Skip to Main Content
HBS Home
  • About
  • Academic Programs
  • Alumni
  • Faculty & Research
  • Baker Library
  • Giving
  • Harvard Business Review
  • Initiatives
  • News
  • Recruit
  • Map / Directions
Faculty & Research
  • Faculty
  • Research
  • Featured Topics
  • Academic Units
  • …→
  • Harvard Business School→
  • Faculty & Research→
  • Academic Units
    • Academic Units
    • Accounting & Management
    • Business, Government & the International Economy
    • Entrepreneurial Management
    • Finance
    • General Management
    • Marketing
    • Negotiation, Organizations & Markets
    • Organizational Behavior
    • Strategy
    • Technology & Operations Management
    →
  • Technology & Operations Management
    • Technology & Operations Management
    • Faculty
    • Curriculum
    • Seminars & Conferences
    • Awards & Honors
    • Doctoral Students
    →

Technology & Operations Management

Technology & Operations Management

  • Faculty
  • Curriculum
  • Seminars & Conferences
  • Awards & Honors
  • Doctoral Students
Overview Faculty Curriculum Seminars & Conferences Awards & Honors Doctoral Students
    • 2023
    • Article

    Probabilistically Robust Recourse: Navigating the Trade-offs between Costs and Robustness in Algorithmic Recourse

    By: Martin Pawelczyk, Teresa Datta, Johannes van-den-Heuvel, Gjergji Kasneci and Himabindu Lakkaraju

    As machine learning models are increasingly being employed to make consequential decisions in real-world settings, it becomes critical to ensure that individuals who are adversely impacted (e.g., loan denied) by the predictions of these models are provided with a means for recourse. While several approaches have been proposed to construct recourses for affected individuals, the recourses output by these methods either achieve low costs (i.e., ease-of-implementation) or robustness to small perturbations (i.e., noisy implementations of recourses), but not both due to the inherent trade-offs between the recourse costs and robustness. Furthermore, prior approaches do not provide end users with any agency over navigating the aforementioned trade-offs. In this work, we address the above challenges by proposing the first algorithmic framework which enables users to effectively manage the recourse cost vs. robustness trade-offs. More specifically, our framework Probabilistically ROBust rEcourse (PROBE) lets users choose the probability with which a recourse could get invalidated (recourse invalidation rate) if small changes are made to the recourse i.e., the recourse is implemented somewhat noisily. To this end, we propose a novel objective function which simultaneously minimizes the gap between the achieved (resulting) and desired recourse invalidation rates, minimizes recourse costs, and also ensures that the resulting recourse achieves a positive model prediction. We develop novel theoretical results to characterize the recourse invalidation rates corresponding to any given instance w.r.t. different classes of underlying models (e.g., linear models, tree based models etc.), and leverage these results to efficiently optimize the proposed objective. Experimental evaluation with multiple real world datasets demonstrates the efficacy of the proposed framework.

    • 2023
    • Article

    Probabilistically Robust Recourse: Navigating the Trade-offs between Costs and Robustness in Algorithmic Recourse

    By: Martin Pawelczyk, Teresa Datta, Johannes van-den-Heuvel, Gjergji Kasneci and Himabindu Lakkaraju

    As machine learning models are increasingly being employed to make consequential decisions in real-world settings, it becomes critical to ensure that individuals who are adversely impacted (e.g., loan denied) by the predictions of these models are provided with a means for recourse. While several approaches have been proposed to construct...

    • 2023
    • Article

    Association Between Regulatory Submission Characteristics and Recalls of Medical Devices Receiving 510(k) Clearance

    By: Alexander O. Everhart, Soumya Sen, Ariel D. Stern, Yi Zhu and Pinar Karaca-Mandic

    Importance: Most regulated medical devices enter the U.S. market via the 510(k) regulatory submission pathway, wherein manufacturers demonstrate that applicant devices are “substantially equivalent” to 1 or more “predicate” devices (legally marketed medical devices with similar intended use). Most recalled medical devices are 510(k) devices. Objective: To examine the association between characteristics of predicate medical devices and recall probability for 510(k) devices. Design, Setting, and Participants: In this exploratory cross-sectional analysis of medical devices cleared by the US Food and Drug Administration (FDA) between 2003 and 2018 via the 510(k) regulatory submission pathway, linear probability models were used to examine associations between a 510(k) device’s recall status and characteristics of its predicate medical devices. Public documents for the 510(k) medical devices were collected using FDA databases. A text extraction algorithm was applied to identify predicate medical devices cited in 510(k) regulatory submissions. Algorithm-derived metadata were combined with 2003-2020 FDA recall data. Exposures: Citation of predicate medical devices with certain characteristics in 510(k) regulatory submissions, including the total number of predicate medical devices cited by the applicant device, the age of the predicate medical devices, the lack of similarity of the predicate medical devices to the applicant device, and the recall status of the predicate medical devices. Main Outcomes and Measures: Class I or class II recall of a 510(k) medical device between its FDA regulatory clearance date and December 31, 2020. Results:The sample included 35 176 medical devices, of which 4007 (11.4%) were recalled. The applicant devices cited a mean of 2.6 predicate medical devices, with mean ages of 3.6 years and 7.4 years for the newest and oldest, respectively, predicate medical devices. Of the applicant devices, 93.9% cited predicate medical devices with no ongoing recalls, 4.3% cited predicate medical devices with 1 ongoing class I or class II recall, 1.0% cited predicate medical devices with 2 ongoing recalls, and 0.8% cited predicate medical devices with 3 or more ongoing recalls. Applicant devices citing predicate medical devices with 3 or more ongoing recalls were significantly associated with a 9.31–percentage-point increase (95% CI, 2.84-15.77 percentage points) in recall probability compared with devices without ongoing recalls of predicate medical devices, or an 81.2% increase in recall probability relative to the mean recall probability. A 1-SD increase in the total number of predicate medical devices cited by the applicant device was significantly associated with a 1.25–percentage-point increase (95% CI, 0.62-1.87 percentage points) in recall probability, or an 11.0% increase in recall probability relative to the mean recall probability. A 1-SD increase in the newest age of a predicate medical device was significantly associated with a 0.78–percentage-point decrease (95% CI, 1.29-0.30 percentage points) in recall probability, or a 6.8% decrease in recall probability relative to the mean recall probability. Conclusions and Relevance: This exploratory cross-sectional study of 510(k) medical devices cleared by the FDA between 2003 and 2018 demonstrated significant associations between 510(k) submission characteristics and recalls of medical devices. Further research is needed to understand the implications of these associations.

    • 2023
    • Article

    Association Between Regulatory Submission Characteristics and Recalls of Medical Devices Receiving 510(k) Clearance

    By: Alexander O. Everhart, Soumya Sen, Ariel D. Stern, Yi Zhu and Pinar Karaca-Mandic

    Importance: Most regulated medical devices enter the U.S. market via the 510(k) regulatory submission pathway, wherein manufacturers demonstrate that applicant devices are “substantially equivalent” to 1 or more “predicate” devices (legally marketed medical devices with similar intended use). Most recalled medical devices are 510(k) devices....

    • January 2023
    • Case

    Vertex and the Cure for Type 1 Diabetes

    By: Amitabh Chandra, William J Anderson and Silvia Mare

    • January 2023
    • Case

    Vertex and the Cure for Type 1 Diabetes

    By: Amitabh Chandra, William J Anderson and Silvia Mare

About the Unit

As the world of operations has changed, so have interests and priorities within the Unit. Historically, the TOM Unit focused on manufacturing and the development of physical products. Over the past several years, we have expanded our research, course development, and course offerings to encompass new issues in information technology, supply chains, and service industries.

The field of TOM is concerned with the design, management, and improvement of operating systems and processes. As we seek to understand the challenges confronting firms competing in today's demanding environment, the focus of our work has broadened to include the multiple activities comprising a firm's "operating core":

  • the multi-function, multi-firm system that includes basic research, design, engineering, product and process development and production of goods and services within individual operating units;
  • the networks of information and material flows that tie operating units together and the systems that support these networks;
  • the distribution and delivery of goods and services to customers.

Recent Publications

Probabilistically Robust Recourse: Navigating the Trade-offs between Costs and Robustness in Algorithmic Recourse

By: Martin Pawelczyk, Teresa Datta, Johannes van-den-Heuvel, Gjergji Kasneci and Himabindu Lakkaraju
  • 2023 |
  • Article |
  • International Conference on Learning Representations (ICLR)
As machine learning models are increasingly being employed to make consequential decisions in real-world settings, it becomes critical to ensure that individuals who are adversely impacted (e.g., loan denied) by the predictions of these models are provided with a means for recourse. While several approaches have been proposed to construct recourses for affected individuals, the recourses output by these methods either achieve low costs (i.e., ease-of-implementation) or robustness to small perturbations (i.e., noisy implementations of recourses), but not both due to the inherent trade-offs between the recourse costs and robustness. Furthermore, prior approaches do not provide end users with any agency over navigating the aforementioned trade-offs. In this work, we address the above challenges by proposing the first algorithmic framework which enables users to effectively manage the recourse cost vs. robustness trade-offs. More specifically, our framework Probabilistically ROBust rEcourse (PROBE) lets users choose the probability with which a recourse could get invalidated (recourse invalidation rate) if small changes are made to the recourse i.e., the recourse is implemented somewhat noisily. To this end, we propose a novel objective function which simultaneously minimizes the gap between the achieved (resulting) and desired recourse invalidation rates, minimizes recourse costs, and also ensures that the resulting recourse achieves a positive model prediction. We develop novel theoretical results to characterize the recourse invalidation rates corresponding to any given instance w.r.t. different classes of underlying models (e.g., linear models, tree based models etc.), and leverage these results to efficiently optimize the proposed objective. Experimental evaluation with multiple real world datasets demonstrates the efficacy of the proposed framework.
Citation
Read Now
Related
Pawelczyk, Martin, Teresa Datta, Johannes van-den-Heuvel, Gjergji Kasneci, and Himabindu Lakkaraju. "Probabilistically Robust Recourse: Navigating the Trade-offs between Costs and Robustness in Algorithmic Recourse." International Conference on Learning Representations (ICLR) (2023).

On the Privacy Risks of Algorithmic Recourse

By: Martin Pawelczyk, Himabindu Lakkaraju and Seth Neel
  • 2023 |
  • Article |
  • International Conference on Artificial Intelligence and Statistics (AISTATS)
As predictive models are increasingly being employed to make consequential decisions, there is a growing emphasis on developing techniques that can provide algorithmic recourse to affected individuals. While such recourses can be immensely beneficial to affected individuals, potential adversaries could also exploit these recourses to compromise privacy. In this work, we make the first attempt at investigating if and how an adversary can leverage recourses to infer private information about the underlying model’s training data. To this end, we propose a series of novel membership inference attacks which leverage algorithmic recourse. More specifically, we extend the prior literature on membership inference attacks to the recourse setting by leveraging the distances between data instances and their corresponding counterfactuals output by state-of-the-art recourse methods. Extensive experimentation with real world and synthetic datasets demonstrates significant privacy leakage through recourses. Our work establishes unintended privacy leakage as an important risk in the widespread adoption of recourse methods.
Citation
Read Now
Related
Pawelczyk, Martin, Himabindu Lakkaraju, and Seth Neel. "On the Privacy Risks of Algorithmic Recourse." International Conference on Artificial Intelligence and Statistics (AISTATS) (2023).

Association Between Regulatory Submission Characteristics and Recalls of Medical Devices Receiving 510(k) Clearance

By: Alexander O. Everhart, Soumya Sen, Ariel D. Stern, Yi Zhu and Pinar Karaca-Mandic
  • 2023 |
  • Article |
  • JAMA, the Journal of the American Medical Association
Importance: Most regulated medical devices enter the U.S. market via the 510(k) regulatory submission pathway, wherein manufacturers demonstrate that applicant devices are “substantially equivalent” to 1 or more “predicate” devices (legally marketed medical devices with similar intended use). Most recalled medical devices are 510(k) devices. Objective: To examine the association between characteristics of predicate medical devices and recall probability for 510(k) devices. Design, Setting, and Participants: In this exploratory cross-sectional analysis of medical devices cleared by the US Food and Drug Administration (FDA) between 2003 and 2018 via the 510(k) regulatory submission pathway, linear probability models were used to examine associations between a 510(k) device’s recall status and characteristics of its predicate medical devices. Public documents for the 510(k) medical devices were collected using FDA databases. A text extraction algorithm was applied to identify predicate medical devices cited in 510(k) regulatory submissions. Algorithm-derived metadata were combined with 2003-2020 FDA recall data. Exposures: Citation of predicate medical devices with certain characteristics in 510(k) regulatory submissions, including the total number of predicate medical devices cited by the applicant device, the age of the predicate medical devices, the lack of similarity of the predicate medical devices to the applicant device, and the recall status of the predicate medical devices. Main Outcomes and Measures: Class I or class II recall of a 510(k) medical device between its FDA regulatory clearance date and December 31, 2020. Results:The sample included 35 176 medical devices, of which 4007 (11.4%) were recalled. The applicant devices cited a mean of 2.6 predicate medical devices, with mean ages of 3.6 years and 7.4 years for the newest and oldest, respectively, predicate medical devices. Of the applicant devices, 93.9% cited predicate medical devices with no ongoing recalls, 4.3% cited predicate medical devices with 1 ongoing class I or class II recall, 1.0% cited predicate medical devices with 2 ongoing recalls, and 0.8% cited predicate medical devices with 3 or more ongoing recalls. Applicant devices citing predicate medical devices with 3 or more ongoing recalls were significantly associated with a 9.31–percentage-point increase (95% CI, 2.84-15.77 percentage points) in recall probability compared with devices without ongoing recalls of predicate medical devices, or an 81.2% increase in recall probability relative to the mean recall probability. A 1-SD increase in the total number of predicate medical devices cited by the applicant device was significantly associated with a 1.25–percentage-point increase (95% CI, 0.62-1.87 percentage points) in recall probability, or an 11.0% increase in recall probability relative to the mean recall probability. A 1-SD increase in the newest age of a predicate medical device was significantly associated with a 0.78–percentage-point decrease (95% CI, 1.29-0.30 percentage points) in recall probability, or a 6.8% decrease in recall probability relative to the mean recall probability. Conclusions and Relevance: This exploratory cross-sectional study of 510(k) medical devices cleared by the FDA between 2003 and 2018 demonstrated significant associations between 510(k) submission characteristics and recalls of medical devices. Further research is needed to understand the implications of these associations.
Keywords: Recalls; Governing Rules, Regulations, and Reforms; Medical Devices and Supplies Industry
Citation
Find at Harvard
Purchase
Related
Everhart, Alexander O., Soumya Sen, Ariel D. Stern, Yi Zhu, and Pinar Karaca-Mandic. "Association Between Regulatory Submission Characteristics and Recalls of Medical Devices Receiving 510(k) Clearance." JAMA, the Journal of the American Medical Association 329, no. 2 (2023): 144–156.

Global Sourcing at Nike

By: Nien-hê Hsieh and Michael W. Toffel
  • January 2023 |
  • Supplement |
  • Faculty Research
Powerpoint Supplement for “Global Sourcing at Nike,” HBS No. 619-008
Citation
Purchase
Related
Hsieh, Nien-hê, and Michael W. Toffel. "Global Sourcing at Nike." Harvard Business School PowerPoint Supplement 623-709, January 2023.

Vertex and the Cure for Type 1 Diabetes

By: Amitabh Chandra, William J Anderson and Silvia Mare
  • January 2023 |
  • Case |
  • Faculty Research
Keywords: Biotechnology Industry
Citation
Educators
Related
Chandra, Amitabh, William J Anderson, and Silvia Mare. "Vertex and the Cure for Type 1 Diabetes ." Harvard Business School Case 623-053, January 2023.

The Subjective Expected Utility Approach and a Framework for Defining Project Risk in Terms of Novelty and Feasibility—A Response to Franzoni and Stephan (2023), ‘Uncertainty and Risk-Taking in Science’

By: Jacqueline N. Lane
  • 2023 |
  • Working Paper |
  • Faculty Research
In their Discussion Paper, Franzoni and Stephan (F&S, 2023) discuss the shortcomings of existing peer review models in shaping the funding of risky science. Their discussion offers a conceptual framework for incorporating risk into peer review models of research proposals by leveraging the Subjective Expected Utility (SEU) approach to decouple reviewers’ assessments of a project’s potential value from its risk. In my Response, I build on F&S’s discussion and attempt to shed light on three additional yet core considerations of risk in science: 1) how risk and reward in science are related to assessments of a project’s novelty and feasibility; 2) how the sunk cost literature can help articulate why reviewers tend to perceive new research areas as riskier than continued investigation of existing lines of research; and 3) how drawing on different types of expert reviewers (i.e., based on domain and technical expertise) can result in alternative evaluation assessments to better inform resource allocation decisions. The spirit of my Response is to sharpen our understanding of risk in science and to offer insights on how future theoretical and empirical work—leveraging experiments— can test and validate the SEU approach for the purposes of funding more risky science that advances the knowledge frontier.
Keywords: Risk and Uncertainty; Research; Resource Allocation; Perception
Citation
Read Now
Related
Lane, Jacqueline N. "The Subjective Expected Utility Approach and a Framework for Defining Project Risk in Terms of Novelty and Feasibility—A Response to Franzoni and Stephan (2023), ‘Uncertainty and Risk-Taking in Science’." Harvard Business School Working Paper, No. 23-037, January 2023.

When Algorithms Explain Themselves: AI Adoption and Accuracy of Experts' Decisions

By: Himabindu Lakkaraju and Chiara Farronato
  • 2023 |
  • Working Paper |
  • Faculty Research
Citation
Related
Lakkaraju, Himabindu, and Chiara Farronato. "When Algorithms Explain Themselves: AI Adoption and Accuracy of Experts' Decisions." Working Paper, 2023.

Achieving Universal Health Insurance Coverage in the United States: Addressing Market Failures or Providing a Social Floor?

By: Katherine Baicker, Amitabh Chandra and Mark Shepard
  • 2023 |
  • Working Paper |
  • Faculty Research
The United States spends substantially more on health care than most developed countries, yet leaves a greater share of the population uninsured. We suggest that incremental insurance expansions focused on addressing market failures will propagate inefficiencies and are not likely to facilitate active policy decisions that align with societal coverage goals. By instead defining a basic bundle of services that is publicly financed for all, while allowing individuals to purchase additional coverage, policymakers could both expand coverage and maintain incentives for innovation, fostering universal access to innovative care in an affordable system.
Keywords: Public Sector; Insurance; Health Care and Treatment; Policy; Innovation and Invention
Citation
Purchase
Related
Baicker, Katherine, Amitabh Chandra, and Mark Shepard. "Achieving Universal Health Insurance Coverage in the United States: Addressing Market Failures or Providing a Social Floor?" NBER Working Paper Series, No. 30854, January 2023.
More Publications

In the News

    • 27 Jan 2023
    • Financial Post

    Why the Canadian Once in Charge of Google’s Finances Thinks the Tech Giant Should Be Broken Up

    Re: Amy Edmondson
    • 26 Jan 2023
    • Vox

    The Death of the Customer Service Hotline

    Re: Ryan Buell
    • 25 Jan 2023
    • Business Insider

    Googlers Prize ‘Psychological Safety,’ but It Won’t Save Them from Layoffs

    Re: Amy Edmondson
→More Faculty News

HBS Working Knowledge

    • 17 Jan 2023

    8 Trends to Watch in 2023

    by Avery Forman
    • 16 Dec 2022

    Why Technology Alone Can't Solve AI's Bias Problem

    Re: Himabindu Lakkaraju
    • 29 Nov 2022

    How Much More Would Holiday Shoppers Pay to Wear Something Rare?

    Re: Chiara Farronato
→More Working Knowledge Articles

Harvard Business Publishing

    • July–August 2013
    • Article

    Leadership Lessons from the Chilean Mine Rescue

    By: Faaiza Rashid, Amy C. Edmondson and Herman B. Leonard
    • December 2022
    • Case

    Ed Catmull: Lessons from Leading Pixar Animation Studios

    By: Francesca Gino, Linda Hill, Gary Pisano and Ruth Page
    • 2012
    • Book

    Producing Prosperity: Why America Needs a Manufacturing Renaissance

    By: Gary P. Pisano and Willy Shih
→More Harvard Business Publishing

Seminars & Conferences

Feb 16
  • 16 Feb 2023

Kate Kellogg, MIT

Technology & Operations Management (TOM) Seminar
→More Seminars & Conferences

Faculty Positions

Harvard Business School seeks candidates in all fields for full time positions. Candidates with outstanding records in PhD or DBA programs are encouraged to apply.
→Learn More

Contact Information

Technology & Operations Management Unit
Harvard Business School
Morgan Hall
Soldiers Field
Boston, MA 02163
tomunit@hbs.edu

ǁ
Campus Map
Harvard Business School
Soldiers Field
Boston, MA 02163
→Map & Directions
→More Contact Information
  • Make a Gift
  • Site Map
  • Jobs
  • Harvard University
  • Trademarks
  • Policies
  • Accessibility
  • Digital Accessibility
Copyright © President & Fellows of Harvard College