Publications

Publications

Filter Results : (29) Arrow Down

Show Results For

Experimental Methods Remove Experimental Methods →

Page 1 of 29 Results →

Are you looking for?

→ Search All HBS Web
  • 2022
  • Working Paper

Nailing Prediction: Experimental Evidence on the Value of Tools in Predictive Model Development
By: Daniel Yue, Paul Hamilton and Iavor Bojinov
Predictive model development is understudied despite its importance to modern businesses. Although prior discussions highlight advances in methods (along the dimensions of data, computing power, and algorithms) as the primary driver of model quality, the value of tools...  View Details
Keywords: Analytics and Data Science
Related
  • Article

Reliable Post hoc Explanations: Modeling Uncertainty in Explainability
By: Dylan Slack, Sophie Hilgard, Sameer Singh and Himabindu Lakkaraju
As black box explanations are increasingly being employed to establish model credibility in high stakes settings, it is important to ensure that these explanations are accurate and reliable. However, prior work demonstrates that explanations generated by...  View Details
Keywords: Black Box Explanations; Bayesian Modeling; Decision Making; Risk and Uncertainty; Information Technology
Related
  • Article

Towards the Unification and Robustness of Perturbation and Gradient Based Explanations
By: Sushant Agarwal, Shahin Jabbari, Chirag Agarwal, Sohini Upadhyay, Steven Wu and Himabindu Lakkaraju
As machine learning black boxes are increasingly being deployed in critical domains such as healthcare and criminal justice, there has been a growing emphasis on developing techniques for explaining these black boxes in a post hoc manner. In this work, we analyze two...  View Details
Keywords: Machine Learning; Black Box Explanations; Decision Making; Forecasting and Prediction; Information Technology
Related
  • 2021
  • Working Paper

Population Interference in Panel Experiments
By: Iavor I Bojinov, Kevin Wu Han and Guillaume Basse
The phenomenon of population interference, where a treatment assigned to one experimental unit affects another experimental unit's outcome, has received considerable attention in standard randomized experiments. The complications produced by population interference in...  View Details
Keywords: Finite Population; Potential Outcomes; Dynamic Causal Effects; Mathematical Methods
Related
  • Article

Beyond Individualized Recourse: Interpretable and Interactive Summaries of Actionable Recourses
By: Kaivalya Rawal and Himabindu Lakkaraju
As predictive models are increasingly being deployed in high-stakes decision-making, there has been a lot of interest in developing algorithms which can provide recourses to affected individuals. While developing such tools is important, it is even more critical to...  View Details
Keywords: Predictive Models; Decision Making; Framework; Mathematical Methods
Related
  • 2020
  • Working Paper

Design and Analysis of Switchback Experiments
By: Iavor I Bojinov, David Simchi-Levi and Jinglong Zhao
In switchback experiments, a firm sequentially exposes an experimental unit to a random treatment, measures its response, and repeats the procedure for several periods to determine which treatment leads to the best outcome. Although practitioners have widely adopted...  View Details
Keywords: Switchback Experiments; Design; Analysis; Mathematical Methods
Related
  • August 2020 (Revised September 2020)
  • Technical Note

Assessing Prediction Accuracy of Machine Learning Models
By: Michael W. Toffel, Natalie Epstein, Kris Ferreira and Yael Grushka-Cockayne
The note introduces a variety of methods to assess the accuracy of machine learning prediction models. The note begins by briefly introducing machine learning, overfitting, training versus test datasets, and cross validation. The following accuracy metrics and tools...  View Details
Keywords: Machine Learning; Statistics; Econometric Analyses; Experimental Methods; Data Analysis; Data Analytics; Forecasting and Prediction; Analytics and Data Science; Analysis; Mathematical Methods
Related
  • August 2020
  • Technical Note

Comparing Two Groups: Sampling and t-Testing
By: Iavor I Bojinov, Chiara Farronato, Yael Grushka-Cockayne, Willy C. Shih and Michael W. Toffel
This note describes sampling and t-tests, two fundamental statistical concepts.  View Details
Keywords: Statistics; Econometric Analyses; Experimental Methods; Data Analysis; Data Analytics; Analytics and Data Science; Analysis; Surveys; Mathematical Methods
Related
  • April 2020
  • Article

Field Comparisons of Incentive-Compatible Preference Elicitation Techniques
By: Shawn A. Cole, A. Nilesh Fernando, Daniel Stein and Jeremy Tobacman
Knowledge of consumer demand is important for firms, policy makers, and economists. One common tool for incentive-compatible demand elicitation, the Becker-DeGroot-Marschak (BDM) mechanism, has been widely used in laboratory settings but rarely evaluated for...  View Details
Keywords: Incentive-compatible Elicitation; Experimental Methods; Weather Insurance; Rainfall Insurance; Agricultural Extension; Demand and Consumers
Related
  • March 2020
  • Module Note

The Role of Experiments in Organizations
By: Michael Luca
This note outlines the structure and content of a four-class module—The Role of Experiments in Organizations—that is designed to introduce students to the role of experimental methods in managerial decisions.  View Details
Keywords: Experiments; Experimental Methods; Decision Making; Information
Related
  • Article

Statistical Physics of Human Cooperation
By: Matjaž Perc, Jillian J. Jordan, David G. Rand, Zhen Wang, Stefano Boccaletti and Attila Szolnoki
Extensive cooperation among unrelated individuals is unique to humans, who often sacrifice personal benefits for the common good and work together to achieve what they are unable to execute alone. The evolutionary success of our species is indeed due, to a large...  View Details
Keywords: Human Cooperation; Evolutionary Game Theory; Public Goods; Reward; Punishment; Tolerance; Self-organization; Pattern Formation; Cooperation; Behavior; Game Theory
Related
  • December 2016
  • Article

The Effects of Endowment Size and Strategy Method on Third Party Punishment
By: Jillian J. Jordan, Katherine McAuliffe and David G. Rand
Numerous experiments have shown that people often engage in third-party punishment (3PP) of selfish behavior. This evidence has been used to argue that people respond to selfishness with anger, and get utility from punishing those who mistreat others. Elements of the...  View Details
Keywords: Third-party Punishment; Norm-enforcement; Strategy Method; Economic Games; Cooperation; Emotions; Fairness
Related
  • Article

Gathering Data for Archival, Field, Survey, and Experimental Accounting Research
By: Robert Bloomfield, Mark W. Nelson and Eugene F. Soltes
In the published proceedings of the first Journal of Accounting Research Conference, Vatter (1966) lamented that “Gathering direct and original facts is a tedious and difficult task, and it is not surprising that such work is avoided.” For the 50th JAR Conference,...  View Details
Keywords: Archival; Data; Experiment; Empirical Methods; Field Study; Analytics and Data Science; Surveys; Financial Reporting
Related
  • March 2016 (Revised January 2020)
  • Teaching Note

Behavioural Insights Team (A) and (B)
By: Michael Luca and Patrick Rooney
The Behavioural Insights Team case introduces students to the concept of choice architecture and the value of experimental methods (sometimes called A/B testing) within organizational contexts. The exercise provides an opportunity for students to apply these principles...  View Details
Keywords: Behavioral Economics; Experiments; Choice Architecture; Public Entrepreneurship; Decision Choices and Conditions; Mathematical Methods; United Kingdom
Related
  • March 2016
  • Supplement

Advertising Experiments at RestaurantGrades
By: Weijia Dai, Hyunjin Kim and Michael Luca
This exercise provides students with a data set consisting of results from a hypothetical experiment, and asks students to make recommendations based on the data. Through this process, the exercise teaches students to analyze, design, and interpret experiments. The...  View Details
Keywords: Experimental Methods; Analytics; Web Technology; Marketing; Digital Marketing; Analysis; Performance Effectiveness; Social Media
Related
  • March 2015 (Revised January 2020)
  • Case

Behavioural Insights Team (A)
By: Michael Luca and Patrick Rooney
The Behavioural Insights Team case introduces students to the concept of choice architecture and the value of experimental methods (sometimes called A/B testing) within organizational contexts. The exercise provides an opportunity for students to apply these principles...  View Details
Keywords: Behavioral Economics; Experiments; Choice Architecture; Public Entrepreneurship; Decision Choices and Conditions; Consumer Behavior; Taxation; Economics; Public Administration Industry; United Kingdom
Related
  • March 2015 (Revised January 2020)
  • Supplement

Behavioural Insights Team (B)
By: Michael Luca and Patrick Rooney
The Behavioural Insights Team case introduces students to the concept of choice architecture and the value of experimental methods (sometimes called A/B testing) within organizational contexts. The exercise provides an opportunity for students to apply these principles...  View Details
Keywords: Behavioral Economics; Experiments; Choice Architecture; Public Entrepreneurship; United Kingdom
Related
  • August 2012
  • Article

Organ Allocation Policy and the Decision to Donate
By: Judd B. Kessler and Alvin E. Roth
Organ donations from deceased donors (cadavers) provide the majority of transplanted organs in the United States, and one deceased donor can save numerous lives by providing multiple organs. Nevertheless, most Americans are not registered organ donors despite the...  View Details
Keywords: Organ Donation; Health; Philanthropy and Charitable Giving; Decision Making; Resource Allocation; Mathematical Methods; United States
Related
  • September 2011 (Revised July 2012)
  • Case

Building Watson: Not So Elementary, My Dear!
By: Willy Shih
This case is set inside IBM Research's efforts to build a computer that can successfully take on human challengers playing the game show Jeopardy! It opens with the machine named Watson offering the incorrect answer "Toronto" to a seemingly simple question during the...  View Details
Keywords: Technological Innovation; Standards; Product Development; Organizational Change and Adaptation; Mathematical Methods; Research and Development; Information Technology
Related
  • 2011
  • Working Paper

Free to Punish? The American Dream and the Harsh Treatment of Criminals
By: Rafael Di Tella and Juan Dubra
We describe the evolution of selective aspects of punishment in the U.S. over the period 1980-2004. We note that imprisonment increased around 1980, a period that coincides with the "Reagan revolution" in economic matters. We build an economic model where beliefs about...  View Details
Keywords: Crime and Corruption; Economy; Moral Sensibility; Mathematical Methods; Opportunities; Behavior; United States
Related

Are you looking for?

→ Search All HBS Web