More from the Author

• Working Paper | HBS Working Paper Series | 2019

  Do Experts Listen to Other Experts? Field Experimental Evidence from Scientific Peer Review

  Misha Teplitskiy, Hardeep Ranu, Gary Gray, Michael Menietti, Eva Guinan and Karim R. Lakhani

  Organizations in science and elsewhere often rely on committees of experts to make important decisions, such as evaluating early-stage projects and ideas. However, very little is known about how experts influence each other’s opinions and how that influence affects final evaluations. Here, we use a field experiment in scientific peer review to examine experts’ susceptibility to the opinions of others. We recruited 277 faculty members at seven U.S. medical schools to evaluate 47 early stage research proposals in biomedicine. In our experiment, evaluators (1) completed independent reviews of research ideas, (2) received (artificial) scores attributed to anonymous “other reviewers” from the same or a different discipline, and (3) decided whether to update their initial scores. Evaluators did not meet in person and were not otherwise aware of each other. We find that, even in a completely anonymous setting and controlling for a range of career factors, women updated their scores 13% more often than men, while very highly cited “superstar” reviewers updated 24% less often than others. Women in male-dominated subfields were particularly likely to update, updating 8% more for every 10% decrease in subfield representation. Very low scores were particularly “sticky” and seldom updated upward, suggesting a possible source of conservatism in evaluation. These systematic differences in how world-class experts respond to external opinions can lead to substantial gender and status disparities in whose opinion ultimately matters in collective expert judgment.

  Keywords: social influence; expert judgment; evaluation; Performance Evaluation; Judgments; Gender; Status and Position; Power and Influence;

  Citation:

  Teplitskiy, Misha, Hardeep Ranu, Gary Gray, Michael Menietti, Eva Guinan, and Karim R. Lakhani. "Do Experts Listen to Other Experts? Field Experimental Evidence from Scientific Peer Review." Harvard Business School Working Paper, No. 19-107, April 2019.  View Details

  CiteView Details Read Now

  Related

• Article | Journal of Economic Behavior & Organization | April 2019

  Incentives for Public Goods Inside Organizations: Field Experimental Evidence

  Andrea Blasco, Olivia S. Jung, Karim R. Lakhani and Michael Menietti

  Understanding why employees go the extra mile at work is a key problem for many organizations. We conduct a field experiment at a medical organization to study motivations for employees to submit project proposals for organizational improvement. In total, we analyze 1,237 employees, 118 proposals, and quality evaluations for more than 12,000 evaluator-proposal pairs. The analysis shows that solicitations offering a personal reward for top submissions boost participation rates without affecting submission quality. We show that this is due to workers partially internalizing the positive effects of their submissions on the other individuals in the workplace. We also find that offering employees project funding to implement their own proposals potentially backfires, undermining participation. And solicitations emphasizing mission-oriented goals, like improving patient care, are sensitive to the solicited person’s gender, with women responding more than men. These results shed light on the factors that drive employees’ engagement in organizational tasks, beyond regular duties, and provide insights on how to design incentives to foster contributions to public goods inside organizations.

  Keywords: field experiment; innovation; contest; incentives; free-rider problem; healthcare organizations; Employees; Motivation and Incentives; Innovation and Invention; Organizations; Performance Improvement; Perspective;

  Citation:

  Blasco, Andrea, Olivia S. Jung, Karim R. Lakhani, and Michael Menietti. "Incentives for Public Goods Inside Organizations: Field Experimental Evidence." Journal of Economic Behavior & Organization 160 (April 2019): 214–229.  View Details

  CiteView DetailsFind at Harvard

  Related

• Working Paper | HBS Working Paper Series | 2019

  Advancing Computational Biology and Bioinformatics Research Through Open Innovation Competitions

  Andrea Blasco, Michael G. Endres, Rinat A. Sergeev, Anup Jonchhe, Max Macaluso, Rajiv Narayan, Ted Natoli, Jin H. Paik, Bryan Briney, Chunlei Wu, Andrew I. Su, Aravind Subramanian and Karim R. Lakhani

  Open data science and algorithm development competitions offer a unique avenue for rapid discovery of better computational strategies. We highlight three examples in computational biology and bioinformatics research where the use of competitions has yielded significant performance gains over established algorithms. These include studying algorithms for antibody clustering, imputing gene expression data, and querying the Connectivity Map (CMap). Performance gains are evaluated quantitatively using realistic, albeit sanitized, data sets. The solutions produced through these competitions are then examined with respect to their utility and the prospects for implementation in the field. We present the decision process and competition design considerations that lead to these successful outcomes as a model for researchers who want to use competitions and non-domain crowds as collaborators to further their research.

  Keywords: computational biology; bioinformatics; innovation competitions; Research; Mathematical Methods; Strategy;

  Citation:

  Blasco, Andrea, Michael G. Endres, Rinat A. Sergeev, Anup Jonchhe, Max Macaluso, Rajiv Narayan, Ted Natoli, Jin H. Paik, Bryan Briney, Chunlei Wu, Andrew I. Su, Aravind Subramanian, and Karim R. Lakhani. "Advancing Computational Biology and Bioinformatics Research Through Open Innovation Competitions." Harvard Business School Working Paper, No. 19-097, March 2019.  View Details

  CiteView Details Read Now

  Related