Skip to Main Content
HBS Home
  • About
  • Academic Programs
  • Alumni
  • Faculty & Research
  • Baker Library
  • Giving
  • Harvard Business Review
  • Initiatives
  • News
  • Recruit
  • Map / Directions
Faculty & Research
  • Faculty
  • Research
  • Featured Topics
  • Academic Units
  • …→
  • Harvard Business School→
  • Faculty & Research→
Publications
Publications
  • 2020
  • Working Paper
  • HBS Working Paper Series

When Do Experts Listen to Other Experts? The Role of Negative Information in Expert Evaluations for Novel Projects

By: Jacqueline N. Lane, Misha Teplitskiy, Gary Gray, Hardeep Ranu, Michael Menietti, Eva C. Guinan and Karim R. Lakhani
  • Format:Print
  • | Language:English
  • | Pages:45
ShareBar

Abstract

The evaluation of novel projects lies at the heart of scientific and technological innovation, and yet literature suggests that this process is subject to inconsistency and potential biases. This paper investigates the role of information sharing among experts as the driver of evaluation decisions. We designed and executed two field experiments in two separate grant funding opportunities at a leading research university to explore evaluators’ receptivity to assessments from other evaluators. Collectively, our experiments mobilized 369 evaluators from seven universities to evaluate 97 projects resulting in 761 proposal-evaluation pairs and over $300,000 in awards. We exogenously varied the relative valence (positive and negative) of others’ scores, to determine how exposures to higher and lower scores affect the focal evaluator’s propensity to change the initial score. We found causal evidence of negativity bias, where evaluators are more likely to lower their scores after seeing critical scores than raise them after seeing better scores. Qualitative coding and topic modelling of the evaluators’ justifications for score changes reveal that exposures to lower scores prompted greater attention to uncovering weaknesses, whereas exposures to neutral or higher scores were associated with strengths, along with greater emphasis on non-evaluation criteria, such as confidence in one’s judgment. Overall, information sharing among expert evaluators can lead to more conservative allocation decisions that favor protecting against failure more than maximizing success.

Keywords

Project Evaluation; Innovation; Knowledge Frontier; Negativity Bias; Projects; Innovation and Invention; Information; Diversity; Judgments

Citation

Lane, Jacqueline N., Misha Teplitskiy, Gary Gray, Hardeep Ranu, Michael Menietti, Eva C. Guinan, and Karim R. Lakhani. "When Do Experts Listen to Other Experts? The Role of Negative Information in Expert Evaluations for Novel Projects." Harvard Business School Working Paper, No. 21-007, July 2020. (Revised November 2020.)
  • SSRN
  • Read Now

About The Authors

Jacqueline Ng Lane

Technology and Operations Management
→More Publications

Karim R. Lakhani

Technology and Operations Management
→More Publications

More from the Authors

    • 2023
    • Faculty Research

    Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of AI on Knowledge Worker Productivity and Quality

    By: Frabrizio Dell'Acqua, Edward McFowland III, Ethan Mollick, Hila Lifshitz-Assaf, Katherine C. Kellogg, Saran Rajendran, Lisa Krayer, François Candelon and Karim R. Lakhani
    • 2023
    • Faculty Research

    The Crowdless Future? How Generative AI Is Shaping the Future of Human Crowdsourcing

    By: Léonard Boussioux, Jacqueline N. Lane, Miaomiao Zhang, Vladimir Jacimovic and Karim R. Lakhani
    • April 2023
    • Research Policy

    The Subjective Expected Utility Approach and a Framework for Defining Project Risk in Terms of Novelty <i>and</i> Feasibility—A Response to Franzoni and Stephan (2023), ‘Uncertainty and Risk-Taking in Science’

    By: Jacqueline N. Lane
More from the Authors
  • Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of AI on Knowledge Worker Productivity and Quality By: Frabrizio Dell'Acqua, Edward McFowland III, Ethan Mollick, Hila Lifshitz-Assaf, Katherine C. Kellogg, Saran Rajendran, Lisa Krayer, François Candelon and Karim R. Lakhani
  • The Crowdless Future? How Generative AI Is Shaping the Future of Human Crowdsourcing By: Léonard Boussioux, Jacqueline N. Lane, Miaomiao Zhang, Vladimir Jacimovic and Karim R. Lakhani
  • The Subjective Expected Utility Approach and a Framework for Defining Project Risk in Terms of Novelty <i>and</i> Feasibility—A Response to Franzoni and Stephan (2023), ‘Uncertainty and Risk-Taking in Science’ By: Jacqueline N. Lane
ǁ
Campus Map
Harvard Business School
Soldiers Field
Boston, MA 02163
→Map & Directions
→More Contact Information
  • Make a Gift
  • Site Map
  • Jobs
  • Harvard University
  • Trademarks
  • Policies
  • Accessibility
  • Digital Accessibility
Copyright © President & Fellows of Harvard College