Skip to Main Content
HBS Home
  • About
  • Academic Programs
  • Alumni
  • Faculty & Research
  • Baker Library
  • Giving
  • Harvard Business Review
  • Initiatives
  • News
  • Recruit
  • Map / Directions
Faculty & Research
  • Faculty
  • Research
  • Featured Topics
  • Academic Units
  • …→
  • Harvard Business School→
  • Faculty & Research→
Publications
Publications
  • 2020
  • Working Paper
  • HBS Working Paper Series

When Do Experts Listen to Other Experts? The Role of Negative Information in Expert Evaluations for Novel Projects

By: Jacqueline N. Lane, Misha Teplitskiy, Gary Gray, Hardeep Ranu, Michael Menietti, Eva C. Guinan and Karim R. Lakhani
  • Format:Print
  • | Language:English
  • | Pages:45
ShareBar

Abstract

The evaluation of novel projects lies at the heart of scientific and technological innovation, and yet literature suggests that this process is subject to inconsistency and potential biases. This paper investigates the role of information sharing among experts as the driver of evaluation decisions. We designed and executed two field experiments in two separate grant funding opportunities at a leading research university to explore evaluators’ receptivity to assessments from other evaluators. Collectively, our experiments mobilized 369 evaluators from seven universities to evaluate 97 projects resulting in 761 proposal-evaluation pairs and over $300,000 in awards. We exogenously varied the relative valence (positive and negative) of others’ scores, to determine how exposures to higher and lower scores affect the focal evaluator’s propensity to change the initial score. We found causal evidence of negativity bias, where evaluators are more likely to lower their scores after seeing critical scores than raise them after seeing better scores. Qualitative coding and topic modelling of the evaluators’ justifications for score changes reveal that exposures to lower scores prompted greater attention to uncovering weaknesses, whereas exposures to neutral or higher scores were associated with strengths, along with greater emphasis on non-evaluation criteria, such as confidence in one’s judgment. Overall, information sharing among expert evaluators can lead to more conservative allocation decisions that favor protecting against failure more than maximizing success.

Keywords

Project Evaluation; Innovation; Knowledge Frontier; Negativity Bias; Projects; Innovation and Invention; Information; Diversity; Judgments

Citation

Lane, Jacqueline N., Misha Teplitskiy, Gary Gray, Hardeep Ranu, Michael Menietti, Eva C. Guinan, and Karim R. Lakhani. "When Do Experts Listen to Other Experts? The Role of Negative Information in Expert Evaluations for Novel Projects." Harvard Business School Working Paper, No. 21-007, July 2020. (Revised November 2020.)
  • SSRN
  • Read Now

About The Authors

Jacqueline Ng Lane

Technology and Operations Management
→More Publications

Karim R. Lakhani

Technology and Operations Management
→More Publications

More from the Authors

    • April 2023
    • Research Policy

    The Subjective Expected Utility Approach and a Framework for Defining Project Risk in Terms of Novelty <i>and</i> Feasibility—A Response to Franzoni and Stephan (2023), ‘Uncertainty and Risk-Taking in Science’

    By: Jacqueline N. Lane
    • March 2023
    • Faculty Research

    VideaHealth: Building the AI Factory

    By: Karim R. Lakhani
    • March 2023
    • Faculty Research

    Moderna (B) Case Supplement

    By: Karim R. Lakhani, Allison J. Wigen and Dave Habeeb
More from the Authors
  • The Subjective Expected Utility Approach and a Framework for Defining Project Risk in Terms of Novelty <i>and</i> Feasibility—A Response to Franzoni and Stephan (2023), ‘Uncertainty and Risk-Taking in Science’ By: Jacqueline N. Lane
  • VideaHealth: Building the AI Factory By: Karim R. Lakhani
  • Moderna (B) Case Supplement By: Karim R. Lakhani, Allison J. Wigen and Dave Habeeb
ǁ
Campus Map
Harvard Business School
Soldiers Field
Boston, MA 02163
→Map & Directions
→More Contact Information
  • Make a Gift
  • Site Map
  • Jobs
  • Harvard University
  • Trademarks
  • Policies
  • Accessibility
  • Digital Accessibility
Copyright © President & Fellows of Harvard College