Looking Across and Looking Beyond the Knowledge Frontier: Intellectual Distance and Resource Allocation in Science

Kevin J. Boudreau; Eva  Guinan; Karim R. Lakhani; Christoph  Riedl

Article | Management Science | October 2016

Looking Across and Looking Beyond the Knowledge Frontier: Intellectual Distance and Resource Allocation in Science

by Kevin J. Boudreau, Eva Guinan, Karim R. Lakhani and Christoph Riedl

Abstract

Selecting among alternative innovative projects is a core management task in all innovating organizations. In this paper, we focus on the evaluation of frontier scientific research projects. We argue that the "intellectual distance" between the knowledge embodied in research proposals and an evaluator's own expertise systematically relates to the evaluations given (and consequent resource allocation). We empirically evaluate effects in data collected from a grant proposal process at a leading research university in which we randomized the assignment of evaluators and proposals to generate 2,130 evaluator-proposal pairs. We find evaluators systematically give lower scores to research proposals closer to their own areas of expertise and to highly novel research proposals. We interpret the empirical patterns in relation to a range of theoretical mechanisms and discuss implications for policy, managerial intervention, and allocation of resources in the ongoing accumulation of scientific knowledge.

Keywords: knowledge; innovation; novelty; evaluation; resource allocation; Resource Allocation; Decision Choices and Conditions; Innovation and Management; Science-Based Business; Experience and Expertise;

Format: Print

SSRN Find at Harvard

Citation:

Boudreau, Kevin J., Eva Guinan, Karim R. Lakhani, and Christoph Riedl. "Looking Across and Looking Beyond the Knowledge Frontier: Intellectual Distance and Resource Allocation in Science." Management Science 62, no. 10 (October 2016).

Working Paper | HBS Working Paper Series | 2020

When Do Experts Listen to Other Experts? The Role of Negative Information in Expert Evaluations for Novel Projects

Jacqueline N. Lane, Misha Teplitskiy, Gary Gray, Hardeep Ranu, Michael Menietti, Eva C. Guinan and Karim R. Lakhani

The evaluation of novel projects lies at the heart of scientific and technological innovation, and yet literature suggests that this process is subject to inconsistency and potential biases. This paper investigates the role of information sharing among experts as the driver of evaluation decisions. We designed and executed two field experiments in two separate grant funding opportunities at a leading research university to explore evaluators’ receptivity to assessments from other evaluators. Collectively, our experiments mobilized 369 evaluators from seven universities to evaluate 97 projects resulting in 760 proposal-evaluation pairs and over $300,000 in awards. We exogenously varied two key aspects of information sharing: 1) the intellectual distance between each focal evaluator and the other evaluators and 2) the relative valence (positive and negative) of others’ scores, to determine how these treatments affect the focal evaluator’s propensity to change the initial score. Although the intellectual similarity treatment did not yield a measurable effect, we found causal evidence of negativity bias, where evaluators are more likely to lower their scores after seeing critical scores than raise them after seeing better scores. Qualitative coding and topic modeling of the evaluators’ justifications for score changes reveal that exposures to low scores prompted greater attention to uncovering weaknesses, whereas exposures to neutral or high scores were associated with strengths, along with greater emphasis on non-evaluation criteria, such as confidence in one’s judgment. Overall, information sharing among expert evaluators can lead to more conservative allocation decisions that favors protecting against failure than maximizing success.

Keywords: Project evaluation; innovation; knowledge frontier; negativity bias; Projects; Innovation and Invention; Information; Diversity; Judgments;

Citation:

Lane, Jacqueline N., Misha Teplitskiy, Gary Gray, Hardeep Ranu, Michael Menietti, Eva C. Guinan, and Karim R. Lakhani. "When Do Experts Listen to Other Experts? The Role of Negative Information in Expert Evaluations for Novel Projects." Harvard Business School Working Paper, No. 21-007, July 2020. View Details

Cite View Details SSRN Read Now

Faculty & Research

Looking Across and Looking Beyond the Knowledge Frontier: Intellectual Distance and Resource Allocation in Science

Abstract

Citation:

About the Author

More from the Author