Performance Responses to Competition Across Skill-Levels in Rank Order Tournaments: Field Evidence and Implications for Tournament Design

Kevin J. Boudreau; Karim R. Lakhani; Michael E. Menietti

Article | RAND Journal of Economics | Spring 2016

Performance Responses to Competition Across Skill-Levels in Rank Order Tournaments: Field Evidence and Implications for Tournament Design

by Kevin J. Boudreau, Karim R. Lakhani and Michael E. Menietti

Abstract

Tournaments are widely used in the economy to organize production and innovation. We study individual contestant-level data from 2,796 contestants in 774 software algorithm design contests with random assignment. Precisely conforming to theory predictions, the performance response to added contestants varies non-monotonically across contestants of different abilities, most respond negatively to competition, and highest-skilled contestants respond positively. In counterfactual simulations, we interpret a number of tournament design policies (number of competitors, prize allocation and structure, divisionalization, open entry) as a means of reconciling non-monotonic incentive responses to competition, effectively manipulating the number and skills distribution of contestants facing one another.

Keywords: Competition; Innovation Strategy;

Format: Print

Find at Harvard Read Now

Citation:

Boudreau, Kevin J., Karim R. Lakhani, and Michael E. Menietti. "Performance Responses to Competition Across Skill-Levels in Rank Order Tournaments: Field Evidence and Implications for Tournament Design." RAND Journal of Economics 47, no. 1 (Spring 2016): 140–165.

Working Paper | HBS Working Paper Series | 2020

When Do Experts Listen to Other Experts? The Role of Negative Information in Expert Evaluations for Novel Projects

Jacqueline N. Lane, Misha Teplitskiy, Gary Gray, Hardeep Ranu, Michael Menietti, Eva C. Guinan and Karim R. Lakhani

The evaluation of novel projects lies at the heart of scientific and technological innovation, and yet literature suggests that this process is subject to inconsistency and potential biases. This paper investigates the role of information sharing among experts as the driver of evaluation decisions. We designed and executed two field experiments in two separate grant funding opportunities at a leading research university to explore evaluators’ receptivity to assessments from other evaluators. Collectively, our experiments mobilized 369 evaluators from seven universities to evaluate 97 projects resulting in 760 proposal-evaluation pairs and over $300,000 in awards. We exogenously varied two key aspects of information sharing: 1) the intellectual distance between each focal evaluator and the other evaluators and 2) the relative valence (positive and negative) of others’ scores, to determine how these treatments affect the focal evaluator’s propensity to change the initial score. Although the intellectual similarity treatment did not yield a measurable effect, we found causal evidence of negativity bias, where evaluators are more likely to lower their scores after seeing critical scores than raise them after seeing better scores. Qualitative coding and topic modeling of the evaluators’ justifications for score changes reveal that exposures to low scores prompted greater attention to uncovering weaknesses, whereas exposures to neutral or high scores were associated with strengths, along with greater emphasis on non-evaluation criteria, such as confidence in one’s judgment. Overall, information sharing among expert evaluators can lead to more conservative allocation decisions that favors protecting against failure than maximizing success.

Keywords: Project evaluation; innovation; knowledge frontier; negativity bias; Projects; Innovation and Invention; Information; Diversity; Judgments;

Citation:

Lane, Jacqueline N., Misha Teplitskiy, Gary Gray, Hardeep Ranu, Michael Menietti, Eva C. Guinan, and Karim R. Lakhani. "When Do Experts Listen to Other Experts? The Role of Negative Information in Expert Evaluations for Novel Projects." Harvard Business School Working Paper, No. 21-007, July 2020. View Details

Cite View Details SSRN Read Now

Faculty & Research

Performance Responses to Competition Across Skill-Levels in Rank Order Tournaments: Field Evidence and Implications for Tournament Design

Abstract

Citation:

About the Author

More from the Author