Skip to Main Content
HBS Home
  • About
  • Academic Programs
  • Alumni
  • Faculty & Research
  • Baker Library
  • Giving
  • Harvard Business Review
  • Initiatives
  • News
  • Recruit
  • Map / Directions
Faculty & Research
  • Faculty
  • Research
  • Featured Topics
  • Academic Units
  • …→
  • Harvard Business School→
  • Faculty & Research→
Publications
Publications
  • October 2021
  • Article
  • Management Science

Judgment Aggregation in Creative Production: Evidence from the Movie Industry

By: Hong Luo, Jeffrey T. Macher and Michael Wahlen
  • Format:Print
ShareBar

Abstract

We study a novel, low-cost approach to aggregating judgment from a large number of industry experts on ideas that they encounter in their normal course of business. Our context is the movie industry, in which customer appeal is difficult to predict and investment costs are high. The Black List, an annual publication, ranks unproduced scripts based on anonymous nominations from film executives. This approach entails an inherent trade-off: low participation costs enable high response rates; but nominations lack standard criteria, and which voters see which ideas is unobservable and influenced by various factors. Despite these challenges, we find that such aggregation is predictive: listed scripts are substantially more likely to be released than observably similar but unlisted scripts, and, conditional on release and investment levels, listed scripts generate higher box office revenues. We also find that this method mitigates entry barriers for less-experienced writers as: (i) their scripts are more likely to be listed than those by experienced writers and to rank higher if listed; (ii) within scripts by less-experienced writers, being listed is associated with a higher release rate. Yet, the gap in release probabilities relative to experienced writers remains large, even for top-ranked scripts. These results can be explained by the premise that scripts from less-experienced writers are more visible among eligible voters than scripts from experienced writers. This highlights idea visibility as an important determinant of votes and surfaces the trade-offs, as well as potential limitations, associated with such methods.

Keywords

Judgment Aggregation; Quality Uncertainty; Creative Industry; Project Evaluation And Selection; Creativity; Film Entertainment; Judgments; Motion Pictures and Video Industry

Citation

Luo, Hong, Jeffrey T. Macher, and Michael Wahlen. "Judgment Aggregation in Creative Production: Evidence from the Movie Industry." Management Science 67, no. 10 (October 2021): 6358–6377.
  • Find it at Harvard
  • Read Now

About The Author

Hong Luo

Strategy
→More Publications

More from the Authors

    • 2022
    • Faculty Research

    Laboratory Safety and Research Productivity

    By: Alberto Galasso, Hong Luo and Brooklynn Zhu
    • May 2022
    • American Economic Journal: Economic Policy

    When Does Product Liability Risk Chill Innovation? Evidence from Medical Implants

    By: Alberto Galasso and Hong Luo
    • March 2022 (Revised April 2022)
    • Faculty Research

    Spotify's Audio-First Strategy: Leading the Podcasting Market

    By: Hong Luo and Carol Lin
More from the Authors
  • Laboratory Safety and Research Productivity By: Alberto Galasso, Hong Luo and Brooklynn Zhu
  • When Does Product Liability Risk Chill Innovation? Evidence from Medical Implants By: Alberto Galasso and Hong Luo
  • Spotify's Audio-First Strategy: Leading the Podcasting Market By: Hong Luo and Carol Lin
ǁ
Campus Map
Harvard Business School
Soldiers Field
Boston, MA 02163
→Map & Directions
→More Contact Information
  • Make a Gift
  • Site Map
  • Jobs
  • Harvard University
  • Trademarks
  • Policies
  • Accessibility
  • Digital Accessibility
Copyright © President & Fellows of Harvard College