Skip to Main Content
HBS Home
  • About
  • Academic Programs
  • Alumni
  • Faculty & Research
  • Baker Library
  • Giving
  • Harvard Business Review
  • Initiatives
  • News
  • Recruit
  • Map / Directions
Faculty & Research
  • Faculty
  • Research
  • Featured Topics
  • Academic Units
  • …→
  • Harvard Business School→
  • Faculty & Research→
Publications
Publications
  • 2020
  • Working Paper
  • HBS Working Paper Series

The Effects of Hierarchy on Learning and Performance in Business Experimentation

By: Sourobh Ghosh, Stefan Thomke and Hazjier Pourkhalkhali
  • Format:Print
  • | Language:English
  • | Pages:42
ShareBar

Abstract

Do senior managers help or hurt business experiments? Despite the widespread adoption of business experiments to guide strategic decision-making, we lack a scholarly understanding of what role senior managers play in firm experimentation. Using proprietary data of live business experiments from the widely used A/B testing platform, this paper estimates the association of management hierarchy with learning from experiments and their performance outcomes across industries and contexts. Our findings suggest that senior management’s association is mixed. On the one hand, senior managers’ involvement associates with bolder experiments that create more statistically significant learning signals aiding in the exploration of new strategic directions. On the other hand, their involvement associates with less cause-and-effect learning that is instrumental to optimization and performance improvements. Our results contribute to a burgeoning literature on experimentation in strategy, while helping articulate limits that organizational design might place on data-driven decision-making. Furthermore, we describe different experimental learning modes in the formation of strategy, offering important implications for how managers can modulate search and performance outcomes.

Keywords

Experimentation; Innovation; Search; New Product Development; Innovation and Invention; Organizational Design; Learning; Performance

Citation

Ghosh, Sourobh, Stefan Thomke, and Hazjier Pourkhalkhali. "The Effects of Hierarchy on Learning and Performance in Business Experimentation." Harvard Business School Working Paper, No. 20-081, February 2020.
  • Read Now

About The Author

Stefan H. Thomke

Technology and Operations Management
→More Publications

More from the Authors

    • Harvard Business Review Digital Articles

    Capture New Value from Your Existing Tech Infrastructure

    By: Stefan Thomke and Anthony Rodrigo
    • Harvard Business Review

    Act Like a Scientist: Great Leaders Challenge Assumptions, Run Experiments, and Follow the Evidence

    By: Stefan Thomke and Gary W. Loveman
    • Research-Technology Management

    High Velocity Business Experiments

    By: Stefan Thomke and Jim Euchner
More from the Authors
  • Capture New Value from Your Existing Tech Infrastructure By: Stefan Thomke and Anthony Rodrigo
  • Act Like a Scientist: Great Leaders Challenge Assumptions, Run Experiments, and Follow the Evidence By: Stefan Thomke and Gary W. Loveman
  • High Velocity Business Experiments By: Stefan Thomke and Jim Euchner
ǁ
Campus Map
Harvard Business School
Soldiers Field
Boston, MA 02163
→Map & Directions
→More Contact Information
  • Make a Gift
  • Site Map
  • Jobs
  • Harvard University
  • Trademarks
  • Policies
  • Accessibility
  • Digital Accessibility
Copyright © President & Fellows of Harvard College