Skip to Main Content
HBS Home
  • About
  • Academic Programs
  • Alumni
  • Faculty & Research
  • Baker Library
  • Giving
  • Harvard Business Review
  • Initiatives
  • News
  • Recruit
  • Map / Directions
Faculty & Research
  • Faculty
  • Research
  • Featured Topics
  • Academic Units
  • …→
  • Harvard Business School→
  • Faculty & Research→
Publications
Publications
  • December 2014
  • Article
  • Harvard Business Review

The Discipline of Business Experimentation

By: Stefan Thomke and Jim Manzi
  • Format:Print
ShareBar

Abstract

The data you already have can't tell you how customers will react to innovations. To discover if a truly novel concept will succeed, you must subject it to a rigorous experiment. In most companies, tests do not adhere to scientific and statistical principles. As a result, managers often end up interpreting statistical noise as causation—and making bad decisions. To conduct experiments that are worth the expense and effort, companies need to ask themselves several questions: Does the experiment have a clear purpose? Managers must figure out exactly what they want to learn in order to determine if testing is the best approach. Have stakeholders made a commitment to abide by the results? Are they willing to walk away from a project if the findings suggest they should? Is the experiment doable? The complexity of the variables in a business experiment and their interactions can make it difficult to determine cause-and-effect relationships. Choosing the right sample size is important. How can we ensure reliable results? Randomized field trials, "blind" tests, and big data can help. Have we gotten the most value out of the experiment? Conducting the experiment is just the beginning. Use the data to assess which components of a new initiative might have the highest ROI or the markets where it is most likely to be successful.

Citation

Thomke, Stefan, and Jim Manzi. "The Discipline of Business Experimentation." Harvard Business Review 92, no. 12 (December 2014): 70–79.
  • Find it at Harvard
  • Register to Read

About The Author

Stefan H. Thomke

Technology and Operations Management
→More Publications

More from the Authors

    • Harvard Business Review Digital Articles

    Capture New Value from Your Existing Tech Infrastructure

    By: Stefan Thomke and Anthony Rodrigo
    • Harvard Business Review

    Act Like a Scientist: Great Leaders Challenge Assumptions, Run Experiments, and Follow the Evidence

    By: Stefan Thomke and Gary W. Loveman
    • Research-Technology Management

    High Velocity Business Experiments

    By: Stefan Thomke and Jim Euchner
More from the Authors
  • Capture New Value from Your Existing Tech Infrastructure By: Stefan Thomke and Anthony Rodrigo
  • Act Like a Scientist: Great Leaders Challenge Assumptions, Run Experiments, and Follow the Evidence By: Stefan Thomke and Gary W. Loveman
  • High Velocity Business Experiments By: Stefan Thomke and Jim Euchner
ǁ
Campus Map
Harvard Business School
Soldiers Field
Boston, MA 02163
→Map & Directions
→More Contact Information
  • Make a Gift
  • Site Map
  • Jobs
  • Harvard University
  • Trademarks
  • Policies
  • Accessibility
  • Digital Accessibility
Copyright © President & Fellows of Harvard College