Course Overview:

Experiments are increasingly used to guide managerial decision-making. Large tech companies run thousands of experiments per year to test everything from product design to pricing to which markets to enter and how. Human resource and people analytics departments use experiments to understand which incentive systems are most effective. Marketers run experiments to understand which advertisements increase sales. Governments run experiments to evaluate policies in areas ranging from education to encouraging entrepreneurship.

Effective strategy and managerial judgment in today’s world requires a foundational understanding of experiments, a core component of the modern analytics toolkit. The overarching goal of the course is to help students gain effective tools and leverage experimental insights to inform managerial and strategic decisions in an increasingly complex and data-driven world.

Course Objectives

The course will focus on three broad questions:

• When should organizations run experiments? Through case studies, the course will help students to approach managerial problems with an experimental mindset, and shed light on the types of insight that can be gained from experiments, as well as the disasters that can be avoided through careful experimental analysis.
• What are the core concepts involved in designing and interpreting an experiment? The course will develop frameworks for interpreting, designing, and analyzing experiments, provide a solid understanding of experimental methods, and offer insights on how to identify the mistakes even smart leaders and managers make when they try to gather and interpret data that will guide their strategic decisions.
• How should managers incorporate experimental evidence into managerial decisions? This course will develop frameworks for students to better understand and address the challenges that arise in using experiments to inform managerial judgment. Examining real-world experiments that organizations have run, students will gain an understanding of how experimental evidence fits into the broader picture and how managers can move from experimental evidence to decisions.

This course will explore a variety of contexts in which experiments are used, such as platform design, strategy, operations, human resources, and marketing. The course will consist of a combination of case studies, book chapters and articles, as well as guest discussions and panels. Tentative guests include managers and data scientists from organizations such as Uber, Facebook, Airbnb, ThredUp, Yelp, and the Behavioural Insights Team. The course will involve a final project. No technical background is required.

For more discussion of the content of the types of topics covered in the course, please see this article: https://hbswk.hbs.edu/item/steer-clear-of-the-blind-spots-that-derail-experiments

Copyright © 2021 President & Fellows of Harvard College. All Rights Reserved.