Skip to Main Content
HBS Home
  • About
  • Academic Programs
  • Alumni
  • Faculty & Research
  • Baker Library
  • Giving
  • Harvard Business Review
  • Initiatives
  • News
  • Recruit
  • Map / Directions
Faculty & Research
  • Faculty
  • Research
  • Featured Topics
  • Academic Units
  • …→
  • Harvard Business School→
  • Faculty & Research→
Publications
Publications
  • March–April 2020
  • Article
  • Harvard Business Review

Avoid the Pitfalls of A/B Testing

By: Iavor I. Bojinov, Guillaume Sait-Jacques and Martin Tingley
  • Format:Print
ShareBar

Abstract

Online experiments measuring whether “A,” usually the current approach, is inferior to “B,” a proposed improvement, have become integral to the product-development cycle, especially at digital enterprises. But often firms make serious mistakes in conducting these tests: They focus on the average, instead of looking at how a change impacts different customer segments. They forget that customers are connected and that their interactions can affect test outcomes. And they run tests for too short a period, failing to recognize that customers’ reactions can change over time. This article describes how to avoid all those traps by applying techniques that LinkedIn and Netflix have used to produce better insights.

Keywords

A/B Testing; Experiment Design; Social Networks; Product Development; Performance Improvement; Measurement and Metrics; Social Media

Citation

Bojinov, Iavor I., Guillaume Sait-Jacques, and Martin Tingley. "Avoid the Pitfalls of A/B Testing." Harvard Business Review 98, no. 2 (March–April 2020): 48–53.
  • Find it at Harvard
  • Register to Read

About The Author

Iavor I. Bojinov

Technology and Operations Management
→More Publications

More from the Authors

    • June 2022
    • Faculty Research

    Causal Inference

    By: Iavor Bojinov, Michael Parzen and Paul J. Hamilton
    • March 2022 (Revised March 2022)
    • Faculty Research

    Prediction & Machine Learning

    By: Iavor I. Bojinov, Michael Parzen and Paul J. Hamilton
    • March 2022 (Revised March 2022)
    • Faculty Research

    Linear Regression

    By: Iavor I. Bojinov, Michael Parzen and Paul J. Hamilton
More from the Authors
  • Causal Inference By: Iavor Bojinov, Michael Parzen and Paul J. Hamilton
  • Prediction & Machine Learning By: Iavor I. Bojinov, Michael Parzen and Paul J. Hamilton
  • Linear Regression By: Iavor I. Bojinov, Michael Parzen and Paul J. Hamilton
ǁ
Campus Map
Harvard Business School
Soldiers Field
Boston, MA 02163
→Map & Directions
→More Contact Information
  • Make a Gift
  • Site Map
  • Jobs
  • Harvard University
  • Trademarks
  • Policies
  • Accessibility
  • Digital Accessibility
Copyright © President & Fellows of Harvard College