Skip to Main Content
HBS Home
  • About
  • Academic Programs
  • Alumni
  • Faculty & Research
  • Baker Library
  • Giving
  • Harvard Business Review
  • Initiatives
  • News
  • Recruit
  • Map / Directions
Faculty & Research
  • Faculty
  • Research
  • Featured Topics
  • Academic Units
  • …→
  • Harvard Business School→
  • Faculty & Research→
Publications
Publications
  • March–April 2020
  • Article
  • Harvard Business Review

Avoid the Pitfalls of A/B Testing

By: Iavor I. Bojinov, Guillaume Sait-Jacques and Martin Tingley
  • Format:Print
ShareBar

Abstract

Online experiments measuring whether “A,” usually the current approach, is inferior to “B,” a proposed improvement, have become integral to the product-development cycle, especially at digital enterprises. But often firms make serious mistakes in conducting these tests: They focus on the average, instead of looking at how a change impacts different customer segments. They forget that customers are connected and that their interactions can affect test outcomes. And they run tests for too short a period, failing to recognize that customers’ reactions can change over time. This article describes how to avoid all those traps by applying techniques that LinkedIn and Netflix have used to produce better insights.

Keywords

A/B Testing; Experiment Design; Social Networks; Product Development; Performance Improvement; Measurement and Metrics; Social Media

Citation

Bojinov, Iavor I., Guillaume Sait-Jacques, and Martin Tingley. "Avoid the Pitfalls of A/B Testing." Harvard Business Review 98, no. 2 (March–April 2020): 48–53.
  • Find it at Harvard
  • Register to Read

About The Author

Iavor I. Bojinov

Technology and Operations Management
→More Publications

More from the Authors

    • April 2025 (Revised April 2025)
    • Faculty Research

    JPMorganChase: Leadership in the Age of GenAI

    By: Iavor I. Bojinov, Karim R. Lakhani and David Lane
    • March 2025
    • Faculty Research

    Building an AI Factory at Procter & Gamble

    By: Iavor I Bojinov, Karim R. Lakhani and Alexis Lefort
    • February 2025
    • Faculty Research

    Building a Generative AI-Powered MBA Course Scheduler

    By: Iavor Bojinov and Annika Hildebrandt
More from the Authors
  • JPMorganChase: Leadership in the Age of GenAI By: Iavor I. Bojinov, Karim R. Lakhani and David Lane
  • Building an AI Factory at Procter & Gamble By: Iavor I Bojinov, Karim R. Lakhani and Alexis Lefort
  • Building a Generative AI-Powered MBA Course Scheduler By: Iavor Bojinov and Annika Hildebrandt
ǁ
Campus Map
Harvard Business School
Soldiers Field
Boston, MA 02163
→Map & Directions
→More Contact Information
  • Make a Gift
  • Site Map
  • Jobs
  • Harvard University
  • Trademarks
  • Policies
  • Accessibility
  • Digital Accessibility
Copyright © President & Fellows of Harvard College.