Publications
Publications
- March–April 2020
- Harvard Business Review
Avoid the Pitfalls of A/B Testing
By: Iavor I. Bojinov, Guillaume Sait-Jacques and Martin Tingley
Abstract
Online experiments measuring whether “A,” usually the current approach, is inferior to “B,” a proposed improvement, have become integral to the product-development cycle, especially at digital enterprises. But often firms make serious mistakes in conducting these tests: They focus on the average, instead of looking at how a change impacts different customer segments. They forget that customers are connected and that their interactions can affect test outcomes. And they run tests for too short a period, failing to recognize that customers’ reactions can change over time. This article describes how to avoid all those traps by applying techniques that LinkedIn and Netflix have used to produce better insights.
Keywords
A/B Testing; Experiment Design; Social Networks; Product Development; Performance Improvement; Measurement and Metrics; Social Media
Citation
Bojinov, Iavor I., Guillaume Sait-Jacques, and Martin Tingley. "Avoid the Pitfalls of A/B Testing." Harvard Business Review 98, no. 2 (March–April 2020): 48–53.