Skip to Main Content
HBS Home
  • About
  • Academic Programs
  • Alumni
  • Faculty & Research
  • Baker Library
  • Giving
  • Harvard Business Review
  • Initiatives
  • News
  • Recruit
  • Map / Directions
Faculty & Research
  • Faculty
  • Research
  • Featured Topics
  • Academic Units
  • …→
  • Harvard Business School→
  • Faculty & Research→
Publications
Publications
  • 2020
  • Working Paper
  • HBS Working Paper Series

Digital Experimentation and Startup Performance: Evidence from A/B Testing

By: Rembrand Koning, Sharique Hasan and Aaron Chatterji
  • Format:Print
  • | Language:English
  • | Pages:77 
ShareBar

Abstract

Recent work argues that experimentation is the appropriate framework for entrepreneurial strategy. We investigate this proposition by exploiting the time-varying adoption of A/B testing technology, which has drastically reduced the cost of experimentally testing business ideas. This paper provides the first evidence on how digital experimentation affects a large sample of high-technology startups using data that tracks their growth, technology use, and product launches. Despite its prominence in the business press, relatively few firms have adopted A/B testing tools. However, among those that do, we find increased performance on several critical dimensions, including page views, code changes, and new product features. These results are robust to our use of instrumental variables and synthetic control models, where we leverage Google’s launch of a new A/B testing tool. Our results also show that the value of A/B testing increases over time and is positively related to tail outcomes, with adopting firms both scaling and failing faster. Our results inform the emerging literature on entrepreneurial strategy and illustrate the growing importance of digitization and data-driven decision-making.

Keywords

Experimentation; A/b Testing; Data-driven Decision-making; Entrepreneurship; Strategy; Business Startups; Technology; Performance

Citation

Koning, Rembrand, Sharique Hasan, and Aaron Chatterji. "Digital Experimentation and Startup Performance: Evidence from A/B Testing." Harvard Business School Working Paper, No. 20-018, August 2019. (Revised September 2020. SSRN Working Paper Series, No. 3440291, August 2019)
  • SSRN
  • Read Now

About The Author

Rembrand M. Koning

Strategy
→More Publications

More from the Authors

    • January 2021
    • Faculty Research

    mPharma (B)

    By: Rembrand Koning, John D. Macomber, Pippa Tubman Armerding and Wale Lawal
    • January 2021
    • Faculty Research

    mPharma (A)

    By: Rembrand Koning, John D. Macomber, Pippa Tubman Armerding and Wale Lawal
    • December 2020
    • Faculty Research

    Thinx, Inc.—Breaking Barriers in Feminine Care (Abridged)

    By: Rembrand Koning, Elie Ofek and Nicole Keller
More from the Authors
  • mPharma (B) By: Rembrand Koning, John D. Macomber, Pippa Tubman Armerding and Wale Lawal
  • mPharma (A) By: Rembrand Koning, John D. Macomber, Pippa Tubman Armerding and Wale Lawal
  • Thinx, Inc.—Breaking Barriers in Feminine Care (Abridged) By: Rembrand Koning, Elie Ofek and Nicole Keller
ǁ
Campus Map
Harvard Business School
Soldiers Field
Boston, MA 02163
→Map & Directions
→More Contact Information
  • Make a Gift
  • Site Map
  • Jobs
  • Harvard University
  • Trademarks
  • Policies
  • Digital Accessibility
Copyright © President & Fellows of Harvard College