Skip to Main Content
HBS Home
  • About
  • Academic Programs
  • Alumni
  • Faculty & Research
  • Baker Library
  • Giving
  • Harvard Business Review
  • Initiatives
  • News
  • Recruit
  • Map / Directions
Faculty & Research
  • Faculty
  • Research
  • Featured Topics
  • Academic Units
  • …→
  • Harvard Business School→
  • Faculty & Research→
Publications
Publications
  • September 2022
  • Article
  • Management Science

Experimentation and Startup Performance: Evidence from A/B Testing

By: Rembrand Koning, Sharique Hasan and Aaron Chatterji
  • Format:Print
  • | Pages:20
ShareBar

Abstract

Recent scholarship has argued that experimentation should be the organizing principle for entrepreneurial strategy. Experimentation leads to organizational learning, which drives improvements in firm performance. We investigate this proposition by exploiting the time-varying adoption of A/B testing technology, which has drastically reduced the cost of testing business ideas. Our results provide the first evidence on how digital experimentation affects a large sample of high-technology startups using data that tracks their growth, technology use, and products. We find that while relatively few firms adopt A/B testing, among those that do, performance improves by 30% to 100% after a year of use. We then argue that this substantial effect and relatively low adoption rate arises because startups do not only test one-off incremental changes but also use A/B testing as part of a broader strategy of experimentation. Qualitative insights and additional quantitative analyses show that experimentation improves organizational learning, which helps startups develop more new products, identify and scale promising ideas, and fail faster when they receive negative signals. These findings inform the literatures on entrepreneurial strategy, organizational learning, and data-driven decision-making.

Keywords

Experimentation; A/B Testing; Data-driven Decision-making; Organizational Learning; Entrepreneurship; Strategy; Business Startups; Learning; Performance; Decision Making

Citation

Koning, Rembrand, Sharique Hasan, and Aaron Chatterji. "Experimentation and Startup Performance: Evidence from A/B Testing." Management Science 68, no. 9 (September 2022): 6434–6453.
  • Find it at Harvard
  • Read Now

About The Author

Rembrand M. Koning

Strategy
→More Publications

More from the Authors

    • October 2022
    • Faculty Research

    Lyra Health: Transforming Mental Health

    By: Rembrand Koning and Nicole Keller
    • October 2022
    • Faculty Research

    Single Earth: Science White Paper Supplement

    By: Rembrand Koning and Emer Moloney
    • October 2022
    • Faculty Research

    Single.Earth

    By: Rembrand Koning and Emer Moloney
More from the Authors
  • Lyra Health: Transforming Mental Health By: Rembrand Koning and Nicole Keller
  • Single Earth: Science White Paper Supplement By: Rembrand Koning and Emer Moloney
  • Single.Earth By: Rembrand Koning and Emer Moloney
ǁ
Campus Map
Harvard Business School
Soldiers Field
Boston, MA 02163
→Map & Directions
→More Contact Information
  • Make a Gift
  • Site Map
  • Jobs
  • Harvard University
  • Trademarks
  • Policies
  • Accessibility
  • Digital Accessibility
Copyright © President & Fellows of Harvard College