Skip to Main Content
HBS Home
  • About
  • Academic Programs
  • Alumni
  • Faculty & Research
  • Baker Library
  • Giving
  • Harvard Business Review
  • Initiatives
  • News
  • Recruit
  • Map / Directions
Faculty & Research
  • Faculty
  • Research
  • Featured Topics
  • Academic Units
  • …→
  • Harvard Business School→
  • Faculty & Research→
Publications
Publications
  • 2022
  • Working Paper
  • HBS Working Paper Series

Sampling Bias in Entrepreneurial Experiments

By: Ruiqing Cao, Rembrand Koning and Ramana Nanda
  • Format:Print
  • | Language:English
  • | Pages:99
ShareBar

Abstract

Using data from a prominent online platform for launching new digital products, we document that ‘sampling bias’—defined as the difference between a startup’s target customer base and the actual sample on which early ‘beta tests’ are conducted—has a systematic and persistent impact on the venture’s success. Specifically, we show that products with a female-focused target market launching on a typical day, when nine in ten users on this platform are men, experience 45% less growth a year after launch than those for whom the target market is more male-focused. By isolating exogenous variation in the composition of beta testers unrelated to the characteristics of launched products on that day, we find that on days when there are unexpectedly more women beta testers on the platform—reducing the amount of sampling bias for female-focused products—the gender-performance gap shrinks towards zero. Our results highlight how sampling bias can lead to fewer successfully commercialized innovations for consumers who are underrepresented among early users.

Keywords

Target Market; Sampling Biases; Beta Testing; Product Launch; Entrepreneurship

Citation

Cao, Ruiqing, Rembrand Koning, and Ramana Nanda. "Sampling Bias in Entrepreneurial Experiments." Harvard Business School Working Paper, No. 21-059, November 2020. (Revised July 2022.)
  • SSRN
  • Read Now

About The Authors

Rembrand M. Koning

Strategy
→More Publications

Ramana Nanda

Entrepreneurial Management
→More Publications

More from the Authors

    • October 2022
    • Faculty Research

    Lyra Health: Transforming Mental Health

    By: Rembrand Koning and Nicole Keller
    • October 2022
    • Faculty Research

    Single Earth: Science White Paper Supplement

    By: Rembrand Koning and Emer Moloney
    • October 2022
    • Faculty Research

    Single.Earth

    By: Rembrand Koning and Emer Moloney
More from the Authors
  • Lyra Health: Transforming Mental Health By: Rembrand Koning and Nicole Keller
  • Single Earth: Science White Paper Supplement By: Rembrand Koning and Emer Moloney
  • Single.Earth By: Rembrand Koning and Emer Moloney
ǁ
Campus Map
Harvard Business School
Soldiers Field
Boston, MA 02163
→Map & Directions
→More Contact Information
  • Make a Gift
  • Site Map
  • Jobs
  • Harvard University
  • Trademarks
  • Policies
  • Accessibility
  • Digital Accessibility
Copyright © President & Fellows of Harvard College