Biased Sampling of Early Users and the Direction of Startup Innovation
Using data from a prominent online platform for launching new digital products, we document that the composition of the platform's ‘beta testers’ on the day a new product is launched has a systematic and persistent impact on the venture's success. Specifically, we use word embedding methods to classify products launched on this platform as more or less focused on the needs of female customers, and show that female-focused products launched on a typical day—when nine-in-ten users on the platform are men—experience up to 40% less growth a year after launch. By isolating variation in the composition of beta testers that is unrelated to the characteristics of launched products, we further show that a startup's user growth and success with VC financing can be traced to variation in the number of female beta-testers on the platform the day its product is launched. Overall, our findings suggest that the composition of early users can create systematic bias in signals of a startup's market potential, with consequential real effects—including a dearth of innovations aimed at consumers who are underrepresented among early users.
Citation:
Cao, Ruiqing, Rembrand Koning, and Ramana Nanda. "Biased Sampling of Early Users and the Direction of Startup Innovation." Harvard Business School Working Paper, No. 21-059, November 2020. View Details