Experimentation and Startup Performance

Rembrand  Koning; Sharique  Hasan; Aaron  Chatterji

Working Paper | HBS Working Paper Series | 2019

Experimentation and Startup Performance: Evidence from A/B testing

by Rembrand Koning, Sharique Hasan and Aaron Chatterji

Abstract

Recent work argues that experimentation is the appropriate framework for entrepreneurial strategy. We investigate this proposition by exploiting the time-varying adoption of A/B testing technology, which has drastically reduced the cost of experimentally testing business ideas. This paper provides the first evidence of how digital experimentation affects the performance of a large sample of high-technology startups using data that tracks their growth, technology use, and product launches. We find that, despite its prominence in the business press, relatively few firms have adopted A/B testing. However, among those that do, we find increased performance on several critical dimensions, including page views and new product features. Furthermore, A/B testing is positively related to tail outcomes, with younger ventures failing faster and older firms being more likely to scale. Firms with experienced managers also derive more benefits from A/B testing. Our results inform the emerging literature on entrepreneurial strategy and how digitization and data-driven decision-making are shaping strategy.

Keywords: experimentation; A/B Testing; Entrepreneurship; Strategy; Business Startups; Technology; Performance;

Language: English

Format: Print

48 pages

Read Now

Citation:

Koning, Rembrand, Sharique Hasan, and Aaron Chatterji. "Experimentation and Startup Performance: Evidence from A/B testing." Harvard Business School Working Paper, No. 20-018, August 2019. (SSRN Working Paper Series, No. 3440291, August 2019.)

When do conversations lead people to generate better ideas? We conducted a field experiment at a startup boot camp to evaluate the impact of informal conversations on the quality of product ideas generated by participants. Specifically, we examine how the personality of an innovator (openness to experience, capturing creativity) and the personalities of her randomly assigned conversational peers (extroversion, measuring willingness to share information) affects the innovator's ideas. We find that open innovators who spoke with extroverted peers generated significantly better ideas than others at the boot camp. However, closed individuals produced mediocre ideas regardless of the individuals they interviewed, suggesting limited benefits of conversations for these people. More surprisingly, open individuals, who are believed to be inherently creative, produced worse ideas after they spoke with introverted peers, suggesting individual creativity's dependence on external information. Our study demonstrates the importance of considering the traits of both innovators and their conversational peers in predicting who will generate the best ideas.

We conduct a field experiment at an entrepreneurship bootcamp to investigate whether interaction with proximate peers shapes a nascent startup team's performance. We find that teams whose members lack prior ties to others at the bootcamp experience peer effects that influence the quality of their product prototypes. A one-standard-deviation increase in the performance of proximate teams is related to a two-thirds standard-deviation improvement for a focal team. In contrast, we find that teams whose members have many prior ties interact less frequently with proximate peers, and thus their performance is unaffected by nearby teams. Our findings highlight how prior social connections, which are often a source of knowledge and influence, can limit new interactions and thus the ability of organizations to leverage peer effects to improve the performance of their members.

Working Paper | 2019

The Value of Communication: Evidence from a Field Experiment with Entrepreneurs in Togo

Stefan Dimitriadis and Rembrand Koning

Why do some entrepreneurs benefit from their portfolio of peer advisers while others do not? In this study, we argue that communication practices are an important but overlooked factor in the formation of useful advice relationships between entrepreneurs, particularly in the context of developing economies. We hypothesize that improving entrepreneurs’ communication practices will affect the relationships they form and have implications for their business performance. To test our theory, we conducted a field experiment in Togo with 301 entrepreneurs who were randomized into a communication practices intervention that was embedded in a business training program. We found that entrepreneurs who were exposed to better communication practices perceived interactions more cooperatively and exchanged more information during those interactions. Moreover, improving communication practices also led to a 50 percent increase in the number of relationships entrepreneurs formed with peers. These relationships exhibited more matching based on skill and were more ethnically diverse. Finally, communication practices training also substantially increased entrepreneurs’ business performance. Our findings highlight how communication practices play a central role in entrepreneurs’ ability to form portfolios of relationships and perform in challenging business environments.

Keywords: communication practices; business relationships; Entrepreneurship; Communication; Developing Countries and Economies; Africa; Togo;

Citation:

Dimitriadis, Stefan, and Rembrand Koning. "The Value of Communication: Evidence from a Field Experiment with Entrepreneurs in Togo." SSRN Working Paper Series, No. 3459643, September 2019. View Details

Cite View Details Read Now