More from the Author

• Article | Harvard Business Review | March–April 2020

  Building A Culture of Experimentation

  Stefan H. Thomke

  Why don’t organizations test more? After examining this question for several years, I can tell you that the central reason is culture. As companies try to scale up their experimentation capacity, they often find that the obstacles are not tools and technology but shared behaviors, beliefs, and values. For every online experiment that succeeds, nearly 10 don’t—and in the eyes of many organizations that emphasize efficiency, predictability, and “winning,” those failures are wasteful. To successfully innovate, companies need to make experimentation an integral part of everyday life—even when budgets are tight. That means creating an environment where employees’ curiosity is nurtured, data trumps opinion, anyone (not just people in R&D) can conduct or commission a test, all experiments are done ethically, and managers embrace a new model of leadership. In this article, I look at several companies that have managed to do those things well.

  Keywords: experimentation; culture; innovation; online; customer experience; Organizational Culture; Innovation and Invention; Online Technology; Attitudes; Decision Making; Change; Leadership;

  Citation:

  Thomke, Stefan H. "Building A Culture of Experimentation." Harvard Business Review 98, no. 2 (March–April 2020): 40–48.  View Details

  CiteView DetailsFind at Harvard Register to Read

  Related

• Working Paper | HBS Working Paper Series | 2020

  The Effects of Hierarchy on Learning and Performance in Business Experimentation

  Sourobh Ghosh, Stefan H. Thomke and Hazjier Pourkhalkhali

  Do senior managers help or hurt business experiments? Despite the widespread adoption of business experiments to guide strategic decision-making, we lack a scholarly understanding of what role senior managers play in firm experimentation. Using proprietary data of live business experiments from the widely-used A/B testing platform, Optimizely, this paper estimates the association of management hierarchy with learning from experiments and their performance outcomes across industries and contexts. Our findings suggest that senior management’s association is mixed. On the one hand, senior managers’ involvement associates with bolder experiments that create more statistically significant learning signals aiding in the exploration of new strategic directions. On the other hand, their involvement associates with less cause-and-effect learning that is instrumental to optimization and performance improvements. Our results contribute to a burgeoning literature on experimentation in strategy, while helping articulate limits that organizational design might place on data-driven decision-making. Furthermore, we describe different experimental learning modes in the formation of strategy, offering important implications for how managers can modulate search and performance outcomes.

  Keywords: experimentation; innovation; search; New Product Development; Innovation and Invention; Organizational Design; Learning; Performance;

  Citation:

  Ghosh, Sourobh, Stefan H. Thomke, and Hazjier Pourkhalkhali. "The Effects of Hierarchy on Learning and Performance in Business Experimentation." Harvard Business School Working Paper, No. 20-081, February 2020.  View Details

  CiteView Details Read Now

  Related

• Book | 2020

  Experimentation Works: The Surprising Power of Business Experiments

  Stefan H. Thomke

  Don’t fly blind. See how the power of experiments works for you. When it comes to improving customer experiences, trying out new business models, or developing new products, even the most experienced managers often get it wrong. They discover that intuition, experience and big data alone don’t work. What does work? Running disciplined business experiments. And what if companies roll out new products, customer experiences or business models without doing so? They fly blind.
  
  That’s what Harvard Business School professor Stefan Thomke shows in this rigorously researched and eye-opening book. It guides you through what constitutes best experimentation practice, illustrates how that works at leading companies and answers some fundamental questions. What makes a good experiment? How do you test in online and brick-and-mortar businesses? In B2B and B2C? How do you build an experimentation culture? Moreover, best experimentation practice means running many experiments. Indeed, some hugely successful companies, such as Amazon, Booking.com and Microsoft, run tens of thousands of controlled experiments annually, engaging millions of users. Thomke shows us how these and many other organizations have discovered that their experimentation capabilities provide considerable competitive advantages.
  
  How can managers create such a capability at their companies? Essential is developing an experimentation organization that prizes the science of testing and puts the discipline of experimentation at the center of its innovation process. It used to take companies years to develop the tools for such large-scale experimentation but they are now at the fingertips of most organizations. By combining the power of software and the rigor of controlled experiments, today's managers can make better decisions, create magical customer experiences, and generate big financial returns.
  
  Experimentation Works introduces you to the power of experiments and will guide you through its implementation in your own organizations.

  Keywords: experimentation; experiments; Market research; Innovation and Invention; Innovation and Management; Customers; Research;

  Citation:

  Thomke, Stefan H. Experimentation Works: The Surprising Power of Business Experiments. Boston, MA: Harvard Business Review Press, 2020.  View Details

  CiteView DetailsFind at HarvardPurchase

  Related