Skip to Main Content
HBS Home
  • About
  • Academic Programs
  • Alumni
  • Faculty & Research
  • Baker Library
  • Giving
  • Harvard Business Review
  • Initiatives
  • News
  • Recruit
  • Map / Directions
Faculty & Research
  • Faculty
  • Research
  • Featured Topics
  • Academic Units
  • …→
  • Harvard Business School→
  • Faculty & Research→
  • Research
    • Research
    • Publications
    • Global Research Centers
    • Case Development
    • Initiatives & Projects
    • Research Services
    • Seminars & Conferences
    →
  • Publications→

Publications

Publications

Filter Results : (21) Arrow Down
Filter Results : (21) Arrow Down Arrow Up

Show Results For

  • All HBS Web  (67)
    • Faculty Publications  (21)

    Show Results For

    • All HBS Web  (67)
      • Faculty Publications  (21)

      A/B Testing Remove A/B Testing →

      Page 1 of 21 Results →

      Are you looking for?

      → Search All HBS Web
      • September 2022
      • Article

      Experimentation and Startup Performance: Evidence from A/B Testing

      By: Rembrand Koning, Sharique Hasan and Aaron Chatterji
      Recent scholarship has argued that experimentation should be the organizing principle for entrepreneurial strategy. Experimentation leads to organizational learning, which drives improvements in firm performance. We investigate this proposition by exploiting the...  View Details
      Keywords: Experimentation; A/B Testing; Data-driven Decision-making; Organizational Learning; Entrepreneurship; Strategy; Business Startups; Learning; Performance; Decision Making
      Citation
      Find at Harvard
      Read Now
      Related
      Koning, Rembrand, Sharique Hasan, and Aaron Chatterji. "Experimentation and Startup Performance: Evidence from A/B Testing." Management Science 68, no. 9 (September 2022): 6434–6453.
      • Article

      Online Experimentation: Benefits, Operational and Methodological Challenges, and Scaling Guide

      By: Iavor Bojinov and Somit Gupta
      In the past decade, online controlled experimentation, or A/B testing, at scale has proved to be a significant driver of business innovation. The practice was first pioneered by the technology sector and, more recently, has been adopted by traditional companies...  View Details
      Keywords: A/B Testing; Experimentation; Data-driven Culture; Product Development; Innovation and Invention; Digital Transformation
      Citation
      Read Now
      Related
      Bojinov, Iavor, and Somit Gupta. "Online Experimentation: Benefits, Operational and Methodological Challenges, and Scaling Guide." Harvard Data Science Review, no. 4.3 (Summer, 2022).
      • August 2021
      • Case

      Orchadio’s First Two Split Experiments

      By: Iavor I. Bojinov, Marco Iansiti and David Lane
      Orchadio, a direct-to-consumer grocery business, needs to conduct its first two A/B tests—one to evaluate the effectiveness and functioning of its newly redesigned website, and one to market-test four versions of a new banner for the website. To do so, it will rely on...  View Details
      Keywords: Information Management; Technological Innovation; Knowledge Use and Leverage; Resource Allocation; Marketing; Measurement and Metrics; Customization and Personalization; Information Technology; Internet and the Web; Digital Platforms; Information Technology Industry; Food and Beverage Industry
      Citation
      Educators
      Purchase
      Related
      Bojinov, Iavor I., Marco Iansiti, and David Lane. "Orchadio’s First Two Split Experiments." Harvard Business School Case 622-015, August 2021.
      • June 23, 2021
      • Article

      Research: When A/B Testing Doesn't Tell You the Whole Story

      By: Eva Ascarza
      When it comes to churn prevention, marketers traditionally start by identifying which customers are most likely to churn, and then running A/B tests to determine whether a proposed retention intervention will be effective at retaining those high-risk customers. While...  View Details
      Keywords: Customer Retention; Churn; Targeting; Market Research; Marketing; Investment Return; Customers; Retention; Research
      Citation
      Find at Harvard
      Register to Read
      Related
      Ascarza, Eva. "Research: When A/B Testing Doesn't Tell You the Whole Story." Harvard Business Review Digital Articles (June 23, 2021).
      • March 2021
      • Supplement

      Artea (A), (B), (C), and (D): Designing Targeting Strategies

      By: Eva Ascarza and Ayelet Israeli
      Power Point Supplement to Teaching Note for HBS No. 521-021,521-022,521-037,521-043. This collection of exercises aims to teach students about 1)Targeting Policies; and 2)Algorithmic bias in marketing—implications, causes, and possible solutions. Part (A) focuses on...  View Details
      Keywords: Targeted Advertising; Targeting; Algorithmic Data; Bias; A/B Testing; Experiment; Advertising; Gender; Race; Diversity; Marketing; Customer Relationship Management; Prejudice and Bias; Analytics and Data Science; Retail Industry; Apparel and Accessories Industry; Technology Industry; United States
      Citation
      Purchase
      Related
      Ascarza, Eva, and Ayelet Israeli. "Artea (A), (B), (C), and (D): Designing Targeting Strategies." Harvard Business School PowerPoint Supplement 521-719, March 2021.
      • February 2021
      • Tutorial

      T-tests: Theory and Practice

      By: Michael Parzen, Natalie Epstein, Chiara Farronato and Michael Toffel
      This video provides an introduction to hypothesis testing, sampling, t-tests, and p-values. It provides examples of A/B testing and t-testing to assess whether difference between two groups are statistically significant. This video can be assigned in conjunction with...  View Details
      Keywords: Data Analysis; Data Analytics; Experiment Design; Experimentation; Analytics and Data Science; Analysis
      Citation
      Purchase
      Related
      Parzen, Michael, Natalie Epstein, Chiara Farronato, and Michael Toffel. T-tests: Theory and Practice. Harvard Business School Tutorial 621-707, February 2021.
      • September 2020 (Revised July 2022)
      • Teaching Note

      Artea (A), (B), (C), and (D): Designing Targeting Strategies

      By: Eva Ascarza and Ayelet Israeli
      Teaching Note for HBS No. 521-021,521-022,521-037,521-043. This collection of exercises aims to teach students about 1)Targeting Policies; and 2)Algorithmic bias in marketing—implications, causes, and possible solutions. Part (A) focuses on A/B testing analysis and...  View Details
      Keywords: Targeted Advertising; Targeting; Race; Gender; Diversity; Marketing; Customer Relationship Management; Prejudice and Bias; Analytics and Data Science; Retail Industry; Apparel and Accessories Industry; Technology Industry; United States
      Citation
      Purchase
      Related
      Ascarza, Eva, and Ayelet Israeli. "Artea (A), (B), (C), and (D): Designing Targeting Strategies." Harvard Business School Teaching Note 521-041, September 2020. (Revised July 2022.)
      • September 2020 (Revised July 2022)
      • Exercise

      Artea (B): Including Customer-level Demographic Data

      By: Eva Ascarza and Ayelet Israeli
      This collection of exercises aims to teach students about 1)Targeting Policies; and 2)Algorithmic bias in marketing—implications, causes, and possible solutions. Part (A) focuses on A/B testing analysis and targeting. Parts (B),(C),(D) Introduce algorithmic bias. The...  View Details
      Keywords: Targeting; Algorithmic Bias; Race; Gender; Marketing; Diversity; Customer Relationship Management; Demographics; Prejudice and Bias; Retail Industry; Apparel and Accessories Industry; Technology Industry; United States
      Citation
      Purchase
      Related
      Ascarza, Eva, and Ayelet Israeli. "Artea (B): Including Customer-level Demographic Data." Harvard Business School Exercise 521-022, September 2020. (Revised July 2022.)
      • September 2020 (Revised July 2022)
      • Exercise

      Artea (C): Potential Discrimination through Algorithmic Targeting

      By: Eva Ascarza and Ayelet Israeli
      This collection of exercises aims to teach students about 1)Targeting Policies; and 2)Algorithmic bias in marketing—implications, causes, and possible solutions. Part (A) focuses on A/B testing analysis and targeting. Parts (B),(C),(D) Introduce algorithmic bias. The...  View Details
      Keywords: Targeting; Algorithmic Bias; Race; Gender; Marketing; Diversity; Customer Relationship Management; Prejudice and Bias; Retail Industry; Apparel and Accessories Industry; Technology Industry; United States
      Citation
      Purchase
      Related
      Ascarza, Eva, and Ayelet Israeli. "Artea (C): Potential Discrimination through Algorithmic Targeting." Harvard Business School Exercise 521-037, September 2020. (Revised July 2022.)
      • September 2020 (Revised July 2022)
      • Exercise

      Artea (D): Discrimination through Algorithmic Bias in Targeting

      By: Eva Ascarza and Ayelet Israeli
      This collection of exercises aims to teach students about 1)Targeting Policies; and 2)Algorithmic bias in marketing—implications, causes, and possible solutions. Part (A) focuses on A/B testing analysis and targeting. Parts (B),(C),(D) Introduce algorithmic bias. The...  View Details
      Keywords: Targeted Advertising; Discrimination; Algorithmic Data; Bias; Advertising; Race; Gender; Marketing; Diversity; Customer Relationship Management; Prejudice and Bias; Analytics and Data Science; Retail Industry; Apparel and Accessories Industry; Technology Industry; United States
      Citation
      Purchase
      Related
      Ascarza, Eva, and Ayelet Israeli. "Artea (D): Discrimination through Algorithmic Bias in Targeting." Harvard Business School Exercise 521-043, September 2020. (Revised July 2022.)
      • September 2020 (Revised April 2021)
      • Exercise

      Artea: Designing Targeting Strategies

      By: Eva Ascarza and Ayelet Israeli
      This collection of exercises aims to teach students about 1)Targeting Policies; and 2)Algorithmic bias in marketing—implications, causes, and possible solutions. Part (A) focuses on A/B testing analysis and targeting. Parts (B),(C),(D) Introduce algorithmic bias. The...  View Details
      Keywords: Algorithmic Data; Race And Ethnicity; Experimentation; Promotion; "Marketing Analytics"; Marketing And Society; Big Data; Privacy; Data-driven Management; Data Analytics; Data Analysis; E-Commerce Strategy; Discrimination; Targeted Advertising; Targeted Policies; Targeting; Pricing Algorithms; A/B Testing; Ethical Decision Making; Customer Base Analysis; Customer Heterogeneity; Coupons; Marketing; Race; Gender; Diversity; Customer Relationship Management; Marketing Communications; Advertising; Decision Making; Ethics; E-commerce; Analytics and Data Science; Retail Industry; Apparel and Accessories Industry; United States
      Citation
      Purchase
      Related
      Ascarza, Eva, and Ayelet Israeli. "Artea: Designing Targeting Strategies." Harvard Business School Exercise 521-021, September 2020. (Revised April 2021.)
      • September 2020 (Revised July 2022)
      • Supplement

      Spreadsheet Supplement to Artea Teaching Note

      By: Eva Ascarza and Ayelet Israeli
      Spreadsheet Supplement to Artea Teaching Note 521-041. This collection of exercises aims to teach students about 1)Targeting Policies; and 2)Algorithmic bias in marketing—implications, causes, and possible solutions. Part (A) focuses on A/B testing analysis and...  View Details
      Keywords: Targeted Advertising; Algorithmic Data; Bias; Advertising; Race; Gender; Diversity; Marketing; Customer Relationship Management; Prejudice and Bias; Analytics and Data Science; Retail Industry; Apparel and Accessories Industry; Technology Industry; United States
      Citation
      Purchase
      Related
      Ascarza, Eva, and Ayelet Israeli. "Spreadsheet Supplement to Artea Teaching Note." Harvard Business School Spreadsheet Supplement 521-705, September 2020. (Revised July 2022.)
      • March–April 2020
      • Article

      Avoid the Pitfalls of A/B Testing

      By: Iavor I. Bojinov, Guillaume Sait-Jacques and Martin Tingley
      Online experiments measuring whether “A,” usually the current approach, is inferior to “B,” a proposed improvement, have become integral to the product-development cycle, especially at digital enterprises. But often firms make serious mistakes in conducting these...  View Details
      Keywords: A/B Testing; Experiment Design; Social Networks; Product Development; Performance Improvement; Measurement and Metrics; Social Media
      Citation
      Find at Harvard
      Register to Read
      Related
      Bojinov, Iavor I., Guillaume Sait-Jacques, and Martin Tingley. "Avoid the Pitfalls of A/B Testing." Harvard Business Review 98, no. 2 (March–April 2020): 48–53.
      • 2020
      • Working Paper

      The Effects of Hierarchy on Learning and Performance in Business Experimentation

      By: Sourobh Ghosh, Stefan Thomke and Hazjier Pourkhalkhali
      Do senior managers help or hurt business experiments? Despite the widespread adoption of business experiments to guide strategic decision-making, we lack a scholarly understanding of what role senior managers play in firm experimentation. Using proprietary data of live...  View Details
      Keywords: Experimentation; Innovation; Search; New Product Development; Innovation and Invention; Organizational Design; Learning; Performance
      Citation
      Read Now
      Related
      Ghosh, Sourobh, Stefan Thomke, and Hazjier Pourkhalkhali. "The Effects of Hierarchy on Learning and Performance in Business Experimentation." Harvard Business School Working Paper, No. 20-081, February 2020.
      • 2020
      • Working Paper

      Digital Experimentation and Startup Performance: Evidence from A/B Testing

      By: Rembrand Koning, Sharique Hasan and Aaron Chatterji
      Recent work argues that experimentation is the appropriate framework for entrepreneurial strategy. We investigate this proposition by exploiting the time-varying adoption of A/B testing technology, which has drastically reduced the cost of experimentally testing...  View Details
      Keywords: Experimentation; A/B Testing; Data-driven Decision-making; Entrepreneurship; Strategy; Business Startups; Information Technology; Performance
      Citation
      SSRN
      Read Now
      Related
      Koning, Rembrand, Sharique Hasan, and Aaron Chatterji. "Digital Experimentation and Startup Performance: Evidence from A/B Testing." Harvard Business School Working Paper, No. 20-018, August 2019. (Revised September 2020. SSRN Working Paper Series, No. 3440291, August 2019)
      • January 2019
      • Teaching Note

      Hubble Contact Lenses: Data Driven Direct-to-Consumer Marketing

      By: Ayelet Israeli
      Teaching Note for HBS No. 519-011. As its Series A extension round approaches, the founders of Hubble, a subscription-based, social-media fueled, direct-to-consumer (DTC) brand of contact lenses, are reflecting on the marketing strategies that have taken them to a...  View Details
      Keywords: DTC; Direct To Consumer Marketing; Health Care; Mobile; Attribution; Experimentation; Experiments; Churn/retention; Customer Lifetime Value; Internet Marketing; Big Data; Analytics; A/B Testing; CRM; Advertising; Marketing; Marketing Channels; Marketing Strategy; Media; Brands and Branding; Marketing Communications; Digital Marketing; Acquisition; Growth and Development Strategy; Customer Focus and Relationships; Consumer Behavior; Social Media; E-commerce
      Citation
      Purchase
      Related
      Israeli, Ayelet. "Hubble Contact Lenses: Data Driven Direct-to-Consumer Marketing." Harvard Business School Teaching Note 519-056, January 2019.
      • October 2018
      • Case

      Booking.com

      By: Stefan Thomke and Daniela Beyersdorfer
      The case reveals how Booking.com has become the world's leading travel accommodation platform. The company has put online experimentation at the heart of how it designs digital experiences for its customers and partners. To unlock the potential of large-scale testing,...  View Details
      Keywords: Travel; Product Innovation; Experimentation; A/B Testing; User Experience Design; Product Development; Product Design; Innovation and Management; Transformation; Information Technology; Digital Transformation; Travel Industry
      Citation
      Educators
      Purchase
      Related
      Thomke, Stefan, and Daniela Beyersdorfer. "Booking.com." Harvard Business School Case 619-015, October 2018.
      • August 2018 (Revised December 2020)
      • Case

      Hubble Contact Lenses: Data Driven Direct-to-Consumer Marketing

      By: Jill Avery and Ayelet Israeli
      As its Series A extension round approaches, the founders of Hubble, a subscription-based, social-media fueled, direct-to-consumer (DTC) brand of contact lenses, are reflecting on the marketing strategies that have taken them to a valuation of $200 million and debating...  View Details
      Keywords: DTC; Direct To Consumer Marketing; Health Care; Mobile; Attribution; Experimentation; Experiments; Churn/retention; Customer Lifetime Value; Internet Marketing; Big Data; Analytics; A/B Testing; CRM; Advertising; Marketing; Marketing Channels; Marketing Strategy; Media; Brands and Branding; Marketing Communications; Digital Marketing; Consumer Behavior; Acquisition; Growth and Development Strategy; Customer Focus and Relationships; Social Media; E-commerce; Analytics and Data Science; Health Industry; Consumer Products Industry; United States; North America; Europe
      Citation
      Educators
      Purchase
      Related
      Avery, Jill, and Ayelet Israeli. "Hubble Contact Lenses: Data Driven Direct-to-Consumer Marketing." Harvard Business School Case 519-011, August 2018. (Revised December 2020.)
      • September–October 2017
      • Article

      The Surprising Power of Online Experiments: Getting the Most Out of A/B and Other Controlled Tests

      By: Ron Kohavi and Stefan Thomke
      In the fast-moving digital world, even experts have a hard time assessing new ideas. Case in point: At Bing, a small headline change an employee proposed was deemed a low priority and shelved for months until one engineer decided to do a quick online controlled...  View Details
      Keywords: Experiments; A/B Testing; Research; Consumer Behavior
      Citation
      Find at Harvard
      Register to Read
      Related
      Kohavi, Ron, and Stefan Thomke. "The Surprising Power of Online Experiments: Getting the Most Out of A/B and Other Controlled Tests." Harvard Business Review 95, no. 5 (September–October 2017): 74–82.
      • September 2015 (Revised March 2017)
      • Technical Note

      FIELD Global Immersion: Developing Customer Empathy

      By: Jill Avery
      The Design Thinking process begins with empathizing with potential customers. Empathizing, being aware of, interpreting, and understanding the thoughts of others, as well as being able to vicariously experience them oneself, requires the careful and deliberate study of...  View Details
      Keywords: Market Research; Design Thinking; Customer Behavior; Ethnography; Interviews; Surveys; A/B Testing; Experimentation; Marketing; Customer Focus and Relationships; Consumer Behavior; Demand and Consumers
      Citation
      Educators
      Related
      Avery, Jill. "FIELD Global Immersion: Developing Customer Empathy." Harvard Business School Technical Note 316-082, September 2015. (Revised March 2017.)
      • 1
      • 2
      • →

      Are you looking for?

      → Search All HBS Web
      ǁ
      Campus Map
      Harvard Business School
      Soldiers Field
      Boston, MA 02163
      →Map & Directions
      →More Contact Information
      • Make a Gift
      • Site Map
      • Jobs
      • Harvard University
      • Trademarks
      • Policies
      • Accessibility
      • Digital Accessibility
      Copyright © President & Fellows of Harvard College