Skip to Main Content
HBS Home
  • About
  • Academic Programs
  • Alumni
  • Faculty & Research
  • Baker Library
  • Giving
  • Harvard Business Review
  • Initiatives
  • News
  • Recruit
  • Map / Directions
Faculty & Research
  • Faculty
  • Research
  • Featured Topics
  • Academic Units
  • …→
  • Harvard Business School→
  • Faculty & Research→
Publications
Publications
  • July 2023
  • Article
  • Management Science

Design and Analysis of Switchback Experiments

By: Iavor I Bojinov, David Simchi-Levi and Jinglong Zhao
  • Format:Print
  • | Pages:19
ShareBar

Abstract

In switchback experiments, a firm sequentially exposes an experimental unit to a random treatment, measures its response, and repeats the procedure for several periods to determine which treatment leads to the best outcome. Although practitioners have widely adopted this experimental design technique, the development of its theoretical properties and the derivation of optimal design procedures have been, to the best of our knowledge, elusive. In this paper, we address these limitations by establishing the necessary results to ensure that practitioners can apply this powerful class of experiments with minimal assumptions. Our main result is the derivation of the optimal design of switchback experiments under a range of different assumptions on the order of carryover effect—that is, the length of time a treatment persists in impacting the outcome. We cast the experimental design problem as a minimax discrete robust optimization problem, identify the worst-case adversarial strategy, establish structural results for the optimal design, and finally solve the problem via a continuous relaxation. For the optimal design, we derive two approaches for performing inference after running the experiment. The first provides exact randomization based p-values and the second uses a finite population central limit theorem to conduct conservative hypothesis tests and build confidence intervals. We further provide theoretical results for our inferential procedures when the order of the carryover effect is misspecified. For firms that possess the capability to run multiple switchback experiments, we also provide a data-driven strategy to identify the likely order of carryover effect. To study the empirical properties of our results, we conduct extensive simulations. We conclude the paper by providing some practical suggestions.

Keywords

Switchback Experiments; Design; Analysis; Mathematical Methods

Citation

Bojinov, Iavor I., David Simchi-Levi, and Jinglong Zhao. "Design and Analysis of Switchback Experiments." Management Science 69, no. 7 (July 2023): 3759–3777.
  • Find it at Harvard
  • Read Now
  • Purchase

About The Author

Iavor I. Bojinov

Technology and Operations Management
→More Publications

More from the Authors

    • April 2025 (Revised April 2025)
    • Faculty Research

    JPMorganChase: Leadership in the Age of GenAI

    By: Iavor I. Bojinov, Karim R. Lakhani and David Lane
    • March 2025
    • Faculty Research

    Building an AI Factory at Procter & Gamble

    By: Iavor I Bojinov, Karim R. Lakhani and Alexis Lefort
    • February 2025
    • Faculty Research

    Building a Generative AI-Powered MBA Course Scheduler

    By: Iavor Bojinov and Annika Hildebrandt
More from the Authors
  • JPMorganChase: Leadership in the Age of GenAI By: Iavor I. Bojinov, Karim R. Lakhani and David Lane
  • Building an AI Factory at Procter & Gamble By: Iavor I Bojinov, Karim R. Lakhani and Alexis Lefort
  • Building a Generative AI-Powered MBA Course Scheduler By: Iavor Bojinov and Annika Hildebrandt
ǁ
Campus Map
Harvard Business School
Soldiers Field
Boston, MA 02163
→Map & Directions
→More Contact Information
  • Make a Gift
  • Site Map
  • Jobs
  • Harvard University
  • Trademarks
  • Policies
  • Accessibility
  • Digital Accessibility
Copyright © President & Fellows of Harvard College.