Estimating Causal Effects in the Presence of Partial Interference Using Multivariate Bayesian Structural Time Series Models

Fiammetta  Menchetti; Iavor  Bojinov

Working Paper | HBS Working Paper Series | 2020

Estimating Causal Effects in the Presence of Partial Interference Using Multivariate Bayesian Structural Time Series Models

by Fiammetta Menchetti and Iavor Bojinov

Abstract

Researchers regularly use synthetic control methods for estimating causal effects when a sub-set of units receive a single persistent treatment, and the rest are unaffected by the change. In many applications, however, units not assigned to treatment are nevertheless impacted by the intervention because of cross-unit interactions. This paper extends the synthetic control methods to accommodate partial interference, allowing interactions within predefined groups, but not between them. Focusing on a class of causal estimands that capture the effect both on the treated and control units, we develop a multivariate Bayesian structural time series model for generating synthetic controls that would have occurred in the absence of an intervention enabling us to estimate our novel effects. In a simulation study, we explore our Bayesian procedure’s empirical properties and show that it achieves good frequentists coverage even when the model is misspecified. Our work is motivated by an analysis of a marketing campaign’s effectiveness by an Italian supermarket chain that permanently reduced the price of hundreds of store-brand products. We use our new methodology to make causal statements about the impact on sales of the affected store-brands and their direct competitors. Our proposed approach is implemented in the CausalMBSTS R package.

Language: English

Format: Print

57 pages

SSRN Read Now

Citation:

Menchetti, Fiammetta, and Iavor Bojinov. "Estimating Causal Effects in the Presence of Partial Interference Using Multivariate Bayesian Structural Time Series Models." Harvard Business School Working Paper, No. 21-048, October 2020.

Working Paper | HBS Working Paper Series | 2020

Design and Analysis of Switchback Experiments

Iavor I Bojinov, David Simchi-Levi and Jinglong Zhao

In switchback experiments, a firm sequentially exposes an experimental unit to a random treatment, measures its response, and repeats the procedure for several periods to determine which treatment leads to the best outcome. Although practitioners have widely adopted this experimental design technique, the development of its theoretical properties and the derivation of optimal design procedures have been, to the best of our knowledge, elusive. In this paper, we address these limitations by establishing the necessary results to ensure that practitioners can apply this powerful class of experiments with minimal assumptions. Our main result is the derivation of the optimal design of switchback experiments under a range of different assumptions on the order of carryover effect—that is, the length of time a treatment persists in impacting the outcome. We cast the experimental design problem as a minimax discrete robust optimization problem, identify the worst-case adversarial strategy, establish structural results for the optimal design, and finally solve the problem via a continuous relaxation. For the optimal design, we derive two approaches for performing inference after running the experiment. The first provides exact randomization based p-values and the second uses a finite population central limit theorem to conduct conservative hypothesis tests and build confidence intervals. We further provide theoretical results for our inferential procedures when the order of the carryover effect is misspecified. For firms that possess the capability to run multiple switchback experiments, we also provide a data-driven strategy to identify the likely order of carryover effect. To study the empirical properties of our results, we conduct extensive simulations. We conclude the paper by providing some practical suggestions.

Keywords: switchback experiments; Design; Analysis; Mathematical Methods;

Citation:

Bojinov, Iavor I., David Simchi-Levi, and Jinglong Zhao. "Design and Analysis of Switchback Experiments." Harvard Business School Working Paper, No. 21-034, September 2020. View Details

Cite View Details Read Now

Causal inference is the study of how actions, interventions, or treatments affect outcomes of interest. The methods that have received the lion’s share of attention in the data science literature for establishing causation are variations of randomized experiments. Unfortunately, randomized experiments are not always feasible for a variety of reasons, such as an inability to fully control the treatment assignment, high cost, and potential negative impacts. In such settings, statisticians and econometricians have developed methods for extracting causal estimates from observational (i.e., nonexperimental) data. Data scientists’ adoption of observational study methods for causal inference, however, has been rather slow and concentrated on a few specific applications. In this article, we attempt to catalyze interest in this area by providing case studies of how data scientists used observational studies to deliver valuable insights at LinkedIn. These case studies employ a variety of methods, and we highlight some themes and practical considerations. Drawing on our learnings, we then explain how firms can develop an organizational culture that embraces causal inference by investing in three key components: education, automation, and certification.

Faculty & Research

Estimating Causal Effects in the Presence of Partial Interference Using Multivariate Bayesian Structural Time Series Models

Abstract

Citation:

About the Author

More from the Author