Iavor Bojinov is an assistant professor of business administration in the Technology and Operations Management unit at Harvard Business School. He teaches the first-year Technology and Operations Management course.
Professor Bojinov’s research interest is at the interface of causal inference, experimental design, and large-scale computing with the overall goal of democratizing statistical methods in order to help firms innovate and grow. He is interested in understanding the value brought by experimentation and data-driven decisions in achieving a competitive advantage. His work has been featured in top Statistics journals such as Biometrika and the Journal of the American Statistical Association.
Prior to joining Harvard Business School, Professor Bojinov worked as a data scientist leading the causal inference effort within the Applied Research Group at LinkedIn. He holds a Ph.D. and an MA in Statistics from Harvard and a MSci in Mathematics from King’s College London.
Online experiments measuring whether “A,” usually the current approach, is inferior to “B,” a proposed improvement, have become integral to the product-development cycle, especially at digital enterprises. But often firms make serious mistakes in conducting these tests: They focus on the average, instead of looking at how a change impacts different customer segments. They forget that customers are connected and that their interactions can affect test outcomes. And they run tests for too short a period, failing to recognize that customers’ reactions can change over time. This article describes how to avoid all those traps by applying techniques that LinkedIn and Netflix have used to produce better insights.
Florian M. Hollenbach, Iavor I. Bojinov, Shahryar Minhas, Nils W. Metternich, Michael D. Ward and Alexander Volfovsky
Missing observations are pervasive throughout empirical research, especially in the social sciences. Despite multiple approaches to dealing adequately with missing data, many scholars still fail to address this vital issue. In this paper, we present a simple-to-use method for generating multiple imputations using a Gaussian copula. The Gaussian copula for multiple imputation (Hoff, 2007) allows scholars to attain estimation results that have good coverage and small bias. The use of copulas to model the dependence among variables will enable researchers to construct valid joint distributions of the data, even without knowledge of the actual underlying marginal distributions. Multiple imputations are then generated by drawing observations from the resulting posterior joint distribution and replacing the missing values. Using simulated and observational data from published social science research, we compare imputation via Gaussian copulas with two other widely used imputation methods: MICE and Amelia II. Our results suggest that the Gaussian copula approach has a slightly smaller bias, higher coverage rates, and narrower confidence intervals compared to the other methods. This is especially true when the variables with missing data are not normally distributed. These results, combined with theoretical guarantees and ease-of-use suggest that the approach examined provides an attractive alternative for applied researchers undertaking multiple imputations.
Hollenbach, Florian M., Iavor I. Bojinov, Shahryar Minhas, Nils W. Metternich, Michael D. Ward, and Alexander Volfovsky. "Multiple Imputation Using Gaussian Copulas."Sociological Methods & Research (forthcoming). (Pre-published online Oct. 18, 2018.)
View Details
We define causal estimands for experiments on single time series, extending the potential outcome framework to dealing with temporal data. Our approach allows the estimation of a broad class of these estimands and exact randomization based p-values for testing causal effects, without imposing stringent assumptions. We further derive a general central limit theorem that can be used to conduct conservative tests and build confidence intervals for causal effects. Finally, we provide three methods for generalizing our approach to multiple units that are receiving the same class of treatment, over time. We test our methodology on simulated “potential autoregressions," which have a causal interpretation. Our methodology is partially inspired by data from a large number of experiments carried out by a financial company who compared the impact of two different ways of trading equity futures contracts. We use our methodology to make causal statements about their trading methods.
Models for analyzing multivariate data sets with missing values require strong, often assessable, assumptions. The most common of these is that the mechanism that created the missing data is ignorable—a twofold assumption dependent on the mode of inference. The first part, which is the focus here, under the Bayesian and direct-likelihood paradigms, requires that the missing data are missing at random; in contrast, the frequentist-likelihood paradigm demands that the missing data mechanism always produces missing at random data, a condition known as missing always at random. Under certain regularity conditions, assuming missing always at random leads to an assumption that can be tested using the observed data alone, namely, the missing data indicators only depend on fully observed variables. Here, we propose three different diagnostic tests that not only indicate when this assumption is incorrect but also suggest which variables are the most likely culprits. Although missing always at random is not a necessary condition to ensure validity under the Bayesian and direct-likelihood paradigms, it is sufficient, and evidence for its violation should encourage the careful statistician to conduct targeted sensitivity analyses.
In panel experiments, we randomly expose multiple units to different treatments and measure their subsequent outcomes, sequentially repeating the procedure numerous times. Using the potential outcomes framework, we define finite population dynamic causal effects that capture the relative effectiveness of alternative treatment paths. For the leading example, known as the lag-p dynamic causal effects, we provide a nonparametric estimator that is unbiased over the randomization distribution. We then derive the finite population limiting distribution of our estimators as either the sample size or the duration of the experiment increases. Our approach provides a new technique for deriving finite population central limit theorems that exploits the underlying Martingale property of unbiased estimators. We further describe two methods for conducting inference on dynamic causal effects: a conservative test for weak null hypotheses of zero average causal effects using the limiting distribution and an exact randomization-based test for sharp null hypotheses. We also derive the finite population limiting distribution of commonly used linear fixed effects estimators showing that these estimators perform poorly in the presence of dynamic causal effects. We conclude with a simulation study and an empirical application in which we reanalyze a lab experiment on cooperation.
What does it mean to say that a quantity is identifiable from the data? Statisticians seem to agree
on a definition in the context of parametric statistical models — roughly, a parameter θ in a model
P = {Pθ : θ ∈ Θ} is identifiable if the mapping θ 7→ Pθ is injective. This definition raises important
questions: Are parameters the only quantities that can be identified? Is the concept of identification
meaningful outside of parametric statistics? Does it even require the notion of a statistical model? Partial and idiosyncratic answers to these questions have been discussed in econometrics, biological modeling, and in some subfields of statistics like causal inference. This paper proposes a unifying theory of identification that incorporates existing definitions for parametric and nonparametric models and formalizes the process of identification analysis. The applicability of this framework is illustrated through a series of examples and two extended case studies.
