Iavor I. Bojinov

Causal inference is the study of how actions, interventions, or treatments affect outcomes of interest. The methods that have received the lion’s share of attention in the data science literature for establishing causation are variations of randomized experiments. Unfortunately, randomized experiments are not always feasible for a variety of reasons, such as an inability to fully control the treatment assignment, high cost, and potential negative impacts. In such settings, statisticians and econometricians have developed methods for extracting causal estimates from observational (i.e., nonexperimental) data. Data scientists’ adoption of observational study methods for causal inference, however, has been rather slow and concentrated on a few specific applications. In this article, we attempt to catalyze interest in this area by providing case studies of how data scientists used observational studies to deliver valuable insights at LinkedIn. These case studies employ a variety of methods, and we highlight some themes and practical considerations. Drawing on our learnings, we then explain how firms can develop an organizational culture that embraces causal inference by investing in three key components: education, automation, and certification.

My research focuses on overcoming the methodological and operational challenges of developing data science capabilities, what I call data science operations. Today, within leading digital companies, data science is no longer confined to technical teams but is pervasive throughout the entire enterprise. From screening resumes to prioritizing sales leads, algorithms are augmenting or replacing tasks typically done by humans within every business function to achieve unparalleled scale. Traditional organizations are also starting to build data science capabilities and integrate them into their existing operating model to stay competitive. The rapid and widespread permeation of data science, the ability of algorithms to learn causal relationships, and the need for employees and algorithms to work side-by-side have created a host of critical managerial questions that my research in data science operations aims to answer. Data science capabilities can either be deployed externally, providing new products or services (like the Netflix recommendation engine, Google translator, and Uber passenger-driver matching), or internally, augmenting or replacing employee decision-making (like A/B testing, sales prospect rankings, and fraudulent transaction detection). Successful internal data science applications require the correct methodology, technology, processes, and culture to develop the capability. Methodological challenges arise because the statistical theory underpinning modern data science was developed decades ago for applications that are significantly different from the current business use cases. For example, the fundamentals of experimental design were first introduced one hundred years ago for agriculture settings with few experimental units and outcomes; today, companies run hundreds of experiments on millions of connected people, tracking thousands of outcomes. The technology and processes are necessary to transform data scientists from solving specific tasks to developing software platforms that democratize and scale the practice. Finally, developing the right culture to embrace and benefit from data science presents new challenges as it requires humans and algorithms to work together to achieve operational benefits. Data science operations seeks to understand how companies can develop internal data science capabilities by integrating these four components. My research initially focused on how managers can leverage causal inference to help augment innovation and evaluation—a central topic within data science operations. Causal inference methods are categorized as either experimental or non-experimental (referred to as observational studies). Experimentation is now a core capability for most digital firms and conventional companies undergoing a digital transformation. However, the surge in usage has created numerous methodological pitfalls and operational challenges that my research aims to overcome. On the other hand, the adoption of observational studies for decision-making is a more recent trend that was, in part, catalyzed by my article (The Importance of Being Causal) describing how LinkedIn created an observational study software platform. More recently, I've broadened my scope to study the four areas of data science operations: methodology, technology, processes, and culture