Does the adoption of descriptive analytics impact online retailer performance, and if so, how? We use the synthetic difference-in-differences method to analyze the staggered adoption of a retail analytics dashboard by more than 1,500 e-commerce websites, and we find an increase of 4%–10% in average weekly revenues postadoption. We demonstrate that only retailers that adopt and use the dashboard reap these benefits.
The increase in revenue is not explained by price changes or advertising optimization. Instead, it is consistent with the addition of customer relationship management, personalization, and prospecting technologies to retailer websites. The adoption and usage of descriptive analytics also increases the diversity of products sold, the number of transactions, the numbers of website visitors and unique customers, and the revenue from repeat customers. In contrast, there is no change in basket size. These findings are consistent with a complementary effect of descriptive analytics that serve as a monitoring device that helps retailers control additional martech tools and amplify their value. Without using the descriptive dashboard, retailers are unable to reap the benefits associated with these technologies.
An inherent risk of algorithmic personalization is disproportionate targeting of individuals from certain groups (or demographic characteristics such as gender or race), even when the decision maker does not intend to discriminate based on those “protected” attributes. This unintended discrimination is often caused by underlying correlations in the data between protected attributes and other observed characteristics used by the algorithm (or machine learning (ML) tool) to create predictions and target individuals optimally. Because these correlations are hidden in high dimensional data, removing protected attributes from the database does not solve the discrimination problem; instead, removing those attributes often exacerbates the problem by making it undetectable and, in some cases, even increases the bias generated by the algorithm.
We propose BEAT (Bias-Eliminating Adapted Trees) to address these issues. This approach allows decision makers to target individuals based on differences in their predicted behavior—hence capturing value from personalization—while ensuring a balanced allocation of resources across individuals, guaranteeing both group and individual fairness. Essentially, the method only extracts heterogeneity in the data that is unrelated to protected attributes. To do so, we build on the General Random Forest (GRF) framework (S. Athey et al., Ann. Stat. 47, 1148–1178 (2019)) and develop a targeting allocation that is “balanced” with respect to protected attributes. We validate BEAT using simulations and an online experiment with N=3,146 participants. This approach can be applied to any type of allocation decision that is based on prediction algorithms, such as medical treatments, hiring decisions, product recommendations, or dynamic pricing.
Ayelet Israeli is
the Marvin Bower Associate Professor of Business Administration at the Harvard Business School Marketing Unit. She teaches the
Rethinking Retail course in the MBA elective curriculum (EC), the Data Driven Marketing course in the Harvard Business Analytics Program (HBAP), and in various Executive Education programs. She has previously taught the Marketing course in the first year MBA required curriculum (RC), and the E-Commerce: Strategy, Growth, and Analytics course in the EC. In her research, Ayelet studies omni-channel and e-commerce markets. In particular, she focuses on pricing, channel management, online marketing, and marketing analytics.
Her research has been published in leading marketing journals including Marketing Science, Journal of Marketing Research, and
Management Science. Her dissertation won the 2014 INFORMS Society for Marketing Science Doctoral Dissertation Proposal Award,
and she was named a finalist for the 2018 and the 2019 Frank M. Bass Award. Her work has been cited by
The Wall Street Journal,
The Atlantic,
MSN Money, and
Harvard Business Review.
Ayelet received her PhD in marketing from the Kellogg School of Management at Northwestern University. She holds an MBA from the Hebrew University of Jerusalem, where she also earned her MSc and BSc in computer science. In addition to her academic experience, Ayelet served as a lieutenant in the Intelligence Corps of the Israeli Defense Forces and worked as an engineer at Israel Aerospace Industries and at Intel Corporation in Israel.
- Featured Work
-
Does the adoption of descriptive analytics impact online retailer performance, and if so, how? We use the synthetic difference-in-differences method to analyze the staggered adoption of a retail analytics dashboard by more than 1,500 e-commerce websites, and we find an increase of 4%–10% in average weekly revenues postadoption. We demonstrate that only retailers that adopt and use the dashboard reap these benefits.
The increase in revenue is not explained by price changes or advertising optimization. Instead, it is consistent with the addition of customer relationship management, personalization, and prospecting technologies to retailer websites. The adoption and usage of descriptive analytics also increases the diversity of products sold, the number of transactions, the numbers of website visitors and unique customers, and the revenue from repeat customers. In contrast, there is no change in basket size. These findings are consistent with a complementary effect of descriptive analytics that serve as a monitoring device that helps retailers control additional martech tools and amplify their value. Without using the descriptive dashboard, retailers are unable to reap the benefits associated with these technologies.
An inherent risk of algorithmic personalization is disproportionate targeting of individuals from certain groups (or demographic characteristics such as gender or race), even when the decision maker does not intend to discriminate based on those “protected” attributes. This unintended discrimination is often caused by underlying correlations in the data between protected attributes and other observed characteristics used by the algorithm (or machine learning (ML) tool) to create predictions and target individuals optimally. Because these correlations are hidden in high dimensional data, removing protected attributes from the database does not solve the discrimination problem; instead, removing those attributes often exacerbates the problem by making it undetectable and, in some cases, even increases the bias generated by the algorithm.
We propose BEAT (Bias-Eliminating Adapted Trees) to address these issues. This approach allows decision makers to target individuals based on differences in their predicted behavior—hence capturing value from personalization—while ensuring a balanced allocation of resources across individuals, guaranteeing both group and individual fairness. Essentially, the method only extracts heterogeneity in the data that is unrelated to protected attributes. To do so, we build on the General Random Forest (GRF) framework (S. Athey et al., Ann. Stat. 47, 1148–1178 (2019)) and develop a targeting allocation that is “balanced” with respect to protected attributes. We validate BEAT using simulations and an online experiment with N=3,146 participants. This approach can be applied to any type of allocation decision that is based on prediction algorithms, such as medical treatments, hiring decisions, product recommendations, or dynamic pricing.
- Journal Articles
-
- Berman, Ron, and Ayelet Israeli. "The Value of Descriptive Analytics: Evidence from Online Retailers." Marketing Science (forthcoming). (Available online.) View Details
- Ascarza, Eva, and Ayelet Israeli. "Eliminating Unintended Bias in Personalized Policies Using Bias-Eliminating Adapted Trees (BEAT)." e2115126119. Proceedings of the National Academy of Sciences 119, no. 11 (March 8, 2022). View Details
- Israeli, Ayelet, Fiona Scott Morton, Jorge Silva-Risso, and Florian Zettelmeyer. "How Market Power Affects Dynamic Pricing: Evidence from Inventory Fluctuations at Car Dealerships." Management Science 68, no. 2 (February 2022): 895–916. View Details
- Israeli, Ayelet, and Eugene F. Zelek Jr. "Pricing Policies that Protect your Brand." Harvard Business Review 98, no. 2 (March–April 2020): 76–83. View Details
- Israeli, Ayelet. "Online MAP Enforcement: Evidence from a Quasi-Experiment." Marketing Science 37, no. 5 (September–October 2018): 710–732. View Details
- Busse, Meghan, Ayelet Israeli, and Florian Zettelmeyer. "Repairing the Damage: The Effect of Price Knowledge and Gender on Auto-Repair Price Quotes." Journal of Marketing Research (JMR) 54, no. 1 (February 2017): 75–95. View Details
- Israeli, Ayelet, Eric Anderson, and Anne Coughlan. "Minimum Advertised Pricing: Patterns of Violation in Competitive Retail Markets." Marketing Science 35, no. 4 (July–August 2016): 539–564. (Lead article.) View Details
- Working Papers
-
- Anderson, Eric, Chaoqun Chen, Ayelet Israeli, and Duncan Simester. "Canary Categories." Working Paper, August 2022. View Details
- Other Publications and Materials
-
- Israeli, Ayelet, Eva Ascarza, and Laura Castrillo. "Beyond Pajamas: Sizing Up the Pandemic Shopper." Harvard Business School Working Knowledge (March 17, 2021). View Details
- Israeli, Ayelet. "REMOTE—A Framework for Teaching Online." Harvard Business Publishing, 2020. View Details
- Israeli, Ayelet. "Encouraging Student Participation Online—and Assessing It Fairly: Techniques and Methods to Involve More Voices in Virtual Classes." Harvard Business Publishing, 2020. Electronic. View Details
- Israeli, Ayelet, Eric Anderson, and Anne Coughlan. "Online Discounting: Who is Leading the Race to the Bottom?" Harvard Business Review 94, no. 3 (March 2016): 24–24. (Idea Watch.) View Details
- Cases and Teaching Materials
-
- Israeli, Ayelet, and Anne V. Wilson. "Athletic Brewing Company: Crafting the U.S. Non-Alcoholic Beer Category." Harvard Business School Case 523-021, July 2022. View Details
- Israeli, Ayelet. "PittaRosso (B): Human and Machine Learning." Harvard Business School Supplement 522-047, November 2021. (Revised December 2021.) View Details
- Israeli, Ayelet, and Fabrizio Fantini. "PittaRosso: Artificial Intelligence-Driven Pricing and Promotion." Harvard Business School Spreadsheet Supplement 522-710, October 2021. (Revised March 2022.) View Details
- Israeli, Ayelet. "PittaRosso: Artificial Intelligence-Driven Pricing and Promotion." Harvard Business School Case 522-046, October 2021. (Revised June 2022.) View Details
- Israeli, Ayelet, Jenyfeer Martinez Buitrago, and Carla Larangeira. "Yummy: Delivering Value to Venezuela." Harvard Business School Case 522-034, August 2021. View Details
- Israeli, Ayelet, and Anne Wilson. "Headspace vs. Calm: A Mindful Competition." Harvard Business School Teaching Plan 521-116, June 2021. View Details
- Israeli, Ayelet, and Anne Wilson. "Headspace vs. Calm: A Mindful Competition." Harvard Business School Case 521-102, May 2021. (Revised May 2022.) View Details
- Israeli, Ayelet. "eGrocery and the Role of Data and E-Commerce Analytics for CPG Firms." Harvard Business School Teaching Note 523-012, July 2022. View Details
- Israeli, Ayelet. "Solution for E-Commerce Analytics for CPG Firms (A): Estimating Sales." Harvard Business School Spreadsheet Supplement 523-704, July 2022. View Details
- Israeli, Ayelet. "Solution for E-Commerce Analytics for CPG Firms (B): Optimizing Assortment for a New Retailer." Harvard Business School Spreadsheet Supplement 523-705, July 2022. View Details
- Israeli, Ayelet. "Solution for E-Commerce Analytics for CPG Firms (C): Free Delivery Terms." Harvard Business School Spreadsheet Supplement 523-706, July 2022. View Details
- Israeli, Ayelet, Fedor (Ted) Lisitsyn, and Mark A. Irwin. "eGrocery and the Role of Data for CPG Firms." Harvard Business School Background Note 521-077, February 2021. (Revised February 2021.) View Details
- Israeli, Ayelet, and Fedor (Ted) Lisitsyn. "E-Commerce Analytics for CPG Firms (C): Free Delivery Terms." Harvard Business School Exercise 521-080, January 2021. (Revised March 2021.) View Details
- Israeli, Ayelet, and Fedor (Ted) Lisitsyn. "E-Commerce Analytics for CPG Firms (B): Optimizing Assortment for a New Retailer." Harvard Business School Exercise 521-079, January 2021. View Details
- Israeli, Ayelet, and Fedor (Ted) Lisitsyn. "E-Commerce Analytics for CPG Firms (A): Estimating Sales." Harvard Business School Exercise 521-078, January 2021. (Revised March 2021.) View Details
- Israeli, Ayelet. "AptDeco: Circular Economy Furniture Marketplace." Harvard Business School Teaching Note 522-078, February 2022. View Details
- Israeli, Ayelet, and Jamie Merkrebs. "AptDeco: Circular Economy Furniture Marketplace." Harvard Business School Case 521-069, February 2021. (Revised March 2021.) View Details
- Israeli, Ayelet, and Jill Avery. "THE YES: Reimagining the Future of E-Commerce with Artificial Intelligence (AI)." Harvard Business School Teaching Note 521-097, May 2021. (Revised February 2022.) View Details
- Avery, Jill, Ayelet Israeli, and Emma von Maur. "THE YES: Reimagining the Future of E-Commerce with Artificial Intelligence (AI)." Harvard Business School Case 521-070, January 2021. (Revised March 2021.) View Details
- Ascarza, Eva, and Ayelet Israeli. "Amazon Shopper Panel: Paying Customers for Their Data." Harvard Business School Teaching Note 522-011, July 2021. (Revised January 2022.) View Details
- Ascarza, Eva, and Ayelet Israeli. "Amazon Shopper Panel: Paying Customers for Their Data." Harvard Business School Case 521-058, January 2021. (Revised May 2021.) View Details
- Israeli, Ayelet, Fares Khrais, and Menna Hassan. "Arçelik (A), (B): From a Dealer Network to an Omnichannel Experience." Harvard Business School Teaching Note 523-009, July 2022. View Details
- Israeli, Ayelet, and Fares Khrais. "Arçelik: COVID-19 Fueled Omnichannel Growth (B)." Harvard Business School Supplement 521-068, February 2021. (Revised March 2021.) View Details
- Israeli, Ayelet, and Fares Khrais. "Arçelik: From a Dealer Network to an Omnichannel Experience." Harvard Business School Case 521-067, January 2021. (Revised March 2022.) View Details
- Ascarza, Eva, and Ayelet Israeli. "Spreadsheet Supplement to Artea Teaching Note." Harvard Business School Spreadsheet Supplement 521-705, September 2020. (Revised July 2022.) View Details
- Ascarza, Eva, and Ayelet Israeli. "Artea (A), (B), (C), and (D): Designing Targeting Strategies." Harvard Business School PowerPoint Supplement 521-719, March 2021. View Details
- Ascarza, Eva, and Ayelet Israeli. "Artea (A), (B), (C), and (D): Designing Targeting Strategies." Harvard Business School Teaching Note 521-041, September 2020. (Revised July 2022.) View Details
- Ascarza, Eva, and Ayelet Israeli. "Artea (D): Discrimination through Algorithmic Bias in Targeting." Harvard Business School Exercise 521-043, September 2020. (Revised July 2022.) View Details
- Ascarza, Eva, and Ayelet Israeli. "Spreadsheet Supplement to Artea (B) and (C)." Harvard Business School Spreadsheet Supplement 521-704, September 2020. (Revised July 2022.) View Details
- Ascarza, Eva, and Ayelet Israeli. "Artea (C): Potential Discrimination through Algorithmic Targeting." Harvard Business School Exercise 521-037, September 2020. (Revised July 2022.) View Details
- Ascarza, Eva, and Ayelet Israeli. "Artea (B): Including Customer-level Demographic Data." Harvard Business School Exercise 521-022, September 2020. (Revised July 2022.) View Details
- Ascarza, Eva, and Ayelet Israeli. Spreadsheet Supplement to "Artea: Designing Targeting Strategies". Harvard Business School Spreadsheet Supplement 521-703, September 2020. (Revised July 2022.) View Details
- Ascarza, Eva, and Ayelet Israeli. "Artea: Designing Targeting Strategies." Harvard Business School Exercise 521-021, September 2020. (Revised April 2021.) View Details
- Israeli, Ayelet, and Eva Ascarza. "Algorithmic Bias in Marketing." Harvard Business School Teaching Note 521-035, September 2020. (Revised July 2022.) View Details
- Israeli, Ayelet, and Eva Ascarza. "Algorithmic Bias in Marketing." Harvard Business School Technical Note 521-020, September 2020. (Revised July 2022.) View Details
- Israeli, Ayelet, and Jill Avery. "Thingtesting: Launching a Brand Discovery and Testing Digital Community." Harvard Business School Teaching Note 521-094, April 2021. View Details
- Israeli, Ayelet, and Jill Avery. "Thingtesting: Launching a Brand Discovery and Testing Digital Community." Harvard Business School Case 520-086, March 2020. View Details
- Avery, Jill, and Ayelet Israeli. "Influencer Marketing." Harvard Business School Technical Note 520-075, March 2020. View Details
- Israeli, Ayelet, Danilo Tauro, and Sarah Gulick. "Sizmek Chapter 11: Surviving Walled Gardens in Their Ad Tech Empire." Harvard Business School Case 520-087, March 2020. View Details
- Israeli, Ayelet, and Carla Larangeira. "Banorte Móvil: Data-Driven Mobile Growth." Harvard Business School Teaching Note 522-095, April 2022. View Details
- Israeli, Ayelet, Carla Larangeira, and Mariana Cal. "Banorte Móvil: Data-Driven Mobile Growth." Harvard Business School Case 520-068, January 2020. View Details
- Israeli, Ayelet. "The DivaCup: Navigating Distribution and Growth." Harvard Business School Teaching Note 523-008, July 2022. View Details
- Israeli, Ayelet. "The DivaCup: Navigating Distribution and Growth." Harvard Business School Case 519-055, March 2019. (Revised April 2021.) View Details
- Israeli, Ayelet. "DayTwo: Going to Market with Gut Microbiome." Harvard Business School Teaching Note 521-052, November 2020. View Details
- Israeli, Ayelet, and David Lane. "DayTwo: Going to Market with Gut Microbiome." Harvard Business School Case 519-010, March 2019. View Details
- Israeli, Ayelet. "Hubble Contact Lenses: Data Driven Direct-to-Consumer Marketing." Harvard Business School Teaching Note 519-056, January 2019. View Details
- Avery, Jill, and Ayelet Israeli. "Hubble Contact Lenses: Data Driven Direct-to-Consumer Marketing." Harvard Business School Case 519-011, August 2018. (Revised December 2020.) View Details
- Israeli, Ayelet, and Jill Avery. "Predicting Consumer Tastes with Big Data at Gap." Harvard Business School Teaching Note 518-053, November 2017. View Details
- Israeli, Ayelet, and Jill Avery. "Predicting Consumer Tastes with Big Data at Gap." Harvard Business School Case 517-115, May 2017. (Revised March 2018.) View Details
- Israeli, Ayelet, and Robert J. Dolan. "Angie's List: Ratings Pioneer Turns 20." Harvard Business School Teaching Note 517-123, May 2017. (Revised January 2019.) View Details
- Dolan, Robert J., and Ayelet Israeli. "Angie's List: Ratings Pioneer Turns 20." Harvard Business School Case 517-016, September 2016. (Revised February 2017.) View Details
- Research Summary
-
Professor Israeli utilizes econometric methods and field experiments to study data driven decision making in marketing context. In her work, she focuses on pricing and channel management in omni-channel and e-commerce markets, and on marketing analytics. She has examined how the prevalence of the online channel affects the interactions between manufacturers and the retailers who are their downstream channel partners. Her findings indicate that a manufacturer is able to improve retailers’ compliance with pricing policies by creating policies that address the challenges of the online retail environment and credibly signal to the retailers that the manufacturer is monitoring their behavior and is prepared to enforce the policies.
Professor Israeli has also studied how the online channel affects the interactions between sellers and consumers in a large-scale field experiment in which callers requested price quotes from automotive repair shops. A key finding is that sellers alter their quotes depending on how informed individual consumers appear to be about market prices. This work demonstrates the benefit to consumers of conducting simple online research in order to appear savvy. The benefits are greater for women, for whom having the correct information alleviates price discrimination in repair shops. In another paper, she found that availability of inventory data online allows consumers to obtain lower prices for new cars.
In another stream of her research, Professor Israeli examines the value of analytics to firms. The main finding is that descriptive analytics are valuable for retailers. Specifically, adopting a descriptive dashboard resulted in increased revenue for online retailers. The underlying mechanism for these benefits was the use of descriptive analytics as a monitoring device that helped retailers monitor other marketing technologies (martech) and amplify their value.
Professor Israeli also investigates unintended consequences of leveraging data. An inherent risk of algorithmic personalization is disproportionate targeting of individuals from certain groups, even when the decision maker does not intend to discriminate based on any “protected” attributes. In this stream of work, she provides a practical solution for firms that want to avoid this bias, but still leverage data for personalization.
- Awards & Honors
-
Received the 2020–2021 Robert F. Greenhill Award.
Received the 2021 Apgar Award for Innovation in Teaching in recognition of outstanding work in the transition to online and hybrid teaching and learning.
Finalist for the 2019 Frank M. Bass Dissertation Paper Award for “Online MAP Enforcement: Evidence from a Quasi-Experiment” (Marketing Science, 2018). Awarded in 2020 for the best 2018–2019 marketing paper based on a Ph.D. thesis published in an INFORMS journal.
Winner of the 2020 Case Centre Award in the Marketing Category for “Predicting Consumer Tastes with Big Data at Gap” (HBS case 517-115) with Jill Avery.
Finalist for the 2018 Frank M. Bass Dissertation Paper Award for “Online MAP Enforcement: Evidence from a Quasi-Experiment” (Marketing Science, 2018). Awarded in 2019 for the best 2017–2018 marketing paper based on a Ph.D. thesis published in an INFORMS journal.
Winner of a 2014 Informs Society for Marketing Science (ISMS) Doctoral Dissertation Proposal Competition Award.
Named a 2012 AMA-Sheth Foundation Doctoral Consortium Fellow.
- Additional Information
-
- Areas of Interest
-
- In The News
-
