Skip to Main Content
HBS Home
  • About
  • Academic Programs
  • Alumni
  • Faculty & Research
  • Baker Library
  • Giving
  • Harvard Business Review
  • Initiatives
  • News
  • Recruit
  • Map / Directions
Faculty & Research
  • Faculty
  • Research
  • Featured Topics
  • Academic Units
  • …→
  • Harvard Business School→
  • Faculty & Research→
Publications
Publications
  • 2020
  • Working Paper

Targeting for Long-Term Outcomes

By: Jeremy Yang, Dean Eckles, Paramveer Dhillon and Sinan Aral
  • Format:Electronic
  • | Language:English
ShareBar

Abstract

Decision makers often want to target interventions so as to maximize an outcome that is observed only in the long term. This typically requires delaying decisions until the outcome is observed or relying on simple short-term proxies for the long-term outcome. Here we build on the statistical surrogacy and policy learning literature to impute the missing long-term outcomes and then approximate the optimal targeting policy on the imputed outcomes via a doubly-robust approach. We first show that conditions for the validity of average treatment effect estimation with imputed outcomes are also sufficient for valid policy evaluation and optimization; furthermore, these conditions can be somewhat relaxed for policy optimization. We apply our approach in two large-scale proactive churn management experiments at The Boston Globe by targeting optimal discounts to its digital subscribers with the aim of maximizing long-term revenue. Using the first experiment, we evaluate this approach empirically by comparing the policy learned using imputed outcomes with a policy learned on the ground-truth, long-term outcomes. The performance of these two policies is statistically indistinguishable, and we rule out large losses from relying on surrogates. Our approach also outperforms a policy learned on short-term proxies for the long-term outcome. In a second field experiment, we implement the optimal targeting policy with additional randomized exploration, which allows us to update the optimal policy for future subscribers. Over three years, our approach had a net-positive revenue impact in the range of $4-5 million compared to the status quo.

Keywords

Targeted Marketing; Optimization; Churn Management; Marketing; Customer Relationship Management; Policy; Learning; Outcome or Result

Citation

Yang, Jeremy, Dean Eckles, Paramveer Dhillon, and Sinan Aral. "Targeting for Long-Term Outcomes." Working Paper, October 2020.
  • Read Now

About The Author

Jeremy Yang

Marketing
→More Publications

More from the Authors

    • June 2023 (Revised December 2023)
    • Faculty Research

    Hometown Foods Pricing Calculator

    By: Julian De Freitas and Jeremy Yang
    • November 2022
    • Faculty Research

    The Future of E-Commerce: Lessons from the Livestream Wars in China

    By: Ayelet Israeli, Jeremy Yang and Billy Chan
    • July 2022
    • Faculty Research

    Hometown Foods

    By: Julian De Freitas, Jeremy Yang and Das Narayandas
More from the Authors
  • Hometown Foods Pricing Calculator By: Julian De Freitas and Jeremy Yang
  • The Future of E-Commerce: Lessons from the Livestream Wars in China By: Ayelet Israeli, Jeremy Yang and Billy Chan
  • Hometown Foods By: Julian De Freitas, Jeremy Yang and Das Narayandas
ǁ
Campus Map
Harvard Business School
Soldiers Field
Boston, MA 02163
→Map & Directions
→More Contact Information
  • Make a Gift
  • Site Map
  • Jobs
  • Harvard University
  • Trademarks
  • Policies
  • Accessibility
  • Digital Accessibility
Copyright © President & Fellows of Harvard College.