2025
Working Paper
HBS Working Paper Series

Lessons from an App Update at Replika AI: Identity Discontinuity in Human-AI Relationships

By: Julian De Freitas, Noah Castelo, Ahmet Kaan Uğuralp and Zeliha Oğuz-Uğuralp

Format:Print
| Language:English
| Pages:47

Abstract

As consumers increasingly interact with AI applications specialized for social relationships, what is the nature and depth of these relationships among actual users, and can company actions influence these dynamics? We find that active users of the US-based AI companion, Replika, feel closer to their AI companion than even their best human friend, and anticipate mourning the loss of their AI companion more than any other technology. We then leverage an app-update event in which Replika removed its erotic role play (ERP) feature, preventing intimate interactions between consumers and chatbots that were previously possible. We find that this event triggered negative reactions typical of losing a partner in human relationships, including mourning and deteriorated mental health. In short, users of AI companions are forming relationships that are very close and show characteristics typical of human relationships, and company actions can perturb these dynamics, creating risks to consumer welfare. Follow-up studies investigate the psychological mechanisms and what companies can do to mitigate these risks.

Keywords

AI and Machine Learning; Welfare; Loss; Well-being; Identity; Perception; Relationships

Citation

De Freitas, Julian, Noah Castelo, Ahmet Kaan Uğuralp, and Zeliha Oğuz-Uğuralp. "Lessons from an App Update at Replika AI: Identity Discontinuity in Human-AI Relationships." Harvard Business School Working Paper, No. 25-018, October 2024. (Revised May 2025.)