Skip to Main Content
HBS Home
  • About
  • Academic Programs
  • Alumni
  • Faculty & Research
  • Baker Library
  • Giving
  • Harvard Business Review
  • Initiatives
  • News
  • Recruit
  • Map / Directions
Faculty & Research
  • Faculty
  • Research
  • Featured Topics
  • Academic Units
  • …→
  • Harvard Business School→
  • Faculty & Research→
Publications
Publications
  • 2023
  • Working Paper
  • HBS Working Paper Series

The Dark Side of Generative AI: Chatbots and Mental Health

By: Julian De Freitas, Ahmet Kaan Uğuralp, Zeliha Uğuralp and Stefano Puntoni
  • Format:Print
  • | Language:English
  • | Pages:26
ShareBar

Abstract

Chatbots are now able to engage in sophisticated conversations with consumers. Due to the ‘black box’ nature of the algorithms, it is impossible to predict in advance how these conversations will unfold. Behavioral research provides little insight into potential safety issues emerging from the current rapid deployment of this technology at scale. We begin to address this urgent question by focusing on the context of mental health and “companion AI”: applications designed to provide consumers with synthetic interaction partners. A Pilot Study reports an extensive performance test of several commercially available companion AIs. Studies 1 and 2 present field evidence: actual consumer interactions with two different companion AIs. Study 3 is an experiment testing consumer reaction to risky and unhelpful chatbot responses. The findings show that (1) companions AIs are often unable to recognize, and respond appropriately to, signs of distress; (2) mental health crises are apparent in a non-negligible minority of conversations with users; and (3) consumers display negative reactions to unhelpful and risky chatbot responses, highlighting emerging reputational risks for generative AI companies.

Keywords

Autonomy; Artificial Intelligence; Chatbots; New Technology; Brand Crises; Ethics; Mental Health; AI and Machine Learning; Well-being; Health; Applications and Software

Citation

De Freitas, Julian, Ahmet Kaan Uğuralp, Zeliha Uğuralp, and Stefano Puntoni. "The Dark Side of Generative AI: Chatbots and Mental Health." Harvard Business School Working Paper, No. 23-011, August 2022. (Revised May 2023.)
  • SSRN
  • Read Now

About The Author

Julian De Freitas

Marketing
→More Publications

More from the Authors

    • March 31, 2023
    • Harvard Business Review (website)

    What Is the Optimal Pattern of a Customer Journey?

    By: Julian De Freitas
    • 2023
    • Faculty Research

    Unselfish Alibis Increase Choices of Selfish Autonomous Vehicles

    By: Julian De Freitas
    • January 28, 2023
    • Wall Street Journal

    Will We Blame Self-Driving Cars? A New Study Finds That People Are Likely to Hold Autonomous Vehicles Liable for Accidents Even When They’re Not at Fault

    By: Julian De Freitas
More from the Authors
  • What Is the Optimal Pattern of a Customer Journey? By: Julian De Freitas
  • Unselfish Alibis Increase Choices of Selfish Autonomous Vehicles By: Julian De Freitas
  • Will We Blame Self-Driving Cars? A New Study Finds That People Are Likely to Hold Autonomous Vehicles Liable for Accidents Even When They’re Not at Fault By: Julian De Freitas
ǁ
Campus Map
Harvard Business School
Soldiers Field
Boston, MA 02163
→Map & Directions
→More Contact Information
  • Make a Gift
  • Site Map
  • Jobs
  • Harvard University
  • Trademarks
  • Policies
  • Accessibility
  • Digital Accessibility
Copyright © President & Fellows of Harvard College