Publications
Publications
- May 2024
- Nature Medicine
The Health Risks of Generative AI-Based Wellness Apps
By: Julian De Freitas and G. Cohen
Abstract
Artifcial intelligence (AI)-enabled chatbots are increasingly being used to
help people manage their mental health. Chatbots for mental health and
particularly ‘wellness’ applications currently exist in a regulatory ‘gray area’.
Indeed, most generative AI-powered wellness apps will not be reviewed
by health regulators. However, recent fndings suggest that users of these
apps sometimes use them to share mental health problems and even to seek
support during crises, and that the apps sometimes respond in a manner
that increases the risk of harm to the user, a challenge that the current US
regulatory structure is not well equipped to address. In this Perspective, we
discuss the regulatory landscape and potential health risks of AI-enabled
wellness apps. Although we focus on the United States, there are similar
challenges for regulators across the globe. We discuss the problems that
arise when AI-based wellness apps cross into medical territory and the
implications for app developers and regulatory bodies, and we outline
outstanding priorities for the feld.
Keywords
AI and Machine Learning; Well-being; Governing Rules, Regulations, and Reforms; Applications and Software
Citation
De Freitas, Julian, and G. Cohen. "The Health Risks of Generative AI-Based Wellness Apps." Nature Medicine 30, no. 5 (May 2024): 1269–1275.