Regulatory Oversight, Causal Inference, and Safe and Effective Health Care Machine Learning

Ariel Dora Stern; W Nicholson Price, II

Article | Biostatistics | April 2020

Regulatory Oversight, Causal Inference, and Safe and Effective Health Care Machine Learning

by Ariel Dora Stern and W Nicholson Price, II

Abstract

In recent years, the applications of Machine Learning (ML) in the health care delivery setting have grown to become both abundant and compelling. Regulators have taken notice of these developments and the U.S. Food and Drug Administration (FDA) has been engaging actively in thinking about how best to facilitate safe and effective use. Although the scope of its oversight for software-driven products is limited, if FDA takes the lead in promoting and facilitating appropriate applications of causal inference as a part of ML development, that leadership is likely to have implications well beyond regulated products.

Keywords: Machine learning; causal inference; Health Care and Treatment; Safety; Governing Rules, Regulations, and Reforms;

Format: Print

Find at Harvard Read Now

Citation:

Stern, Ariel Dora, and W Nicholson Price, II. "Regulatory Oversight, Causal Inference, and Safe and Effective Health Care Machine Learning." Biostatistics 21, no. 2 (April 2020): 363–367.

In 2019, the co-founders of the Swedish medical start-up 1928 Diagnostics, CEO Dr. Kristina Lagerstedt and COO Dr. Susanne Staaf, had to pick the right business model to commercialize their novel technology to hospitals and health care providers. Developed in partnership with research hospitals to help fight the global antibiotic resistance crisis, the firm’s cloud-based technology platform helped partners identify resistant genes and mutations in bacteria more quickly and accurately, allowing for easier outbreak cluster tracking in support of hospital infection control management, as well as better diagnostics and antibiotic selection. By 2019, they had raised $5 million, employed 16 people, and had their tool deployed at 24 partner sites in 10 different countries. Their decisions on which markets to focus on and with which business model would crucially impact the young firm’s chances at successfully converting existing users and attracting new clients.

Emerging gene therapy and gene-editing technologies will have a growing impact on patient lives and health-care delivery. We analyzed a decade of data on clinical trials and venture capital investments to understand the likely trajectory of genetically focused therapies in the years ahead. The number of clinical trials and venture capital deals increased substantially from 2006–2017. We observed particularly notable growth in both industry-sponsored trials and trials involving specialty fields of drug development, including oncology, neurology, hematology, ophthalmology, and neurology. As the number of gene-therapy and gene-editing technologies continues to grow, we expect that such therapies will have a significant and heterogeneous impact on health-care delivery, including a shifting of resources from chronic disease management to more intense acute episodic care, an increase in the complexity of required intellectual property and manufacturing know-how, and the potential expansion of biopharmaceutical companies into service-based business models.

Article | JMIR mHealth and uHealth

Assessing the Food and Drug Administration's Risk-Based Framework for Software Precertification with Top Health Apps in the United States: Quality Improvement Study

Noy Alon, Ariel Dora Stern and John Torous

BACKGROUND: As the development of mobile health apps continues to accelerate, the need to implement a framework that can standardize categorizing these apps to allow for efficient, yet robust regulation grows. However, regulators and researchers are faced with numerous challenges as apps have a wide variety of features, constant updates, and fluid use cases for consumers. As past regulatory efforts have failed to match the rapid innovation of these apps, the United States Food and Drug Administration (FDA) has proposed that the Software Precertification (Pre-Cert) Program and new risk-based framework could be the solution. OBJECTIVE: The objective of this study was to determine if the risk-based framework proposed by the FDA's Pre-Cert Program could standardize categorizing top U.S. health apps. METHODS: In this quality improvement study during the summer 2019, the top 10 apps for six disease conditions (addiction, anxiety, depression, diabetes, high blood pressure, and schizophrenia) in the United States Apple iTunes and Android Google Play Store were classified using the FDA's risk-based framework. Data on the presence of well-defined apps features, user engagement methods, popularity metrics, medical claims, and scientific backing were collected. RESULTS: The FDA's risk-based framework classifies an app's risk by the disease condition it targets and what information that app provides. Out of the 120 apps tested, 95 apps were categorized as targeting a non-serious health condition, while only seven targeted a serious condition and 18 apps targeted a critical condition. Since the majority of apps targeted a non-serious condition, their risk categorization was largely determined by what information they provided. The apps that were assessed as not requiring FDA review were more likely than those assessed as requiring review to be associated with the integration of external devices (15/58, 26% vs. 5/62, 8%; P=.03) and health information collection (24/58, 41% vs. 9/62, 15%; P =.008). Apps exempt from review were less likely to offer health information (25/58, 43% vs 45/62, 72%; P <.001), to connect users with professional care (7/58, 12% vs 14/62, 23%; P =.04), and to include an intervention (8/58, 14% vs. 35/62, 55%; P <.001). CONCLUSIONS: The FDA's risk-based framework has the potential to improve the efficiency of the regulatory review process for health apps. However, we were unable to identify a standard measure that differentiated apps requiring regulatory review from those that would not. Apps exempt from review also carried concerns regarding privacy and data security. Before the framework is used to assess the need for formal review of digital health tools, further research and regulatory guidance are needed in order to ensure that the Pre-Cert Program operates in the greatest interest of the public health.

Keywords: mobile health; smartphone; Food and Drug Administration; risk-based framework; Health Care and Treatment; Mobile Technology; Software;

Citation:

Alon, Noy, Ariel Dora Stern, and John Torous. "Assessing the Food and Drug Administration's Risk-Based Framework for Software Precertification with Top Health Apps in the United States: Quality Improvement Study." JMIR mHealth and uHealth 8, no. 10 (October 2020). View Details

Cite View Details Read Now

Faculty & Research

Regulatory Oversight, Causal Inference, and Safe and Effective Health Care Machine Learning

Abstract

Citation:

About the Author

More from the Author