• 2023
• Article

As machine learning models are increasingly being employed to make consequential decisions in real-world settings, it becomes critical to ensure that individuals who are adversely impacted (e.g., loan denied) by the predictions of these models are provided with a means for recourse. While several approaches have been proposed to construct recourses for affected individuals, the recourses output by these methods either achieve low costs (i.e., ease-of-implementation) or robustness to small perturbations (i.e., noisy implementations of recourses), but not both due to the inherent trade-offs between the recourse costs and robustness. Furthermore, prior approaches do not provide end users with any agency over navigating the aforementioned trade-offs. In this work, we address the above challenges by proposing the first algorithmic framework which enables users to effectively manage the recourse cost vs. robustness trade-offs. More specifically, our framework Probabilistically ROBust rEcourse (PROBE) lets users choose the probability with which a recourse could get invalidated (recourse invalidation rate) if small changes are made to the recourse i.e., the recourse is implemented somewhat noisily. To this end, we propose a novel objective function which simultaneously minimizes the gap between the achieved (resulting) and desired recourse invalidation rates, minimizes recourse costs, and also ensures that the resulting recourse achieves a positive model prediction. We develop novel theoretical results to characterize the recourse invalidation rates corresponding to any given instance w.r.t. different classes of underlying models (e.g., linear models, tree based models etc.), and leverage these results to efficiently optimize the proposed objective. Experimental evaluation with multiple real world datasets demonstrates the efficacy of the proposed framework.

• 2023
• Article

Importance: Most regulated medical devices enter the U.S. market via the 510(k) regulatory submission pathway, wherein manufacturers demonstrate that applicant devices are “substantially equivalent” to 1 or more “predicate” devices (legally marketed medical devices with similar intended use). Most recalled medical devices are 510(k) devices. Objective: To examine the association between characteristics of predicate medical devices and recall probability for 510(k) devices. Design, Setting, and Participants: In this exploratory cross-sectional analysis of medical devices cleared by the US Food and Drug Administration (FDA) between 2003 and 2018 via the 510(k) regulatory submission pathway, linear probability models were used to examine associations between a 510(k) device’s recall status and characteristics of its predicate medical devices. Public documents for the 510(k) medical devices were collected using FDA databases. A text extraction algorithm was applied to identify predicate medical devices cited in 510(k) regulatory submissions. Algorithm-derived metadata were combined with 2003-2020 FDA recall data. Exposures: Citation of predicate medical devices with certain characteristics in 510(k) regulatory submissions, including the total number of predicate medical devices cited by the applicant device, the age of the predicate medical devices, the lack of similarity of the predicate medical devices to the applicant device, and the recall status of the predicate medical devices. Main Outcomes and Measures: Class I or class II recall of a 510(k) medical device between its FDA regulatory clearance date and December 31, 2020. Results:The sample included 35 176 medical devices, of which 4007 (11.4%) were recalled. The applicant devices cited a mean of 2.6 predicate medical devices, with mean ages of 3.6 years and 7.4 years for the newest and oldest, respectively, predicate medical devices. Of the applicant devices, 93.9% cited predicate medical devices with no ongoing recalls, 4.3% cited predicate medical devices with 1 ongoing class I or class II recall, 1.0% cited predicate medical devices with 2 ongoing recalls, and 0.8% cited predicate medical devices with 3 or more ongoing recalls. Applicant devices citing predicate medical devices with 3 or more ongoing recalls were significantly associated with a 9.31–percentage-point increase (95% CI, 2.84-15.77 percentage points) in recall probability compared with devices without ongoing recalls of predicate medical devices, or an 81.2% increase in recall probability relative to the mean recall probability. A 1-SD increase in the total number of predicate medical devices cited by the applicant device was significantly associated with a 1.25–percentage-point increase (95% CI, 0.62-1.87 percentage points) in recall probability, or an 11.0% increase in recall probability relative to the mean recall probability. A 1-SD increase in the newest age of a predicate medical device was significantly associated with a 0.78–percentage-point decrease (95% CI, 1.29-0.30 percentage points) in recall probability, or a 6.8% decrease in recall probability relative to the mean recall probability. Conclusions and Relevance: This exploratory cross-sectional study of 510(k) medical devices cleared by the FDA between 2003 and 2018 demonstrated significant associations between 510(k) submission characteristics and recalls of medical devices. Further research is needed to understand the implications of these associations.

• January 2023
• Case