A Jay Holmgren is a doctoral candidate in the health policy management track at Harvard Business School. Prior to HBS, A Jay graduated from the University of Michigan with a Master of Health Informatics degree in 2017; he also holds a BA in History from the university. A Jay previously worked as a research assistant with the Lab for Health IT Impact and Dr. Julia Adler-Milstein, studying the adoption and efficacy of information technology in healthcare. His work has been published in journals including JAMA, Health Affairs, The American Journal of Managed Care, and The Journal of Hospital Medicine. A Jay’s research interests are in digital health. His work seeks to understand how healthcare organizations can best implement and optimize their use of information technology, especially health information exchange and data interoperability, to reduce costs and improve the quality of care.
Prescription drug monitoring programs (PDMPs) have become a widely embraced policy solution to the opioid epidemic in the US. PDMPs offer prescribers a comprehensive view of patients’ controlled substance prescription history and can be used to monitor and reduce inappropriate opioid prescribing. However, poor usability and lack of integration with electronic health records (EHRs) have limited their effectiveness. Furthermore, without PDMP integration in EHRs, prescribers are forced to manage multiple disconnected software systems that interrupt clinical workflow, which may exacerbate technology-driven physician burnout and result in prescribers neglecting to check the PDMP before writing opioid prescriptions. Efforts to integrate PDMPs with EHRs may be associated with reduced prescriber burden and improved PDMP effectiveness; however, the extent to which this adoption has occurred remains unknown on a national scale in the US. Extant literature has focused on usability of PDMP systems rather than adoption and integration. PDMP integration rates are particularly important among hospitals, which are major sources of ambulatory care and thus potential opioid prescribing. Furthermore, hospitals in areas with high opioid prescribing rates may gain the most from PDMP integration efforts because physicians are more likely to provide treatment to patients with current or past opioid prescriptions in these regions. In this study, we assessed the level of EHR and PDMP integration in hospitals, comparing hospitals located in US counties with vs without high opioid prescribing rates.
Prescription drug monitoring programs (PDMPs) have become a widely embraced policy to address the US opioid crisis. Despite mixed scientific evidence on their effectiveness at improving health and reducing overdose deaths, 49 states and Washington, DC have adopted PDMPs, and they have received strong bipartisan legislative support. This article explores the history of PDMPs, tracking their evolution from paper-based administrative databases in the early 1900s to modern-day electronic systems that intervene at the point of care. We focus on two questions: how did PDMPs become so widely adopted in the United States, and how did they gain popularity as an intervention in the contemporary opioid crisis? Through this historical approach, we evaluate what PDMPs reflect about national drug policy and broader cultural understandings of substance use disorder in the United States today.
David Classen, A Jay Holmgren, Zoe Co, Lisa Newmark, Diane Seger, Melissa Danforth and David Bates
Importance Despite the broad adoption of electronic health record (EHR) systems across the continuum of care, safety problems persist.
Objective To measure the safety performance of operational EHRs in hospitals across the country during a 10-year period.
Design, Setting, and Participants This case series included all US adult hospitals nationwide that used the National Quality Forum Health IT Safety Measure EHR computerized physician order entry safety test administered by the Leapfrog Group between 2009 and 2018. Data were analyzed from July 1, 2018 to December 1, 2019.
Exposure The Health IT Safety Measure test, which uses simulated medication orders that have either injured or killed patients previously to evaluate how well hospital EHRs could identify medication errors with potential for patient harm.
Main Outcomes and Measures Descriptive statistics for performance on the assessment test over time were calculated at the overall test score level, type of decision support category level, and EHR vendor level.
Results Among 8657 hospital-years observed during the study, mean (SD) scores on the overall test increased from 53.9% (18.3%) in 2009 to 65.6% (15.4%) in 2018. Mean (SD) hospital score for the categories representing basic clinical decision support increased from 69.8% (20.8%) in 2009 to 85.6% (14.9%) in 2018. For the categories representing advanced clinical decision support, the mean (SD) score increased from 29.6% (22.4%) in 2009 to 46.1% (21.6%) in 2018. There was considerable variation in test performance by EHR.
Conclusions and Relevance These findings suggest that despite broad adoption and optimization of EHR systems in hospitals, wide variation in the safety performance of operational EHR systems remains across a large sample of hospitals and EHR vendors. Hospitals using some EHR vendors had significantly higher test scores. Overall, substantial safety risk persists in current hospital EHR systems.
A Jay Holmgren, Nate Apathy and Julia Adler-Milstein
We sought to identify barriers to hospital reporting of electronic surveillance data to local, state, and federal public health agencies and the impact on areas projected to be overwhelmed by the COVID-19 pandemic. Using 2018 American Hospital Association data, we identified barriers to surveillance data reporting, and combined this with data on the projected impact of the COVID-19 pandemic on hospital capacity at the hospital referral region (HRR) level.
Our results find the most common barrier was public health agencies lacked the capacity to electronically receive data, with 41.2% of all hospitals reporting it. We also identified 31 HRRs in the top quartile of projected bed capacity needed for COVID-19 patients in which over half of hospitals in the area reported that the relevant public health agency was unable to receive electronic data.
Public health agencies’ inability to receive electronic data is the most prominent hospital-reported barrier to effective syndromic surveillance. This reflects the policy commitment of investing in information technology for hospitals without a concomitant investment in IT infrastructure for state and local public health agencies.
David Levine, Zo Co, Lisa Newmark, Alissa Groisser, A Jay Holmgren, Jennifer Haas and David Bates
Mobile health applications (“apps”) have rapidly proliferated, yet their ability to improve outcomes for patients remains unclear. A validated tool that addresses apps’ potentially important dimensions has not been available to patients and clinicians. The objective of this study was to develop and preliminarily assess a usable, valid, and open-source rating tool to objectively measure the risks and benefits of health apps. We accomplished this by using a Delphi process, where we constructed an app rating tool called THESIS that could promote informed app selection. We used a systematic process to select chronic disease apps with ≥4 stars and <4-stars and then rated them with THESIS to examine the tool’s interrater reliability and internal consistency. We rated 211 apps, finding they performed fair overall (3.02 out of 5 [95% CI, 2.96–3.09]), but especially poorly for privacy/security (2.21 out of 5 [95% CI, 2.11–2.32]), interoperability (1.75 [95% CI, 1.59–1.91]), and availability in multiple languages (1.43 out of 5 [95% CI, 1.30–1.56]). Ratings using THESIS had fair interrater reliability (κ = 0.3–0.6) and excellent scale reliability (ɑ = 0.85). Correlation with traditional star ratings was low (r = 0.24), suggesting THESIS captures issues beyond general user acceptance. Preliminary testing of THESIS suggests apps that serve patients with chronic disease could perform much better, particularly in privacy/security and interoperability. THESIS warrants further testing and may guide software and policymakers to further improve app performance, so apps can more consistently improve patient outcomes.
A Jay Holmgren, Zoe Co, Lisa Newmark, Melissa Danforth, David Classen and David Bates
Background Electronic health records (EHR) can improve safety via computerised physician order entry with clinical decision support, designed in part to alert providers and prevent potential adverse drug events at entry and before they reach the patient. However, early evidence suggested performance at preventing adverse drug events was mixed.
Methods We used data from a national, longitudinal sample of 1527 hospitals in the USA from 2009 to 2016 who took a safety performance assessment test using simulated medication orders to test how well their EHR prevented medication errors with potential for patient harm. We calculated the descriptive statistics on performance on the assessment over time, by years of hospital experience with the test and across hospital characteristics. Finally, we used ordinary least squares regression to identify hospital characteristics associated with higher test performance.
Results The average hospital EHR system correctly prevented only 54.0% of potential adverse drug events tested on the 44-order safety performance assessment in 2009; this rose to 61.6% in 2016. Hospitals that took the assessment multiple times performed better in subsequent years than those taking the test the first time, from 55.2% in the first year of test experience to 70.3% in the eighth, suggesting efforts to participate in voluntary self-assessment and improvement may be helpful in improving medication safety performance.
Conclusion Hospital medication order safety performance has improved over time but is far from perfect. The specifics of EHR medication safety implementation and improvement play a key role in realising the benefits of computerising prescribing, as organisations have substantial latitude in terms of what they implement. Intentional quality improvement efforts appear to be a critical part of high safety performance and may indicate the importance of a culture of safety.
Horizontal consolidation in the hospital industry has gained momentum in the United States despite concerns over rising costs and lower quality. Hospital systems frequently point to potential gains in interoperability and electronic exchange of patient information as consolidation benefits. We sought to assess whether hospitals in different health system structures varied in their interoperable data sharing.
A Jay Holmgren, Julia Adler-Milstein and Jeffrey McCullough
Objective
The federal electronic health record (EHR) certification process was intended to ensure a baseline level of system quality and the ability to support meaningful use criteria. We sought to assess whether there was variation across EHR vendors in the degree to which hospitals using products from those vendors were able to achieve high levels of performance on meaningful use criteria.
Materials and Methods
We created a cross-sectional national hospital sample from the Office of the National Coordinator for Health Information Technology EHR Products Used for Meaningful Use Attestation public use file and the Centers for Medicare & Medicaid Services Medicare EHR Incentive Program Eligible Hospitals public use file. We used regression models to assess the relationship between vendor and hospital performance on 6 Stage 2 Meaningful Use criteria, controlling for hospital characteristics. We also calculated how much variation in performance is explained by vendor choice.
Results
We found significant associations between specific vendor and level of hospital performance for all 6 meaningful use criteria. Epic was associated with significantly higher performance on 5 of the 6 criteria; relationships for other vendors were mixed, with some associated with significantly worse performance on multiple criteria. EHR vendor choice accounted for between 7% and 34% of performance variation across the 6 criteria.
Discussion
A nontrivial proportion of variation in hospital meaningful use performance is explained by vendor choice, and certain vendors are more often associated with better meaningful use performance than others. Our results suggest that policy-makers should improve the certification process by including more “real-world” scenario testing and provider feedback or ratings to reduce this variation. Hospitals can use these results to guide interactions with vendors.
Conclusion
Vendor choice accounts for a meaningful proportion of variation in hospital meaningful use performance, and specific vendors are consistently associated with higher or lower performance across criteria.
A Jay Holmgren, Vaishali Patel and Julia Adler-Milstein
Achieving an interoperable health care system remains a top US policy priority. Despite substantial efforts to encourage interoperability, the first set of national data in 2014 suggested that hospitals’ engagement levels were low. With 2015 data now available, we examined the first national trends in engagement in four domains of interoperability: finding, sending, receiving, and integrating electronic patient information from outside providers. We found small gains, with 29.7 percent of hospitals engaging in all four domains in 2015 compared to 24.5 percent in 2014. The two domains with the most progress were sending (with an increase of 8.1 percentage points) and receiving (an increase of 8.4 percentage points) information, while there was no change in integrating systems. Hospitals’ use for patient care of data from outside providers was low, with only 18.7 percent of hospitals reporting that they “often” used these data. Our results reveal that hospitals’ progress toward interoperability is slow and that progress is focused on moving information between hospitals, not on ensuring usability of information in clinical decisions.
A Jay Holmgren, Eric Pfeifer, Milisa Manojlovich and Julia Adler-Milstein
Citation:
Holmgren, A Jay, Eric Pfeifer, Milisa Manojlovich, and Julia Adler-Milstein. "A Novel Survey to Examine the Relationship between Health IT Adoption and Nurse-Physician Communication." Applied Clinical Informatics 7, no. 4 (December 2016): 1182–1201.
View Details
