Electronic Health Record Optimization and Clinician Well-Being: A Potential Roadmap Toward Action

 

 

In the United States, 86 percent of office-based and 94 percent of hospital-based physicians currently use an electronic health record (EHR), incentivized by the 2009 Health Information Technology for Economic and Clinical Health Act [1,2]. While intended to improve care quality and efficiency, the EHR has inadvertently burdened clinicians and is now considered a leading cause of their burnout [3,4,5]. Clinician burnout (a syndrome characterized by emotional exhaustion, depersonalization, and a low sense of personal accomplishment) is associated with higher rates of medical errors, health care costs, and clinician turnover [6]. In February 2020, the Office of the National Coordinator for Health Information Technology published a strategy on reducing EHR-related burden, further signaling the urgency for health care leaders to optimize the EHR [7]. To shift the pendulum from clinician burnout to well-being, it is imperative that health care organizations take action to optimize the EHR.

EHR optimization relies on human factors engineering, a science that considers the benefits and fallibility of human interaction with a system. Optimization requires a tailored, multipronged strategy that incorporates an organization’s clinician-identified pain points, clinical informatics and technology resources, and clinician and leadership buy-in. This paper provides strategies to help health care organizations embark on their EHR optimization journey toward improved patient care and clinician well-being.

 

Problem Overview: Electronic Health Records and Clinician Well-Being

While EHRs have offered many benefits, EHR design currently emphasizes billing and administrative functions over care delivery and clinical decision making. Furthermore, archaic paper-based workflows were commonly transferred into electronic form instead of creating more streamlined, technology-enabled workflows. Clinicians spend one-half to two-thirds of each workday on EHR and desk work instead of on direct patient care [8].

Indirectly, the EHR may disrupt the patient-clinician relationship and increase inefficiencies, cognitive load, and the risk of clinician burnout. A focus on ensuring data elements for billing purposes are entered into the EHR has led to “note bloat,” or excessively long notes. U.S. physicians’ notes are four times as long as their counterparts in other countries [9]. Many EHRs require an excessive number of steps, or clicks, to carry out a simple function, such as reviewing a patient’s allergies or renewing prescriptions. This phenomenon is so ubiquitous that it is known as “death by a thousand clicks” [10].

EHRs are often designed to send alerts and messages about patient care tasks directly to the physician, regardless of the skill set needed to process the tasks (e.g., medication refill requests, reminders to schedule cancer screenings, or obtaining test results). Consequently, the physician faces extra cognitive load from having to review or complete patient care tasks that could be delegated. A recent assessment of physicians’ in-basket messages, or EHR-based “e-mail inbox” notifications, indicates that almost half are auto-generated by the EHR and that clinicians are at twice as much risk for burnout when their inboxes included an above-average number of EHR-generated messages [11].

 

Optimizing the Electronic Health Record: Strategies and Approaches to Address Well-Being

Fortunately, interventions are available that help address clinician burnout associated with EHR use. Some of these strategies are available now, and others will be available for future implementation—the proximity to implementation is noted underneath the title of the intervention in each of the following sections.

 

EHR Optimization “Sprints” and Work Design Approaches

Available Now

An EHR optimization “Sprint” process (i.e., a short, intensive, team-based intervention) at the University of Colorado Health in Aurora, Colorado, significantly increased EHR satisfaction (e.g., Net Promoter Score increased from –15 to +12) and reduced documentation time for clinicians [12]. The Sprint team was composed of 11 staff members who carried out a three-pronged intervention: (1) training clinicians to use the EHR more efficiently, (2) streamlining multidisciplinary clinic workflows, and (3) building new specialty-specific EHR tools. The team met with clinical and office leaders 90, 60, and 30 days prior to the Sprint to understand clinician pain points and current office workflows as well as to create customized tools. Sprints lasted 5 to 20 days depending on the size of the practice and consisted of on-site kick-off meetings, 1:1 observations and EHR efficiency training sessions, and deployment of new clinic workflows using protected clinician time.

Erlanger Health System in Chattanooga, Tennessee, with assistance from new EHR-based protocols, was able to increase the rate of paperwork completion and office visit documentation within 1 day of the patient visit to 83 percent and reduce turnaround time for medication refill authorizations to within 1.2 days [13]. For medication refills, primary care (PC) physicians created EHR-based protocols to delegate processing these refills to registered nurses or medical assistants (MAs). Importantly, the health system added additional personnel during EHR optimization to conduct listening sessions with EHR “super users,” or actively practicing clinicians who can serve as experts for their colleagues in the field, to ensure that pain points for actual workflows were addressed. Erlanger also tackled clinician-identified low- or no-value documentation requirements with insurers as part of broader efforts to reduce administrative EHR burden.

Hawaii Pacific Health in Honolulu, Hawaii, implemented an EHR campaign called “Getting Rid of Stupid Stuff,” in which physicians and nurses across the system were asked by senior leadership to nominate anything in the EHR that they felt was “poorly designed, unnecessary, or just plain stupid” [14]. According to Ashton (2018), seventy-eight percent of the requests came from nurses, and most nominations were about eliminating documentation that was not required by regulations or internal policies, improving efficiency of workflows, and creating ways to document more efficiently. More than half of the changes suggested had been completed or were in process by November 2018. While a formal assessment of impact on documentation burden or clinician burnout was not conducted, Ashton reported that the initiative was well received by employees.

Several other institutions—including Nemours Children’s Health System, Oregon Health & Science University, and Allina Health—are actively engaged in multipronged EHR optimization approaches as well, which indicates that optimization is both immediately possible and highly important to health systems and clinicians [15,16].

 

In-Basket Management Techniques

Available Now

Reliant Medical Group in Worcester, Massachusetts, rerouted certain types of in-basket messages to the appropriate clinical team members rather than sending all messages to physicians first, resulting in a 25 percent reduction in physician in-basket volume [17]. Reliant also created guidelines for staff to reduce e-mail volume and ensure that e-mailed messages indicated a clear response or action needed for the recipient.

The Department of Veterans Affairs (VA) health system implemented a program to standardize and restrict EHR notifications to types on a recommended evidence-based list and provided training to clinicians focused on customizing and processing notifications generated by the EHR. The program reduced the mean number of daily notifications from 128 to 116 for each PC physician, which resulted in estimated time savings of 1.5 hours per week [18] and ancillary benefits on clinician well-being due to reduced notification and administrative burden, described earlier in this paper.

The American Medical Association (AMA) also has an educational module containing a six-step plan to restructure EHR in-basket messages, relying on team-based care (TBC), in which a clinician is partnered with two to three MAs or nurses [19]. In addition to insights on how to achieve optimization by redirecting EHR messages to the right team member, this module provides guidance on using other TBC techniques to reduce a clinician’s in-basket load, such as creating and utilizing a team in-basket that is accessed and worked on by multiple team members.

 

Documentation Strategies and Team-Based Workflow

Available Now

The use of medical scribes, or non-clinical staff who collect and enter data into EHRs in real-time for the clinician, effectively reduces clinician time spent on documentation and provides other distinct advantages [20,21]. One study using scribes in PC offices showed significant improvements in physician productivity and job satisfaction [22]. While potential disadvantages include the added expense of scribes, the frequency of scribe turnover, and associated training costs [23], these drawbacks can be offset by an increase in clinician willingness and ability to see more patients within the same amount of time [24,25]. A few studies show high patient acceptance of scribes [20,26,27]; however, this area needs further research.

TBC is also a mechanism to address EHR inefficiencies. For example, a nurse and a physician may conduct the in-person visit together, with the clinician focusing on the patient and the nurse assisting with documentation and orders [28]. In between patient visits, clinical support staff process in-basket messages that have arisen during the day. At VA PC practices, a fully staffed care team model was independently associated with lower clinician burnout (odds ratio 0.79) [29]. Additionally, VA PC practices with highly functioning TBC were found to be associated with lower staff burnout, higher quality of clinical care, and higher patient satisfaction compared to practices with lower functioning TBC [30].

 

EHR Training

Available Now

Training clinicians on how to use the EHR, both during implementation and at regular intervals thereafter, is key to maximize efficiency and reduce the risk of burnout. In a survey of health care workers across 156 organizations, physicians that received poor training were 3.5 times more likely to report that their EHR does not “enable them to deliver high-quality care” [31]. Major barriers for training are a lack of protected time for clinicians, customized educational resources to roles and specialty, and consensus on metrics for exemplary (i.e., effective, efficient, and proficient) EHR use. Similar to other clinical skills, “at-the-elbow” training during protected time is perhaps more valuable to clinicians, as real-time feedback can address unique needs alongside peer-to-peer learning.

Novice learners can be taught to utilize best practices upfront instead of needing to “un-learn” ineffective behaviors. Training for learners should be multifaceted, with clear guidelines on roles and responsibilities for EHR-based work, provision of streamlined note templates, and formal education to improve note writing [32]. A widely accepted EHR training curriculum for students is needed but does not currently exist. The AMA launched a consortium of 32 medical schools to accelerate change in medical education, including an EHR training program that had more than 19,000 participants as of 2018 [33]. Despite its significance for patient safety and clinician well-being, graduate medical training programs have neither focused on teaching trainees how to extract or aggregate clinical information efficiently nor fully addressed the hidden agenda that fosters note bloat from coding and billing inquiries.

 

Artificial Intelligence and “Add-On Apps”

Just Down the Road

Use of artificial intelligence (AI) tools such as natural language processing (NLP) and speech recognition (SR) to automate documentation are increasingly used in the health care sector [34]. NLP can reliably extract data from free text sections in notes for billing purposes and create summaries to speed up a clinician’s chart review [35]. SR software packages such as Dragon are commonly used to dictate into the EHR and have been in use since 1994. A recent survey showed that 77 percent of physicians who use SR stated that it improved efficiency [36].

Emerging technology called “digital scribes” seeks to combine SR, NLP, and other AI functionality to fully automate EHR-based documentation and work tasks (e.g., programs such as Suki and Nuance). Exam tools, such as an otoscope, can also capture a variety of clinically useful information that can be directly digitized and transferred to the EHR. While these technologies are promising, they have yet to accurately capture all human interactions with patients (e.g., gesturing) and human-EHR interactions to obviate the need for a clinician to directly interact with the EHR. Furthermore, there are no sector-wide standards for this technology to ensure high quality in a real-world setting, so full integration of these tools into the clinical environment remains a future goal.

Beyond data capture and entry, AI will be increasingly useful as a tool for clinical decision support. In imaging, AI algorithms have been developed and deployed to improve detection of lesions likely to be cancerous such as from mammograms. Clinical utilization of these tools is currently variable and dependent on a range of factors, primarily due to concerns regarding generalizability, which can affect clinician trust in predictive models [37].

One emerging application is “add-on apps” that can seamlessly integrate with and enhance the functionality of EHRs. Made possible by Health Level Seven International’s Substitutable Medical Applications and Reusable Technologies (SMART) and Fast Healthcare Interoperability Resources (FHIR) standards, these apps, coined SMART-on-FHIR, have been successfully developed by both institutions and private industry, often to specifically aid in the management of specific diseases or populations [38].

 

 

Electronic Health Record Optimization: Measurement and Governance

Measurement

Measurement of usability and subsequent optimization efforts should continue after initial EHR deployment. Among other factors, EHRs should be evaluated for the following three aspects based on human factors engineering: (1) their direct interface with the clinician, (2) their impact on unique workflows of the user’s particular specialty, and (3) the physical environment in which they will reside. User experience should also be measured specifically for satisfaction and burnout. A plan should also be developed for how selected metrics will be validated for the specific user’s care team role, activity, specialty, and practice setting.

Organizations should start with soliciting feedback from clinicians about which metrics they feel are most important to monitor for EHR usability and EHR-related burnout. Every attempt should be made to use existing and automated data, including internal workplace and well-being surveys and vendor-provided metrics. Vendor-provided metrics include the number of mouse clicks and keystrokes to complete a specific and isolated task [39,40]; provider efficiency reports, which measure use of EHR-embedded efficiency tools such as order sets, macros, and hot keys, are also useful [8,12]. “Pajama time,” or after-hours EHR use, is directly correlated with clinician burnout and should be trended [8,41]. For in-basket monitoring, volume, types of messages being received, origin (i.e., system- or provider-generated,) clinical value, and message processing time should be measured [42,43,44].

For additional needs, the Agency for Healthcare Research and Quality provides a compendium of validated health information technology (IT) tools [45]. Surveys that allow for external benchmarking such as the KLAS Arch Collaborative survey, which has more than 200 provider organization participants, should be considered [46]. Among multiple available survey tools for user satisfaction, a simple and well-validated approach is the System Usability Scale [47,48]. Eye-tracking and composite measures of workload, such as the National Aeronautics and Space Administration Task Load Index, can be helpful for measurement and to aid iterative workflow redesign [49].

 

Governance

The importance of health IT governance structures should not be overlooked to sustain EHR optimization. Smaller organizations may have an informal governance structure, while larger organizations may formalize a standing governance committee [50]. A standing governance committee should be multidisciplinary and include IT, finance, operational, and clinical leaders with representation across clinical sites of care. Large organizations often employ a chief medical informatics officer (physician), a chief nursing informatics officer (nurse), and/or a chief health information officer (with varied clinical or nonclinical background) to lead optimization work. Recently created roles such as the chief wellness officer may also aid in EHR optimization.

Clinical informatics teams should be established to provide readily available support and to proactively seek out EHR challenges so that they can be addressed before they become problematic [51]. These teams should include IT personnel and clinician super users who can tackle specialty- and discipline-specific EHR nuances. A lack of physician informaticists or informatics staff resources are common reasons for EHR optimization failure and clinician burnout. While research is limited, studies suggest that a staffing ratio of 1 IT full-time equivalent (FTE) for every 50–60 non-IT FTEs is sufficient to support EHR usability [52].

 

Designing for the Future: Emerging Technologies, Evaluation, and Implementation

There will need to be continued emphasis on user-centric EHR design, especially as new applications extend legacy EHR platforms. As referenced above, AI and addon apps will continue to positively disrupt care delivery and should be embraced, rigorously tested, and deployed for use in the clinical setting.

Simulation is an ideal evaluation and testing method for new technologies or new technology applications in addition to improving EHR proficiency and efficiency [53]. Robust simulated patient records that are specialty and workflow congruent and contain enough data density and complexity to accurately assess efficacy for patient outcomes are key [53]. Simulation can also improve the training and onboarding plan for new users. In contrast to demonstration, simulation activities not only effectively train the future user on the new technology but also allow for rapid-cycle improvement for the training process.

Beyond EHRs, further advances in IT are on the horizon. One emerging direction is integration of aggregate data into a common data model from multiple sources (outside the EHR). Data from diverse sources can run through algorithms, which yield knowledge suited for individual use cases. EHRs will be a data entry point and a place to receive AI for expression back to the patients and clinicians in real time at the moment of care. However, for these and others to follow, issues of bias, liability, and transparency (while beyond this paper’s scope) need to be addressed. Furthermore, it is essential that all technologies are tested and evaluated in the context of the providers using them and the environment in which they are used [54].

 

Conclusion

EHRs are a quintessential information platform for modern health care, yet they possess major design and implementation challenges. Fortunately, there are many tools available today to optimize EHRs for enhanced productivity, clinical care, and provider well-being. Going forward, additional best practices need to be developed, and future generations of EHRs and other health IT tools need to be designed with human factors engineering principles to ensure a focus on usability. Strategies in use and/or under development include:

  • engaging clinicians often in the initial stages of EHR implementation and during evaluation and optimization of EHR usability,
  • implementing peer-led training programs and periodically training clinicians to improve EHR efficiency after initial deployment,
  • focusing on workflow redesign as part of EHR optimization,
  • implementing TBC models and including ancillary staff such as scribes,
  • developing strategies and governance to reduce low-value in-basket messages and limit other notifications, and
  • investing in human factors assessment of the work environment to ensure that “right person, right place, right time” is adhered to for all health care workers.

 

Join the conversation!

 

While meant to improve care quality and efficiency, the EHR has inadvertently burdened clinicians. A new #NAMPerspectives outlines interventions available now that can move the EHR toward supporting #ClinicianWellBeing: https://doi.org/10.31478/202008a

Simply implementing targeted, tailored training on the EHR can contribute significantly to its efficiency and effectiveness in the clinical workplace. A new #NAMPerspectives examines this intervention and more: https://doi.org/10.31478/202008a #ClinicianWellBeing

#AI and machine learning may be able to be leveraged in the near future to ensure that the EHR contributes to #ClinicianWellBeing. Read a new #NAMPerspectives to hear about this and other interventions available just down the road: https://doi.org/10.31478/202008a

 

Download the graphics below and share them on social media!

 

 

 

References

  1. Myrick, K. L., D. F. Ogburn, and B. W. Ward. 2019. Percentage of office-based physicians using any electronic health record (EHR)/electronic medical record (EMR) system and physicians that have a certified EHR/EMR system, by U.S. state: National Electronic Health Records Survey, 2017. National Center for Health Statistics. Available at: https://www.cdc.gov/nchs/data/nehrs/2017_NEHRS_Web_Table_EHR_State.pdf (accessed June 1, 2020).
  2. Office of the National Coordinator for Health Information Technology. 2019. Hospitals’ use of electronic health records data, 2015–2017. ONC Data Brief No. 46. Available at: https://www.healthit.gov/sites/default/files/page/2019-04/AHAEHRUseData-Brief.pdf (accessed June 1, 2020).
  3. Friedberg, M. W., P. G. Chen, K. R. Van Busum, F. Aunon, C. Pham, J. P. Caloyeras, S. Mattke, E. Pitchforth, D. D. Quigley, R. H. Brook, F. J. Crosson, and M. Tutty. 2013. Factors affecting physician professional satisfaction and their implications for patient care, health systems, and health policy. Santa Monica, CA: RAND Corporation. Available at: https://pubmed.ncbi.nlm.nih.gov/28083306 (accessed June 1, 2020).
  4. Shanafelt, T. D., L. N. Dyrbye, C. Sinsky, O. Hasan, D. Satele, J. Sloan, and C. P. West. 2016. Relationship between clerical burden and characteristics of the electronic environment with physician burnout and professional satisfaction. Mayo Clinic Proceedings 91(7):836–848. https://doi.org/ 10.1016/j.mayocp.2016.05.007.
  5. Stanford Medicine. 2018. How doctors feel about electronic health records: National physician poll. The Harris Poll. Available at: http://www.med.stanford.edu/content/dam/sm/ehr/documents/EHR-Poll-Presentation.pdf (accessed February 28, 2020).
  6. Dyrbye, L. N., T. D. Shanafelt, C. A. Sinsky, P. F. Cipriano, J. Bhatt, A. Ommaya, C. P. West, and D. Meyers. 2017. Burnout among health care professionals: A call to explore and address this underrecognized threat to safe, high-quality care. NAM Perspectives. Discussion Paper, National Academy of Medicine, Washington, DC. https://doi.org/10.31478/201707b.
  7. The Office of the National Coordinator for Health Information Technology. 2020. Strategy on reducing regulatory and administrative burden relating to the use of health IT and EHRs. Available at: https://www.healthit.gov/sites/default/fi les/webform/reducing_burden_report/draft-strategy-on-reducing-regulatory-and-administrative-burden-relating—rkb-comments.pdf (accessed March 10, 2020).
  8. Arndt, B. G., J. W. Beasley, M. D. Watkinson, J. L. Temte, W. Tuan, C. A. Sinsky, and V. J. Gilchrist. 2017. Tethered to the EHR: PC physician workload assessment using EHR event log data and time-motion observations. Annals of Family Medicine 15(5):419–426. https://doi.org/10.1370/afm.2121.
  9. Downing, N. L., D. W. Bates, and C. A. Longhurst. 2018. Physician burnout in the electronic health record era: Are we ignoring the real cause? Annals of Internal Medicine 169:50–51. https://doi.org/10.7326/M18-0139.
  10. Schulte, F., and E. Fry. 2019. Death by a thousand clicks: Where electronic health records went wrong. Kaiser Health News. Available at: https://www.healthleadersmedia.com/innovation/deaththousand-clicks-where-electronic-health-recordswent-wrong (accessed May 13, 2020).
  11. Tai-Seale, M., E. C. Dillon, Y. Yang, R. Nordgren, R. L. Steinberg, T. Nauenberg, T. C. Lee, A. Meehan, J. Li, A. S. Chan, and D. L. Frosch. 2019. Physicians’ well-being linked to in-basket messages generated by algorithms in electronic health records. Health Affairs 38(7):1073–1078. https://doi.org/10.1377/hlthaff .2018.05509.
  12. Sieja, A., K. Markley, J. Pell, C. Gonzalez, B. Redig, P. Kneeland, and C. Lin. 2019. Optimization Sprints: Improving clinician satisfaction and teamwork by rapidly reducing electronic health record burden. Mayo Clinic Proceedings 94(5):793–802. https://doi.org/10.1016/j.mayocp.2018.08.036.
  13. American Hospital Association Physician Alliance. 2018. Erlanger Health System. Available at: https://www.aha.org/system/files/2018-11/plf-case-studyerlanger.pdf (accessed February 28, 2020).
  14. Ashton, M. 2018. Getting rid of stupid stuff. New England Journal of Medicine 8(19):1789–1791. https://doi.org/10.1056/NEJMp1809698.
  15. Raths, D. 2019. With physician burnout rising, EHR optimization takes center stage. Healthcare Innovation. Available at: https://www.hcinnovationgroup.com/clinical-it/article/21089517/with-physicianburnout-rising-ehr-optimization-takes-centerstage (accessed February 28, 2020).
  16. American Hospital Association. 2020. Be well case studies. Available at: https://www.aha.org/be-wellcase-studies (accessed March 10, 2020).
  17. Massachusetts Medical Society. 2019. Changing the EHR from a liability to an asset to reduce physician burnout: The Reliant Medical Group story. Available at: http://www.massmed.org/News-and-Publications/Changing-EHR-Physician-Burnout (accessed February 28, 2020).
  18. Shah, T., S. Patel-Teague, L. Kroupa, A. Meyer, and H. Singh. 2019. Impact of a national QI programme on reducing electronic health record notifications to clinicians. BMJ Quality and Safety 28(1):10–14. https://doi.org/10.1136/bmjqs-2017-007447.
  19. Jerzak, J., and C. Sinsky. 2017. EHR in-basket restructuring for improved efficiency. American Medical Association Steps Forward. Available at: https://edhub.ama-assn.org/steps-forward/module/2702694 (accessed May 13, 2020).
  20. Gidwani, R., C. Nguyen, A. Kofoed, C. Carragee, T. Rydel, I. Nelligan, A. Sattler, M. Mahoney, and S. Lin. 2017. Impact of scribes on physician satisfaction, patient satisfaction, and charting efficiency: A randomized controlled trial. Annals of Family Medicine 15(5):427–433. https://doi.org/10.1370/afm.2122.
  21. Smith, C. D., C. Balatbat, S. Corbridge, A. L. Dopp, J. Fried, R. Harter, S. Landefeld, C. Martin, F. Opelka, L. Sandy, L. Sato, and C. Sinsky. 2018. Implementing optimal team-based care to reduce clinician burnout. NAM Perspectives. Discussion Paper, National Academy of Medicine, Washington, DC. https://doi.org/10.31478/201809c.
  22. Mishra, P., J. C. Kiang, and R. W. Grant. 2018. Association of medical scribes in PC with physician workflow and patient experience. JAMA Internal Medicine 178(11):1467–1472. https://doi.org/10.1001/jamainternmed.2018.3956.
  23. Williams, M. 2016. The pros and cons of using scribes. ACP Internist. Available at: https://acpinternist.org/archives/2016/10/tips.htm (accessed June 1, 2020).
  24. Bank, A. J., C. Obetz, A. Konrardy, A. Khan, K. M. Pillai, B. J. McKinley, R. M. Gage, M. A. Turnbull, and W. O. Kenney. 2013. Impact of scribes on patient interaction, productivity, and revenue in a cardiology clinic: A prospective study. ClinicoEconomics and Outcomes Research 5:399–406. https://doi.org/10.2147/ceor.s49010.
  25. Nambudiri, V. E., A. J. Watson, E. A. Buzney, T. S. Kupper, M. H. Rubenstein, and F. C. Yang. 2018. Medical scribes in an academic dermatology practice. JAMA Dermatology 54(1):101–103. https://doi.org/10.1001/jamadermatol.2017.3658.
  26. Pozdnyakova, A., N. Laiteerapong, A. Volerman, L. D. Feld, W. Wan, D. L. Burnet, and W. W. Lee. 2018. Impact of medical scribes on physician and patient satisfaction in primary care. Journal of General Internal Medicine 33(7):1109–1115. https://doi.org/10.1007/s11606-018-4434-6.
  27. Zallman, L., K. Finnegan, D. Roll, M. Todaro, R. Oneiz, and A. Sayah. 2018. Impact of medical scribes in PC on productivity, face-to-face time, and patient comfort. Journal of the American Board of Family Medicine 7(1):612–619. https://doi.org/10.3122/jabfm.2018.04.170325.
  28. Sinsky, C. A., and T. Bodenheimer. 2019. Powering-up primary care teams: Advanced team care with in-room support. Annals of Family Medicine 17(4):367–371. https://doi.org/10.1370/afm.2422.
  29. Helfrich, C. D., E. D. Dolan, J. Simonetti, R. J. Reid, S. Joos, B. J. Wakefield, G. Schectman, R. Stark, S. D. Fihn, H. B. Harvey, and K. Nelson. 2014. Elements of team-based care in a patient-centered medical home are associated with lower burnout among VA primary care employees. Journal of General Internal Medicine 29:659–666. https://doi.org/10.1007/s11606-013-2702-z.
  30. Nelson, K. M., C. Helfrich, H. Sun, P. L. Hebert, C. F. Liu, E. Dolan, L. Taylor, E. Wong, C. Maynard, S. E. Hernandez, W. Sanders, I. Randall, I. Curtis, G. Schectman, R. Stark, and S. D. Fihn. 2014. Implementation of the patient-centered medical home in the Veterans Health Administration: Associations with patient satisfaction, quality of care, staff burnout, and hospital and emergency department use. JAMA Internal Medicine 174(8):1350–1358. https://doi.org/10.1001/jamainternmed.2014.2488.
  31. Longhurst, C. A., T. Davis, A. Maneker, H. C. Eschenroeder Jr., R. Dunscombe, G. Reynolds, B. Clay, T. Moran, D. B. Graham, S. M. Dean, and J. Adler-Milstein. 2019. Local investment in training drives electronic health record user satisfaction. Applied Clinical Informatics 10(2):331–335. https://doi.org/10.1055/s-0039-1688753.
  32. Dean, S. M., J. C. Eickhoff , and L. A. Bakel. 2015. Improving notes in the EHR. Journal of Hospital Medicine 2:104–107. https://doi.org/10.1002/jhm.2283.
  33. Monica, K. 2018. AMA expanding EHR training, medical education consortium. EHR Intelligence. Available at: https://ehrintelligence.com/news/ama-expanding-ehr-training-medical-educationconsortium (accessed March 10, 2020).
  34. Coiera, E., B. Kocaballi, J. Halamka, and L. Laranjo. 2018. The digital scribe. NPJ Digital Medicine 1:58. https://doi.org/10.1038/s41746-018-0066-9.
  35. Ommaya, A. K., P. F. Cipriano, D. B. Hoyt, K. A. Horvath, P. Tang, H. L. Paz, M. S. DeFrancesco, S. T. Hingle, S. Butler, and C. A. Sinsky. 2018. Care-centered clinical documentation in the digital environment: Solutions to alleviate burnout. NAM Perspectives. Discussion Paper, National Academy of Medicine, Washington, DC. https://doi.org/10.31478/201801c.
  36. Goss, F. R., S. V. Blackley, C. A. Ortega, L. T. Kowalski, A. B. Landman, C. T. Lin, M. Meteer, S. Bakes, S. C. Gradwohl, D. W. Bates, and L. Zhou. 2019. A clinician survey of using speech recognition for clinical documentation in the electronic health record. International Journal of Medical Informatics 130. https://doi.org/10.1016/j.ijmedinf.2019.07.017.
  37. Kelly, C. J., A. Karthikesalingam, M. Suleyman, G. Corrado, and D. King. 2019. Key challenges for delivering clinical impact with artificial intelligence. BMC Medicine 17:195. https://doi.org/10.1186/s12916-019-1426-2.
  38. Kawamoto, K., P. Kukhareva, J. H. Shakib, H. Kramer, S. Rodriguez, P. B. Warner, D. Shields, C. Weir, G. Del Fiol, T. Taft, and C. H. Stipelman. 2019. Association of an Electronic Health Record add-on app for neonatal bilirubin management with physician efficiency and care quality. JAMA Network Open 2(11):e1915343. https://doi.org/10.1001/jamanetworkopen.2019.15343.
  39. Ratwani, R. M., E. Savage, A. Will, A. Fong, D. Karavite, N. Muthu, A. J. Rivera, C. Gibson, D. Asmonga, B. Moscovitch. R. Grundmeier, and J. Rising. 2018. Identifying electronic health record usability and safety challenges in pediatric settings. Health Affairs 37(11):1752–1759. https://doi.org/ 10.1377/hlthaff .2018.0699.
  40. Gold, J. A., L. E. Stephenson, A. Gorsuch, K. Parthasarathy, and V. Mohan. 2015. Feasibility of utilizing a commercial eye tracker to assess electronic health record use during patient simulation. Health Informatics Journal 22(3):744–757. https://doi.org/10.1177/1460458215590250.
  41. Gardner, R. L., E. Cooper, J. Haskell, D. A. Harris, S. Poplau, P. J. Kroth, and M. Linzer. 2019. Physician stress and burnout: The impact of health information technology. Journal of the American Medical Informatics Association 26(2):106–114. https://doi.org/10.1093/jamia/ocy145.
  42. Kizzier-Carnahan, V., K. A. Artis, V. Mohan, and J. A. Gold. 2019. Frequency of passive EHR alerts in the ICU: Another form of alert fatigue? Journal of Patient Safety 15(3):246–250. https://doi.org/10.1097/PTS.0000000000000270.
  43. Lin, C. P., T. H. Payne, W. P. Nichol, P. J. Hoey, C. L. Anderson, and J. H. Gennari. 2008. Evaluating clinical decision support systems: Monitoring CPOE order check override rates in the Department of Veterans Affairs’ computerized patient record system. Journal of the American Medical Informatics Association 15(5):620–626. https://doi.org/10.1197/jamia.M2453.
  44. Spirk, D., A. K. Stuck, A. Hager, R. P. Engelberger, D. Aujesky, and N. Kucher. 2017. Electronic alert system for improving appropriate thromboprophylaxis in hospitalized medical patients: A randomized controlled trial. Journal of Thrombosis and Haemostasis 15(11):2138–2146. https://doi.org/10.1111/jth.13812.
  45. Digital Healthcare Research. 2020. Health IT survey compendium. Agency for Healthcare Research and Quality. Available at: https://digital.ahrq.gov/health-it-tools-and-resources/evaluation-resources/health-it-survey-compendium-search (accessed March 10, 2020).
  46. KLAS. 2020. Home. Available at: https://klasresearch.com/home (accessed March 10, 2020).
  47. Brooke, J. 1996. SUS: A “quick and dirty” usability scale. In Usability evaluation in industry, edited by P. W. Jordan, B. Thomas, B. A. Weerdmeester, and I. L. McClelland, pp. 189–194. London: Taylor & Francis.
  48. Sandefer, R., D. Brinda, J. Wapola, S. E. Maki, and D. Marc. 2013. EHR usability on mobile devices. Educational Perspectives in Health Informatics and Information Management 1–11. Available at: https://pdfs.semanticscholar.org/09e3/ae0769cf0c9b7e32d69550c4846aafe472bf.pdf?_ga=2.22700091.1287485016.1591114417-1171844949.1589903450 (accessed June 2, 2020).
  49. Hart, S. G. 2006. Nasa-Task Load Index (NASA-TLX); 20 Years Later. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 50:904–908. https://doi.org/10.1177/154193120605000909.
  50. The Office of the National Coordinator for Health Information Technology. 2020. Data Governance. Available at: https://www.healthhit.gov/playbook/ambulatory-guide/data-governance (accessed March 10, 2020).
  51. Rajaram, A., H. Zachary, P. Nimesh, N. Joseph, and B. Wolfrom. 2020. Training medical students and residents in the use of electronic health records: A systematic review of the literature. Journal of the American Medical Informatics Association 27(1):175–180. https://doi.org/10.1093/jamia/ocz178.
  52. Hersh, W. 2010. The health information technology workforce: Estimations of demands and a framework for requirements. Applied Clinical Informatics 1(2):197–212. https://doi.org/10.4338/ACI-2009-11-R-0011.
  53. Mohan, V., D. Woodcock, K. McGrath, G. Scholl, R. Pranaat, J. W. Doberne, D. A. Chase, J. A. Gold, and J. S. Ash. 2016. Using simulations to improve electronic health record use, clinician training and patient safety: Recommendations from a consensus conference. AMIA Annual Symposium Proceedings Archive 904–913. Available at: https://pubmed.ncbi.nlm.nih.gov/28269887 (accessed June 2, 2020).
  54. Takeuchi, W., S. Tarumi, S. Rodriquez, D. Shields, P. Warner, M. Flynn, K. Turner, F. Sakaguchi, H. Ban, and K. Kawamoto. 2018. EHR-integrated, machine learning-driven SMART on FHIR pharmacotherapy decision support system for type-2 diabetes mellitus. Paper presented at the AMIA 2018 Informatics Summit, San Francisco, CA. Available at: https://knowledge.amia.org/amia-66728-cri2018-1.4079151/t004-1.4080149/f004-1.4080150/a090-1.4080157/an090-1.4080158?qr=1 (accessed June 2, 2020).