Abstract
Objective
The objective of this paper is to share challenges, recommendations, and lessons learned regarding the development and implementation of a Patient Safety Learning Laboratory (PSLL) project, an innovative and complex intervention comprised of a suite of Health Information Technology (HIT) tools integrated with a newly implemented Electronic Health Record (EHR) vendor system in the acute care setting at a large academic center.
Materials and Methods
The PSLL Administrative Core engaged stakeholders and study personnel throughout all phases of the project: problem analysis, design, development, implementation, and evaluation. Implementation challenges and recommendations were derived from direct observations and the collective experience of PSLL study personnel.
Results
The PSLL intervention was implemented on 12 inpatient units during the 18-month study period, potentially impacting 12,628 patient admissions. Challenges to implementation included stakeholder engagement, project scope/complexity, technology/governance, and team structure. Recommendations to address each of these challenges were generated, some enacted during the trial, others as lessons learned for future iterative refinements of the intervention and its implementation.
Conclusion
Designing, implementing, and evaluating a suite of tools integrated within a vendor EHR to improve patient safety has a variety of challenges. Keys to success include continuous stakeholder engagement, involvement of systems and human factors engineers within a multidisciplinary team, an iterative approach to user-centered design, and a willingness to think outside of current workflows and processes to change health system culture around adverse event prevention.
Keywords: patient safety, health information technology, quality improvement, consumer health informatics, patient-centered care
INTRODUCTION
Research evaluating health information technology (HIT) often narrowly focuses on its effects on quality, efficiency, and cost.1 Less well understood are the technical, organizational, human factors, and project management components that are key to its successful implementation.1 The complex and competing interplays between hospital priorities, provider satisfaction, patient experience, limitations in technical resources, patient safety, and health services research must be balanced for successful uptake of these technologies.2–4
Implementation frameworks have been developed to guide user adoption and foster successful integration of evidence-based interventions and technologies into clinical work flow, but there is more work to be done in developing best practices for implementing novel, large scale HIT that integrates well into both vendor electronic health record (EHR) and provider work flows.5 Because HIT implementation faces obstacles, establishing standards and sharing best practices is critical to understanding the impact of large HIT interventions on care delivery.1,2 The Patient Safety Learning Laboratory (PSLL) was the first set of HIT tools to be integrated with a newly implemented vendor EHR (Epic, Verona, WI) in our clinical setting, with no established precedent for how to innovate within the newly formed governance and technology support structures.6
Using our experiences implementing a complex PSLL study in a large academic medical center, we share challenges, lessons learned, and provide guidance on implementation practices for organizations seeking to improve patient care through large scale patient-centered HIT. This article provides actionable information for healthcare organizations aiming to implement research interventions that complement and interact with vendor EHR systems. While there is no one-size-fits-all implementation approach, our aim is to provide recommendations that can be adapted to other complex healthcare settings.
MATERIALS AND METHODS
Overview of patient safety learning laboratory study and aims
The PSLL project, a 4-year Agency for Healthcare Research and Quality (AHRQ) funded study (2014–2018), was a collaboration between the Brigham and Women’s Hospital (BWH) Center for Patient Safety Research and Practice and the Northeastern University Healthcare Systems Engineering Institute (HSyE).6 The focus was to make acute care more patient-centered and improve patient safety by developing tools and interventions that engage patients along with the care team in mitigating preventable harm during hospitalization. In collaboration with patients and providers, the PSLL team utilized systems engineering approaches to develop a suite of patient- and provider-facing tools to raise patients’ awareness of safety issues and prevention strategies, encourage patients to give input into their care, and provide tools for clinicians to monitor patient safety risks in real time.
Patient safety learning laboratory project structure
Governance Structure:A steering committee was created that: 1) guided decisions regarding conduct of the study; and 2) provided multiple methods for gathering stakeholder input. The governance structure is described in Appendix 1, and the PSLL project’s division of responsibility is below:
Administrative Core: responsible for the overall leadership, organization and coordination of PSLL activities. Because PSLL was a complex interdisciplinary project spanning multiple hospital units and services, this core focused the project teams on the overarching goals and provided project management oversight of the following activities: 1) collaboration and communication between team members, collaborators, and stakeholders; 2) selection of strategies for utilizing HIT to facilitate patient activation in reducing harm in hospital settings; 3) scientific support for study design and analyses; 4) adherence to allotted time frames and budgets; and, 5) coordination of HIT development and use of resources across projects.
Systems Engineering (SE) and Human Factors (HF) Core:supported project teams during development, iterative refinement, and implementation of the suite of EHR-integrated HIT tools. The core, a collaboration between BWH and HSyE, drew from a compendium of SE/HF methods during each project phase and encouraged project teams to consider the “systems-of-systems” perspective: how individual tools interact as a single, unified system to identify, assess, and mitigate threats to preventable hospital-acquired harms.6
Project Teams:Each project, comprised of a coinvestigator and research assistant, shared a goal of creating an electronic patient-centered tool to improve patient safety. The projects included:
Fall TIPS (Tailoring Interventions for Patient Safety): An evidence-based fall prevention protocol and associated tools to reduce falls by engaging patients and family in identifying fall risks, helping providers create a personalized fall prevention plan and consistently following the plan.7,8
MySafeCare (MSC): A reporting tool that enabled patients and caregivers to report safety concerns, which were then displayed on a clinician dashboard to be addressed by providers and unit-based clinical leaders.9,10 Patients and caregivers could choose to report anonymously or identify themselves.
Patient Safety Dashboard: A real-time safety display for interdisciplinary care team members, complemented by a patient-facing safety portal, which shared information from the safety dashboard with patients/caregivers, and a bedside display with safety and plan information shown on a monitor in the patient room.11
PSLL EHR-integrated health information technology tools
The 3 project teams worked together to create a set of tools with overlapping functionality (Figure 1). Many of the tools achieved the goals of more than 1 project and were integrated with our newly implemented vendor EHR (Epic, Verona, WI) using live data services to pull real-time clinical information, manipulate and display it to be maximally useful for clinicians, patients, caregivers, and unit-based clinical leadership. The PSLL team also readied the environment for implementation of the tools in the clinical setting using the Patient SatisfActive model.2,4,12,13
Figure 1.
Patient Safety Learning Laboratory (PSLL): EHR-Integrated Health Information Technology Tools. The Patient Safety Learning Laboratory developed tools to help patients become better informed and more involved in their care and provided clinicians with information to facilitate better and faster decisions about patient care. Patients used the interactive patient portal; a bedside display showed patient care plan information on a monitor in the patient room for patients, families and hospital staff; clinicians viewed the patient safety dashboard which contained patient data to alert hospital staff of potential patient risks.7–11 All of this was built upon the Patient SatisfActive model, a communication system that helped clinicians identify, assess, and address patient needs and expectations and helped patients and their caregivers become an active part of their care and decision-making.2,4,12,13
Conceptual models and framework
The PSLL utilized the reach, effectiveness, adoption, implementation, and maintenance (RE-AIM) framework to guide the study design.14 RE-AIM informs implementation and evaluation with guiding questions in each domain, and its focus on external validity makes it well suited for technology innovation projects.14,15
The broad research questions from each RE-AIM component were used to guide the study through all phases, providing practical measures of how well the intervention worked in real-world clinical settings.15,16Figure 2 describes how the PSLL applied RE-AIM to our study and how each outcome measure was associated with each RE-AIM component and with the study as a whole.
Figure 2.
Reach, Effectiveness, Adoption, Implementation and Maintenance (RE-AIM) framework: addressing the barriers to translation of research into practice in the Patient Safety Learning Laboratory (PSLL) study. The PSLL applied the RE-AIM framework to our overall implementation by 1) Formulating research questions from each RE-AIM component to our associated outcome measures, 2) Addressing implementation and dissemination through all phases of the research project lifecycle study-wide; and 3) Providing practical measures of how well the intervention worked in varied clinical settings.15,16
Project methodology
The PSLL project was phased in accordance with the AHRQ systems engineering project life cycle: 1) problem analysis, 2) design, 3) development, 4) implementation, and 5) evaluation.17 During implementation planning, we adopted a socio-technical approach to understand the cultural and environmental context.18
Our study was approved by the Partners Human Research Committee, the Institutional Review Board of Partners HealthCare.
Throughout each phase, the Administrative Core: 1) engaged with stakeholders at all levels of the organization to establish buy-in, assist with key decisions and strategies, and plan for the implementation; 2) discussed project risks and mitigation plans with project teams; and 3) ensured project teams connected intervention development with overarching design and implementation goals. Table 1 describes the activities and implementation goals during each phase of the PSLL project.
Table 1.
PSLL project phases, activities, and implementation goals
| Project Phase | Activities | Implementation Goals |
|---|---|---|
| Problem Analysis | Engage stakeholders to gauge the impact of tools on communication and safety |
|
| Design | Leverage team’s expertise to ensure that differing perspectives and prototypes are refined through focus groups with stakeholders using develop-test-revise iterations |
|
| Development | Collaborate with stakeholders in refinement of tools until a sufficiently mature version is developed that stakeholders agree will both facilitate team communication and engage patients and families |
|
| Implementation | Implement PSLL tools on 12 units on varied services as a stepped wedge cluster-randomized controlled trial design |
|
| Evaluation | Evaluate process and outcome measures related to each RE-AIM component |
|
Abbreviations: BWH, Brigham and Women’s Hospital; IT, information technology; PSLL, patient safety learning laboratory; RE-AIM, Reach, Effectiveness, Adoption, Implementation and Maintenance framework.
The PSLL research study was designed as a stepped wedge cluster-randomized trial for 12 inpatient units at BWH across 3 services: General Medicine, Oncology, and Neurology (Appendix 2). During the 18-month study period (12/1/2016 through 7/31/17), each unit or group of units (cluster) started in usual care and then moved to the intervention at a prespecified randomized time point. The PSLL tools were implemented on 1 cluster (“step”) every 6 weeks. This design allowed us to focus implementation efforts on 1 group of clinicians at a time and allowed each unit to be compared to itself (adjusting for different micro-cultures and patient populations) while adjusting for temporal trends.19,20
RESULTS
The PSLL was implemented on 12 inpatient units at BWH. Nurses were trained on the tools via small-group sessions and during unit-based practice council meetings. Physicians and physician assistants were trained via weekly orientation sessions (prior to starting a rotation on 1 of the study units). All clinicians were supported “at the elbow” by PSLL team members and by clinical champions that were identified and trained by the team. The patient safety dashboard and bedside display were available for every patient admitted to the study units during the study period, for a total of 12 628 patient admissions. The patient safety dashboard was opened by 184 nurses (nurses, patient care assistants, and nursing students), 179 prescribers (attending physicians, physician assistants, nurse practitioners, fellows, residents, and medical students), and 19 unit leadership staff (administrators and nurse/medical directors). The patient portal was given to 1761 patients out of 3002 patients approached. The PSLL team’s challenges and recommendations implementing the suite of tools is described in Table 2.
Table 2.
Patient safety learning laboratory project: implementation challenges and recommendations by category
| Category | Challenge | Recommendation |
|---|---|---|
| Stakeholder Engagement | User variety: (MD, PA, RN, Patients, Caregivers, Unit Directors, etc.), 3 clinical services (Oncology, General Medicine, Neurology) | |
| Project Complexity |
|
|
| Change fatigue |
|
|
| Requirement for Enduring Engagement |
|
|
| Project Scope/Complexity | Multiple products designed for different user groups (clinician vs patient) with different use cases |
|
| Multiplicity of sites (units) for implementation |
|
|
| Varied implementation/ training needs |
|
|
| Technology/ Governance | Technical bugs |
|
| Project and scope management |
|
|
| Development of usable, helpful products |
|
|
| Data access |
|
|
| Team Structure | Varied and wide-ranging research assistant (RA) tasks |
|
| Co-investigator involvement in training and support |
|
|
| Risk mitigation |
|
Abbreviations: MD, medical doctor; PA, physician assistant; RA, research assistant; RN, registered nurse.
Because PSLL was the first large-scale HIT research study to be conducted at our institution after the deployment of a vendor EHR, there were many technical hurdles to overcome (like developing new live data services), and governance issues to be resolved (eg, permissions to develop new services or provide a link within the EHR environment to launch the PSLL tools). Figure 3 describes the key implementation lessons learned by project phase.
Figure 3.
Implementation Lessons Learned by Project Phase. The Patient Safety Learning Laboratory projects followed the Agency for Healthcare Research and Quality Systems Design lifecycle: problem analysis, design, development, implementation, evaluation.17 During each project phase, the PSLL administrative core prepared for implementation and documented lessons learned.
DISCUSSION
The complexity of the PSLL research project involved management of multiple dimensions: research, hospital operations, technology, the newly formed BWH/HSyE collaboration, and engagement with multiple stakeholders (eg, patients, family members, clinicians, and hospital leaders) working amid newly formed governance structures within the organization. The PSLL study was a test case for this hospital, with no established precedent for how to innovate and integrate with a newly implemented vendor EHR system. The RE-AIM provided a framework for the PSLL team to design, implement, and evaluate tools to engage patients in their care and improve patient safety.
The success of the PSLL project hinged on building a multi-disciplinary team focused on a well-articulated vision, mission, and goals. Establishing buy-in internally about the “why” of our project, then communicating a clear and consistent message to organizational leaders and champions was key to ensure successful adoption of the PSLL technologies into the work flow, and more broadly, into the organization (Figure 3).
The PSLL team benefited from having multiple projects under 1 infrastructure, and implementing multiple tools simultaneously resulted in culture change for a broad group of users across awide array of disciplines. The interventions and tools were synergistic, requiring a common set of data for all projects. The health systems engineering collaboration helped us focus on the whole system and address variability in work flows. The complexity of this study placed high demands on the clinical and administrative infrastructure, and these logistical supports, along with a comprehensive implementation framework, helped ameliorate that demand.
CONCLUSION
A key metric of success for HIT projects of this magnitude is organizational adoption. BWH has committed to adopting the PSLL HIT innovations and is planning to spread the PSLL tools to all clinical units across BWH. This process, which we are calling “productization,” has its own set of challenges, but some of the lessons learned regarding implementation during the PSLL study can be applied to this next phase.
Another major highlight of this research is that we successfully extracted data in real-time from the newly implemented vendor EHR and displayed it to providers and patients in novel ways, engaging patients, family members and clinical care teams to improve patient safety and clinical care. We plan to use approaches we developed for future projects, and with the utilization of open application programming interfaces (API) becoming more mainstream, real-time data extraction from vendor systems and the creation of novel applications built “around” a vendor EHR may be possible for many organizations and across multiple vendors.
The evaluation phase of the PSLL study—evaluating outcome measures related to each component of the RE-AIM framework (Figure 2)—is currently underway. Results of the intervention and its impact on clinical care will be forthcoming. Future publications about the BWH/HSyE PSLL will report specifically on our outcome and process measures.
The PSLL vision of health systems collaborating with engineers to engage patients, families, and clinicians in strategies to eliminate harm in acute care hospital settings involves applying new approaches to adverse event prevention. A successful implementation of this magnitude requires a multi-disciplinary team that is willing to think outside of current work flows and processes to change health system culture. Applying user-centered design and systems engineering tools utilized in this PSLL has the potential to redesign the way health systems work.
FUNDING
The PSLL project is supported by a grant from the Agency for Healthcare Research and Quality (P30-HS023535). The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality. This funding agency was not involved in the design and conduct of the study; collection, management, analysis, and interpretation of the data; or preparation, review, or approval of the manuscript. All authors had full access to the data.
AUTHOR CONTRIBUTIONS
All authors have contributed sufficiently and meaningfully to the conception, design, and conduct of the study; data acquisition, analysis, and interpretation; and/or drafting, editing, and revising the manuscript.
SUPPLEMENTARY MATERIAL
Supplementary material is available at Journal of the American Medical Informatics Association online.
Supplementary Material
ACKNOWLEDGMENTS
We appreciate the efforts of numerous PSLL project investigators and research staff including: Lisa S Lehmann, Anthony Massaro, Stuart R Lipsitz, Frank Chang, Wai Yin Leung, Eli Mlaver, Brittany Couture, Megan Duckworth, Jenzel Espares, Kerrin Bersani, and Taylor Christiansen. We also appreciate the efforts and support of BWH leadership including Adam Landman, and the support of Partners Healthcare System application development led by Nina Plaks.
Conflict of Interest statement.
Dr Jeffrey Schnipper received funding from Mallinckrodt Pharmaceuticals to conduct an investigator-initiated study of opioid-related adverse drug events among hospitalized post-operative patients; and from Portola Pharmaceuticals to conduct an investigator-initiated study of hospitalized patients who decline administration of subcutaneous medications to prevent venous thromboembolism. All other authors report no conflicts of interest.
REFERENCES
- 1. Shekelle PG, Morton SC, Keeler EB.. Costs and benefits of health information technology. Evid Rep Technol Assess (Full Rep) 2006; (132): 1–71. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Wells S, Rozenblum R, Park A, et al. Organizational strategies for promoting patient and provider uptake of personal health records. J Am Med Inform Assoc 2015; 22 (1): 213–22. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Grando MA, Rozenblum R, Bates DW, eds. Information Technology for Patient Empowerment in Healthcare. 1st ed Berlin: Walter de Gruyter; 2015. [Google Scholar]
- 4. Rozenblum R, Miller P, Pearson D, et al. Patient-centered healthcare, patient engagement and health information technology: the perfect storm In: Grando MA, Rozenblum R, Bates DW, eds. Information Technology for Patient Empowerment in Healthcare. 1st ed.Berlin: Walter de Gruyter; 2015: 3–22. [Google Scholar]
- 5. Abott PA, Foster J, Marin Hde F, et al. Complexity and the science of implementation in health IT—knowledge gaps and future visions. Int J Med Inform 2014; 83 (7): e12–22. [DOI] [PubMed] [Google Scholar]
- 6. Dalal A, Fuller T, Garabedian P, et al. Systems engineering and human factors support of a system of novel EHR-integrated tools to prevent harm in the hospital. J Am Med Inform Assoc 2019; 26 (6): 553–60. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Dykes PC, Carroll DL, Hurley A, et al. Fall prevention in acute care hospitals: a randomized trial. JAMA 2010; 304 (17): 1912–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Dykes PC, Duckworth M, Cunningham S, et al. Pilot testing Fall TIPS (Tailoring Interventions for Patient Safety): a patient-centered fall prevention toolkit. Jt Comm J Qual Patient Saf 2017; 43 (8): 403–13. [DOI] [PubMed] [Google Scholar]
- 9. Collins SA, Couture B, Smith AD, et al. Mixed-methods evaluation of real-time safety reporting by hospitalized patients and their care partners: the mysafecare application. J Patient Saf 2018. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Couture B, Lilley E, Chang F, et al. Applying user-centered design methods to the development of an mHealth application for use in the hospital setting by patients and care partners. Appl Clin Inform 2018; 9 (2): 302–12. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Mlaver E, Schnipper JL, Boxer RB, et al. User-centered collaborative design and development of an inpatient safety dashboard. Jt Comm J Qual Patient Saf 2017; 43 (12): 676–85. [DOI] [PubMed] [Google Scholar]
- 12.Rozenblum R, Bates DW. The Patient-SatisfActive® Model: A Proactive Approach to Patient-Centered Care; 2010: USA. [Google Scholar]
- 13. Rozenblum R, Lisby M, Hockey PM, et al. The patient satisfaction chasm: the gap between hospital management and frontline clinicians. BMJ Qual Saf 2013; 22 (3): 242–50. [DOI] [PubMed] [Google Scholar]
- 14. Glasgow RE, Vogt TM, Boles SM.. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health 1999; 89 (9): 1322–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Bakken S, Ruland CM.. Translating clinical informatics interventions into routine clinical care: how can the RE-AIM framework help? J Am Med Inform Assoc 2009; 16 (6): 889–97. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Glasgow RE, McKay HG, Piette JD, et al. The RE-AIM framework for evaluating interventions: what can it tell us about approaches to chronic illness management? Patient Educ Couns 2001; 44 (2): 119–27. [DOI] [PubMed] [Google Scholar]
- 17.Department of Health and Human Services: Agency for Healthcare Research and Quality RFA for Patient Safety Learning Laboratories: Innovative Design and Development to Improve Healthcare Delivery Systems (P30); posted 2013: funding opportunity description specific objectives. https://grants.nih.gov/grants/guide/rfa-files/RFA-HS-14-005.html Accessed July 3, 2019.
- 18. Baxter G, Sommerville I.. Socio-technical systems: From design methods to systems engineering. Interact Comput 2011; 23 (1): 4–17. [Google Scholar]
- 19. Hemming K, Haines TP, Chilton PJ, et al. The stepped wedge cluster randomised trial: rationale, design, analysis, and reporting. BMJ 2015; 350 (feb06 1): h391. [DOI] [PubMed] [Google Scholar]
- 20. Collins S, Rozenblum R, Leung WY, et al. Acute care patient portals: a qualitative study of stakeholder perspectives on current practices. J Am Med Inform Assoc 2017; 24 (e1): e9–e17. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.