%0 Journal Article %@ 2292-9495 %I JMIR Publications %V 11 %N %P e52301 %T The Added Value of Using Video in Out-of-Hours Primary Care Telephone Triage Among General Practitioners: Cross-Sectional Survey Study %A Nebsbjerg,Mette Amalie %A Bomholt,Katrine Bjørnshave %A Vestergaard,Claus Høstrup %A Christensen,Morten Bondo %A Huibers,Linda %K primary health care %K after-hours care %K referral and consultation %K general practitioners %K triage %K remote consultation %K telemedicine %D 2024 %7 15.11.2024 %9 %J JMIR Hum Factors %G English %X Background: Many countries have introduced video consultations in primary care both inside and outside of office hours. Despite some relational and technical limitations, general practitioners (GPs) have reported the benefits of video use in the daytime as it provides faster and more flexible access to health care. Studies have indicated that video may be specifically valuable in out-of-hours primary care (OOH-PC), but additional information on the added value of video use is needed. Objective: This study aimed to investigate triage GPs’ perspectives on video use in GP-led telephone triage in OOH-PC by exploring their reasons for choosing video use and its effect on triage outcome, the decision-making process, communication, and invested time. Methods: We conducted a cross-sectional questionnaire study among GPs performing telephone triage in the OOH-PC service in the Central Denmark Region from September 5, 2022, until December 21, 2022. The questionnaire was integrated into the electronic patient registration system as a pop-up window appearing after every third video contact. This setup automatically linked background data on the contact, patient, and GP to the questionnaire data. We used descriptive analyses to describe reasons for and effects of video use and GP evaluation, stratified by patient age. Results: A total of 2456 questionnaires were completed. The most frequent reasons for video use were to assess the severity (n=1951, 79.4%), to increase the probability of self-care (n=1279, 52.1%), and to achieve greater certainty in decision-making (n=810, 33%) (multiple answers were possible for reasons of video use). In 61.9% (n=1516) of contacts, the triage GPs anticipated that the contact would have resulted in a different triage outcome if video had not been used. Use of video resulted in a downgrading of severity level in 88.3% (n=1338) of cases. Triage GPs evaluated the use of video as positive in terms of their decision-making process (n=2358, 96%), communication (n=2214, 90.1%), and invested time (n=2391, 97.3%). Conclusions: Triage GPs assessed that the use of video in telephone triage did affect their triage outcome, mostly by downgrading the level of care needed. The participating triage GPs found video in OOH-PC to be of added value, particularly in communication and the decision-making process. %R 10.2196/52301 %U https://humanfactors.jmir.org/2024/1/e52301 %U https://doi.org/10.2196/52301 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 11 %N %P e55667 %T Clinical Acceptability of a Quality Improvement Program for Reducing Cardiovascular Disease Risk in People With Chronic Kidney Disease in Australian General Practice: Qualitative Study %A McBride,Caroline %A Hunter,Barbara %A Lumsden,Natalie %A Somasundaram,Kaleswari %A McMorrow,Rita %A Boyle,Douglas %A Emery,Jon %A Nelson,Craig %A Manski-Nankervis,Jo-Anne %K clinical decision support %K general practice %K GP %K primary care %K family medicine %K general medicine %K family physician %K implementation science %K chronic kidney disease %K CKD %K nephrology %K nephrologist %K chronic disease %K cardiovascular risk %K cardiology %K quality improvement %K EHR %K electronic health record %K clinical software %D 2024 %7 13.11.2024 %9 %J JMIR Hum Factors %G English %X Background: Future Health Today (FHT) is a technology program that integrates with general practice clinical software to provide point of care (PoC) clinical decision support and a quality improvement dashboard. This qualitative study looks at the use of FHT in the context of cardiovascular disease risk in chronic kidney disease (CKD). Objective: This study aims to explore factors influencing clinical implementation of the FHT module focusing on cardiovascular risk in CKD, from the perspectives of participating general practitioner staff. Methods: Practices in Victoria were recruited to participate in a pragmatic cluster randomized controlled trial using FHT, of which 19 practices were randomly assigned to use FHT’s cardiovascular risk in CKD program. A total of 13 semistructured interviews were undertaken with a nominated general practitioner (n=7) or practice nurse (n=6) from 10 participating practices. Interview questions focused on the clinical usefulness of the tool and its place in clinical workflows. Qualitative data were coded by 2 researchers and analyzed using framework analysis and Clinical Performance Feedback Intervention Theory. Results: All 13 interviewees had used the FHT PoC tool, and feedback was largely positive. Overall, clinicians described engaging with the tool as a “prompt” or “reminder” system. Themes reflected that the tool’s goals and clinical content were aligned with clinician’s existing priorities and knowledge, and the tool’s design facilitated easy integration into existing workflows. The main barrier to implementation identified by 2 clinicians was notification fatigue. A total of 7 interviewees had used the FHT dashboard tool. The main barriers to use were its limited integration into clinical workflows, such that some participants did not know of its existence; clinicians’ competing clinical priorities; and limited time to learn and use the tool. Conclusions: This study identified many facilitators for the successful use of the FHT PoC program, in the context of cardiovascular risk in CKD, and barriers to the use of the dashboard program. This work will be used to inform the wider implementation of FHT, as well as the development of future modules of FHT for other risk or disease states. Trial Registration: Australian New Zealand Clinical Trial Registry ACTRN12620000993998; https://www.anzctr.org.au/Trial/Registration/TrialReview.aspx?id=380119&is %R 10.2196/55667 %U https://humanfactors.jmir.org/2024/1/e55667 %U https://doi.org/10.2196/55667 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e54022 %T Patterns and Perceptions of Standard Order Set Use Among Physicians Working Within a Multihospital System: Mixed Methods Study %A Naicker,Sundresan %A Tariq,Amina %A Donovan,Raelene %A Magon,Honor %A White,Nicole %A Simmons,Joshua %A McPhail,Steven M %+ Australian Centre for Health Services Innovation and Centre for Healthcare Transformation, School of Public Health and Social Work, Faculty of Health, Queensland University of Technology, 60 Musk Avenue, Kelvin Grove, 4059, Australia, 61 449876034, sundresan.naicker@qut.edu.au %K medical informatics %K adoption and implementation %K behavior %K health systems %K testing %K electronic medical records %K behavioral model %K quantitative data %K semistructured interview %K clinical practice %K user preference %K user %K user experience %D 2024 %7 8.11.2024 %9 Original Paper %J JMIR Form Res %G English %X Background: Electronic standard order sets automate the ordering of specific treatment, testing, and investigative protocols by physicians. These tools may help reduce unwarranted clinical variation and improve health care efficiency. Despite their routine implementation within electronic medical records (EMRs), little is understood about how they are used and what factors influence their adoption in practice. Objective: This study aims to (1) describe the patterns of use of standard order sets implemented in a widely used EMR (PowerPlans and Cerner Millennium) within a multihospital digital health care system; (2) explore the experiences and perceptions of implementers and users regarding the factors contributing to the use of these standard order sets; and (3) map these findings to the Capability, Opportunity, and Motivation Behavior (COM-B) model of behavior change to assist those planning to develop, improve, implement, and iterate the use of standard order sets in hospital settings. Methods: Quantitative data on standard order set usage were captured from 5 hospitals over 5-month intervals for 3 years (2019, 2020, and 2021). Qualitative data, comprising unstructured and semistructured interviews (n=15), were collected and analyzed using a reflexive thematic approach. Interview themes were then mapped to a theory-informed model of behavior change (COM-B) to identify determinants of standard order set usage in routine clinical practice. The COM-B model is an evidence-based, multicomponent framework that posits that human actions result from multiple contextual influences, which can be categorized across 3 dimensions: capability, opportunity, and motivation, all of which intersect. Results: The total count of standard order set usage across the health system during the 2019 observation period was 267,253, increasing to 293,950 in 2020 and 335,066 in 2021. There was a notable shift toward using specialty order sets that received upgrades during the study period. Four emergent themes related to order set use were derived from clinician interviews: (1) Knowledge and Skills; (2) Perceptions; (3) Technical Dependencies; and (4) Unintended Consequences, all of which were mapped to the COM-B model. Findings indicate a user preference for customized order sets that respond to local context and user experience. Conclusions: The study findings suggest that ongoing investment in the development and functionality of specialty order sets has the potential to enhance usage as these sets continue to be customized in response to local context and user experience. Sustained and continuous uptake of appropriate Computerized Provider Order Entry use may require implementation strategies that address the capability, opportunity, and motivational influencers of behavior. %M 39514274 %R 10.2196/54022 %U https://formative.jmir.org/2024/1/e54022 %U https://doi.org/10.2196/54022 %U http://www.ncbi.nlm.nih.gov/pubmed/39514274 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e55766 %T Health Care Professionals’ Experience of Using AI: Systematic Review With Narrative Synthesis %A Ayorinde,Abimbola %A Mensah,Daniel Opoku %A Walsh,Julia %A Ghosh,Iman %A Ibrahim,Siti Aishah %A Hogg,Jeffry %A Peek,Niels %A Griffiths,Frances %+ Division of Health Sciences, Warwick Medical School, University of Warwick, Medical School Building, Gibbet Hill Road, Coventry, CV4 7AL, United Kingdom, 44 2476151098, a.ayorinde.1@warwick.ac.uk %K artificial intelligence %K clinical decision support systems %K CDSS %K decision-making %K quality assessment %K clinician experience %K health care professionals %K health care delivery %D 2024 %7 30.10.2024 %9 Review %J J Med Internet Res %G English %X Background: There has been a substantial increase in the development of artificial intelligence (AI) tools for clinical decision support. Historically, these were mostly knowledge-based systems, but recent advances include non–knowledge-based systems using some form of machine learning. The ability of health care professionals to trust technology and understand how it benefits patients or improves care delivery is known to be important for their adoption of that technology. For non–knowledge-based AI tools for clinical decision support, these issues are poorly understood. Objective: The aim of this study is to qualitatively synthesize evidence on the experiences of health care professionals in routinely using non–knowledge-based AI tools to support their clinical decision-making. Methods: In June 2023, we searched 4 electronic databases, MEDLINE, Embase, CINAHL, and Web of Science, with no language or date limit. We also contacted relevant experts and searched reference lists of the included studies. We included studies of any design that reported the experiences of health care professionals using non–knowledge-based systems for clinical decision support in their work settings. We completed double independent quality assessment for all included studies using the Mixed Methods Appraisal Tool. We used a theoretically informed thematic approach to synthesize the findings. Results: After screening 7552 titles and 182 full-text articles, we included 25 studies conducted in 9 different countries. Most of the included studies were qualitative (n=13), and the remaining were quantitative (n=9) and mixed methods (n=3). Overall, we identified 7 themes: health care professionals’ understanding of AI applications, level of trust and confidence in AI tools, judging the value added by AI, data availability and limitations of AI, time and competing priorities, concern about governance, and collaboration to facilitate the implementation and use of AI. The most frequently occurring are the first 3 themes. For example, many studies reported that health care professionals were concerned about not understanding the AI outputs or the rationale behind them. There were issues with confidence in the accuracy of the AI applications and their recommendations. Some health care professionals believed that AI provided added value and improved decision-making, and some reported that it only served as a confirmation of their clinical judgment, while others did not find it useful at all. Conclusions: Our review identified several important issues documented in various studies on health care professionals’ use of AI tools in real-world health care settings. Opinions of health care professionals regarding the added value of AI tools for supporting clinical decision-making varied widely, and many professionals had concerns about their understanding of and trust in this technology. The findings of this review emphasize the need for concerted efforts to optimize the integration of AI tools in real-world health care settings. Trial Registration: PROSPERO CRD42022336359; https://tinyurl.com/2yunvkmb %M 39476382 %R 10.2196/55766 %U https://www.jmir.org/2024/1/e55766 %U https://doi.org/10.2196/55766 %U http://www.ncbi.nlm.nih.gov/pubmed/39476382 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e60306 %T Enhancing Clinicians’ Use of Electronic Patient-Reported Outcome Measures in Outpatient Care: Mixed Methods Study %A van Engen,Veerle %A Bonfrer,Igna %A Ahaus,Kees %A Den Hollander-Ardon,Monique %A Peters,Ingrid %A Buljac-Samardzic,Martina %+ Erasmus School of Health Policy & Management, Erasmus University Rotterdam, Burgemeester Oudlaan 50, Rotterdam, 3062 PA, Netherlands, 31 010 408 8555, vanengen@eshpm.eur.nl %K patient-reported outcome measure %K value-based health care %K implementation %K clinician %K behavior %K barrier %K facilitator %K strategies %K professionalization %K mixed methods %D 2024 %7 18.10.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: Despite the increasing use of patient-reported outcome measures (PROMs) for collecting self-reported data among hospital outpatients, clinicians’ use of these data remains suboptimal. Insight into this issue and strategies to enhance the use of PROMs are critical but limited. Objective: This study aimed to examine clinicians’ use of PROM data for value-based outpatient consultations and identify efforts to enhance their use of PROMs in a Dutch university hospital. First, we aimed to investigate clinicians’ use of outpatients’ PROM data in 2023, focusing on adoption, implementation, and maintenance. Second, we aimed to develop insights into the organizational-level strategies implemented to enhance clinicians’ use of PROM data from 2020 to 2023. This included understanding the underlying rationales for these strategies and identifying strategies that appeared to be missing to address barriers or leverage facilitators. Third, we aimed to explore the key factors driving and constraining clinicians’ use of PROMs in 2023. Methods: We integrated data from 4 sources: 1-year performance data on clinicians’ use of PROMs (n=70 subdepartments), internal hospital documents from a central support team (n=56), a survey among clinicians (n=47), and interviews with individuals contributing to the organizational-level implementation of PROMs (n=20). The Reach, Effectiveness, Adoption, Implementation, and Maintenance framework was used to analyze clinicians’ adoption, implementation, and maintenance of PROMs. Strategies were analyzed using the Expert Recommendations for Implementing Change taxonomy, and results were structured around the constructs of capability, opportunity, and motivation. Results: On average, around 2023, clinicians accessed PROM data for approximately 3 of 20 (14%) patients during their outpatient consultation, despite numerous strategies to improve this practice. We identified issues in adoption, implementation, and maintenance. The hospital’s strategies, shaped organically and pragmatically, were related to 27 (37%) out of 73 Expert Recommendations for Implementing Change strategies. These strategies focused on enhancing clinicians’ capability, opportunity, and motivation. We found shortcomings in the quality of execution and completeness of strategies in relation to addressing all barriers and leveraging facilitators. We identified variations in the factors influencing the use of PROMs among frequent PROM users, occasional users, and nonusers. Challenges to effective facilitation were apparent, with certain desired strategies being unfeasible or impeded. Conclusions: Enhancing clinicians’ use of PROMs has remained challenging despite various strategies aimed at improving their capability, opportunity, and motivation. The use of PROMs may require more substantial changes than initially expected, necessitating a shift in clinicians’ professional attitudes and practices. Hospitals can facilitate rather than manage clinicians’ genuine use of PROMs. They must prioritize efforts to engage clinicians with PROMs for value-based outpatient care. Specific attention to their professionalization may be warranted. Tailored strategies, designed to address within-group differences in clinicians’ needs and motivation, hold promise for future efforts. %M 39422999 %R 10.2196/60306 %U https://www.jmir.org/2024/1/e60306 %U https://doi.org/10.2196/60306 %U http://www.ncbi.nlm.nih.gov/pubmed/39422999 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e58144 %T Co-Designing Remote Patient Monitoring Technologies for Inpatients: Systematic Review %A Sumner,Jennifer %A Tan,Si Ying %A Wang,Yuchen %A Keck,Camille Hui Sze %A Xin Lee,Eunice Wei %A Chew,Emily Hwee Hoon %A Yip,Alexander Wenjun %+ Medical Affairs-Research, Alexandra Hospital, 378 Alexandra Road, Singapore, 159964, Singapore, 65 6908 2222, jennyssumner@gmail.com %K remote patient monitoring %K technology %K inpatient %K care transition %K systematic review %K health technology %K patient-centeredness %K technology use %K effectiveness %K study design %K assessment %K pilot testing %K health care %K technologies %K terminology %K quality and consistency %K telehealth %K telemonitoring %D 2024 %7 15.10.2024 %9 Review %J J Med Internet Res %G English %X Background: The co-design of health technology enables patient-centeredness and can help reduce barriers to technology use. Objective: The study objectives were to identify what remote patient monitoring (RPM) technology has been co-designed for inpatients and how effective it is, to identify and describe the co-design approaches used to develop RPM technologies and in which contexts they emerge, and to identify and describe barriers and facilitators of the co-design process. Methods: We conducted a systematic review of co-designed RPM technologies for inpatients or for the immediate postdischarge period and assessed (1) their effectiveness in improving health outcomes, (2) the co-design approaches used, and (3) barriers and facilitators to the co-design process. Eligible records included those involving stakeholders co-designing RPM technology for use in the inpatient setting or during the immediate postdischarge period. Searches were limited to the English language within the last 10 years. We searched MEDLINE, Embase, CINAHL, PsycInfo, and Science Citation Index (Web of Science) in April 2023. We used the Joanna Briggs Institute critical appraisal checklist for quasi-experimental studies and qualitative research. Findings are presented narratively. Results: We screened 3334 reports, and 17 projects met the eligibility criteria. Interventions were designed for pre- and postsurgical monitoring (n=6), intensive care monitoring (n=2), posttransplant monitoring (n=3), rehabilitation (n=4), acute inpatients (n=1), and postpartum care (n=1). No projects evaluated the efficacy of their co-designed RPM technology. Three pilot studies reported clinical outcomes; their risk of bias was low to moderate. Pilot evaluations (11/17) also focused on nonclinical outcomes such as usability, usefulness, feasibility, and satisfaction. Common co-design approaches included needs assessment or ideation (16/17), prototyping (15/17), and pilot testing (11/17). The most commonly reported challenge to the co-design process was the generalizability of findings, closely followed by time and resource constraints and participant bias. Stakeholders’ perceived value was the most frequently reported enabler of co-design. Other enablers included continued stakeholder engagement and methodological factors (ie, the use of flexible mixed method approaches and prototyping). Conclusions: Co-design methods can help enhance interventions’ relevance, usability, and adoption. While included studies measured usability, satisfaction, and acceptability—critical factors for successful implementation and uptake—we could not determine the clinical effectiveness of co-designed RPM technologies. A stronger commitment to clinical evaluation is needed. Studies’ use of diverse co-design approaches can foster stakeholder inclusivity, but greater standardization in co-design terminology is needed to improve the quality and consistency of co-design research. %M 39405106 %R 10.2196/58144 %U https://www.jmir.org/2024/1/e58144 %U https://doi.org/10.2196/58144 %U http://www.ncbi.nlm.nih.gov/pubmed/39405106 %0 Journal Article %@ 2563-6316 %I JMIR Publications %V 5 %N %P e55903 %T Development and Assessment of a Point-of-Care Application (Genomic Medicine Guidance) for Heritable Thoracic Aortic Disease %A Patil,Rohan %A Ashraf,Fatima %A Abu Dayeh,Samer %A Prakash,Siddharth K %K genomic medicine %K point of care %K thoracic aortic aneurysm %K aortic dissection %K decision support %D 2024 %7 8.10.2024 %9 %J JMIRx Med %G English %X Background: Genetic testing can determine familial and personal risks for heritable thoracic aortic aneurysms and dissections (TAD). The 2022 American College of Cardiology/American Heart Association guidelines for TAD recommend management decisions based on the specific gene mutation. However, many clinicians lack sufficient comfort or insight to integrate genetic information into clinical practice. Objective: We therefore developed the Genomic Medicine Guidance (GMG) application, an interactive point-of-care tool to inform clinicians and patients about TAD diagnosis, treatment, and surveillance. GMG is a REDCap-based application that combines publicly available genetic data and clinical recommendations based on the TAD guidelines into one translational education tool. Methods: TAD genetic information in GMG was sourced from the Montalcino Aortic Consortium, a worldwide collaboration of TAD centers of excellence, and the National Institutes of Health genetic repositories ClinVar and ClinGen. Results: The application streamlines data on the 13 most frequently mutated TAD genes with 2286 unique pathogenic mutations that cause TAD so that users receive comprehensive recommendations for diagnostic testing, imaging, surveillance, medical therapy, and preventative surgical repair, as well as guidance for exercise safety and management during pregnancy. The application output can be displayed in a clinician view or exported as an informative pamphlet in a patient-friendly format. Conclusions: The overall goal of the GMG application is to make genomic medicine more accessible to clinicians and patients while serving as a unifying platform for research. We anticipate that these features will be catalysts for collaborative projects aiming to understand the spectrum of genetic variants contributing to TAD. %R 10.2196/55903 %U https://xmed.jmir.org/2024/1/e55903 %U https://doi.org/10.2196/55903 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e55267 %T A Collection of Components to Design Clinical Dashboards Incorporating Patient-Reported Outcome Measures: Qualitative Study %A Bischof,Anja Yvonne %A Kuklinski,David %A Salvi,Irene %A Walker,Carla %A Vogel,Justus %A Geissler,Alexander %+ Chair of Health Economics, Policy and Management, School of Medicine, University of St. Gallen, St. Jakob-Strasse 21, St. Gallen, 9000, Switzerland, 41 712243220, anja.bischof@unisg.ch %K clinical dashboards %K design components %K patient-reported outcome measures (PROMs) %K patient-reported outcomes (PROs) %D 2024 %7 2.10.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: A clinical dashboard is a data-driven clinical decision support tool visualizing multiple key performance indicators in a single report while minimizing time and effort for data gathering. Studies have shown that including patient-reported outcome measures (PROMs) in clinical dashboards supports the clinician’s understanding of how treatments impact patients’ health status, helps identify changes in health-related quality of life at an early stage, and strengthens patient-physician communication. Objective: This study aims to determine design components for clinical dashboards incorporating PROMs to inform software producers and users (ie, physicians). Methods: We conducted interviews with software producers and users to test preselected design components. Furthermore, the interviews allowed us to derive additional components that are not outlined in existing literature. Finally, we used inductive and deductive coding to derive a guide on which design components need to be considered when building a clinical dashboard incorporating PROMs. Results: A total of 25 design components were identified, of which 16 were already surfaced during the literature search. Furthermore, 9 additional components were derived inductively during our interviews. The design components are clustered in a generic dashboard, PROM-related, adjacent information, and requirements for adoption components. Both software producers and users agreed on the primary purpose of a clinical dashboard incorporating PROMs to enhance patient communication in outpatient settings. Dashboard benefits include enhanced data visualization and improved workflow efficiency, while interoperability and data collection were named as adoption challenges. Consistency in dashboard design components is preferred across different episodes of care, with adaptations only for disease-specific PROMs. Conclusions: Clinical dashboards have the potential to facilitate informed treatment decisions if certain design components are followed. This study establishes a comprehensive framework of design components to guide the development of effective clinical dashboards incorporating PROMs in health care practice. %M 39357042 %R 10.2196/55267 %U https://www.jmir.org/2024/1/e55267 %U https://doi.org/10.2196/55267 %U http://www.ncbi.nlm.nih.gov/pubmed/39357042 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e55164 %T Usability, Engagement, and Report Usefulness of Chatbot-Based Family Health History Data Collection: Mixed Methods Analysis %A Nguyen,Michelle Hoang %A Sedoc,João %A Taylor,Casey Overby %+ Institute for Computational Medicine, Johns Hopkins University, Hackerman Hall, Baltimore, MD, 21218, United States, 1 (410) 516 4116, mnguye79@jhmi.edu %K family health history %K chatbots %K conversational agents %K digital health tools %K usability %K engagement %K report usefulness %K evaluation %K crowdsourcing %K mixed methods %D 2024 %7 30.9.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: Family health history (FHx) is an important predictor of a person’s genetic risk but is not collected by many adults in the United States. Objective: This study aims to test and compare the usability, engagement, and report usefulness of 2 web-based methods to collect FHx. Methods: This mixed methods study compared FHx data collection using a flow-based chatbot (KIT; the curious interactive test) and a form-based method. KIT’s design was optimized to reduce user burden. We recruited and randomized individuals from 2 crowdsourced platforms to 1 of the 2 FHx methods. All participants were asked to complete a questionnaire to assess the method’s usability, the usefulness of a report summarizing their experience, user-desired chatbot enhancements, and general user experience. Engagement was studied using log data collected by the methods. We used qualitative findings from analyzing free-text comments to supplement the primary quantitative results. Results: Participants randomized to KIT reported higher usability than those randomized to the form, with a mean System Usability Scale score of 80.2 versus 61.9 (P<.001), respectively. The engagement analysis reflected design differences in the onboarding process. KIT users spent less time entering FHx information and reported more conditions than form users (mean 5.90 vs 7.97 min; P=.04; and mean 7.8 vs 10.1 conditions; P=.04). Both KIT and form users somewhat agreed that the report was useful (Likert scale ratings of 4.08 and 4.29, respectively). Among desired enhancements, personalization was the highest-rated feature (188/205, 91.7% rated medium- to high-priority). Qualitative analyses revealed positive and negative characteristics of both KIT and the form-based method. Among respondents randomized to KIT, most indicated it was easy to use and navigate and that they could respond to and understand user prompts. Negative comments addressed KIT’s personality, conversational pace, and ability to manage errors. For KIT and form respondents, qualitative results revealed common themes, including a desire for more information about conditions and a mutual appreciation for the multiple-choice button response format. Respondents also said they wanted to report health information beyond KIT’s prompts (eg, personal health history) and for KIT to provide more personalized responses. Conclusions: We showed that KIT provided a usable way to collect FHx. We also identified design considerations to improve chatbot-based FHx data collection: First, the final report summarizing the FHx collection experience should be enhanced to provide more value for patients. Second, the onboarding chatbot prompt may impact data quality and should be carefully considered. Finally, we highlighted several areas that could be improved by moving from a flow-based chatbot to a large language model implementation strategy. %M 39348188 %R 10.2196/55164 %U https://www.jmir.org/2024/1/e55164 %U https://doi.org/10.2196/55164 %U http://www.ncbi.nlm.nih.gov/pubmed/39348188 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 11 %N %P e56872 %T User Requirements for an Electronic Patient Recruitment System: Semistructured Interview Analysis After First Implementation in 3 German University Hospitals %A Stein,Alexandra %A Blasini,Romina %A Strantz,Cosima %A Fitzer,Kai %A Gulden,Christian %A Leddig,Torsten %A Hoffmann,Wolfgang %+ Institute for Community Medicine, Section Epidemiology of Health Care and Community Health, University Medicine Greifswald, Ellernholzstr 1-2, Greifswald, 17487, Germany, 49 383486 ext 22708, alexandra.stein@med.uni-greifswald.de %K patient recruitment system %K clinical trial recruitment support system %K clinical trials %K recruit %K recruitment %K recruiting %K participant %K participants %K research %K digital health %K usability %K interview %K interviews %K qualitative %K experience %K experiences %K attitude %K attitudes %K opinion %K perception %K perceptions %K perspective %K perspectives %K database %K databases %K information system %K information systems %K search %K searches %K searching %K retrieval %D 2024 %7 27.9.2024 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Clinical trials are essential for medical research and medical progress. Nevertheless, trials often fail to reach their recruitment goals. Patient recruitment systems aim to support clinical trials by providing an automated search for eligible patients in the databases of health care institutions like university hospitals. To integrate patient recruitment systems into existing workflows, previous works have assessed user requirements for these tools. In this study, we tested patient recruitment systems KAS+ and recruIT as part of the MIRACUM (Medical Informatics in Research and Care in University Medicine) project. Objective: Our goal was to investigate whether and to what extent the 2 different evaluated tools can meet the requirements resulting from the first requirements analysis, which was performed in 2018-2019. A user survey was conducted to determine whether the tools are usable in practice and helpful for the trial staff. Furthermore, we investigated whether the test phase revealed further requirements for recruitment tools that were not considered in the first place. Methods: We performed semistructured interviews with 10 participants in 3 German university hospitals who used the patient recruitment tools KAS+ or recruIT for at least 1 month with currently recruiting trials. Thereafter, the interviews were transcribed and analyzed by Meyring method. The identified statements of the interviewees were categorized into 5 groups of requirements and sorted by their frequency. Results: The evaluated recruIT and KAS+ tools fulfilled 7 and 11 requirements of the 12 previously identified requirements, respectively. The interviewed participants mentioned the need for different notification schedules, integration into their workflow, different patient characteristics, and pseudonymized screening lists. This resulted in a list of new requirements for the implementation or enhancement of patient recruitment systems. Conclusions: Trial staff report a huge need of support for the identification of eligible trial participants. Moreover, the workflows in patient recruitment differ across trials. For better suitability of the recruitment systems in the workflow of different kinds of trials, we recommend the implementation of an adjustable notification schedule for screening lists, a detailed workflow analysis, broad patient filtering options, and the display of all information needed to identify the persons on the list. Despite criticisms, all participants confirmed to use the patient recruitment systems again. %M 39331958 %R 10.2196/56872 %U https://humanfactors.jmir.org/2024/1/e56872 %U https://doi.org/10.2196/56872 %U http://www.ncbi.nlm.nih.gov/pubmed/39331958 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 11 %N %P e55099 %T Triage Accuracy and the Safety of User-Initiated Symptom Assessment With an Electronic Symptom Checker in a Real-Life Setting: Instrument Validation Study %A Liu,Ville %A Kaila,Minna %A Koskela,Tuomas %+ Faculty of Medicine, University of Helsinki, Ruusulankatu 21 B 32, Helsinki, 00250, Finland, 358 400642104, villeliu@hotmail.com %K nurse triage %K emergency department triage %K triage %K symptom assessment %K health services accessibility %K telemedicine %K eHealth %K remote consultation %K eHealth %K primary health care %K primary care %K urgent care %K health services research %K health services %D 2024 %7 26.9.2024 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Previous studies have evaluated the accuracy of the diagnostics of electronic symptom checkers (ESCs) and triage using clinical case vignettes. National Omaolo digital services (Omaolo) in Finland consist of an ESC for various symptoms. Omaolo is a medical device with a Conformité Européenne marking (risk class: IIa), based on Duodecim Clinical Decision Support, EBMEDS. Objective: This study investigates how well triage performed by the ESC nurse triage within the chief symptom list available in Omaolo (anal region symptoms, cough, diarrhea, discharge from the eye or watery or reddish eye, headache, heartburn, knee symptom or injury, lower back pain or injury, oral health, painful or blocked ear, respiratory tract infection, sexually transmitted disease, shoulder pain or stiffness or injury, sore throat or throat symptom, and urinary tract infection). In addition, the accuracy, specificity, sensitivity, and safety of the Omaolo ESC were assessed. Methods: This is a clinical validation study in a real-life setting performed at multiple primary health care (PHC) centers across Finland. The included units were of the walk-in model of primary care, where no previous phone call or contact was required. Upon arriving at the PHC center, users (patients) answered the ESC questions and received a triage recommendation; a nurse then assessed their triage. Findings on 877 patients were analyzed by matching the ESC recommendations with triage by the triage nurse. Results: Safe assessments by the ESC accounted for 97.6% (856/877; 95% CI 95.6%-98.0%) of all assessments made. The mean of the exact match for all symptom assessments was 53.7% (471/877; 95% CI 49.2%-55.9%). The mean value of the exact match or overly conservative but suitable for all (ESC’s assessment was 1 triage level higher than the nurse’s triage) symptom assessments was 66.6% (584/877; 95% CI 63.4%-69.7%). When the nurse concluded that urgent treatment was needed, the ESC’s exactly matched accuracy was 70.9% (244/344; 95% CI 65.8%-75.7%). Sensitivity for the Omaolo ESC was 62.6% and specificity of 69.2%. A total of 21 critical assessments were identified for further analysis: there was no indication of compromised patient safety. Conclusions: The primary objectives of this study were to evaluate the safety and to explore the accuracy, specificity, and sensitivity of the Omaolo ESC. The results indicate that the ESC is safe in a real-life setting when appraised with assessments conducted by triage nurses. Furthermore, the Omaolo ESC exhibits the potential to guide patients to appropriate triage destinations effectively, helping them to receive timely and suitable care. International Registered Report Identifier (IRRID): RR2-10.2196/41423 %M 39326038 %R 10.2196/55099 %U https://humanfactors.jmir.org/2024/1/e55099 %U https://doi.org/10.2196/55099 %U http://www.ncbi.nlm.nih.gov/pubmed/39326038 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e57633 %T A Clinical Decision Support Tool for Intimate Partner Violence Screening Among Women Veterans: Development and Qualitative Evaluation of Provider Perspectives %A Rossi,Fernanda S %A Wu,Justina %A Timko,Christine %A Nevedal,Andrea L %A Wiltsey Stirman,Shannon %+ Department of Psychiatry and Behavioral Sciences, Stanford University School of Medicine, 1070 Arastradero Rd, Stanford, CA, 94304, United States, 1 650 721 3990, fsrossi@stanford.edu %K intimate partner violence %K clinical decision support %K intimate partner violence screening %K women veterans %D 2024 %7 25.9.2024 %9 Original Paper %J JMIR Form Res %G English %X Background: Women veterans, compared to civilian women, are especially at risk of experiencing intimate partner violence (IPV), pointing to the critical need for IPV screening and intervention in the Veterans Health Administration (VHA). However, implementing paper-based IPV screening and intervention in the VHA has revealed substantial barriers, including health care providers’ inadequate IPV training, competing demands, time constraints, and discomfort addressing IPV and making decisions about the appropriate type or level of intervention. Objective: This study aimed to address IPV screening implementation barriers and hence developed and tested a novel IPV clinical decision support (CDS) tool for physicians in the Women’s Health Clinic (WHC), a primary care clinic within the Veterans Affairs Palo Alto Health Care System. This tool provides intelligent, evidence-based, step-by-step guidance on how to conduct IPV screening and intervention. Methods: Informed by existing CDS development frameworks, developing the IPV CDS tool prototype involved six steps: (1) identifying the scope of the tool, (2) identifying IPV screening and intervention content, (3) incorporating IPV-related VHA and clinic resources, (4) identifying the tool’s components, (5) designing the tool, and (6) conducting initial tool revisions. We obtained preliminary physician feedback on user experience and clinical utility of the CDS tool via the System Usability Scale (SUS) and semistructured interviews with 6 WHC physicians. SUS scores were examined using descriptive statistics. Interviews were analyzed using rapid qualitative analysis to extract actionable feedback to inform design updates and improvements. Results: This study includes a detailed description of the IPV CDS tool. Findings indicated that the tool was generally well received by physicians, who indicated good tool usability (SUS score: mean 77.5, SD 12.75). They found the tool clinically useful, needed in their practice, and feasible to implement in primary care. They emphasized that it increased their confidence in managing patients reporting IPV but expressed concerns regarding its length, workflow integration, flexibility, and specificity of information. Several physicians, for example, found the tool too time consuming when encountering patients at high risk; they suggested multiple uses of the tool (eg, an educational tool for less-experienced health care providers and a checklist for more-experienced health care providers) and including more detailed information (eg, a list of local shelters). Conclusions: Physician feedback on the IPV CDS tool is encouraging and will be used to improve the tool. This study offers an example of an IPV CDS tool that clinics can adapt to potentially enhance the quality and efficiency of their IPV screening and intervention process. Additional research is needed to determine the tool’s clinical utility in improving IPV screening and intervention rates and patient outcomes (eg, increased patient safety, reduced IPV risk, and increased referrals to mental health treatment). %M 39321455 %R 10.2196/57633 %U https://formative.jmir.org/2024/1/e57633 %U https://doi.org/10.2196/57633 %U http://www.ncbi.nlm.nih.gov/pubmed/39321455 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 11 %N %P e57984 %T User-Centered Design and Usability of Voxe as a Pediatric Electronic Patient-Reported Outcome Measure Platform: Mixed Methods Evaluation Study %A Anthony,Samantha J %A Pol,Sarah J %A Selkirk,Enid K %A Matthiesen,Amarens %A Klaassen,Robert J %A Manase,Dorin %A Silva,Amanda %A Barwick,Melanie %A Stinson,Jennifer N %A Damer,Alameen %A Ayibiowu,Mowa %A Dong,Selina X %A Oreskovich,Stephan %A Brudno,Michael %+ Child Health Evaluative Sciences, Research Institute, Peter Gilgan Centre for Research and Learning, The Hospital for Sick Children, 686 Bay Street, Toronto, ON, M5G 0A4, Canada, 1 416 813 7654 ext 303126, samantha.anthony@sickkids.ca %K eHealth %K end user engagement %K mobile phone %K patient-reported outcome %K patient-reported outcome measures %K pediatric %K user-centered design %D 2024 %7 19.9.2024 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Electronic patient-reported outcome measures (ePROMs) are standardized digital instruments integrated into clinical care to collect subjective data regarding patients’ health-related quality of life, functional status, and symptoms. In documenting patient-reported progress, ePROMs can guide treatment decisions and encourage measurement-based care practices. Voxe is a pediatric and user-centered ePROM platform for patients with chronic health conditions. Objective: We aimed to describe the user-centered design approach involving feedback from end users and usability testing of Voxe’s platform features to support implementation in a pediatric health care setting. Methods: Purposive sampling was used to recruit patients aged 8-17 years from 2 chronic illness populations in 2 pediatric hospitals in Canada. Patients’ health care team members were also purposively recruited. One-on-one iterative testing sessions were conducted digitally by research team members with participants to obtain feedback on the appearance and functionalities of the Voxe platform prototype. Patients and health care providers (HCPs) completed Voxe-related task-based activities. International Organization for Standardization key performance indicators were tracked during HCP task-based activities. HCPs also completed the System Usability Scale. To test platform usability, the think-aloud technique was used by participants during the completion of structured tasks. After completing all task-based activities, patient participants selected 5 words from the Microsoft Desirability Toolkit to describe their overall impression and experience with the Voxe platform. Qualitative data about likes, dislikes, and ease of use were collected through semistructured interviews. Feedback testing sessions were conducted with patients and HCPs until Voxe was acceptable to participating end users, with no further refinements identified. Quantitative and qualitative data analysis were completed using descriptive statistics and content analysis. Results: A total of 49 patients and 38 HCPs were recruited. Patients were positive about Voxe’s child-centered design characteristics and notification settings. HCPs rated Voxe as user-friendly and efficient, with the time to complete tasks decreasing over time. HCPs were satisfied with the Voxe platform functionalities and identified the value of Voxe’s system notifications, summarized display of ePROM results, and its capacity to integrate with electronic medical records. Patients’ and HCPs’ high satisfaction rates with the Voxe prototype highlight the importance of being responsive to user suggestions from the inception of eHealth platform developments to ensure their efficient and effective design. Conclusions: This paper describes the user-centered creation and usability testing of Voxe as an ePROM platform for implementation into clinical care for pediatric patients with chronic health conditions. As a patient-facing platform that can be integrated into electronic medical records, Voxe aligns with measurement-based care practices to foster quality patient-centered approaches to care. End users’ positive feedback and evaluation of the platform’s user-friendliness and efficiency suggest that Voxe represents a valuable and promising solution to systematically integrate patient-related outcome (PRO) data into complex and dynamic clinical health care settings. International Registered Report Identifier (IRRID): RR2-10.1136/bmjopen-2021-053119 %M 39298749 %R 10.2196/57984 %U https://humanfactors.jmir.org/2024/1/e57984 %U https://doi.org/10.2196/57984 %U http://www.ncbi.nlm.nih.gov/pubmed/39298749 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e56591 %T Acceptance of Electronic Labeling for Medicinal Product Information Among Malaysian Hospital Patients: Cross-Sectional Study %A Loh,Xin Yee %A Woo,Ai Ling %A Haris,Azwa %A Pereira,Cheryl Shajini %A Tan,Bee Kim %+ School of Pharmacy, Faculty of Health and Medical Sciences, Taylor's University, 1 Jalan Taylors, Subang Jaya, 47500, Malaysia, 60 3 5629 5000, beekim.tan@taylors.edu.my %K electronic labeling %K e-labeling %K electronic medication information %K patient preference %K acceptance %K hospital ambulatory care patient %D 2024 %7 18.9.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: While perceptions of electronic labeling (e-labeling) in developed countries have been generally positive, existing data primarily come from studies involving hospital pharmacists, community pharmacy customers who may not be frequent medication users, and individuals receiving COVID-19 vaccines. Objective: This study aims to assess e-labeling acceptance, perceptions of its benefits, challenges with its implementation, and preferences among hospital ambulatory care patients in Malaysia. Additionally, the study investigates the factors influencing patients’ acceptance of e-labeling. Methods: A cross-sectional study using a 28-item questionnaire was conducted at the outpatient pharmacy department of a quaternary hospital in Kuala Lumpur, Malaysia, from May to June 2023. The questionnaire was developed based on a review of published literature related to e-labeling and was guided by the Unified Theory of Acceptance and Use of Technology, second version (UTAUT2). Patients aged 18 years and above were recruited using a stratified sampling method to ensure representative age-related medication usage. A mobile tablet was provided to patients for self-completion of the e-survey in their preferred language (English, Malay, or Mandarin). Categorical data on e-labeling acceptance, perceptions, and preferences were analyzed using descriptive statistics. Qualitative content analysis was performed to characterize participants’ responses to open-ended questions. Univariate and multivariate binomial logistic regression analyses were conducted to identify predictors of e-labeling acceptance. Results: Out of 462 patients approached, 387 (83.8%) participated in the survey, with 283 (73.1%) accepting e-labeling. Most participants perceived the electronic version of the package insert as beneficial, particularly for understanding their medication better through the choice of language (352/387, 91.0%). However, around half of the participants (197/387, 50.9%) expressed concerns about the potential risks of obtaining illegal medication information via e-labeling. Most participants (302/387, 78.0%) preferred to access electronic leaflets through government websites. However, 221/387 (57.1%) still wanted the option to request printed leaflets. Significant predictors of e-labeling acceptance included perceived benefits such as better understanding of medication (adjusted odds ratio [AOR] 8.02, 95% CI 2.80-22.97, P<.001), environmental protection (AOR 7.24, 95% CI 3.00-17.51, P<.001), and flexibility in information retrieval (AOR 2.66, 95% CI 1.11-6.35, P=.03). Conversely, being of Chinese ethnicity compared with Malay (AOR 0.28, 95% CI 0.13-0.60, P=.005) and perceived lack of self-efficacy in browsing electronic leaflets (AOR 0.25, 95% CI 0.11-0.56, P<.001) were associated with lower acceptance. Conclusions: The acceptance rate for e-labeling among hospital ambulatory care patients was moderately high and was significantly influenced by ethnicity as well as patients’ perceived benefits and challenges related to its implementation. Future strategies to enhance e-labeling uptake should address patient concerns regarding the challenges of using the digital platform and emphasize the benefits of e-labeling. %M 39293048 %R 10.2196/56591 %U https://www.jmir.org/2024/1/e56591 %U https://doi.org/10.2196/56591 %U http://www.ncbi.nlm.nih.gov/pubmed/39293048 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e56370 %T A Behavior-Based Model to Validate Electronic Systems Designed to Collect Patient-Reported Outcomes: Model Development and Application %A Attamimi,Sultan %A Marshman,Zoe %A Deery,Christopher %A Radley,Stephen %A Gilchrist,Fiona %+ Academic Unit of Oral Health Dentistry and Society, University of Sheffield, 19 Claremont Cres, Broomhall, Sheffield, S10 2TA, United Kingdom, 44 0114 2717990, su.altamimi@uoh.edu.sa %K patient-reported outcome %K PRO %K electronic PRO %K user acceptance testing %K system validation %K patient-reported outcomes %K electronic PROs %K user acceptance %K validation model %K paediatric dentistry %D 2024 %7 17.9.2024 %9 Original Paper %J JMIR Form Res %G English %X Background: The merits of technology have been adopted in capturing patient-reported outcomes (PROs) by incorporating PROs into electronic systems. Following the development of an electronic system, evaluation of system performance is crucial to ensuring the collection of meaningful data. In contemporary PRO literature, electronic system validation is overlooked, and evidence on validation methods is lacking. Objective: This study aims to introduce a generalized concept to guide electronic patient-reported outcome (ePRO) providers in planning for system-specific validation methods. Methods: Since electronic systems are essentially products of software engineering endeavors, electronic systems used to collect PRO should be viewed from a computer science perspective with consideration to the health care environment. On this basis, a testing model was blueprinted and applied to a newly developed ePRO system designed for clinical use in pediatric dentistry (electronic Personal Assessment Questionnaire-Paediatric Dentistry) to investigate its thoroughness. Results: A behavior-based model of ePRO system validation was developed based on the principles of user acceptance testing and patient-centered care. The model allows systematic inspection of system specifications and identification of technical errors through simulated positive and negative usage pathways in open and closed environments. The model was able to detect 15 positive errors with 1 unfavorable response when applied to electronic Personal Assessment Questionnaire-Paediatric Dentistry system testing. Conclusions: The application of the behavior-based model to a newly developed ePRO system showed a high ability for technical error detection in a systematic fashion. The proposed model will increase confidence in the validity of ePRO systems as data collection tools in future research and clinical practice. %M 39288407 %R 10.2196/56370 %U https://formative.jmir.org/2024/1/e56370 %U https://doi.org/10.2196/56370 %U http://www.ncbi.nlm.nih.gov/pubmed/39288407 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e46901 %T User Experiences of Transitioning From a Homegrown Electronic Health Record to a Vendor-Based Product in the Department of Veterans Affairs: Qualitative Findings From a Mixed Methods Evaluation %A Anderson,Ekaterina %A Moldestad,Megan %A Brunner,Julian %A Ball,Sherry %A Helfrich,Christian %A Orlander,Jay %A Rinne,Seppo %A Sayre,George %+ Center for Health Optimization and Implementation Research, Veterans Affairs Bedford Healthcare System, 200 Springs Road, Bedford, MA, 01730, United States, 1 7816874936, Ekaterina.Anderson@va.gov %K electronic health records %K United States Department of Veterans Affairs %K Veterans Affairs %K organizational change %K delivery of health care %K integrated %K medical informatics %D 2024 %7 10.9.2024 %9 Original Paper %J JMIR Form Res %G English %X Background: The Department of Veterans Affairs (VA), the largest nationally integrated health system in the United States, is transitioning from its homegrown electronic health record (EHR) to a new vendor-based EHR, Oracle Cerner. Experiences of the first VA site to transition have been widely discussed in the media, but in-depth accounts based on rigorous research are lacking. Objective: We sought to explore employee perspectives on the rationale for, and value of, transitioning from a VA-tailored EHR to a vendor-based product. Methods: As part of a larger mixed methods, multisite, formative evaluation of VA clinician and staff experiences with the EHR transition, we conducted semistructured interviews at the Mann-Grandstaff VA Medical Center before, during, and after going live in October 2020. In total, we completed 122 interviews with 26 participants across multiple departments. Results: Before the new vendor-based EHR went live, participants initially expressed cautious optimism about the transition. However, in subsequent interviews following the go-live, participants increasingly critiqued the vendor’s understanding of VA’s needs, values, and workflows, as well as what they perceived as an inadequate fit between the functionalities of the new vendor-based EHR system and VA’s characteristic approach to care. As much as a year after going live, participants reiterated these concerns while also expressing a desire for substantive changes to the transition process, with some questioning the value of continuing with the transition. Conclusions: VA’s transition from a homegrown EHR to a vendor-based EHR system has presented substantial challenges, both practical and cultural in nature. Consequently, it is a valuable case study for understanding the sociotechnical dimension of EHR-to-EHR transitions. These findings have implications for both VA leadership and the broader community of policy makers, vendors, informaticists, and others involved in large-scale health information technology implementations. %M 39255006 %R 10.2196/46901 %U https://formative.jmir.org/2024/1/e46901 %U https://doi.org/10.2196/46901 %U http://www.ncbi.nlm.nih.gov/pubmed/39255006 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 11 %N %P e44662 %T Design and Implementation of an Opioid Scorecard for Hospital System–Wide Peer Comparison of Opioid Prescribing Habits: Observational Study %A Slovis,Benjamin Heritier %A Huang,Soonyip %A McArthur,Melanie %A Martino,Cara %A Beers,Tasia %A Labella,Meghan %A Riggio,Jeffrey M %A Pribitkin,Edmund deAzevedo %K opioids %K peer comparison %K quality %K scorecard %K prescribing %K design %K implementation %K opioid %K morbidity %K mortality %K opioid usage %K opioid dependence %K drug habits %D 2024 %7 9.9.2024 %9 %J JMIR Hum Factors %G English %X Background: Reductions in opioid prescribing by health care providers can lead to a decreased risk of opioid dependence in patients. Peer comparison has been demonstrated to impact providers’ prescribing habits, though its effect on opioid prescribing has predominantly been studied in the emergency department setting. Objective: The purpose of this study is to describe the development of an enterprise-wide opioid scorecard, the architecture of its implementation, and plans for future research on its effects. Methods: Using data generated by the author’s enterprise vendor–based electronic health record, the enterprise analytics software, and expertise from a dedicated group of informaticists, physicians, and analysts, the authors developed an opioid scorecard that was released on a quarterly basis via email to all opioid prescribers at our institution. These scorecards compare providers’ opioid prescribing habits on the basis of established metrics to those of their peers within their specialty throughout the enterprise. Results: At the time of this study’s completion, 2034 providers have received at least 1 scorecard over a 5-quarter period ending in September 2021. Poisson regression demonstrated a 1.6% quarterly reduction in opioid prescribing, and chi-square analysis demonstrated pre-post reductions in the proportion of prescriptions longer than 5 days’ duration and a morphine equivalent daily dose of >50. Conclusions: To our knowledge, this is the first peer comparison effort with high-quality evidence-based metrics of this scale published in the literature. By sharing this process for designing the metrics and the process of distribution, the authors hope to influence other health systems to attempt to curb the opioid pandemic through peer comparison. Future research examining the effects of this intervention could demonstrate significant reductions in opioid prescribing, thus potentially reducing the progression of individual patients to opioid use disorder and the associated increased risk of morbidity and mortality. %R 10.2196/44662 %U https://humanfactors.jmir.org/2024/1/e44662 %U https://doi.org/10.2196/44662 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 11 %N %P e55790 %T Evaluating the Usability and Quality of a Clinical Mobile App for Assisting Physicians in Head Computed Tomography Scan Ordering: Mixed Methods Study %A Meidani,Zahra %A Omidvar,Aydine %A Akbari,Hossein %A Asghari,Fatemeh %A Khajouei,Reza %A Nazemi,Zahra %A Nabovati,Ehsan %A Holl,Felix %+ Health Information Management Research Center, Kashan University of Medical Sciences, Qotbe Ravandi, Pezeshk Blvd, Kashan, 8715973449, Iran, 98 55589343, nabovati@kaums.ac.ir %K mobile apps %K user-centered design %K user-computer interface %K physicians %K tomography %K x-ray computed %K mobile phone %D 2024 %7 9.9.2024 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Among the numerous factors contributing to health care providers’ engagement with mobile apps, including user characteristics (eg, dexterity, anatomy, and attitude) and mobile features (eg, screen and button size), usability and quality of apps have been introduced as the most influential factors. Objective: This study aims to investigate the usability and quality of the Head Computed Tomography Scan Appropriateness Criteria (HAC) mobile app for physicians’ computed tomography scan ordering. Methods: Our study design was primarily based on methodological triangulation by using mixed methods research involving quantitative and qualitative think-aloud usability testing, quantitative analysis of the Mobile Apps Rating Scale (MARS) for quality assessment, and debriefing across 3 phases. In total, 16 medical interns participated in quality assessment and testing usability characteristics, including efficiency, effectiveness, learnability, errors, and satisfaction with the HAC app. Results: The efficiency and effectiveness of the HAC app were deemed satisfactory, with ratings of 97.8% and 96.9%, respectively. MARS assessment scale indicated the overall favorable quality score of the HAC app (82 out of 100). Scoring 4 MARS subscales, Information (73.37 out of 100) and Engagement (73.48 out of 100) had the lowest scores, while Aesthetics had the highest score (87.86 out of 100). Analysis of the items in each MARS subscale revealed that in the Engagement subscale, the lowest score of the HAC app was “customization” (63.6 out of 100). In the Functionality subscale, the HAC app’s lowest value was “performance” (67.4 out of 100). Qualitative think-aloud usability testing of the HAC app found notable usability issues grouped into 8 main categories: lack of finger-friendly touch targets, poor search capabilities, input problems, inefficient data presentation and information control, unclear control and confirmation, lack of predictive capabilities, poor assistance and support, and unclear navigation logic. Conclusions: Evaluating the quality and usability of mobile apps using a mixed methods approach provides valuable information about their functionality and disadvantages. It is highly recommended to embrace a more holistic and mixed methods strategy when evaluating mobile apps, because results from a single method imperfectly reflect trustworthy and reliable information regarding the usability and quality of apps. %M 39250788 %R 10.2196/55790 %U https://humanfactors.jmir.org/2024/1/e55790 %U https://doi.org/10.2196/55790 %U http://www.ncbi.nlm.nih.gov/pubmed/39250788 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 12 %N %P e58080 %T Exploring Impediments Imposed by the Medical Device Regulation EU 2017/745 on Software as a Medical Device %A Svempe,Liga %+ Faculty of Social Sciences, Riga Stradins University, Dzirciema 16, Riga, LV1007, Latvia, 371 67409120, liga.svempe@rsu.edu.lv %K software %K artificial intelligence %K medical device regulation %K rights %K digital health %D 2024 %7 5.9.2024 %9 Viewpoint %J JMIR Med Inform %G English %X In light of rapid technological advancements, the health care sector is undergoing significant transformation with the continuous emergence of novel digital solutions. Consequently, regulatory frameworks must continuously adapt to ensure their main goal to protect patients. In 2017, the new Medical Device Regulation (EU) 2017/745 (MDR) came into force, bringing more complex requirements for development, launch, and postmarket surveillance. However, the updated regulation considerably impacts the manufacturers, especially small- and medium-sized enterprises, and consequently, the accessibility of medical devices in the European Union market, as many manufacturers decide to either discontinue their products, postpone the launch of new innovative solutions, or leave the European Union market in favor of other regions such as the United States. This could lead to reduced health care quality and slower industry innovation efforts. Effective policy calibration and collaborative efforts are essential to mitigate these effects and promote ongoing advancements in health care technologies in the European Union market. This paper is a narrative review with the objective of exploring hindering factors to software as a medical device development, launch, and marketing brought by the new regulation. It exclusively focuses on the factors that engender obstacles. Related regulations, directives, and proposals were discussed for comparison and further analysis. %M 39235850 %R 10.2196/58080 %U https://medinform.jmir.org/2024/1/e58080 %U https://doi.org/10.2196/58080 %U http://www.ncbi.nlm.nih.gov/pubmed/39235850 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 11 %N %P e48359 %T Toward Personalized Care and Patient Empowerment and Perspectives on a Personal Health Record in Hemophilia Care: Qualitative Interview Study %A Brands,Martijn R %A Haverman,Lotte %A Muis,Jelmer J %A Driessens,Mariëtte H E %A Meijer,Stephan %A van der Meer,Felix J M %A de Jong,Marianne %A van der Bom,Johanna G %A Cnossen,Marjon H %A Fijnvandraat,Karin %A Gouw,Samantha C %+ Department of Pediatric Hematology, Emma Children’s Hospital, Amsterdam UMC location University of Amsterdam, Meibergdreef 9, 1105 AZ, Amsterdam, Netherlands, 31 020 566 2727, s.c.gouw@amsterdamumc.nl %K hemophilia %K telemedicine %K health records %K personal %K decision-making %K shared %K patient participation %D 2024 %7 3.9.2024 %9 Original Paper %J JMIR Hum Factors %G English %X Background: To enable personalized treatment and shared decision-making in chronic care, relevant health information is collected. However, health information is often fragmented across hospital information systems, digital health apps, and questionnaire portals. This also pertains to hemophilia care, in which scattered information hampers integrated care. We intend to co-design a nationwide digital personal health record (PHR) for patients to help manage their health information. For this, user perspectives are crucial. Objective: This study aims to assess patients’ and health care providers’ perspectives regarding the use of a PHR in hemophilia care in the Netherlands, required functionalities, and expectations and concerns. Methods: In this semistructured interview study, 19 pediatric and adult persons with hemophilia, parents, and women with other inherited bleeding disorders, as well as 18 health care providers working within and outside of hemophilia treatment centers, participated. Perspectives of patients and providers were explored separately. To explore requirements, participants were asked to prioritize functionalities. Results: Participants expected a PHR would increase the transparency of health information, improve patients’ understanding of their illness, and help the coordination of care between health care providers and institutions. Prioritized functionalities included the integration of relevant health information and patient-entered data. Formulated expectations and concerns focused on 4 themes: usability, safety, inclusiveness, and implementation. While patients expressed worries over medicalization (ie, more confrontational reminders of their illness), providers were concerned about an increased workload. Conclusions: People with hemophilia, their parents, and health care providers welcomed the development of a PHR, as they expected it would result in better coordinated care. Formulated expectations and concerns will contribute to the successful development of a PHR for persons with hemophilia, and ultimately, for all persons with a chronic condition. %M 39226550 %R 10.2196/48359 %U https://humanfactors.jmir.org/2024/1/e48359 %U https://doi.org/10.2196/48359 %U http://www.ncbi.nlm.nih.gov/pubmed/39226550 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e56332 %T Leveraging an Electronic Health Record Patient Portal to Help Patients Formulate Their Health Care Goals: Mixed Methods Evaluation of Pilot Interventions %A Naimark,Jody %A Tinetti,Mary E %A Delbanco,Tom %A Dong,Zhiyong %A Harcourt,Kendall %A Esterson,Jessica %A Charpentier,Peter %A Walker,Jan %+ Division of General Medicine, Beth Israel Deaconess Medical Center, Harvard Medical School, 330 Brookline Ave, HVMA Annex, Suite 2200, Boston, MA, 02215, United States, 1 6177541416, jwalker1@bidmc.harvard.edu %K electronic pre-visit questionnaire %K patient portal %K multimorbidity %K care priorities %K goal %K goals %K priority %K priorities %K care plan %K care plans %K questionnaire %K questionnaires %K previsit %K pre-visit %K portal %K portals %K electronic health record %K EHR %K records %K record %K engagement %K experience %K experiences %K attitude %K attitudes %K opinion %K perception %K perceptions %K perspective %K perspectives %D 2024 %7 29.8.2024 %9 Original Paper %J JMIR Form Res %G English %X Background: Persons with multiple chronic conditions face complex medical regimens and clinicians may not focus on what matters most to these patients who vary widely in their health priorities. Patient Priorities Care is a facilitator-led process designed to identify patients’ priorities and align decision-making and care, but the need for a facilitator has limited its widespread adoption. Objective: The aims of this study are to design and test mechanisms for patients to complete a self-directed process for identifying priorities and providing their priorities to clinicians. Methods: The study involved patients of at least 65 years of age at 2 family medicine practices with 5 physicians each. We first tested 2 versions of an interactive website and asked patients to bring their results to their visit. We then tested an Epic previsit questionnaire derived from the website’s questions and included standard previsit materials. We completed postintervention phone interviews and an online survey with participating patients and collected informal feedback and conducted a focus group with participating physicians. Results: In the test of the first website version, 17.3% (35/202) of invited patients went to the website, 11.4% (23/202) completed all of the questions, 2.5% (5/202) brought results to their visits, and the median session time was 43.0 (IQR 28.0) minutes. Patients expressed confusion about bringing results to the visit. After clarifying that issue in the second version, 15.1% (32/212) of patients went to the website, 14.6% (31/212) completed the questions, 1.9% (4/212) brought results to the visit, and the median session time was 35.0 (IQR 35.0) minutes. In the test of the Epic questionnaire, 26.4% (198/750) of patients completed the questionnaire before at least 1 visit, and the median completion time was 14.0 (IQR 23.0) minutes. The 8 main questions were answered 62.9% (129/205) to 95.6% (196/205) of the time. Patients who completed questionnaires were younger than those who did not (72.3 vs 76.1 years) and were more likely to complete at least 1 of their other assigned questionnaires (99.5%, 197/198) than those who did not (10.3%, 57/552). A total of 140 of 198 (70.7%) patients responded to a survey, and 86 remembered completing the questionnaire; 78 (90.7%) did not remember having difficulty answering the questions and 57 (68.7%) agreed or somewhat agreed that it helped them and their clinicians to understand their priorities. Doctors noted that the sickest patients did not complete the questionnaire and that the discussion provided a good segue into end-of-life care. Conclusions: Embedding questionnaires assaying patient priorities into patient portals holds promise for expanding access to priorities-concordant care. %M 39207829 %R 10.2196/56332 %U https://formative.jmir.org/2024/1/e56332 %U https://doi.org/10.2196/56332 %U http://www.ncbi.nlm.nih.gov/pubmed/39207829 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e55752 %T Sociotechnical Cross-Country Analysis of Contextual Factors That Impact Patients’ Access to Electronic Health Records in 4 European Countries: Framework Evaluation Study %A Moll,Jonas %A Scandurra,Isabella %A Bärkås,Annika %A Blease,Charlotte %A Hägglund,Maria %A Hörhammer,Iiris %A Kane,Bridget %A Kristiansen,Eli %A Ross,Peeter %A Åhlfeldt,Rose-Mharie %A Klein,Gunnar O %+ Centre for Empirical Research on Information systems, School of Business, Örebro University, Nova building, 4th floor, Fakultetsgatan 1, Örebro, 70182, Sweden, 46 0703174383, jonas.moll@oru.se %K electronic health record %K EHR %K health data %K national survey %K web-based medical record %K web-based record access %K patient access %K patient-accessible electronic health record %K patient portal %K sociotechnical analysis %D 2024 %7 26.8.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: The NORDeHEALTH project studies patient-accessible electronic health records (PAEHRs) in Estonia, Finland, Norway, and Sweden. Such country comparisons require an analysis of the sociotechnical context of these services. Although sociotechnical analyses of PAEHR services have been carried out in the past, a framework specifically tailored to in-depth cross-country analysis has not been developed. Objective: This study aims to develop and evaluate a method for a sociotechnical analysis of PAEHRs that advances a framework for sociotechnical analysis of eHealth solutions first presented by Sittig and Singh. This first article in a series presents the development of the method and a cross-country comparison of the contextual factors that enable PAEHR access and use. Methods: The dimensions of the framework for sociotechnical analysis were thoroughly discussed and extended in a series of workshops with international stakeholders, all being eHealth researchers focusing on PAEHRs. All countries were represented in the working group to make sure that important national perspectives were covered. A spreadsheet with relevant questions related to the studied services and the various dimensions of the sociotechnical framework was constructed and distributed to the 4 participating countries, and the project participants researched various national sources to provide the relevant data for the comparisons in the 10 sociotechnical dimensions. Results: In total, 3 dimensions were added to the methodology of Sittig and Singh to separate clinical content from features and functions of PAEHRs and demonstrate basic characteristics of the different countries regarding national and regional steering of health care and information and communications technology developments. The final framework contained the following dimensions: metadata; hardware and software computing infrastructure; features and functions; clinical content shared with patients; human-computer interface; people; workflow and communication; the health care organization’s internal policies, procedures, and culture; national rules, regulations, and incentives; system measurement and monitoring; and health care system context. The dimensions added during the study mostly concerned background information needed for cross-country comparisons in particular. Several similarities were identified among the compared countries, especially regarding hardware and software computing infrastructure. All countries had, for example, one national access point, and patients are provided a PAEHR automatically. Most of the differences could be identified in the health care system context dimension. One important difference concerned the governing of information and communications technology development, where different levels (state, region, and municipality) were responsible in different countries. Conclusions: This is the first large-scale international sociotechnical analysis of services for patients to access their electronic health records; this study compared services in Estonia, Finland, Norway, and Sweden. A methodology for such an analysis was developed and is presented to enable comparison studies in other national contexts to enable future implementations and evaluations of PAEHRs. %M 39186760 %R 10.2196/55752 %U https://www.jmir.org/2024/1/e55752 %U https://doi.org/10.2196/55752 %U http://www.ncbi.nlm.nih.gov/pubmed/39186760 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e55717 %T Impacts of Clinical Decision Support Systems on the Relationship, Communication, and Shared Decision-Making Between Health Care Professionals and Patients: Multistakeholder Interview Study %A Funer,Florian %A Schneider,Diana %A Heyen,Nils B %A Aichinger,Heike %A Klausen,Andrea Diana %A Tinnemeyer,Sara %A Liedtke,Wenke %A Salloch,Sabine %A Bratan,Tanja %+ Competence Center Emerging Technologies, Fraunhofer Institute for Systems and Innovation Research ISI, Breslauer Straße 48, Karlsruhe, 76139, Germany, 49 0721 6809 58, diana.schneider@isi.fraunhofer.de %K clinical decision support system %K CDSS %K health care professionals %K patients %K relationships %K communication %K shared decision-making %K Germany %D 2024 %7 23.8.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: Clinical decision support systems (CDSSs) are increasingly being introduced into various domains of health care. Little is known so far about the impact of such systems on the health care professional–patient relationship, and there is a lack of agreement about whether and how patients should be informed about the use of CDSSs. Objective: This study aims to explore, in an empirically informed manner, the potential implications for the health care professional–patient relationship and to underline the importance of this relationship when using CDSSs for both patients and future professionals. Methods: Using a methodological triangulation, 15 medical students and 12 trainee nurses were interviewed in semistructured interviews and 18 patients were involved in focus groups between April 2021 and April 2022. All participants came from Germany. Three examples of CDSSs covering different areas of health care (ie, surgery, nephrology, and intensive home care) were used as stimuli in the study to identify similarities and differences regarding the use of CDSSs in different fields of application. The interview and focus group transcripts were analyzed using a structured qualitative content analysis. Results: From the interviews and focus groups analyzed, three topics were identified that interdependently address the interactions between patients and health care professionals: (1) CDSSs and their impact on the roles of and requirements for health care professionals, (2) CDSSs and their impact on the relationship between health care professionals and patients (including communication requirements for shared decision-making), and (3) stakeholders’ expectations for patient education and information about CDSSs and their use. Conclusions: The results indicate that using CDSSs could restructure established power and decision-making relationships between (future) health care professionals and patients. In addition, respondents expected that the use of CDSSs would involve more communication, so they anticipated an increased time commitment. The results shed new light on the existing discourse by demonstrating that the anticipated impact of CDSSs on the health care professional–patient relationship appears to stem less from the function of a CDSS and more from its integration in the relationship. Therefore, the anticipated effects on the relationship between health care professionals and patients could be specifically addressed in patient information about the use of CDSSs. %M 39178023 %R 10.2196/55717 %U https://www.jmir.org/2024/1/e55717 %U https://doi.org/10.2196/55717 %U http://www.ncbi.nlm.nih.gov/pubmed/39178023 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 12 %N %P e57153 %T Evaluating and Enhancing the Fitness-for-Purpose of Electronic Health Record Data: Qualitative Study on Current Practices and Pathway to an Automated Approach Within the Medical Informatics for Research and Care in University Medicine Consortium %A Kamdje Wabo,Gaetan %A Moorthy,Preetha %A Siegel,Fabian %A Seuchter,Susanne A %A Ganslandt,Thomas %+ Center for Preventive Medicine and Digital Health Baden-Wuerttemberg, Department of Biomedical Informatics, Medical Faculty of Mannheim, University of Heidelberg, Building 3, Level 4, Theodor-Kutzer-Ufer 1-3, Mannheim, 68167, Germany, 49 621 383 8088, gaetankamdje.wabo@medma.uni-heidelberg.de %K data quality %K fitness-for-purpose %K secondary use %K thematic analysis %K EHR data %K electronic health record %K data integration center %K Medical Informatics Initiative %K MIRACUM consortium %K Medical Informatics for Research and Care in University Medicine %K data science %K integration %K data use %K visualization %K visualizations %K record %K records %K EHR %K EHRs %K survey %K surveys %K medical informatics %D 2024 %7 19.8.2024 %9 Original Paper %J JMIR Med Inform %G English %X Background: Leveraging electronic health record (EHR) data for clinical or research purposes heavily depends on data fitness. However, there is a lack of standardized frameworks to evaluate EHR data suitability, leading to inconsistent quality in data use projects (DUPs). This research focuses on the Medical Informatics for Research and Care in University Medicine (MIRACUM) Data Integration Centers (DICs) and examines empirical practices on assessing and automating the fitness-for-purpose of clinical data in German DIC settings. Objective: The study aims (1) to capture and discuss how MIRACUM DICs evaluate and enhance the fitness-for-purpose of observational health care data and examine the alignment with existing recommendations and (2) to identify the requirements for designing and implementing a computer-assisted solution to evaluate EHR data fitness within MIRACUM DICs. Methods: A qualitative approach was followed using an open-ended survey across DICs of 10 German university hospitals affiliated with MIRACUM. Data were analyzed using thematic analysis following an inductive qualitative method. Results: All 10 MIRACUM DICs participated, with 17 participants revealing various approaches to assessing data fitness, including the 4-eyes principle and data consistency checks such as cross-system data value comparison. Common practices included a DUP-related feedback loop on data fitness and using self-designed dashboards for monitoring. Most experts had a computer science background and a master’s degree, suggesting strong technological proficiency but potentially lacking clinical or statistical expertise. Nine key requirements for a computer-assisted solution were identified, including flexibility, understandability, extendibility, and practicability. Participants used heterogeneous data repositories for evaluating data quality criteria and practical strategies to communicate with research and clinical teams. Conclusions: The study identifies gaps between current practices in MIRACUM DICs and existing recommendations, offering insights into the complexities of assessing and reporting clinical data fitness. Additionally, a tripartite modular framework for fitness-for-purpose assessment was introduced to streamline the forthcoming implementation. It provides valuable input for developing and integrating an automated solution across multiple locations. This may include statistical comparisons to advanced machine learning algorithms for operationalizing frameworks such as the 3×3 data quality assessment framework. These findings provide foundational evidence for future design and implementation studies to enhance data quality assessments for specific DUPs in observational health care settings. %M 39158950 %R 10.2196/57153 %U https://medinform.jmir.org/2024/1/e57153 %U https://doi.org/10.2196/57153 %U http://www.ncbi.nlm.nih.gov/pubmed/39158950 %0 Journal Article %@ 2291-9694 %I %V 12 %N %P e52934 %T Data Flow Construction and Quality Evaluation of Electronic Source Data in Clinical Trials: Pilot Study Based on Hospital Electronic Medical Records in China %A Yuan,Yannan %A Mei,Yun %A Zhao,Shuhua %A Dai,Shenglong %A Liu,Xiaohong %A Sun,Xiaojing %A Fu,Zhiying %A Zhou,Liheng %A Ai,Jie %A Ma,Liheng %A Jiang,Min %K clinical trials %K electronic source data %K EHRs %K electronic data capture systems %K data quality %K electronic health records %D 2024 %7 27.6.2024 %9 %J JMIR Med Inform %G English %X Background: The traditional clinical trial data collection process requires a clinical research coordinator who is authorized by the investigators to read from the hospital’s electronic medical record. Using electronic source data opens a new path to extract patients’ data from electronic health records (EHRs) and transfer them directly to an electronic data capture (EDC) system; this method is often referred to as eSource. eSource technology in a clinical trial data flow can improve data quality without compromising timeliness. At the same time, improved data collection efficiency reduces clinical trial costs. Objective: This study aims to explore how to extract clinical trial–related data from hospital EHR systems, transform the data into a format required by the EDC system, and transfer it into sponsors’ environments, and to evaluate the transferred data sets to validate the availability, completeness, and accuracy of building an eSource dataflow. Methods: A prospective clinical trial study registered on the Drug Clinical Trial Registration and Information Disclosure Platform was selected, and the following data modules were extracted from the structured data of 4 case report forms: demographics, vital signs, local laboratory data, and concomitant medications. The extracted data was mapped and transformed, deidentified, and transferred to the sponsor’s environment. Data validation was performed based on availability, completeness, and accuracy. Results: In a secure and controlled data environment, clinical trial data was successfully transferred from a hospital EHR to the sponsor’s environment with 100% transcriptional accuracy, but the availability and completeness of the data could be improved. Conclusions: Data availability was low due to some required fields in the EDC system not being available directly in the EHR. Some data is also still in an unstructured or paper-based format. The top-level design of the eSource technology and the construction of hospital electronic data standards should help lay a foundation for a full electronic data flow from EHRs to EDC systems in the future. %R 10.2196/52934 %U https://medinform.jmir.org/2024/1/e52934 %U https://doi.org/10.2196/52934 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e54518 %T Key Considerations for Designing Clinical Studies to Evaluate Digital Health Solutions %A Bolinger,Elaina %A Tyl,Benoit %+ Integrated Evidence Generation & Business Innovation, Bayer HealthCare SAS, 10 Place de Belgique, La Garenne Colombes, F-92254, France, 33 6 80 29 07 79, benoit.tyl@bayer.com %K evidence generation %K clinical robustness %K clinical trials %K digital health %K solutions %K digital health solutions %K DHS %K health care system %K patients %K patient %K individuals %K individual %K healthcare system %K control arm adaptations %K randomization methods %K real-world data %K platform research %D 2024 %7 17.6.2024 %9 Viewpoint %J J Med Internet Res %G English %X Evidence of clinical impact is critical to unlock the potential of digital health solutions (DHSs), yet many solutions are failing to deliver positive clinical results. We argue in this viewpoint that this failure is linked to current approaches to DHS evaluation design, which neglect numerous key characteristics (KCs) requiring specific scientific and design considerations. We first delineate the KCs of DHSs: (1) they are implemented at health care system and patient levels; (2) they are “complex” interventions; (3) they can drive multiple clinical outcomes indirectly through a multitude of smaller clinical benefits; (4) their mechanism of action can vary between individuals and change over time based on patient needs; and (5) they develop through short, iterative cycles—optimally within a real-world use context. Following our objective to drive better alignment between clinical evaluation design and the unique traits of DHSs, we then provide methodological suggestions that better address these KCs, including tips on mechanism-of-action mapping, alternative randomization methods, control-arm adaptations, and novel end-point selection, as well as innovative methods utilizing real-world data and platform research. %M 38885020 %R 10.2196/54518 %U https://www.jmir.org/2024/1/e54518 %U https://doi.org/10.2196/54518 %U http://www.ncbi.nlm.nih.gov/pubmed/38885020 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 11 %N %P e50939 %T Assessing the Utility, Impact, and Adoption Challenges of an Artificial Intelligence–Enabled Prescription Advisory Tool for Type 2 Diabetes Management: Qualitative Study %A Yoon,Sungwon %A Goh,Hendra %A Lee,Phong Ching %A Tan,Hong Chang %A Teh,Ming Ming %A Lim,Dawn Shao Ting %A Kwee,Ann %A Suresh,Chandran %A Carmody,David %A Swee,Du Soon %A Tan,Sarah Ying Tse %A Wong,Andy Jun-Wei %A Choo,Charlotte Hui-Min %A Wee,Zongwen %A Bee,Yong Mong %+ Health Services and Systems Research, Duke-NUS Medical School, 8 College Road, Singapore, 169857, Singapore, 65 66013198, sungwon.yoon@duke-nus.edu.sg %K clinical decision support system %K artificial intelligence %K endocrinology %K diabetes management %K human factors %D 2024 %7 13.6.2024 %9 Original Paper %J JMIR Hum Factors %G English %X Background: The clinical management of type 2 diabetes mellitus (T2DM) presents a significant challenge due to the constantly evolving clinical practice guidelines and growing array of drug classes available. Evidence suggests that artificial intelligence (AI)–enabled clinical decision support systems (CDSSs) have proven to be effective in assisting clinicians with informed decision-making. Despite the merits of AI-driven CDSSs, a significant research gap exists concerning the early-stage implementation and adoption of AI-enabled CDSSs in T2DM management. Objective: This study aimed to explore the perspectives of clinicians on the use and impact of the AI-enabled Prescription Advisory (APA) tool, developed using a multi-institution diabetes registry and implemented in specialist endocrinology clinics, and the challenges to its adoption and application. Methods: We conducted focus group discussions using a semistructured interview guide with purposively selected endocrinologists from a tertiary hospital. The focus group discussions were audio-recorded and transcribed verbatim. Data were thematically analyzed. Results: A total of 13 clinicians participated in 4 focus group discussions. Our findings suggest that the APA tool offered several useful features to assist clinicians in effectively managing T2DM. Specifically, clinicians viewed the AI-generated medication alterations as a good knowledge resource in supporting the clinician’s decision-making on drug modifications at the point of care, particularly for patients with comorbidities. The complication risk prediction was seen as positively impacting patient care by facilitating early doctor-patient communication and initiating prompt clinical responses. However, the interpretability of the risk scores, concerns about overreliance and automation bias, and issues surrounding accountability and liability hindered the adoption of the APA tool in clinical practice. Conclusions: Although the APA tool holds great potential as a valuable resource for improving patient care, further efforts are required to address clinicians’ concerns and improve the tool’s acceptance and applicability in relevant contexts. %M 38869934 %R 10.2196/50939 %U https://humanfactors.jmir.org/2024/1/e50939 %U https://doi.org/10.2196/50939 %U http://www.ncbi.nlm.nih.gov/pubmed/38869934 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 12 %N %P e54811 %T Evaluating the Prevalence of Burnout Among Health Care Professionals Related to Electronic Health Record Use: Systematic Review and Meta-Analysis %A Wu,Yuxuan %A Wu,Mingyue %A Wang,Changyu %A Lin,Jie %A Liu,Jialin %A Liu,Siru %+ Information Center, West China Hospital, Sichuan University, 37 Guoxue Road, Chengdu, 610041, China, 86 28 85422306, Dljl8@163.com %K clinical decision support system %K electronic health record %K electronic medical record %K health information technology %K alert fatigue %K burnout %K health care professionals %K health care service %K EHR %K systematic review %K meta-analysis %K health information system %K clinician burnout %K health informatics %D 2024 %7 12.6.2024 %9 Review %J JMIR Med Inform %G English %X Background: Burnout among health care professionals is a significant concern, with detrimental effects on health care service quality and patient outcomes. The use of the electronic health record (EHR) system has been identified as a significant contributor to burnout among health care professionals. Objective: This systematic review and meta-analysis aims to assess the prevalence of burnout among health care professionals associated with the use of the EHR system, thereby providing evidence to improve health information systems and develop strategies to measure and mitigate burnout. Methods: We conducted a comprehensive search of the PubMed, Embase, and Web of Science databases for English-language peer-reviewed articles published between January 1, 2009, and December 31, 2022. Two independent reviewers applied inclusion and exclusion criteria, and study quality was assessed using the Joanna Briggs Institute checklist and the Newcastle-Ottawa Scale. Meta-analyses were performed using R (version 4.1.3; R Foundation for Statistical Computing), with EndNote X7 (Clarivate) for reference management. Results: The review included 32 cross-sectional studies and 5 case-control studies with a total of 66,556 participants, mainly physicians and registered nurses. The pooled prevalence of burnout among health care professionals in cross-sectional studies was 40.4% (95% CI 37.5%-43.2%). Case-control studies indicated a higher likelihood of burnout among health care professionals who spent more time on EHR-related tasks outside work (odds ratio 2.43, 95% CI 2.31-2.57). Conclusions: The findings highlight the association between the increased use of the EHR system and burnout among health care professionals. Potential solutions include optimizing EHR systems, implementing automated dictation or note-taking, employing scribes to reduce documentation burden, and leveraging artificial intelligence to enhance EHR system efficiency and reduce the risk of burnout. Trial Registration: PROSPERO International Prospective Register of Systematic Reviews CRD42021281173; https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42021281173 %M 38865188 %R 10.2196/54811 %U https://medinform.jmir.org/2024/1/e54811 %U https://doi.org/10.2196/54811 %U http://www.ncbi.nlm.nih.gov/pubmed/38865188 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 11 %N %P e47631 %T Evaluation of a Computer-Aided Clinical Decision Support System for Point-of-Care Use in Low-Resource Primary Care Settings: Acceptability Evaluation Study %A Tegenaw,Geletaw Sahle %A Sori,Demisew Amenu %A Teklemariam,Girum Ketema %A Verbeke,Frank %A Cornelis,Jan %A Jansen,Bart %+ Department of Electronics and Informatics, Vrije Universiteit Brussel, Pleinlaan 2, Brussel, 1050, Belgium, 32 485744961, gtegenaw@vub.be %K low-resource setting %K clinical decision support system %K point-of-care instrument %K evaluation %K user acceptance %K structural equation modeling %K partial least squares structural equation modeling %K decision-making %K decision making %K decision support %K caregiver %K users %K acceptance %K system quality %D 2024 %7 11.6.2024 %9 Original Paper %J JMIR Hum Factors %G English %X Background: A clinical decision support system (CDSS) based on the logic and philosophy of clinical pathways is critical for managing the quality of health care and for standardizing care processes. Using such a system at a point-of-care setting is becoming more frequent these days. However, in a low-resource setting (LRS), such systems are frequently overlooked. Objective: The purpose of the study was to evaluate the user acceptance of a CDSS in LRSs. Methods: The CDSS evaluation was carried out at the Jimma Health Center and the Jimma Higher Two Health Center, Jimma, Ethiopia. The evaluation was based on 22 parameters organized into 6 categories: ease of use, system quality, information quality, decision changes, process changes, and user acceptance. A Mann-Whitney U test was used to investigate whether the difference between the 2 health centers was significant (2-tailed, 95% CI; α=.05). Pearson correlation and partial least squares structural equation modeling (PLS-SEM) was used to identify the relationship and factors influencing the overall acceptance of the CDSS in an LRS. Results: On the basis of 116 antenatal care, pregnant patient care, and postnatal care cases, 73 CDSS evaluation responses were recorded. We found that the 2 health centers did not differ significantly on 16 evaluation parameters. We did, however, detect a statistically significant difference in 6 parameters (P<.05). PLS-SEM results showed that the coefficient of determination, R2, of perceived user acceptance was 0.703. More precisely, the perceived ease of use (β=.015, P=.91) and information quality (β=.149, P=.25) had no positive effect on CDSS acceptance but, rather, on the system quality and perceived benefits of the CDSS, with P<.05 and β=.321 and β=.486, respectively. Furthermore, the perceived ease of use was influenced by information quality and system quality, with an R2 value of 0.479, indicating that the influence of information quality on the ease of use is significant but the influence of system quality on the ease of use is not, with β=.678 (P<.05) and β=.021(P=.89), respectively. Moreover, the influence of decision changes (β=.374, P<.05) and process changes (β=.749, P<.05) both was significant on perceived benefits (R2=0.983). Conclusions: This study concludes that users are more likely to accept and use a CDSS at the point of care when it is easy to grasp the perceived benefits and system quality in terms of health care professionals’ needs. We believe that the CDSS acceptance model developed in this study reveals specific factors and variables that constitute a step toward the effective adoption and deployment of a CDSS in LRSs. %M 38861298 %R 10.2196/47631 %U https://humanfactors.jmir.org/2024/1/e47631 %U https://doi.org/10.2196/47631 %U http://www.ncbi.nlm.nih.gov/pubmed/38861298 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 11 %N %P e53976 %T Implementing a Hospital Call Center Service for Mental Health in Uganda: User-Centered Design Approach %A Kabukye,Johnblack K %A Namagembe,Rosemary %A Nakku,Juliet %A Kiberu,Vincent %A Sjölinder,Marie %A Nilsson,Susanne %A Wamala-Larsson,Caroline %+ SPIDER - The Swedish Program for ICT in Developing Regions, Department of Computer and Systems Sciences, Stockholm University, Borgarfjordsgatan 12 (NOD Building), PO Box 7003 Kista, Stockholm, SE-164 07, Sweden, 46 8162000, kabukye@dsv.su.se %K mHealth %K mobile health %K digital health %K digital solution %K digital solutions %K digital intervention %K digital interventions %K mental health %K awareness %K Uganda %K Africa %K African %K user centred %K user centered %K design %K qualitative %K focus group %K focus groups %K call centre %K call centres %K call center %K call centers %K mental %K experience %K experiences %K attitude %K attitudes %K opinion %K perception %K perceptions %K perspective %K perspectives %K cocreated %K cocreation %K service %K services %K mobile phone %D 2024 %7 6.6.2024 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Mental health conditions are a significant public health problem globally, responsible for >8 million deaths per year. In addition, they lead to lost productivity, exacerbate physical illness, and are associated with stigma and human rights violations. Uganda, like many low- and middle-income countries, faces a massive treatment gap for mental health conditions, and numerous sociocultural challenges exacerbate the burden of mental health conditions. Objective: This study aims to describe the development and formative evaluation of a digital health intervention for improving access to mental health care in Uganda. Methods: This qualitative study used user-centered design and design science research principles. Stakeholders, including patients, caregivers, mental health care providers, and implementation experts (N=65), participated in focus group discussions in which we explored participants’ experience of mental illness and mental health care, experience with digital interventions, and opinions about a proposed digital mental health service. Data were analyzed using the Consolidated Framework for Implementation Research to derive requirements for the digital solution, which was iteratively cocreated with users and piloted. Results: Several challenges were identified, including a severe shortage of mental health facilities, unmet mental health information needs, heavy burden of caregiving, financial challenges, stigma, and negative beliefs related to mental health. Participants’ enthusiasm about digital solutions as a feasible, acceptable, and convenient method for accessing mental health services was also revealed, along with recommendations to make the service user-friendly, affordable, and available 24×7 and to ensure anonymity. A hospital call center service was developed to provide mental health information and advice in 2 languages through interactive voice response and live calls with health care professionals and peer support workers (recovering patients). In the 4 months after launch, 456 calls, from 236 unique numbers, were made to the system, of which 99 (21.7%) calls went to voicemails (out-of-office hours). Of the remaining 357 calls, 80 (22.4%) calls stopped at the interactive voice response, 231 (64.7%) calls were answered by call agents, and 22 (6.2%) calls were not answered. User feedback was positive, with callers appreciating the inclusion of peer support workers who share their recovery journeys. However, some participant recommendations (eg, adding video call options) or individualized needs (eg, prescriptions) could not be accommodated due to resource limitations or technical feasibility. Conclusions: This study demonstrates a systematic and theory-driven approach to developing contextually appropriate digital solutions for improving mental health care in Uganda and similar contexts. The positive reception of the implemented service underscores its potential impact. Future research should address the identified limitations and evaluate clinical outcomes of long-term adoption. %M 38843515 %R 10.2196/53976 %U https://humanfactors.jmir.org/2024/1/e53976 %U https://doi.org/10.2196/53976 %U http://www.ncbi.nlm.nih.gov/pubmed/38843515 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 11 %N %P e51666 %T The Solutions in Health Analytics for Rural Equity Across the Northwest (SHARE-NW) Dashboard for Health Equity in Rural Public Health: Usability Evaluation %A Heitkemper,Elizabeth %A Hulse,Scott %A Bekemeier,Betty %A Schultz,Melinda %A Whitman,Greg %A Turner,Anne M %+ School of Nursing, The University of Texas at Austin, 1710 Red River Street, Austin, TX, 78712, United States, 1 512 232 4228, e.heit@utexas.edu %K data dashboard %K rural health %K health equity %K usability %K nursing informatics %K dashboard %K rural %K informatics %K satisfaction %K think aloud %K content analysis %K user experience %K public health %K visualization %K information systems %D 2024 %7 5.6.2024 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Given the dearth of resources to support rural public health practice, the solutions in health analytics for rural equity across the northwest dashboard (SHAREdash) was created to support rural county public health departments in northwestern United States with accessible and relevant data to identify and address health disparities in their jurisdictions. To ensure the development of useful dashboards, assessment of usability should occur at multiple stages throughout the system development life cycle. SHAREdash was refined via user-centered design methods, and upon completion, it is critical to evaluate the usability of SHAREdash. Objective: This study aims to evaluate the usability of SHAREdash based on the system development lifecycle stage 3 evaluation goals of efficiency, satisfaction, and validity. Methods: Public health professionals from rural health departments from Washington, Idaho, Oregon, and Alaska were enrolled in the usability study from January to April 2022. The web-based evaluation consisted of 2 think-aloud tasks and a semistructured qualitative interview. Think-aloud tasks assessed efficiency and effectiveness, and the interview investigated satisfaction and overall usability. Verbatim transcripts from the tasks and interviews were analyzed using directed content analysis. Results: Of the 9 participants, all were female and most worked at a local health department (7/9, 78%). A mean of 10.1 (SD 1.4) clicks for task 1 (could be completed in 7 clicks) and 11.4 (SD 2.0) clicks for task 2 (could be completed in 9 clicks) were recorded. For both tasks, most participants required no prompting—89% (n=8) participants for task 1 and 67% (n=6) participants for task 2, respectively. For effectiveness, all participants were able to complete each task accurately and comprehensively. Overall, the participants were highly satisfied with the dashboard with everyone remarking on the utility of using it to support their work, particularly to compare their jurisdiction to others. Finally, half of the participants stated that the ability to share the graphs from the dashboard would be “extremely useful” for their work. The only aspect of the dashboard cited as problematic is the amount of missing data that was present, which was a constraint of the data available about rural jurisdictions. Conclusions: Think-aloud tasks showed that the SHAREdash allows users to complete tasks efficiently. Overall, participants reported being very satisfied with the dashboard and provided multiple ways they planned to use it to support their work. The main usability issue identified was the lack of available data indicating the importance of addressing the ongoing issues of missing and fragmented public health data, particularly for rural communities. %M 38837192 %R 10.2196/51666 %U https://humanfactors.jmir.org/2024/1/e51666 %U https://doi.org/10.2196/51666 %U http://www.ncbi.nlm.nih.gov/pubmed/38837192 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 11 %N %P e53071 %T Electronic Immunization Registry in Rwanda: Qualitative Study of Health Worker Experiences %A Uwera,Thaoussi %A Venkateswaran,Mahima %A Bhutada,Kiran %A Papadopoulou,Eleni %A Rukundo,Enock %A K Tumusiime,David %A Frøen,J Frederik %+ Centre of Excellence in Biomedical Engineering and eHealth, University of Rwanda, KG 11 Ave Gasabo, Kigali, Rwanda, 250 788229321, uwera04@gmail.com %K childhood immunization %K electronic immunization registry %K digital health interventions %D 2024 %7 28.5.2024 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Monitoring childhood immunization programs is essential for health systems. Despite the introduction of an electronic immunization registry called e-Tracker in Rwanda, challenges such as lacking population denominators persist, leading to implausible reports of coverage rates of more than 100%. Objective: This study aimed to assess the extent to which the immunization e-Tracker responds to stakeholders’ needs and identify key areas for improvement. Methods: In-depth interviews were conducted with all levels of e-Tracker users including immunization nurses, data managers, and supervisors from health facilities in 5 districts of Rwanda. We used an interview guide based on the constructs of the Human, Organization, and Technology–Fit (HOT-Fit) framework, and we analyzed and summarized our findings using the framework. Results: Immunization nurses reported using the e-Tracker as a secondary data entry tool in addition to paper-based forms, which resulted in considerable dissatisfaction among nurses. While users acknowledged the potential of a digital tool compared to paper-based systems, they also reported the need for improvement of functionalities to support their work, such as digital client appointment lists, lists of defaulters, search and register functions, automated monthly reports, and linkages to birth notifications and the national identity system. Conclusions: Reducing dual documentation for users can improve e-Tracker use and user satisfaction. Our findings can help identify additional digital health interventions to support and strengthen the health information system for the immunization program. %M 38805254 %R 10.2196/53071 %U https://humanfactors.jmir.org/2024/1/e53071 %U https://doi.org/10.2196/53071 %U http://www.ncbi.nlm.nih.gov/pubmed/38805254 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e51952 %T User-Centered Framework for Implementation of Technology (UFIT): Development of an Integrated Framework for Designing Clinical Decision Support Tools Packaged With Tailored Implementation Strategies %A Ray,Jessica %A Finn,Emily Benjamin %A Tyrrell,Hollyce %A Aloe,Carlin F %A Perrin,Eliana M %A Wood,Charles T %A Miner,Dean S %A Grout,Randall %A Michel,Jeremy J %A Damschroder,Laura J %A Sharifi,Mona %+ Department of Health Outcomes and Biomedical Informatics, University of Florida College of Medicine, 1889 Museum Road, Gainesville, FL, 32611, United States, 1 321 356 4346, jessica.ray@ufl.edu %K user-centered design %K implementation science %K clinical decision support %K human factors %K implementation %K decision support %K develop %K development %K framework %K frameworks %K design %K user-centered %K digital health %K health technology %K health technologies %K need %K needs %K tailor %K tailoring %K guidance %K guideline %K guidelines %K pediatric %K pediatrics %K child %K children %K obese %K obesity %K weight %K overweight %K primary care %D 2024 %7 21.5.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: Electronic health record–based clinical decision support (CDS) tools can facilitate the adoption of evidence into practice. Yet, the impact of CDS beyond single-site implementation is often limited by dissemination and implementation barriers related to site- and user-specific variation in workflows and behaviors. The translation of evidence-based CDS from initial development to implementation in heterogeneous environments requires a framework that assures careful balancing of fidelity to core functional elements with adaptations to ensure compatibility with new contexts. Objective: This study aims to develop and apply a framework to guide tailoring and implementing CDS across diverse clinical settings. Methods: In preparation for a multisite trial implementing CDS for pediatric overweight or obesity in primary care, we developed the User-Centered Framework for Implementation of Technology (UFIT), a framework that integrates principles from user-centered design (UCD), human factors/ergonomics theories, and implementation science to guide both CDS adaptation and tailoring of related implementation strategies. Our transdisciplinary study team conducted semistructured interviews with pediatric primary care clinicians and a diverse group of stakeholders from 3 health systems in the northeastern, midwestern, and southeastern United States to inform and apply the framework for our formative evaluation. Results: We conducted 41 qualitative interviews with primary care clinicians (n=21) and other stakeholders (n=20). Our workflow analysis found 3 primary ways in which clinicians interact with the electronic health record during primary care well-child visits identifying opportunities for decision support. Additionally, we identified differences in practice patterns across contexts necessitating a multiprong design approach to support a variety of workflows, user needs, preferences, and implementation strategies. Conclusions: UFIT integrates theories and guidance from UCD, human factors/ergonomics, and implementation science to promote fit with local contexts for optimal outcomes. The components of UFIT were used to guide the development of Improving Pediatric Obesity Practice Using Prompts, an integrated package comprising CDS for obesity or overweight treatment with tailored implementation strategies. Trial Registration: ClinicalTrials.gov NCT05627011; https://clinicaltrials.gov/study/NCT05627011 %M 38771622 %R 10.2196/51952 %U https://www.jmir.org/2024/1/e51952 %U https://doi.org/10.2196/51952 %U http://www.ncbi.nlm.nih.gov/pubmed/38771622 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e53122 %T Exploring the Impact of In Basket Metrics on the Adoption of a New Electronic Health Record System Among Specialists in a Tertiary Hospital in Alberta: Descriptive Study %A Avdagovska,Melita %A Kuziemsky,Craig %A Koosha,Helia %A Hadizadeh,Maliheh %A Pauly,Robert P %A Graham,Timothy %A Stafinski,Tania %A Bigam,David %A Kassam,Narmin %A Menon,Devidas %+ School of Public Health, University of Alberta, 11405 87 Avenue, Edmonton Clinic Health Academy, Edmonton, AB, T6G 1C9, Canada, 1 780 908 3334, avdagovs@ualberta.ca %K electronic health records %K In Basket %K metrics %K descriptive study %K inpatients %D 2024 %7 29.4.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: Health care organizations implement electronic health record (EHR) systems with the expectation of improved patient care and enhanced provider performance. However, while these technologies hold the potential to create improved care and system efficiencies, they can also lead to unintended negative consequences, such as patient safety issues, communication problems, and provider burnout. Objective: This study aims to document metrics related to the In Basket communication hub (time in In Basket per day, time in In Basket per appointment, In Basket messages received per day, and turnaround time) of the EHR system implemented by Alberta Health Services, the province-wide health delivery system called Connect Care (Epic Systems). The objective was to identify how a newly implemented EHR system was used, the timing of its use, and the duration of use specifically related to In Basket activities. Methods: A descriptive study was conducted. Due to the diversity of specialties, the providers were grouped into medical and surgical based on previous similar studies. The participants were further subgrouped based on their self-reported clinical full-time equivalent (FTE ) measure. This resulted in 3 subgroups for analysis: medical FTE <0.5, medical FTE >0.5, and surgical (all of whom reported FTE >0.5). The analysis was limited to outpatient clinical interactions and explicitly excluded inpatient activities. Results: A total of 72 participants from 19 different specialties enrolled in this study. The providers had, on average, 8.31 appointments per day during the reporting periods. The providers received, on average, 21.93 messages per day, and they spent 7.61 minutes on average in the time in In Basket per day metric and 1.84 minutes on average in the time in In Basket per appointment metric. The time for the providers to mark messages as done (turnaround time) was on average 11.45 days during the reporting period. Although the surgical group had, on average, approximately twice as many appointments per scheduled day, they spent considerably less connected time (based on almost all time metrics) than the medical group. However, the surgical group took much longer than the medical group to mark messages as done (turnaround time). Conclusions: We observed a range of patterns with no consistent direction. There does not seem to be evidence of a “learning curve,” which would have shown a consistent reduction in time spent on the system over time due to familiarity and experience. While this study does not show how the included metrics could be used as predictors of providers’ satisfaction or feelings of burnout, the use trends could be used to start discussions about future Canadian studies needed in this area. %M 38684079 %R 10.2196/53122 %U https://www.jmir.org/2024/1/e53122 %U https://doi.org/10.2196/53122 %U http://www.ncbi.nlm.nih.gov/pubmed/38684079 %0 Journal Article %@ 2369-2960 %I JMIR Publications %V 10 %N %P e52191 %T The Role of Trust as a Driver of Private-Provider Participation in Disease Surveillance: Cross-Sectional Survey From Nigeria %A Mitchell,Ellen MH %A Adejumo,Olusola Adedeji %A Abdur-Razzaq,Hussein %A Ogbudebe,Chidubem %A Gidado,Mustapha %+ Mycobacterial Diseases and Neglected Tropical Diseases Unit, Department of Public Health, Institute for Tropical Medicine, 155 Nationaalestraat, Roche, Antwerp, 2000, Belgium, 31 634790509, emitchell@itg.be %K surveillance %K trust %K Integrated Disease Surveillance and Response %K IDSR %K tuberculosis %K notification %K public-private mix %K infectious disease %K disease surveillance %K surveillance behavior %K health care worker %K health professional %K public health %K Nigeria %K survey %K behavior %K self-reported %D 2024 %7 25.4.2024 %9 Original Paper %J JMIR Public Health Surveill %G English %X Background: Recognition of the importance of valid, real-time knowledge of infectious disease risk has renewed scrutiny into private providers’ intentions, motives, and obstacles to comply with an Integrated Disease Surveillance Response (IDSR) framework. Appreciation of how private providers’ attitudes shape their tuberculosis (TB) notification behaviors can yield lessons for the surveillance of emerging pathogens, antibiotic stewardship, and other crucial public health functions. Reciprocal trust among actors and institutions is an understudied part of the “software” of surveillance. Objective: We aimed to assess the self-reported knowledge, motivation, barriers, and TB case notification behavior of private health care providers to public health authorities in Lagos, Nigeria. We measured the concordance between self-reported notification, TB cases found in facility records, and actual notifications received. Methods: A representative, stratified sample of 278 private health care workers was surveyed on TB notification attitudes, behavior, and perceptions of public health authorities using validated scales. Record reviews were conducted to identify the TB treatment provided and facility case counts were abstracted from the records. Self-reports were triangulated against actual notification behavior for 2016. The complex health system framework was used to identify potential predictors of notification behavior. Results: Noncompliance with the legal obligations to notify infectious diseases was not attributable to a lack of knowledge. Private providers who were uncomfortable notifying TB cases via the IDSR system scored lower on the perceived benevolence subscale of trust. Health care workers who affirmed “always” notifying via IDSR monthly reported higher median trust in the state’s public disease control capacity. Although self-reported notification behavior was predicted by age, gender, and positive interaction with public health bodies, the self-report numbers did not tally with actual TB notifications. Conclusions: Providers perceived both risks and benefits to recording and reporting TB cases. To improve private providers’ public health behaviors, policy makers need to transcend instrumental and transactional approaches to surveillance to include building trust in public health, simplifying the task, and enhancing the link to improved health. Renewed attention to the “software” of health systems (eg, norms, values, and relationships) is vital to address pandemic threats. Surveys with private providers may overestimate their actual participation in public health surveillance. %M 38506095 %R 10.2196/52191 %U https://publichealth.jmir.org/2024/1/e52191 %U https://doi.org/10.2196/52191 %U http://www.ncbi.nlm.nih.gov/pubmed/38506095 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 11 %N %P e52592 %T Academic Detailing as a Health Information Technology Implementation Method: Supporting the Design and Implementation of an Emergency Department–Based Clinical Decision Support Tool to Prevent Future Falls %A Barton,Hanna J %A Maru,Apoorva %A Leaf,Margaret A %A Hekman,Daniel J %A Wiegmann,Douglas A %A Shah,Manish N %A Patterson,Brian W %+ BerbeeWalsh Department of Emergency Medicine, University of Wisconsin-Madison, 800 University Bay Dr., Madison, WI, 53705, United States, 1 (608) 890 8682, hbarton@wisc.edu %K emergency medicine %K clinical decision support %K health IT %K human factors %K work systems %K SEIPS %K Systems Engineering Initiative for Patient Safety %K educational outreach %K academic detailing %K implementation method %K department-based %K CDS %K clinical care %K evidence-based %K CDS tool %K gerontology %K geriatric %K geriatrics %K older adult %K older adults %K elder %K elderly %K older person %K older people %K preventative intervention %K team-based analysis %K machine learning %K high-risk patient %K high-risk patients %K pharmaceutical %K pharmaceutical sales %K United States %K fall-risk prediction %K EHR %K electronic health record %K interview %K ED environment %K emergency department %D 2024 %7 18.4.2024 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Clinical decision support (CDS) tools that incorporate machine learning–derived content have the potential to transform clinical care by augmenting clinicians’ expertise. To realize this potential, such tools must be designed to fit the dynamic work systems of the clinicians who use them. We propose the use of academic detailing—personal visits to clinicians by an expert in a specific health IT tool—as a method for both ensuring the correct understanding of that tool and its evidence base and identifying factors influencing the tool’s implementation. Objective: This study aimed to assess academic detailing as a method for simultaneously ensuring the correct understanding of an emergency department–based CDS tool to prevent future falls and identifying factors impacting clinicians’ use of the tool through an analysis of the resultant qualitative data. Methods: Previously, our team designed a CDS tool to identify patients aged 65 years and older who are at the highest risk of future falls and prompt an interruptive alert to clinicians, suggesting the patient be referred to a mobility and falls clinic for an evidence-based preventative intervention. We conducted 10-minute academic detailing interviews (n=16) with resident emergency medicine physicians and advanced practice providers who had encountered our CDS tool in practice. We conducted an inductive, team-based content analysis to identify factors that influenced clinicians’ use of the CDS tool. Results: The following categories of factors that impacted clinicians’ use of the CDS were identified: (1) aspects of the CDS tool’s design (2) clinicians’ understanding (or misunderstanding) of the CDS or referral process, (3) the busy nature of the emergency department environment, (4) clinicians’ perceptions of the patient and their associated fall risk, and (5) the opacity of the referral process. Additionally, clinician education was done to address any misconceptions about the CDS tool or referral process, for example, demonstrating how simple it is to place a referral via the CDS and clarifying which clinic the referral goes to. Conclusions: Our study demonstrates the use of academic detailing for supporting the implementation of health information technologies, allowing us to identify factors that impacted clinicians’ use of the CDS while concurrently educating clinicians to ensure the correct understanding of the CDS tool and intervention. Thus, academic detailing can inform both real-time adjustments of a tool’s implementation, for example, refinement of the language used to introduce the tool, and larger scale redesign of the CDS tool to better fit the dynamic work environment of clinicians. %M 38635318 %R 10.2196/52592 %U https://humanfactors.jmir.org/2024/1/e52592 %U https://doi.org/10.2196/52592 %U http://www.ncbi.nlm.nih.gov/pubmed/38635318 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 11 %N %P e52625 %T Influence of Disease-Related Stigma on Patients’ Decisions to Upload Medical Reports to the German Electronic Health Record: Randomized Controlled Trial %A von Kalckreuth,Niklas %A Feufel,Markus A %+ Division of Ergonomics, Department of Psychology and Ergonomics, Technische Universität Berlin, Straße des 17 Juni 135, Berlin, 10623, Germany, 49 3031470747, niklas.vkalckreuth@tu-berlin.de %K electronic health record %K EHR %K technology acceptance %K upload behavior %K health-related stigma %K intention to use %K intention-behavior gap %K medical reports %K stigma %K Germany %K patient decision %K digital transformation %K implementation %K risk %K decision %K risk perception %K social stigma %K safety %D 2024 %7 10.4.2024 %9 Original Paper %J JMIR Hum Factors %G English %X Background: The rollout of the electronic health record (EHR) represents a central component of the digital transformation of the German health care system. Although the EHR promises more effective, safer, and faster treatment of patients from a systems perspective, the successful implementation of the EHR largely depends on the patient. In a recent survey, 3 out of 4 Germans stated that they intend to use the EHR, whereas other studies show that the intention to use a technology is not a reliable and sufficient predictor of actual use. Objective: Controlling for patients’ intention to use the EHR, we investigated whether disease-specific risk perceptions related to the time course of the disease and disease-related stigma explain the additional variance in patients’ decisions to upload medical reports to the EHR. Methods: In an online user study, 241 German participants were asked to interact with a randomly assigned medical report that varied systematically in terms of disease-related stigma (high vs low) and disease time course (acute vs chronic) and to decide whether to upload it to the EHR. Results: Disease-related stigma (odds ratio 0.154, P<.001) offset the generally positive relationship between intention to use and the upload decision (odds ratio 2.628, P<.001), whereas the disease time course showed no effect. Conclusions: Even if patients generally intend to use the EHR, risk perceptions such as those related to diseases associated with social stigma may deter people from uploading related medical reports to the EHR. To ensure the reliable use of this key technology in a digitalized health care system, transparent and easy-to-comprehend information about the safety standards of the EHR are warranted across the board, even for populations that are generally in favor of using the EHR. %M 38598271 %R 10.2196/52625 %U https://humanfactors.jmir.org/2024/1/e52625 %U https://doi.org/10.2196/52625 %U http://www.ncbi.nlm.nih.gov/pubmed/38598271 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 11 %N %P e46698 %T Usability of an Automated System for Real-Time Monitoring of Shared Decision-Making for Surgery: Mixed Methods Evaluation %A Hoffmann,Christin %A Avery,Kerry %A Macefield,Rhiannon %A Dvořák,Tadeáš %A Snelgrove,Val %A Blazeby,Jane %A Hopkins,Della %A Hickey,Shireen %A Gibbison,Ben %A Rooshenas,Leila %A Williams,Adam %A Aning,Jonathan %A Bekker,Hilary L %A McNair,Angus GK %A , %+ National Institute for Health and Care Research Bristol Biomedical Research Centre, Bristol Centre for Surgical Research, Bristol Medical School: Population Health Sciences, University of Bristol, Canynge Hall, 39 Whatley Road, Bristol, BS8 2PS, United Kingdom, 44 117 455 5993, c.hoffmann@bristol.ac.uk %K surgery %K shared decision-making %K patient participation %K mixed methods %K surgery %K real-time measurement %K patient-reported measure %K electronic data collection %K usability %K data collection %K patient reported %K satisfaction %K mobile phone %D 2024 %7 10.4.2024 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Improving shared decision-making (SDM) for patients has become a health policy priority in many countries. Achieving high-quality SDM is particularly important for approximately 313 million surgical treatment decisions patients make globally every year. Large-scale monitoring of surgical patients’ experience of SDM in real time is needed to identify the failings of SDM before surgery is performed. We developed a novel approach to automating real-time data collection using an electronic measurement system to address this. Examining usability will facilitate its optimization and wider implementation to inform interventions aimed at improving SDM. Objective: This study examined the usability of an electronic real-time measurement system to monitor surgical patients’ experience of SDM. We aimed to evaluate the metrics and indicators relevant to system effectiveness, system efficiency, and user satisfaction. Methods: We performed a mixed methods usability evaluation using multiple participant cohorts. The measurement system was implemented in a large UK hospital to measure patients’ experience of SDM electronically before surgery using 2 validated measures (CollaboRATE and SDM-Q-9). Quantitative data (collected between April 1 and December 31, 2021) provided measurement system metrics to assess system effectiveness and efficiency. We included adult patients booked for urgent and elective surgery across 7 specialties and excluded patients without the capacity to consent for medical procedures, those without access to an internet-enabled device, and those undergoing emergency or endoscopic procedures. Additional groups of service users (group 1: public members who had not engaged with the system; group 2: a subset of patients who completed the measurement system) completed user-testing sessions and semistructured interviews to assess system effectiveness and user satisfaction. We conducted quantitative data analysis using descriptive statistics and calculated the task completion rate and survey response rate (system effectiveness) as well as the task completion time, task efficiency, and relative efficiency (system efficiency). Qualitative thematic analysis identified indicators of and barriers to good usability (user satisfaction). Results: A total of 2254 completed surveys were returned to the measurement system. A total of 25 service users (group 1: n=9; group 2: n=16) participated in user-testing sessions and interviews. The task completion rate was high (169/171, 98.8%) and the survey response rate was good (2254/5794, 38.9%). The median task completion time was 3 (IQR 2-13) minutes, suggesting good system efficiency and effectiveness. The qualitative findings emphasized good user satisfaction. The identified themes suggested that the measurement system is acceptable, easy to use, and easy to access. Service users identified potential barriers and solutions to acceptability and ease of access. Conclusions: A mixed methods evaluation of an electronic measurement system for automated, real-time monitoring of patients’ experience of SDM showed that usability among patients was high. Future pilot work will optimize the system for wider implementation to ultimately inform intervention development to improve SDM. International Registered Report Identifier (IRRID): RR2-10.1136/bmjopen-2023-079155 %M 38598276 %R 10.2196/46698 %U https://humanfactors.jmir.org/2024/1/e46698 %U https://doi.org/10.2196/46698 %U http://www.ncbi.nlm.nih.gov/pubmed/38598276 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 13 %N %P e54787 %T Understanding Physician’s Perspectives on AI in Health Care: Protocol for a Sequential Multiple Assignment Randomized Vignette Study %A Kim,Jane Paik %A Yang,Hyun-Joon %A Kim,Bohye %A Ryan,Katie %A Roberts,Laura Weiss %+ Department of Psychiatry and Behavioral Sciences, Stanford University School of Medicine, 1070 Arastradero Rd, Stanford, CA, 94304, United States, 1 650 736 8996, janepkim@stanford.edu %K AI-based clinical decision support %K decision-making %K hypothetical vignettes %K physician perspective %K web-based survey %K hypothesis-driven research %K ethics %K stakeholder attitudes %D 2024 %7 4.4.2024 %9 Protocol %J JMIR Res Protoc %G English %X Background: As the availability and performance of artificial intelligence (AI)–based clinical decision support (CDS) systems improve, physicians and other care providers poised to be on the front lines will be increasingly tasked with using these tools in patient care and incorporating their outputs into clinical decision-making processes. Vignette studies provide a means to explore emerging hypotheses regarding how context-specific factors, such as clinical risk, the amount of information provided about the AI, and the AI result, may impact physician acceptance and use of AI-based CDS tools. To best anticipate how such factors influence the decision-making of frontline physicians in clinical scenarios involving AI decision-support tools, hypothesis-driven research is needed that enables scenario testing before the implementation and deployment of these tools. Objective: This study’s objectives are to (1) design an original, web-based vignette-based survey that features hypothetical scenarios based on emerging or real-world applications of AI-based CDS systems that will vary systematically by features related to clinical risk, the amount of information provided about the AI, and the AI result; and (2) test and determine causal effects of specific factors on the judgments and perceptions salient to physicians’ clinical decision-making. Methods: US-based physicians with specialties in family or internal medicine will be recruited through email and mail (target n=420). Through a web-based survey, participants will be randomized to a 3-part “sequential multiple assignment randomization trial (SMART) vignette” detailing a hypothetical clinical scenario involving an AI decision support tool. The SMART vignette design is similar to the SMART design but adapted to a survey design. Each respondent will be randomly assigned to 1 of the possible vignette variations of the factors we are testing at each stage, which include the level of clinical risk, the amount of information provided about the AI, and the certainty of the AI output. Respondents will be given questions regarding their hypothetical decision-making in response to the hypothetical scenarios. Results: The study is currently in progress and data collection is anticipated to be completed in 2024. Conclusions: The web-based vignette study will provide information on how contextual factors such as clinical risk, the amount of information provided about an AI tool, and the AI result influence physicians’ reactions to hypothetical scenarios that are based on emerging applications of AI in frontline health care settings. Our newly proposed “SMART vignette” design offers several benefits not afforded by the extensively used traditional vignette design, due to the 2 aforementioned features. These advantages are (1) increased validity of analyses targeted at understanding the impact of a factor on the decision outcome, given previous outcomes and other contextual factors; and (2) balanced sample sizes across groups. This study will generate a better understanding of physician decision-making within this context. International Registered Report Identifier (IRRID): DERR1-10.2196/54787 %M 38573756 %R 10.2196/54787 %U https://www.researchprotocols.org/2024/1/e54787 %U https://doi.org/10.2196/54787 %U http://www.ncbi.nlm.nih.gov/pubmed/38573756 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 11 %N %P e55802 %T Clinical Decision Support Requirements for Ventricular Tachycardia Diagnosis Within the Frameworks of Knowledge and Practice: Survey Study %A Hu,Zhao %A Wang,Min %A Zheng,Si %A Xu,Xiaowei %A Zhang,Zhuxin %A Ge,Qiaoyue %A Li,Jiao %A Yao,Yan %+ Arrhythmia Center, Fuwai Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College/National Center for Cardiovascular Diseases, Beilishi Road 167, Beijing, 100037, China, 86 10 88322401, ianyao@263.net.cn %K clinical decision support system %K requirements analysis %K ventricular tachycardia %K knowledge %K clinical practice %K questionnaires %D 2024 %7 26.3.2024 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Ventricular tachycardia (VT) diagnosis is challenging due to the similarity between VT and some forms of supraventricular tachycardia, complexity of clinical manifestations, heterogeneity of underlying diseases, and potential for life-threatening hemodynamic instability. Clinical decision support systems (CDSSs) have emerged as promising tools to augment the diagnostic capabilities of cardiologists. However, a requirements analysis is acknowledged to be vital for the success of a CDSS, especially for complex clinical tasks such as VT diagnosis. Objective: The aims of this study were to analyze the requirements for a VT diagnosis CDSS within the frameworks of knowledge and practice and to determine the clinical decision support (CDS) needs. Methods: Our multidisciplinary team first conducted semistructured interviews with seven cardiologists related to the clinical challenges of VT and expected decision support. A questionnaire was designed by the multidisciplinary team based on the results of interviews. The questionnaire was divided into four sections: demographic information, knowledge assessment, practice assessment, and CDS needs. The practice section consisted of two simulated cases for a total score of 10 marks. Online questionnaires were disseminated to registered cardiologists across China from December 2022 to February 2023. The scores for the practice section were summarized as continuous variables, using the mean, median, and range. The knowledge and CDS needs sections were assessed using a 4-point Likert scale without a neutral option. Kruskal-Wallis tests were performed to investigate the relationship between scores and practice years or specialty. Results: Of the 687 cardiologists who completed the questionnaire, 567 responses were eligible for further analysis. The results of the knowledge assessment showed that 383 cardiologists (68%) lacked knowledge in diagnostic evaluation. The overall average score of the practice assessment was 6.11 (SD 0.55); the etiological diagnosis section had the highest overall scores (mean 6.74, SD 1.75), whereas the diagnostic evaluation section had the lowest scores (mean 5.78, SD 1.19). A majority of cardiologists (344/567, 60.7%) reported the need for a CDSS. There was a significant difference in practice competency scores between general cardiologists and arrhythmia specialists (P=.02). Conclusions: There was a notable deficiency in the knowledge and practice of VT among Chinese cardiologists. Specific knowledge and practice support requirements were identified, which provide a foundation for further development and optimization of a CDSS. Moreover, it is important to consider clinicians’ specialization levels and years of practice for effective and personalized support. %M 38530337 %R 10.2196/55802 %U https://humanfactors.jmir.org/2024/1/e55802 %U https://doi.org/10.2196/55802 %U http://www.ncbi.nlm.nih.gov/pubmed/38530337 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 11 %N %P e41557 %T Exploring the Use of Persuasive System Design Principles to Enhance Medication Incident Reporting and Learning Systems: Scoping Reviews and Persuasive Design Assessment %A Oyibo,Kiemute %A Gonzalez,Paola A %A Ejaz,Sarah %A Naheyan,Tasneem %A Beaton,Carla %A O’Donnell,Denis %A Barker,James R %+ Department of Electrical Engineering and Computer Science, Lassonde Research Centre, York University, 4751 Keele Street, North York, ON, M3J 2N9, Canada, 1 416 736 5053, kiemute.oyibo@yorku.ca %K medication incident %K reporting system %K persuasive technology %K persuasive design %K medication %K persuasive system design %K pharmacy %K pharmaceutic %K pharmacology %K drug reporting %K drug event %K adverse event %K incident management %D 2024 %7 21.3.2024 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Medication incidents (MIs) causing harm to patients have far-reaching consequences for patients, pharmacists, public health, business practice, and governance policy. Medication Incident Reporting and Learning Systems (MIRLS) have been implemented to mitigate such incidents and promote continuous quality improvement in community pharmacies in Canada. They aim to collect and analyze MIs for the implementation of incident preventive strategies to increase safety in community pharmacy practice. However, this goal remains inhibited owing to the persistent barriers that pharmacies face when using these systems. Objective: This study aims to investigate the harms caused by medication incidents and technological barriers to reporting and identify opportunities to incorporate persuasive design strategies in MIRLS to motivate reporting. Methods: We conducted 2 scoping reviews to provide insights on the relationship between medication errors and patient harm and the information system–based barriers militating against reporting. Seven databases were searched in each scoping review, including PubMed, Public Health Database, ProQuest, Scopus, ACM Library, Global Health, and Google Scholar. Next, we analyzed one of the most widely used MIRLS in Canada using the Persuasive System Design (PSD) taxonomy—a framework for analyzing, designing, and evaluating persuasive systems. This framework applies behavioral theories from social psychology in the design of technology-based systems to motivate behavior change. Independent assessors familiar with MIRLS reported the degree of persuasion built into the system using the 4 categories of PSD strategies: primary task, dialogue, social, and credibility support. Results: Overall, 17 articles were included in the first scoping review, and 1 article was included in the second scoping review. In the first review, significant or serious harm was the most frequent harm (11/17, 65%), followed by death or fatal harm (7/17, 41%). In the second review, the authors found that iterative design could improve the usability of an MIRLS; however, data security and validation of reports remained an issue to be addressed. Regarding the MIRLS that we assessed, participants considered most of the primary task, dialogue, and credibility support strategies in the PSD taxonomy as important and useful; however, they were not comfortable with some of the social strategies such as cooperation. We found that the assessed system supported a number of persuasive strategies from the PSD taxonomy; however, we identified additional strategies such as tunneling, simulation, suggestion, praise, reward, reminder, authority, and verifiability that could further enhance the perceived persuasiveness and value of the system. Conclusions: MIRLS, equipped with persuasive features, can become powerful motivational tools to promote safer medication practices in community pharmacies. They have the potential to highlight the value of MI reporting and increase the readiness of pharmacists to report incidents. The proposed persuasive design guidelines can help system developers and community pharmacy managers realize more effective MIRLS. %M 38512325 %R 10.2196/41557 %U https://humanfactors.jmir.org/2024/1/e41557 %U https://doi.org/10.2196/41557 %U http://www.ncbi.nlm.nih.gov/pubmed/38512325 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 13 %N %P e53627 %T Data Visualization Support for Tumor Boards and Clinical Oncology: Protocol for a Scoping Review %A Boehm,Dominik %A Strantz,Cosima %A Christoph,Jan %A Busch,Hauke %A Ganslandt,Thomas %A Unberath,Philipp %+ Medical Center for Information and Communication Technology, Universitätsklinikum Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Glückstraße 11, Erlangen, 91054, Germany, 49 91318546478, dominik.boehm@uk-erlangen.de %K clinical oncology %K tumor board %K cancer conference %K multidisciplinary %K visualization %K software %K tool %K scoping review %K tumor %K malignant %K benign %K data sets %K oncology %K interactive visualization %K data %K patient %K patients %K physicians %K medical practitioners %K medical practitioner %K conference %D 2024 %7 5.3.2024 %9 Protocol %J JMIR Res Protoc %G English %X Background: Complex and expanding data sets in clinical oncology applications require flexible and interactive visualization of patient data to provide the maximum amount of information to physicians and other medical practitioners. Interdisciplinary tumor conferences in particular profit from customized tools to integrate, link, and visualize relevant data from all professions involved. Objective: The scoping review proposed in this protocol aims to identify and present currently available data visualization tools for tumor boards and related areas. The objective of the review will be to provide not only an overview of digital tools currently used in tumor board settings, but also the data included, the respective visualization solutions, and their integration into hospital processes. Methods: The planned scoping review process is based on the Arksey and O’Malley scoping study framework. The following electronic databases will be searched for articles published in English: PubMed, Web of Knowledge, and SCOPUS. Eligible articles will first undergo a deduplication step, followed by the screening of titles and abstracts. Second, a full-text screening will be used to reach the final decision about article selection. At least 2 reviewers will independently screen titles, abstracts, and full-text reports. Conflicting inclusion decisions will be resolved by a third reviewer. The remaining literature will be analyzed using a data extraction template proposed in this protocol. The template includes a variety of meta information as well as specific questions aiming to answer the research question: “What are the key features of data visualization solutions used in molecular and organ tumor boards, and how are these elements integrated and used within the clinical setting?” The findings will be compiled, charted, and presented as specified in the scoping study framework. Data for included tools may be supplemented with additional manual literature searches. The entire review process will be documented in alignment with the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) flowchart. Results: The results of this scoping review will be reported per the expanded PRISMA-ScR guidelines. A preliminary search using PubMed, Web of Knowledge, and Scopus resulted in 1320 articles after deduplication that will be included in the further review process. We expect the results to be published during the second quarter of 2024. Conclusions: Visualization is a key process in leveraging a data set’s potentially available information and enabling its use in an interdisciplinary setting. The scoping review described in this protocol aims to present the status quo of visualization solutions for tumor board and clinical oncology applications and their integration into hospital processes. International Registered Report Identifier (IRRID): DERR1-10.2196/53627 %M 38441925 %R 10.2196/53627 %U https://www.researchprotocols.org/2024/1/e53627 %U https://doi.org/10.2196/53627 %U http://www.ncbi.nlm.nih.gov/pubmed/38441925 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 11 %N %P e47081 %T Intention to Use an Electronic Community Health Information System Among Health Extension Workers in Rural Northwest Ethiopia: Cross-Sectional Study Using the Unified Theory of Acceptance and Use of Technology 2 Model %A Hailemariam,Tesfahun %A Atnafu,Asmamaw %A Gezie,Lemma %A Kaasbøll,Jens %A Klein,Jorn %A Tilahun,Binyam %+ Department of Health Informatics, Institute of Public Health, College of Medicine and Health Sciences, University of Gondar, Chechela street, Gondar, 196, Ethiopia, 251 913305250, tesfahunhailemariam@gmail.com %K data capturing %K data use %K eCHIS %K electronic community health information system %K health extension worker %K HEW %K intention to use %K service provision %K Unified Theory of Acceptance and Use of Technology 2 %K UTAUT2 model %D 2024 %7 4.3.2024 %9 Original Paper %J JMIR Hum Factors %G English %X Background: IT has brought remarkable change in bridging the digital gap in resource-constrained regions and advancing the health care system worldwide. Community-based information systems and mobile apps have been extensively developed and deployed to quantify and support health services delivered by community health workers. The success and failure of a digital health information system depends on whether and how it is used. Ethiopia is scaling up its electronic community health information system (eCHIS) to support the work of health extension workers (HEWs). For successful implementation, more evidence was required about the factors that may affect the willingness of HEWs to use the eCHIS. Objective: This study aimed to assess HEWs’ intentions to use the eCHIS for health data management and service provision. Methods: A cross-sectional study design was conducted among 456 HEWs in 6 pilot districts of the Central Gondar zone, Northwest Ethiopia. A Unified Theory of Acceptance and Use of Technology model was used to investigate HEWs’ intention to use the eCHIS. Data were cleaned, entered into Epi-data (version 4.02; EpiData Association), and exported to SPSS (version 26; IBM Corp) for analysis using the AMOS 23 Structural Equation Model. The statistical significance of dependent and independent variables in the model was reported using a 95% CI with a corresponding P value of <.05. Results: A total of 456 HEWs participated in the study, with a response rate of 99%. The mean age of the study participants was 28 (SD 4.8) years. Our study revealed that about 179 (39.3%; 95% CI 34.7%-43.9%) participants intended to use the eCHIS for community health data generation, use, and service provision. Effort expectancy (β=0.256; P=.007), self-expectancy (β=0.096; P=.04), social influence (β=0.203; P=.02), and hedonic motivation (β=0.217; P=.03) were significantly associated with HEWs’ intention to use the eCHIS. Conclusions: HEWs need to be computer literate and understand their role with the eCHIS. Ensuring that the system is easy and enjoyable for them to use is important for implementation and effective health data management. %M 38437008 %R 10.2196/47081 %U https://humanfactors.jmir.org/2024/1/e47081 %U https://doi.org/10.2196/47081 %U http://www.ncbi.nlm.nih.gov/pubmed/38437008 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 13 %N %P e51002 %T Enhancing the Efficiency of a Radiation Oncology Department Using Electronic Medical Records: Protocol for Preparing Radiotherapy %A Cheng,Hao-Shen %A You,Weir-Chiang %A Chen,Ni-Wei %A Hsieh,Mu-Chih %A Tsai,Che-Fu %A Ho,Chia-Jing %A Chen,Chien-Chih %+ Taichung Veterans General Hospital, 1650 Taiwan Boulevard Sect. 4, Taichung, 40705, Taiwan, 886 4 2359 2525 ext 5601, chiencheh@gmail.com %K efficiency %K electronic medical records %K Hospital Information System %K protocol %K radiation oncology %D 2024 %7 23.2.2024 %9 Protocol %J JMIR Res Protoc %G English %X Background: Electronic medical records (EMRs) streamline medical processes, improve quality control, and facilitate data sharing among hospital departments. They also reduce maintenance costs and storage space needed for paper records, while saving time and providing structured data for future research. Objective: This study aimed to investigate whether the integration of the radiation oncology information system and the hospital information system enhances the efficiency of the department of radiation oncology. Methods: We held multidisciplinary discussions among physicians, physicists, medical radiation technologists, nurses, and engineers. We integrated paper records from the radiation oncology department into the existing hospital information system within the hospital. A new electronic interface was designed. A comparison was made between the time taken to retrieve information from either the paper records or the EMRs for radiation preparation. A total of 30 cases were randomly allocated in both the old paper-based system and the new EMR system. The time spent was calculated manually at every step during the process, and we performed an independent 1-tailed t test to evaluate the difference between the 2 systems. Results: Since the system was launched in August 2020, more than 1000 medical records have been entered into the system, and this figure continues to increase. The total time needed for the radiation preparation process was reduced from 286.8 minutes to 154.3 minutes (P<.001)—a reduction of 46.2%. There was no longer any need to arrange for a nurse to organize the radiotherapy paper records, saving a workload of 16 hours per month. Conclusions: The implementation of the integrated EMR system has resulted in a significant reduction in the number of steps involved in radiotherapy preparation, as well as a decrease in the amount of time required for the process. The new EMR system has provided numerous benefits for the department, including a decrease in workload, a simplified workflow, and conserving more patient data within a confined space. %M 38393753 %R 10.2196/51002 %U https://www.researchprotocols.org/2024/1/e51002 %U https://doi.org/10.2196/51002 %U http://www.ncbi.nlm.nih.gov/pubmed/38393753 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 11 %N %P e48445 %T Usability and Evaluation of a Health Information System in the Emergency Department: Mixed Methods Study %A Østervang,Christina %A Jensen,Charlotte Myhre %A Coyne,Elisabeth %A Dieperink,Karin B %A Lassen,Annmarie %+ Department of Emergency Medicine, Odense University Hospital, Kløvervænget 25, Odense, 5000, Denmark, 45 21757009, christina.oestervang@rsyd.dk %K consumer %K eHealth %K elderly %K emergency department %K emergency %K family members %K healthcare professionals %K information system %K mixed methods research: patients %K qualitative interview %K questionnaire %K technology %K usability %K usable %D 2024 %7 21.2.2024 %9 Original Paper %J JMIR Hum Factors %G English %X Background: A lack of information during an emergency visit leads to the experience of powerlessness for patients and their family members, who may also feel unprepared to cope with acute symptoms. The ever-changing nature and fast-paced workflow in the emergency department (ED) often affect how health care professionals can tailor information and communication to the needs of the patient. Objective: This study aimed to evaluate the usability and experience of a newly developed information system. The system was developed together with patients and their family members to help provide the information needed in the ED. Methods: We conducted a mixed methods study consisting of quantitative data obtained from the System Usability Scale questionnaire and qualitative interview data obtained from purposively selected participants included in the quantitative part of the study. Results: A total of 106 patients and 14 family members (N=120) answered the questionnaire. A total of 10 patients and 3 family members participated in the interviews. Based on the System Usability Scale score, the information system was rated close to excellent, with a mean score of 83.6 (SD 12.8). Most of the participants found the information system easy to use and would like to use it again. The participants reported that the system helped them feel in control, and the information was useful. Simplifications were needed to improve the user experience for the older individuals. Conclusions: This study demonstrates that the usability of the information system is rated close to excellent. It was perceived to be useful as it enabled understanding and predictability of the patient’s trajectory in the ED. Areas for improvement include making the system more usable by older individuals. The study provides an example of how a technological solution can be used to diminish the information gap in an ED context. %M 38381502 %R 10.2196/48445 %U https://humanfactors.jmir.org/2024/1/e48445 %U https://doi.org/10.2196/48445 %U http://www.ncbi.nlm.nih.gov/pubmed/38381502 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e49497 %T Data Representation Structure to Support Clinical Decision-Making in the Pediatric Intensive Care Unit: Interview Study and Preliminary Decision Support Interface Design %A Yakob,Najia %A Laliberté,Sandrine %A Doyon-Poulin,Philippe %A Jouvet,Philippe %A Noumeir,Rita %+ Pediatric Intensive Care Unit, CHU Sainte-Justine, 3175 Côte-Sainte-Catherine, Montreal, QC, H3T 1C5, Canada, 1 514 345 4927, philippe.jouvet@umontreal.ca %K data representation %K decision support %K critical care %K clinical workflow %K clinical decision-making %K prototype %K design %K intensive care unit %D 2024 %7 1.2.2024 %9 Original Paper %J JMIR Form Res %G English %X Background: Clinical decision-making is a complex cognitive process that relies on the interpretation of a large variety of data from different sources and involves the use of knowledge bases and scientific recommendations. The representation of clinical data plays a key role in the speed and efficiency of its interpretation. In addition, the increasing use of clinical decision support systems (CDSSs) provides assistance to clinicians in their practice, allowing them to improve patient outcomes. In the pediatric intensive care unit (PICU), clinicians must process high volumes of data and deal with ever-growing workloads. As they use multiple systems daily to assess patients’ status and to adjust the health care plan, including electronic health records (EHR), clinical systems (eg, laboratory, imaging and pharmacy), and connected devices (eg, bedside monitors, mechanical ventilators, intravenous pumps, and syringes), clinicians rely mostly on their judgment and ability to trace relevant data for decision-making. In these circumstances, the lack of optimal data structure and adapted visual representation hinder clinician’s cognitive processes and clinical decision-making skills. Objective: In this study, we designed a prototype to optimize the representation of clinical data collected from existing sources (eg, EHR, clinical systems, and devices) via a structure that supports the integration of a home-developed CDSS in the PICU. This study was based on analyzing end user needs and their clinical workflow. Methods: First, we observed clinical activities in a PICU to secure a better understanding of the workflow in terms of staff tasks and their use of EHR on a typical work shift. Second, we conducted interviews with 11 clinicians from different staff categories (eg, intensivists, fellows, nurses, and nurse practitioners) to compile their needs for decision support. Third, we structured the data to design a prototype that illustrates the proposed representation. We used a brain injury care scenario to validate the relevance of integrated data and the utility of main functionalities in a clinical context. Fourth, we held design meetings with 5 clinicians to present, revise, and adapt the prototype to meet their needs. Results: We created a structure with 3 levels of abstraction—unit level, patient level, and system level—to optimize clinical data representation and display for efficient patient assessment and to provide a flexible platform to host the internally developed CDSS. Subsequently, we designed a preliminary prototype based on this structure. Conclusions: The data representation structure allows prioritizing patients via criticality indicators, assessing their conditions using a personalized dashboard, and monitoring their courses based on the evolution of clinical values. Further research is required to define and model the concepts of criticality, problem recognition, and evolution. Furthermore, feasibility tests will be conducted to ensure user satisfaction. %M 38300695 %R 10.2196/49497 %U https://formative.jmir.org/2024/1/e49497 %U https://doi.org/10.2196/49497 %U http://www.ncbi.nlm.nih.gov/pubmed/38300695 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e49347 %T Evaluation of Eligibility Criteria Relevance for the Purpose of IT-Supported Trial Recruitment: Descriptive Quantitative Analysis %A Blasini,Romina %A Strantz,Cosima %A Gulden,Christian %A Helfer,Sven %A Lidke,Jakub %A Prokosch,Hans-Ulrich %A Sohrabi,Keywan %A Schneider,Henning %+ Institute of Medical Informatics, Justus Liebig University, Rudolf-Buchheim-Strasse 6, Giessen, 35392, Germany, 49 06419941386, romina.blasini@informatik.med.uni-giessen.de %K CTRSS %K clinical trial recruitment support system %K PRS %K patient recruitment system %K clinical trials %K classifications %K data groups %K data elements %K data classification %K criteria %K relevance %K automated clinical trials %K participants %K clinical trial %D 2024 %7 31.1.2024 %9 Original Paper %J JMIR Form Res %G English %X Background: Clinical trials (CTs) are crucial for medical research; however, they frequently fall short of the requisite number of participants who meet all eligibility criteria (EC). A clinical trial recruitment support system (CTRSS) is developed to help identify potential participants by performing a search on a specific data pool. The accuracy of the search results is directly related to the quality of the data used for comparison. Data accessibility can present challenges, making it crucial to identify the necessary data for a CTRSS to query. Prior research has examined the data elements frequently used in CT EC but has not evaluated which criteria are actually used to search for participants. Although all EC must be met to enroll a person in a CT, not all criteria have the same importance when searching for potential participants in an existing data pool, such as an electronic health record, because some of the criteria are only relevant at the time of enrollment. Objective: In this study, we investigated which groups of data elements are relevant in practice for finding suitable participants and whether there are typical elements that are not relevant and can therefore be omitted. Methods: We asked trial experts and CTRSS developers to first categorize the EC of their CTs according to data element groups and then to classify them into 1 of 3 categories: necessary, complementary, and irrelevant. In addition, the experts assessed whether a criterion was documented (on paper or digitally) or whether it was information known only to the treating physicians or patients. Results: We reviewed 82 CTs with 1132 unique EC. Of these 1132 EC, 350 (30.9%) were considered necessary, 224 (19.8%) complementary, and 341 (30.1%) total irrelevant. To identify the most relevant data elements, we introduced the data element relevance index (DERI). This describes the percentage of studies in which the corresponding data element occurs and is also classified as necessary or supplementary. We found that the query of “diagnosis” was relevant for finding participants in 79 (96.3%) of the CTs. This group was followed by “date of birth/age” with a DERI of 85.4% (n=70) and “procedure” with a DERI of 35.4% (n=29). Conclusions: The distribution of data element groups in CTs has been heterogeneously described in previous works. Therefore, we recommend identifying the percentage of CTs in which data element groups can be found as a more reliable way to determine the relevance of EC. Only necessary and complementary criteria should be included in this DERI. %M 38294862 %R 10.2196/49347 %U https://formative.jmir.org/2024/1/e49347 %U https://doi.org/10.2196/49347 %U http://www.ncbi.nlm.nih.gov/pubmed/38294862 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e46857 %T Natural Language Processing of Referral Letters for Machine Learning–Based Triaging of Patients With Low Back Pain to the Most Appropriate Intervention: Retrospective Study %A Fudickar,Sebastian %A Bantel,Carsten %A Spieker,Jannik %A Töpfer,Heinrich %A Stegeman,Patrick %A Schiphorst Preuper,Henrica R %A Reneman,Michiel F %A Wolff,André P %A Soer,Remko %+ Institute of Medical Informatics, University of Lübeck, Ratzeburger Allee 160, Lübeck, D-23538, Germany, 49 160 7979077, sebastian.fudickar@uni-luebeck.de %K decision support %K triaging %K NLP %K natural language processing %K neural network %K LBP %K low back pain %K back %K pain %K decision-making %K machine learning %K artificial intelligence %K clinical application %K patient records %K qualitative data %K support system %K questionnaire %K quality of life %K psychosocial %D 2024 %7 30.1.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: Decision support systems (DSSs) for suggesting optimal treatments for individual patients with low back pain (LBP) are currently insufficiently accurate for clinical application. Most of the input provided to train these systems is based on patient-reported outcome measures. However, with the appearance of electronic health records (EHRs), additional qualitative data on reasons for referrals and patients’ goals become available for DSSs. Currently, no decision support tools cover a wide range of biopsychosocial factors, including referral letter information to help clinicians triage patients to the optimal LBP treatment. Objective: The objective of this study was to investigate the added value of including qualitative data from EHRs and referral letters to the accuracy of a quantitative DSS for patients with LBP. Methods: A retrospective study was conducted in a clinical cohort of Dutch patients with LBP. Patients filled out a baseline questionnaire about demographics, pain, disability, work status, quality of life, medication, psychosocial functioning, comorbidity, history, and duration of pain. Referral reasons and patient requests for help (patient goals) were extracted via natural language processing (NLP) and enriched in the data set. For decision support, these data were considered independent factors for triage to neurosurgery, anesthesiology, rehabilitation, or minimal intervention. Support vector machine, k-nearest neighbor, and multilayer perceptron models were trained for 2 conditions: with and without consideration of the referral letter content. The models’ accuracies were evaluated via F1-scores, and confusion matrices were used to predict the treatment path (out of 4 paths) with and without additional referral parameters. Results: Data from 1608 patients were evaluated. The evaluation indicated that 2 referral reasons from the referral letters (for anesthesiology and rehabilitation intervention) increased the F1-score accuracy by up to 19.5% for triaging. The confusion matrices confirmed the results. Conclusions: This study indicates that data enriching by adding NLP-based extraction of the content of referral letters increases the model accuracy of DSSs in suggesting optimal treatments for individual patients with LBP. Overall model accuracies were considered low and insufficient for clinical application. %M 38289669 %R 10.2196/46857 %U https://www.jmir.org/2024/1/e46857 %U https://doi.org/10.2196/46857 %U http://www.ncbi.nlm.nih.gov/pubmed/38289669 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 12 %N %P e53002 %T Application of Failure Mode and Effects Analysis to Improve the Quality of the Front Page of Electronic Medical Records in China: Cross-Sectional Data Mapping Analysis %A Zhan,Siyi %A Ding,Liping %A Li,Hui %A Su,Aonan %+ Zhejiang Provincial People's Hospital, No. 158, Shangtang Rd, Hangzhou, 310000, China, 86 18814885258, suaonan_512917@126.com %K front page %K EMR system %K electronic medical record %K failure mode and effects analysis %K FMEA %K measures %D 2024 %7 19.1.2024 %9 Original Paper %J JMIR Med Inform %G English %X Background: The completeness and accuracy of the front pages of electronic medical records (EMRs) are crucial for evaluating hospital performance and for health insurance payments to inpatients. However, the quality of the first page of EMRs in China's medical system is not satisfactory, which can be partly attributed to deficiencies in the EMR system. Failure mode and effects analysis (FMEA) is a proactive risk management tool that can be used to investigate the potential failure modes in an EMR system and analyze the possible consequences. Objective: The purpose of this study was to preemptively identify the potential failures of the EMR system in China and their causes and effects in order to prevent such failures from recurring. Further, we aimed to implement corresponding improvements to minimize system failure modes. Methods: From January 1, 2020, to May 31, 2022, 10 experts, including clinicians, engineers, administrators, and medical record coders, in Zhejiang People’s Hospital conducted FMEA to improve the quality of the front page of the EMR. The completeness and accuracy of the front page and the risk priority numbers were compared before and after the implementation of specific improvement measures. Results: We identified 2 main processes and 6 subprocesses for improving the EMR system. We found that there were 13 potential failure modes, including data messaging errors, data completion errors, incomplete quality control, and coding errors. A questionnaire survey administered to random physicians and coders showed 7 major causes for these failure modes. Therefore, we established quality control rules for medical records and embedded them in the system. We also integrated the medical insurance system and the front page of the EMR on the same interface and established a set of intelligent front pages in the EMR management system. Further, we revamped the quality management systems such as communicating with physicians regularly and conducting special training seminars. The overall accuracy and integrity rate of the front page (P<.001) of the EMR increased significantly after implementation of the improvement measures, while the risk priority number decreased. Conclusions: In this study, we were able to identify the potential failure modes in the front page of the EMR system by using the FMEA method and implement corresponding improvement measures in order to minimize recurring errors in the health care services in China. %M 38241064 %R 10.2196/53002 %U https://medinform.jmir.org/2024/1/e53002 %U https://doi.org/10.2196/53002 %U http://www.ncbi.nlm.nih.gov/pubmed/38241064 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 12 %N %P e47761 %T The Implementation of an Electronic Medical Record in a German Hospital and the Change in Completeness of Documentation: Longitudinal Document Analysis %A Wurster,Florian %A Beckmann,Marina %A Cecon-Stabel,Natalia %A Dittmer,Kerstin %A Hansen,Till Jes %A Jaschke,Julia %A Köberlein-Neu,Juliane %A Okumu,Mi-Ran %A Rusniok,Carsten %A Pfaff,Holger %A Karbach,Ute %+ Chair of Quality Development and Evaluation in Rehabilitation, Institute of Medical Sociology, Health Services Research, and Rehabilitation Science, Faculty of Human Sciences & Faculty of Medicine and University Hospital Cologne, University of Cologne, Eupener Str. 129, Cologne, 50933, Germany, 49 22147897116, florian.wurster@uni-koeln.de %K clinical documentation %K digital transformation %K document analysis %K electronic medical record %K EMR %K Germany %K health services research %K hospital %K implementation %D 2024 %7 19.1.2024 %9 Original Paper %J JMIR Med Inform %G English %X Background: Electronic medical records (EMR) are considered a key component of the health care system’s digital transformation. The implementation of an EMR promises various improvements, for example, in the availability of information, coordination of care, or patient safety, and is required for big data analytics. To ensure those possibilities, the included documentation must be of high quality. In this matter, the most frequently described dimension of data quality is the completeness of documentation. In this regard, little is known about how and why the completeness of documentation might change after the implementation of an EMR. Objective: This study aims to compare the completeness of documentation in paper-based medical records and EMRs and to discuss the possible impact of an EMR on the completeness of documentation. Methods: A retrospective document analysis was conducted, comparing the completeness of paper-based medical records and EMRs. Data were collected before and after the implementation of an EMR on an orthopaedical ward in a German academic teaching hospital. The anonymized records represent all treated patients for a 3-week period each. Unpaired, 2-tailed t tests, chi-square tests, and relative risks were calculated to analyze and compare the mean completeness of the 2 record types in general and of 10 specific items in detail (blood pressure, body temperature, diagnosis, diet, excretions, height, pain, pulse, reanimation status, and weight). For this purpose, each of the 10 items received a dichotomous score of 1 if it was documented on the first day of patient care on the ward; otherwise, it was scored as 0. Results: The analysis consisted of 180 medical records. The average completeness was 6.25 (SD 2.15) out of 10 in the paper-based medical record, significantly rising to an average of 7.13 (SD 2.01) in the EMR (t178=–2.469; P=.01; d=–0.428). When looking at the significant changes of the 10 items in detail, the documentation of diet (P<.001), height (P<.001), and weight (P<.001) was more complete in the EMR, while the documentation of diagnosis (P<.001), excretions (P=.02), and pain (P=.008) was less complete in the EMR. The completeness remained unchanged for the documentation of pulse (P=.28), blood pressure (P=.47), body temperature (P=.497), and reanimation status (P=.73). Conclusions: Implementing EMRs can influence the completeness of documentation, with a possible change in both increased and decreased completeness. However, the mechanisms that determine those changes are often neglected. There are mechanisms that might facilitate an improved completeness of documentation and could decrease or increase the staff’s burden caused by documentation tasks. Research is needed to take advantage of these mechanisms and use them for mutual profit in the interests of all stakeholders. Trial Registration: German Clinical Trials Register DRKS00023343; https://drks.de/search/de/trial/DRKS00023343 %M 38241076 %R 10.2196/47761 %U https://medinform.jmir.org/2024/1/e47761 %U https://doi.org/10.2196/47761 %U http://www.ncbi.nlm.nih.gov/pubmed/38241076 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 11 %N %P e52495 %T Characterizing and Comparing Adverse Drug Events Documented in 2 Spontaneous Reporting Systems in the Lower Mainland of British Columbia, Canada: Retrospective Observational Study %A Lau,Erica Y %A Cragg,Amber %A Small,Serena S %A Butcher,Katherine %A Hohl,Corinne M %+ Department of Emergency Medicine, University of British Columbia, 7th Floor, 828 West 10th Avenue Research Pavilion, Vancouver, BC, V5Z 1M9, Canada, 1 6048754111 ext 68926, erica.lau@ubc.ca %K adverse drug event reporting systems %K side effect %K side effects %K drug %K drugs %K pharmacy %K pharmacology %K pharmacotherapy %K pharmaceutic %K pharmaceutics %K pharmaceuticals %K pharmaceutical %K medication %K medications %K patient safety %K health information technology %K pharmacovigilance %K adverse %K safety %K HIT %K information system %K information systems %K reporting %K descriptive statistics %K monitoring %D 2024 %7 18.1.2024 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Robust adverse drug event (ADE) reporting systems are crucial to monitor and identify drug safety signals, but the quantity and type of ADEs captured may vary by system characteristics. Objective: We compared ADEs reported in 2 different reporting systems in the same jurisdictions, the Patient Safety and Learning System–Adverse Drug Reaction (PSLS-ADR) and ActionADE, to understand report variation. Methods: This retrospective observational study analyzed reports entered into PSLS-ADR and ActionADE systems between December 1, 2019, and December 31, 2022. We conducted a comprehensive analysis including all events from both reporting systems to examine coverage and usage and understand the types of events captured in both systems. We calculated descriptive statistics for reporting facility type, patient demographics, serious events, and most reported drugs. We conducted a subanalysis focused on adverse drug reactions to enable direct comparisons between systems in terms of the volume and events reported. We stratified results by reporting system. Results: We performed the comprehensive analysis on 3248 ADE reports, of which 12.4% (375/3035) were reported in PSLS-ADR and 87.6% (2660/3035) were reported in ActionADE. Distribution of all events and serious events varied slightly between the 2 systems. Iohexol, gadobutrol, and empagliflozin were the most common culprit drugs (173/375, 46.2%) in PSLS-ADR, while hydrochlorothiazide, apixaban, and ramipril (308/2660, 11.6%) were common in ActionADE. We included 2728 reports in the subanalysis of adverse drug reactions, of which 12.9% (353/2728) were reported in PSLS-ADR and 86.4% (2357/2728) were reported in ActionADE. ActionADE captured 4- to 6-fold more comparable events than PSLS-ADR over this study’s period. Conclusions: User-friendly and robust reporting systems are vital for pharmacovigilance and patient safety. This study highlights substantial differences in ADE data that were generated by different reporting systems. Understanding system factors that lead to varying reporting patterns can enhance ADE monitoring and should be taken into account when evaluating drug safety signals. %M 38236629 %R 10.2196/52495 %U https://humanfactors.jmir.org/2024/1/e52495 %U https://doi.org/10.2196/52495 %U http://www.ncbi.nlm.nih.gov/pubmed/38236629 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 12 %N %P e49007 %T Additional Value From Free-Text Diagnoses in Electronic Health Records: Hybrid Dictionary and Machine Learning Classification Study %A Mehra,Tarun %A Wekhof,Tobias %A Keller,Dagmar Iris %+ Department for Medical Oncology and Hematology, University Hospital of Zurich, Rämistrasse 100, Zurich, 8091, Switzerland, 41 44255 ext 1111, tarun.mehra@usz.ch %K electronic health records %K free text %K natural language processing %K NLP %K artificial intelligence %K AI %D 2024 %7 17.1.2024 %9 Original Paper %J JMIR Med Inform %G English %X Background: Physicians are hesitant to forgo the opportunity of entering unstructured clinical notes for structured data entry in electronic health records. Does free text increase informational value in comparison with structured data? Objective: This study aims to compare information from unstructured text-based chief complaints harvested and processed by a natural language processing (NLP) algorithm with clinician-entered structured diagnoses in terms of their potential utility for automated improvement of patient workflows. Methods: Electronic health records of 293,298 patient visits at the emergency department of a Swiss university hospital from January 2014 to October 2021 were analyzed. Using emergency department overcrowding as a case in point, we compared supervised NLP-based keyword dictionaries of symptom clusters from unstructured clinical notes and clinician-entered chief complaints from a structured drop-down menu with the following 2 outcomes: hospitalization and high Emergency Severity Index (ESI) score. Results: Of 12 symptom clusters, the NLP cluster was substantial in predicting hospitalization in 11 (92%) clusters; 8 (67%) clusters remained significant even after controlling for the cluster of clinician-determined chief complaints in the model. All 12 NLP symptom clusters were significant in predicting a low ESI score, of which 9 (75%) remained significant when controlling for clinician-determined chief complaints. The correlation between NLP clusters and chief complaints was low (r=−0.04 to 0.6), indicating complementarity of information. Conclusions: The NLP-derived features and clinicians’ knowledge were complementary in explaining patient outcome heterogeneity. They can provide an efficient approach to patient flow management, for example, in an emergency medicine setting. We further demonstrated the feasibility of creating extensive and precise keyword dictionaries with NLP by medical experts without requiring programming knowledge. Using the dictionary, we could classify short and unstructured clinical texts into diagnostic categories defined by the clinician. %M 38231569 %R 10.2196/49007 %U https://medinform.jmir.org/2024/1/e49007 %U https://doi.org/10.2196/49007 %U http://www.ncbi.nlm.nih.gov/pubmed/38231569 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e45391 %T Clinical Needs Assessment of a Machine Learning–Based Asthma Management Tool: User-Centered Design Approach %A Zheng,Lu %A Ohde,Joshua W %A Overgaard,Shauna M %A Brereton,Tracey A %A Jose,Kristelle %A Wi,Chung-Il %A Peterson,Kevin J %A Juhn,Young J %+ Center for Digital Health, Mayo Clinic, 200 1st Street South West, Rochester, MN, United States, 1 480 758 0664, zheng.lu@mayo.edu %K asthma %K formative research %K user-centered design %K machine learning (ML) %K artificial intelligence (AI) %K qualitative %K user needs. %D 2024 %7 15.1.2024 %9 Original Paper %J JMIR Form Res %G English %X Background: Personalized asthma management depends on a clinician’s ability to efficiently review patient’s data and make timely clinical decisions. Unfortunately, efficient and effective review of these data is impeded by the varied format, location, and workflow of data acquisition, storage, and processing in the electronic health record. While machine learning (ML) and clinical decision support tools are well-positioned as potential solutions, the translation of such frameworks requires that barriers to implementation be addressed in the formative research stages. Objective: We aimed to use a structured user-centered design approach (double-diamond design framework) to (1) qualitatively explore clinicians’ experience with the current asthma management system, (2) identify user requirements to improve algorithm explainability and Asthma Guidance and Prediction System prototype, and (3) identify potential barriers to ML-based clinical decision support system use. Methods: At the “discovery” phase, we first shadowed to understand the practice context. Then, semistructured interviews were conducted digitally with 14 clinicians who encountered pediatric asthma patients at 2 outpatient facilities. Participants were asked about their current difficulties in gathering information for patients with pediatric asthma, their expectations of ideal workflows and tools, and suggestions on user-centered interfaces and features. At the “define” phase, a synthesis analysis was conducted to converge key results from interviewees’ insights into themes, eventually forming critical “how might we” research questions to guide model development and implementation. Results: We identified user requirements and potential barriers associated with three overarching themes: (1) usability and workflow aspects of the ML system, (2) user expectations and algorithm explainability, and (3) barriers to implementation in context. Even though the responsibilities and workflows vary among different roles, the core asthma-related information and functions they requested were highly cohesive, which allows for a shared information view of the tool. Clinicians hope to perceive the usability of the model with the ability to note patients’ high risks and take proactive actions to manage asthma efficiently and effectively. For optimal ML algorithm explainability, requirements included documentation to support the validity of algorithm development and output logic, and a request for increased transparency to build trust and validate how the algorithm arrived at the decision. Acceptability, adoption, and sustainability of the asthma management tool are implementation outcomes that are reliant on the proper design and training as suggested by participants. Conclusions: As part of our comprehensive informatics-based process centered on clinical usability, we approach the problem using a theoretical framework grounded in user experience research leveraging semistructured interviews. Our focus on meeting the needs of the practice with ML technology is emphasized by a user-centered approach to clinician engagement through upstream technology design. %M 38224482 %R 10.2196/45391 %U https://formative.jmir.org/2024/1/e45391 %U https://doi.org/10.2196/45391 %U http://www.ncbi.nlm.nih.gov/pubmed/38224482 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 11 %N %P e46030 %T A Novel Continuous Real-Time Vital Signs Viewer for Intensive Care Units: Design and Evaluation Study %A Yang,Shiming %A Galvagno,Samuel %A Badjatia,Neeraj %A Stein,Deborah %A Teeter,William %A Scalea,Thomas %A Shackelford,Stacy %A Fang,Raymond %A Miller,Catriona %A Hu,Peter %A , %+ Department of Anesthesiology, University of Maryland School of Medicine, 11 S Paca St. LL01, Baltimore, MD, 21201, United States, 1 4103284179, syang@som.umaryland.edu %K clinical decision-making %K health information technology %K intensive care units %K patient care prioritization %K physiological monitoring %K visualization %K vital signs %D 2024 %7 5.1.2024 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Clinicians working in intensive care units (ICUs) are immersed in a cacophony of alarms and a relentless onslaught of data. Within this frenetic environment, clinicians make high-stakes decisions using many data sources and are often oversaturated with information of varying quality. Traditional bedside monitors only depict static vital signs data, and these data are not easily viewable remotely. Clinicians must rely on separate nursing charts—handwritten or electric—to review physiological patterns, including signs of potential clinical deterioration. An automated physiological data viewer has been developed to provide at-a-glance summaries and to assist with prioritizing care for multiple patients who are critically ill. Objective: This study aims to evaluate a novel vital signs viewer system in a level 1 trauma center by subjectively assessing the viewer’s utility in a high-volume ICU setting. Methods: ICU attendings were surveyed during morning rounds. Physicians were asked to conduct rounds normally, using data reported from nurse charts and briefs from fellows to inform their clinical decisions. After the physician finished their assessment and plan for the patient, they were asked to complete a questionnaire. Following completion of the questionnaire, the viewer was presented to ICU physicians on a tablet personal computer that displayed the patient’s physiologic data (ie, shock index, blood pressure, heart rate, temperature, respiratory rate, and pulse oximetry), summarized for up to 72 hours. After examining the viewer, ICU physicians completed a postview questionnaire. In both questionnaires, the physicians were asked questions regarding the patient’s stability, status, and need for a higher or lower level of care. A hierarchical clustering analysis was used to group participating ICU physicians and assess their general reception of the viewer. Results: A total of 908 anonymous surveys were collected from 28 ICU physicians from February 2015 to June 2017. Regarding physicians’ perception of whether the viewer enhanced the ability to assess multiple patients in the ICU, 5% (45/908) strongly agreed, 56.6% (514/908) agreed, 35.3% (321/908) were neutral, 2.9% (26/908) disagreed, and 0.2% (2/908) strongly disagreed. Conclusions: Morning rounds in a trauma center ICU are conducted in a busy environment with many data sources. This study demonstrates that organized physiologic data and visual assessment can improve situation awareness, assist clinicians with recognizing changes in patient status, and prioritize care. %M 38180791 %R 10.2196/46030 %U https://humanfactors.jmir.org/2024/1/e46030 %U https://doi.org/10.2196/46030 %U http://www.ncbi.nlm.nih.gov/pubmed/38180791 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 11 %N %P e47755 %T Usability of an App for Medical History Taking in General Practice From the Patients’ Perspective: Cross-Sectional Study %A Albrink,Klara %A Schröder,Dominik %A Joos,Carla %A Müller,Frank %A Noack,Eva Maria %+ Department of General Practice, University Medical Center Göttingen, Humboldtallee 38, Göttingen, 37073, Germany, 49 551 39 65745, evamaria.noack@med.uni-goettingen.de %K digitization %K application software %K usability %K mHealth %K history of present illness %K medical history taking %D 2024 %7 5.1.2024 %9 Original Paper %J JMIR Hum Factors %G English %X Background: A future shortage of physicians, especially in general practice, will result in an increasing workload for health care providers as a whole. Therefore, it is important to optimize patient-encounter processes to increase time efficiency related to visits. Utilizing digital tools to record patients’ medical histories prior to a consultation offers great potential to achieve this goal. The collected information can be stored into the practice’s electronic medical record, allowing for the general practitioner to review structured information of the patients’ complaints and related medical history beforehand, thereby saving time during the encounter. However, the low usability of new digital developments in this setting often hinders implementation. Objective: The aim of this study was to evaluate the usability of an app designed for medical history taking in general practice to capture the patients’ perspective. Methods: Between November 2021 and January 2022, we recruited 406 patients with acute complaints in one out-of-hour urgent care and seven general practice clinics. These study participants used the app during their waiting time and subsequently assessed its usability by completing the System Usability Scale (SUS), a robust and well-established 10-question survey measuring the perceived usability of products and technologies. Additionally, we collected general participant information, including age, sex, media usage, health literacy, and native language. Descriptive and inferential statistics were applied to identify patient characteristics associated with low or high SUS scores. Results: We analyzed data from 397 patients (56.7% female, 43.3% male). The mean total SUS score was 77.8 points; 54.4% (216/397) of participants had SUS scores of 80 points or higher, indicating high usability of the app. In a multiple linear regression predicting SUS score, male sex and higher age (65 years or older) were significantly negatively associated with the SUS score. Conversely, a higher health literacy score and German as the native language were significantly positively associated with the SUS score. Conclusions: Usability testing based on the SUS anticipates successful implementation of the app. However, not all patients will easily adapt to utilizing the app, as exemplified by the participants of older age in this study who reported lower perceived usability. Further research should examine these groups of people, identify the exact problems in operating such an app, and provide targeted solutions. Trial Registration: German Clinical Trials Register World Health Organization Trial Registration Data Set DRKS00026659; https://trialsearch.who.int/Trial2.aspx?TrialID=DRKS00026659 %M 38180798 %R 10.2196/47755 %U https://humanfactors.jmir.org/2024/1/e47755 %U https://doi.org/10.2196/47755 %U http://www.ncbi.nlm.nih.gov/pubmed/38180798 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 7 %N %P e48296 %T Involving Patients and Clinicians in the Design of Wireframes for Cancer Medicines Electronic Patient Reported Outcome Measures in Clinical Care: Mixed Methods Study %A Dunlop,Emma %A Ferguson,Aimee %A Mueller,Tanja %A Baillie,Kelly %A Laskey,Jennifer %A Clarke,Julie %A Kurdi,Amanj %A Wales,Ann %A Connolly,Thomas %A Bennie,Marion %+ Strathclyde Institute of Pharmacy & Biomedical Sciences, University of Strathclyde, 161 Cathedral St, Glasgow, G4 0RE, United Kingdom, 44 01415482478, emma.dunlop@strath.ac.uk %K cancer %K clinicians %K mHealth %K mixed methods study %K patient reported outcome measures %K patients %K technology acceptance model %D 2023 %7 21.12.2023 %9 Original Paper %J JMIR Form Res %G English %X Background: Cancer treatment is a key component of health care systems, and the increasing number of cancer medicines is expanding the treatment landscape. However, evidence of the impact on patients has been focused more on chemotherapy toxicity and symptom control and less on the effect of cancer medicines more broadly on patients’ lives. Evolving electronic patient-reported outcome measures (ePROMs) presents the opportunity to secure early engagement of patients and clinicians in shaping the collection of quality-of-life metrics and presenting these data to better support the patient-clinician decision-making process. Objective: The aim of this study was to obtain initial feedback from patients and clinicians on the wireframes of a digital solution (patient app and clinician dashboard) for the collection and use of cancer medicines ePROMs. Methods: We adopted a 2-stage, mixed methods approach. Stage 1 (March to June 2019) consisted of interviews and focus groups with cancer clinicians and patients with cancer to explore the face validity of the wireframes, informed by the technology acceptance model constructs (perceived ease of use, perceived usefulness, and behavioral intention to use). In stage 2 (October 2019 to February 2020), the revised wireframes were assessed through web-based, adapted technology acceptance model questionnaires. Qualitative data (stage 1) underwent a framework analysis, and descriptive statistics were performed on quantitative data (stage 2). Clinicians and patients with cancer were recruited from NHS Greater Glasgow & Clyde, the largest health board in Scotland. Results: A total of 14 clinicians and 19 patients participated in a combination of stage 1 interviews and focus groups. Clinicians and patients indicated that the wireframes of a patient app and clinician dashboard for the collection of cancer medicines ePROMs would be easy to use and could focus discussions, and they would be receptive to using such tools in the future. In stage 1, clinicians raised the potential impact on workload, and both groups identified the need for adequate IT skills to use each technology. Changes to the wireframes were made, and in stage 2, clinicians (n=8) and patients (n=16) indicated it was “quite likely” that the technologies would be easy to use and they would be “quite likely” to use them in the future. Notably, clinicians indicated that they would use the dashboard to enable treatment decisions “with around half” of their patients. Conclusions: This study emphasizes the importance of consulting both patients and clinicians in the design of digital solutions. The wireframes were perceived positively by patients and clinicians who were willing to use such technologies if available in the future as part of routine care. However, challenges were raised, and some differences were identified between participant groups, which warrant further research. %M 38127422 %R 10.2196/48296 %U https://formative.jmir.org/2023/1/e48296 %U https://doi.org/10.2196/48296 %U http://www.ncbi.nlm.nih.gov/pubmed/38127422 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e50158 %T Holistic Human-Serving Digitization of Health Care Needs Integrated Automated System-Level Assessment Tools %A Welzel,Cindy %A Cotte,Fabienne %A Wekenborg,Magdalena %A Vasey,Baptiste %A McCulloch,Peter %A Gilbert,Stephen %+ Else Kröner Fresenius Center for Digital Health, TUD Dresden University of Technology, Fetscherstraße 74, Dresden, 01307, Germany, 49 35145819630, stephen.gilbert@ukdd.de %K health technology assessment %K human factors %K postmarket surveillance %K software as a medical device %K digital health tools %K quality assessment %K quality improvement %K regulatory framework %K user experience %K health care %D 2023 %7 20.12.2023 %9 Viewpoint %J J Med Internet Res %G English %X Digital health tools, platforms, and artificial intelligence– or machine learning–based clinical decision support systems are increasingly part of health delivery approaches, with an ever-greater degree of system interaction. Critical to the successful deployment of these tools is their functional integration into existing clinical routines and workflows. This depends on system interoperability and on intuitive and safe user interface design. The importance of minimizing emergent workflow stress through human factors research and purposeful design for integration cannot be overstated. Usability of tools in practice is as important as algorithm quality. Regulatory and health technology assessment frameworks recognize the importance of these factors to a certain extent, but their focus remains mainly on the individual product rather than on emergent system and workflow effects. The measurement of performance and user experience has so far been performed in ad hoc, nonstandardized ways by individual actors using their own evaluation approaches. We propose that a standard framework for system-level and holistic evaluation could be built into interacting digital systems to enable systematic and standardized system-wide, multiproduct, postmarket surveillance and technology assessment. Such a system could be made available to developers through regulatory or assessment bodies as an application programming interface and could be a requirement for digital tool certification, just as interoperability is. This would enable health systems and tool developers to collect system-level data directly from real device use cases, enabling the controlled and safe delivery of systematic quality assessment or improvement studies suitable for the complexity and interconnectedness of clinical workflows using developing digital health technologies. %M 38117545 %R 10.2196/50158 %U https://www.jmir.org/2023/1/e50158 %U https://doi.org/10.2196/50158 %U http://www.ncbi.nlm.nih.gov/pubmed/38117545 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e46306 %T Testing and Practical Implementation of a User-Friendly Personalized and Long-Term Electronic Informed Consent Prototype in Clinical Research: Mixed Methods Study %A De Sutter,Evelien %A Geerts,David %A Yskout,Koen %A Verreydt,Stef %A Borry,Pascal %A Barbier,Liese %A Huys,Isabelle %+ Catholic University (KU) Leuven Digital Society Institute, Catholic University (KU) Leuven, Parkstraat 45, Box 3605, Leuven, 3000, Belgium, 32 16323195, david.geerts@kuleuven.be %K human-centered design %K digital health %K qualitative research %K informed consent %K trial %K stakeholders %K implementation %D 2023 %7 19.12.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: Over the years, there has been increasing interest in electronic informed consent (eIC) in clinical research. The user-friendliness of an eIC application and its acceptance by stakeholders plays a central role in achieving successful implementation. Objective: This study aims to identify insights for the design and implementation of a user-friendly, personalized, and long-term eIC application based on a usability study with (potential) research participants and semistructured interviews with stakeholders on the practical integration of such an application into their daily practice. Methods: An eIC prototype was evaluated and refined through usability testing among Belgian citizens and iterative redesign. On the basis of a digital literacy questionnaire, a heterogeneous sample of participants was established. Participants needed to complete a series of usability tasks related to personalization and long-term interaction with the research team while using the “think aloud” technique. In addition, usability tests involved completing the System Usability Scale questionnaire and taking part in a semistructured feedback interview. Furthermore, semistructured interviews were conducted with ethics committee members, health care professionals, and pharmaceutical industry representatives active in Belgium and involved in clinical research. Thematic analysis was undertaken using the NVivo software (Lumivero). Results: In total, 3 iterations of usability tests were conducted with 10 participants each. Each cycle involved some participants who reported having low digital skills. The System Usability Scale scores related to the tasks on personalization and long-term interaction increased after each iteration and reached 69.5 (SD 8.35) and 71.3 (SD 16.1) out of 100, respectively, which represents above-average usability. Semistructured interviews conducted with health care professionals (n=4), ethics committee members (n=8), and pharmaceutical industry representatives (n=5) identified the need for an eIC system that can be easily set up. For example, a library could be established enabling stakeholders to easily provide background information about a clinical study, presented in the second layer of the interface. In contrast, some functionalities, such as informing participants about new studies through an eIC system, were not considered useful by stakeholders. Conclusions: This study provides insights for the implementation of a user-friendly personalized and long-term eIC application. The study findings showed that usability testing is key to assessing and increasing the user-friendliness of an eIC application. Although this eIC system has the potential to be usable by a wide audience, participants with low digital literacy may not be able to use it successfully, highlighting the need for additional support for participants or other alternatives to an eIC system. In addition, key lessons emerging from the interviews included ensuring that the application is easy to implement in practice and is interoperable with other established systems. %M 38113088 %R 10.2196/46306 %U https://www.jmir.org/2023/1/e46306 %U https://doi.org/10.2196/46306 %U http://www.ncbi.nlm.nih.gov/pubmed/38113088 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 11 %N %P e44773 %T Standardized Comparison of Voice-Based Information and Documentation Systems to Established Systems in Intensive Care: Crossover Study %A Peine,Arne %A Gronholz,Maike %A Seidl-Rathkopf,Katharina %A Wolfram,Thomas %A Hallawa,Ahmed %A Reitz,Annika %A Celi,Leo Anthony %A Marx,Gernot %A Martin,Lukas %+ Department of Intensive Care Medicine and Intermediate Care, University Hospital RWTH Aachen, Pauwelsstr. 30, Aachen, 52070, Germany, 49 241 800, apeine@ukaachen.de %K artificial intelligence %K documentation %K ICU %K intensive care medicine %K speech-recognition %K user perception %K workload %D 2023 %7 28.11.2023 %9 Original Paper %J JMIR Med Inform %G English %X Background: The medical teams in intensive care units (ICUs) spend increasing amounts of time at computer systems for data processing, input, and interpretation purposes. As each patient creates about 1000 data points per hour, the available information is abundant, making the interpretation difficult and time-consuming. This data flood leads to a decrease in time for evidence-based, patient-centered care. Information systems, such as patient data management systems (PDMSs), are increasingly used at ICUs. However, they often create new challenges arising from the increasing documentation burden. Objective: New concepts, such as artificial intelligence (AI)–based assistant systems, are hence introduced to the workflow to cope with these challenges. However, there is a lack of standardized, published metrics in order to compare the various data input and management systems in the ICU setting. The objective of this study is to compare established documentation and retrieval processes with newer methods, such as PDMSs and voice information and documentation systems (VIDSs). Methods: In this crossover study, we compare traditional, paper-based documentation systems with PDMSs and newer AI-based VIDSs in terms of performance (required time), accuracy, mental workload, and user experience in an intensive care setting. Performance is assessed on a set of 6 standardized, typical ICU tasks, ranging from documentation to medical interpretation. Results: A total of 60 ICU-experienced medical professionals participated in the study. The VIDS showed a statistically significant advantage compared to the other 2 systems. The tasks were completed significantly faster with the VIDS than with the PDMS (1-tailed t59=12.48; Cohen d=1.61; P<.001) or paper documentation (t59=20.41; Cohen d=2.63; P<.001). Significantly fewer errors were made with VIDS than with the PDMS (t59=3.45; Cohen d=0.45; P=.03) and paper-based documentation (t59=11.2; Cohen d=1.45; P<.001). The analysis of the mental workload of VIDS and PDMS showed no statistically significant difference (P=.06). However, the analysis of subjective user perception showed a statistically significant perceived benefit of the VIDS compared to the PDMS (P<.001) and paper documentation (P<.001). Conclusions: The results of this study show that the VIDS reduced error rate, documentation time, and mental workload regarding the set of 6 standardized typical ICU tasks. In conclusion, this indicates that AI-based systems such as the VIDS tested in this study have the potential to reduce this workload and improve evidence-based and safe patient care. %M 38015593 %R 10.2196/44773 %U https://medinform.jmir.org/2023/1/e44773 %U https://doi.org/10.2196/44773 %U http://www.ncbi.nlm.nih.gov/pubmed/38015593 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e46146 %T Co-Designing an Initiative to Increase Shared Access to Older Adults’ Patient Portals: Stakeholder Engagement %A Dukhanin,Vadim %A Wolff,Jennifer L %A Salmi,Liz %A Harcourt,Kendall %A Wachenheim,Deborah %A Byock,Ira %A Gonzales,Matthew J %A Niehus,Doug %A Parshley,Marianne %A Reay,Caroline %A Epstein,Sara %A Mohile,Supriya %A Farrell,Timothy W %A Supiano,Mark A %A Jajodia,Anushka %A DesRoches,Catherine M %A , %+ Department of Medicine, Harvard Medical School, 133 Brookline Avenue, 2nd Floor Annex, Boston, MA, 02115, United States, 1 617 975 7613, cdesroch@bidmc.harvard.edu %K patient portal %K electronic health record %K care partners %K stakeholder engagement %K patient engagement %K human-centered design %K mobile phone %K design %K older adults %K digital platform %K awareness %K development %K engagement %K stakeholder %K education %D 2023 %7 22.11.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: The patient portal is a widely available secure digital platform offered by care delivery organizations that enables patients to communicate electronically with clinicians and manage their care. Many organizations allow patients to authorize family members or friends—“care partners”—to share access to patient portal accounts, thus enabling care partners to receive their own identity credentials. Shared access facilitates trilateral information exchange among patients, clinicians, and care partners; however, uptake and awareness of this functionality are limited. Objective: We partnered with 3 health care organizations to co-design an initiative that aimed to increase shared access registration and use and that can be implemented using existing patient portals. Methods: In 2020, we undertook a rigorous selection process to identify 3 geographically diverse health care organizations that had engaged medical informatics teams and clinical champions within service delivery lines caring for older adults. We prioritized selecting organizations that serve racially and socioeconomically diverse populations and possess sophisticated reporting capabilities, a stable patient portal platform, a sufficient volume of older adult patients, and active patient and family advisory councils. Along with patients and care partners, clinicians, staff, and other stakeholders, the study team co-designed an initiative to increase the uptake of shared access guided by either an iterative, human-centered design process or rapid assessment procedures of stakeholders’ inputs. Results: Between February 2020 and April 2022, 73 stakeholder engagements were conducted with patients and care partners, clinicians and clinic staff, medical informatics teams, marketing and communications staff, and administrators, as well as with funders and thought leaders. We collected insights regarding (1) barriers to awareness, registration, and use of shared access; (2) features of consumer-facing educational materials to address identified barriers; (3) features of clinician- and staff-facing materials to address identified barriers; and (4) approaches to fit the initiative into current workflows. Using these inputs iteratively via a human-centered design process, we produced brochures and posters, co-designed organization-specific web pages detailing shared access registration processes, and developed clinician and staff talking points about shared access and staff tip sheets that outline shared access registration steps. Educational materials emphasized the slogan “People remember less than half of what their doctors say,” which was selected from 9 candidate alternatives as resonating best with the full range of the initiative’s stakeholders. The materials were accompanied by implementation toolkits specifying and reinforcing workflows involving both in-person and telehealth visits. Conclusions: Meaningful and authentic stakeholder engagement allowed our deliberate, iterative, and human-centered co-design aimed at increasing the use of shared access. Our initiative has been launched as a part of a 12-month demonstration that will include quantitative and qualitative analysis of registration and use of shared access. Educational materials are publicly available at Coalition for Care Partners. %M 37991827 %R 10.2196/46146 %U https://www.jmir.org/2023/1/e46146 %U https://doi.org/10.2196/46146 %U http://www.ncbi.nlm.nih.gov/pubmed/37991827 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e49100 %T Correlation Between Remote Symptom Reporting by Caregivers and Adverse Clinical Outcomes: Mixed Methods Study %A Oakley-Girvan,Ingrid %A Yunis,Reem %A Fonda,Stephanie J %A Longmire,Michelle %A Veuthey,Tess L %A Shieh,Jennifer %A Aghaee,Sara %A Kubo,Ai %A Davis,Sharon W %A Liu,Raymond %A Neeman,Elad %+ Medable Inc, 525 University Ave, Suite A70, Palo Alto, CA, 94103, United States, 1 877 820 6259, oakley@stanford.edu %K adverse events %K cancer %K decentralized clinical trials %K electronic patient-reported outcomes %K ePROs %K mobile health app %K observer-reported outcomes %K Patient-Reported Outcomes Measurement Information System Patient-Reported Outcome Common Terminology Criteria for Adverse Events %K patient-reported outcomes %K PRO-CTCAE %K PROMIS %K remote clinical trials %K remote monitoring %K smartphone %D 2023 %7 21.11.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: Timely collection of patient-reported outcomes (PROs) decreases emergency department visits and hospitalizations and increases survival. However, little is known about the outcome predictivity of unpaid informal caregivers’ reporting using similar clinical outcome assessments. Objective: The aim of this study is to assess whether caregivers and adults with cancer adhered to a planned schedule for electronically collecting patient-reported outcomes (PROs) and if PROs were associated with future clinical events. Methods: We developed 2 iPhone apps to collect PROs, one for patients with cancer and another for caregivers. We enrolled 52 patient-caregiver dyads from Kaiser Permanente Northern California in a nonrandomized study. Participants used the apps independently for 4 weeks. Specific clinical events were obtained from the patients’ electronic health records up to 6 months following the study. We used logistic and quasi-Poisson regression analyses to test associations between PROs and clinical events. Results: Participants completed 97% (251/260) of the planned Patient-Reported Outcomes Common Terminology Criteria for Adverse Events (PRO-CTCAE) surveys and 98% (254/260) of the Patient-Reported Outcomes Measurement Information System (PROMIS) surveys. PRO-CTCAE surveys completed by caregivers were associated with patients’ hospitalizations or emergency department visits, grade 3-4 treatment-related adverse events, dose reductions (P<.05), and hospice referrals (P=.03). PROMIS surveys completed by caregivers were associated with hospice referrals (P=.02). PRO-CTCAE surveys completed by patients were not associated with any clinical events, but their baseline PROMIS surveys were associated with mortality (P=.03), while their antecedent or final PROMIS surveys were associated with all clinical events examined except for total days of treatment breaks. Conclusions: In this study, caregivers and patients completed PROs using smartphone apps as requested. The association of caregiver PRO-CTCAE surveys with patient clinical events suggests that this is a feasible approach to reducing patient burden in clinical trial data collection and may help provide early information about increasing symptom severity. %M 37988151 %R 10.2196/49100 %U https://www.jmir.org/2023/1/e49100 %U https://doi.org/10.2196/49100 %U http://www.ncbi.nlm.nih.gov/pubmed/37988151 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 10 %N %P e48950 %T Think Aloud Testing of a Smartphone App for Lifestyle Change Among Persons at Risk of Type 2 Diabetes: Usability Study %A Lunde,Pernille %A Skoglund,Gyri %A Olsen,Cecilie Fromholt %A Hilde,Gunvor %A Bong,Way Kiat %A Nilsson,Birgitta Blakstad %+ Department of Rehabilitation Science and Health Technology, Faculty of Health Sciences, Oslo Metropolitan University, P44, PB 4, Street Olavs Plass, Oslo, 0130, Norway, 47 48063537, plunde@oslomet.no %K mHealth %K mobile phone app %K smartphone %K lifestyle %K usability %K diabetes %K diabetic %K mobile health %K smartphone %K app %K apps %K application %K applications %K think-aloud %K think aloud %K user experience %K mobile phone %D 2023 %7 15.11.2023 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Type 2 diabetes mellitus (DM2) is a leading cause of morbidity and mortality worldwide and is considered a global epidemic. Despite the growing evidence on the effectiveness of mobile health interventions in the management of DM2, the evidence on the effect of mobile health interventions in prevention of DM2 is sparse. Therefore, we have developed an app aiming to promote initiation of behavioral change and adherence to healthy behavior. Before commencing a small-scale randomized controlled trial to assess the feasibility of using an app for initiation and adherence of healthy behavior in people at risk of DM2, testing the usability of the app in the target population is warranted. Objective: The aim of this study was to assess the usability of an app among people at risk of DM2. Methods: A qualitative study with the use of a think aloud (TA) procedure was conducted from April to November 2022. The TA procedure consisted of 10 problem-solving tasks and a semistructured interview which was carried out after the tasks. These interviews served to gain more in-depth knowledge of the users experience of the problem-solving tasks. The TA-sessions and the postactivity interviews were recorded and transcribed verbatim, and the data were coded and analyzed following the principles of thematic analysis. Results: In total, 7 people at risk of DM2 with a median age of 66 (range 41-75) years participated in this study. The analysis resulted in the following themes: (1) user interface design; and (2) suggestions for improvements of the functionality of the app. Conclusions: Overall, the participants were satisfied with the usability of the app. Through the TA-sessions, real time perspective on the appeal, relevance, and utility of the app were gained. Only minor changes to the functionality of the prototype app were reported as necessary to improve the usability of the app. Points of guidance from the participants in this study have been adopted and incorporated into the final design of the app now being assessed for feasibility in a small-scale randomized controlled trial. %M 37966894 %R 10.2196/48950 %U https://humanfactors.jmir.org/2023/1/e48950 %U https://doi.org/10.2196/48950 %U http://www.ncbi.nlm.nih.gov/pubmed/37966894 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 10 %N %P e47718 %T Developing an Audit and Feedback Dashboard for Family Physicians: User-Centered Design Process %A Shuldiner,Jennifer %A Kiran,Tara %A Agarwal,Payal %A Daneshvarfard,Maryam %A Eldridge,Kirsten %A Kim,Susie %A Greiver,Michelle %A Jokhio,Iffat %A Ivers,Noah %+ Women's College Hospital, 76 Grenville, Toronto, ON, M5S 1B2, Canada, 1 6474720232, jennifer.shuldiner@wchospital.ca %K audit and feedback %K primary care %K design %K user-centered %K design %K audit %K feedback %K development %K dashboard %K family physician %K clinical performance %K implementation %K users %K primary care %K care %D 2023 %7 9.11.2023 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Audit and feedback (A&F), the summary and provision of clinical performance data, is a common quality improvement strategy. Successful design and implementation of A&F—or any quality improvement strategy—should incorporate evidence-informed best practices as well as context-specific end user input. Objective: We used A&F theory and user-centered design to inform the development of a web-based primary care A&F dashboard. We describe the design process and how it influenced the design of the dashboard. Methods: Our design process included 3 phases: prototype development based on A&F theory and input from clinical improvement leaders; workshop with family physician quality improvement leaders to develop personas (ie, fictional users that represent an archetype character representative of our key users) and application of those personas to design decisions; and user-centered interviews with family physicians to learn about the physician’s reactions to the revised dashboard. Results: The team applied A&F best practices to the dashboard prototype. Personas were used to identify target groups with challenges and behaviors as a tool for informed design decision-making. Our workshop produced 3 user personas, Dr Skeptic, Frazzled Physician, and Eager Implementer, representing common users based on the team’s experience of A&F. Interviews were conducted to further validate findings from the persona workshop and found that (1) physicians were interested in how they compare with peers; however, if performance was above average, they were not motivated to improve even if gaps compared to other standards in their care remained; (2) burnout levels were high as physicians are trying to catch up on missed care during the pandemic and are therefore less motivated to act on the data; and (3) additional desired features included integration within the electronic medical record, and more up-to-date and accurate data. Conclusions: We found that carefully incorporating data from user interviews helped operationalize generic best practices for A&F to achieve an acceptable dashboard that could meet the needs and goals of physicians. We demonstrate such a design process in this paper. A&F dashboards should address physicians’ data skepticism, present data in a way that spurs action, and support physicians to have the time and capacity to engage in quality improvement work; the steps we followed may help those responsible for quality improvement strategy implementation achieve these aims. %M 37943586 %R 10.2196/47718 %U https://humanfactors.jmir.org/2023/1/e47718 %U https://doi.org/10.2196/47718 %U http://www.ncbi.nlm.nih.gov/pubmed/37943586 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 11 %N %P e45636 %T Clinical Decision Support to Reduce Opioid Prescriptions for Dental Extractions using SMART on FHIR: Implementation Report %A Rindal,D Brad %A Pasumarthi,Dhavan Prasad %A Thirumalai,Vijayakumar %A Truitt,Anjali R %A Asche,Stephen E %A Worley,Donald C %A Kane,Sheryl M %A Gryczynski,Jan %A Mitchell,Shannon G %+ HealthPartners Institute, 8170 33rd Ave S, Minneapolis, MN, 55425, United States, 1 952 967 5026, donald.b.rindal@healthpartners.com %K clinical decision support systems %K dentistry %K analgesics %K electronic health records %K EHR %K algorithm %K design %K implementation %K decision support %K development %K dentists %K pain management %K patient care %K application %K tool %K Fast Healthcare Interoperability Resources %K FHIR %K Substitutable Medical Applications and Reusable Technologies %K SMART %D 2023 %7 7.11.2023 %9 Implementation Report %J JMIR Med Inform %G English %X Background: Clinical decision support (CDS) has the potential to improve clinical decision-making consistent with evidence-based care. CDS can be designed to save health care providers time and help them provide safe and personalized analgesic prescribing. Objective: The aim of this report is to describe the development of a CDS system designed to provide dentists with personalized pain management recommendations to reduce opioid prescribing following extractions. The use of CDS is also examined. Methods: This study was conducted in HealthPartners, which uses an electronic health record (EHR) system that integrates both medical and dental information upon which the CDS application was developed based on SMART (Substitutable Medical Applications and Reusable Technologies) on FHIR (Fast Healthcare Interoperability Resources). The various tools used to bring relevant medical conditions, medications, patient history, and other relevant data into the CDS interface are described. The CDS application runs a drug interaction algorithm developed by our organization and provides patient-specific recommendations. The CDS included access to the state Prescription Monitoring Program database. Implementation (Results): The pain management CDS was implemented as part of a study examining opioid prescribing among patients undergoing dental extraction procedures from February 17, 2020, to May 14, 2021. Provider-level use of CDS at extraction encounters ranged from 0% to 87.4% with 12.1% of providers opening the CDS for no encounters, 39.4% opening the CDS for 1%-20% of encounters, 36.4% opening it for 21%-50% of encounters, and 12.1% opening it for 51%-87% of encounters. Conclusions: The pain management CDS is an EHR-embedded, provider-facing tool to help dentists make personalized pain management recommendations following dental extractions. The SMART on FHIR–based pain management CDS adapted well to the point-of-care dental setting and led to the design of a scalable CDS tool that is EHR vendor agnostic. Trial Registration: ClinicalTrials.gov NCT03584789; https://clinicaltrials.gov/study/NCT03584789 %M 37934572 %R 10.2196/45636 %U https://medinform.jmir.org/2023/1/e45636 %U https://doi.org/10.2196/45636 %U http://www.ncbi.nlm.nih.gov/pubmed/37934572 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 10 %N %P e47624 %T Patient-Caregiver Portal System in Palliative Oncology: Assessment of Usability and Perceived Benefit %A Longacre,Margaret L %A Chwistek,Marcin %A Keleher,Cynthia %A Siemon,Mark %A Egleston,Brian L %A Collins,Molly %A Fang,Carolyn Y %+ College of Health Sciences, Arcadia University, 450 South Easton Road, 241 Easton Hall, Glenside, PA, 19038, United States, 1 215 582 4275, longacrm@arcadia.edu %K caregiving %K patient portal, health policy %K palliative oncology %K oncology %K engagement %K family caregiver %K caregiver %K communication %K usage %K usability %K clinical care %K cancer %D 2023 %7 2.11.2023 %9 Original Paper %J JMIR Hum Factors %G English %X Background: The engagement of family caregivers in oncology is not universal or systematic. Objective: We implemented a process intervention (ie, patient-caregiver portal system) with an existing patient portal system to (1) allow a patient to specify their caregiver and communication preferences with that caregiver, (2) connect the caregiver to a unique caregiver-specific portal page to indicate their needs, and (3) provide an electronic notification of the dyad’s responses to the care team to inform clinicians and connect the caregiver to resources as needed. Methods: We assessed usability and satisfaction with this patient-caregiver portal system among patients with cancer receiving palliative care, their caregivers, and clinicians. Results: Of 31 consented patient-caregiver dyads, 20 patients and 19 caregivers logged in. Further, 60% (n=12) of patients indicated a preference to communicate equally or together with their caregiver. Caregivers reported high emotional (n=9, 47.3%), financial (n=6, 31.6%), and physical (n=6, 31.6%) caregiving-related strain. The care team received all patient-caregiver responses electronically. Most patients (86.6%, 13/15 who completed the user experience interview) and caregivers (94%, 16/17 who completed the user experience interview) were satisfied with the system, while, of the 6 participating clinicians, 66.7% agreed “quite a bit” (n=1, 16.7%) or “very much” (n=3, 50%) that the system allowed them to provide better care. Conclusions: Our findings demonstrate system usability, including a systematic way to identify caregiver needs and share with the care team in a way that is acceptable to patients and caregivers and perceived by clinicians to benefit clinical care. Integration of a patient-caregiver portal system may be an effective approach for systematically engaging caregivers. These findings highlight the need for additional research among caregivers of patients with less advanced cancer or with different illnesses. %M 37917129 %R 10.2196/47624 %U https://humanfactors.jmir.org/2023/1/e47624 %U https://doi.org/10.2196/47624 %U http://www.ncbi.nlm.nih.gov/pubmed/37917129 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e49605 %T Characteristics and Admission Preferences of Pediatric Emergency Patients and Their Waiting Time Prediction Using Electronic Medical Record Data: Retrospective Comparative Analysis %A Guo,Lin Lin %A Guo,Lin Ying %A Li,Jiao %A Gu,Yao Wen %A Wang,Jia Yang %A Cui,Ying %A Qian,Qing %A Chen,Ting %A Jiang,Rui %A Zheng,Si %+ Institute of Medical Information, Chinese Academy of Medical Sciences and Peking Union Medical College, 3 Yabao Rd, Chaoyang District, Beijing, 100020, China, 86 010 52328745, zheng.si@imicams.ac.cn %K pediatric emergency department %K characteristics %K admission preferences %K waiting time %K machine learning %K electronic medical record %D 2023 %7 1.11.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: The growing number of patients visiting pediatric emergency departments could have a detrimental impact on the care provided to children who are triaged as needing urgent attention. Therefore, it has become essential to continuously monitor and analyze the admissions and waiting times of pediatric emergency patients. Despite the significant challenge posed by the shortage of pediatric medical resources in China’s health care system, there have been few large-scale studies conducted to analyze visits to the pediatric emergency room. Objective: This study seeks to examine the characteristics and admission patterns of patients in the pediatric emergency department using electronic medical record (EMR) data. Additionally, it aims to develop and assess machine learning models for predicting waiting times for pediatric emergency department visits. Methods: This retrospective analysis involved patients who were admitted to the emergency department of Children’s Hospital Capital Institute of Pediatrics from January 1, 2021, to December 31, 2021. Clinical data from these admissions were extracted from the electronic medical records, encompassing various variables of interest such as patient demographics, clinical diagnoses, and time stamps of clinical visits. These indicators were collected and compared. Furthermore, we developed and evaluated several computational models for predicting waiting times. Results: In total, 183,024 eligible admissions from 127,368 pediatric patients were included. During the 12-month study period, pediatric emergency department visits were most frequent among children aged less than 5 years, accounting for 71.26% (130,423/183,024) of the total visits. Additionally, there was a higher proportion of male patients (104,147/183,024, 56.90%) compared with female patients (78,877/183,024, 43.10%). Fever (50,715/183,024, 27.71%), respiratory infection (43,269/183,024, 23.64%), celialgia (9560/183,024, 5.22%), and emesis (6898/183,024, 3.77%) were the leading causes of pediatric emergency room visits. The average daily number of admissions was 501.44, and 18.76% (34,339/183,204) of pediatric emergency department visits resulted in discharge without a prescription or further tests. The median waiting time from registration to seeing a doctor was 27.53 minutes. Prolonged waiting times were observed from April to July, coinciding with an increased number of arrivals, primarily for respiratory diseases. In terms of waiting time prediction, machine learning models, specifically random forest, LightGBM, and XGBoost, outperformed regression methods. On average, these models reduced the root-mean-square error by approximately 17.73% (8.951/50.481) and increased the R2 by approximately 29.33% (0.154/0.525). The SHAP method analysis highlighted that the features “wait.green” and “department” had the most significant influence on waiting times. Conclusions: This study offers a contemporary exploration of pediatric emergency room visits, revealing significant variations in admission rates across different periods and uncovering certain admission patterns. The machine learning models, particularly ensemble methods, delivered more dependable waiting time predictions. Patient volume awaiting consultation or treatment and the triage status emerged as crucial factors contributing to prolonged waiting times. Therefore, strategies such as patient diversion to alleviate congestion in emergency departments and optimizing triage systems to reduce average waiting times remain effective approaches to enhance the quality of pediatric health care services in China. %M 37910168 %R 10.2196/49605 %U https://www.jmir.org/2023/1/e49605 %U https://doi.org/10.2196/49605 %U http://www.ncbi.nlm.nih.gov/pubmed/37910168 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e48236 %T Implications for Electronic Surveys in Inpatient Settings Based on Patient Survey Response Patterns: Cross-Sectional Study %A Gregory,Megan E %A Sova,Lindsey N %A Huerta,Timothy R %A McAlearney,Ann Scheck %+ The Center for the Advancement of Team Science, Analytics, and Systems Thinking in Health Services and Implementation Science Research (CATALYST), College of Medicine, The Ohio State University, 700 Ackerman Rd, Suite 4000, Columbus, OH, 43202, United States, 1 614 293 8973, Ann.McAlearney@osumc.edu %K surveys %K patient satisfaction %K patient experience %K patient surveys %K electronic survey %K cross-sectional study %K quality of life %K mental health %K symptoms %K data quality %K hospitalization %D 2023 %7 1.11.2023 %9 Original Paper %J J Med Internet Res %G English %X Background:  Surveys of hospitalized patients are important for research and learning about unobservable medical issues (eg, mental health, quality of life, and symptoms), but there has been little work examining survey data quality in this population whose capacity to respond to survey items may differ from the general population. Objective:  The aim of this study is to determine what factors drive response rates, survey drop-offs, and missing data in surveys of hospitalized patients. Methods:  Cross-sectional surveys were distributed on an inpatient tablet to patients in a large, midwestern US hospital. Three versions were tested: 1 with 174 items and 2 with 111 items; one 111-item version had missing item reminders that prompted participants when they did not answer items. Response rate, drop-off rate (abandoning survey before completion), and item missingness (skipping items) were examined to investigate data quality. Chi-square tests, Kaplan-Meyer survival curves, and distribution charts were used to compare data quality among survey versions. Response duration was computed for each version. Results: Overall, 2981 patients responded. Response rate did not differ between the 174- and 111-item versions (81.7% vs 83%, P=.53). Drop-off was significantly reduced when the survey was shortened (65.7% vs 20.2% of participants dropped off, P<.001). Approximately one-quarter of participants dropped off by item 120, with over half dropping off by item 158. The percentage of participants with missing data decreased substantially when missing item reminders were added (77.2% vs 31.7% of participants, P<.001). The mean percentage of items with missing data was reduced in the shorter survey (40.7% vs 20.3% of items missing); with missing item reminders, the percentage of items with missing data was further reduced (20.3% vs 11.7% of items missing). Across versions, for the median participant, each item added 24.6 seconds to a survey’s duration. Conclusions:  Hospitalized patients may have a higher tolerance for longer surveys than the general population, but surveys given to hospitalized patients should have a maximum of 120 items to ensure high rates of completion. Missing item prompts should be used to reduce missing data. Future research should examine generalizability to nonhospitalized individuals. %M 37910163 %R 10.2196/48236 %U https://www.jmir.org/2023/1/e48236 %U https://doi.org/10.2196/48236 %U http://www.ncbi.nlm.nih.gov/pubmed/37910163 %0 Journal Article %@ 1929-073X %I JMIR Publications %V 12 %N %P e49973 %T A Pragmatic Mapping of Perceptions and Use of Digital Information Systems in Primary Care in Sweden: Survey Study %A Sant’Anna,Anita %A Nygren,Jens %+ School of Health and Welfare, Halmstad University, Box 823, Halmstad, 30118, Sweden, 46 035167100, jens.nygren@hh.se %K digital information systems %K implementation %K primary care %K health care professionals %K information system %K information systems %K usability %K adoption %K perception %K perceptions %K technology use %K perspective %K perspectives %D 2023 %7 25.10.2023 %9 Original Paper %J Interact J Med Res %G English %X Background: Electronic health records and IT infrastructure in primary care allow for digital documentation and access to information, which can be used to guide evidence-based care and monitor patient safety and quality of care. Quality indicators specified by regulatory authorities can be automatically computed and presented to primary care staff. However, the implementation of digital information systems (DIS) in health care can be challenging, and understanding factors such as relative advantage, compatibility, complexity, trialability, and observability is needed to improve the success and rate of adoption and diffusion. Objective: This study aims to explore how DIS are used and perceived by health care professionals in primary care. Methods: This study used quantitative assessment to gather survey data on the use and potential of DIS in health care in Sweden from the perspectives of primary care personnel in various roles. The digital questionnaire was designed to be short and contained 3 sections covering respondent characteristics, current use of platforms, and perceptions of decision support tools. Data were analyzed using descriptive statistics, nonparametric hypothesis testing, ordinal coefficient α, and confirmatory factor analysis. Results: The study collected responses from participants across 10 regions of Sweden, comprising 31.9% (n=22) from private clinics and 68.1% (n=47) from public clinics. Participants included administrators (18/69, 26.1%), a medical strategist (1/69, 1.4%), and physicians (50/69, 72.5%). Usage frequency varied as follows: 11.6% (n=8) used DIS weekly, 24.6% (n=17) monthly, 27.5% (n=19) a few times a year, 26.1% (n=18) very rarely, and 10.1% (n=7) lacked access. Administrators used DIS more frequently than physicians (P=.005). DIS use centered on quality improvement and identifying high-risk patients, with differences by role. Physicians were more inclined to use DIS out of curiosity (P=.01). Participants desired DIS for patient follow-up, lifestyle guidance, treatment suggestions, reminders, and shared decision-making. Administrators favored predictive analysis (P<.001), while physicians resisted immediate patient identification (P=.03). The 5 innovation attributes showed high internal consistency (α>.7). These factors explained 78.5% of questionnaire variance, relating to complexity, competitive advantage, compatibility, trialability, and observability. Factors 2, 3, and 4 predicted intention to use DIS, with factor 2 alone achieving the best accuracy (root-mean-square=0.513). Conclusions: Administrators and physicians exhibited role-based DIS use patterns highlighting the need for tailored approaches to promote DIS adoption. The study reveals a link between positive perceptions and intention to use DIS, emphasizing the significance of considering all factors for successful health care integration. The results suggest various directions for future studies. These include refining the trialability and observability questions for increased reliability and validity, investigating a larger sample with more specific target groups to improve generalization, and exploring the relevance of different groups’ perspectives and needs in relation to decisions about and use of DIS. %M 37878357 %R 10.2196/49973 %U https://www.i-jmr.org/2023/1/e49973 %U https://doi.org/10.2196/49973 %U http://www.ncbi.nlm.nih.gov/pubmed/37878357 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 10 %N %P e49025 %T Role of Individual Clinician Authority in the Implementation of Informatics Tools for Population-Based Medication Management: Qualitative Semistructured Interview Study %A Ranusch,Allison %A Lin,Ying-Jen %A Dorsch,Michael P %A Allen,Arthur L %A Spoutz,Patrick %A Seagull,F Jacob %A Sussman,Jeremy B %A Barnes,Geoffrey D %+ Center for Bioethics and Social Sciences in Medicine, University of Michigan, 2800 Plymouth Rd, B14 G214, Ann Arbor, MI, 48109, United States, 1 734 763 0047, gbarnes@umich.edu %K direct oral anticoagulant %K population management %K implementation science %K medical informatics %K individual clinician authority %K electronic health record %K health records %K EHR %K EHRs %K implementation %K clotting %K clot %K clots %K anticoagulant %K anticoagulants %K dashboard %K DOAC %K satisfaction %K interview %K interviews %K pharmacist %K pharmacy %K pharmacology %K medication %K prescribe %K prescribing %D 2023 %7 24.10.2023 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Direct oral anticoagulant (DOAC) medications are frequently associated with inappropriate prescribing and adverse events. To improve the safe use of DOACs, health systems are implementing population health tools within their electronic health record (EHR). While EHR informatics tools can help increase awareness of inappropriate prescribing of medications, a lack of empowerment (or insufficient empowerment) of nonphysicians to implement change is a key barrier. Objective: This study examined how the individual authority of clinical pharmacists and anticoagulation nurses is impacted by and changes the implementation success of an EHR DOAC Dashboard for safe DOAC medication prescribing. Methods: We conducted semistructured interviews with pharmacists and nurses following the implementation of the EHR DOAC Dashboard at 3 clinical sites. Interview transcripts were coded according to the key determinants of implementation success. The intersections between individual clinician authority and other determinants were examined to identify themes. Results: A high level of individual clinician authority was associated with high levels of key facilitators for effective use of the DOAC Dashboard (communication, staffing and work schedule, job satisfaction, and EHR integration). Conversely, a lack of individual authority was often associated with key barriers to effective DOAC Dashboard use. Positive individual authority was sometimes present with a negative example of another determinant, but no evidence was found of individual authority co-occurring with a positive instance of another determinant. Conclusions: Increased individual clinician authority is a necessary antecedent to the effective implementation of an EHR DOAC Population Management Dashboard and positively affects other aspects of implementation. International Registered Report Identifier (IRRID): RR2-10.1186/s13012-020-01044-5 %M 37874636 %R 10.2196/49025 %U https://humanfactors.jmir.org/2023/1/e49025 %U https://doi.org/10.2196/49025 %U http://www.ncbi.nlm.nih.gov/pubmed/37874636 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 10 %N %P e48632 %T Digital Patient Reported Outcome Measures Platform for Post–COVID-19 Condition and Other Long-Term Conditions: User-Centered Development and Technical Description %A Sivan,Manoj %A Rocha Lawrence,Román %A O'Brien,Paul %+ Leeds Institute of Rheumatic and Musculoskeletal Medicine, University of Leeds, Martin Wing, Leeds General Infirmary, Leeds, LS1 3EX, United Kingdom, 44 01133922564, M.Sivan@leeds.ac.uk %K post–COVID-19 condition %K post–COVID-19 syndrome %K post-acute COVID-19 syndrome %K C19-YRS %K Yorkshire Rehabilitation Scale %K ELAROS %K chronic conditions %K mobile app %K digital technology %K smartphone %K chronic %K respiratory %K COVID-19 %K Sars-CoV-2 %K coronavirus %K platform %K patient-reported %K outcome measure %K outcome measures %K health record %K health records %K report %K reports %K data sharing %K information sharing %K mobile phone %D 2023 %7 20.10.2023 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Post–COVID-19 condition (PCC), colloquially known as long COVID, is a multisystem condition characterized by persistent symptoms beyond 4 weeks after the SARS-CoV-2 infection. More than 60 million people with PCC worldwide need prompt assessment, diagnosis, and monitoring, with many requiring specialist help from a multidisciplinary team of health care professionals (HCPs). Consequently, a scalable digital system is required for both people with PCC and HCPs to capture the breadth of symptoms and their impact on health, using patient-reported outcome measures (PROMs) and patient-reported experience measures (PREMs). Objective: We aim to develop and implement a novel PCC digital PROM (DPROM) platform for (1) securely collecting PROM and PREM data from people with PCC, (2) enabling users to monitor symptoms longitudinally and assess response to treatment, (3) generating reports for the electronic health records (EHRs), (4) providing summary reports on PCC services based on national requirements, and (5) facilitating the sharing of relevant data with authorized research teams to accelerate our understanding of this new condition and evaluate new strategies to manage PCC. Methods: We (1) undertook requirement analysis with people with PCC, HCPs, and researchers to identify the needs of the DPROM platform and determine its required functionalities; (2) designed and developed a clinically useful web portal for staff and a mobile app for patients, with a web-based alternative app to improve patient and staff choice, limit the risk of digital exclusion, and account for variability across services; (3) determined the PROMs and PREMs that PCC services would prefer to use on the platform; and (4) designed the summary report function that can be generated for each user for the EHR and for reporting to national health authorities. Results: A DPROM platform to record PCC symptom profile, condition severity, functional disability, and quality of life, based on the C19-YRS (Yorkshire Rehabilitation Scale) and other PROMs and PREMs, was developed. Individual-level medical information and details on the COVID-19 illness can be captured systematically. The platform generates easy-to-understand scores, radar plots and line graphs for people with PCC to self-monitor their condition and for HCPs to assess the natural course of the condition and the response to interventions. Clinics can configure a suite of PROMs and PREMs based on their local and national service and commissioning requirements and support research studies which require large-scale data collection on PROMs. The DPROM platform enables automatic aggregate data analysis for services to undertake service evaluation and cost-effectiveness analysis. The DPROM platform generated summary report can be uploaded to the EHRs of people with PCC. Conclusions: A multifunctional DPROM platform to assess, grade, and monitor PCC has been developed. Future research will analyze the system’s usability in specialist PCC clinical services and other long-term conditions. %M 37665334 %R 10.2196/48632 %U https://humanfactors.jmir.org/2023/1/e48632 %U https://doi.org/10.2196/48632 %U http://www.ncbi.nlm.nih.gov/pubmed/37665334 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e43765 %T Patient and Health Care Provider Perspectives on Patient Access to Test Results via Web Portals: Scoping Review %A Petrovskaya,Olga %A Karpman,Albina %A Schilling,Joanna %A Singh,Simran %A Wegren,Larissa %A Caine,Vera %A Kusi-Appiah,Elizabeth %A Geen,Willow %+ School of Nursing, University of Victoria, PO Box 1700 STN CSC, Victoria, BC, V8W 2Y2, Canada, 1 250 472 4607, olgap@uvic.ca %K patient portal %K web portal %K MyChart %K electronic health records %K personal health records %K patient access to records %K laboratory tests %K radiology reports %K diagnostic imaging %K laboratory test results %K result release %K embargo %K the Cures Act %D 2023 %7 19.10.2023 %9 Review %J J Med Internet Res %G English %X Background: A frequently used feature of electronic patient portals is the viewing of test results. Research on patient portals is abundant and offers evidence to help portal implementers make policy and practice decisions. In contrast, no comparable comprehensive summary of research addresses the direct release of and patient access to test results. Objective: This scoping review aims to analyze and synthesize published research focused on patient and health care provider perspectives on the direct release of laboratory, imaging, and radiology results to patients via web portals. Methods: PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines were followed. Searches were conducted in CINAHL, MEDLINE, and other databases. Citations were screened in Covidence using the inclusion and exclusion criteria. Primary studies that focused on patient and health care provider perspectives on patient access to laboratory and imaging results via web portals were included. An updated search was conducted up to August 2023. Our review included 27 articles—20 examining patient views, 3 examining provider views, and 4 examining both patient and provider views. Data extraction and inductive data analysis were informed by sensitizing concepts from sociomaterial perspectives, and 15 themes were generated. Results: Patient perspectives (24 papers) were synthesized using nine themes: (1) patterns of use and patient characteristics; (2) emotional response when viewing the results and uncertainty about their implications; (3) understanding test results; (4) preferences for mode and timing of result release; (5). information seeking and patients’ actions motivated by viewing results via a portal; (6) contemplating changes in behavior and managing own health; (7) benefits of accessing test results via a portal; (8) limitations of accessing test results via a portal; and (9) suggestions for portal improvement. Health care provider perspectives (7 papers) were synthetized into six themes: (1) providers’ view of benefits of patient access to results via the portal; (2) effects on health care provider workload; (3) concerns about patient anxiety; (4) timing of result release into the patient portal; (5) the method of result release into the patient portal: manual versus automatic release; and (6) the effects of hospital health information technology system on patient quality outcomes. Conclusions: The timing of the release of test results emerged as a particularly important topic. In some countries, the policy context may motivate immediate release of most tests directly into patient portals. However, our findings aim to make policy makers, health administrators, and other stakeholders aware of factors to consider when making decisions about the timing of result release. This review is sensitive to the characteristics of patient populations and portal technology and can inform result release framework policies. The findings are timely, as patient portals have become more common internationally. %M 37856174 %R 10.2196/43765 %U https://www.jmir.org/2023/1/e43765 %U https://doi.org/10.2196/43765 %U http://www.ncbi.nlm.nih.gov/pubmed/37856174 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 7 %N %P e44065 %T Integrating Clinical Decision Support Into Electronic Health Record Systems Using a Novel Platform (EvidencePoint): Developmental Study %A Solomon,Jeffrey %A Dauber-Decker,Katherine %A Richardson,Safiya %A Levy,Sera %A Khan,Sundas %A Coleman,Benjamin %A Persaud,Rupert %A Chelico,John %A King,D'Arcy %A Spyropoulos,Alex %A McGinn,Thomas %+ Institute of Health System Science, Feinstein Institutes for Medical Research, 350 Community Drive, Manhasset, NY, 11030, United States, 1 516 600 1422, jsolomon6@northwell.edu %K clinical decision support system %K cloud based %K decision support %K development %K EHR %K electronic health record %K evidence-based medicine %K health information technology %K platform %K user-centered design %D 2023 %7 19.10.2023 %9 Original Paper %J JMIR Form Res %G English %X Background: Through our work, we have demonstrated how clinical decision support (CDS) tools integrated into the electronic health record (EHR) assist providers in adopting evidence-based practices. This requires confronting technical challenges that result from relying on the EHR as the foundation for tool development; for example, the individual CDS tools need to be built independently for each different EHR. Objective: The objective of our research was to build and implement an EHR-agnostic platform for integrating CDS tools, which would remove the technical constraints inherent in relying on the EHR as the foundation and enable a single set of CDS tools that can work with any EHR. Methods: We developed EvidencePoint, a novel, cloud-based, EHR-agnostic CDS platform, and we will describe the development of EvidencePoint and the deployment of its initial CDS tools, which include EHR-integrated applications for clinical use cases such as prediction of hospitalization survival for patients with COVID-19, venous thromboembolism prophylaxis, and pulmonary embolism diagnosis. Results: The results below highlight the adoption of the CDS tools, the International Medical Prevention Registry on Venous Thromboembolism-D-Dimer, the Wells’ criteria, and the Northwell COVID-19 Survival (NOCOS), following development, usability testing, and implementation. The International Medical Prevention Registry on Venous Thromboembolism-D-Dimer CDS was used in 5249 patients at the 2 clinical intervention sites. The intervention group tool adoption was 77.8% (4083/5249 possible uses). For the NOCOS tool, which was designed to assist with triaging patients with COVID-19 for hospital admission in the event of constrained hospital resources, the worst-case resourcing scenario never materialized and triaging was never required. As a result, the NOCOS tool was not frequently used, though the EvidencePoint platform’s flexibility and customizability enabled the tool to be developed and deployed rapidly under the emergency conditions of the pandemic. Adoption rates for the Wells’ criteria tool will be reported in a future publication. Conclusions: The EvidencePoint system successfully demonstrated that a flexible, user-friendly platform for hosting CDS tools outside of a specific EHR is feasible. The forthcoming results of our outcomes analyses will demonstrate the adoption rate of EvidencePoint tools as well as the impact of behavioral economics “nudges” on the adoption rate. Due to the EHR-agnostic nature of EvidencePoint, the development process for additional forms of CDS will be simpler than traditional and cumbersome IT integration approaches and will benefit from the capabilities provided by the core system of EvidencePoint. %M 37856193 %R 10.2196/44065 %U https://formative.jmir.org/2023/1/e44065 %U https://doi.org/10.2196/44065 %U http://www.ncbi.nlm.nih.gov/pubmed/37856193 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e48583 %T Characterizing the Patterns of Electronic Health Record–Integrated Secure Messaging Use: Cross-Sectional Study %A Baratta,Laura R %A Harford,Derek %A Sinsky,Christine A %A Kannampallil,Thomas %A Lou,Sunny S %+ Department of Anesthesiology, Washington University School of Medicine, 660 South Euclid, Campus Box 8054, Saint Louis, MO, 63110, United States, 1 314 362 1196, slou@wustl.edu %K clinical care %K clinician burden %K communication %K electronic health record %K EHR %K interprofessional communication %K medical assistant %K messaging %K nurses %K observational study %K physicians %K secure messaging %K users %D 2023 %7 6.10.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: Communication among health care professionals is essential for the delivery of safe clinical care. Secure messaging has rapidly emerged as a new mode of asynchronous communication. Despite its popularity, relatively little is known about how secure messaging is used and how such use contributes to communication burden. Objective: This study aims to characterize the use of an electronic health record–integrated secure messaging platform across 14 hospitals and 263 outpatient clinics within a large health care system. Methods: We collected metadata on the use of the Epic Systems Secure Chat platform for 6 months (July 2022 to January 2023). Information was retrieved on message volume, response times, message characteristics, messages sent and received by users, user roles, and work settings (inpatient vs outpatient). Results: A total of 32,881 users sent 9,639,149 messages during the study. Median daily message volume was 53,951 during the first 2 weeks of the study and 69,526 during the last 2 weeks, resulting in an overall increase of 29% (P=.03). Nurses were the most frequent users of secure messaging (3,884,270/9,639,149, 40% messages), followed by physicians (2,387,634/9,639,149, 25% messages), and medical assistants (1,135,577/9,639,149, 12% messages). Daily message frequency varied across users; inpatient advanced practice providers and social workers interacted with the highest number of messages per day (median 19). Conversations were predominantly between 2 users (1,258,036/1,547,879, 81% conversations), with a median of 2 conversational turns and a median response time of 2.4 minutes. The largest proportion of inpatient messages was from nurses to physicians (972,243/4,749,186, 20% messages) and physicians to nurses (606,576/4,749,186, 13% messages), while the largest proportion of outpatient messages was from physicians to nurses (344,048/2,192,488, 16% messages) and medical assistants to other medical assistants (236,694/2,192,488, 11% messages). Conclusions: Secure messaging was widely used by a diverse range of health care professionals, with ongoing growth throughout the study and many users interacting with more than 20 messages per day. The short message response times and high messaging volume observed highlight the interruptive nature of secure messaging, raising questions about its potentially harmful effects on clinician workflow, cognition, and errors. %M 37801359 %R 10.2196/48583 %U https://www.jmir.org/2023/1/e48583 %U https://doi.org/10.2196/48583 %U http://www.ncbi.nlm.nih.gov/pubmed/37801359 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 10 %N %P e47390 %T Readiness for Change in the Implementation of a 3D Printing Initiative in a Catalan Tertiary Hospital Using the Normalization Process Theory: Survey Study %A López Seguí,Francesc %A Cos Codina,Joan %A Ricou Ríos,Laura %A Martínez Segura,María Isabel %A Miró Mezquita,Laura %A Escrich Navarro,Raquel %A Davins Riu,Meritxell %A Estrada Cuxart,Oriol %A Anashkin Kachalin,German %A Moreno-Martínez,Daniel %+ Research Group on Innovation, Health Economics and Digital Transformation, Institut Germans Trias i Pujol, Camí de les Escoles, s/n, Badalona, 08916, Spain, 34 644 26 91 34, dmorenom.germanstrias@gencat.cat %K change management %K normalization process theory %K NPT %K 3D printing %K readiness for change %K Normalization Measure Development questionnaire %K NoMAD %K implementation %K tertiary hospital %K barrier %K readiness %K printing %K survey %K development %K questionnaire %K support %K communicate %K assessment %K users %K transformation %D 2023 %7 6.10.2023 %9 Original Paper %J JMIR Hum Factors %G English %X Background: The high failure rate of innovation projects motivates us to understand the perceptions about resistances and barriers of the main stakeholders to improving success rates. Objective: This study aims to analyze the readiness for change in the implementation of a 3D printing project in a Catalan tertiary hospital prior to its implementation. Methods: We used a web-based, voluntary, and anonymous survey using the Normalization Measurement Development questionnaire (NoMAD) to gather views and perceptions from a selected group of health care professionals at Germans Trias i Pujol University Hospital. Results: In this study, 58 professionals, including heads of service (n=30, 51%), doctors (n=18, 31%), nurses (n=7, 12%), and support staff (n=3, 5%), responded to the questionnaire. All groups saw the value of the project and were willing to enroll and support it. Respondents reported the highest scores (out of 5) in cognitive participation (mean 4.45, SD 0.04), coherence (mean 3.72, SD 0.13), and reflective monitoring (mean 3.80, SD 0.25). The weakest score was in collective action (mean 3.52, SD 0.12). There were no statistically significant differences in scores among professions in the survey. Conclusions: The 3D printing project implementation should pay attention to preparing, defining, sharing, and supporting the operational work involved in its use and implementation. It should also understand, assess, and communicate the ways in which the new set of practices can affect the users and others around them. We suggest that health officers and politicians consider this experience as a solid ground toward the development of a more efficient health innovation system and as a catalyst for transformation. %M 37801353 %R 10.2196/47390 %U https://humanfactors.jmir.org/2023/1/e47390 %U https://doi.org/10.2196/47390 %U http://www.ncbi.nlm.nih.gov/pubmed/37801353 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 10 %N %P e49438 %T Design and Evaluation of an Intensive Care Unit Dashboard Built in Response to the COVID-19 Pandemic: Semistructured Interview Study %A Wac,Marceli %A Craddock,Ian %A Chantziara,Sofia %A Campbell,Tabitha %A Santos-Rodriguez,Raul %A Davidson,Brittany %A McWilliams,Chris %+ Faculty of Engineering, University of Bristol, Queen's Building, University Walk, Bristol, BS8 1TR, United Kingdom, 44 117 331 5830, m.wac@bristol.ac.uk %K software engineering %K dashboard %K interactive display %K COVID-19 %K intensive care %K critical care %K intensive care unit %K ICU %K human-centered design %K participatory design %K health %K design %K interview %K electronic health record %K EHR %K electronic patient record %K EPR %K clinical information system %K CIS %K thematic analysis %D 2023 %7 26.9.2023 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Dashboards and interactive displays are becoming increasingly prevalent in most health care settings and have the potential to streamline access to information, consolidate disparate data sources and deliver new insights. Our research focuses on intensive care units (ICUs) which are heavily instrumented, critical care environments that generate vast amounts of data and frequently require individualized support for each patient. Consequently, clinicians experience a high cognitive load, which can translate to suboptimal performance. The global COVID-19 pandemic exacerbated this problem by generating a large number of additional hospitalizations, which necessitated a new tool that would help manage ICUs’ census. In a previous study, we interviewed clinicians at the University Hospitals Bristol and Weston National Health Service Foundation Trust to capture the requirements for bespoke dashboards that would alleviate this problem. Objective: This study aims to design, implement, and evaluate an ICU dashboard to allow for monitoring of the high volume of patients in need of critical care, particularly tailored to high-demand situations, such as those seen during the COVID-19 pandemic. Methods: Building upon the previously gathered requirements, we developed a dashboard, integrated it within the ICU of a National Health Service trust, and allowed all staff to access our tool. For evaluation purposes, participants were recruited and interviewed following a 25-day period during which they were able to use the dashboard clinically. The semistructured interviews followed a topic guide aimed at capturing the usability of the dashboard, supplemented with additional questions asked post hoc to probe themes established during the interview. Interview transcripts were analyzed using a thematic analysis framework that combined inductive and deductive approaches and integrated the Technology Acceptance Model. Results: A total of 10 participants with 4 different roles in the ICU (6 consultants, 2 junior doctors, 1 nurse, and 1 advanced clinical practitioner) participated in the interviews. Our analysis generated 4 key topics that prevailed across the data: our dashboard met the usability requirements of the participants and was found useful and intuitive; participants perceived that it impacted their delivery of patient care by improving the access to the information and better equipping them to do their job; the tool was used in a variety of ways and for different reasons and tasks; and there were barriers to integration of our dashboard into practice, including familiarity with existing systems, which stifled the adoption of our tool. Conclusions: Our findings show that the perceived utility of the dashboard had a positive impact on the clinicians’ workflows in the ICU. Improving access to information translated into more efficient patient care and transformed some of the existing processes. The introduction of our tool was met with positive reception, but its integration during the COVID-19 pandemic limited its adoption into practice. %M 37751239 %R 10.2196/49438 %U https://humanfactors.jmir.org/2023/1/e49438 %U https://doi.org/10.2196/49438 %U http://www.ncbi.nlm.nih.gov/pubmed/37751239 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 7 %N %P e45376 %T Developing a Semantically Based Query Recommendation for an Electronic Medical Record Search Engine: Query Log Analysis and Design Implications %A Wu,Danny T Y %A Hanauer,David %A Murdock,Paul %A Vydiswaran,V G Vinod %A Mei,Qiaozhu %A Zheng,Kai %+ Department of Biomedical Informatics, University of Cincinnati College of Medicine, 231 Albert Sabin Way, ML0840, Cincinnati, OH, 45267, United States, 1 5135586464, wutz@ucmail.uc.edu %K electronic health records %K information retrieval %K user-centered evaluation %K query recommendation %K query log analysis %K clinical research informatics %D 2023 %7 15.9.2023 %9 Original Paper %J JMIR Form Res %G English %X Background: An effective and scalable information retrieval (IR) system plays a crucial role in enabling clinicians and researchers to harness the valuable information present in electronic health records. In a previous study, we developed a prototype medical IR system, which incorporated a semantically based query recommendation (SBQR) feature. The system was evaluated empirically and demonstrated high perceived performance by end users. To delve deeper into the factors contributing to this perceived performance, we conducted a follow-up study using query log analysis. Objective: One of the primary challenges faced in IR is that users often have limited knowledge regarding their specific information needs. Consequently, an IR system, particularly its user interface, needs to be thoughtfully designed to assist users through the iterative process of refining their queries as they encounter relevant documents during their search. To address these challenges, we incorporated “query recommendation” into our Electronic Medical Record Search Engine (EMERSE), drawing inspiration from the success of similar features in modern IR systems for general purposes. Methods: The query log data analyzed in this study were collected during our previous experimental study, where we developed EMERSE with the SBQR feature. We implemented a logging mechanism to capture user query behaviors and the output of the IR system (retrieved documents). In this analysis, we compared the initial query entered by users with the query formulated with the assistance of the SBQR. By examining the results of this comparison, we could examine whether the use of SBQR helped in constructing improved queries that differed from the original ones. Results: Our findings revealed that the first query entered without SBQR and the final query with SBQR assistance were highly similar (Jaccard similarity coefficient=0.77). This suggests that the perceived positive performance of the system was primarily attributed to the automatic query expansion facilitated by the SBQR rather than users manually manipulating their queries. In addition, through entropy analysis, we observed that search results converged in scenarios of moderate difficulty, and the degree of convergence correlated strongly with the perceived system performance. Conclusions: The study demonstrated the potential contribution of the SBQR in shaping participants' positive perceptions of system performance, contingent upon the difficulty of the search scenario. Medical IR systems should therefore consider incorporating an SBQR as a user-controlled option or a semiautomated feature. Future work entails redesigning the experiment in a more controlled manner and conducting multisite studies to demonstrate the effectiveness of EMERSE with SBQR for patient cohort identification. By further exploring and validating these findings, we can enhance the usability and functionality of medical IR systems in real-world settings. %M 37713239 %R 10.2196/45376 %U https://formative.jmir.org/2023/1/e45376 %U https://doi.org/10.2196/45376 %U http://www.ncbi.nlm.nih.gov/pubmed/37713239 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 10 %N %P e50232 %T Telehealth Satisfaction in Patients Receiving Virtual Atrial Fibrillation Care: Quantitative Exploratory Study %A Rush,Kathy L %A Burton,Lindsay %A Seaton,Cherisse L %A Loewen,Peter %A O'Connor,Brian P %A Moroz,Lana %A Corman,Kendra %A Smith,Mindy A %A Andrade,Jason G %+ School of Nursing, University of British Columbia, Okanagan, 1147 Research Road, Kelowna, BC, Canada, 1 250 807 9561, kathy.rush@ubc.ca %K atrial fibrillation %K telehealth %K telehealth satisfaction %K self-efficacy %K attitudes toward technology %K attitude %K attitudes %K satisfaction %K telemedicine %K cardiology %K heart %K adoption %K eHealth %K perception %K perceptions %D 2023 %7 14.9.2023 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Telehealth can optimize access to specialty care for patients with atrial fibrillation (AF). Virtual AF care, however, may not fit with the complex needs of patients with AF. Objective: This study aims to explore the correlation among attitudes toward health care technologies, self-efficacy, and telehealth satisfaction as part of the future planning of virtual AF clinic care. Methods: Patients with AF older than 18 years from an urban-based, highly specialized AF clinic who had an upcoming telehealth visit were invited to participate in a web-based survey. The survey asked about demographic characteristics; use of technology; general, computer, and health care technology self-efficacy (HTSE) and health care technology attitudes, using a validated 30-item tool; and telehealth satisfaction questionnaire using a validated 14-item questionnaire. Data were analyzed with descriptive statistics, correlational analyses, and linear regression modeling. Results: Participants (n=195 of 579 invited, for a 34% response rate) were primarily older, male, and White, had postsecondary schooling or more, and had high self-reported overall and mental health ratings. A variety of technologies were used in their daily lives and for health care, with the majority of technologies comprising desktop and laptop computers, smartphones, and tablets. Self-efficacy and telehealth satisfaction questionnaire scores were high overall, with male participants having higher general self-efficacy, computer self-efficacy, HTSE, and technology attitude scores. After controlling for age and sex, only HTSE was significantly related to individuals’ attitudes toward health care technology. Both general self-efficacy and attitude toward health care technology were positively related to telehealth satisfaction. Conclusions: Consistent with a previous study, only HTSE significantly influenced attitudes toward health care technology. This finding confirms that, in this regard, self-efficacy is not a general perception but is domain specific. Considering participants’ predominant use of the telephone for virtual care, it follows that general self-efficacy and attitude toward health care technology were significant contributors to telehealth satisfaction. Given our patients’ frequent use of technology and high computer self-efficacy and HTSE scores, the use of video for telehealth appointments could be supported. %M 37707881 %R 10.2196/50232 %U https://humanfactors.jmir.org/2023/1/e50232 %U https://doi.org/10.2196/50232 %U http://www.ncbi.nlm.nih.gov/pubmed/37707881 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 10 %N %P e46324 %T Exploring the Challenges and Opportunities of Adopting and Using Telemedicine for Diabetes Care and Management: Qualitative Semistructured Interview Study Among Health Care Providers and Patients With Diabetes %A Altabtabaei,Rabab %A Alhuwail,Dari %+ Information Science Department, Kuwait University, Sabah Al-Salem University City, Kuwait City, 13060, Kuwait, 965 24633214, dari.alhuwail@ku.edu.kw %K health %K telemedicine %K diabetes %K challenges %K Kuwait %K technology %K informatics %K diabetes care %K management %K obstacle %K health care provider %K qualitative study %K mobile phone %D 2023 %7 7.9.2023 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Around the world, over half of the global population experiences noncommunicable diseases, resulting in premature death. Health care providers (HCPs) can deliver medical treatment from a distance by using digital advancements such as telemedicine. However, there is a limited understanding of the difficulties and opportunities of implementing telemedicine solutions in different socioeconomic and cultural environments, including Kuwait. Objective: The purpose of this study is to (1) examine the obstacles and benefits of telemedicine in the context of diabetes treatment and management, as perceived by both HCPs and patients with diabetes; (2) investigate the nonfunctional requirements for telemedicine applications used in diabetes care and management; and (3) provide suggestions to enhance the integration and adoption of telemedicine in Kuwait’s health care system for diabetes care and management. Methods: The research used a qualitative and exploratory design, with semistructured interviews as the main data collection method. Participants were recruited on the internet through social media platforms due to the COVID-19 pandemic. The results were analyzed using thematic analysis and the Framework Method. The “diffusion of innovation” model was used as a perspective to interpret the findings. Results: A total of 20 participants were included in this study—10 HCPs and 10 patients with diabetes—all of whom supported telemedicine. The HCPs reported that many diabetes cases could be managed through telemedicine, with only a few requiring in-person visits. Patients with diabetes noted the convenience and time-saving aspect of telemedicine. Both groups recommended the creation of a secure and user-friendly telemedicine system similar to popular social media platforms. Additionally, participants emphasized the importance of telemedicine during the pandemic as a way to prioritize patient safety. Conclusions: The results of this study provide valuable insights into the needs and preferences of both HCPs and patients with diabetes in a resource-rich country like Kuwait to embrace telemedicine. The COVID-19 pandemic has changed the way medical care is provided and has pushed both groups to consider digital solutions for ongoing diabetes management and treatment. %M 37676711 %R 10.2196/46324 %U https://humanfactors.jmir.org/2023/1/e46324 %U https://doi.org/10.2196/46324 %U http://www.ncbi.nlm.nih.gov/pubmed/37676711 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 10 %N %P e41552 %T Alerts and Collections for Automating Patients’ Sensemaking and Organizing of Their Electronic Health Record Data for Reflection, Planning, and Clinical Visits: Qualitative Research-Through-Design Study %A Nakikj,Drashko %A Kreda,David %A Gehlenborg,Nils %+ Department of Biomedical Informatics, Harvard Medical School, Harvard University, 10 Shattuck Street, Suite 514, Boston, MA, 02115, United States, 1 857 272 5075, nils@hms.harvard.edu %K patients %K electronic health records %K sensemaking %K pattern detection %K data organization %K alerts %K reports %K collections %D 2023 %7 21.8.2023 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Electronic health record (EHR) data from multiple providers often exhibit important but convoluted and complex patterns that patients find hard and time-consuming to identify and interpret. However, existing patient-facing applications lack the capability to incorporate automatic pattern detection robustly and toward supporting making sense of the patient’s EHR data. In addition, there is no means to organize EHR data in an efficient way that suits the patient’s needs and makes them more actionable in real-life settings. These shortcomings often result in a skewed and incomplete picture of the patient’s health status, which may lead to suboptimal decision-making and actions that put the patient at risk. Objective: Our main goal was to investigate patients’ attitudes, needs, and use scenarios with respect to automatic support for surfacing important patterns in their EHR data and providing means for organizing them that best suit patients’ needs. Methods: We conducted an inquisitive research-through-design study with 14 participants. Presented in the context of a cutting-edge application with strong emphasis on independent EHR data sensemaking, called Discovery, we used high-level mock-ups for the new features that were supposed to support automatic identification of important data patterns and offer recommendations—Alerts—and means for organizing the medical records based on patients’ needs, much like photos in albums—Collections. The combined audio recording transcripts and in-study notes were analyzed using the reflexive thematic analysis approach. Results: The Alerts and Collections can be used for raising awareness, reflection, planning, and especially evidence-based patient-provider communication. Moreover, patients desired carefully designed automatic pattern detection with safe and actionable recommendations, which produced a well-tailored and scoped landscape of alerts for both potential threats and positive progress. Furthermore, patients wanted to contribute their own data (eg, progress notes) and log feelings, daily observations, and measurements to enrich the meaning and enable easier sensemaking of the alerts and collections. On the basis of the findings, we renamed Alerts to Reports for a more neutral tone and offered design implications for contextualizing the reports more deeply for increased actionability; automatically generating the collections for more expedited and exhaustive organization of the EHR data; enabling patient-generated data input in various formats to support coarser organization, richer pattern detection, and learning from experience; and using the reports and collections for efficient, reliable, and common-ground patient-provider communication. Conclusions: Patients need to have a flexible and rich way to organize and annotate their EHR data; be introduced to insights from these data—both positive and negative; and share these artifacts with their physicians in clinical visits or via messaging for establishing shared mental models for clear goals, agreed-upon priorities, and feasible actions. %M 37603400 %R 10.2196/41552 %U https://humanfactors.jmir.org/2023/1/e41552 %U https://doi.org/10.2196/41552 %U http://www.ncbi.nlm.nih.gov/pubmed/37603400 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e46478 %T Video Recording Patients for Direct Care Purposes: Systematic Review and Narrative Synthesis of International Empirical Studies and UK Professional Guidance %A Lear,Rachael %A Ellis,Sophia %A Ollivierre-Harris,Tiffany %A Long,Susannah %A Mayer,Erik K %+ Imperial Clinical Analytics, Research & Evaluation (iCARE), Digital Collaboration Space, St Mary's Hospital Campus, 1a Sheldon Square, Imperial College London, London, W2 6PY, United Kingdom, 44 07969064617, r.lear12@imperial.ac.uk %K video recording %K audio-visual media %K digital health %K electronic health records %K mobile phone %D 2023 %7 16.8.2023 %9 Review %J J Med Internet Res %G English %X Background: Video recordings of patients may offer advantages to supplement patient assessment and clinical decision-making. However, little is known about the practice of video recording patients for direct care purposes. Objective: We aimed to synthesize empirical studies published internationally to explore the extent to which video recording patients is acceptable and effective in supporting direct care and, for the United Kingdom, to summarize the relevant guidance of professional and regulatory bodies. Methods: Five electronic databases (MEDLINE, Embase, APA PsycINFO, CENTRAL, and HMIC) were searched from 2012 to 2022. Eligible studies evaluated an intervention involving video recording of adult patients (≥18 years) to support diagnosis, care, or treatment. All study designs and countries of publication were included. Websites of UK professional and regulatory bodies were searched to identify relevant guidance. The acceptability of video recording patients was evaluated using study recruitment and retention rates and a framework synthesis of patients’ and clinical staff’s perspectives based on the Theoretical Framework of Acceptability by Sekhon. Clinically relevant measures of impact were extracted and tabulated according to the study design. The framework approach was used to synthesize the reported ethico-legal considerations, and recommendations of professional and regulatory bodies were extracted and tabulated. Results: Of the 14,221 abstracts screened, 27 studies met the inclusion criteria. Overall, 13 guidance documents were retrieved, of which 7 were retained for review. The views of patients and clinical staff (16 studies) were predominantly positive, although concerns were expressed about privacy, technical considerations, and integrating video recording into clinical workflows; some patients were anxious about their physical appearance. The mean recruitment rate was 68.2% (SD 22.5%; range 34.2%-100%; 12 studies), and the mean retention rate was 73.3% (SD 28.6%; range 16.7%-100%; 17 studies). Regarding effectiveness (10 studies), patients and clinical staff considered video recordings to be valuable in supporting assessment, care, and treatment; in promoting patient engagement; and in enhancing communication and recall of information. Observational studies (n=5) favored video recording, but randomized controlled trials (n=5) did not demonstrate that video recording was superior to the controls. UK guidelines are consistent in their recommendations around consent, privacy, and storage of recordings but lack detailed guidance on how to operationalize these recommendations in clinical practice. Conclusions: Video recording patients for direct care purposes appears to be acceptable, despite concerns about privacy, technical considerations, and how to incorporate recording into clinical workflows. Methodological quality prevents firm conclusions from being drawn; therefore, pragmatic trials (particularly in older adult care and the movement disorders field) should evaluate the impact of video recording on diagnosis, treatment monitoring, patient-clinician communication, and patient safety. Professional and regulatory documents should signpost to practical guidance on the implementation of video recording in routine practice. Trial Registration: PROSPERO CRD42022331825: https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=331825 %M 37585249 %R 10.2196/46478 %U https://www.jmir.org/2023/1/e46478 %U https://doi.org/10.2196/46478 %U http://www.ncbi.nlm.nih.gov/pubmed/37585249 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e45043 %T Examining Care Planning Efficiency and Clinical Decision Support Adoption in a System Tailoring to Nurses’ Graph Literacy: National, Web-Based Randomized Controlled Trial %A Yao,Yingwei %A Dunn Lopez,Karen %A Bjarnadottir,Ragnhildur I %A Macieira,Tamara G R %A Dos Santos,Fabiana Cristina %A Madandola,Olatunde O %A Cho,Hwayoung %A Priola,Karen J B %A Wolf,Jessica %A Wilkie,Diana J %A Keenan,Gail %+ University of Florida College of Nursing, 1225 Center Dr, Gainesville, FL, 32610, United States, 1 352 273 6352, gkeenan@ufl.edu %K clinical decision support %K nurse decision-making %K nurse care planning %K simulation %K remote testing %K tailored interfaces %K graph literacy %K cognitive workload %D 2023 %7 11.8.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: The proliferation of health care data in electronic health records (EHRs) is fueling the need for clinical decision support (CDS) that ensures accuracy and reduces cognitive processing and documentation burden. The CDS format can play a key role in achieving the desired outcomes. Building on our laboratory-based pilot study with 60 registered nurses (RNs) from 1 Midwest US metropolitan area indicating the importance of graph literacy (GL), we conducted a fully powered, innovative, national, and web-based randomized controlled trial with 203 RNs. Objective: This study aimed to compare care planning time (CPT) and the adoption of evidence-based CDS recommendations by RNs randomly assigned to 1 of 4 CDS format groups: text only (TO), text+table (TT), text+graph (TG), and tailored (based on the RN’s GL score). We hypothesized that the tailored CDS group will have faster CPT (primary) and higher adoption rates (secondary) than the 3 nontailored CDS groups. Methods: Eligible RNs employed in an adult hospital unit within the past 2 years were recruited randomly from 10 State Board of Nursing lists representing the 5 regions of the United States (Northeast, Southeast, Midwest, Southwest, and West) to participate in a randomized controlled trial. RNs were randomly assigned to 1 of 4 CDS format groups—TO, TT, TG, and tailored (based on the RN’s GL score)—and interacted with the intervention on their PCs. Regression analysis was performed to estimate the effect of tailoring and the association between CPT and RN characteristics. Results: The differences between the tailored (n=46) and nontailored (TO, n=55; TT, n=54; and TG, n=48) CDS groups were not significant for either the CPT or the CDS adoption rate. RNs with low GL had longer CPT interacting with the TG CDS format than the TO CDS format (P=.01). The CPT in the TG CDS format was associated with age (P=.02), GL (P=.02), and comfort with EHRs (P=.047). Comfort with EHRs was also associated with CPT in the TT CDS format (P<.001). Conclusions: Although tailoring based on GL did not improve CPT or adoption, the study reinforced previous pilot findings that low GL is associated with longer CPT when graphs were included in care planning CDS. Higher GL, younger age, and comfort with EHRs were associated with shorter CPT. These findings are robust based on our new innovative testing strategy in which a diverse national sample of RN participants (randomly derived from 10 State Board of Nursing lists) interacted on the web with the intervention on their PCs. Future studies applying our innovative methodology are recommended to cost-effectively enhance the understanding of how the RN’s GL, combined with additional factors, can inform the development of efficient CDS for care planning and other EHR components before use in practice. %M 37566456 %R 10.2196/45043 %U https://www.jmir.org/2023/1/e45043 %U https://doi.org/10.2196/45043 %U http://www.ncbi.nlm.nih.gov/pubmed/37566456 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 7 %N %P e48363 %T Challenges and Solutions in Implementing eSource Technology for Real-World Studies in China: Qualitative Study Among Different Stakeholders %A Wang,Bin %A Lai,Junkai %A Liao,Xiwen %A Jin,Feifei %A Yao,Chen %+ Peking University Clinical Research Institute, Peking University First Hospital, No. 8 Xishiku Street, Xicheng District, Beijing, 100191, China, 86 01066551053, yaochen@hsc.pku.edu.cn %K electronic medical record %K electronic source %K eSource %K challenge %K real-world data %K interoperability %D 2023 %7 10.8.2023 %9 Original Paper %J JMIR Form Res %G English %X Background: eSources consist of data that were initially documented in an electronic structure. Typically, an eSource encompasses the direct acquisition, compilation, and retention of electronic information (such as electronic health records [EHRs] or wearable devices), which serves to streamline clinical research. eSources have the potential to enhance the accuracy of data, promote patient safety, and minimize expenses associated with clinical trials. An opinion study published in September 2020 by TransCelerate outlined a road map for the future application of eSource technology and identified 5 key areas of challenges. The background of this study concerns the use of eSource technology in clinical research. Objective: The aim of this study was to present challenges and possible solutions for the implementation of eSource technology in real-world studies by summarizing team experiences and lessons learned from an eSource record (ESR) project. Methods: After initially developing a simple prototype of the ESR software that can be demonstrated systematically, the researchers conducted in-depth interviews and interacted with different stakeholders to obtain guidance and suggestions. The researchers selected 5 different roles for interviewees: regulatory authorities, pharmaceutical company representatives, hospital information department employees, medical system providers, and clinicians. Results: After screening all consultants, the researchers concluded that there were 25 representative consultants. The hospital information department needs to implement many demands from various stakeholders, which makes the existing EHR system unable to meet all the demands of eSources. The emergence of an ESR is intended to divert the burden of the hospital information department from the enormous functional requirements of the outdated EHR system. The entire research process emphasizes multidisciplinary and multibackground expert opinions and considers the complexity of the knowledge backgrounds of personnel involved in the chain of clinical source data collection, processing, quality control, and application in real-world scenarios. To increase the readability of the results, the researchers classified the main results in accordance with the paragraph titles in “Use of Electronic Health Record Data in Clinical Investigations,” a guide released by the US Food and Drug Administration. Conclusions: This study introduces the requirement dependencies of different stakeholders and the challenges and recommendations for designing ESR software when implementing eSource technology in China. Experiences based on ESR projects will provide new insights into the disciplines that advance the eSource research field. Future studies should engage patients directly to understand their experiences, concerns, and preferences regarding the implementation of eSource technology. Moreover, involving additional stakeholders, including community health care providers and social workers, will provide valuable insights into the challenges and potential solutions across various health care settings. %M 37561551 %R 10.2196/48363 %U https://formative.jmir.org/2023/1/e48363 %U https://doi.org/10.2196/48363 %U http://www.ncbi.nlm.nih.gov/pubmed/37561551 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e47958 %T Automated Electronic Health Record to Electronic Data Capture Transfer in Clinical Studies in the German Health Care System: Feasibility Study and Gap Analysis %A Mueller,Christian %A Herrmann,Patrick %A Cichos,Stephan %A Remes,Bernhard %A Junker,Erwin %A Hastenteufel,Tobias %A Mundhenke,Markus %+ Bayer Vital GmbH, Kaiser-Wilhelm-Allee 70, Leverkusen, 51373, Germany, 49 175 3005134, christian.mueller4@bayer.com %K digital transformation %K automated data transfer %K electronic medical record %K electronic data capture %K EDC %K data transfer %K electronic health record %K EHR %K digital transfer %K barrier %K clinical practice %K EHR2EDC %K health care system %D 2023 %7 4.8.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: Data transfer between electronic health records (EHRs) at the point of care and electronic data capture (EDC) systems for clinical research is still mainly carried out manually, which is error-prone as well as cost- and time-intensive. Automated digital transfer from EHRs to EDC systems (EHR2EDC) would enable more accurate and efficient data capture but has so far encountered technological barriers primarily related to data format and the technological environment: in Germany, health care data are collected at the point of care in a variety of often individualized practice management systems (PMSs), most of them not interoperable. Data quality for research purposes within EDC systems must meet the requirements of regulatory authorities for standardized submission of clinical trial data and safety reports. Objective: We aimed to develop a model for automated data transfer as part of an observational study that allows data of sufficient quality to be captured at the point of care, extracted from various PMSs, and automatically transferred to electronic case report forms in EDC systems. This required addressing aspects of data security, as well as the lack of compatibility between EHR health care data and the data quality required in EDC systems for clinical research. Methods: The SaniQ software platform (Qurasoft GmbH) is already used to extract and harmonize predefined variables from electronic medical records of different Compu Group Medical–hosted PMSs. From there, data are automatically transferred to the validated AlcedisTRIAL EDC system (Alcedis GmbH) for data collection and management. EHR2EDC synchronization occurs automatically overnight, and real-time updates can be initiated manually following each data entry in the EHR. The electronic case report form (eCRF) contains 13 forms with 274 variables. Of these, 5 forms with 185 variables contain 67 automatically transferable variables (67/274, 24% of all variables and 67/185, 36% of eligible variables). Results: This model for automated data transfer bridges the current gap between clinical practice data capture at the point of care and the data sets required by regulatory agencies; it also enables automated EHR2EDC data transfer in compliance with the General Data Protection Regulation (GDPR). It addresses feasibility, connectivity, and system compatibility of currently used PMSs in health care and clinical research and is therefore directly applicable. Conclusions: This use case demonstrates that secure, consistent, and automated end-to-end data transmission from the treating physician to the regulatory authority is feasible. Automated data transmission can be expected to reduce effort and save resources and costs while ensuring high data quality. This may facilitate the conduct of studies for both study sites and sponsors, thereby accelerating the development of new drugs. Nevertheless, the industry-wide implementation of EHR2EDC requires policy decisions that set the framework for the use of research data based on routine PMS data. %M 37540555 %R 10.2196/47958 %U https://www.jmir.org/2023/1/e47958 %U https://doi.org/10.2196/47958 %U http://www.ncbi.nlm.nih.gov/pubmed/37540555 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 10 %N %P e46528 %T Optimizing Patient-Reported Outcome Collection and Documentation in Medical Music Therapy: Process-Improvement Study %A Rodgers-Melnick,Samuel N %A Block,Seneca %A Rivard,Rachael L %A Dusek,Jeffery A %+ University Hospitals Connor Whole Health, 11100 Euclid Avenue, Cleveland, OH, 44106, United States, 1 2168447727, Samuel.RodgersMelnick@UHhospitals.org %K electronic health record %K integrative medicine %K music therapy %K pain %K quality improvement %D 2023 %7 27.7.2023 %9 Original Paper %J JMIR Hum Factors %G English %X Background: To measure the effectiveness of nonpharmacologic interventions delivered during clinical care, investigators need to ensure robust and routine data collection without disrupting individualized patient care or adding unnecessary documentation burden. Objective: A process-improvement study was undertaken to improve documentation consistency and increase the capture of patient-reported outcomes (PROs; ie, stress, pain, anxiety, and coping) within a medical music therapy (MT) team. Methods: We used 2 Plan-Do-Study-Act (PDSA) cycles to improve documentation processes among an MT team (13.3 clinical full-time equivalent staff). Trainings focused on providing skills and resources for optimizing pre- and postsession PRO collection, specific guidelines for entering session data in the electronic health record, and opportunities for the team to provide feedback. Two comparisons of therapists’ PRO collection rates were conducted: (1) between the 6 months before PDSA Cycle 1 (T0) and PDSA Cycle 1 (T1), and (2) between T1 and PDSA Cycle 2 (T2). Results: Music therapists’ rates of capturing any PRO within MT sessions increased significantly (P<.001) from T0 to T1 and from T1 to T2 for all domains, including stress (4/2758, 0.1% at T0; 1012/2786, 36.3% at T1; and 393/775, 50.7% at T2), pain (820/2758, 29.7% at T0; 1444/2786, 51.8% at T1; and 476/775, 61.4% at T2), anxiety (499/2758, 18.1% at T0; 950/2786, 34.1% at T1; and 400/775, 51.6% at T2), and coping (0/2758, 0% at T0; 571/2786, 20.5% at T1; and 319/775, 41.2% at T2). Music therapists’ feedback and findings from a retrospective analysis were used to create an improved electronic health record documentation template. Conclusions: Rates of PRO data collection improved within the medical MT team. Although the process improvement in this study was applied to a nonpharmacologic MT intervention, the principles are applicable to numerous inpatient clinical providers. As hospitals continue to implement nonpharmacologic therapies in response to the Joint Commission’s recommendations, routine PRO collection will provide future researchers with the ability to evaluate the impact of these therapies on pain relief and opioid use. %M 37498646 %R 10.2196/46528 %U https://humanfactors.jmir.org/2023/1/e46528 %U https://doi.org/10.2196/46528 %U http://www.ncbi.nlm.nih.gov/pubmed/37498646 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 10 %N %P e42993 %T The Polarization of Clinician and Service Staff Perspectives After the Use of Health Information Technology in Youth Mental Health Services: Implementation and Evaluation Study %A McKenna,Sarah %A Piper,Sarah %A Capon,William %A Crowley,Alison %A Lira,Lucas %A LaMonica,Haley M %A Chong,Min Kyung %A Scott,Elizabeth %A Hickie,Ian %A Iorfino,Frank %+ Faculty of Medicine and Health, Brain and Mind Centre, The University of Sydney, 94 Mallett Street, Camperdown, 2050, Australia, 61 0419517723, sarah.mckenna@sydney.edu.au %K mental health %K youth %K adolescent %K service delivery %K implementation science %K digital technologies %K measurement-based care %K health information technology %K information system %K perspective %K provider %K health care staff %K health care worker %K health care professional %D 2023 %7 25.7.2023 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Highly personalized care is substantially improved by technology platforms that assess and track patient outcomes. However, evidence regarding how to successfully implement technology in real-world mental health settings is limited. Objective: This study aimed to naturalistically monitor how a health information technology (HIT) platform was used within 2 real-world mental health service settings to gain practical insights into how HIT can be implemented and sustained to improve mental health service delivery. Methods: An HIT (The Innowell Platform) was naturally implemented in 2 youth mental health services in Sydney, Australia. Web-based surveys (n=19) and implementation logs were used to investigate staff attitudes toward technology before and after implementation. Descriptive statistics were used to track staff attitudes over time, whereas qualitative thematic analysis was used to explore implementation log data to gain practical insights into useful implementation strategies in real-world settings. Results: After the implementation, the staff were nearly 3 times more likely to agree that the HIT would improve care for their clients (3/12, 25% agreed before the implementation compared with 7/10, 70% after the implementation). Despite this, there was also an increase in the number of staff who disagreed that the HIT would improve care (from 1/12, 8% to 2/10, 20%). There was also decreased uncertainty (from 6/12, 50% to 3/10, 30%) about the willingness of the service to implement the technology for its intended purpose, with similar increases in the number of staff who agreed and disagreed with this statement. Staff were more likely to be uncertain about whether colleagues in my service are receptive to changes in clinical processes (not sure rose from 5/12, 42% to 7/10, 70%). They were also more likely to report that their service already provides the best mental health care (agreement rose from 7/12, 58% to 8/10, 80%). After the implementation, a greater proportion of participants reported that the HIT enabled shared or collaborative decision-making with young people (2/10, 20%, compared with 1/12, 8%), enabled clients to proactively work on their mental health care through digital technologies (3/10, 30%, compared with 2/12, 16%), and improved their response to suicidal risk (4/10, 40% compared with 3/12, 25%). Conclusions: This study raises important questions about why clinicians, who have the same training and support in using technology, develop more polarized opinions on its usefulness after implementation. It seems that the uptake of HIT is heavily influenced by a clinician’s underlying beliefs and attitudes toward clinical practice in general as well as the role of technology, rather than their knowledge or the ease of use of the HIT in question. %M 37490321 %R 10.2196/42993 %U https://humanfactors.jmir.org/2023/1/e42993 %U https://doi.org/10.2196/42993 %U http://www.ncbi.nlm.nih.gov/pubmed/37490321 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 10 %N %P e44747 %T Understanding the Role of Patient Portals in Fostering Interprofessional Collaboration Within Mental Health Care Settings: Mixed Methods Study %A Durocher,Keri %A Shin,Hwayeon Danielle %A Lo,Brian %A Chen,Sheng %A Ma,Clement %A Strudwick,Gillian %+ Campbell Family Mental Health Research Institute, Centre for Addiction and Mental Health, 60 White Squirrel Way, Toronto, ON, M6J 1H4, Canada, 1 (416) 599 1448, kduroch3@uwo.ca %K mental health %K patient portal %K mixed methods %K interprofessional collaboration %K communication %K self-empowerment %K nursing informatics %K digital health %D 2023 %7 19.7.2023 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Patient portals are web-based systems through which patients can access their personal health information and communicate with their clinicians. The integration of patient portals into mental health care settings has been evolving over the past decade, as cumulated research to date has highlighted the potential role of portals in facilitating positive health outcomes. However, it is currently unknown whether portal use can foster interprofessional collaboration between clinicians and patients or whether the portal is a tool to support an already established collaborative relationship. Objective: This mixed methods study aimed to understand how the use of a patient portal within mental health settings can impact the level of interprofessional collaboration between clinicians and patients. Methods: This study was conducted in a large mental health care organization in Ontario, Canada. A convergent mixed methods design was used, where the primary data collection methods included questionnaires and semistructured interviews with patients who had experience using a portal for their mental health care. For the quantitative strand, participants completed the Health Care Communication Questionnaire and the Self-Empowerment subscale of the Mental Health Recovery Measure at 3 time points (baseline, 3 months of use, and 6 months of use) to measure changes in scores over time. For the qualitative strand, semistructured interviews were conducted at the 3-month time point to assess the elements of interprofessional collaboration associated with the portal. Results: For the quantitative strand, 113 participants completed the questionnaire. For the Health Care Communication Questionnaire scores, the raw means of the total scores at the 3 time points were as follows: baseline, 43.01 (SD 7.28); three months, 43.19 (SD 6.65); and 6 months, 42.74 (SD 6.84). In the univariate model with time as the only independent variable, the scores did not differ significantly across the 3 time points (P=.70). For the Mental Health Recovery Measure scores, the raw mean total scores at the 3 time points were as follows: baseline, 10.77 (SD 3.63); three months, 11.09 (SD 3.81); and 6 months, 11.10 (SD 3.33). In the univariate model with time as the only independent variable, the scores did not differ significantly across the 3 time points (P=.34). For the qualitative strand, 10 participants were interviewed and identified various elements of how interprofessional collaboration can be supplemented through the use of a patient portal, including improved team functioning, communication, and conflict resolution. Conclusions: Although the quantitative data produced nonsignificant findings in interprofessional collaboration scores over time, the patients’ narrative accounts described how the portal can support various interprofessional collaboration concepts, such as communication, leadership, and conflict resolution. This provides useful information for clinicians to support the interprofessional relationship when using a portal within a mental health setting. International Registered Report Identifier (IRRID): RR2-10.1136/bmjopen-2018-025508 %M 37467024 %R 10.2196/44747 %U https://humanfactors.jmir.org/2023/1/e44747 %U https://doi.org/10.2196/44747 %U http://www.ncbi.nlm.nih.gov/pubmed/37467024 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 10 %N %P e45598 %T Involving Health Care Professionals in the Development of Electronic Health Records: Scoping Review %A Busse,Theresa Sophie %A Jux,Chantal %A Laser,Johannes %A Rasche,Peter %A Vollmar,Horst Christian %A Ehlers,Jan P %A Kernebeck,Sven %+ Institute of General Practice and Family Medicine (AM RUB), Medical Faculty, Ruhr University Bochum, Universitätsstraße 150, MA 1/158, Bochum, 44801, Germany, 49 23432 ext 27782, theresa.busse@ruhr-uni-bochum.de %K user-centered design %K electronic health records %K electronic medical records %K digital technology %K technology development %K stakeholder participation %D 2023 %7 10.7.2023 %9 Review %J JMIR Hum Factors %G English %X Background: Electronic health records (EHRs) are a promising approach to document and map (complex) health information gathered in health care worldwide. However, possible unintended consequences during use, which can occur owing to low usability or the lack of adaption to existing workflows (eg, high cognitive load), may pose a challenge. To prevent this, the involvement of users in the development of EHRs is crucial and growing. Overall, involvement is designed to be very multifaceted, for example, in terms of the timing, frequency, or even methods used to capture user preferences. Objective: Setting, users and their needs, and the context and practice of health care must be considered in the design and subsequent implementation of EHRs. Many different approaches to user involvement exist, each requiring a variety of methodological choices. The aim of the study was to provide an overview of the existing forms of user involvement and the circumstances they need and to provide support for the planning of new involvement processes. Methods: We conducted a scoping review to provide a database for future projects on which design of inclusion is worthwhile and to show the diversity of reporting. Using a very broad search string, we searched the PubMed, CINAHL, and Scopus databases. In addition, we searched Google Scholar. Hits were screened according to scoping review methodology and then examined, focusing on methods and materials, participants, frequency and design of the development, and competencies of the researchers involved. Results: In total, 70 articles were included in the final analysis. There was a wide range of methods of involvement. Physicians and nurses were the most frequently included groups and, in most cases, were involved only once in the process. The approach of involvement (eg, co-design) was not specified in most of the studies (44/70, 63%). Further qualitative deficiencies in the reporting were evident in the presentation of the competences of members of the research and development teams. Think-aloud sessions, interviews, and prototypes were frequently used. Conclusions: This review provides insights into the diversity of health care professionals’ involvement in the development of EHRs. It provides an overview of the different approaches in various fields of health care. However, it also shows the necessity of considering quality standards in the development of EHRs together with future users and the need for reporting this in future studies. %M 37428524 %R 10.2196/45598 %U https://humanfactors.jmir.org/2023/1/e45598 %U https://doi.org/10.2196/45598 %U http://www.ncbi.nlm.nih.gov/pubmed/37428524 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e45944 %T The Integration of Clinical Decision Support Systems Into Telemedicine for Patients With Multimorbidity in Primary Care Settings: Scoping Review %A Wiwatkunupakarn,Nutchar %A Aramrat,Chanchanok %A Pliannuom,Suphawita %A Buawangpong,Nida %A Pinyopornpanish,Kanokporn %A Nantsupawat,Nopakoon %A Mallinson,Poppy Alice Carson %A Kinra,Sanjay %A Angkurawaranon,Chaisiri %+ Department of Family Medicine, Faculty of Medicine, Chiang Mai University, 110 Intawaroros Road, Chiang Mai, 50200, Thailand, 66 616852307, chaisiri.a@cmu.ac.th %K telemedicine %K clinical decision support system %K CDSS %K primary care %K multimorbidity %K polypharmacy %K chronic disease %K pharmacy %K pharmaceutic %K telehealth %K decision support %K scoping %K search strategy %K review %D 2023 %7 28.6.2023 %9 Review %J J Med Internet Res %G English %X Background: Multimorbidity, the presence of more than one condition in a single individual, is a global health issue in primary care. Multimorbid patients tend to have a poor quality of life and suffer from a complicated care process. Clinical decision support systems (CDSSs) and telemedicine are the common information and communication technologies that have been used to reduce the complexity of patient management. However, each element of telemedicine and CDSSs is often examined separately and with great variability. Telemedicine has been used for simple patient education as well as more complex consultations and case management. For CDSSs, there is variability in data inputs, intended users, and outputs. Thus, there are several gaps in knowledge about how to integrate CDSSs into telemedicine and to what extent these integrated technological interventions can help improve patient outcomes for those with multimorbidity. Objective: Our aims were to (1) broadly review system designs for CDSSs that have been integrated into each function of telemedicine for multimorbid patients in primary care, (2) summarize the effectiveness of the interventions, and (3) identify gaps in the literature. Methods: An online search for literature was conducted up to November 2021 on PubMed, Embase, CINAHL, and Cochrane. Searching from the reference lists was done to find additional potential studies. The eligibility criterion was that the study focused on the use of CDSSs in telemedicine for patients with multimorbidity in primary care. The system design for the CDSS was extracted based on its software and hardware, source of input, input, tasks, output, and users. Each component was grouped by telemedicine functions: telemonitoring, teleconsultation, tele–case management, and tele-education. Results: Seven experimental studies were included in this review: 3 randomized controlled trials (RCTs) and 4 non-RCTs. The interventions were designed to manage patients with diabetes mellitus, hypertension, polypharmacy, and gestational diabetes mellitus. CDSSs can be used for various telemedicine functions: telemonitoring (eg, feedback), teleconsultation (eg, guideline suggestions, advisory material provisions, and responses to simple queries), tele–case management (eg, sharing information across facilities and teams), and tele-education (eg, patient self-management). However, the structure of CDSSs, such as data input, tasks, output, and intended users or decision-makers, varied. With limited studies examining varying clinical outcomes, there was inconsistent evidence of the clinical effectiveness of the interventions. Conclusions: Telemedicine and CDSSs have a role in supporting patients with multimorbidity. CDSSs can likely be integrated into telehealth services to improve the quality and accessibility of care. However, issues surrounding such interventions need to be further explored. These issues include expanding the spectrum of medical conditions examined; examining tasks of CDSSs, particularly for screening and diagnosis of multiple conditions; and exploring the role of the patient as the direct user of the CDSS. %M 37379066 %R 10.2196/45944 %U https://www.jmir.org/2023/1/e45944 %U https://doi.org/10.2196/45944 %U http://www.ncbi.nlm.nih.gov/pubmed/37379066 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 7 %N %P e44549 %T Evaluation of the EsteR Toolkit for COVID-19 Decision Support: Sensitivity Analysis and Usability Study %A Alpers,Rieke %A Kühne,Lisa %A Truong,Hong-Phuc %A Zeeb,Hajo %A Westphal,Max %A Jäckle,Sonja %+ Fraunhofer Institute for Digital Medicine MEVIS, Max-von-Laue-Str. 2, Bremen, 28209, Germany, 49 421 218 ext 59228, rieke.alpers@mevis.fraunhofer.de %K COVID-19 %K public health %K decision support tool %K sensitivity analysis %K web application %K usability study %D 2023 %7 27.6.2023 %9 Original Paper %J JMIR Form Res %G English %X Background: During the COVID-19 pandemic, local health authorities were responsible for managing and reporting current cases in Germany. Since March 2020, employees had to contain the spread of COVID-19 by monitoring and contacting infected persons as well as tracing their contacts. In the EsteR project, we implemented existing and newly developed statistical models as decision support tools to assist in the work of the local health authorities. Objective: The main goal of this study was to validate the EsteR toolkit in two complementary ways: first, investigating the stability of the answers provided by our statistical tools regarding model parameters in the back end and, second, evaluating the usability and applicability of our web application in the front end by test users. Methods: For model stability assessment, a sensitivity analysis was carried out for all 5 developed statistical models. The default parameters of our models as well as the test ranges of the model parameters were based on a previous literature review on COVID-19 properties. The obtained answers resulting from different parameters were compared using dissimilarity metrics and visualized using contour plots. In addition, the parameter ranges of general model stability were identified. For the usability evaluation of the web application, cognitive walk-throughs and focus group interviews were conducted with 6 containment scouts located at 2 different local health authorities. They were first asked to complete small tasks with the tools and then express their general impressions of the web application. Results: The simulation results showed that some statistical models were more sensitive to changes in their parameters than others. For each of the single-person use cases, we determined an area where the respective model could be rated as stable. In contrast, the results of the group use cases highly depended on the user inputs, and thus, no area of parameters with general model stability could be identified. We have also provided a detailed simulation report of the sensitivity analysis. In the user evaluation, the cognitive walk-throughs and focus group interviews revealed that the user interface needed to be simplified and more information was necessary as guidance. In general, the testers rated the web application as helpful, especially for new employees. Conclusions: This evaluation study allowed us to refine the EsteR toolkit. Using the sensitivity analysis, we identified suitable model parameters and analyzed how stable the statistical models were in terms of changes in their parameters. Furthermore, the front end of the web application was improved with the results of the conducted cognitive walk-throughs and focus group interviews regarding its user-friendliness. %M 37368487 %R 10.2196/44549 %U https://formative.jmir.org/2023/1/e44549 %U https://doi.org/10.2196/44549 %U http://www.ncbi.nlm.nih.gov/pubmed/37368487 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e43838 %T Personalized Risk Analysis to Improve the Psychological Resilience of Women Undergoing Treatment for Breast Cancer: Development of a Machine Learning–Driven Clinical Decision Support Tool %A C Manikis,Georgios %A Simos,Nicholas J %A Kourou,Konstantina %A Kondylakis,Haridimos %A Poikonen-Saksela,Paula %A Mazzocco,Ketti %A Pat-Horenczyk,Ruth %A Sousa,Berta %A Oliveira-Maia,Albino J %A Mattson,Johanna %A Roziner,Ilan %A Marzorati,Chiara %A Marias,Kostas %A Nuutinen,Mikko %A Karademas,Evangelos %A Fotiadis,Dimitrios %+ Foundation for Research and Technology-Hellas, Biomedical Research Institute, University campus of Ioannina, Building E5, Ioannina, GR 45110, Greece, 30 2651009006, fotiadis@uoi.gr %K breast cancer %K classification %K machine learning %K mental health %K well-being %K explainability %K interventions %K risk assessment %D 2023 %7 12.6.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: Health professionals are often faced with the need to identify women at risk of manifesting poor psychological resilience following the diagnosis and treatment of breast cancer. Machine learning algorithms are increasingly used to support clinical decision support (CDS) tools in helping health professionals identify women who are at risk of adverse well-being outcomes and plan customized psychological interventions for women at risk. Clinical flexibility, cross-validated performance accuracy, and model explainability permitting person-specific identification of risk factors are highly desirable features of such tools. Objective: This study aimed to develop and cross-validate machine learning models designed to identify breast cancer survivors at risk of poor overall mental health and global quality of life and identify potential targets of personalized psychological interventions according to an extensive set of clinical recommendations. Methods: A set of 12 alternative models was developed to improve the clinical flexibility of the CDS tool. All models were validated using longitudinal data from a prospective, multicenter clinical pilot at 5 major oncology centers in 4 countries (Italy, Finland, Israel, and Portugal; the Predicting Effective Adaptation to Breast Cancer to Help Women to BOUNCE Back [BOUNCE] project). A total of 706 patients with highly treatable breast cancer were enrolled shortly after diagnosis and before the onset of oncological treatments and were followed up for 18 months. An extensive set of demographic, lifestyle, clinical, psychological, and biological variables measured within 3 months after enrollment served as predictors. Rigorous feature selection isolated key psychological resilience outcomes that could be incorporated into future clinical practice. Results: Balanced random forest classifiers were successful at predicting well-being outcomes, with accuracies ranging between 78% and 82% (for 12-month end points after diagnosis) and between 74% and 83% (for 18-month end points after diagnosis). Explainability and interpretability analyses built on the best-performing models were used to identify potentially modifiable psychological and lifestyle characteristics that, if addressed systematically in the context of personalized psychological interventions, would be most likely to promote resilience for a given patient. Conclusions: Our results highlight the clinical utility of the BOUNCE modeling approach by focusing on resilience predictors that can be readily available to practicing clinicians at major oncology centers. The BOUNCE CDS tool paves the way for personalized risk assessment methods to identify patients at high risk of adverse well-being outcomes and direct valuable resources toward those most in need of specialized psychological interventions. %M 37307043 %R 10.2196/43838 %U https://www.jmir.org/2023/1/e43838 %U https://doi.org/10.2196/43838 %U http://www.ncbi.nlm.nih.gov/pubmed/37307043 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e42649 %T Involving Health Professionals in the Development of Quality and Safety Dashboards: Qualitative Study %A van de Baan,Frank Christian %A Lambregts,Stijn %A Bergman,Esther %A Most,Jasper %A Westra,Daan %+ Department of Health Services Research, Care and Public Health Research Institute, Maastricht University, Duboisdomein 30, Maastricht, 6229GT, Netherlands, 31 043 38 81 553, f.vandebaan@maastrichtuniversity.nl %K quality improvement %K dashboard %K user involvement %K innovation %K health care provider %K health care professional %K feedback %K opinion %K perspective %K qualitative %K constant comparative method %D 2023 %7 12.6.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: Dashboards are an important tool for hospitals to improve quality and safety performance. However, implementing quality and safety dashboards often does not increase performance due to a lack of use by health professionals. Including health professionals in the development process of quality and safety dashboards can improve their use in practice. Yet, it remains unclear how a development process involving health professionals can be executed successfully. Objective: The aim of this study is twofold: (1) to delineate how a process whereby health professionals are included in the development of quality and safety dashboards can be facilitated and (2) to identify the factors that are important to consider in order to make that process successful. Methods: We conducted a qualitative, in-depth exploratory case study in which we analyzed 150 pages of internal documents and interviewed 13 staff members regarding the development of quality and safety dashboards within 2 care pathways of a hospital that has experience in such development. The data were analyzed inductively using the constant comparative method. Results: We found that the development of quality and safety dashboards in collaboration with health professionals was facilitated through a five-stage process: (1) familiarizing participants with dashboards and the development process; (2) brainstorming about potential indicators to be included in the dashboard; (3) prioritizing, defining, and selecting indicators to be included in the dashboard; (4) examining how the indicators can be visualized; and (5) implementing the dashboard and following up on its use. To enhance the success of the process, 3 factors were deemed important. The first is to create and maintain broad involvement, ensuring that various professions are represented and take ownership of the dashboard. Here, potential barriers include gaining engagement from peers not directly involved in the process and maintaining involvement after the initial implementation of the dashboard. Second, unburdening, whereby quality and safety staff facilitate a structured process that has little additional burden for professionals. For this, time management and a lack of collaboration with departments responsible for delivering the data might be an issue. Lastly, focusing on relevance for health professionals, which refers to the inclusion of indicators with value for health professionals. For this factor, a lack of consensus on how indicators should be defined and registered might be a barrier. Conclusions: Health care organizations seeking to develop quality and safety dashboards in collaboration with health professionals can use a 5-stage process. To enhance the success of the process, organizations are advised to focus on 3 key factors. For each of the key factors, potential barriers should be taken into account. Engaging in this process and attaining the key factors could increase the likelihood that the dashboards are used in practice. %M 37307058 %R 10.2196/42649 %U https://www.jmir.org/2023/1/e42649 %U https://doi.org/10.2196/42649 %U http://www.ncbi.nlm.nih.gov/pubmed/37307058 %0 Journal Article %@ 2562-7600 %I JMIR Publications %V 6 %N %P e40000 %T A Nurse-Led Telehealth Program for Diabetes Foot Care: Feasibility and Usability Study %A Ju,Hsiao-Hui %A Momin,Rashmi %A Cron,Stanley %A Jularbal,Jed %A Alford,Jeffery %A Johnson,Constance %+ The University of Texas Health Science Center at Houston Cizik School of Nursing, 6901 Bertner Avenue, SON 748, Houston, TX, 77030, United States, 1 713 500 2278, hsiao-hui.ju@uth.tmc.edu %K diabetes mellitus, type 2 %K telehealth %K telemedicine %K foot care education %K self-management %K self-care %K preventive health services %K patient education %K diabetes complications %K diabetic foot %D 2023 %7 6.6.2023 %9 Original Paper %J JMIR Nursing %G English %X Background: Diabetes mellitus can lead to severe and debilitating foot complications, such as infections, ulcerations, and amputations. Despite substantial progress in diabetes care, foot disease remains a major challenge in managing this chronic condition that causes serious health complications worldwide. Objective: The primary aim of this study was to examine the feasibility and usability of a telehealth program focused on preventive diabetes foot care. A secondary aim was to descriptively measure self-reported changes in diabetes knowledge, self-care, and foot care behaviors before and after participating in the program. Methods: The study used a single-arm, pre-post design in 2 large family medical practice clinics in Texas. Participants met individually with the nurse practitioner once a month for 3 months using synchronous telehealth videoconferencing. Each participant received diabetes foot education guided by the Integrated Theory of Health Behavior Change. Feasibility was measured with rates of enrollment and program and assessment completion. Usability was measured with the Telehealth Usability Questionnaire. Diabetes knowledge, self-care, and foot care behaviors were measured with validated survey instruments at baseline, 1.5 months, and 3 months. Results: Of 50 eligible individuals, 39 (78%) enrolled; 34 of 39 (87%) completed the first videoconference and 29 of 39 (74%) completed the second and third videoconferences. Of the 39 who consented, 37 (95%) completed the baseline assessment; 50% (17/34) of those who attended the first videoconference completed the assessment at 1.5 months, and 100% (29/29) of those who attended the subsequent videoconferences completed the final assessment. Overall, participants reported a positive attitude toward the use of telehealth, with a mean Telehealth Usability Questionnaire score of 6.24 (SD 0.98) on a 7-point scale. Diabetes knowledge increased by a mean of 15.82 (SD 16.69) points of 100 (P<.001) from baseline to 3 months. The values for the Summary of Diabetes Self-Care Activities measure demonstrated better self-care, with participants performing foot care on average 1.74 (SD 2.04) more days per week (P<.001), adhering to healthy eating habits on average 1.57 (SD 2.12) more days per week (P<.001), and being physically active on average 1.24 (SD 2.21) more days per week (P=.005). Participants also reported an improvement in the frequency of foot self-examinations and general foot care behaviors. The mean scores for foot care increased by a mean of 7.65 (SD 7.04) points (scale of 7 to 35) from baseline to 3 months postintervention (P<.001). Conclusions: This study demonstrates that a nurse-led telehealth educational program centered on diabetes foot care is feasible, acceptable, and has the potential to improve diabetes knowledge and self-care, which are precursors to preventing debilitating foot complications. %M 37279046 %R 10.2196/40000 %U https://nursing.jmir.org/2023/1/e40000 %U https://doi.org/10.2196/40000 %U http://www.ncbi.nlm.nih.gov/pubmed/37279046 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 10 %N %P e43551 %T Decision Aids for Patients With Head and Neck Cancer: Qualitative Elicitation of Design Recommendations From Patient End Users %A Stringer,Eleah %A Lum,Julian J %A Livergant,Jonathan %A Kushniruk,Andre W %+ School of Health Information Science, University of Victoria, 3800 Finnerty Road, Victoria, BC, V8P 5C2, Canada, 1 250 721 6461, eleah@uvic.ca %K decision support %K decision aid %K app design %K oncology %K head and neck cancer %K patient information needs %K qualitative %D 2023 %7 5.6.2023 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Patients with head and neck cancer (HNC) carry a clinically significant symptom burden, have alterations in function (eg, impaired ability to chew, swallow, and talk), and decrease in quality of life. Furthermore, treatment impacts social activities and interactions as patients report reduced sexuality and shoulder the highest rates of depression across cancer types. Patients suffer undue anxiety because they find the treatment incomprehensible, which is partially a function of limited, understandable information. Patients’ perceptions of having obtained adequate information prior to and during treatment are predictive of positive outcomes. Providing patient-centered decision support and utilizing visual images may increase understanding of treatment options and associated risks to improve satisfaction with their decision and consultation, while reducing decisional conflict. Objective: This study aims to gather requirements from survivors of HNC on the utility of key visual components to be used in the design of an electronic decision aid (eDA) to assist with decision-making on treatment options. Methods: Informed by a scoping review on eDAs for patients with HNC, screens and visualizations for an eDA were created and then presented to 12 survivors of HNC for feedback on their utility, features, and further requirements. The semistructured interviews were video-recorded and thematically analyzed to inform co-design recommendations. Results: A total of 9 themes were organized into 2 categories. The first category, eDAs and decision support, included 3 themes: familiarity with DAs, support of concept, and versatility of the prototype. The second category, evaluation of mock-up, contained 6 themes: reaction to the screens and visualizations, favorite features, complexity, preference for customizability, presentation device, and suggestions for improvement. Conclusions: All participants felt an eDA, used in the presence of their oncologist, would support a more thorough and transparent explanation of treatment or augment the quality of education received. Participants liked the simple design of the mock-ups they were shown but, ultimately, desired customizability to adapt the eDA to their individual information needs. This research highlights the value of user-centered design, rooted in acceptability and utility, in medical health informatics, recognizing cancer survivors as the ultimate knowledge holders. This research highlights the value of incorporating visuals into technology-based innovations to engage all patients in treatment decisions. %M 37276012 %R 10.2196/43551 %U https://humanfactors.jmir.org/2023/1/e43551 %U https://doi.org/10.2196/43551 %U http://www.ncbi.nlm.nih.gov/pubmed/37276012 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e47659 %T The Effects of Online Access to General Practice Medical Records Perceived by Patients: Longitudinal Survey Study %A Thielmann,Rosa R L C %A Hoving,Ciska %A Cals,Jochen W L %A Crutzen,Rik %+ School for Public Health and Primary Care, Faculty of Health, Medicine and Life Sciences, Maastricht University, Debyeplein 1, Maastricht, 6226 HA, Netherlands, 31 38 82423, r.thielmann@maastrichtuniversity.nl %K electronic health records %K personal health records %K medical records %K patient access to records %K patient portals %K patient participation %K informed decision-making %K patient empowerment %D 2023 %7 2.6.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: Patient online access to medical records is assumed to facilitate patient empowerment and advance patient-centered health care. However, to date, the actual effects of online access to medical records perceived by patients and other outcomes are insufficiently empirically tested. Objective: This study aimed to investigate the effects of online access to medical records on patient empowerment, informed decision-making, and the patient-provider relationship perceived by patients. Methods: A nationwide, 2-wave, longitudinal survey study was conducted among Dutch adults (N=2402). Linear regression analyses were performed. In model 1, the perceived effects of online access to medical records (measured at T1 [first measurement; July 2021]) on 16 outcomes (measured at T2 [second measurement; January 2022]), which were associated with the use of online access to general practice medical records in previous research, were investigated. Model 2 included sociodemographic factors and patient characteristics as confounders. Results: Users indicated more strongly than nonusers that online access to medical records would increase their participation in health care, improve the relationship with their general practitioner, and support informed decision-making. These results were robust when adjusted for the influence of confounders. Effect sizes were very small, with unstandardized regression coefficients (B) ranging between −0.39 and 0.28. Higher digital and health literacy were associated with higher ratings of almost all effects. Conclusions: Online access to medical records has the potential to empower patients and foster informed decision-making among patients. The effects in this study were small but might grow over time. Other factors, such as the attitude of general practitioners toward online access to medical records, might moderate these effects. The results indicate that the potential benefits of online access to medical records might be unevenly distributed. We suggest future exploration of the conditions under which online access to medical records can improve health care system functioning and efficiency without increasing health inequality. %M 37266981 %R 10.2196/47659 %U https://www.jmir.org/2023/1/e47659 %U https://doi.org/10.2196/47659 %U http://www.ncbi.nlm.nih.gov/pubmed/37266981 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 7 %N %P e44567 %T Migrating a Well-Established Longitudinal Cohort Database From Oracle SQL to Research Electronic Data Entry (REDCap): Data Management Research and Design Study %A Kusejko,Katharina %A Smith,Daniel %A Scherrer,Alexandra %A Paioni,Paolo %A Kohns Vasconcelos,Malte %A Aebi-Popp,Karoline %A Kouyos,Roger D %A Günthard,Huldrych F %A Kahlert,Christian R %A , %+ Institute of Medical Virology, University of Zurich, Universitaetsstrasse 84, Zurich, 8006, Switzerland, 41 44 634 1913, katharina.kusejko@usz.ch %K REDCap %K cohort study %K data collection %K electronic case report forms %K eCRF %K software %K digital solution %K electronic data entry %K HIV %D 2023 %7 31.5.2023 %9 Original Paper %J JMIR Form Res %G English %X Background: Providing user-friendly electronic data collection tools for large multicenter studies is key for obtaining high-quality research data. Research Electronic Data Capture (REDCap) is a software solution developed for setting up research databases with integrated graphical user interfaces for electronic data entry. The Swiss Mother and Child HIV Cohort Study (MoCHiV) is a longitudinal cohort study with around 2 million data entries dating back to the early 1980s. Until 2022, data collection in MoCHiV was paper-based. Objective: The objective of this study was to provide a user-friendly graphical interface for electronic data entry for physicians and study nurses reporting MoCHiV data. Methods: MoCHiV collects information on obstetric events among women living with HIV and children born to mothers living with HIV. Until 2022, MoCHiV data were stored in an Oracle SQL relational database. In this project, R and REDCap were used to develop an electronic data entry platform for MoCHiV with migration of already collected data. Results: The key steps for providing an electronic data entry option for MoCHiV were (1) design, (2) data cleaning and formatting, (3) migration and compliance, and (4) add-on features. In the first step, the database structure was defined in REDCap, including the specification of primary and foreign keys, definition of study variables, and the hierarchy of questions (termed “branching logic”). In the second step, data stored in Oracle were cleaned and formatted to adhere to the defined database structure. Systematic data checks ensured compliance to all branching logic and levels of categorical variables. REDCap-specific variables and numbering of repeated events for enabling a relational data structure in REDCap were generated using R. In the third step, data were imported to REDCap and then systematically compared to the original data. In the last step, add-on features, such as data access groups, redirections, and summary reports, were integrated to facilitate data entry in the multicenter MoCHiV study. Conclusions: By combining different software tools—Oracle SQL, R, and REDCap—and building a systematic pipeline for data cleaning, formatting, and comparing, we were able to migrate a multicenter longitudinal cohort study from Oracle SQL to REDCap. REDCap offers a flexible way for developing customized study designs, even in the case of longitudinal studies with different study arms (ie, obstetric events, women, and mother-child pairs). However, REDCap does not offer built-in tools for preprocessing large data sets before data import. Additional software is needed (eg, R) for data formatting and cleaning to achieve the predefined REDCap data structure. %M 37256686 %R 10.2196/44567 %U https://formative.jmir.org/2023/1/e44567 %U https://doi.org/10.2196/44567 %U http://www.ncbi.nlm.nih.gov/pubmed/37256686 %0 Journal Article %@ 2369-3762 %I JMIR Publications %V 9 %N %P e38079 %T Examining Pediatric Resident Electronic Health Records Use During Prerounding: Mixed Methods Observational Study %A Alami,Jawad %A Hammonds,Clare %A Hensien,Erin %A Khraibani,Jenan %A Borowitz,Stephen %A Hellems,Martha %A Riggs,Sara %+ Department of Systems and Information Engineering, University of Virginia, 151 Engineer's Way, Charlottesville, VA, 22903, United States, 1 434 243 5342, aalami@virginia.edu %K EHR %K pediatric %K usability %K prerounding %K training %K electronic health record %K eHealth %D 2023 %7 10.5.2023 %9 Original Paper %J JMIR Med Educ %G English %X Background: Electronic health records (EHRs) play a substantial role in modern health care, especially during prerounding, when residents gather patient information to inform daily care decisions of the care team. The effective use of the EHR system is crucial for efficient and frustration-free prerounding. Ideally, the system should be designed to support efficient user interactions by presenting data effectively and providing easy navigation between different pages. Additionally, training on the system should aim to make user interactions more efficient by familiarizing the users with best practices that minimize interaction time while using the full potential of the system’s capabilities. However, formal training on EHR systems often falls short of providing residents with all the necessary EHR-related skills, leading to the adoption of inefficient practices and the underuse of the system’s full range of capabilities. Objective: This study aims to examine the efficiency of EHR use during prerounding among pediatric residents, assess the effect of experience level on EHR use, and identify areas for improvement in EHR design and training. Methods: A mixed methods approach was used, involving a self-reported survey and video analysis of prerounding practices of the entire population of pediatric residents from a large teaching hospital in the South Atlantic Region. The residents were stratified by experience level by postgraduate year. Data were collected on the number of pages accessed, duration of prerounding, task completion rates, and effective use of data sources. Observational and qualitative data complemented the quantitative analysis. Our study followed the STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) reporting guidelines, ensuring completeness and transparency of reporting. Results: Of the 30 pediatric residents, 20 were included in the analyses; of these, 16 (80%) missed at least 1 step during prerounding. Although more experienced residents on average omitted fewer steps, 4 (57%) of the 7 most experienced residents still omitted at least 1 step. On average, residents took 6.5 minutes to round each patient and accessed 21 pages within the EHR during prerounding; no statistically significant differences were observed between experience levels for prerounding times (P=.48) or number of pages accessed (P=.92). The use of aggregated data pages within the EHR system neither seem to improve prerounding times nor decrease the number of pages accessed. Conclusions: The findings suggest that EHR design should be improved to better support user needs, and hospitals should adopt more effective training programs to familiarize residents with the system’s capabilities. We recommend implementing prerounding checklists and providing ongoing EHR training programs for health care practitioners. Despite the generalizability of limitations of our study in terms of sample size and specialization, it offers valuable insights for future research to investigate the impact of EHR use on patient outcomes and satisfaction, as well as identify factors that contribute to efficient and effective EHR usage. %M 37163346 %R 10.2196/38079 %U https://mededu.jmir.org/2023/1/e38079 %U https://doi.org/10.2196/38079 %U http://www.ncbi.nlm.nih.gov/pubmed/37163346 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 10 %N %P e44644 %T A Visual Analytic Tool (VIADS) to Assist the Hypothesis Generation Process in Clinical Research: Mixed Methods Usability Study %A Jing,Xia %A Patel,Vimla L %A Cimino,James J %A Shubrook,Jay H %A Zhou,Yuchun %A Draghi,Brooke N %A Ernst,Mytchell A %A Liu,Chang %A De Lacalle,Sonsoles %+ Department of Public Health Sciences, College of Behavioral, Social and Health Sciences, Clemson University, 519 Edwards Hall, Clemson, SC, 29634, United States, 1 8646563347, xia.xjing@gmail.com %K usability %K VIADS %K data-driven hypothesis generation %K visualization %K clinical research %K SUS %K mixed methods study %D 2023 %7 27.4.2023 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Visualization can be a powerful tool to comprehend data sets, especially when they can be represented via hierarchical structures. Enhanced comprehension can facilitate the development of scientific hypotheses. However, the inclusion of excessive data can make visualizations overwhelming. Objective: We developed a visual interactive analytic tool for filtering and summarizing large health data sets coded with hierarchical terminologies (VIADS). In this study, we evaluated the usability of VIADS for visualizing data sets of patient diagnoses and procedures coded in the International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM). Methods: We used mixed methods in the study. A group of 12 clinical researchers participated in the generation of data-driven hypotheses using the same data sets and time frame (a 1-hour training session and a 2-hour study session) utilizing VIADS via the think-aloud protocol. The audio and screen activities were recorded remotely. A modified version of the System Usability Scale (SUS) survey and a brief survey with open-ended questions were administered after the study to assess the usability of VIADS and verify their intense usage experience with VIADS. Results: The range of SUS scores was 37.5 to 87.5. The mean SUS score for VIADS was 71.88 (out of a possible 100, SD 14.62), and the median SUS was 75. The participants unanimously agreed that VIADS offers new perspectives on data sets (12/12, 100%), while 75% (8/12) agreed that VIADS facilitates understanding, presentation, and interpretation of underlying data sets. The comments on the utility of VIADS were positive and aligned well with the design objectives of VIADS. The answers to the open-ended questions in the modified SUS provided specific suggestions regarding potential improvements for VIADS, and the identified problems with usability were used to update the tool. Conclusions: This usability study demonstrates that VIADS is a usable tool for analyzing secondary data sets with good average usability, good SUS score, and favorable utility. Currently, VIADS accepts data sets with hierarchical codes and their corresponding frequencies. Consequently, only specific types of use cases are supported by the analytical results. Participants agreed, however, that VIADS provides new perspectives on data sets and is relatively easy to use. The VIADS functionalities most appreciated by participants were the ability to filter, summarize, compare, and visualize data. International Registered Report Identifier (IRRID): RR2-10.2196/39414 %M 37011112 %R 10.2196/44644 %U https://humanfactors.jmir.org/2023/1/e44644 %U https://doi.org/10.2196/44644 %U http://www.ncbi.nlm.nih.gov/pubmed/37011112 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 10 %N %P e43782 %T Empowering Researchers to Query Medical Data and Biospecimens by Ensuring Appropriate Usability of a Feasibility Tool: Evaluation Study %A Schüttler,Christina %A Zerlik,Maria %A Gruendner,Julian %A Köhler,Thomas %A Rosenau,Lorenz %A Prokosch,Hans-Ulrich %A Sedlmayr,Brita %+ Central Biobank Erlangen, University Hospital Erlangen, Krankenhausstr. 12, Erlangen, 91054, Germany, 49 913185 47290, christina.schuettler@fau.de %K usability evaluation %K ontology %K feasibility queries %K user-centered design %K clinical research informatics %K user interface %D 2023 %7 19.4.2023 %9 Original Paper %J JMIR Hum Factors %G English %X Background: The Aligning Biobanking and Data Integration Centers Efficiently project aims to harmonize technologies and governance structures of German university hospitals and their biobanks to facilitate searching for patient data and biospecimens. The central element will be a feasibility tool for researchers to query the availability of samples and data to determine the feasibility of their study project. Objective: The objectives of the study were as follows: an evaluation of the overall user interface usability of the feasibility tool, the identification of critical usability issues, comprehensibility of the underlying ontology operability, and analysis of user feedback on additional functionalities. From these, recommendations for quality-of-use optimization, focusing on more intuitive usability, were derived. Methods: To achieve the study goal, an exploratory usability test consisting of 2 main parts was conducted. In the first part, the thinking aloud method (test participants express their thoughts aloud throughout their use of the tool) was complemented by a quantitative questionnaire. In the second part, the interview method was combined with supplementary mock-ups to collect users’ opinions on possible additional features. Results: The study cohort rated global usability of the feasibility tool based on the System Usability Scale with a good score of 81.25. The tasks assigned posed certain challenges. No participant was able to solve all tasks correctly. A detailed analysis showed that this was mostly because of minor issues. This impression was confirmed by the recorded statements, which described the tool as intuitive and user friendly. The feedback also provided useful insights regarding which critical usability problems occur and need to be addressed promptly. Conclusions: The findings indicate that the prototype of the Aligning Biobanking and Data Integration Centers Efficiently feasibility tool is headed in the right direction. Nevertheless, we see potential for optimization primarily in the display of the search functions, the unambiguous distinguishability of criteria, and the visibility of their associated classification system. Overall, it can be stated that the combination of different tools used to evaluate the feasibility tool provided a comprehensive picture of its usability. %M 37074765 %R 10.2196/43782 %U https://humanfactors.jmir.org/2023/1/e43782 %U https://doi.org/10.2196/43782 %U http://www.ncbi.nlm.nih.gov/pubmed/37074765 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 10 %N %P e43960 %T Performance of a Web-Based Reference Database With Natural Language Searching Capabilities: Usability Evaluation of DynaMed and Micromedex With Watson %A Rui,Angela %A Garabedian,Pamela M %A Marceau,Marlika %A Syrowatka,Ania %A Volk,Lynn A %A Edrees,Heba H %A Seger,Diane L %A Amato,Mary G %A Cambre,Jacob %A Dulgarian,Sevan %A Newmark,Lisa P %A Nanji,Karen C %A Schultz,Petra %A Jackson,Gretchen Purcell %A Rozenblum,Ronen %A Bates,David W %+ Division of General Internal Medicine, Brigham and Women's Hospital, 75 Francis St, Boston, MA, 02115, United States, 1 978 397 0082, arui@partners.org %K medication safety %K patient safety %K usability %K searching behavior %K efficiency %K quality of care %K web-based databases %K point-of-care information %K POCI %K point-of-care tools %K artificial intelligence %K machine learning %K clinical decision support %K natural language processing %D 2023 %7 17.4.2023 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Evidence-based point-of-care information (POCI) tools can facilitate patient safety and care by helping clinicians to answer disease state and drug information questions in less time and with less effort. However, these tools may also be visually challenging to navigate or lack the comprehensiveness needed to sufficiently address a medical issue. Objective: This study aimed to collect clinicians’ feedback and directly observe their use of the combined POCI tool DynaMed and Micromedex with Watson, now known as DynaMedex. EBSCO partnered with IBM Watson Health, now known as Merative, to develop the combined tool as a resource for clinicians. We aimed to identify areas for refinement based on participant feedback and examine participant perceptions to inform further development. Methods: Participants (N=43) within varying clinical roles and specialties were recruited from Brigham and Women’s Hospital and Massachusetts General Hospital in Boston, Massachusetts, United States, between August 10, 2021, and December 16, 2021, to take part in usability sessions aimed at evaluating the efficiency and effectiveness of, as well as satisfaction with, the DynaMed and Micromedex with Watson tool. Usability testing methods, including think aloud and observations of user behavior, were used to identify challenges regarding the combined tool. Data collection included measurements of time on task; task ease; satisfaction with the answer; posttest feedback on likes, dislikes, and perceived reliability of the tool; and interest in recommending the tool to a colleague. Results: On a 7-point Likert scale, pharmacists rated ease (mean 5.98, SD 1.38) and satisfaction (mean 6.31, SD 1.34) with the combined POCI tool higher than the physicians, nurse practitioner, and physician’s assistants (ease: mean 5.57, SD 1.64, and satisfaction: mean 5.82, SD 1.60). Pharmacists spent longer (mean 2 minutes, 26 seconds, SD 1 minute, 41 seconds) on average finding an answer to their question than the physicians, nurse practitioner, and physician’s assistants (mean 1 minute, 40 seconds, SD 1 minute, 23 seconds). Conclusions: Overall, the tool performed well, but this usability evaluation identified multiple opportunities for improvement that would help inexperienced users. %M 37067858 %R 10.2196/43960 %U https://humanfactors.jmir.org/2023/1/e43960 %U https://doi.org/10.2196/43960 %U http://www.ncbi.nlm.nih.gov/pubmed/37067858 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 7 %N %P e39383 %T Usability and Acceptability of Electronic Immunization Registry Data Entry Workflows From the Health Care Worker Perspective in Siaya, Kenya (Part 3): Pre-Post Study %A Wittenauer,Rachel %A Dolan,Samantha B %A Njoroge,Anne %A Onyango,Penina %A Owiso,George %A Rabinowitz,Peter %A Puttkammer,Nancy %+ Department of Global Health, University of Washington, 3980 15th Ave NE, Seattle, WA, 98195, United States, 1 6502694771, rwitten1@uw.edu %K workflow %K digital health %K electronic immunization registry %K usability %K acceptability %K health care worker %K technology adoption %D 2023 %7 30.3.2023 %9 Original Paper %J JMIR Form Res %G English %X Background: Digital health tools such as electronic immunization registries (EIRs) have the potential to improve patient care and alleviate the challenges that arise from the use of paper-based clinic records for reporting. To address some of these challenges, the Kenya Ministry of Health and the International Training and Education Center for Health Kenya implemented an EIR system in 161 immunizing clinics in Siaya County between 2018 and 2019. The successful implementation of digital health tools depends on many factors, one of which is alignment between the technology and the context in which it is used. One important aspect of that implementation context is the perceptions of the health care workers (HCWs) using the EIR. Objective: This study aimed to evaluate HCWs’ perceptions of the usability and acceptability of multiple clinic workflows using the new EIR. Methods: We performed a mixed methods pre-post study using semistructured interviews with HCWs at 6 facilities in Siaya County, Kenya. We interviewed HCWs at each facility 4 times: at baseline and once after the implementation of 3 different workflow modifications (n=24 interviews). The baseline state was dual data entry with paper records and the EIR. We then implemented 3 workflow modifications for 1 full day each: fully paperless data entry, preparation of an appointment diary before patient visits for the day, and a combination of the 2 workflows. We compared ratings and themes across interviews after each of the 4 workflows to understand the changes in the usability and acceptability of the EIR. Results: HCWs considered the EIR clinic workflows to be usable and acceptable. Of the modified workflows, HCWs perceived the fully paperless workflow most favorably. In all workflows, HCWs’ perceived benefits included ease of clinical decision-making using the EIR, reduced mental burden of data entry when using the EIR, and ease of identification of errors. Perceived barriers to the workflow included contextual challenges such as staffing shortages and lack of network connectivity, EIR platform challenges such as errors in saving records and missing fields, and workflow challenges such as the dual data entry burden of using paper and digital tools simultaneously. Conclusions: Fully paperless EIR implementation shows great promise from a workflow acceptability standpoint, contingent upon the presence of supporting contextual clinic factors and the resolution of system performance and design challenges. Rather than trying to identify a singular best workflow, future efforts should provide adequate flexibility for HCWs to implement the new system in their unique clinic context. Future EIR implementation stands to benefit from continued monitoring of EIR adoption acceptability during implementation, both for Siaya’s program and for other efforts around the globe, as digital health interventions become more widely used. %M 36995755 %R 10.2196/39383 %U https://formative.jmir.org/2023/1/e39383 %U https://doi.org/10.2196/39383 %U http://www.ncbi.nlm.nih.gov/pubmed/36995755 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 7 %N %P e39777 %T Time Utilization Among Immunization Clinics Using an Electronic Immunization Registry (Part 2): Time and Motion Study of Modified User Workflows %A Dolan,Samantha B %A Wittenauer,Rachel %A Njoroge,Anne %A Onyango,Penina %A Owiso,George %A Shearer,Jessica C %A Lober,William B %A Liu,Shan %A Puttkammer,Nancy %A Rabinowitz,Peter %+ Bill and Melinda Gates Foundation, 500 5th Ave N, Seattle, WA, 98109, United States, 1 206 709 3100, sdolan11@gmail.com %K immunizations %K electronic immunization registry %K workflow %K time and motion %K digital health intervention %K mixed methods evaluation %D 2023 %7 16.3.2023 %9 Original Paper %J JMIR Form Res %G English %X Background: Digital health interventions have the potential to improve the provision of health care services through digitized data collection and management. Low- and middle-income countries are beginning to introduce electronic immunization registries (EIRs) into their routine immunization services to better capture and store childhood vaccination information. Especially in Africa, where 25% of children remain unimmunized or underimmunized, technologies that can help identify children due for a vaccination are particularly important for improving vaccination coverage. However, an improved understanding of the effectiveness of these systems is needed to develop and deploy sustainable EIRs in low- and middle-income countries. Objective: We conducted an interventional pretest-posttest design study that sought to improve time efficiency through workflow modifications in Kenyan immunization clinics. Our aim was to describe how activity times differed after introducing workflow modifications that could potentially reduce the time needed to perform routine data entry activities. Our intent was to demonstrate changes in efficiency when moving from the existing dual–data entry workflow to a future paperless workflow by health facility size and experience length of health care workers (HCWs). Methods: We tested how 3 workflow modifications would affect time utilization among HCWs using the EIR at the point of care compared with baseline immunization clinic workflows. Our outcome of interest was the time taken to complete individual activities and a patient’s total time in the clinic where we compared the time spent during the baseline workflow with that during the modified workflow. We used a standardized tool to observe and document the immunization clinic workflow. To estimate differences in time utilization, we used bivariate analyses and fit multivariate linear mixed-effects models. Results: Our study found that for HCWs using an EIR, the introduction of modified workflows decreased the amount of time needed to provide services to children seen in the immunization clinic. With a baseline mean time of 10 minutes spent per child, this decreased by about 3 minutes when the preparation modification was introduced and almost 5 minutes for the paperless and combined modifications. Results pertaining to the EIR’s performance and ability to connect to the internet were particularly insightful about potential causes of delays. Conclusions: We were able to conduct a concise clinical simulation exercise by introducing modified workflows and estimating their impact on time utilization in immunization clinics using an EIR. We found that the paperless workflow provided the largest time savings when delivering services, although this was threatened by poor EIR performance and internet connectivity. This study demonstrated that not only should digital health interventions be built and adapted for particular use cases but existing user workflows also need to adapt to new technology. %M 36927606 %R 10.2196/39777 %U https://formative.jmir.org/2023/1/e39777 %U https://doi.org/10.2196/39777 %U http://www.ncbi.nlm.nih.gov/pubmed/36927606 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 7 %N %P e39775 %T Integration of a Digital Health Intervention Into Immunization Clinic Workflows in Kenya: Qualitative, Realist Evaluation of Technology Usability %A Dolan,Samantha B %A Wittenauer,Rachel  %A Shearer,Jessica C %A Njoroge,Anne %A Onyango,Penina %A Owiso,George %A Lober,William B %A Liu,Shan %A Puttkammer,Nancy %A Rabinowitz,Peter %+ Bill and Melinda Gates Foundation, 500 5th Ave N, Seattle, WA, 98109, United States, 1 206 709 3100, sdolan11@gmail.com %K immunizations %K electronic immunization registry %K workflow %K usability %K realist research %D 2023 %7 14.3.2023 %9 Original Paper %J JMIR Form Res %G English %X Background: In an effort to increase vaccination coverage in low-resource settings, digital tools have been introduced to better track immunization records, improve data management practices, and provide improved access to vaccination coverage data for decision-making. Despite the potential of these electronic systems to improve the provision of health services, few digital health interventions have been institutionalized at scale in low- and middle-income countries. Objective: In this paper, we aimed to describe how health care workers in Kenya had integrated an electronic immunization registry into their immunization clinic workflows and to use these findings to inform the development of a refined program theory on the registry’s usability. Methods: Informed by realist methodology, we developed a program theory to explain usability of the electronic immunization registry. We designed a qualitative study based on our theory to describe the barriers and facilitators influencing data entry and use. Qualitative data were collected through semistructured interviews with users and workflow observations of immunization clinic sessions. Our findings were summarized by context-mechanism-outcome relationships formed after analyzing our key themes across interviews and workflow observations. Using these relationships, we were able to identify common rules for future implementers. Results: Across the 12 facilities included in our study, 19 health care workers were interviewed, and 58 workflow sessions were observed. The common rules developed from our qualitative findings are as follows: rule 1—ensure that the users complete training to build familiarity with the system, understand the value of the system and data, and know where to find support; rule 2—confirm that the system captures all data needed for users to provide routine health care services and is easy to navigate; rule 3—identify work-arounds for poor network, system performance, and too few staff or resources; and rule 4—make users aware of expected changes to their workflow, and how these changes might differ over time and by facility size or number of patients. Upon study completion, we revised the program theory to reflect the importance of the goals and workflows of electronic immunization registries aligning with reality. Conclusions: We created a deeper understanding of the underlying mechanisms for usability of the registry. We found that the electronic immunization registry had high acceptability among users; however, there were numerous barriers to using the system, even under ideal conditions, causing a misalignment between the system and the reality of the users’ workflows and their environment. Human-centered design and human-factors methods can assist during pilot stages to better align systems with users’ needs and again after scale-up to ensure that interventions are suitable for all user settings. %M 36917157 %R 10.2196/39775 %U https://formative.jmir.org/2023/1/e39775 %U https://doi.org/10.2196/39775 %U http://www.ncbi.nlm.nih.gov/pubmed/36917157 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 10 %N %P e43103 %T The Priorities of End Users of Emergency Department Electronic Health Records: Modified Delphi Study %A Yip,Matthew %A Ackery,Alun %A Jamieson,Trevor %A Mehta,Shaun %+ Temerty Faculty of Medicine, University of Toronto, 1 King's College Circle, Toronto, ON, M5S 1A8, Canada, 1 6479092917, matt.yip@mail.utoronto.ca %K Delphi %K EHR %K electronic health record %K emergency medicine %K emergency %K functionality %K health information exchange %K health system %K medical informatics %K patient-physician relationship %K usability %D 2023 %7 10.3.2023 %9 Original Paper %J JMIR Hum Factors %G English %X Background: The needs of the emergency department (ED) pose unique challenges to modern electronic health record (EHR) systems. A diverse case load of high-acuity, high-complexity presentations, and ambulatory patients, all requiring multiple transitions of care, creates a rich environment through which to critically examine EHRs. Objective: This investigation aims to capture and analyze the perspective of end users of EHR about the strengths, limitations, and future priorities for EHR in the setting of the ED. Methods: In the first phase of this investigation, a literature search was conducted to identify 5 key usage categories of ED EHRs. Using key usage categories in the first phase, a modified Delphi study was conducted with a group of 12 panelists with expertise in both emergency medicine and health informatics. Across 3 rounds of surveys, panelists generated and refined a list of strengths, limitations, and key priorities. Results: The findings from this investigation highlighted the preference of panelists for features maximizing functionality of basic clinical features relative to features of disruptive innovation. Conclusions: By capturing the perspectives of end users in the ED, this investigation highlights areas for the improvement or development of future EHRs in acute care settings. %M 36897633 %R 10.2196/43103 %U https://humanfactors.jmir.org/2023/1/e43103 %U https://doi.org/10.2196/43103 %U http://www.ncbi.nlm.nih.gov/pubmed/36897633 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 10 %N %P e39051 %T The Perceived Ease of Use and Perceived Usefulness of a Web-Based Interprofessional Communication and Collaboration Platform in the Hospital Setting: Interview Study With Health Care Providers %A Nie,Jason Xin %A Heidebrecht,Christine %A Zettler,Andrea %A Pearce,Jacklyn %A Cunha,Rafael %A Quan,Sherman %A Mansfield,Elizabeth %A Tang,Terence %+ Institute for Better Health, Trillium Health Partners, 2200 Eglinton Avenue West, Mississauga, ON, L5M 2N1, Canada, 1 9058132200, terence.tang@thp.ca %K health information technology %K communication and collaboration %K teamwork %K design %K technology acceptance model %K qualitative method %K communication %K collaboration %K hospital %K care %K team %K professional %K support %K health information %K technology %K clinician %K members %K complex %K lesson %K education %D 2023 %7 23.1.2023 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Hospitalized patients with complex care needs require an interprofessional team of health professionals working together to support their care in hospitals and during discharge planning. However, interprofessional communication and collaboration in inpatient settings are often fragmented and inefficient, leading to poor patient outcomes and provider frustration. Health information technology can potentially help improve team communication and collaboration; however, to date, evidence of its effectiveness is lacking. There are also concerns that current implementations might further fragment communication and increase the clinician burden without proven benefits. Objective: In this study, we aimed to generate transferrable lessons for future designers of health information technology tools that facilitate team communication and collaboration. Methods: A secondary analysis of the qualitative component of the mixed methods evaluation was performed. The electronic communication and collaboration platform was implemented in 2 general internal medicine wards in a large community teaching hospital in Mississauga, Ontario, Canada. Fifteen inpatient clinicians in those wards, including nurses, physicians, and allied health care providers, were recruited to participate in semistructured interviews about their experience with a co-designed electronic communication and collaboration tool. Data were analyzed using the Technology Acceptance Model, and themes related to the constructs of perceived ease of use (PEOU) and perceived usefulness (PU) were identified. Results: A secondary analysis guided by the Technology Acceptance Model highlighted important points. Intuitive design precluded training as a barrier to use, but lack of training may hinder participants’ PEOU if features designed for efficiency are not discovered by users. Organized information was found to be useful for creating a comprehensive clinical picture of each patient and facilitating improved handovers. However, information needs to be both comprehensive and succinct, and information overload may negatively impact PEOU. The mixed paper and electronic practice environment also negatively impacted PEOU owing to unavoidable double documentation and the need for printing. Participants perceived the tool to be useful as it improved efficiency in information retrieval and documentation, improved the handover process, afforded another mode of communication when face-to-face communication was impractical, and improved shared awareness. The PU of this tool depends on its optimal use by all team members. Conclusions: Electronic tools can support communication and collaboration among interprofessional teams caring for patients with complex needs. There are transferable lessons learned that can improve the PU and PEOU of future systems. %M 36689261 %R 10.2196/39051 %U https://humanfactors.jmir.org/2023/1/e39051 %U https://doi.org/10.2196/39051 %U http://www.ncbi.nlm.nih.gov/pubmed/36689261 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 12 %N %P e41533 %T Measurement of Vital Signs by Lifelight Software in Comparison to Standard of Care Multisite Development (VISION-MD): Protocol for an Observational Study %A Wiffen,Laura %A Brown,Thomas %A Brogaard Maczka,Annika %A Kapoor,Melissa %A Pearce,Laurence %A Chauhan,Milan %A Chauhan,Anoop J %A Saxena,Manish %A , %+ Mind Over Matter MedTech Ltd, Kemp House, 160 City Road, London, EC1V 2NX, United Kingdom, 44 7881 927063, melissa@mind-medtech.com %K general practice %K vital signs/methods %K vital signs/standards %K photoplethysmography %K remote photoplethysmography %K rPPG %K Lifelight %K contactless %K software %D 2023 %7 11.1.2023 %9 Protocol %J JMIR Res Protoc %G English %X Background: Measuring vital signs (VS) is an important aspect of clinical care but is time-consuming and requires multiple pieces of equipment and trained staff. Interest in the contactless measurement of VS has grown since the COVID-19 pandemic, including in nonclinical situations. Lifelight is an app being developed as a medical device for the contactless measurement of VS using remote photoplethysmography (rPPG) via the camera on smart devices. The VISION-D (Measurement of Vital Signs by Lifelight Software in Comparison to the Standard of Care—Development) and VISION-V (Validation) studies demonstrated the accuracy of Lifelight compared with standard-of-care measurement of blood pressure, pulse rate, and respiratory rate, supporting the certification of Lifelight as a class I Conformité Européenne (CE) medical device. Objective: To support further development of the Lifelight app, the observational VISION Multisite Development (VISION-MD) study is collecting high-quality data from a broad range of patients, including those with VS measurements outside the normal healthy range and patients who are critically ill. Methods: The study is recruiting adults (aged ≥16 years) who are inpatients (some critically ill), outpatients, and healthy volunteers, aiming to cover a broad range of normal and clinically relevant VS values; there are no exclusion criteria. High-resolution 60-second videos of the face are recorded by the Lifelight app while simultaneously measuring VS using standard-of-care methods (automated sphygmomanometer for blood pressure; finger clip sensor for pulse rate and oxygen saturation; manual counting of respiratory rate). Feedback from patients and nurses who use Lifelight is collected via a questionnaire. Data to estimate the cost-effectiveness of Lifelight compared with standard-of-care VS measurement are also being collected. A new method for rPPG signal processing is currently being developed, based on the identification of small areas of high-quality signals in each individual. Anticipated recruitment is 1950 participants, with the expectation that data from approximately 1700 will be used for software development. Data from 250 participants will be retained to test the performance of Lifelight against predefined performance targets. Results: Recruitment began in May 2021 but was hindered by the restrictions instigated during the COVID-19 pandemic. The development of data processing methodology is in progress. The data for analysis will become available from September 2022, and the algorithms will be refined continuously to improve clinical accuracy. The performance of Lifelight compared with that of the standard-of-care measurement of VS will then be tested. Recruitment will resume if further data are required. The analyses are expected to be completed in early 2023. Conclusions: This study will support the refinement of data collection and processing toward the development of a robust app that is suitable for routine clinical use. Trial Registration: ClinicalTrials.gov NCT04763746; https://clinicaltrials.gov/ct2/show/NCT04763746 International Registered Report Identifier (IRRID): DERR1-10.2196/41533 %M 36630158 %R 10.2196/41533 %U https://www.researchprotocols.org/2023/1/e41533 %U https://doi.org/10.2196/41533 %U http://www.ncbi.nlm.nih.gov/pubmed/36630158 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 10 %N %P e38736 %T Characterizing the Gaps Between Best-Practice Implementation Strategies and Real-world Implementation: Qualitative Study Among Family Physicians Who Engaged With Audit and Feedback Reports %A Rouleau,Geneviève %A Reis,Catherine %A Ivers,Noah %A Desveaux,Laura %+ Institute for Health System Solutions and Virtual Care, Women’s College Hospital, 76, Grenville Street, Toronto, ON, M5S 1B2, Canada, 1 4383921857, Genevieve.Rouleau@wchospital.ca %K audit and feedback %K family physicians %K primary care %K qualitative %D 2023 %7 6.1.2023 %9 Original Paper %J JMIR Hum Factors %G English %X Background: In Ontario, Canada, a government agency known as Ontario Health is responsible for making audit and feedback reports available to all family physicians to encourage ongoing quality improvement. The confidential report provides summary data on 3 key areas of practice: safe prescription, cancer screening, and diabetes management. Objective: This report was redesigned to improve its usability in line with evidence. The objective of this study was to explore how the redesign was perceived, with an emphasis on recipients’ understanding of the report and their engagement with it. Methods: We conducted qualitative semistructured interviews with family physicians who had experience with both versions of the report recruited through purposeful and snowball sampling. We analyzed the transcripts following an emergent and iterative approach. Results: Saturation was reached after 17 family physicians participated. In total, 2 key themes emerged as factors that affected the perceived usability of the report: alignment between the report and the recipients’ expectations and capacity to engage in quality improvement. Family physicians expected the report and its quality indicators to reflect best practices and to be valid and accurate. They also expected the report to offer feedback on the clinical activities they perceived to be within their control to change. Furthermore, family physicians expected the goal of the report to be aligned with their perspective on feasible quality improvement activities. Most of these expectations were not met, limiting the perceived usability of the report. The capacity to engage with audit and feedback was hindered by several organizational and physician-level barriers, including the lack of fit with the existing workflow, competing priorities, time constraints, and insufficient skills for bridging the gaps between their data and the corresponding desired actions. Conclusions: Despite recognized improvements in the design of the report to better align with best practices, it was not perceived as highly usable. Improvements in the presentation of the data could not overcome misalignment with family physicians’ expectations or the limited capacity to engage with the report. Integrating iterative evaluations informed by user-centered design can complement evidence-based guidance for implementation strategies. Creating a space for bringing together audit and feedback designers and recipients may help improve usability and effectiveness. %M 36607715 %R 10.2196/38736 %U https://humanfactors.jmir.org/2023/1/e38736 %U https://doi.org/10.2196/38736 %U http://www.ncbi.nlm.nih.gov/pubmed/36607715 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 10 %N %P e39114 %T Intensive Care Unit Physicians’ Perspectives on Artificial Intelligence–Based Clinical Decision Support Tools: Preimplementation Survey Study %A van der Meijden,Siri L %A de Hond,Anne A H %A Thoral,Patrick J %A Steyerberg,Ewout W %A Kant,Ilse M J %A Cinà,Giovanni %A Arbous,M Sesmu %+ Department of Intensive Care Medicine, Leiden University Medical Center, Albinusdreef 2, Leiden, 2333 ZA, Netherlands, 31 71 526 9111, S.L.van_der_meijden@lumc.nl %K intensive care unit %K hospital %K discharge %K artificial intelligence %K AI %K clinical decision support %K clinical support %K acceptance %K decision support %K decision-making %K digital health %K eHealth %K survey %K perspective %K attitude %K opinion %K adoption %K prediction %K risk %D 2023 %7 5.1.2023 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Artificial intelligence–based clinical decision support (AI-CDS) tools have great potential to benefit intensive care unit (ICU) patients and physicians. There is a gap between the development and implementation of these tools. Objective: We aimed to investigate physicians’ perspectives and their current decision-making behavior before implementing a discharge AI-CDS tool for predicting readmission and mortality risk after ICU discharge. Methods: We conducted a survey of physicians involved in decision-making on discharge of patients at two Dutch academic ICUs between July and November 2021. Questions were divided into four domains: (1) physicians’ current decision-making behavior with respect to discharging ICU patients, (2) perspectives on the use of AI-CDS tools in general, (3) willingness to incorporate a discharge AI-CDS tool into daily clinical practice, and (4) preferences for using a discharge AI-CDS tool in daily workflows. Results: Most of the 64 respondents (of 93 contacted, 69%) were familiar with AI (62/64, 97%) and had positive expectations of AI, with 55 of 64 (86%) believing that AI could support them in their work as a physician. The respondents disagreed on whether the decision to discharge a patient was complex (23/64, 36% agreed and 22/64, 34% disagreed); nonetheless, most (59/64, 92%) agreed that a discharge AI-CDS tool could be of value. Significant differences were observed between physicians from the 2 academic sites, which may be related to different levels of involvement in the development of the discharge AI-CDS tool. Conclusions: ICU physicians showed a favorable attitude toward the integration of AI-CDS tools into the ICU setting in general, and in particular toward a tool to predict a patient’s risk of readmission and mortality within 7 days after discharge. The findings of this questionnaire will be used to improve the implementation process and training of end users. %M 36602843 %R 10.2196/39114 %U https://humanfactors.jmir.org/2023/1/e39114 %U https://doi.org/10.2196/39114 %U http://www.ncbi.nlm.nih.gov/pubmed/36602843 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 4 %P e40400 %T Usability Evaluation of the Preoperative ISBAR (Identification, Situation, Background, Assessment, and Recommendation) Desktop Virtual Reality Application: Qualitative Observational Study %A Andreasen,Eva Mari %A Høigaard,Rune %A Berg,Helen %A Steinsbekk,Aslak %A Haraldstad,Kristin %+ Department of Health and Nursing Sciences, University of Agder, P.O. Box 422, Kristiansand, 4604, Norway, 47 90642121, eva.mari.andreasen@uia.no %K desktop virtual reality %K handover %K ISBAR %K preoperative %K undergraduate nursing students %K usability evaluation %K usability %K nursing %K health care education %K student %K medical education %K medical training %K VR %K virtual reality %K surgery %K surgical %K System Usability Scale %K communication %K self-instruction %K self-guided %K nurse %K training %K undergraduate %K health care professional %K health care provider %D 2022 %7 29.12.2022 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Systematic communication, such as the ISBAR (identification, situation, background, assessment, recommendation) approach, comprises a generic, transferable nontechnical skill. It can be used during the handover of patients set to undergo surgery and can be practiced in various ways, including virtual reality (VR). VR increasingly has been implemented and valued in nursing education as a positive contribution to teach students about pre- and postoperative nursing. A new nonimmersive 3D learning activity called the Preoperative ISBAR Desktop VR Application has been developed for undergraduate nursing students to learn preoperative handover using the ISBAR approach. However, the usability of this learning activity has not been studied. Objective: This study aimed to investigate how second-year undergraduate nursing students evaluated the usability of the Preoperative ISBAR Desktop VR Application. Methods: This was a qualitative study with observation and interviews. The inclusion criteria were undergraduate second-year nursing students of varying ages, gender, and anticipated technological competence. The System Usability Scale (SUS) questionnaire was used to get a score on overall usability. Results: A total of 9 second-year nursing students aged 22-29 years participated in the study. The average score on the SUS was 83 (range 0-100), which equals a “B” on the graded scale and is excellent for an adjective-grade rating. The students expressed increased motivation to learn while working in self-instructed desktop VR. Still, a few technical difficulties occurred, and some students reported that they experienced some problems comprehending the instructions provided in the application. Long written instructions and a lack of self-pacing built into the application were considered limitations. Conclusions: The nursing students found the application to be usable overall, giving it an excellent usability score and noting that the application provided opportunities for active participation, which was motivational and facilitated their perceived learning outcomes. The next version of the application, to be used in a randomized controlled trial, will be upgraded to address technological and comprehension issues. %M 36580357 %R 10.2196/40400 %U https://humanfactors.jmir.org/2022/4/e40400 %U https://doi.org/10.2196/40400 %U http://www.ncbi.nlm.nih.gov/pubmed/36580357 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 4 %P e38701 %T A Mobile App Leveraging Citizenship Engagement to Perform Anonymized Longitudinal Studies in the Context of COVID-19 Adverse Drug Reaction Monitoring: Development and Usability Study %A Di Filippo,Marzia %A Avellone,Alessandro %A Belingheri,Michael %A Paladino,Maria Emilia %A Riva,Michele Augusto %A Zambon,Antonella %A Pescini,Dario %+ Department of Statistics and Quantitative Methods, University of Milano-Bicocca, Via Bicocca degli Arcimboldi 8, Milan, 20126, Italy, 39 0264485835, dario.pescini@unimib.it %K ADR reporting %K adverse drug reaction–reporting systems %K mobile apps %K longitudinal studies %K COVID-19 vaccination campaign %K COVID-19 %K vaccine %K apps %K adverse drug reaction %K pharmacovigilance %K anonymity %D 2022 %7 4.11.2022 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Over the past few years, studies have increasingly focused on the development of mobile apps as complementary tools to existing traditional pharmacovigilance surveillance systems for improving and facilitating adverse drug reaction (ADR) reporting. Objective: In this research, we evaluated the potentiality of a new mobile app (vaxEffect@UniMiB) to perform longitudinal studies, while preserving the anonymity of the respondents. We applied the app to monitor the ADRs during the COVID-19 vaccination campaign in a sample of the Italian population. Methods: We administered vaxEffect@UniMiB to a convenience sample of academic subjects vaccinated at the Milano-Bicocca University hub for COVID-19 during the Italian national vaccination campaign. vaxEffect@UniMiB was developed for both Android and iOS devices. The mobile app asks users to send their medical history and, upon every vaccine administration, their vaccination data and the ADRs that occurred within 7 days postvaccination, making it possible to follow the ADR dynamics for each respondent. The app sends data over the web to an application server. The server, along with receiving all user data, saves the data in a SQL database server and reminds patients to submit vaccine and ADR data by push notifications sent to the mobile app through Firebase Cloud Messaging (FCM). On initial startup of the app, a unique user identifier (UUID) was generated for each respondent, so its anonymity was completely ensured, while enabling longitudinal studies. Results: A total of 3712 people were vaccinated during the first vaccination wave. A total of 2733 (73.6%) respondents between the ages of 19 and 80 years, coming from the University of Milano-Bicocca (UniMiB) and the Politecnico of Milan (PoliMi), participated in the survey. Overall, we collected information about vaccination and ADRs to the first vaccine dose for 2226 subjects (60.0% of the first dose vaccinated), to the second dose for 1610 subjects (43.4% of the second dose vaccinated), and, in a nonsponsored fashion, to the third dose for 169 individuals (4.6%). Conclusions: vaxEffect@UniMiB was revealed to be the first attempt in performing longitudinal studies to monitor the same subject over time in terms of the reported ADRs after each vaccine administration, while guaranteeing complete anonymity of the subject. A series of aspects contributed to the positive involvement from people in using this app to report their ADRs to vaccination: ease of use, availability from multiple platforms, anonymity of all survey participants and protection of the submitted data, and the health care workers’ support. %M 35930561 %R 10.2196/38701 %U https://humanfactors.jmir.org/2022/4/e38701 %U https://doi.org/10.2196/38701 %U http://www.ncbi.nlm.nih.gov/pubmed/35930561 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 4 %P e36976 %T Desired Characteristics of a Clinical Decision Support System for Early Sepsis Recognition: Interview Study Among Hospital-Based Clinicians %A Silvestri,Jasmine A %A Kmiec,Tyler E %A Bishop,Nicholas S %A Regli,Susan H %A Weissman,Gary E %+ Palliative and Advanced Illness Research Center, University of Pennsylvania Perelman School of Medicine, 300 Blockley Hall, 423 Guardian Drive, Philadelphia, PA, 19104, United States, 1 215 746 2887, gary.weissman@pennmedicine.upenn.edu %K sepsis %K predictive information %K clinical decision support %K human factors %K sepsis onset %D 2022 %7 21.10.2022 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Sepsis is a major burden for health care systems in the United States, with over 750,000 cases annually and a total cost of approximately US $20 billion. The hallmark of sepsis treatment is early and appropriate initiation of antibiotic therapy. Although sepsis clinical decision support (CDS) systems can provide clinicians with early predictions of suspected sepsis or imminent clinical decline, such systems have not reliably demonstrated improvements in clinical outcomes or care processes. Growing evidence suggests that the challenges of integrating sepsis CDS systems into clinical workflows, gaining the trust of clinicians, and making sepsis CDS systems clinically relevant at the bedside are all obstacles to successful deployment. However, there are significant knowledge gaps regarding the achievement of these implementation and deployment goals. Objective: We aimed to identify perceptions of predictive information in sepsis CDS systems based on clinicians’ past experiences, explore clinicians’ perceptions of a hypothetical sepsis CDS system, and identify the characteristics of a CDS system that would be helpful in promoting timely recognition and management of suspected sepsis in a multidisciplinary, team-based clinical setting. Methods: We conducted semistructured interviews with practicing bedside nurses, advanced practice providers, and physicians at a large academic medical center between September 2020 and March 2021. We used modified human factor methods (contextual interview and cognitive walkthrough performed over video calls because of the COVID-19 pandemic) and conducted a thematic analysis using an abductive approach for coding to identify important patterns and concepts in the interview transcripts. Results: We interviewed 6 bedside nurses and 9 clinicians responsible for ordering antibiotics (advanced practice providers or physicians) who had a median of 4 (IQR 4-6.5) years of experience working in an inpatient setting. We then synthesized critical content from the thematic analysis of the data into four domains: clinician perceptions of prediction models and alerts; previous experiences of clinician encounters with predictive information and risk scores; desired characteristics of a CDS system build, including predictions, supporting information, and delivery methods for a potential alert; and the clinical relevance and potential utility of a CDS system. These 4 domains were strongly linked to clinicians’ perceptions of the likelihood of adoption and the impact on clinical workflows when diagnosing and managing patients with suspected sepsis. Ultimately, clinicians desired a trusted and actionable CDS system to improve sepsis care. Conclusions: Building a trusted and actionable sepsis CDS alert is paramount to achieving acceptability and use among clinicians. These findings can inform the development, implementation, and deployment strategies for CDS systems that support the early detection and treatment of sepsis. This study also highlights several key opportunities when eliciting clinician input before the development and deployment of prediction models. %M 36269653 %R 10.2196/36976 %U https://humanfactors.jmir.org/2022/4/e36976 %U https://doi.org/10.2196/36976 %U http://www.ncbi.nlm.nih.gov/pubmed/36269653 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 3 %P e39234 %T Electronic Diagnostic Support in Emergency Physician Triage: Qualitative Study With Thematic Analysis of Interviews %A Sibbald,Matthew %A Abdulla,Bashayer %A Keuhl,Amy %A Norman,Geoffrey %A Monteiro,Sandra %A Sherbino,Jonathan %+ McMaster Education Research, Innovation & Theory (MERIT) Program, Department of Medicine, McMaster University, 100 Main Street West, Hamilton, ON, L8P 1H6, Canada, 1 905 921 2101 ext 44477, matthew.sibbald@medportal.ca %K electronic differential diagnostic support %K clinical reasoning %K natural language processing %K triage %K diagnostic error %K human factors %K diagnosis %K diagnostic %K emergency %K artificial intelligence %K adoption %K attitude %K support system %K automation %D 2022 %7 30.9.2022 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Not thinking of a diagnosis is a leading cause of diagnostic error in the emergency department, resulting in delayed treatment, morbidity, and excess mortality. Electronic differential diagnostic support (EDS) results in small but significant reductions in diagnostic error. However, the uptake of EDS by clinicians is limited. Objective: We sought to understand physician perceptions and barriers to the uptake of EDS within the emergency department triage process. Methods: We conducted a qualitative study using a research associate to rapidly prototype an embedded EDS into the emergency department triage process. Physicians involved in the triage assessment of a busy emergency department were provided the output of an EDS based on the triage complaint by an embedded researcher to simulate an automated system that would draw from the electronic medical record. Physicians were interviewed immediately after their experience. Verbatim transcripts were analyzed by a team using open and axial coding, informed by direct content analysis. Results: In all, 4 themes emerged from 14 interviews: (1) the quality of the EDS was inferred from the scope and prioritization of the diagnoses present in the EDS differential; (2) the trust of the EDS was linked to varied beliefs around the diagnostic process and potential for bias; (3) clinicians foresaw more benefit to EDS use for colleagues and trainees rather than themselves; and (4) clinicians felt strongly that EDS output should not be included in the patient record. Conclusions: The adoption of an EDS into an emergency department triage process will require a system that provides diagnostic suggestions appropriate for the scope and context of the emergency department triage process, transparency of system design, and affordances for clinician beliefs about the diagnostic process and addresses clinician concern around including EDS output in the patient record. %M 36178728 %R 10.2196/39234 %U https://humanfactors.jmir.org/2022/3/e39234 %U https://doi.org/10.2196/39234 %U http://www.ncbi.nlm.nih.gov/pubmed/36178728 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 3 %P e37894 %T Defining Recommendations to Guide User Interface Design: Multimethod Approach %A Diehl,Ceci %A Martins,Ana %A Almeida,Ana %A Silva,Telmo %A Ribeiro,Óscar %A Santinha,Gonçalo %A Rocha,Nelson %A Silva,Anabela G %+ Center for Health Technology and Services Research, School of Health Sciences, University of Aveiro, Campus Universitário Santiago, Aveiro, 3810-193, Portugal, 351 234 372 444, asilva@ua.pt %K user interface design %K usability principles %K interaction paradigm %K generic recommendations %K specific recommendations %D 2022 %7 30.9.2022 %9 Original Paper %J JMIR Hum Factors %G English %X Background: For the development of digital solutions, different aspects of user interface design must be taken into consideration. Different technologies, interaction paradigms, user characteristics and needs, and interface design components are some of the aspects that designers and developers should pay attention to when designing a solution. Many user interface design recommendations for different digital solutions and user profiles are found in the literature, but these recommendations have numerous similarities, contradictions, and different levels of detail. A detailed critical analysis is needed that compares, evaluates, and validates existing recommendations and allows the definition of a practical set of recommendations. Objective: This study aimed to analyze and synthesize existing user interface design recommendations and propose a practical set of recommendations that guide the development of different technologies. Methods: Based on previous studies, a set of recommendations on user interface design was generated following 4 steps: (1) interview with user interface design experts; (2) analysis of the experts’ feedback and drafting of a set of recommendations; (3) reanalysis of the shorter list of recommendations by a group of experts; and (4) refining and finalizing the list. Results: The findings allowed us to define a set of 174 recommendations divided into 12 categories, according to usability principles, and organized into 2 levels of hierarchy: generic (69 recommendations) and specific (105 recommendations). Conclusions: This study shows that user interface design recommendations can be divided according to usability principles and organized into levels of detail. Moreover, this study reveals that some recommendations, as they address different technologies and interaction paradigms, need further work. %M 36178714 %R 10.2196/37894 %U https://humanfactors.jmir.org/2022/3/e37894 %U https://doi.org/10.2196/37894 %U http://www.ncbi.nlm.nih.gov/pubmed/36178714 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 3 %P e34568 %T Evaluation of a Health Information Exchange System for Geriatric Health Care in Rural Areas: Development and Technical Acceptance Study %A Pfeuffer,Nils %A Beyer,Angelika %A Penndorf,Peter %A Leiz,Maren %A Radicke,Franziska %A Hoffmann,Wolfgang %A van den Berg,Neeltje %+ Section Epidemiology of Health Care and Community Health, Institute for Community Medicine, University Medicine Greifswald, Ellernholzstr. 1-2, Greifswald, 17489, Germany, 49 3834867618, nils.pfeuffer@med.uni-greifswald.de %K electronic health records %K health information exchange %K geriatrics %K community-based participatory research %K technical acceptance %K usability %K health information network %K postacute care %K patient-centered care %D 2022 %7 15.9.2022 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Patients of geriatrics are often treated by several health care providers at the same time. The spatial, informational, and organizational separation of these health care providers can hinder the effective treatment of these patients. Objective: This study aimed to develop a regional health information exchange (HIE) system to improve HIE in geriatric treatment. This study also evaluated the usability of the regional HIE system and sought to identify barriers to and facilitators of its implementation. Methods: The development of the regional HIE system followed the community-based participatory research approach. The primary outcomes were the usability of the regional HIE system, expected implementation barriers and facilitators, and the quality of the developmental process. Data were collected and analyzed using a mixed methods approach. Results: A total of 3 focus regions were identified, 22 geriatric health care providers participated in the development of the regional HIE system, and 11 workshops were conducted between October 2019 and September 2020. In total, 12 participants responded to a questionnaire. The main results were that the regional HIE system should support the exchange of assessments, diagnoses, medication, assistive device supply, and social information. The regional HIE system was expected to be able to improve the quality and continuity of care. In total, 5 adoption facilitators were identified. The main points were adaptability of the regional HIE system to local needs, availability to different patient groups and treatment documents, web-based design, trust among the users, and computer literacy. A total of 13 barriers to adoption were identified. The main expected barriers to implementation were lack of resources, interoperability issues, computer illiteracy, lack of trust, privacy concerns, and ease-of-use issues. Conclusions: Participating health care professionals shared similar motivations for developing the regional HIE system, including improved quality of care, reduction of unnecessary examinations, and more effective health care provision. An overly complicated registration process for health care professionals and the patients’ free choice of their health care providers hinder the effectiveness of the regional HIE system, resulting in incomplete patient health information. However, the web-based design of the system bridges interoperability problems that exist owing to the different technical and organizational structures of the health care facilities involved. The regional HIE system is better accepted by health care professionals who are already engaged in an interdisciplinary, geriatric-focused network. This might indicate that pre-existing cross-organizational structures and processes are prerequisites for using HIE systems. The participatory design supports the development of technologies that are adaptable to regional needs. Health care providers are interested in participating in the development of an HIE system, but they often lack the required time, knowledge, and resources. %M 36107474 %R 10.2196/34568 %U https://humanfactors.jmir.org/2022/3/e34568 %U https://doi.org/10.2196/34568 %U http://www.ncbi.nlm.nih.gov/pubmed/36107474 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 8 %P e34858 %T Physicians’ Perceptions as Predictors of the Future Use of the National Death Information System in Peru: Cross-sectional Study %A Vargas-Herrera,Javier %A Meneses,Giovanni %A Cortez-Escalante,Juan %+ Department of Preventive Medicine and Public Health, National University of San Marcos, Av German Amezaga 375, Lima, 15081, Peru, 51 945029342, jvargash@unmsm.edu.pe %K death certificates %K health information system %K mortality %K vital statistics %K Technology Acceptance Model %K model %K acceptance model %K certificates %K information system %K physicians %K predictors %K cross-sectional study %K analysis %K death %D 2022 %7 15.8.2022 %9 Original Paper %J J Med Internet Res %G English %X Background: A computer application called the National Death Information System (SINADEF) was implemented in Peru so that physicians can prepare death certificates in electronic format and the information is available online. In 2018, only half of the estimated deaths in Peru were certified using SINADEF. When a death is certified in paper format, the probability being entered in the mortality database decreases. It is important to know, from the user’s perspective, the factors that can influence the successful implementation of SINADEF. SINADEF can only be successfully implemented if it is known whether physicians believe that it is useful and easy to operate. Objective: The aim of this study was to identify the perceptions of physicians and other factors as predictors of their behavioral intention to use SINADEF to certify a death. Methods: This study had an observational, cross-sectional design. A survey was provided to physicians working in Peru, who used SINADEF to certify a death for a period of 12 months, starting in November 2019. A questionnaire was adapted based on the Technology Acceptance Model. The questions measured the dimensions of subjective norm, image, job relevance, output quality, demonstrability of results, perceived usefulness, perceived ease of use, and behavioral intention to use. Chi-square and logistic regression tests were used in the analysis, and a confidence level of 95% was chosen to support a significant association. Results: In this study, 272 physicians responded to the survey; 184 (67.6%) were men and the average age was 45.3 (SD 10.1) years. The age range was 24 to 73 years. In the bivariate analysis, the intention to use SINADEF was found to be associated with (1) perceived usefulness, expressed as “using SINADEF avoids falsifying a death certificate” (P<.001), “using SINADEF reduces the risk of errors” (P<.001), and “using SINADEF allows for filling out a certificate in less time” (P<.001); and (2) perceived ease of use, expressed as “I think SINADEF is easy to use” (P<.001). In the logistic regression, perceived usefulness (odds ratio [OR] 8.5, 95% CI 2.2-32.3; P=.002), perceived ease of use (OR 10.1, 95% CI 2.4-41.8; P=.001), and training in filling out death certificates (OR 8.3, 95% CI 1.6-42.8; P=.01) were found to be predictors of the behavioral intention to use SINADEF. Conclusions: The behavioral intention to use SINADEF was related to the perception that it is an easy-to-use system, the belief that it improves the performance of physicians in carrying out the task at hand, and with training in filling out death certificates. %M 35969435 %R 10.2196/34858 %U https://www.jmir.org/2022/8/e34858 %U https://doi.org/10.2196/34858 %U http://www.ncbi.nlm.nih.gov/pubmed/35969435 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 6 %N 8 %P e34572 %T A Novel Hospital-to-Home System for Children With Medical Complexities: Usability Testing Study %A Bird,Marissa %A Carter,Nancy %A Lim,Audrey %A Kazmie,Nadia %A Fajardo,Cindy %A Reaume,Shannon %A McGillion,Michael H %+ School of Nursing, McMaster University, 1280 Main Street West, Hamilton, ON, L8S1K4, Canada, 1 905 525 9140, birdm3@mcmaster.ca %K usability testing %K digital health %K children with medical complexities %K children %K chronic disease %K pediatrics %K health care %K parenting %K virtual health %K care provider %K youth %K family needs %K home care %K usability %D 2022 %7 12.8.2022 %9 Original Paper %J JMIR Form Res %G English %X Background: Children with medical complexity (CMC) are a group of young people who have severe complex chronic conditions, substantial family-identified service needs, functional limitations, and high health care resource use. Technology-enabled hospital-to-home interventions designed to deliver comprehensive care in the home setting are needed to ease CMC family stress, provide proactive and comprehensive care to this fragile population, and avoid hospital admissions, where possible. Objective: In this usability testing study, we aimed to assess areas of strength and opportunity within the DigiComp Kids system, a hospital-to-home intervention for CMC and their families and care providers. Methods: Hospital-based clinicians, family members of medically complex children, and home-based clinicians participated in DigiComp Kids usability testing. Participants were recorded and tasked to think aloud while completing usability testing tasks. Participants were scored on the metrics of effectiveness, efficiency, and satisfaction, and the total usability score was calculated using the Single Usability Metric. Participants also provided insights into user experiences during the postusability testing interviews. Results: A total of 15 participants (5 hospital-based clinicians, 6 family members, and 4 home-based clinicians) participated in DigiComp Kids usability testing. The participants were able to complete all assigned tasks independently. Error-free rates for tasks ranged from 58% to 100%; the average satisfaction rating across groups was ≥80%, as measured by the Single Ease Question. Task times of participants were variable compared with the task times of an expert DigiComp Kids user. Single Usability Metric scores ranged from 80.5% to 89.5%. In qualitative interviews, participants stressed the need to find the right fit between user needs and the effort required to use the system. Interviews also revealed that the value of the DigiComp Kids system was in its ability to create a digital bridge between hospital and home, enabling participants to foster and maintain connections across boundaries. Conclusions: Usability testing revealed strong scores across the groups. Insights gained include the importance of tailoring the implementation of the system to match individual user needs, streamlining key system features, and consideration of the meaning attached to system use by participants to allow for insight into system adoption and sustainment. %M 35969456 %R 10.2196/34572 %U https://formative.jmir.org/2022/8/e34572 %U https://doi.org/10.2196/34572 %U http://www.ncbi.nlm.nih.gov/pubmed/35969456 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 3 %P e36069 %T Telehealth Perceptions Among US Immigrant Patients at an Academic Internal Medicine Practice: Cross-sectional Study %A Levine,Susan %A Gupta,Richa %A Alkwatli,Kenda %A Almoushref,Allaa %A Cherian,Saira %A Jimenez,Dominique Feterman %A Cordero Baez,Greishka Nicole %A Hart,Angela %A Weinstock,Clara %+ UConn Health, University of Connecticut, 263 Farmington Ave, MC 8021, Farmington, CT, 06030, United States, 1 860 679 1781, slevine@uchc.edu %K telemedicine %K immigrant patient %K immigrant %K ethnic minority %K cultural minority %K patient satisfaction %K telehealth %K satisfaction %K health care %K health equity %K digital health %K patient experience %D 2022 %7 10.8.2022 %9 Original Paper %J JMIR Hum Factors %G English %X Background: The use of telemedicine has increased dramatically through the COVID-19 pandemic. Although data are available about patient satisfaction with telemedicine, little is known about immigrant patients’ experience. Objective: We sought to investigate patients’ experiences with telehealth compared to in- person visits between immigrants and nonimmigrants. We wanted to identify and describe next visit preferences within the Farmington University of Connecticut Internal Medicine practice to ultimately guide suggestions for more equitable use and accessibility of visit options. Methods: A total of 270 patients including 122 immigrants and 148 nonimmigrants were seen by 4 Internal Medicine providers in an in-person (n=132) or telemedicine (n=138) university practice setting. Patients were queried between February and April 2021, using an adaptation of a previously validated patient satisfaction survey that contained standard questions developed by the Consumer Assessment of Healthcare Providers and Systems Program. Patients seen via in-person visits completed a paper copy of the survey. The same survey was administered by a follow-up phone call for telemedicine visits. Patients surveyed spoke English, Spanish, or Arabic and were surveyed in their preferred language. For televisits, the same survey was read to the patient by a certified translator. The survey consisted of 10 questions on a Likert scale of 1-5. Of them, 9 questions assessed patient satisfaction under the categories of access to care, interpersonal interaction, and quality of care. An additional question asked patients to describe and explain the reasons behind next visit preferences. Survey question responses were compared by paired t tests. Results: Across both immigrant and nonimmigrant patient populations, satisfaction with perceived quality of care was high, regardless of visit type (P=.80, P=.60 for televisits and P=.76, P=.37 for in-person visits). During televisits, immigrants were more likely to feel providers spent sufficient time with them (P<.001). Different perceptions were noted among nonimmigrant patients. Nonimmigrants tended to perceive more provider time during in-person visits (P=.006). When asked to comment on reasons behind next televisit preference, nonimmigrant patients prioritized convenience, whereas immigrants noted not having to navigate office logistics. For those who chose in-person visits, both groups prioritized the need for a physical exam. Conclusions: Although satisfaction was high for both telemedicine and in-person visits across immigrant and nonimmigrant populations, significant differences in patient priorities were identified. Immigrants found televisits desirable because they felt they spent more time with providers and were able to avoid additional office logistics that are often challenging barriers for non-English speakers. This suggests opportunities to use information technology to provide cultural and language-appropriate information throughout immigrants’ in-person and telemedicine visit experience. A focus on diminishing these barriers will help reduce health care inequities among immigrant patients. %M 35947438 %R 10.2196/36069 %U https://humanfactors.jmir.org/2022/3/e36069 %U https://doi.org/10.2196/36069 %U http://www.ncbi.nlm.nih.gov/pubmed/35947438 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 7 %P e37928 %T Development of an Interoperable and Easily Transferable Clinical Decision Support System Deployment Platform: System Design and Development Study %A Yoo,Junsang %A Lee,Jeonghoon %A Min,Ji Young %A Choi,Sae Won %A Kwon,Joon-myoung %A Cho,Insook %A Lim,Chiyeon %A Choi,Mi Young %A Cha,Won Chul %+ Department of Emergency Medicine, Samsung Medical Center, Sungkyunkwan University School of Medicine, 81 Irwon-ro, Gangnam-gu, Seoul, 06351, Republic of Korea, 82 3410 2053, wc.cha@samsung.com %K clinical decision support system %K decision making %K decision aid %K decision support %K common data model %K model %K development %K electronic health record %K medical record %K EHR %K EMR %K Fast Healthcare Interoperability Resource %K interoperability %K machine learning %K clinical decision %K health technology %K algorithm %K intelligent algorithm network %K modeling %D 2022 %7 27.7.2022 %9 Original Paper %J J Med Internet Res %G English %X Background: A clinical decision support system (CDSS) is recognized as a technology that enhances clinical efficacy and safety. However, its full potential has not been realized, mainly due to clinical data standards and noninteroperable platforms. Objective: In this paper, we introduce the common data model–based intelligent algorithm network environment (CANE) platform that supports the implementation and deployment of a CDSS. Methods: CDSS reasoning engines, usually represented as R or Python objects, are deployed into the CANE platform and converted into C# objects. When a clinician requests CANE-based decision support in the electronic health record (EHR) system, patients’ information is transformed into Health Level 7 Fast Healthcare Interoperability Resources (FHIR) format and transmitted to the CANE server inside the hospital firewall. Upon receiving the necessary data, the CANE system’s modules perform the following tasks: (1) the preprocessing module converts the FHIRs into the input data required by the specific reasoning engine, (2) the reasoning engine module operates the target algorithms, (3) the integration module communicates with the other institutions’ CANE systems to request and transmit a summary report to aid in decision support, and (4) creates a user interface by integrating the summary report and the results calculated by the reasoning engine. Results: We developed a CANE system such that any algorithm implemented in the system can be directly called through the RESTful application programming interface when it is integrated with an EHR system. Eight algorithms were developed and deployed in the CANE system. Using a knowledge-based algorithm, physicians can screen patients who are prone to sepsis and obtain treatment guides for patients with sepsis with the CANE system. Further, using a nonknowledge-based algorithm, the CANE system supports emergency physicians’ clinical decisions about optimum resource allocation by predicting a patient’s acuity and prognosis during triage. Conclusions: We successfully developed a common data model–based platform that adheres to medical informatics standards and could aid artificial intelligence model deployment using R or Python. %M 35896020 %R 10.2196/37928 %U https://www.jmir.org/2022/7/e37928 %U https://doi.org/10.2196/37928 %U http://www.ncbi.nlm.nih.gov/pubmed/35896020 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 3 %P e35403 %T Automated Decision Support For Community Mental Health Services Using National Electronic Health Records: Qualitative Implementation Case Study %A van Kasteren,Yasmin %A Strobel,Jörg %A Bastiampillai,Tarun %A Linedale,Ecushla %A Bidargaddi,Niranjan %+ Flinders Health and Medical Research Institute, College of Medicine and Public Health, Flinders University, PO Box 2100, Adelaide, 5001, Australia, 61 8 82018840, niranjan.bidargaddi@flinders.edu.au %K implementation %K computerised clinical decision system %K decision system %K decision support %K participatory action framework %K psychotropic medication %K psychotropic %K nonadherence %K monitoring %K medication adherence %K algorithms %K algorithm %K electronic health records %K EHR %K health record %K normalization process theory %K automated alerts %K automated alert %K mental health %K mental illness %K adherence %K medication %K eHealth %K web-based %D 2022 %7 5.7.2022 %9 Original Paper %J JMIR Hum Factors %G English %X Background: A high proportion of patients with severe mental illness relapse due to nonadherence to psychotropic medication. In this paper, we use the normalization process theory (NPT) to describe the implementation of a web-based clinical decision support system (CDSS) for Community Mental Health Services (CMHS) called Actionable Intime Insights or AI2. AI2 has two distinct functions: (1) it provides an overview of medication and treatment history to assist in reviewing patient adherence and (2) gives alerts indicating nonadherence to support early intervention. Objective: Our objective is to evaluate the pilot implementation of the AI2 application to better understand the challenges of implementing a web-based CDSS to support medication adherence and early intervention in CMHS. Methods: The NPT and participatory action framework were used to both explore and support implementation. Qualitative data were collected over the course of the 14-month implementation, in which researchers were active participants. Data were analyzed and coded using the NPT framework. Qualitative data included discussions, meetings, and work products, including emails and documents. Results: This study explores the barriers and enablers of implementing a CDSS to support early intervention within CMHS using Medicare data from Australia’s national electronic record system, My Health Record (MyHR). The implementation was a series of ongoing negotiations, which resulted in a staged implementation with compromises on both sides. Clinicians were initially hesitant about using a CDSS based on MyHR data and expressed concerns about the changes to their work practice required to support early intervention. Substantial workarounds were required to move the implementation forward. This pilot implementation allowed us to better understand the challenges of implementation and the resources and support required to implement and sustain a model of care based on automated alerts to support early intervention. Conclusions: The use of decision support based on electronic health records is growing, and while implementation is challenging, the potential benefits of early intervention to prevent relapse and hospitalization and ensure increased efficiency of the health care system are worth pursuing. %M 35788103 %R 10.2196/35403 %U https://humanfactors.jmir.org/2022/3/e35403 %U https://doi.org/10.2196/35403 %U http://www.ncbi.nlm.nih.gov/pubmed/35788103 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 2 %P e37192 %T Usability of the Swedish Accessible Electronic Health Record: Qualitative Survey Study %A Hägglund,Maria %A Scandurra,Isabella %+ Healthcare Sciences and e-Health, Department of Women's and Children's Health, Uppsala University, Akademiska Sjukhuset, Dag Hammarskjölds väg 14B, 1tr, Uppsala, 751 85, Sweden, 46 729999381, maria.hagglund@kbh.uu.se %K usability %K evaluation %K patient-accessible electronic health records %K open notes %K patient portals %K mobile phone %D 2022 %7 23.6.2022 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Patient portals are increasingly being implemented worldwide to ensure that patients have timely access to their health data, including patients’ access to their electronic health records. In Sweden, the e-service Journalen is a national patient-accessible electronic health record (PAEHR), accessible on the web through the national patient portal. User characteristics and perceived benefits of using a PAEHR will influence behavioral intentions to use and adoption; however, poor usability, which increases effort expectancy, may have a negative impact. Therefore, it is of interest to further explore how users of the PAEHR Journalen perceive its usability and usefulness. Objective: On the basis of the analysis of the survey respondents’ experiences of the usability of the Swedish PAEHR, this study aimed to identify specific usability problems that may need to be addressed in the future. Methods: A survey study was conducted to elicit opinions and experiences of patients using Journalen. Data were collected from June to October 2016. The questionnaire included a free-text question regarding the usability of the system, and the responses were analyzed using content analysis with a sociotechnical framework as guidance when grouping identified usability issues. Results: During the survey period, 423,141 users logged into Journalen, of whom 2587 (0.61%) completed the survey (unique users who logged in; response rate 0.61%). Of the 2587 respondents, 186 (7.19%) provided free-text comments on the usability questions. The analysis resulted in 19 categories, which could be grouped under 7 of the 8 dimensions in the sociotechnical framework of Sittig and Singh. The most frequently mentioned problems were related to regional access limitations, structure and navigation of the patient portal, and language and understanding. Conclusions: Although the survey respondents, who were also end users of the PAEHR Journalen, were overall satisfied with its usability, they also experienced important challenges when accessing their records. For all patients to be able to reap the benefits of record access, it is essential to understand both the usability challenges they encounter and, more broadly, how policies, regulations, and technical implementation decisions affect the usefulness of record access. The results presented here are specific to the Swedish PAEHR Journalen but also provide important insights into how design and implementation of record access can be improved in any context. %M 35737444 %R 10.2196/37192 %U https://humanfactors.jmir.org/2022/2/e37192 %U https://doi.org/10.2196/37192 %U http://www.ncbi.nlm.nih.gov/pubmed/35737444 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 6 %N 6 %P e34141 %T Finding Primary Care—Repurposing Physician Registration Data to Generate a Regionally Accurate List of Primary Care Clinics: Development and Validation of an Open-Source Algorithm %A Cooper,Ian R %A Lindsay,Cameron %A Fraser,Keaton %A Hill,Tiffany T %A Siu,Andrew %A Fletcher,Sarah %A Klimas,Jan %A Hamilton,Michee-Ana %A Frazer,Amanda D %A Humphrys,Elka %A Koepke,Kira %A Hedden,Lindsay %A Price,Morgan %A McCracken,Rita K %+ Innovation Support Unit, Department of Family Practice, University of British Columbia, 3rd Floor David Strangway Building, Vancouver, BC, V6T 1Z3, Canada, 1 604 827 4168, rita.mccracken@ubc.ca %K physicians, primary care %K primary health care %K health services accessibility %K practice patterns, physicians %K physicians’ offices %K computing methodologies %K algorithms %D 2022 %7 22.6.2022 %9 Original Paper %J JMIR Form Res %G English %X Background: Some Canadians have limited access to longitudinal primary care, despite its known advantages for population health. Current initiatives to transform primary care aim to increase access to team-based primary care clinics. However, many regions lack a reliable method to enumerate clinics, limiting estimates of clinical capacity and ongoing access gaps. A region-based complete clinic list is needed to effectively describe clinic characteristics and to compare primary care outcomes at the clinic level. Objective: The objective of this study is to show how publicly available data sources, including the provincial physician license registry, can be used to generate a verifiable, region-wide list of primary care clinics in British Columbia, Canada, using a process named the Clinic List Algorithm (CLA). Methods: The CLA has 10 steps: (1) collect data sets, (2) develop clinic inclusion and exclusion criteria, (3) process data sets, (4) consolidate data sets, (5) transform from list of physicians to initial list of clinics, (6) add additional metadata, (7) create working lists, (8) verify working lists, (9) consolidate working lists, and (10) adjust processing steps based on learnings. Results: The College of Physicians and Surgeons of British Columbia Registry contained 13,726 physicians, at 2915 unique addresses, 6942 (50.58%) of whom were family physicians (FPs) licensed to practice in British Columbia. The CLA identified 1239 addresses where primary care was delivered by 4262 (61.39%) FPs. Of the included addresses, 84.50% (n=1047) were in urban locations, and there was a median of 2 (IQR 2-4, range 1-23) FPs at each unique address. Conclusions: The CLA provides a region-wide description of primary care clinics that improves on simple counts of primary care providers or self-report lists. It identifies the number and location of primary care clinics and excludes primary care providers who are likely not providing community-based primary care. Such information may be useful for estimates of capacity of primary care, as well as for policy planning and research in regions engaged in primary care evaluation or transformation. %M 35731556 %R 10.2196/34141 %U https://formative.jmir.org/2022/6/e34141 %U https://doi.org/10.2196/34141 %U http://www.ncbi.nlm.nih.gov/pubmed/35731556 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 2 %P e31029 %T A Reference Architecture for Data-Driven and Adaptive Internet-Delivered Psychological Treatment Systems: Software Architecture Development and Validation Study %A Mukhiya,Suresh Kumar %A Lamo,Yngve %A Rabbi,Fazle %+ Western Norway University of Applied Sciences, Inndalsveien 28, 5063 Bergen, Bergen, 5063, Norway, 47 55 58 58 00, itsmeskm99@gmail.com %K software architecture %K adaptive system %K IDPT system %K health care systems %K ICBT %K adaptive strategies %K personalized therapies %K reference architecture %D 2022 %7 20.6.2022 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Internet-delivered psychological treatment (IDPT) systems are software applications that offer psychological treatments via the internet. Such IDPT systems have become one of the most commonly practiced and widely researched forms of psychotherapy. Evidence shows that psychological treatments delivered by IDPT systems can be an effective way of treating mental health morbidities. However, current IDPT systems have high dropout rates and low user adherence. The primary reason is that the current IDPT systems are not flexible, adaptable, and personalized as they follow a fixed tunnel-based treatment architecture. A fixed tunnel-based architecture follows predefined, sequential treatment content for every patient, irrespective of their context, preferences, and needs. Moreover, current IDPT systems have poor interoperability, making it difficult to reuse and share treatment materials. There is a lack of development and documentation standards, conceptual frameworks, and established (clinical) guidelines for such IDPT systems. As a result, several ad hoc forms of IDPT models exist. Consequently, developers and researchers have tended to reinvent new versions of IDPT systems, making them more complex and less interoperable. Objective: This study aimed to design, develop, and evaluate a reference architecture (RA) for adaptive systems that can facilitate the design and development of adaptive, interoperable, and reusable IDPT systems. Methods: This study was conducted in collaboration with a large interdisciplinary project entitled INTROMAT (Introducing Mental Health through Adaptive Technology), which brings together information and communications technology researchers, information and communications technology industries, health researchers, patients, clinicians, and patients’ next of kin to reach its vision. First, we investigated previous studies and state-of-the-art works based on the project’s problem domain and goals. On the basis of the findings from these investigations, we identified 2 primary gaps in current IDPT systems: lack of adaptiveness and limited interoperability. Second, we used model-driven engineering and Domain-Driven Design techniques to design, develop, and validate the RA for building adaptive, interoperable, and reusable IDPT systems to address these gaps. Third, based on the proposed RA, we implemented a prototype as the open-source software. Finally, we evaluated the RA and open-source implementation using empirical (case study) and nonempirical approaches (software architecture analysis method, expert evaluation, and software quality attributes). Results: This paper outlines an RA that supports flexible user modeling and the adaptive delivery of treatments. To evaluate the proposed RA, we developed an open-source software based on the proposed RA. The open-source framework aims to improve development productivity, facilitate interoperability, increase reusability, and expedite communication with domain experts. Conclusions: Our results showed that the proposed RA is flexible and capable of adapting interventions based on patients’ needs, preferences, and context. Furthermore, developers and researchers can extend the proposed RA to various health care interventions. %M 35723905 %R 10.2196/31029 %U https://humanfactors.jmir.org/2022/2/e31029 %U https://doi.org/10.2196/31029 %U http://www.ncbi.nlm.nih.gov/pubmed/35723905 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 2 %P e35032 %T An Electronic Data Capture Tool for Data Collection During Public Health Emergencies: Development and Usability Study %A Brown,Joan %A Bhatnagar,Manas %A Gordon,Hugh %A Goodner,Jared %A Cobb,J Perren %A Lutrick,Karen %+ Clinical Operations Business Intelligence, The Keck School of Medicine of the University of Southern California, 1520 San Pablo St, Los Angeles, CA, 90033, United States, 1 310 245 8079, joancbrown@gmail.com %K clinical research design %K disaster management %K informatics %K public health emergencies %K electronic data capture %K design tenet %K public health emergency %K electronic data %K EDCT %K real time data %D 2022 %7 9.6.2022 %9 Original Paper %J JMIR Hum Factors %G English %X Background: The Discovery Critical Care Research Network Program for Resilience and Emergency Preparedness (Discovery PREP) partnered with a third-party technology vendor to design and implement an electronic data capture tool that addressed multisite data collection challenges during public health emergencies (PHE) in the United States. The basis of the work was to design an electronic data capture tool and to prospectively gather data on usability from bedside clinicians during national health system stress queries and influenza observational studies. Objective: The aim of this paper is to describe the lessons learned in the design and implementation of a novel electronic data capture tool with the goal of significantly increasing the nation’s capability to manage real-time data collection and analysis during PHE. Methods: A multiyear and multiphase design approach was taken to create an electronic data capture tool, which was used to pilot rapid data capture during a simulated PHE. Following the pilot, the study team retrospectively assessed the feasibility of automating the data captured by the electronic data capture tool directly from the electronic health record. In addition to user feedback during semistructured interviews, the System Usability Scale (SUS) questionnaire was used as a basis to evaluate the usability and performance of the electronic data capture tool. Results: Participants included Discovery PREP physicians, their local administrators, and data collectors from tertiary-level academic medical centers at 5 different institutions. User feedback indicated that the designed system had an intuitive user interface and could be used to automate study communication tasks making for more efficient management of multisite studies. SUS questionnaire results classified the system as highly usable (SUS score 82.5/100). Automation of 17 (61%) of the 28 variables in the influenza observational study was deemed feasible during the exploration of automated versus manual data abstraction. The creation and use of the Project Meridian electronic data capture tool identified 6 key design requirements for multisite data collection, including the need for the following: (1) scalability irrespective of the type of participant; (2) a common data set across sites; (3) automated back end administrative capability (eg, reminders and a self-service status board); (4) multimedia communication pathways (eg, email and SMS text messaging); (5) interoperability and integration with local site information technology infrastructure; and (6) natural language processing to extract nondiscrete data elements. Conclusions: The use of the electronic data capture tool in multiple multisite Discovery PREP clinical studies proved the feasibility of using the novel, cloud-based platform in practice. The lessons learned from this effort can be used to inform the improvement of ongoing global multisite data collection efforts during the COVID-19 pandemic and transform current manual data abstraction approaches into reliable, real time, and automated information exchange. Future research is needed to expand the ability to perform automated multisite data extraction during a PHE and beyond. %M 35679114 %R 10.2196/35032 %U https://humanfactors.jmir.org/2022/2/e35032 %U https://doi.org/10.2196/35032 %U http://www.ncbi.nlm.nih.gov/pubmed/35679114 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 2 %P e33960 %T Factors Influencing Clinician Trust in Predictive Clinical Decision Support Systems for In-Hospital Deterioration: Qualitative Descriptive Study %A Schwartz,Jessica M %A George,Maureen %A Rossetti,Sarah Collins %A Dykes,Patricia C %A Minshall,Simon R %A Lucas,Eugene %A Cato,Kenrick D %+ Department of Biomedical Informatics, Columbia University, 622 West 168th Street, PH20 3720, New York, NY, 10032, United States, 1 212 305 5334, jms2468@cumc.columbia.edu %K clinical decision support systems %K machine learning %K inpatient %K nurses %K physicians %K qualitative research %D 2022 %7 12.5.2022 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Clinician trust in machine learning–based clinical decision support systems (CDSSs) for predicting in-hospital deterioration (a type of predictive CDSS) is essential for adoption. Evidence shows that clinician trust in predictive CDSSs is influenced by perceived understandability and perceived accuracy. Objective: The aim of this study was to explore the phenomenon of clinician trust in predictive CDSSs for in-hospital deterioration by confirming and characterizing factors known to influence trust (understandability and accuracy), uncovering and describing other influencing factors, and comparing nurses’ and prescribing providers’ trust in predictive CDSSs. Methods: We followed a qualitative descriptive methodology conducting directed deductive and inductive content analysis of interview data. Directed deductive analyses were guided by the human-computer trust conceptual framework. Semistructured interviews were conducted with nurses and prescribing providers (physicians, physician assistants, or nurse practitioners) working with a predictive CDSS at 2 hospitals in Mass General Brigham. Results: A total of 17 clinicians were interviewed. Concepts from the human-computer trust conceptual framework—perceived understandability and perceived technical competence (ie, perceived accuracy)—were found to influence clinician trust in predictive CDSSs for in-hospital deterioration. The concordance between clinicians’ impressions of patients’ clinical status and system predictions influenced clinicians’ perceptions of system accuracy. Understandability was influenced by system explanations, both global and local, as well as training. In total, 3 additional themes emerged from the inductive analysis. The first, perceived actionability, captured the variation in clinicians’ desires for predictive CDSSs to recommend a discrete action. The second, evidence, described the importance of both macro- (scientific) and micro- (anecdotal) evidence for fostering trust. The final theme, equitability, described fairness in system predictions. The findings were largely similar between nurses and prescribing providers. Conclusions: Although there is a perceived trade-off between machine learning–based CDSS accuracy and understandability, our findings confirm that both are important for fostering clinician trust in predictive CDSSs for in-hospital deterioration. We found that reliance on the predictive CDSS in the clinical workflow may influence clinicians’ requirements for trust. Future research should explore the impact of reliance, the optimal explanation design for enhancing understandability, and the role of perceived actionability in driving trust. %M 35550304 %R 10.2196/33960 %U https://humanfactors.jmir.org/2022/2/e33960 %U https://doi.org/10.2196/33960 %U http://www.ncbi.nlm.nih.gov/pubmed/35550304 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 2 %P e35325 %T Audio Recording Patient-Nurse Verbal Communications in Home Health Care Settings: Pilot Feasibility and Usability Study %A Zolnoori,Maryam %A Vergez,Sasha %A Kostic,Zoran %A Jonnalagadda,Siddhartha Reddy %A V McDonald,Margaret %A Bowles,Kathryn K H %A Topaz,Maxim %+ School of Nursing, Columbia University, 390 Fort Washington Avenue, New York, NY, 10033, United States, 1 317 515 1950, mz2825@cumc.columbia.edu %K patients %K HHC %K communications %K nurse %K audio recording %K device %D 2022 %7 11.5.2022 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Patients’ spontaneous speech can act as a biomarker for identifying pathological entities, such as mental illness. Despite this potential, audio recording patients’ spontaneous speech is not part of clinical workflows, and health care organizations often do not have dedicated policies regarding the audio recording of clinical encounters. No previous studies have investigated the best practical approach for integrating audio recording of patient-clinician encounters into clinical workflows, particularly in the home health care (HHC) setting. Objective: This study aimed to evaluate the functionality and usability of several audio-recording devices for the audio recording of patient-nurse verbal communications in the HHC settings and elicit HHC stakeholder (patients and nurses) perspectives about the facilitators of and barriers to integrating audio recordings into clinical workflows. Methods: This study was conducted at a large urban HHC agency located in New York, United States. We evaluated the usability and functionality of 7 audio-recording devices in a laboratory (controlled) setting. A total of 3 devices—Saramonic Blink500, Sony ICD-TX6, and Black Vox 365—were further evaluated in a clinical setting (patients’ homes) by HHC nurses who completed the System Usability Scale questionnaire and participated in a short, structured interview to elicit feedback about each device. We also evaluated the accuracy of the automatic transcription of audio-recorded encounters for the 3 devices using the Amazon Web Service Transcribe. Word error rate was used to measure the accuracy of automated speech transcription. To understand the facilitators of and barriers to integrating audio recording of encounters into clinical workflows, we conducted semistructured interviews with 3 HHC nurses and 10 HHC patients. Thematic analysis was used to analyze the transcribed interviews. Results: Saramonic Blink500 received the best overall evaluation score. The System Usability Scale score and word error rate for Saramonic Blink500 were 65% and 26%, respectively, and nurses found it easier to approach patients using this device than with the other 2 devices. Overall, patients found the process of audio recording to be satisfactory and convenient, with minimal impact on their communication with nurses. Although, in general, nurses also found the process easy to learn and satisfactory, they suggested that the audio recording of HHC encounters can affect their communication patterns. In addition, nurses were not aware of the potential to use audio-recorded encounters to improve health care services. Nurses also indicated that they would need to involve their managers to determine how audio recordings could be integrated into their clinical workflows and for any ongoing use of audio recordings during patient care management. Conclusions: This study established the feasibility of audio recording HHC patient-nurse encounters. Training HHC nurses about the importance of the audio-recording process and the support of clinical managers are essential factors for successful implementation. %M 35544296 %R 10.2196/35325 %U https://humanfactors.jmir.org/2022/2/e35325 %U https://doi.org/10.2196/35325 %U http://www.ncbi.nlm.nih.gov/pubmed/35544296 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 2 %P e31758 %T Assessing the Usability of a Clinical Decision Support System: Heuristic Evaluation %A Cho,Hwayoung %A Keenan,Gail %A Madandola,Olatunde O %A Dos Santos,Fabiana Cristina %A Macieira,Tamara G R %A Bjarnadottir,Ragnhildur I %A Priola,Karen J B %A Dunn Lopez,Karen %+ College of Nursing, University of Florida, 1225 Center Dr, Gainesville, FL, 32611, United States, 1 3522736347, hcho@ufl.edu %K usability %K heuristic %K clinical decision support %K electronic health record %K expert review %K evaluation %K user interface %K human-computer interaction %D 2022 %7 10.5.2022 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Poor usability is a primary cause of unintended consequences related to the use of electronic health record (EHR) systems, which negatively impacts patient safety. Due to the cost and time needed to carry out iterative evaluations, many EHR components, such as clinical decision support systems (CDSSs), have not undergone rigorous usability testing prior to their deployment in clinical practice. Usability testing in the predeployment phase is crucial to eliminating usability issues and preventing costly fixes that will be needed if these issues are found after the system’s implementation. Objective: This study presents an example application of a systematic evaluation method that uses clinician experts with human-computer interaction (HCI) expertise to evaluate the usability of an electronic clinical decision support (CDS) intervention prior to its deployment in a randomized controlled trial. Methods: We invited 6 HCI experts to participate in a heuristic evaluation of our CDS intervention. Each expert was asked to independently explore the intervention at least twice. After completing the assigned tasks using patient scenarios, each expert completed a heuristic evaluation checklist developed by Bright et al based on Nielsen’s 10 heuristics. The experts also rated the overall severity of each identified heuristic violation on a scale of 0 to 4, where 0 indicates no problems and 4 indicates a usability catastrophe. Data from the experts’ coded comments were synthesized, and the severity of each identified usability heuristic was analyzed. Results: The 6 HCI experts included professionals from the fields of nursing (n=4), pharmaceutical science (n=1), and systems engineering (n=1). The mean overall severity scores of the identified heuristic violations ranged from 0.66 (flexibility and efficiency of use) to 2.00 (user control and freedom and error prevention), in which scores closer to 0 indicate a more usable system. The heuristic principle user control and freedom was identified as the most in need of refinement and, particularly by nonnursing HCI experts, considered as having major usability problems. In response to the heuristic match between system and the real world, the experts pointed to the reversed direction of our system’s pain scale scores (1=severe pain) compared to those commonly used in clinical practice (typically 1=mild pain); although this was identified as a minor usability problem, its refinement was repeatedly emphasized by nursing HCI experts. Conclusions: Our heuristic evaluation process is simple and systematic and can be used at multiple stages of system development to reduce the time and cost needed to establish the usability of a system before its widespread implementation. Furthermore, heuristic evaluations can help organizations develop transparent reporting protocols for usability, as required by Title IV of the 21st Century Cures Act. Testing of EHRs and CDSSs by clinicians with HCI expertise in heuristic evaluation processes has the potential to reduce the frequency of testing while increasing its quality, which may reduce clinicians’ cognitive workload and errors and enhance the adoption of EHRs and CDSSs. %M 35536613 %R 10.2196/31758 %U https://humanfactors.jmir.org/2022/2/e31758 %U https://doi.org/10.2196/31758 %U http://www.ncbi.nlm.nih.gov/pubmed/35536613 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 2 %P e33967 %T Enhancing Clinical Information Display to Improve Patient Encounters: Human-Centered Design and Evaluation of the Parkinson Disease-BRIDGE Platform %A Brown,Ethan G %A Schleimer,Erica %A Bledsoe,Ian O %A Rowles,William %A Miller,Nicolette A %A Sanders,Stephan J %A Rankin,Katherine P %A Ostrem,Jill L %A Tanner,Caroline M %A Bove,Riley %+ University of California San Francisco Weill Institute for Neurosciences, University of California San Francisco, San Francisco, CA, United States, 1 415 353 2311, riley.bove@ucsf.edu %K human-centered design %K personal health record %K visualization in eHealth %K Parkinson disease %K digital health %D 2022 %7 6.5.2022 %9 Original Paper %J JMIR Hum Factors %G English %X Background: People with Parkinson disease (PD) have a variety of complex medical problems that require detailed review at each clinical encounter for appropriate management. Care of other complex conditions has benefited from digital health solutions that efficiently integrate disparate clinical information. Although various digital approaches have been developed for research and care in PD, no digital solution to personalize and improve communication in a clinical encounter is readily available. Objective: We intend to improve the efficacy and efficiency of clinical encounters with people with PD through the development of a platform (PD-BRIDGE) with personalized clinical information from the electronic health record (EHR) and patient-reported outcome (PRO) data. Methods: Using human-centered design (HCD) processes, we engaged clinician and patient stakeholders in developing PD-BRIDGE through three phases: an inspiration phase involving focus groups and discussions with people having PD, an ideation phase generating preliminary mock-ups for feedback, and an implementation phase testing the platform. To qualitatively evaluate the platform, movement disorders neurologists and people with PD were sent questionnaires asking about the technical validity, usability, and clinical relevance of PD-BRIDGE after their encounter. Results: The HCD process led to a platform with 4 modules. Among these, 3 modules that pulled data from the EHR include a longitudinal module showing motor ratings over time, a display module showing the most recently collected clinical rating scales, and another display module showing relevant laboratory values and diagnoses; the fourth module displays motor symptom fluctuation based on an at-home diary. In the implementation phase, PD-BRIDGE was used in 17 clinical encounters for patients cared for by 1 of 11 movement disorders neurologists. Most patients felt that PD-BRIDGE facilitated communication with their clinician (n=14, 83%) and helped them understand their disease trajectory (n=11, 65%) and their clinician’s recommendations (n=11, 65%). Neurologists felt that PD-BRIDGE improved their ability to understand the patients’ disease course (n=13, 75% of encounters), supported clinical care recommendations (n=15, 87%), and helped them communicate with their patients (n=14, 81%). In terms of improvements, neurologists noted that data in PD-BRIDGE were not exhaustive in 62% (n=11) of the encounters. Conclusions: Integrating clinically relevant information from EHR and PRO data into a visually efficient platform (PD-BRIDGE) can facilitate clinical encounters with people with PD. Developing new modules with more disparate information could improve these complex encounters even further. %M 35522472 %R 10.2196/33967 %U https://humanfactors.jmir.org/2022/2/e33967 %U https://doi.org/10.2196/33967 %U http://www.ncbi.nlm.nih.gov/pubmed/35522472 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 2 %P e32456 %T Exploring Human-Data Interaction in Clinical Decision-making Using Scenarios: Co-design Study %A Tendedez,Helena %A Ferrario,Maria-Angela %A McNaney,Roisin %A Gradinar,Adrian %+ Lancaster Institute for the Contemporary Arts, Lancaster University, The LICA Building, Lancaster, , United Kingdom, 44 1524510302, a.gradinar@lancaster.ac.uk %K data-supported decision-making %K health care professionals %K respiratory care %K scenario-based design %K clinical decision-making %K decision support %K COPD %K respiratory conditions %K digital health %K user-centered design %K health technologies %D 2022 %7 6.5.2022 %9 Original Paper %J JMIR Hum Factors %G English %X Background: When caring for patients with chronic conditions such as chronic obstructive pulmonary disease (COPD), health care professionals (HCPs) rely on multiple data sources to make decisions. Collating and visualizing these data, for example, on clinical dashboards, holds the potential to support timely and informed decision-making. Most studies on data-supported decision-making (DSDM) technologies for health care have focused on their technical feasibility or quantitative effectiveness. Although these studies are an important contribution to the literature, they do not further our limited understanding of how HCPs engage with these technologies and how they can be designed to support specific contexts of use. To advance our knowledge in this area, we must work with HCPs to explore this space and the real-world complexities of health care work and service structures. Objective: This study aimed to qualitatively explore how DSDM technologies could support HCPs in their decision-making regarding COPD care. We created a scenario-based research tool called Respire, which visualizes HCPs’ data needs about their patients with COPD and services. We used Respire with HCPs to uncover rich and nuanced findings about human-data interaction in this context, focusing on the real-world challenges that HCPs face when carrying out their work and making decisions. Methods: We engaged 9 respiratory HCPs from 2 collaborating health care organizations to design Respire. We then used Respire as a tool to investigate human-data interaction in the context of decision-making about COPD care. The study followed a co-design approach that had 3 stages and spanned 2 years. The first stage involved 5 workshops with HCPs to identify data interaction scenarios that would support their work. The second stage involved creating Respire, an interactive scenario-based web app that visualizes HCPs’ data needs, incorporating feedback from HCPs. The final stage involved 11 one-to-one sessions with HCPs to use Respire, focusing on how they envisaged that it could support their work and decisions about care. Results: We found that HCPs trust data differently depending on where it came from and who recorded it, sporadic and subjective data generated by patients have value but create challenges for decision-making, and HCPs require support in interpreting and responding to new data and its use cases. Conclusions: Our study uncovered important lessons for the design of DSDM technologies to support health care contexts. We show that although DSDM technologies have the potential to support patient care and health care delivery, important sociotechnical and human-data interaction challenges influence the design and deployment of these technologies. Exploring these considerations during the design process can ensure that DSDM technologies are designed with a holistic view of how decision-making and engagement with data occur in health care contexts. %M 35522463 %R 10.2196/32456 %U https://humanfactors.jmir.org/2022/2/e32456 %U https://doi.org/10.2196/32456 %U http://www.ncbi.nlm.nih.gov/pubmed/35522463 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 5 %P e33505 %T Health Care Professionals’ Experiences of Web-Based Symptom Checkers for Triage: Cross-sectional Survey Study %A Kujala,Sari %A Hörhammer,Iiris %+ Department of Computer Science, Aalto University, PO Box 15400, Espoo, FI-00076, Finland, 358 503862768, sari.kujala@aalto.fi %K adoption %K symptom checker %K triage %K health care professional %K survey %K online health %K digital health %K health organizations %K health care %D 2022 %7 5.5.2022 %9 Original Paper %J J Med Internet Res %G English %X Background: Web-based symptom checkers are promising tools that provide help to patients seeking guidance on health problems. Many health organizations have started using them to enhance triage. Patients use the symptom checker to report their symptoms online and submit the report to the health care center through the system. Health care professionals (registered nurse, practical nurse, general physician, physiotherapist, etc) receive patient inquiries with urgency rating, decide on actions to be taken, and communicate these to the patients. The success of the adoption, however, depends on whether the tools can efficiently support health care professionals’ workflow and achieve their support. Objective: This study explores the factors influencing health care professionals’ support for a web-based symptom checker for triage. Methods: Data were collected through a web-based survey of 639 health care professionals using either of the two most used web-based symptom checkers in the Finnish public primary care. Linear regression models were fitted to study the associations between the study variables and health care professionals’ support for the symptom checkers. In addition, the health care professionals’ comments collected via survey were qualitatively analyzed to elicit additional insights about the benefits and challenges of the clinical use of symptom checkers. Results: Results show that the perceived beneficial influence of the symptom checkers on health care professionals’ work and the perceived usability of the tools were positively associated with professionals’ support. The perceived benefits to patients and organizational support for use were positively associated, and threat to professionals’ autonomy was negatively associated with health care professionals’ support. These associations were, however, not independent of other factors included in the models. The influences on professionals’ work were both positive and negative; the tools streamlined work by providing preliminary information on patients and reduced the number of phone calls, but they also created extra work as the professionals needed to call patients and ask clarifying questions. Managing time between the use of symptom checkers and other tasks was also challenging. Meanwhile, according to health care professionals’ experience, the symptom checkers benefited patients as they received help quickly with a lower threshold for care. Conclusions: The efficient use of symptom checkers for triage requires usable solutions that support health care professionals’ work. High-quality information about the patients’ conditions and an efficient way of communicating with patients are needed. Using a new eHealth tool also requires that health organizations and teams reorganize their workflows and work distributions to support clinical processes. %M 35511254 %R 10.2196/33505 %U https://www.jmir.org/2022/5/e33505 %U https://doi.org/10.2196/33505 %U http://www.ncbi.nlm.nih.gov/pubmed/35511254 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 2 %P e35094 %T Persuasive Design Techniques and App Design Recommendations to Improve Health Workforce Capability in Rural Health Professionals: What Do Users Want and How Does an App Help? %A Pit,Sabrina %A Ramsden,Robyn %A Tan,Aaron JH %A Payne,Kristy %A Barr,James %A Eames,Benjamin %A Edwards,Mike %A Colbran,Richard %+ New South Wales Rural Doctors Network, 1/53 Cleary Street, Hamilton, 2308, Australia, 61 429455720, sabrina.pit@sydney.edu.au %K health %K wellness %K mobile apps %K persuasive strategies %K behavior change %K review %K health workforce %K capability %K career %K employment %K rural %K workforce planning %K mHealth %K mobile health %K digital health %K health professional %K user experience %K health application %K task support %K social support %K dialog support %D 2022 %7 2.5.2022 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Health professionals’ perceptions of persuasive design techniques for use in technological solutions to improve health workforce capability have not been previously explored. Objective: This study aims to explore rural health professionals’ perceptions of health workforce capability and persuasive design techniques; and translate these into recommendations for designing a health workforce capability app to increase their impact and usefulness. Methods: Qualitative interviews with 13 rural health professionals were conducted. Subsequently, 32 persuasive techniques were used as a framework to deductively analyze the data. Persuasive design technique domains were Primary Task Support, Dialog Support, System Credibility Support, Social Support, and Cialdini’s Principles of Persuasion. Results: Persuasive design techniques can be applied across the factors that influence health workforce capability including health and personal qualities; competencies and skills; values, attitudes, and motivation; and factors that operate outside of work and at the team, organizational, and labor market levels. The majority of the 32 persuasive design techniques were reflected in the data from the interviews and led to recommendations as to how these could be translated into practice, with the exception of scarcity. Many suggestions and persuasive design techniques link back to the need for tailored and localized solutions such as the need for country-specific–based evidence, the wish for localized communities of practice, learning from other rural health professionals, and referral pathways to other clinicians. Participants identified how persuasive design techniques can optimize the user experience to help meet rural health professionals needs for more efficient systems to improve patient access to care, quality care, and to enable working in interprofessional team-based care. Social inclusion plays a vital role for health professionals, indicating the importance of the Social Support domain of persuasive techniques. Overall, health professionals were open to self-monitoring of their work performance and some professionals used wearables to monitor their health. Conclusions: Rural health professionals’ perceptions of health workforce capability informed which persuasive design techniques can be used to optimize the user experience of an app. These were translated into recommendations for designing a health workforce capability app to increase likelihood of adoption. This study has also contributed to the further validation of the Persuasive Systems Design model through empirically aligning elements of the model to increase persuasive system content and functionality with real-world applied data, in this case the health workforce capability of rural health professionals. Our findings confirm that these techniques can be used to develop a future prototype of an app that may assist health professionals in improving or maintaining their health workforce capability which in turn may increase recruitment and retention in rural areas. Health professionals need to be central during the design phase. Interventions are needed to provide a supportive environment to rural and remote health professionals to increase their rural health workforce capability. %M 35499866 %R 10.2196/35094 %U https://humanfactors.jmir.org/2022/2/e35094 %U https://doi.org/10.2196/35094 %U http://www.ncbi.nlm.nih.gov/pubmed/35499866 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 2 %P e29780 %T Health Professionals’ eHealth Literacy and System Experience Before and 3 Months After the Implementation of an Electronic Health Record System: Longitudinal Study %A Kayser,Lars %A Karnoe,Astrid %A Duminski,Emily %A Jakobsen,Svend %A Terp,Rikke %A Dansholm,Susanne %A Roeder,Michael %A From,Gustav %+ Section of Health Service Research, Department of Public Health, University of Copenhagen, Øster Farimagsgade 5, Copenhagen, 1353, Denmark, 45 28757291, lk@sund.ku.dk %K health care professionals %K eHealth literacy %K electronic health record %K implementation %K digital health %K eHealth %K health literacy %K health records %K eHealth records %K patient care %D 2022 %7 29.4.2022 %9 Original Paper %J JMIR Hum Factors %G English %X Background: The implementation of an integrated electronic health record (EHR) system can potentially provide health care providers with support standardization of patient care, pathways, and workflows, as well as provide medical staff with decision support, easier access, and the same interface across features and subsystems. These potentials require an implementation process in which the expectations of the medical staff and the provider of the new system are aligned with respect to the medical staff’s knowledge and skills, as well as the interface and performance of the system. Awareness of the medical staff’s level of eHealth literacy may be a way of understanding and aligning these expectations and following the progression of the implementation process. Objective: The objective of this study was to investigate how a newly developed and modified instrument measuring the medical staff’s eHealth literacy (staff eHealth Literacy Questionnaire [eHLQ]) can be used to inform the system provider and the health care organization in the implementation process and evaluate whether the medical staff’s perceptions of the ease of use change and how this may be related to their level of eHealth literacy. Methods: A modified version of the eHLQ was distributed to the staff of a medical department in Denmark before and 3 months after the implementation of a new EHR system. The survey also included questions related to users’ perceived ease of use and their self-reported information technology skills. Results: The mean age of the 194 participants before implementation was 43.1 (SD 12.4) years, and for the 198 participants after implementation, it was 42.3 (SD 12.5) years. After the implementation, the only difference compared with the preimplementation data was a small decrease in staff eHLQ5 (motivated to engage with digital services; unpaired 2-tailed t test; P=.009; effect size 0.267), and the values of the scales relating to the medical staff’s knowledge and skills (eHLQ1-3) were approximately ≥3 both before and after implementation. The range of scores was narrower after implementation, indicating that some of those with the lowest ability benefited from the training and new experiences with the EHR. There was an association between perceived ease of use and the 3 tested staff eHLQ scales, both before and after implementation. Conclusions: The staff eHLQ may be a good candidate for monitoring the medical staff’s digital competence in and response to the implementation of new digital solutions. This may enable those responsible for the implementation to tailor efforts to the specific needs of segments of users and inform them if the process is not going according to plan with respect to the staff’s information technology–related knowledge and skills, trust in data security, motivation, and experience of a coherent system that suits their needs and supports the workflows and data availability. %M 35486414 %R 10.2196/29780 %U https://humanfactors.jmir.org/2022/2/e29780 %U https://doi.org/10.2196/29780 %U http://www.ncbi.nlm.nih.gov/pubmed/35486414 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 2 %P e29118 %T Effective Communication of Personalized Risks and Patient Preferences During Surgical Informed Consent Using Data Visualization: Qualitative Semistructured Interview Study With Patients After Surgery %A Gisladottir,Undina %A Nakikj,Drashko %A Jhunjhunwala,Rashi %A Panton,Jasmine %A Brat,Gabriel %A Gehlenborg,Nils %+ Department of Biomedical Informatics, Harvard Medical School, Harvard University, 10 Shattuck Street, Suite 514, Boston, MA, 02115, United States, 1 6174321535, nils@hms.harvard.edu %K data visualization %K surgical informed consent %K shared decision-making %K biomedical informatics %D 2022 %7 29.4.2022 %9 Original Paper %J JMIR Hum Factors %G English %X Background: There is no consensus on which risks to communicate to a prospective surgical patient during informed consent or how. Complicating the process, patient preferences may diverge from clinical assumptions and are often not considered for discussion. Such discrepancies can lead to confusion and resentment, raising the potential for legal action. To overcome these issues, we propose a visual consent tool that incorporates patient preferences and communicates personalized risks to patients using data visualization. We used this platform to identify key effective visual elements to communicate personalized surgical risks. Objective: Our main focus is to understand how to best communicate personalized risks using data visualization. To contextualize patient responses to the main question, we examine how patients perceive risks before surgery (research question 1), how suitably the visual consent tool is able to present personalized surgical risks (research question 2), how well our visualizations convey those personalized surgical risks (research question 3), and how the visual consent tool could improve the informed consent process and how it can be used (research question 4). Methods: We designed a visual consent tool to meet the objectives of our study. To calculate and list personalized surgical risks, we used the American College of Surgeons risk calculator. We created multiple visualization mock-ups using visual elements previously determined to be well-received for risk communication. Semistructured interviews were conducted with patients after surgery, and each of the mock-ups was presented and evaluated independently and in the context of our visual consent tool design. The interviews were transcribed, and thematic analysis was performed to identify major themes. We also applied a quantitative approach to the analysis to assess the prevalence of different perceptions of the visualizations presented in our tool. Results: In total, 20 patients were interviewed, with a median age of 59 (range 29-87) years. Thematic analysis revealed factors that influenced the perception of risk (the surgical procedure, the cognitive capacity of the patient, and the timing of consent; research question 1); factors that influenced the perceived value of risk visualizations (preference for rare event communication, preference for risk visualization, and usefulness of comparison with the average; research question 3); and perceived usefulness and use cases of the visual consent tool (research questions 2 and 4). Most importantly, we found that patients preferred the visual consent tool to current text-based documents and had no unified preferences for risk visualization. Furthermore, our findings suggest that patient concerns were not often represented in existing risk calculators. Conclusions: We identified key elements that influence effective visual risk communication in the perioperative setting and pointed out the limitations of the existing calculators in addressing patient concerns. Patient preference is highly variable and should influence choices regarding risk presentation and visualization. %M 35486432 %R 10.2196/29118 %U https://humanfactors.jmir.org/2022/2/e29118 %U https://doi.org/10.2196/29118 %U http://www.ncbi.nlm.nih.gov/pubmed/35486432 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 11 %N 4 %P e31464 %T Overcoming Decisional Gaps in High-Risk Prescribing by Junior Physicians Using Simulation-Based Training: Protocol for a Randomized Controlled Trial %A Lauffenburger,Julie C %A DiFrancesco,Matthew F %A Barlev,Renee A %A Robertson,Ted %A Kim,Erin %A Coll,Maxwell D %A Haff,Nancy %A Fontanet,Constance P %A Hanken,Kaitlin %A Oran,Rebecca %A Avorn,Jerry %A Choudhry,Niteesh K %+ Brigham and Women's Hospital, Harvard Medical School, 1620 Tremont Street Suite 3030, Boston, MA, 02120, United States, 1 6175258865, jlauffenburger@bwh.harvard.edu %K pragmatic trial %K behavioral science %K prescribing %K benzodiazepines %K antipsychotics %K impact evaluation %D 2022 %7 27.4.2022 %9 Protocol %J JMIR Res Protoc %G English %X Background: Gaps between rational thought and actual decisions are increasingly recognized as a reason why people make suboptimal choices in states of heightened emotion, such as stress. These observations may help explain why high-risk medications continue to be prescribed to acutely ill hospitalized older adults despite widely accepted recommendations against these practices. Role playing and other efforts, such as simulation training, have demonstrated benefits to help people avoid decisional gaps but have not been tested to reduce overprescribing of high-risk medications. Objective: This study aims to evaluate the impact of a simulation-based training program designed to address decisional gaps on prescribing of high-risk medications compared with control. Methods: In this 2-arm pragmatic trial, we are randomizing at least 36 first-year medical resident physicians (ie, interns) who provide care on inpatient general medicine services at a large academic medical center to either intervention (simulation-based training) or control (online educational training). The intervention comprises a 40-minute immersive individual simulation training consisting of a reality-based patient care scenario in a simulated environment at the beginning of their inpatient service rotation. The simulation focuses on 3 types of high-risk medications, including benzodiazepines, antipsychotics, and sedative hypnotics (Z-drugs), in older adults, and is specifically designed to help the physicians identify their reactions and prescribing decisions in stressful situations that are common in the inpatient setting. The simulation scenario is followed by a semistructured debriefing with an expert facilitator. The trial’s primary outcome is the number of medication doses for any of the high-risk medications prescribed by the interns to patients aged 65 years or older who were not taking one of the medications upon admission. Secondary outcomes include prescribing by all providers on the care team, being discharged on 1 of the medications, and prescribing of related medications (eg, melatonin, trazodone), or the medications of interest for the control intervention. These outcomes will be measured using electronic health record data. Results: Recruitment of interns began on March 29, 2021. Recruitment for the trial ended in Q42021, with follow-up completed by Q12022. Conclusions: This trial will evaluate the impact of a simulation-based training program designed using behavioral science principles on prescribing of high-risk medications by junior physicians. If the intervention is shown to be effective, this approach could potentially be reproducible by others and for a broader set of behaviors. Trial Registration: ClinicalTrials.gov NCT04668248; https://clinicaltrials.gov/ct2/show/NCT04668248 International Registered Report Identifier (IRRID): PRR1-10.2196/31464 %M 35475982 %R 10.2196/31464 %U https://www.researchprotocols.org/2022/4/e31464 %U https://doi.org/10.2196/31464 %U http://www.ncbi.nlm.nih.gov/pubmed/35475982 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 2 %P e33706 %T Adoption of a Postoperative Pain Self-Report Tool: Qualitative Study %A Thiel,Bram %A Iao,Inez %A Smid,Joris %A de Wit,Emmy %A Koopman,Seppe %A Geerts,Bart %A Godfried,Marc %A Kalkman,Cor %+ Department of Anesthesiology, OLVG Hospital, Oosterpark 9, Amsterdam, 1091AC, Netherlands, 31 0205994773, b.thiel@olvg.nl %K innovation %K eHealth adoption model, mobile health %K pain %K self-report %K perioperative medicine %K postoperative pain %K surgery %D 2022 %7 26.4.2022 %9 Original Paper %J JMIR Hum Factors %G English %X Background: With electronic technologies, patients are provided with tools to easily acquire information and to manage and record their own health status. eHealth interventions are already broadly applied to perioperative care. In a similar way, we aimed to utilize a smartphone application to enable postoperative patients to partially self-manage their postoperative pain. The results of a previously performed proof-of-concept study regarding the application were promising, and nurses as well as patients were optimistic regarding this innovative mobile application. Nevertheless, in reality, it appears that the usage and overall implementation of this application have stagnated since its introduction. Problems with innovation adoption are not novel; various studies have been conducted to explore the reasons for low implementation success of eHealth applications and indicated that adoption is influenced by multiple organizational factors. This study investigated the influence of these organizational factors on the adoption process, aiming to provide more insight in the dos and don’ts for implementing eHealth in the working processes of hospital care. Objective: This study aimed to provide insight in how to successfully implement a technological eHealth innovation in a general nonacademic hospital. Methods: A qualitative study was conducted to explore organizational factors affecting the innovation adoption process. Data were collected by conducting semistructured one-on-one interviews with 11 stakeholders. The data were analyzed using thematic analysis identifying overarching themes. Results: Absorptive capacity, referred to as an organization’s dynamic capability pertaining to knowledge creation and utilization that enhances an organization’s ability to gain and sustain a competitive advantage, was regarded as the most influential factor on the application’s adoption. Accordingly, it appeared that innovation adoption is mainly determined by the capability and willingness to assimilate and transform new information into productive use and the ability to absorb a novel innovation. Absorptive capacity was found to be influenced by the innovation’s benefit and the sense of ownership and responsibility. Organizational readiness and management support were also regarded as essential since absorptive capacity seemed to be mediated by these factors. The size of the hospital influenced eHealth adoption by the amount of resources available and by its organizational structure. Conclusions: In conclusion, absorptive capacity is essential for eHealth adoption, and it is mediated by management support and organizational readiness. It is recommended to increase the degree of willingness and ability to adopt an eHealth innovation by enhancing the relevance, engaging stakeholders, and assigning appropriate leaders to offer guidance. %M 35471472 %R 10.2196/33706 %U https://humanfactors.jmir.org/2022/2/e33706 %U https://doi.org/10.2196/33706 %U http://www.ncbi.nlm.nih.gov/pubmed/35471472 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 2 %P e30523 %T Requirements for a Bespoke Intensive Care Unit Dashboard in Response to the COVID-19 Pandemic: Semistructured Interview Study %A Davidson,Brittany %A Ferrer Portillo,Katiuska Mara %A Wac,Marceli %A McWilliams,Chris %A Bourdeaux,Christopher %A Craddock,Ian %+ Department of Electrical & Electronic Engineering, University of Bristol, 1 Cathedral Square, Bristol, BS1 5DD, United Kingdom, 44 07774286342, m.wac@bristol.ac.uk %K intensive care %K critical care %K COVID-19 %K human-centered design %K dashboard %K eHealth %K disease monitoring %K monitoring %K ICU %K design %K development %K interview %D 2022 %7 13.4.2022 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Intensive care units (ICUs) around the world are in high demand due to patients with COVID-19 requiring hospitalization. As researchers at the University of Bristol, we were approached to develop a bespoke data visualization dashboard to assist two local ICUs during the pandemic that will centralize disparate data sources in the ICU to help reduce the cognitive load on busy ICU staff in the ever-evolving pandemic. Objective: The aim of this study was to conduct interviews with ICU staff in University Hospitals Bristol and Weston National Health Service Foundation Trust to elicit requirements for a bespoke dashboard to monitor the high volume of patients, particularly during the COVID-19 pandemic. Methods: We conducted six semistructured interviews with clinical staff to obtain an overview of their requirements for the dashboard and to ensure its ultimate suitability for end users. Interview questions aimed to understand the job roles undertaken in the ICU, potential uses of the dashboard, specific issues associated with managing COVID-19 patients, key data of interest, and any concerns about the introduction of a dashboard into the ICU. Results: From our interviews, we found the following design requirements: (1) a flexible dashboard, where the functionality can be updated quickly and effectively to respond to emerging information about the management of this new disease; (2) a mobile dashboard, which allows staff to move around on wards with a dashboard, thus potentially replacing paper forms to enable detailed and consistent data entry; (3) a customizable and intuitive dashboard, where individual users would be able to customize the appearance of the dashboard to suit their role; (4) real-time data and trend analysis via informative data visualizations that help busy ICU staff to understand a patient’s clinical trajectory; and (5) the ability to manage tasks and staff, tracking both staff and patient movements, handovers, and task monitoring to ensure the highest quality of care. Conclusions: The findings of this study confirm that digital solutions for ICU use would potentially reduce the cognitive load of ICU staff and reduce clinical errors at a time of notably high demand of intensive health care. %M 35038301 %R 10.2196/30523 %U https://humanfactors.jmir.org/2022/2/e30523 %U https://doi.org/10.2196/30523 %U http://www.ncbi.nlm.nih.gov/pubmed/35038301 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 2 %P e32399 %T Implementation of a Web-Based Tool for Shared Decision-making in Lung Cancer Screening: Mixed Methods Quality Improvement Evaluation %A Lowery,Julie %A Fagerlin,Angela %A Larkin,Angela R %A Wiener,Renda S %A Skurla,Sarah E %A Caverly,Tanner J %+ Center for Clinical Management Research, Ann Arbor VA Healthcare System, 2215 Fuller Road, Ann Arbor, MI, 48105, United States, 1 303 587 1038, tcaverly@med.umich.edu %K shared decision-making %K lung cancer %K screening %K clinical decision support %K academic detailing %K quality improvement %K implementation %D 2022 %7 1.4.2022 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Lung cancer risk and life expectancy vary substantially across patients eligible for low-dose computed tomography lung cancer screening (LCS), which has important consequences for optimizing LCS decisions for different patients. To account for this heterogeneity during decision-making, web-based decision support tools are needed to enable quick calculations and streamline the process of obtaining individualized information that more accurately informs patient-clinician LCS discussions. We created DecisionPrecision, a clinician-facing web-based decision support tool, to help tailor the LCS discussion to a patient’s individualized lung cancer risk and estimated net benefit. Objective: The objective of our study is to test two strategies for implementing DecisionPrecision in primary care at eight Veterans Affairs medical centers: a quality improvement (QI) training approach and academic detailing (AD). Methods: Phase 1 comprised a multisite, cluster randomized trial comparing the effectiveness of standard implementation (adding a link to DecisionPrecision in the electronic health record vs standard implementation plus the Learn, Engage, Act, and Process [LEAP] QI training program). The primary outcome measure was the use of DecisionPrecision at each site before versus after LEAP QI training. The second phase of the study examined the potential effectiveness of AD as an implementation strategy for DecisionPrecision at all 8 medical centers. Outcomes were assessed by comparing absolute tool use before and after AD visits and conducting semistructured interviews with a subset of primary care physicians (PCPs) following the AD visits. Results: Phase 1 findings showed that sites that participated in the LEAP QI training program used DecisionPrecision significantly more often than the standard implementation sites (tool used 190.3, SD 174.8 times on average over 6 months at LEAP sites vs 3.5 SD 3.7 at standard sites; P<.001). However, this finding was confounded by the lack of screening coordinators at standard implementation sites. In phase 2, there was no difference in the 6-month tool use between before and after AD (95% CI −5.06 to 6.40; P=.82). Follow-up interviews with PCPs indicated that the AD strategy increased provider awareness and appreciation for the benefits of the tool. However, other priorities and limited time prevented PCPs from using them during routine clinical visits. Conclusions: The phase 1 findings did not provide conclusive evidence of the benefit of a QI training approach for implementing a decision support tool for LCS among PCPs. In addition, phase 2 findings showed that our light-touch, single-visit AD strategy did not increase tool use. To enable tool use by PCPs, prediction-based tools must be fully automated and integrated into electronic health records, thereby helping providers personalize LCS discussions among their many competing demands. PCPs also need more time to engage in shared decision-making discussions with their patients. Trial Registration: ClinicalTrials.gov NCT02765412; https://clinicaltrials.gov/ct2/show/NCT02765412 %M 35363144 %R 10.2196/32399 %U https://humanfactors.jmir.org/2022/2/e32399 %U https://doi.org/10.2196/32399 %U http://www.ncbi.nlm.nih.gov/pubmed/35363144 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 2 %P e27924 %T Chinese Americans’ Use of Patient Portal Systems: Scoping Review %A Lawrence,Katharine %A Chong,Stella %A Krelle,Holly %A Roberts,Timothy %A Thorpe,Lorna %A Trinh-Shevrin,Chau %A Yi,Stella %A Kwon,Simona %+ Healthcare Innovation Bridging Research, Informatics, and Design (HiBRID) Lab, Department of Population Health, New York University Grossman School of Medicine, 227 E 30th St, 6th Floor, New York, NY, 10016, United States, 1 6465013488, katharine.lawrence@nyulangone.org %K patient portal %K electronic health records %K personal health records %K ehealth %K health equity %K digital divide %K Chinese Americans %K Asian Americans %D 2022 %7 1.4.2022 %9 Review %J JMIR Hum Factors %G English %X Background: Electronic patient portals are increasingly used in health care systems as communication and information-sharing tools and show promise in addressing health care access, quality, and outcomes. However, limited research exists on portal use patterns and practices among diverse patient populations, resulting in the lack of culturally and contextually tailored portal systems for these patients. Objective: This study aimed to summarize existing evidence on the access and use patterns, barriers, and facilitators of patient portals among Chinese Americans, who represent a growing patient population in the United States with unique health care and health technology needs. Methods: The authors conducted a literature search using the PRISMA Protocol for Scoping Reviews (Preferred Reporting Items for Systematic Reviews and Meta-Analyses-ScR) for extracting articles published in major databases (MEDLINE, Embase, and PsycINFO) on patient portals and Chinese Americans. Authors independently reviewed the papers during initial screening and full-text review. The studies were analyzed and coded for the study method type, sample population, and main outcomes of interest. Results: In total, 17 articles were selected for inclusion in the review. The included articles were heterogenous and varied in their study aims, methodologies, sample populations, and outcomes. Major findings identified from the articles include variable patterns of portal access and use among Chinese Americans compared to other racial or ethnic groups, with limited evidence on the specific barriers and facilitators for this group; a preference for cross-sectional quantitative tools such as patient surveys and electronic health record–based data over qualitative or other methodologies; and a pattern of aggregating Chinese American–related data into a larger Asian or Asian American designation. Conclusions: There is limited research evaluating the use patterns, experiences, and needs of Chinese Americans who access and use patient portal systems. Existing research is heterogeneous, largely cross-sectional, and does not disaggregate Chinese Americans from larger Asian demographics. Future research should be devoted to the specific portal use patterns, preferences, and needs of Chinese Americans to help ensure contextually appropriate and acceptable design and implementation of these digital health tools. %M 35363153 %R 10.2196/27924 %U https://humanfactors.jmir.org/2022/2/e27924 %U https://doi.org/10.2196/27924 %U http://www.ncbi.nlm.nih.gov/pubmed/35363153 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 2 %P e33630 %T Evaluation of the First Year(s) of Physicians Collaboration on an Interdisciplinary Electronic Consultation Platform in the Netherlands: Mixed Methods Observational Study %A Sanavro,Sanne M %A van der Worp,Henk %A Jansen,Danielle %A Koning,Paul %A Blanker,Marco H %A , %+ Department of General Practice and Elderly Care Medicine, University of Groningen, PO Box 196, Groningen, 9700 AD, Netherlands, 31 50 3616731, sannesanavro@hotmail.com %K primary care %K digital consultation %K interdisciplinary %K specialist care %D 2022 %7 1.4.2022 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Complexity of health problems and aging of the population create an ongoing burden on the health care system with the general practitioner (GP) being the gatekeeper in primary care. In GPs daily practice, collaboration with specialists and exchange of knowledge from the secondary care play a crucial role in this system. Communication between primary and secondary care has shortcomings for health care workers that want to practice sustainable patient-centered health care. Therefore, a new digital interactive platform was developed: Prisma. Objective: This study aims to describe the development of a digital consultation platform (Prisma) to connect GPs with hospital specialists via the Siilo application and to evaluate the first year of use, including consultations, topic diversity, and number of participating physicians. Methods: We conducted a mixed methods observational study, analyzing qualitative and quantitative data for cases posted on the platform between June 2018 and May 2020. Any GP can post questions to an interdisciplinary group of secondary care specialists, with the platform designed to facilitate discussion and knowledge exchange for all users. Results: In total, 3674 cases were posted by 424 GPs across 16 specialisms. Most questions and answers concerned diagnosis, nonmedical treatment, and medication. Mean response time was 76 minutes (range 44-252). An average of 3 users engaged with each case (up to 7 specialists). Almost half of the internal medicine cases received responses from at least two specialisms in secondary care, contrasting with about one-fifth for dermatology. Of note, the growth in consultations was steepest for dermatology. Conclusions: Digital consultations offer the possibility for GPs to receive quick responses when seeking advice. The interdisciplinary approach of Prisma creates opportunities for digital patient-centered networking. %M 35363155 %R 10.2196/33630 %U https://humanfactors.jmir.org/2022/2/e33630 %U https://doi.org/10.2196/33630 %U http://www.ncbi.nlm.nih.gov/pubmed/35363155 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 2 %P e26825 %T Proposal for Post Hoc Quality Control in Instrumented Motion Analysis Using Markerless Motion Capture: Development and Usability Study %A Röhling,Hanna Marie %A Althoff,Patrik %A Arsenova,Radina %A Drebinger,Daniel %A Gigengack,Norman %A Chorschew,Anna %A Kroneberg,Daniel %A Rönnefarth,Maria %A Ellermeyer,Tobias %A Rosenkranz,Sina Cathérine %A Heesen,Christoph %A Behnia,Behnoush %A Hirano,Shigeki %A Kuwabara,Satoshi %A Paul,Friedemann %A Brandt,Alexander Ulrich %A Schmitz-Hübsch,Tanja %+ Experimental and Clinical Research Center, a cooperation between the Max-Delbrück-Center for Molecular Medicine in the Helmholtz Association and the Charité - Universitätsmedizin Berlin, Lindenberger Weg 80, Berlin, 13125, Germany, 49 30 450539718, hanna-marie.roehling@charite.de %K instrumented motion analysis %K markerless motion capture %K visual perceptive computing %K quality control %K quality reporting %K gait analysis %D 2022 %7 1.4.2022 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Instrumented assessment of motor symptoms has emerged as a promising extension to the clinical assessment of several movement disorders. The use of mobile and inexpensive technologies such as some markerless motion capture technologies is especially promising for large-scale application but has not transitioned into clinical routine to date. A crucial step on this path is to implement standardized, clinically applicable tools that identify and control for quality concerns. Objective: The main goal of this study comprises the development of a systematic quality control (QC) procedure for data collected with markerless motion capture technology and its experimental implementation to identify specific quality concerns and thereby rate the usability of recordings. Methods: We developed a post hoc QC pipeline that was evaluated using a large set of short motor task recordings of healthy controls (2010 recordings from 162 subjects) and people with multiple sclerosis (2682 recordings from 187 subjects). For each of these recordings, 2 raters independently applied the pipeline. They provided overall usability decisions and identified technical and performance-related quality concerns, which yielded respective proportions of their occurrence as a main result. Results: The approach developed here has proven user-friendly and applicable on a large scale. Raters’ decisions on recording usability were concordant in 71.5%-92.3% of cases, depending on the motor task. Furthermore, 39.6%-85.1% of recordings were concordantly rated as being of satisfactory quality whereas in 5.0%-26.3%, both raters agreed to discard the recording. Conclusions: We present a QC pipeline that seems feasible and useful for instant quality screening in the clinical setting. Results confirm the need of QC despite using standard test setups, testing protocols, and operator training for the employed system and by extension, for other task-based motor assessment technologies. Results of the QC process can be used to clean existing data sets, optimize quality assurance measures, as well as foster the development of automated QC approaches and therefore improve the overall reliability of kinematic data sets. %M 35363150 %R 10.2196/26825 %U https://humanfactors.jmir.org/2022/2/e26825 %U https://doi.org/10.2196/26825 %U http://www.ncbi.nlm.nih.gov/pubmed/35363150 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 1 %P e33325 %T Designing a Novel Clinician Decision Support Tool for the Management of Acute Diarrhea in Bangladesh: Formative Qualitative Study %A Rosen,Rochelle K %A Garbern,Stephanie C %A Gainey,Monique %A Lantini,Ryan %A Nasrin,Sabiha %A Nelson,Eric J %A Elshabassi,Nour %A Alam,Nur H %A Sultana,Sufia %A Hasnin,Tahmida %A Qu,Kexin %A Schmid,Christopher H %A Levine,Adam C %+ Center for Behavioral and Preventive Medicine, The Miriam Hospital, 164 Summit St, Coro Building Suite 134, Providence, RI, 02906, United States, 1 401 793 8182, rrosen@lifespan.org %K clinical decision support tools %K diarrhea management %K focus group %K formative qualitative research %K low- and middle-income countries %K mobile phone %D 2022 %7 25.3.2022 %9 Original Paper %J JMIR Hum Factors %G English %X Background: The availability of mobile clinical decision support (CDS) tools has grown substantially with the increased prevalence of smartphone devices and apps. Although health care providers express interest in integrating mobile health (mHealth) technologies into their clinical settings, concerns have been raised, including perceived disagreements between information provided by mobile CDS tools and standard guidelines. Despite their potential to transform health care delivery, there remains limited literature on the provider’s perspective on the clinical utility of mobile CDS tools for improving patient outcomes, especially in low- and middle-income countries. Objective: This study aims to describe providers’ perceptions about the utility of a mobile CDS tool accessed via a smartphone app for diarrhea management in Bangladesh. In addition, feedback was collected on the preliminary components of the mobile CDS tool to address clinicians’ concerns and incorporate their preferences. Methods: From November to December 2020, qualitative data were gathered through 8 web-based focus group discussions with physicians and nurses from 3 Bangladeshi hospitals. Each discussion was conducted in the local language—Bangla—and audio recorded for transcription and translation by the local research team. Transcripts and codes were entered into NVivo (version 12; QSR International), and applied thematic analysis was used to identify themes that explore the clinical utility of an mHealth app for assessing dehydration severity in patients with acute diarrhea. Summaries of concepts and themes were generated from reviews of the aggregated coded data; thematic memos were written and used for the final analysis. Results: Of the 27 focus group participants, 14 (52%) were nurses and 13 (48%) were physicians; 15 (56%) worked at a diarrhea specialty hospital and 12 (44%) worked in government district or subdistrict hospitals. Participants’ experience in their current position ranged from 2 to 14 years, with an average of 10.3 (SD 9.0) years. Key themes from the qualitative data analysis included current experience with CDS, overall perception of the app’s utility and its potential role in clinical care, barriers to and facilitators of app use, considerations of overtreatment and undertreatment, and guidelines for the app’s clinical recommendations. Participants felt that the tool would initially take time to use, but once learned, it could be useful during epidemic cholera. Some felt that clinical experience remains an important part of treatment that can be supplemented, but not replaced, by a CDS tool. In addition, diagnostic information, including mid-upper arm circumference and blood pressure, might not be available to directly inform programming decisions. Conclusions: Participants were positive about the mHealth app and its potential to inform diarrhea management. They provided detailed feedback, which developers used to revise the mobile CDS tool. These formative qualitative data provided timely and relevant feedback to improve the utility of a CDS tool for diarrhea treatment in Bangladesh. %M 35333190 %R 10.2196/33325 %U https://humanfactors.jmir.org/2022/1/e33325 %U https://doi.org/10.2196/33325 %U http://www.ncbi.nlm.nih.gov/pubmed/35333190 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 1 %P e28639 %T Human Factors and Technological Characteristics Influencing the Interaction of Medical Professionals With Artificial Intelligence–Enabled Clinical Decision Support Systems: Literature Review %A Knop,Michael %A Weber,Sebastian %A Mueller,Marius %A Niehaves,Bjoern %+ Department of Information Systems, University of Siegen, Kohlbettstrasse 15, Siegen, 57072, Germany, 49 15755910502, michael.knop@uni-siegen.de %K artificial intelligence %K clinical decision support systems %K CDSS %K decision-making %K diagnostic decision support %K human–computer interaction %K human–AI collaboration %K machine learning %K patient outcomes %K deep learning %K trust %K literature review %D 2022 %7 24.3.2022 %9 Review %J JMIR Hum Factors %G English %X Background: The digitization and automation of diagnostics and treatments promise to alter the quality of health care and improve patient outcomes, whereas the undersupply of medical personnel, high workload on medical professionals, and medical case complexity increase. Clinical decision support systems (CDSSs) have been proven to help medical professionals in their everyday work through their ability to process vast amounts of patient information. However, comprehensive adoption is partially disrupted by specific technological and personal characteristics. With the rise of artificial intelligence (AI), CDSSs have become an adaptive technology with human-like capabilities and are able to learn and change their characteristics over time. However, research has not reflected on the characteristics and factors essential for effective collaboration between human actors and AI-enabled CDSSs. Objective: Our study aims to summarize the factors influencing effective collaboration between medical professionals and AI-enabled CDSSs. These factors are essential for medical professionals, management, and technology designers to reflect on the adoption, implementation, and development of an AI-enabled CDSS. Methods: We conducted a literature review including 3 different meta-databases, screening over 1000 articles and including 101 articles for full-text assessment. Of the 101 articles, 7 (6.9%) met our inclusion criteria and were analyzed for our synthesis. Results: We identified the technological characteristics and human factors that appear to have an essential effect on the collaboration of medical professionals and AI-enabled CDSSs in accordance with our research objective, namely, training data quality, performance, explainability, adaptability, medical expertise, technological expertise, personality, cognitive biases, and trust. Comparing our results with those from research on non-AI CDSSs, some characteristics and factors retain their importance, whereas others gain or lose relevance owing to the uniqueness of human-AI interactions. However, only a few (1/7, 14%) studies have mentioned the theoretical foundations and patient outcomes related to AI-enabled CDSSs. Conclusions: Our study provides a comprehensive overview of the relevant characteristics and factors that influence the interaction and collaboration between medical professionals and AI-enabled CDSSs. Rather limited theoretical foundations currently hinder the possibility of creating adequate concepts and models to explain and predict the interrelations between these characteristics and factors. For an appropriate evaluation of the human-AI collaboration, patient outcomes and the role of patients in the decision-making process should be considered. %M 35323118 %R 10.2196/28639 %U https://humanfactors.jmir.org/2022/1/e28639 %U https://doi.org/10.2196/28639 %U http://www.ncbi.nlm.nih.gov/pubmed/35323118 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 1 %P e30130 %T A Patient Outcomes–Driven Feedback Platform for Emergency Medicine Clinicians: Human-Centered Design and Usability Evaluation of Linking Outcomes Of Patients (LOOP) %A Strauss,Alexandra T %A Morgan,Cameron %A El Khuri,Christopher %A Slogeris,Becky %A Smith,Aria G %A Klein,Eili %A Toerper,Matt %A DeAngelo,Anthony %A Debraine,Arnaud %A Peterson,Susan %A Gurses,Ayse P %A Levin,Scott %A Hinson,Jeremiah %+ Department of Medicine, Johns Hopkins University School of Medicine, 600 N Wolfe St, Block 465, Baltimore, MD, 21287, United States, 1 6098280943, atstrauss13@gmail.com %K emergency medicine %K usability %K human-centered design %K health informatics %K feedback %K practice-based learning and improvement %K emergency room %K ER %K platform %K outcomes %K closed-loop learning %D 2022 %7 23.3.2022 %9 Original Paper %J JMIR Hum Factors %G English %X Background: The availability of patient outcomes–based feedback is limited in episodic care environments such as the emergency department. Emergency medicine (EM) clinicians set care trajectories for a majority of hospitalized patients and provide definitive care to an even larger number of those discharged into the community. EM clinicians are often unaware of the short- and long-term health outcomes of patients and how their actions may have contributed. Despite large volumes of patients and data, outcomes-driven learning that targets individual clinician experiences is meager. Integrated electronic health record (EHR) systems provide opportunity, but they do not have readily available functionality intended for outcomes-based learning. Objective: This study sought to unlock insights from routinely collected EHR data through the development of an individualizable patient outcomes feedback platform for EM clinicians. Here, we describe the iterative development of this platform, Linking Outcomes Of Patients (LOOP), under a human-centered design framework, including structured feedback obtained from its use. Methods: This multimodal study consisting of human-centered design studios, surveys (24 physicians), interviews (11 physicians), and a LOOP application usability evaluation (12 EM physicians for ≥30 minutes each) was performed between August 2019 and February 2021. The study spanned 3 phases: (1) conceptual development under a human-centered design framework, (2) LOOP technical platform development, and (3) usability evaluation comparing pre- and post-LOOP feedback gathering practices in the EHR. Results: An initial human-centered design studio and EM clinician surveys revealed common themes of disconnect between EM clinicians and their patients after the encounter. Fundamental postencounter outcomes of death (15/24, 63% respondents identified as useful), escalation of care (20/24, 83%), and return to ED (16/24, 67%) were determined high yield for demonstrating proof-of-concept in our LOOP application. The studio aided the design and development of LOOP, which integrated physicians throughout the design and content iteration. A final LOOP prototype enabled usability evaluation and iterative refinement prior to launch. Usability evaluation compared to status quo (ie, pre-LOOP) feedback gathering practices demonstrated a shift across all outcomes from “not easy” to “very easy” to obtain and from “not confident” to “very confident” in estimating outcomes after using LOOP. On a scale from 0 (unlikely) to 10 (most likely), the users were very likely (9.5) to recommend LOOP to a colleague. Conclusions: This study demonstrates the potential for human-centered design of a patient outcomes–driven feedback platform for individual EM providers. We have outlined a framework for working alongside clinicians with a multidisciplined team to develop and test a tool that augments their clinical experience and enables closed-loop learning. %M 35319469 %R 10.2196/30130 %U https://humanfactors.jmir.org/2022/1/e30130 %U https://doi.org/10.2196/30130 %U http://www.ncbi.nlm.nih.gov/pubmed/35319469 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 1 %P e29019 %T Barriers to and Facilitators for Acceptance of Comprehensive Clinical Decision Support System–Driven Care Maps for Patients With Thoracic Trauma: Interview Study Among Health Care Providers and Nurses %A Jones,Emma K %A Banks,Alyssa %A Melton,Genevieve B %A Porta,Carolyn M %A Tignanelli,Christopher J %+ Department of Surgery, University of Minnesota, 420 Delaware St SE, Mayo Mail Code 195, Minneapolis, MN, 55455, United States, 1 6126261968, ctignane@umn.edu %K clinical decision support systems %K rib fractures %K trauma %K Unified Theory of Acceptance and Use of Technology %K human computer interaction %D 2022 %7 16.3.2022 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Comprehensive clinical decision support (CDS) care maps can improve the delivery of care and clinical outcomes. However, they are frequently plagued by usability problems and poor user acceptance. Objective: This study aims to characterize factors influencing successful design and use of comprehensive CDS care maps and identify themes associated with end-user acceptance of a thoracic trauma CDS care map earlier in the process than has traditionally been done. This was a planned adaptive redesign stage of a User Acceptance and System Adaptation Design development and implementation strategy for a CDS care map. This stage was based on a previously developed prototype CDS care map guided by the Unified Theory of Acceptance and Use of Technology. Methods: A total of 22 multidisciplinary end users (physicians, advanced practice providers, and nurses) were identified and recruited using snowball sampling. Qualitative interviews were conducted, audio-recorded, and transcribed verbatim. Generation of prespecified codes and the interview guide was informed by the Unified Theory of Acceptance and Use of Technology constructs and investigative team experience. Interviews were blinded and double-coded. Thematic analysis of interview scripts was conducted and yielded descriptive themes about factors influencing the construction and potential use of an acceptable CDS care map. Results: A total of eight dominant themes were identified: alert fatigue (theme 1), automation (theme 2), redundancy (theme 3), minimalistic design (theme 4), evidence based (theme 5), prevent errors (theme 6), comprehensive across the spectrum of disease (theme 7), and malleability (theme 8). Themes 1 to 4 addressed factors directly affecting end users, and themes 5 to 8 addressed factors affecting patient outcomes. More experienced providers prioritized a system that is easy to use. Nurses prioritized a system that incorporated evidence into decision support. Clinicians across specialties, roles, and ages agreed that the amount of extra work generated should be minimal and that the system should help them administer optimal care efficiently. Conclusions: End user feedback reinforces attention toward factors that improve the acceptance and use of a CDS care map for patients with thoracic trauma. Common themes focused on system complexity, the ability of the system to fit different populations and settings, and optimal care provision. Identifying these factors early in the development and implementation process may facilitate user-centered design and improve adoption. %M 35293873 %R 10.2196/29019 %U https://humanfactors.jmir.org/2022/1/e29019 %U https://doi.org/10.2196/29019 %U http://www.ncbi.nlm.nih.gov/pubmed/35293873 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 1 %P e24680 %T Acceptance of the Use of Artificial Intelligence in Medicine Among Japan’s Doctors and the Public: A Questionnaire Survey %A Tamori,Honoka %A Yamashina,Hiroko %A Mukai,Masami %A Morii,Yasuhiro %A Suzuki,Teppei %A Ogasawara,Katsuhiko %+ Faculty of Health Sciences, Hokkaido University, N12-W5, Kita-ku, Sapporo, 0600812, Japan, 81 11 706 3409, oga@hs.hokudai.ac.jp %K artificial intelligence %K technology acceptance %K surveys and questionnaires %K doctors vs public %D 2022 %7 16.3.2022 %9 Original Paper %J JMIR Hum Factors %G English %X Background: The use of artificial intelligence (AI) in the medical industry promises many benefits, so AI has been introduced to medical practice primarily in developed countries. In Japan, the government is preparing for the rollout of AI in the medical industry. This rollout depends on doctors and the public accepting the technology. Therefore it is necessary to consider acceptance among doctors and among the public. However, little is known about the acceptance of AI in medicine in Japan. Objective: This study aimed to obtain detailed data on the acceptance of AI in medicine by comparing the acceptance among Japanese doctors with that among the Japanese public. Methods: We conducted an online survey, and the responses of doctors and members of the public were compared. AI in medicine was defined as the use of AI to determine diagnosis and treatment without requiring a doctor. A questionnaire was prepared referred to as the unified theory of acceptance and use of technology, a model of behavior toward new technologies. It comprises 20 items, and each item was rated on a five-point scale. Using this questionnaire, we conducted an online survey in 2018 among 399 doctors and 600 members of the public. The sample-wide responses were analyzed, and then the responses of the doctors were compared with those of the public using t tests. Results: Regarding the sample-wide responses (N=999), 653 (65.4%) of the respondents believed, in the future, AI in medicine would be necessary, whereas only 447 (44.7%) expressed an intention to use AI-driven medicine. Additionally, 730 (73.1%) believed that regulatory legislation was necessary, and 734 (73.5%) were concerned about where accountability lies. Regarding the comparison between doctors and the public, doctors (mean 3.43, SD 1.00) were more likely than members of the public (mean 3.23, SD 0.92) to express intention to use AI-driven medicine (P<.001), suggesting that optimism about AI in medicine is greater among doctors compared to the public. Conclusions: Many of the respondents were optimistic about the role of AI in medicine. However, when asked whether they would like to use AI-driven medicine, they tended to give a negative response. This trend suggests that concerns about the lack of regulation and about accountability hindered acceptance. Additionally, the results revealed that doctors were more enthusiastic than members of the public regarding AI-driven medicine. For the successful implementation of AI in medicine, it would be necessary to inform the public and doctors about the relevant laws and to take measures to remove their concerns about them. %M 35293878 %R 10.2196/24680 %U https://humanfactors.jmir.org/2022/1/e24680 %U https://doi.org/10.2196/24680 %U http://www.ncbi.nlm.nih.gov/pubmed/35293878 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 1 %P e31021 %T Concept Libraries for Repeatable and Reusable Research: Qualitative Study Exploring the Needs of Users %A Almowil,Zahra %A Zhou,Shang-Ming %A Brophy,Sinead %A Croxall,Jodie %+ Data Science Building, Medical School, Swansea University, Sketty, Swansea, Wales, SA2 8PP, United Kingdom, 44 07552894384, 934467@swansea.ac.uk %K electronic health records %K record linkage %K reproducible research %K clinical codes %K concept libraries %D 2022 %7 15.3.2022 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Big data research in the field of health sciences is hindered by a lack of agreement on how to identify and define different conditions and their medications. This means that researchers and health professionals often have different phenotype definitions for the same condition. This lack of agreement makes it difficult to compare different study findings and hinders the ability to conduct repeatable and reusable research. Objective: This study aims to examine the requirements of various users, such as researchers, clinicians, machine learning experts, and managers, in the development of a data portal for phenotypes (a concept library). Methods: This was a qualitative study using interviews and focus group discussion. One-to-one interviews were conducted with researchers, clinicians, machine learning experts, and senior research managers in health data science (N=6) to explore their specific needs in the development of a concept library. In addition, a focus group discussion with researchers (N=14) working with the Secured Anonymized Information Linkage databank, a national eHealth data linkage infrastructure, was held to perform a SWOT (strengths, weaknesses, opportunities, and threats) analysis for the phenotyping system and the proposed concept library. The interviews and focus group discussion were transcribed verbatim, and 2 thematic analyses were performed. Results: Most of the participants thought that the prototype concept library would be a very helpful resource for conducting repeatable research, but they specified that many requirements are needed before its development. Although all the participants stated that they were aware of some existing concept libraries, most of them expressed negative perceptions about them. The participants mentioned several facilitators that would stimulate them to share their work and reuse the work of others, and they pointed out several barriers that could inhibit them from sharing their work and reusing the work of others. The participants suggested some developments that they would like to see to improve reproducible research output using routine data. Conclusions: The study indicated that most interviewees valued a concept library for phenotypes. However, only half of the participants felt that they would contribute by providing definitions for the concept library, and they reported many barriers regarding sharing their work on a publicly accessible platform. Analysis of interviews and the focus group discussion revealed that different stakeholders have different requirements, facilitators, barriers, and concerns about a prototype concept library. %M 35289755 %R 10.2196/31021 %U https://humanfactors.jmir.org/2022/1/e31021 %U https://doi.org/10.2196/31021 %U http://www.ncbi.nlm.nih.gov/pubmed/35289755 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 1 %P e24172 %T Personas for Better Targeted eHealth Technologies: User-Centered Design Approach %A ten Klooster,Iris %A Wentzel,Jobke %A Sieverink,Floor %A Linssen,Gerard %A Wesselink,Robin %A van Gemert-Pijnen,Lisette %+ Faculty of Behavioural, Management and Social Sciences, University of Twente, Drienerlolaan 5, Enschede, 7522 NB, Netherlands, 31 620730557, i.tenklooster@utwente.nl %K personas %K clustering %K heart failure %K eHealth %K user-centered design %D 2022 %7 15.3.2022 %9 Original Paper %J JMIR Hum Factors %G English %X Background: The full potential of eHealth technologies to support self-management and disease management for patients with chronic diseases is not being reached. A possible explanation for these lacking results is that during the development process, insufficient attention is paid to the needs, wishes, and context of the prospective end users. To overcome such issues, the user-centered design practice of creating personas is widely accepted to ensure the fit between a technology and the target group or end users throughout all phases of development. Objective: In this study, we integrate several approaches to persona development into the Persona Approach Twente to attain a more holistic and structured approach that aligns with the iterative process of eHealth development. Methods: In 3 steps, a secondary analysis was carried out on different parts of the data set using the Partitioning Around Medoids clustering method. First, we used health-related electronic patient record data only. Second, we added person-related data that were gathered through interviews and questionnaires. Third, we added log data. Results: In the first step, 2 clusters were found, with average silhouette widths of 0.12 and 0.27. In the second step, again 2 clusters were found, with average silhouette widths of 0.08 and 0.12. In the third step, 3 clusters were identified, with average silhouette widths of 0.09, 0.12, and 0.04. Conclusions: The Persona Approach Twente is applicable for mixed types of data and allows alignment of this user-centered design method to the iterative approach of eHealth development. A variety of characteristics can be used that stretches beyond (standardized) medical and demographic measurements. Challenges lie in data quality and fitness for (quantitative) clustering. %M 35289759 %R 10.2196/24172 %U https://humanfactors.jmir.org/2022/1/e24172 %U https://doi.org/10.2196/24172 %U http://www.ncbi.nlm.nih.gov/pubmed/35289759 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 1 %P e30655 %T A Remote Patient-Monitoring System for Intensive Care Medicine: Mixed Methods Human-Centered Design and Usability Evaluation %A Poncette,Akira-Sebastian %A Mosch,Lina Katharina %A Stablo,Lars %A Spies,Claudia %A Schieler,Monique %A Weber-Carstens,Steffen %A Feufel,Markus A %A Balzer,Felix %+ Institute of Medical Informatics, Charité – Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Charitéplatz 1, Berlin, 10117, Germany, 49 30450 ext 651166, felix.balzer@charite.de %K digital health %K patient monitoring %K intensive care medicine %K intensive care unit %K technological innovation %K user-centered design %K usability %K user experience %K implementation science %K qualitative research %K interview %K mixed methods %K mobile phone %D 2022 %7 11.3.2022 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Continuous monitoring of vital signs is critical for ensuring patient safety in intensive care units (ICUs) and is becoming increasingly relevant in general wards. The effectiveness of health information technologies such as patient-monitoring systems is highly determined by usability, the lack of which can ultimately compromise patient safety. Usability problems can be identified and prevented by involving users (ie, clinicians). Objective: In this study, we aim to apply a human-centered design approach to evaluate the usability of a remote patient-monitoring system user interface (UI) in the ICU context and conceptualize and evaluate design changes. Methods: Following institutional review board approval (EA1/031/18), a formative evaluation of the monitoring UI was performed. Simulated use tests with think-aloud protocols were conducted with ICU staff (n=5), and the resulting qualitative data were analyzed using a deductive analytic approach. On the basis of the identified usability problems, we conceptualized informed design changes and applied them to develop an improved prototype of the monitoring UI. Comparing the UIs, we evaluated perceived usability using the System Usability Scale, performance efficiency with the normative path deviation, and effectiveness by measuring the task completion rate (n=5). Measures were tested for statistical significance using a 2-sample t test, Poisson regression with a generalized linear mixed-effects model, and the N-1 chi-square test. P<.05 were considered significant. Results: We found 37 individual usability problems specific to monitoring UI, which could be assigned to six subcodes: usefulness of the system, response time, responsiveness, meaning of labels, function of UI elements, and navigation. Among user ideas and requirements for the UI were high usability, customizability, and the provision of audible alarm notifications. Changes in graphics and design were proposed to allow for better navigation, information retrieval, and spatial orientation. The UI was revised by creating a prototype with a more responsive design and changes regarding labeling and UI elements. Statistical analysis showed that perceived usability improved significantly (System Usability Scale design A: mean 68.5, SD 11.26, n=5; design B: mean 89, SD 4.87, n=5; P=.003), as did performance efficiency (normative path deviation design A: mean 8.8, SD 5.26, n=5; design B: mean 3.2, SD 3.03, n=5; P=.001), and effectiveness (design A: 18 trials, failed 7, 39% times, passed 11, 61% times; design B: 20 trials, failed 0 times, passed 20 times; P=.002). Conclusions: Usability testing with think-aloud protocols led to a patient-monitoring UI with significantly improved usability, performance, and effectiveness. In the ICU work environment, difficult-to-use technology may result in detrimental outcomes for staff and patients. Technical devices should be designed to support efficient and effective work processes. Our results suggest that this can be achieved by applying basic human-centered design methods and principles. Trial Registration: ClinicalTrials.gov NCT03514173; https://clinicaltrials.gov/ct2/show/NCT03514173 %M 35275071 %R 10.2196/30655 %U https://humanfactors.jmir.org/2022/1/e30655 %U https://doi.org/10.2196/30655 %U http://www.ncbi.nlm.nih.gov/pubmed/35275071 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 1 %P e30797 %T User-Centered Development and Testing of the Online Patient-Reported Outcomes, Burdens, and Experiences (PROBE) Survey and the myPROBE App and Integration With the Canadian Bleeding Disorder Registry: Mixed Methods Study %A Germini,Federico %A Borg Debono,Victoria %A Page,David %A Zuk,Victoria %A Kucher,Alexandra %A Cotoi,Chris %A Hobson,Nicholas %A Sevestre,Michael %A Skinner,Mark W %A Iorio,Alfonso %A , %+ Department of Health Research Methods, Evidence, and Impact, McMaster University, 1280 Main Street West, 2C Area, Hamilton, ON, L8S 4K1, Canada, 1 905 525 9140 ext 26771, germinif@mcmaster.ca %K health-related quality of life %K EQ-5D %K mobile app %K Patient-Reported Outcomes, Burdens, and Experiences (PROBE) %K hemophilia %K mobile health %K mHealth %K eHealth %K telehealth %K user-centered design %D 2022 %7 2.3.2022 %9 Original Paper %J JMIR Hum Factors %G English %X Background: The Patient-Reported Outcomes, Burdens, and Experiences (PROBE) questionnaire is a tool for assessing the quality of life and disease burden in people living with hemophilia. Objective: The objectives of our study were (1) to assess the needs of relevant stakeholders involved in the use of PROBE, (2) to develop the software infrastructure needed to meet these needs, and (3) to test the usability of the final product. Methods: We conducted a series of semistructured interviews of relevant stakeholders, including PROBE investigators, people with hemophilia, and representatives of the sponsor. Based on these, we developed an online survey and a mobile app for iOS and Android. A user group evaluated the final product using the System Usability Scale (SUS) and an open feedback framework. Results: The online survey was updated, and the myPROBE app for mobile devices and a new application programming interface were developed. The app was tested and modified according to user feedback over multiple cycles. The final version of the app was released in July 2019. Seventeen users aged 23 to 67 years evaluated the final version of the app using the SUS. The median (first, third quartile) SUS score for the app was 85 (68, 88) out of 100. The newly introduced functionalities were as follows: (1) capability to longitudinally track repeated fillings of the questionnaire at different time points by the same participant (as opposed to anonymous completion); (2) linking of the questionnaire with hemophilia registries, starting with the Canadian Bleeding Disorders Registry as a proof of concept; (3) removing or adding questions as needed; and (4) sending notifications to the users (eg, reminders). A new secure database was built for securely storing personal information separately from the questionnaire data. The PROBE online survey is currently available in 96 countries and 34 languages. Conclusions: The online survey was updated successfully, and the myPROBE app was developed, with a SUS score of 85 (out of 100). The app has been released in 81 countries and 34 languages. This will facilitate data collection for research and advocacy purposes, and the use of this tool in everyday clinical practice. %M 35234648 %R 10.2196/30797 %U https://humanfactors.jmir.org/2022/1/e30797 %U https://doi.org/10.2196/30797 %U http://www.ncbi.nlm.nih.gov/pubmed/35234648 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 6 %N 2 %P e30527 %T Embedding and Integrating a Digital Patient Management Platform Into Everyday Primary Care Routines: Qualitative Case Study %A Frennert,Susanne %A Erlingsdóttir,Gudbjörg %A Muhic,Mirella %A Rydenfält,Christofer %A Milos Nymberg,Veronica %A Ekman,Björn %+ Department of Design Sciences, Lund University, Ingvar Kamprad Designcentrum, Sölvegatan 26, Lund, 223 62, Sweden, 46 2220195, susanne.frennert@design.lth.se %K digital patient management platform %K primary care %K normalization process theory coherence %K cognitive participation %K collective action %K reflexive monitoring %D 2022 %7 22.2.2022 %9 Original Paper %J JMIR Form Res %G English %X Background: Traditional primary care is characterized by patient consultations via phone and physical visits. However, the current development in Swedish primary care is to blend digital solutions with traditional solutions. This paper addresses this development by examining the normalization of embedding and integrating a digital health care platform into everyday care routines in a primary care clinic. The digital health care platform enables both synchronous (video calls) and asynchronous (chat) communication, as well as self-registration of patient data using automated questions and forms requiring the patient’s input. Objective: This study aims to explore the work that health care professionals (HCPs) have to undertake to implement and sustain a digital health care platform as part of their everyday work practice. Methods: HCPs were observed and interviewed to assess their individual and collective engagement and the mechanisms involved in the implementation of the digital platform and its effects on everyday work routines. The normalization process theory (NPT) was used to frame the data analysis. Results: The analysis identified several themes related to the four NPT constructs: coherence, cognitive participation, collective action, and reflexive monitoring. The use of these constructs enabled the analysis to identify ways of supporting implementation. For example, it showed the benefits of having implementation champions and scheduling work hours for HCPs to use the platform. The analysis also revealed a theme of materiality that deviated from the NPT constructs, as NPT gives ontological priority to human actors and social structures. Conclusions: Digital health care platform implementation is a complex process. Our findings provide insights into how individual and collective actions can be supported to embed and integrate a digital platform into everyday care routines. Primary health care organizations need to involve HCPs throughout the implementation process by reorganizing work and providing frequent feedback loops. HCPs are more likely to engage with and commit to changing practices if they perceive the digital platform to be beneficial compared with the current practice. However, they also need resources (eg, time, training, and continuous support) to put the platform into practice. Patient engagement and appraisal are important elements in implementation. Unless patients are willing to use the platform, there is no motivation for HCPs to embed the digital platform into everyday care practice. %M 35191845 %R 10.2196/30527 %U https://formative.jmir.org/2022/2/e30527 %U https://doi.org/10.2196/30527 %U http://www.ncbi.nlm.nih.gov/pubmed/35191845 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 1 %P e33651 %T Usage and Usability of a National e-Library for Chemotherapy Regimens: Mixed Methods Study %A Fyhr,AnnSofie %A Persson,Johanna %A Ek,Åsa %+ Regional Cancer Centre South, Region Skåne, Medicon Village, Scheeletorget 1, Lund, SE-223 81, Sweden, 46 46 275 23 51, ann-sofie.fyhr@skane.se %K chemotherapy regimens %K user evaluation %K standardization %K patient safety %K chemotherapy %K safety %K usability %K e-library %K medication errors %D 2022 %7 17.2.2022 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Accurate information about chemotherapy drugs and regimens is needed to reduce chemotherapy errors. A national e-library, as a common knowledge source with standardized chemotherapy nomenclature and content, was developed. Since the information in the library is both complex and extensive, it is central that the users can use the resource as intended. Objective: The aim of this study was to evaluate the usage and usability of an extensive e-library for chemotherapy regimens developed to reduce medication errors, support the health care staff in their work, and increase patient safety. Methods: To obtain a comprehensive evaluation, a mixed methods study was performed for a broad view of the usage, including a compilation of subjective views of the users (web survey, spontaneous user feedback, and qualitative interviews), analysis of statistics from the website, and an expert evaluation of the usability of the webpage. Results: Statistics from the website show an average of just over 2500 visits and 870 unique visitors per month. Most visits took place Mondays to Fridays, but there were 5-10 visits per day on weekends. The web survey, with 292 answers, shows that the visitors were mainly physicians and nurses. Almost 80% (224/292) of respondents searched for regimens and 90% (264/292) found what they were looking for and were satisfied with their visit. The expert evaluation shows that the e-library follows many existing design principles, thus providing some useful improvement suggestions. A total of 86 emails were received in 2020 with user feedback, most of which were from nurses. The main part (78%, 67/86) contained a question, and the rest had discovered errors mainly in some regimen. The interviews reveal that most hospitals use a computerized physician order entry system, and they use the e-library in various ways, import XML files, transfer information, or use it as a reference. One hospital without a system uses the administration schedules from the library. Conclusions: The user evaluation indicates that the e-library is used in the intended manner and that the users can interact without problems. Users have different needs depending on their profession and their workplace, and these can be supported. The combination of methods applied ensures that the design and content comply with the users’ needs and serves as feedback for continuous design and learning. With a broad national usage, the e-library can become a source for organizational and national learning and a source for continuous improvement of cancer care in Sweden. %M 35175199 %R 10.2196/33651 %U https://humanfactors.jmir.org/2022/1/e33651 %U https://doi.org/10.2196/33651 %U http://www.ncbi.nlm.nih.gov/pubmed/35175199 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 2 %P e34560 %T Building a Precision Medicine Delivery Platform for Clinics: The University of California, San Francisco, BRIDGE Experience %A Bove,Riley %A Schleimer,Erica %A Sukhanov,Paul %A Gilson,Michael %A Law,Sindy M %A Barnecut,Andrew %A Miller,Bruce L %A Hauser,Stephen L %A Sanders,Stephan J %A Rankin,Katherine P %+ UCSF Weill Institute for Neurosciences, University of California, San Francisco, 1651 4th Street, San Francisco, CA, 94158, United States, 1 415 353 2069, Riley.bove@ucsf.edu %K precision medicine %K clinical implementation %K in silico trials %K clinical dashboard %K precision %K implementation %K dashboard %K design %K experience %K analytic %K tool %K analysis %K decision-making %K real time %K platform %K human-centered design %D 2022 %7 15.2.2022 %9 Viewpoint %J J Med Internet Res %G English %X Despite an ever-expanding number of analytics with the potential to impact clinical care, the field currently lacks point-of-care technological tools that allow clinicians to efficiently select disease-relevant data about their patients, algorithmically derive clinical indices (eg, risk scores), and view these data in straightforward graphical formats to inform real-time clinical decisions. Thus far, solutions to this problem have relied on either bottom-up approaches that are limited to a single clinic or generic top-down approaches that do not address clinical users’ specific setting-relevant or disease-relevant needs. As a road map for developing similar platforms, we describe our experience with building a custom but institution-wide platform that enables economies of time, cost, and expertise. The BRIDGE platform was designed to be modular and scalable and was customized to data types relevant to given clinical contexts within a major university medical center. The development process occurred by using a series of human-centered design phases with extensive, consistent stakeholder input. This institution-wide approach yielded a unified, carefully regulated, cross-specialty clinical research platform that can be launched during a patient’s electronic health record encounter. The platform pulls clinical data from the electronic health record (Epic; Epic Systems) as well as other clinical and research sources in real time; analyzes the combined data to derive clinical indices; and displays them in simple, clinician-designed visual formats specific to each disorder and clinic. By integrating an application into the clinical workflow and allowing clinicians to access data sources that would otherwise be cumbersome to assemble, view, and manipulate, institution-wide platforms represent an alternative approach to achieving the vision of true personalized medicine. %M 35166689 %R 10.2196/34560 %U https://www.jmir.org/2022/2/e34560 %U https://doi.org/10.2196/34560 %U http://www.ncbi.nlm.nih.gov/pubmed/35166689 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 10 %N 2 %P e32695 %T State-of-the-art Dashboards on Clinical Indicator Data to Support Reflection on Practice: Scoping Review %A Bucalon,Bernard %A Shaw,Tim %A Brown,Kerri %A Kay,Judy %+ Human Centred Technology Cluster, School of Computer Science, The University of Sydney, Level 3, Computer Science Building J12, 1 Cleveland St, Darlington, NSW 2008, Australia, 61 2 8627 0010, bernard.bucalon@sydney.edu.au %K practice analytics dashboards %K data visualization %K reflective practice %K professional learning %K mobile phone %D 2022 %7 14.2.2022 %9 Review %J JMIR Med Inform %G English %X Background: There is an increasing interest in using routinely collected eHealth data to support reflective practice and long-term professional learning. Studies have evaluated the impact of dashboards on clinician decision-making, task completion time, user satisfaction, and adherence to clinical guidelines. Objective: This scoping review aims to summarize the literature on dashboards based on patient administrative, medical, and surgical data for clinicians to support reflective practice. Methods: A scoping review was conducted using the Arksey and O’Malley framework. A search was conducted in 5 electronic databases (MEDLINE, Embase, Scopus, ACM Digital Library, and Web of Science) to identify studies that met the inclusion criteria. Study selection and characterization were performed by 2 independent reviewers (BB and CP). One reviewer extracted the data that were analyzed descriptively to map the available evidence. Results: A total of 18 dashboards from 8 countries were assessed. Purposes for the dashboards were designed for performance improvement (10/18, 56%), to support quality and safety initiatives (6/18, 33%), and management and operations (4/18, 22%). Data visualizations were primarily designed for team use (12/18, 67%) rather than individual clinicians (4/18, 22%). Evaluation methods varied among asking the clinicians directly (11/18, 61%), observing user behavior through clinical indicators and use log data (14/18, 78%), and usability testing (4/18, 22%). The studies reported high scores on standard usability questionnaires, favorable surveys, and interview feedback. Improvements to underlying clinical indicators were observed in 78% (7/9) of the studies, whereas 22% (2/9) of the studies reported no significant changes in performance. Conclusions: This scoping review maps the current literature landscape on dashboards based on routinely collected clinical indicator data. Although there were common data visualization techniques and clinical indicators used across studies, there was diversity in the design of the dashboards and their evaluation. There was a lack of detail regarding the design processes documented for reproducibility. We identified a lack of interface features to support clinicians in making sense of and reflecting on their personal performance data. %M 35156928 %R 10.2196/32695 %U https://medinform.jmir.org/2022/2/e32695 %U https://doi.org/10.2196/32695 %U http://www.ncbi.nlm.nih.gov/pubmed/35156928 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 1 %P e30512 %T Electronic Records With Tablets at the Point of Care in an Internal Medicine Unit: Before-After Time Motion Study %A Pérez-Martí,Montserrat %A Casadó-Marín,Lina %A Guillén-Villar,Abraham %+ Department of Information Technology, Consorci Sanitari Alt Penedès-Garraf, c/Espirall 61, Vilafranca del Penedès, 08720, Spain, 34 938194000 ext 7445, mperezm@csapg.cat %K electronic health records %K nursing %K computer handheld %K equipment and supplies (devices tablets mobile phones, devices and technologies) %K workflow %D 2022 %7 10.2.2022 %9 Original Paper %J JMIR Hum Factors %G English %X Background: There are many benefits of nursing professionals being able to consult electronic health records (EHRs) at the point of care. It promotes quality and patient security, communication, continuity of care, and time dedicated to records. Objective: The aim of this study was to evaluate whether making EHRs available at the point of care with tablets reduces nurses’ time spent on records compared with the current system. The analysis included sociodemographic and qualitative variables, time spent per patient, and work shift. This time difference can be used for direct patient care. Methods: A before-after time motion study was carried out in the internal medicine unit. There was a total of 130 observations of 2 hours to 3 hours in duration of complete patient records that were carried out at the beginning of the nurses' work shifts. We calculated the time dedicated to measuring vital signs, patient evaluation, and EHR recording. The main variable was time spent per patient. Results: The average time spent per patient (total time/patients admitted) was lower with the tablet group (mean 4.22, SD 0.14 minutes) than with the control group (mean 4.66, SD 0.12 minutes); there were statistically significant differences (W=3.20, P=.001) and a low effect (d=.44) between groups. The tablet group saved an average of 0.44 (SD 0.13) minutes per patient. Similar results were obtained for the afternoon shift, which saved an average of 0.60 (SD 0.15) minutes per patient (t34=3.82, P=.01) and high effect (d=.77). However, although there was a mean difference of 0.26 (SD 0.22) minutes per patient for the night shift, this was not statistically significant (t29=1.16, P=.25). The “nonparticipating” average age was higher (49.57, SD 2.92 years) compared with the “afternoon shift participants” and “night shift participants” (P=.007). “Nonparticipants” of the night shift had a worse perception of the project. Conclusions: This investigation determined that, with EHRs at the point of care, the time spent for registration by the nursing staff decreases, because of reduced movements and avoiding data transcription. It eliminates unnecessary work that does not add value, and therefore, care is improved. So, we think EHRs at the point of care should be the future or natural method for nursing to undertake. However, variables that could have a negative effect include age, night shift, and nurses’ perceptions. Therefore, it is proposed that training in the different work platforms and the participation of nurses are fundamental axes that any institution should consider before their implementation. %M 35142624 %R 10.2196/30512 %U https://humanfactors.jmir.org/2022/1/e30512 %U https://doi.org/10.2196/30512 %U http://www.ncbi.nlm.nih.gov/pubmed/35142624 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 1 %P e27887 %T Enriching the Value of Patient Experience Feedback: Web-Based Dashboard Development Using Co-design and Heuristic Evaluation %A Khanbhai,Mustafa %A Symons,Joshua %A Flott,Kelsey %A Harrison-White,Stephanie %A Spofforth,Jamie %A Klaber,Robert %A Manton,David %A Darzi,Ara %A Mayer,Erik %+ Patient Safety Translational Research Centre, Imperial College London, National Institute for Health Research/Institute of Global Health Innovation, Queen Elizabeth the Queen Mother Wing, St Mary's Campus, London, W2 1NY, United Kingdom, 44 02033126428, m.khanbhai@imperial.ac.uk %K patient experience %K friends and family test %K quality dashboard %K co-design %K heuristic evaluation %K usability %D 2022 %7 3.2.2022 %9 Original Paper %J JMIR Hum Factors %G English %X Background: There is an abundance of patient experience data held within health care organizations, but stakeholders and staff are often unable to use the output in a meaningful and timely way to improve care delivery. Dashboards, which use visualized data to summarize key patient experience feedback, have the potential to address these issues. Objective: The aim of this study is to develop a patient experience dashboard with an emphasis on Friends and Family Test (FFT) reporting, as per the national policy drive. Methods: A 2-stage approach was used—participatory co-design involving 20 co-designers to develop a dashboard prototype, followed by iterative dashboard testing. Language analysis was performed on free-text patient experience data from the FFT, and the themes and sentiments generated were used to populate the dashboard with associated FFT metrics. Heuristic evaluation and usability testing were conducted to refine the dashboard and assess user satisfaction using the system usability score. Results: The qualitative analysis from the co-design process informed the development of the dashboard prototype with key dashboard requirements and a significant preference for bubble chart display. The heuristic evaluation revealed that most cumulative scores had no usability problems (18/20, 90%), had cosmetic problems only (7/20, 35%), or had minor usability problems (5/20, 25%). The mean System Usability Scale score was 89.7 (SD 7.9), suggesting an excellent rating. Conclusions: The growing capacity to collect and process patient experience data suggests that data visualization will be increasingly important in turning feedback into improvements to care. Through heuristic usability, we demonstrated that very large FFT data can be presented in a thematically driven, simple visual display without the loss of the nuances and still allow for the exploration of the original free-text comments. This study establishes guidance for optimizing the design of patient experience dashboards that health care providers find meaningful, which in turn drives patient-centered care. %M 35113022 %R 10.2196/27887 %U https://humanfactors.jmir.org/2022/1/e27887 %U https://doi.org/10.2196/27887 %U http://www.ncbi.nlm.nih.gov/pubmed/35113022 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 1 %P e23833 %T An Information Directory App InHouse Call for Streamlining Communication to Optimize Efficiency and Patient Care in a Hospital: Pilot Mixed Methods Design and Utility Study %A Schilling,George %A Villarosa,Leonardo %+ Vidant Medical Center, 2100 Statonsburg Rd, Greenville, NC, 27834, United States, 1 252 847 4441, georgeschilling@gmail.com %K InHouse Call %K communication %K hospital directory %K healthcare %K health care %K health informatics %K mHealth %K mobile app %K digital health %K patient records %K electronic health %D 2022 %7 27.1.2022 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Communication failures disrupt physician workflow, lead to poor patient outcomes, and are associated with significant economic burden. To increase efficiency when contacting a team member in a hospital, we have designed an information directory app, InHouse Call. Objective: This study aimed to describe the design of InHouse Call, objectively compare the usefulness of the app versus that of traditional methods (operator or pocket cards, etc), and determine its subjective usefulness through user surveys and a net promoter score (NPS). Methods: This pilot study utilizing before-after trials was carried out at a tertiary academic hospital and involved 20 clinicians, including physiatrists, hospitalists, internal medicine and family medicine residents, and advanced practice providers/nurse practitioners/physician assistants. InHouse Call was designed to efficiently supply contact information to providers through a simple, user-friendly interface. The participants used InHouse Call in timed trials to contact a health care team member in the hospital via a telephone call. The effectiveness of InHouse Call in connecting the user with a contact in the hospital was measured through timed trials comparing the amount of time spent in attempting to make the connection using traditional methods versus the app. Usability was measured through exit surveys and NPS. Results: The average time spent connecting to the correct contact using traditional methods was 59.5 seconds, compared to 13.8 seconds when using InHouse Call. The degree of variance when using traditional methods was 1544.2, compared to 19.7 with InHouse Call. A call made using the traditional methods deviated from the mean by 39.3 seconds, compared to 4.4 seconds when using InHouse Call. InHouse Call achieved an NPS of 95. Conclusions: InHouse Call significantly reduced the average amount of time spent connecting with the correct contact as well as the variability to complete the task, thus proving to be the superior method of communication for health care providers. The app garnered a high NPS and positive subjective feedback. %M 35084350 %R 10.2196/23833 %U https://humanfactors.jmir.org/2022/1/e23833 %U https://doi.org/10.2196/23833 %U http://www.ncbi.nlm.nih.gov/pubmed/35084350 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 1 %P e16447 %T Uses of Personal Health Records for Communication Among Colorectal Cancer Survivors, Caregivers, and Providers: Interview and Observational Study in a Human-Computer Interaction Laboratory %A Haggstrom,David A %A Carr,Thomas %+ VA HSR&D Center for Health Information and Communication, Roudebush Veterans Affairs Medical Center, 1481 West 10th Street, Indianapolis, IN, 46202, United States, 1 317 274 9304, dahaggst@iupui.edu %K personal health record %K communication %K cancer survivorship %K colorectal cancer %D 2022 %7 25.1.2022 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Personal health records (PHRs) may be useful for patient self-management and participation in communication with their caregivers and health care providers. As each potential participant’s role is different, their perception of the best uses of a PHR may vary. Objective: The perspectives of patients, caregivers, and providers were all evaluated concurrently in relation to a PHR developed for colorectal cancer (CRC) survivors. Methods: We explored group perceptions of a CRC PHR prototype. Scenario-based testing across eight use cases, with semistructured follow-up interviews, was videotaped in a human-computer interaction laboratory with patients, caregivers, and health care providers. Providers included oncologists, gastroenterologists, and primary care physicians. Discrete observations underwent grounded theory visual affinity analysis to identify emergent themes. Results: Observations fell into three major themes: the network (who should be granted access to the PHR by the patient), functions (helpful activities the PHR enabled), and implementation (how to adopt the PHR into workflow). Patients wanted physician access to their PHR, as well as family member access, especially when they lived at a distance. All groups noted the added value of linking the PHR to an electronic health record, self-tracking, self-management, and secure messaging. Patients and caregivers also saw information in the PHR as a useful memory tool given their visits to multiple doctors. Providers had reservations about patients viewing raw data, which they were not prepared to interpret or might be inaccurate; patients and caregivers did not express any reservations about having access to more information. Patients saw PHR communication functions as a potential tool for relationship building. Patients and caregivers valued the journal as a tool for reflection and delivery of emotional support. Providers felt the PHR would facilitate patient-physician communication but worried that sharing journal access would make the doctor-patient relationship less professional and had reservations about the time burden of reviewing. Strategies suggested for efficient adoption into workflow included team delegation. Establishment of parameters for patient uses and provider responses was perceived as good standard practice. Conclusions: PHR perceptions differed by role, with providers seeing the PHR as informational, while patients and caregivers viewed the tool as more relational. Personal health records should be linked to electronic health records for ease of use. Tailoring access, content, and implementation of the PHR is essential. Technology changes have the potential to change the nature of the patient-physician relationship. Patients and providers should establish shared expectations about the optimal use of the PHR and explore how emerging patient-centered technologies can be successfully implemented in modern medical practice to improve the relational quality of care. %M 35076406 %R 10.2196/16447 %U https://humanfactors.jmir.org/2022/1/e16447 %U https://doi.org/10.2196/16447 %U http://www.ncbi.nlm.nih.gov/pubmed/35076406 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 1 %P e28762 %T Comparing International Experiences With Electronic Health Records Among Emergency Medicine Physicians in the United States and Norway: Semistructured Interview Study %A Garcia,Gracie %A Crenner,Christopher %+ Department of History and Philosophy of Medicine, University of Kansas School of Medicine, 3901 Rainbow Blvd, Kansas City, KS, 66160, United States, 1 6209514171, grace.c.garcia1@gmail.com %K electronic health records %K electronic medical records %K health information technology %K health information exchange %K health policy %K international %K emergency medicine %K medical informatics %K meaningful use %K burnout %D 2022 %7 7.1.2022 %9 Original Paper %J JMIR Hum Factors %G English %X Background: The variability in physicians’ attitudes regarding electronic health records (EHRs) is widely recognized. Both human and technological factors contribute to user satisfaction. This exploratory study considers these variables by comparing emergency medicine physician experiences with EHRs in the United States and Norway. Objective: This study is unique as it aims to compare individual experiences with EHRs. It creates an opportunity to expand perspective, challenge the unknown, and explore how this technology affects clinicians globally. Research often highlights the challenge that health information technology has created for users: Are the negative consequences of this technology shared among countries? Does it affect medical practice? What determines user satisfaction? Can this be measured internationally? Do specific factors account for similarities or differences? This study begins by investigating these questions by comparing cohort experiences. Fundamental differences between nations will also be addressed. Methods: We used semistructured, participant-driven, in-depth interviews (N=12) for data collection in conjunction with ethnographic observations. The conversations were recorded and transcribed. Texts were then analyzed using NVivo software (QSR International) to develop codes for direct comparison among countries. Comprehensive understanding of the data required triangulation, specifically using thematic and interpretive phenomenological analysis. Narrative analysis ensured appropriate context of the NVivo (QSR International) query results. Results: Each interview resulted in mixed discussions regarding the benefits and disadvantages of EHRs. All the physicians recognized health care’s dependence on this technology. In Norway, physicians perceived more benefits compared with those based in the United States. Americans reported fewer benefits and disproportionally high disadvantages. Both cohorts believed that EHRs have increased user workload. However, this was mentioned 2.6 times more frequently by Americans (United States [n=40] vs Norway [n=15]). Financial influences regarding health information technology use were of great concern for American physicians but rarely mentioned among Norwegian physicians (United States [n=37] vs Norway [n=6]). Technology dysfunctions were the most common complaint from Norwegian physicians. Participants from each country noted increased frustration among older colleagues. Conclusions: Despite differences spanning geographical, organizational, and cultural boundaries, much is to be learned by comparing individual experiences. Both cohorts experienced EHR-related frustrations, although etiology differed. The overall number of complaints was significantly higher among American physicians. This study augments the idea that policy, regulation, and administration have compelling influence on user experience. Global EHR optimization requires additional investigation, and these results help to establish a foundation for future research. %M 34994702 %R 10.2196/28762 %U https://humanfactors.jmir.org/2022/1/e28762 %U https://doi.org/10.2196/28762 %U http://www.ncbi.nlm.nih.gov/pubmed/34994702 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 1 %P e28783 %T Clinician Perspectives on Unmet Needs for Mobile Technology Among Hospitalists: Workflow Analysis Based on Semistructured Interviews %A Savoy,April %A Saleem,Jason J %A Barker,Barry C %A Patel,Himalaya %A Kara,Areeba %+ Center for Health Information and Communication, Health Services Research and Development Service, Richard L Roudebush Veterans Affairs Medical Center, 1481 West 10th Street, Stop 11H, Indianapolis, IN, 46202, United States, 1 3172782194, asavoy@iu.edu %K electronic health records %K hospital medicine %K user-computer interface %K human-computer interaction %K usability %K mental workload %K workflow analysis %D 2022 %7 4.1.2022 %9 Original Paper %J JMIR Hum Factors %G English %X Background: The hospitalist workday is cognitively demanding and dominated by activities away from patients’ bedsides. Although mobile technologies are offered as solutions, clinicians report lower expectations of mobile technology after actual use. Objective: The purpose of this study is to better understand opportunities for integrating mobile technology and apps into hospitalists’ workflows. We aim to identify difficult tasks and contextual factors that introduce inefficiencies and characterize hospitalists’ perspectives on mobile technology and apps. Methods: We conducted a workflow analysis based on semistructured interviews. At a Midwestern US medical center, we recruited physicians and nurse practitioners from hospitalist and inpatient teaching teams and internal medicine residents. Interviews focused on tasks perceived as frequent, redundant, and difficult. Additionally, participants were asked to describe opportunities for mobile technology interventions. We analyzed contributing factors, impacted workflows, and mobile app ideas. Results: Over 3 months, we interviewed 12 hospitalists. Participants collectively identified chart reviews, orders, and documentation as the most frequent, redundant, and difficult tasks. Based on those tasks, the intake, discharge, and rounding workflows were characterized as difficult and inefficient. The difficulty was associated with a lack of access to electronic health records at the bedside. Contributing factors for inefficiencies were poor usability and inconsistent availability of health information technology combined with organizational policies. Participants thought mobile apps designed to improve team communications would be most beneficial. Based on our analysis, mobile apps focused on data entry and presentation supporting specific tasks should also be prioritized. Conclusions: Based on our results, there are prioritized opportunities for mobile technology to decrease difficulty and increase the efficiency of hospitalists’ workflows. Mobile technology and task-specific mobile apps with enhanced usability could decrease overreliance on hospitalists’ memory and fragmentation of clinical tasks across locations. This study informs the design and implementation processes of future health information technologies to improve continuity in hospital-based medicine. %M 34643530 %R 10.2196/28783 %U https://humanfactors.jmir.org/2022/1/e28783 %U https://doi.org/10.2196/28783 %U http://www.ncbi.nlm.nih.gov/pubmed/34643530 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 8 %N 4 %P e31130 %T Exploring Usability Issues of a Smartphone-Based Physician-to-Physician Teleconsultation App in an Orthopedic Clinic: Mixed Methods Study %A Choemprayong,Songphan %A Charoenlap,Chris %A Piromsopa,Krerk %+ Department of Orthopaedic, Faculty of Medicine, Chulalongkorn University, Rama IV Road, Pathumwan, Bangkok, 10330, Thailand, 66 081 552 4224, chris.cha@chula.ac.th %K teleconsultation %K remote consultation %K mobile applications %K usability %K orthopedics %K physician-to-physician consultation %K electronic medical records %K mobile phone %D 2021 %7 20.12.2021 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Physician-to-physician teleconsultation has increasingly played an essential role in delivering optimum health care services, particularly in orthopedic practice. In this study, the usability of a smartphone app for teleconsultation among orthopedic specialists was investigated to explore issues informing further recommendations for improvement in the following iterations. Objective: This study aimed to explore usability issues emerging from users’ interactions with MEDIC app, a smartphone-based patient-centered physician-to-physician teleconsultation system. Methods: Five attending physicians in the Department of Orthopedics in a large medical school in Bangkok, Thailand, were recruited and asked to perform 5 evaluation tasks, namely, group formation, patient registration, clinical data capturing, case record form creation, and teleconsultation. In addition, one expert user was recruited as the control participant. Think aloud was adopted while performing the tasks. Semistructured interviews were conducted after each task and prior to the exit. Quantitative and qualitative measures were used to identify usability issues in 7 domains based on the People At the Centre of Mobile Application Development model: effectiveness, efficiency, satisfaction, learnability, memorability, error, and cognitive load. Results: Several measures indicate various aspects of usability of the app, including completion rates, time to completion, number of clicks, number of screens, errors, incidents where participants were unable to perform tasks, which had previously been completed, and perceived task difficulty. Major and critical usability issues based on participant feedback were rooted from the limitation of screen size and resolution. Errors in data input (eg, typing errors, miscalculation), action failures, and misinterpretation of data (ie, radiography) were the most critical and common issues found in this study. A few participants did not complete the assigned tasks mostly owing to the navigation design and misreading/misunderstanding icons. However, the novice users were quite positive that they would be able to become familiar with the app in a short period of time. Conclusions: The usability issues in physician-to-physician teleconsultation systems in smartphones, in general, are derived from the limitations of smartphones and their operating systems. Although some recommendations were devised to handle these usability issues, usability evaluation for additional development should still be further investigated. %M 34931991 %R 10.2196/31130 %U https://humanfactors.jmir.org/2021/4/e31130 %U https://doi.org/10.2196/31130 %U http://www.ncbi.nlm.nih.gov/pubmed/34931991 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 8 %N 4 %P e31214 %T Stakeholder Perspectives on an Inpatient Hypoglycemia Informatics Alert: Mixed Methods Study %A Mathioudakis,Nestoras %A Aboabdo,Moeen %A Abusamaan,Mohammed S %A Yuan,Christina %A Lewis Boyer,LaPricia %A Pilla,Scott J %A Johnson,Erica %A Desai,Sanjay %A Knight,Amy %A Greene,Peter %A Golden,Sherita H %+ Division of Endocrinology, Diabetes & Metabolism, Department of Medicine, Johns Hopkins University, 1830 E. Monument Street, Suite 333, Baltimore, MD, 21287, United States, 1 410 502 8089, nmathio1@jhmi.edu %K informatics alert %K clinical decision support %K hypoglycemia %K hospital %K inpatient %D 2021 %7 26.11.2021 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Iatrogenic hypoglycemia is a common occurrence among hospitalized patients and is associated with poor clinical outcomes and increased mortality. Clinical decision support systems can be used to reduce the incidence of this potentially avoidable adverse event. Objective: This study aims to determine the desired features and functionality of a real-time informatics alert to prevent iatrogenic hypoglycemia in a hospital setting. Methods: Using the Agency for Healthcare Research and Quality Five Rights of Effective Clinical Decision Support Framework, we conducted a mixed methods study using an electronic survey and focus group sessions of hospital-based providers. The goal was to elicit stakeholder input to inform the future development of a real-time informatics alert to target iatrogenic hypoglycemia. In addition to perceptions about the importance of the problem and existing barriers, we sought input regarding the content, format, channel, timing, and recipient for the alert (ie, the Five Rights). Thematic analysis of focus group sessions was conducted using deductive and inductive approaches. Results: A 21-item electronic survey was completed by 102 inpatient-based providers, followed by 2 focus group sessions (6 providers per session). Respondents universally agreed or strongly agreed that inpatient iatrogenic hypoglycemia is an important problem that can be addressed with an informatics alert. Stakeholders expressed a preference for an alert that is nonintrusive, accurate, communicated in near real time to the ordering provider, and provides actionable treatment recommendations. Several electronic medical record tools, including alert indicators in the patient header, glucose management report, and laboratory results section, were deemed acceptable formats for consideration. Concerns regarding alert fatigue were prevalent among both survey respondents and focus group participants. Conclusions: The design preferences identified in this study will provide the framework needed for an informatics team to develop a prototype alert for pilot testing and evaluation. This alert will help meet the needs of hospital-based clinicians caring for patients with diabetes who are at a high risk of treatment-related hypoglycemia. %M 34842544 %R 10.2196/31214 %U https://humanfactors.jmir.org/2021/4/e31214 %U https://doi.org/10.2196/31214 %U http://www.ncbi.nlm.nih.gov/pubmed/34842544 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 8 %N 4 %P e26017 %T User-Centered System Design for Communicating Clinical Laboratory Test Results: Design and Evaluation Study %A Zhang,Zhan %A Kmoth,Lukas %A Luo,Xiao %A He,Zhe %+ School of Computer Science and Information Systems, Pace University, 163 William Street, New York, NY, 10078, United States, 1 2123461897, zzhang@pace.edu %K clinical laboratory results %K patient-centered care %K patient portal %K health communication %D 2021 %7 25.11.2021 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Personal clinical data, such as laboratory test results, are increasingly being made available to patients via patient portals. However, laboratory test results are presented in a way that is difficult for patients to interpret and use. Furthermore, the indications of laboratory test results may vary among patients with different characteristics and from different medical contexts. To date, little is known about how to design patient-centered technology to facilitate the interpretation of laboratory test results. Objective: The aim of this study is to explore design considerations for supporting patient-centered communication and comprehension of laboratory test results, as well as discussions between patients and health care providers. Methods: We conducted a user-centered, multicomponent design research consisting of user studies, an iterative prototype design, and pilot user evaluations, to explore design concepts and considerations that are useful for supporting patients in not only viewing but also interpreting and acting upon laboratory test results. Results: The user study results informed the iterative design of a system prototype, which had several interactive features: using graphical representations and clear takeaway messages to convey the concerning nature of the results; enabling users to annotate laboratory test reports; clarifying medical jargon using nontechnical verbiage and allowing users to interact with the medical terms (eg, saving, favoriting, or sorting); and providing pertinent and reliable information to help patients comprehend test results within their medical context. The results of a pilot user evaluation with 8 patients showed that the new patient-facing system was perceived as useful in not only presenting laboratory test results to patients in a meaningful way but also facilitating in situ patient-provider interactions. Conclusions: We draw on our findings to discuss design implications for supporting patient-centered communication of laboratory test results and how to make technology support informative, trustworthy, and empathetic. %M 34842529 %R 10.2196/26017 %U https://humanfactors.jmir.org/2021/4/e26017 %U https://doi.org/10.2196/26017 %U http://www.ncbi.nlm.nih.gov/pubmed/34842529 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 11 %P e28854 %T Analysis of a Web-Based Dashboard to Support the Use of National Audit Data in Quality Improvement: Realist Evaluation %A Alvarado,Natasha %A McVey,Lynn %A Elshehaly,Mai %A Greenhalgh,Joanne %A Dowding,Dawn %A Ruddle,Roy %A Gale,Chris P %A Mamas,Mamas %A Doherty,Patrick %A West,Robert %A Feltbower,Richard %A Randell,Rebecca %+ Faculty of Health Studies, University of Bradford, Great Horton Road, Bradford, BD7 1DP, United Kingdom, 44 07715433565, n.alvarado@bradford.ac.uk %K data %K QualDash %K audit %K dashboards %K support %K quality %D 2021 %7 23.11.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: Dashboards can support data-driven quality improvements in health care. They visualize data in ways intended to ease cognitive load and support data comprehension, but how they are best integrated into working practices needs further investigation. Objective: This paper reports the findings of a realist evaluation of a web-based quality dashboard (QualDash) developed to support the use of national audit data in quality improvement. Methods: QualDash was co-designed with data users and installed in 8 clinical services (3 pediatric intensive care units and 5 cardiology services) across 5 health care organizations (sites A-E) in England between July and December 2019. Champions were identified to support adoption. Data to evaluate QualDash were collected between July 2019 and August 2021 and consisted of 148.5 hours of observations including hospital wards and clinical governance meetings, log files that captured the extent of use of QualDash over 12 months, and a questionnaire designed to assess the dashboard’s perceived usefulness and ease of use. Guided by the principles of realist evaluation, data were analyzed to understand how, why, and in what circumstances QualDash supported the use of national audit data in quality improvement. Results: The observations revealed that variation across sites in the amount and type of resources available to support data use, alongside staff interactions with QualDash, shaped its use and impact. Sites resourced with skilled audit support staff and established reporting systems (sites A and C) continued to use existing processes to report data. A number of constraints influenced use of QualDash in these sites including that some dashboard metrics were not configured in line with user expectations and staff were not fully aware how QualDash could be used to facilitate their work. In less well-resourced services, QualDash automated parts of their reporting process, streamlining the work of audit support staff (site B), and, in some cases, highlighted issues with data completeness that the service worked to address (site E). Questionnaire responses received from 23 participants indicated that QualDash was perceived as useful and easy to use despite its variable use in practice. Conclusions: Web-based dashboards have the potential to support data-driven improvement, providing access to visualizations that can help users address key questions about care quality. Findings from this study point to ways in which dashboard design might be improved to optimize use and impact in different contexts; this includes using data meaningful to stakeholders in the co-design process and actively engaging staff knowledgeable about current data use and routines in the scrutiny of the dashboard metrics and functions. In addition, consideration should be given to the processes of data collection and upload that underpin the quality of the data visualized and consequently its potential to stimulate quality improvement. International Registered Report Identifier (IRRID): RR2-10.1136/bmjopen-2019-033208 %M 34817384 %R 10.2196/28854 %U https://www.jmir.org/2021/11/e28854 %U https://doi.org/10.2196/28854 %U http://www.ncbi.nlm.nih.gov/pubmed/34817384 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 11 %P e29532 %T Introduction of Systematized Nomenclature of Medicine–Clinical Terms Coding Into an Electronic Health Record and Evaluation of its Impact: Qualitative and Quantitative Study %A Pankhurst,Tanya %A Evison,Felicity %A Atia,Jolene %A Gallier,Suzy %A Coleman,Jamie %A Ball,Simon %A McKee,Deborah %A Ryan,Steven %A Black,Ruth %+ NHS Foundation Trust, University Hospitals Birmingham, Mindelsohn Way, Birmingham, B15 2TG, United Kingdom, 44 7811357984, pankhurst.tanya@gmail.com %K coding standards %K clinical decision support %K Clinician led design %K clinician reported experience %K clinical usability %K data sharing %K diagnoses %K electronic health records %K electronic health record standards %K health data exchange %K health data research %K International Classification of Diseases version 10 (ICD-10) %K National Health Service Blueprint %K patient diagnoses %K population health %K problem list %K research %K Systematized Nomenclature Of Medicine–Clinical Terms (SNOMED-CT) %K use of electronic health data %K user-led design %D 2021 %7 23.11.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: This study describes the conversion within an existing electronic health record (EHR) from the International Classification of Diseases, Tenth Revision coding system to the SNOMED-CT (Systematized Nomenclature of Medicine–Clinical Terms) for the collection of patient histories and diagnoses. The setting is a large acute hospital that is designing and building its own EHR. Well-designed EHRs create opportunities for continuous data collection, which can be used in clinical decision support rules to drive patient safety. Collected data can be exchanged across health care systems to support patients in all health care settings. Data can be used for research to prevent diseases and protect future populations. Objective: The aim of this study was to migrate a current EHR, with all relevant patient data, to the SNOMED-CT coding system to optimize clinical use and clinical decision support, facilitate data sharing across organizational boundaries for national programs, and enable remodeling of medical pathways. Methods: The study used qualitative and quantitative data to understand the successes and gaps in the project, clinician attitudes toward the new tool, and the future use of the tool. Results: The new coding system (tool) was well received and immediately widely used in all specialties. This resulted in increased, accurate, and clinically relevant data collection. Clinicians appreciated the increased depth and detail of the new coding, welcomed the potential for both data sharing and research, and provided extensive feedback for further development. Conclusions: Successful implementation of the new system aligned the University Hospitals Birmingham NHS Foundation Trust with national strategy and can be used as a blueprint for similar projects in other health care settings. %M 34817387 %R 10.2196/29532 %U https://medinform.jmir.org/2021/11/e29532 %U https://doi.org/10.2196/29532 %U http://www.ncbi.nlm.nih.gov/pubmed/34817387 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 5 %N 11 %P e23789 %T A Fully Collaborative, Noteless Electronic Medical Record Designed to Minimize Information Chaos: Software Design and Feasibility Study %A Steinkamp,Jackson %A Sharma,Abhinav %A Bala,Wasif %A Kantrowitz,Jacob J %+ Hospital of the University of Pennsylvania, 3400 Spruce Street, Philadelphia, PA, United States, 1 215 662 4000, jacksonsteinkamp@gmail.com %K electronic medical records %K clinical notes %K information chaos %K information overload %K clinician burnout %K software design %K problem-oriented medical record %K medical records %K electronic records %K documentation %K clinical %K software %D 2021 %7 9.11.2021 %9 Original Paper %J JMIR Form Res %G English %X Background: Clinicians spend large amounts of their workday using electronic medical records (EMRs). Poorly designed documentation systems contribute to the proliferation of out-of-date information, increased time spent on medical records, clinician burnout, and medical errors. Beyond software interfaces, examining the underlying paradigms and organizational structures for clinical information may provide insights into ways to improve documentation systems. In particular, our attachment to the note as the major organizational unit for storing unstructured medical data may be a cause of many of the problems with modern clinical documentation. Notes, as currently understood, systematically incentivize information duplication and information scattering, both within a single clinician’s notes over time and across multiple clinicians’ notes. Therefore, it is worthwhile to explore alternative paradigms for unstructured data organization. Objective: The aim of this study is to demonstrate the feasibility of building an EMR that does not use notes as the core organizational unit for unstructured data and which is designed specifically to disincentivize information duplication and information scattering. Methods: We used specific design principles to minimize the incentive for users to duplicate and scatter information. By default, the majority of a patient’s medical history remains the same over time, so users should not have to redocument that information. Clinicians on different teams or services mostly share the same medical information, so all data should be collaboratively shared across teams and services (while still allowing for disagreement and nuance). In all cases where a clinician must state that information has remained the same, they should be able to attest to the information without redocumenting it. We designed and built a web-based EMR based on these design principles. Results: We built a medical documentation system that does not use notes and instead treats the chart as a single, dynamically updating, and fully collaborative workspace. All information is organized by clinical topic or problem. Version history functionality is used to enable granular tracking of changes over time. Our system is highly customizable to individual workflows and enables each individual user to decide which data should be structured and which should be unstructured, enabling individuals to leverage the advantages of structured templating and clinical decision support as desired without requiring programming knowledge. The system is designed to facilitate real-time, fully collaborative documentation and communication among multiple clinicians. Conclusions: We demonstrated the feasibility of building a non–note-based, fully collaborative EMR system. Our attachment to the note as the only possible atomic unit of unstructured medical data should be reevaluated, and alternative models should be considered. %M 34751651 %R 10.2196/23789 %U https://formative.jmir.org/2021/11/e23789 %U https://doi.org/10.2196/23789 %U http://www.ncbi.nlm.nih.gov/pubmed/34751651 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 8 %N 4 %P e29180 %T Evaluating An Automated Compounding Workflow Software for Safety and Efficiency: Implementation Study %A Meren,Ülle Helena %A Waterson,James %+ Medical Affairs, Medication Management Solutions, Becton Dickinson Ltd, 11F Blue Bay Tower, Business Bay, Dubai, 52279, United Arab Emirates, 971 566035154, james.waterson@bd.com %K compounding %K medication safety %K positive patient identification %K gravimetric %K automation %K closed loop %D 2021 %7 2.11.2021 %9 Original Paper %J JMIR Hum Factors %G English %X Background: The forms of automation available to the oncology pharmacy range from compounding robotic solutions through to combination workflow software, which can scale-up to cover the entire workflow from prescribing to administration. A solution that offers entire workflow management for oncology is desirable because (in terms of cytotoxic delivery of a regimen to a patient) the chain that starts with prescription and the assay of the patient’s laboratory results and ends with administration has multiple potential safety gaps and choke points. Objective: The aim of this study was to show how incremental change to a core compounding workflow software solution has helped an organization meet goals of improved patient safety; increasing the number of oncology treatments; improving documentation; and improving communication between oncologists, pharmacists, and nurses. We also aimed to illustrate how using this technology flow beyond the pharmacy has extended medication safety to the patient’s bedside through the deployment of a connected solution for confirming and documenting right patient–right medication transactions. Methods: A compounding workflow software solution was introduced for both preparation and documentation, with pharmacist verification of the order, gravimetric checks, and step-by-step on-screen instructions displayed in the work area for the technician. The software supported the technician during compounding by proposing the required drug vial size, diluents, and consumables. Out-of-tolerance concentrations were auto-alerted via an integrated gravimetric scale. A patient-medication label was created. Integration was undertaken between a prescribing module and the compounding module to reduce the risk of transcription errors. The deployment of wireless-connected handheld barcode scanners was then made to allow nurses to use the patient-medication label on each compounded product and to scan patient identification bands to ensure right patient–right prescription. Results: Despite an increase in compounding, with a growth of 12% per annum and no increase in pharmacy headcount, we doubled our output to 14,000 medications per annum through the application of the compounding solution. The use of a handheld barcode scanning device for nurses reduced the time for medication administration from ≈6 minutes per item to 41 seconds, with a mean average saving of 5 minutes and 19 seconds per item. When calculated against our throughput of 14,000 items per annum (current production rate via pharmacy), this gives a saving of 3 hours and 24 minutes of nursing time per day, equivalent to 0.425 full-time nurses per annum. Conclusions: The addition of prescribing, compounding, and administration software solutions to our oncology medication chain has increased detection and decreased the risk of error at each stage of the process. The double-checks that the system has built in by virtue of its own systems and through the flow of control of drugs and dosages from physician to pharmacist to nurse allow it to integrate fully with our human systems of risk management. %M 34456182 %R 10.2196/29180 %U https://humanfactors.jmir.org/2021/4/e29180 %U https://doi.org/10.2196/29180 %U http://www.ncbi.nlm.nih.gov/pubmed/34456182 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 10 %P e29259 %T Transforming Anesthesia Data Into the Observational Medical Outcomes Partnership Common Data Model: Development and Usability Study %A Lamer,Antoine %A Abou-Arab,Osama %A Bourgeois,Alexandre %A Parrot,Adrien %A Popoff,Benjamin %A Beuscart,Jean-Baptiste %A Tavernier,Benoît %A Moussa,Mouhamed Djahoum %+ Univ. Lille, CHU Lille, ULR 2694 - METRICS: Évaluation des technologies de santé et des pratiques médicales, 2 place de Verdun, Lille, F-59000, France, 33 320626969, antoine.lamer@chru-lille.fr %K data reuse %K common data model %K Observational Medical Outcomes Partnership %K anesthesia %K data warehouse %K reproducible research %D 2021 %7 29.10.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: Electronic health records (EHRs, such as those created by an anesthesia management system) generate a large amount of data that can notably be reused for clinical audits and scientific research. The sharing of these data and tools is generally affected by the lack of system interoperability. To overcome these issues, Observational Health Data Sciences and Informatics (OHDSI) developed the Observational Medical Outcomes Partnership (OMOP) common data model (CDM) to standardize EHR data and promote large-scale observational and longitudinal research. Anesthesia data have not previously been mapped into the OMOP CDM. Objective: The primary objective was to transform anesthesia data into the OMOP CDM. The secondary objective was to provide vocabularies, queries, and dashboards that might promote the exploitation and sharing of anesthesia data through the CDM. Methods: Using our local anesthesia data warehouse, a group of 5 experts from 5 different medical centers identified local concepts related to anesthesia. The concepts were then matched with standard concepts in the OHDSI vocabularies. We performed structural mapping between the design of our local anesthesia data warehouse and the OMOP CDM tables and fields. To validate the implementation of anesthesia data into the OMOP CDM, we developed a set of queries and dashboards. Results: We identified 522 concepts related to anesthesia care. They were classified as demographics, units, measurements, operating room steps, drugs, periods of interest, and features. After semantic mapping, 353 (67.7%) of these anesthesia concepts were mapped to OHDSI concepts. Further, 169 (32.3%) concepts related to periods and features were added to the OHDSI vocabularies. Then, 8 OMOP CDM tables were implemented with anesthesia data and 2 new tables (EPISODE and FEATURE) were added to store secondarily computed data. We integrated data from 5,72,609 operations and provided the code for a set of 8 queries and 4 dashboards related to anesthesia care. Conclusions: Generic data concerning demographics, drugs, units, measurements, and operating room steps were already available in OHDSI vocabularies. However, most of the intraoperative concepts (the duration of specific steps, an episode of hypotension, etc) were not present in OHDSI vocabularies. The OMOP mapping provided here enables anesthesia data reuse. %M 34714250 %R 10.2196/29259 %U https://www.jmir.org/2021/10/e29259 %U https://doi.org/10.2196/29259 %U http://www.ncbi.nlm.nih.gov/pubmed/34714250 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 8 %N 4 %P e27671 %T Medical Specialists' Perspectives on the Influence of Electronic Medical Record Use on the Quality of Hospital Care: Semistructured Interview Study %A van Poelgeest,Rube %A Schrijvers,Augustinus %A Boonstra,Albert %A Roes,Kit %+ Julius Center, University Medical Center, University of Utrecht, Universiteitsweg 100, Utrecht, 3584 CG, Netherlands, 31 620139545, rube.van.poelgeest@planet.nl %K electronic medical record (emr) %K hospitals %K quality %K health care %K medical specialist %D 2021 %7 27.10.2021 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Numerous publications show that electronic medical records (EMRs) may make an important contribution to increasing the quality of care. There are indications that particularly the medical specialist plays an important role in the use of EMRs in hospitals. Objective: The aim of this study was to examine how, and by which aspects, the relationship between EMR use and the quality of care in hospitals is influenced according to medical specialists. Methods: To answer this question, a qualitative study was conducted in the period of August-October 2018. Semistructured interviews of around 90 min were conducted with 11 medical specialists from 11 different Dutch hospitals. For analysis of the answers, we used a previously published taxonomy of factors that can influence the use of EMRs. Results: The professional experience of the participating medical specialists varied between 5 and 27 years. Using the previously published taxonomy, these medical specialists considered technical barriers the most significant for EMR use. The suboptimal change processes surrounding implementation were also perceived as a major barrier. A final major problem is related to the categories “social” (their relationships with the patients and fellow care providers), “psychological” (based on their personal issues, knowledge, and perceptions), and “time” (the time required to select, implement, and learn how to use EMR systems and subsequently enter data into the system). However, the medical specialists also identified potential technical facilitators, particularly in the assured availability of information to all health care professionals involved in the care of a patient. They see promise in using EMRs for medical decision support to improve the quality of care but consider these possibilities currently lacking. Conclusions: The 11 medical specialists shared positive experiences with EMR use when comparing it to formerly used paper records. The fact that involved health care professionals can access patient data at any time they need is considered important. However, in practice, potential quality improvement lags as long as decision support cannot be applied because of the lack of a fully coded patient record. %M 34704955 %R 10.2196/27671 %U https://humanfactors.jmir.org/2021/4/e27671 %U https://doi.org/10.2196/27671 %U http://www.ncbi.nlm.nih.gov/pubmed/34704955 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 8 %N 4 %P e28618 %T A Shared Decision-making Tool for Drug Interactions Between Warfarin and Nonsteroidal Anti-inflammatory Drugs: Design and Usability Study %A Reese,Thomas J %A Del Fiol,Guilherme %A Morgan,Keaton %A Hurwitz,Jason T %A Kawamoto,Kensaku %A Gomez-Lumbreras,Ainhoa %A Brown,Mary L %A Thiess,Henrik %A Vazquez,Sara R %A Nelson,Scott D %A Boyce,Richard %A Malone,Daniel %+ Vanderbilt University, 2525 West End Avenue, Suite 1475, Nashville, TN, 37203, United States, 1 (615) 936 6867, Thomas.Reese@vumc.org %K shared decision-making %K user-centered design %K drug interaction %K clinical decision support %D 2021 %7 26.10.2021 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Exposure to life-threatening drug-drug interactions (DDIs) occurs despite the widespread use of clinical decision support. The DDI between warfarin and nonsteroidal anti-inflammatory drugs is common and potentially life-threatening. Patients can play a substantial role in preventing harm from DDIs; however, the current model for DDI decision-making is clinician centric. Objective: This study aims to design and study the usability of DDInteract, a tool to support shared decision-making (SDM) between a patient and provider for the DDI between warfarin and nonsteroidal anti-inflammatory drugs. Methods: We used an SDM framework and user-centered design methods to guide the design and usability of DDInteract—an SDM electronic health record app to prevent harm from clinically significant DDIs. The design involved iterative prototypes, qualitative feedback from stakeholders, and a heuristic evaluation. The usability evaluation included patients and clinicians. Patients participated in a simulated SDM discussion using clinical vignettes. Clinicians were asked to complete eight tasks using DDInteract and to assess the tool using a survey adapted from the System Usability Scale. Results: The designed DDInteract prototype includes the following features: a patient-specific risk profile, dynamic risk icon array, patient education section, and treatment decision tree. A total of 4 patients and 11 clinicians participated in the usability study. After an SDM session where patients and clinicians review the tool concurrently, patients generally favored pain treatments with less risk of gastrointestinal bleeding. Clinicians successfully completed the tasks with a mean of 144 (SD 74) seconds and rated the usability of DDInteract as 4.32 (SD 0.52) of 5. Conclusions: This study expands the use of SDM to DDIs. The next steps are to determine if DDInteract can improve shared decision-making quality and to implement it across health systems using interoperable technology. %M 34698649 %R 10.2196/28618 %U https://humanfactors.jmir.org/2021/4/e28618 %U https://doi.org/10.2196/28618 %U http://www.ncbi.nlm.nih.gov/pubmed/34698649 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 10 %P e30165 %T Writing Practices Associated With Electronic Progress Notes and the Preferences of Those Who Read Them: Descriptive Study %A Payne,Thomas H %A Keller,Carolyn %A Arora,Pallavi %A Brusati,Allison %A Levin,Jesse %A Salgaonkar,Monica %A Li,Xi %A Zech,Jennifer %A Lees,A Fischer %+ Department of Medicine, University of Washington School of Medicine, Box 359780, 325 Ninth Ave, Seattle, WA, 98104-2499, United States, 1 206 744 1824, tpayne@uw.edu %K electronic documentation %K electronic health records %K hospital progress notes %K copy-paste %K EHR %K patient records %K workflow %K human factors %K clinical communication %K physician communication %K hospital %D 2021 %7 6.10.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: Hospital progress notes can serve as an important communication tool. However, they are criticized for their length, preserved content, and for the time physicians spend writing them. Objective: We aimed to describe hospital progress note content, writing and reading practices, and the preferences of those who create and read them prior to the implementation of a new electronic health record system. Methods: Using a sample of hospital progress notes from 1000 randomly selected admissions, we measured note length, similarity of content in successive daily notes for the same patient, the time notes were signed and read, and who read them. We conducted focus group sessions with note writers, readers, and clinical leaders to understand their preferences. Results: We analyzed 4938 inpatient progress notes from 418 authors. The average length was 886 words, and most were in the Assessment & Plan note section. A total of 29% of notes (n=1432) were signed after 4 PM. Notes signed later in the day were read less often. Notes were highly similar from one day to the next, and 26% (23/88) had clinical risk associated with the preserved content. Note content of the highest value varied according to the reader’s professional role. Conclusions: Progress note length varied widely. Notes were often signed late in the day when they were read less often and were highly similar to the note from the previous day. Measuring note length, signing time, when and by whom notes are read, and the amount and safety of preserved content will be useful metrics for measuring how the new electronic health record system is used, and can aid improvements. %M 34612825 %R 10.2196/30165 %U https://www.jmir.org/2021/10/e30165 %U https://doi.org/10.2196/30165 %U http://www.ncbi.nlm.nih.gov/pubmed/34612825 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 8 %N 3 %P e28501 %T Examining How Internet Users Trust and Access Electronic Health Record Patient Portals: Survey Study %A Yin,Rong %A Law,Katherine %A Neyens,David %+ Department of Industrial Engineering, Clemson University, 100 Freeman Hall, Clemson, SC, United States, 1 8646564719, dneyens@clemson.edu %K internet %K consumer health informatics %K patient portal %K participatory medicine %K electronic health records %K logistic model %K surveys %K questionnaires %D 2021 %7 21.9.2021 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Electronic health record (EHR) patient portals are designed to provide medical health records to patients. Using an EHR portal is expected to contribute to positive health outcomes and facilitate patient-provider communication. Objective: Our objective was to examine how portal users report using their portals and the factors associated with obtaining health information from the internet. We also examined the desired portal features, factors impacting users’ trust in portals, and barriers to using portals. Methods: An internet-based survey study was conducted using Amazon Mechanical Turk. All the participants were adults in the United States who used patient portals. The survey included questions about how the participants used their portals, what factors acted as barriers to using their portals, and how they used and how much they trusted other web-based health information sources as well as their portals. A logistic regression model was used to examine the factors influencing the participants’ trust in their portals. Additionally, the desired features and design characteristics were identified to support the design of future portals. Results: A total of 394 participants completed the survey. Most of the participants were less than 35 years old (212/394, 53.8%), with 36.3% (143/394) aged between 35 and 55 years, and 9.9% (39/394) aged above 55 years. Women accounted for 48.5% (191/394) of the survey participants. More than 78% (307/394) of the participants reported using portals at least monthly. The most common portal features used were viewing lab results, making appointments, and paying bills. Participants reported some barriers to portal use including data security and limited access to the internet. The results of a logistic regression model used to predict the trust in their portals suggest that those comfortable using their portals (odds ratio [OR] 7.97, 95% CI 1.11-57.32) thought that their portals were easy to use (OR 7.4, 95% CI 1.12-48.84), and frequent internet users (OR 43.72, 95% CI 1.83-1046.43) were more likely to trust their portals. Participants reporting that the portals were important in managing their health (OR 28.13, 95% CI 5.31-148.85) and that their portals were a valuable part of their health care (OR 6.75, 95% CI 1.51-30.11) were also more likely to trust their portals. Conclusions: There are several factors that impact the trust of EHR patient portal users in their portals. Designing easily usable portals and considering these factors may be the most effective approach to improving trust in patient portals. The desired features and usability of portals are critical factors that contribute to users’ trust in EHR portals. %M 34546182 %R 10.2196/28501 %U https://humanfactors.jmir.org/2021/3/e28501 %U https://doi.org/10.2196/28501 %U http://www.ncbi.nlm.nih.gov/pubmed/34546182 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 8 %N 3 %P e21810 %T Machine Learning–Based Analysis of Encrypted Medical Data in the Cloud: Qualitative Study of Expert Stakeholders’ Perspectives %A Alaqra,Ala Sarah %A Kane,Bridget %A Fischer-Hübner,Simone %+ Computer Science and Information Systems, Karlstad University, Universitetsgatan 2, Karlstad, 65188, Sweden, 46 054 700 2815, as.alaqra@kau.se %K medical data analysis %K encryption %K privacy-enhancing technologies %K machine learning %K stakeholders %K tradeoffs %K perspectives %K eHealth %K interviews %D 2021 %7 16.9.2021 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Third-party cloud-based data analysis applications are proliferating in electronic health (eHealth) because of the expertise offered and their monetary advantage. However, privacy and security are critical concerns when handling sensitive medical data in the cloud. Technical advances based on “crypto magic” in privacy-preserving machine learning (ML) enable data analysis in encrypted form for maintaining confidentiality. Such privacy-enhancing technologies (PETs) could be counterintuitive to relevant stakeholders in eHealth, which could in turn hinder adoption; thus, more attention is needed on human factors for establishing trust and transparency. Objective: The aim of this study was to analyze eHealth expert stakeholders’ perspectives and the perceived tradeoffs in regard to data analysis on encrypted medical data in the cloud, and to derive user requirements for development of a privacy-preserving data analysis tool. Methods: We used semistructured interviews and report on 14 interviews with individuals having medical, technical, or research expertise in eHealth. We used thematic analysis for analyzing interview data. In addition, we conducted a workshop for eliciting requirements. Results: Our results show differences in the understanding of and in trusting the technology; caution is advised by technical experts, whereas patient safety assurances are required by medical experts. Themes were identified with general perspectives on data privacy and practices (eg, acceptance of using external services), as well as themes highlighting specific perspectives (eg, data protection drawbacks and concerns of the data analysis on encrypted data). The latter themes result in requiring assurances and conformance testing for trusting tools such as the proposed ML-based tool. Communicating privacy, and utility benefits and tradeoffs with stakeholders is essential for trust. Furthermore, stakeholders and their organizations share accountability of patient data. Finally, stakeholders stressed the importance of informing patients about the privacy of their data. Conclusions: Understanding the benefits and risks of using eHealth PETs is crucial, and collaboration among diverse stakeholders is essential. Assurances of the tool’s privacy, accuracy, and patient safety should be in place for establishing trust of ML-based PETs, especially if used in the cloud. %M 34528892 %R 10.2196/21810 %U https://humanfactors.jmir.org/2021/3/e21810 %U https://doi.org/10.2196/21810 %U http://www.ncbi.nlm.nih.gov/pubmed/34528892 %0 Journal Article %@ 2368-7959 %I JMIR Publications %V 8 %N 9 %P e30564 %T Implementation of Electronic Medical Records in Mental Health Settings: Scoping Review %A Zurynski,Yvonne %A Ellis,Louise A %A Tong,Huong Ly %A Laranjo,Liliana %A Clay-Williams,Robyn %A Testa,Luke %A Meulenbroeks,Isabelle %A Turton,Charmaine %A Sara,Grant %+ Centre for Healthcare Resilience and Implementation Science, Australian Institute of Health Innovation, Macquarie University, 75 Talavera Road, Sydney, Australia, 61 9850 ext 2414, Yvonne.Zurynski@mq.edu.au %K electronic medical records %K health information technology %K implementation %K mental health %D 2021 %7 7.9.2021 %9 Review %J JMIR Ment Health %G English %X Background: The success of electronic medical records (EMRs) is dependent on implementation features, such as usability and fit with clinical processes. The use of EMRs in mental health settings brings additional and specific challenges owing to the personal, detailed, narrative, and exploratory nature of the assessment, diagnosis, and treatment in this field. Understanding the determinants of successful EMR implementation is imperative to guide the future design, implementation, and investment of EMRs in the mental health field. Objective: We intended to explore evidence on effective EMR implementation for mental health settings and provide recommendations to support the design, adoption, usability, and outcomes. Methods: The scoping review combined two search strategies that focused on clinician-facing EMRs, one for primary studies in mental health settings and one for reviews of peer-reviewed literature in any health setting. Three databases (Medline, EMBASE, and PsycINFO) were searched from January 2010 to June 2020 using keywords to describe EMRs, settings, and impacts. The Proctor framework for implementation outcomes was used to guide data extraction and synthesis. Constructs in this framework include adoption, acceptability, appropriateness, feasibility, fidelity, cost, penetration, and sustainability. Quality assessment was conducted using a modified Hawker appraisal tool and the Joanna Briggs Institute Critical Appraisal Checklist for Systematic Reviews and Research Syntheses. Results: This review included 23 studies, namely 12 primary studies in mental health settings and 11 reviews. Overall, the results suggested that adoption of EMRs was impacted by financial, technical, and organizational factors, as well as clinician perceptions of appropriateness and acceptability. EMRs were perceived as acceptable and appropriate by clinicians if the system did not interrupt workflow and improved documentation completeness and accuracy. Clinicians were more likely to value EMRs if they supported quality of care, were fit for purpose, did not interfere with the clinician-patient relationship, and were operated with readily available technical support. Evidence on the feasibility of the implemented EMRs was mixed; the primary studies and reviews found mixed impacts on documentation quality and time; one primary study found downward trends in adverse events, whereas a review found improvements in care quality. Five papers provided information on implementation outcomes such as cost and fidelity, and none reported on the penetration and sustainability of EMRs. Conclusions: The body of evidence relating to EMR implementation in mental health settings is limited. Implementation of EMRs could benefit from methods used in general health settings such as co-designing the software and tailoring EMRs to clinical needs and workflows to improve usability and acceptance. Studies in mental health and general health settings rarely focused on long-term implementation outcomes such as penetration and sustainability. Future evaluations of EMRs in all settings should consider long-term impacts to address current knowledge gaps. %M 34491208 %R 10.2196/30564 %U https://mental.jmir.org/2021/9/e30564 %U https://doi.org/10.2196/30564 %U http://www.ncbi.nlm.nih.gov/pubmed/34491208 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 10 %N 8 %P e30431 %T Implementation of Patient Engagement Tools in Electronic Health Records to Enhance Patient-Centered Communication: Protocol for Feasibility Evaluation and Preliminary Results %A Tai-Seale,Ming %A Rosen,Rebecca %A Ruo,Bernice %A Hogarth,Michael %A Longhurst,Christopher A %A Lander,Lina %A Walker,Amanda L %A Stults,Cheryl D %A Chan,Albert %A Mazor,Kathleen %A Garber,Lawrence %A Millen,Marlene %+ Department of Family Medicine, University of California San Diego, 9500 Gilman Dr, La Jolla, CA, 92093, United States, 1 858 246 1939, mtaiseale@health.ucsd.edu %K electronic health record %K patient portal %K patient–physician communication %K agenda setting %K patient priorities %K patient engagement %K health care teams %K electronic health record documentation %K standard work %K digital health %D 2021 %7 26.8.2021 %9 Protocol %J JMIR Res Protoc %G English %X Background: Patient–physician communication during clinical encounters is essential to ensure quality of care. Many studies have attempted to improve patient–physician communication. Incorporating patient priorities into agenda setting and medical decision-making are fundamental to patient-centered communication. Efficient and scalable approaches are needed to empower patients to speak up and prepare physicians to respond. Leveraging electronic health records (EHRs) in engaging patients and health care teams has the potential to enhance the integration of patient priorities in clinical encounters. A systematic approach to eliciting and documenting patient priorities before encounters could facilitate effective communication in such encounters. Objective: In this paper, we report the design and implementation of a set of EHR tools built into clinical workflows for facilitating patient–physician joint agenda setting and the documentation of patient concerns in the EHRs for ambulatory encounters. Methods: We engaged health information technology leaders and users in three health care systems for developing and implementing a set of EHR tools. The goal of these tools is to standardize the elicitation of patient priorities by using a previsit “patient important issue” questionnaire distributed through the patient portal to the EHR. We built additional EHR documentation tools to facilitate patient–staff communication when the staff records the vital signs and the reason for the visit in the EHR while in the examination room, with a simple transmission method for physicians to incorporate patient concerns in EHR notes. Results: The study is ongoing. The anticipated completion date for survey data collection is November 2021. A total of 34,037 primary care patients from three health systems (n=26,441; n=5136; and n=2460 separately recruited from each system) used the previsit patient important issue questionnaire in 2020. The adoption of the digital previsit questionnaire during the COVID-19 pandemic was much higher in one health care system because it expanded the use of the questionnaire from physicians participating in trials to all primary care providers midway through the year. It also required the use of this previsit questionnaire for eCheck-ins, which are required for telehealth encounters. Physicians and staff suggested anecdotally that this questionnaire helped patient–clinician communication, particularly during the COVID-19 pandemic. Conclusions: EHR tools have the potential to facilitate the integration of patient priorities into agenda setting and documentation in real-world primary care practices. Early results suggest the feasibility and acceptability of such digital tools in three health systems. EHR tools can support patient engagement and clinicians’ work during in-person and telehealth visits. They could potentially exert a sustained influence on patient and clinician communication behaviors in contrast to prior ad hoc educational efforts targeting patients or clinicians. Trial Registration: ClinicalTrials.gov NCT03385512; https://clinicaltrials.gov/ct2/show/NCT03385512 International Registered Report Identifier (IRRID): DERR1-10.2196/30431 %M 34435960 %R 10.2196/30431 %U https://www.researchprotocols.org/2021/8/e30431 %U https://doi.org/10.2196/30431 %U http://www.ncbi.nlm.nih.gov/pubmed/34435960 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 8 %P e28245 %T Team Dynamics in Hospital Workflows: An Exploratory Study of a Smartphone Task Manager %A Hettiachchi,Danula %A Hayes,Lachie %A Goncalves,Jorge %A Kostakos,Vassilis %+ School of Computing and Information Systems, The University of Melbourne, Gratten St, Parkville, 3010, Australia, 61 474435815, dhettiachchi@student.unimelb.edu.au %K task assignment %K smartphones %K hospital communication %K clinical workflows %K mobile app %K clinical platform %K mHealth %D 2021 %7 16.8.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: Although convenient and reliable modern messaging apps like WhatsApp enable efficient communication among hospital staff, hospitals are now pivoting toward purpose-built structured communication apps for various reasons, including security and privacy concerns. However, there is limited understanding of how we can examine and improve hospital workflows using the data collected through such apps as an alternative to costly and challenging research methods like ethnography and patient record analysis. Objective: We seek to identify whether the structure of the collected communication data provides insights into hospitals’ workflows. Our analysis also aims to identify ways in which task management platforms can be improved and designed to better support clinical workflows. Methods: We present an exploratory analysis of clinical task records collected over 22 months through a smartphone app that enables structured communication between staff to manage and execute clinical workflows. We collected over 300,000 task records between July 2018 and May 2020 completed by staff members including doctors, nurses, and pharmacists across all wards in an Australian hospital. Results: We show that important insights into how teams function in a clinical setting can be readily drawn from task assignment data. Our analysis indicates that predefined labels such as urgency and task type are important and impact how tasks are accepted and completed. Our results show that both task sent-to-accepted (P<.001) and sent-to-completed (P<.001) times are significantly higher for routine tasks when compared to urgent tasks. We also show how task acceptance varies across teams and roles and that internal tasks are more efficiently managed than external tasks, possibly due to increased trust among team members. For example, task sent-to-accepted time (minutes) is significantly higher (P<.001) for external assignments (mean 22.10, SD 91.45) when compared to internal assignments (mean 19.03, SD 82.66). Conclusions: Smartphone-based task assignment apps can provide unique insights into team dynamics in clinical settings. These insights can be used to further improve how well these systems support clinical work and staff. %M 34398797 %R 10.2196/28245 %U https://medinform.jmir.org/2021/8/e28245 %U https://doi.org/10.2196/28245 %U http://www.ncbi.nlm.nih.gov/pubmed/34398797 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 8 %N 3 %P e26336 %T US Physicians’ Perspective on the Sudden Shift to Telehealth: Survey Study %A Walia,Bhavneet %A Shridhar,Anshu %A Arasu,Pratap %A Singh,Gursimar Kaur %+ Department of Public Health, Syracuse University, 430B Barclay, Syracuse, NY, 13244, United States, 1 3154433930, bwalia@syr.edu %K physician survey %K US telehealth training %K US telehealth care %K COVID-19 %K pandemic %K snowball sampling %K health care access %K health care quality %K telehealth %K telemedicine %K survey %K physician %K perspective %K recommendation %K policy %K public health %K implication %K quality %K access %D 2021 %7 12.8.2021 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Given the sudden shift to telemedicine during the early COVID-19 pandemic, we conducted a survey of practicing physicians’ experience with telehealth during the prepandemic and early pandemic periods. Our survey estimates that most patient visits in the United States during the early COVID-19 pandemic period were conducted via telehealth. Given this magnitude and the potential benefits and challenges of telehealth for the US health care system, in this paper, we obtain, summarize, and analyze telehealth views and experiences of US-based practicing-physicians. Objective: The aim of this study was to examine the extent of shift toward telehealth training and care provision during the early pandemic from the US-based practicing physicians’ perspective. We also sought to determine the short- and long-term implications of this shift on the quality, access, and mode of US health care delivery. Methods: We conducted a purposive, snowball-sampled survey of US practicing-physicians. A total of 148 physician completed the survey. Data were collected from July 17, 2020, through September 4, 2020. Results: Sample training intensity scaled 21-fold during the early pandemic period, and patient-care visits conducted via telehealth increased, on average, from 13.1% directly before the pandemic to 59.7% during the early pandemic period. Surveyed physician respondents reported that telehealth patient visits and face-to-face patient visits are comparable in quality. The difference was not statistically significant based on a nonparametric sign test (P=.11). Moreover, physicians feel that telehealth care should continue to play a larger role (44.9% of total visits) in postpandemic health care in the United States. Our survey findings suggest a high market concentration in telehealth software, which is a market structural characteristic that may have implications on the cost and access of telehealth. The results varied markedly by physician employer type. Conclusions: During the shift toward telehealth, there has been a considerable discovery among physicians regarding US telehealth physicians. Physicians are now better prepared to undertake telehealth care from a training perspective. They are favorable toward a permanently expanded telehealth role, with potential for enhanced health care access, and the realization of enhanced access may depend on market structural characteristics of telehealth software platforms. %M 33938813 %R 10.2196/26336 %U https://humanfactors.jmir.org/2021/3/e26336 %U https://doi.org/10.2196/26336 %U http://www.ncbi.nlm.nih.gov/pubmed/33938813 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 8 %N 3 %P e25568 %T Optimizing e-Consultations to Adolescent Medicine Specialists: Qualitative Synthesis of Feedback From User-Centered Design %A Rankine,Jacquelin %A Yeramosu,Deepika %A Matheo,Loreta %A Sequeira,Gina M %A Miller,Elizabeth %A Ray,Kristin N %+ Division of Adolescent and Young Adult Medicine, Department of Pediatrics, University of Pittsburgh School of Medicine, UPMC Children's Hospital of Pittsburgh, 120 Lytton Avenue, Pittsburgh, PA, 15213, United States, 1 412 692 7711, jacquelin.rankine@chp.edu %K referral %K consultation %K telemedicine %K telehealth %K adolescents %K child health %K child health services %K confidentiality %K access to health care %D 2021 %7 5.8.2021 %9 Original Paper %J JMIR Hum Factors %G English %X Background: e-Consultations between primary care physicians and specialists are a valuable means of improving access to specialty care. Adolescents and young adults (AYAs) face unique challenges in accessing limited adolescent medicine specialty care resources, which contributes to delayed or forgone care. e-Consultations between general pediatricians and adolescent medicine specialists may alleviate these barriers to care. However, the optimal application of this model in adolescent medicine requires careful attention to the nuances of AYA care. Objective: This study aims to qualitatively analyze feedback obtained during the iterative development of an e-consultation system for communication between general pediatricians and adolescent medicine specialists tailored to the specific health care needs of AYAs. Methods: We conducted an iterative user-centered design and evaluation process in two phases. In the first phase, we created a static e-consultation prototype and storyboards and evaluated them with target users (general pediatricians and adolescent medicine specialists). In the second phase, we incorporated feedback to develop a functional prototype within the electronic health record and again evaluated this with general pediatricians and adolescent medicine specialists. In each phase, general pediatricians and adolescent medicine specialists provided think-aloud feedback during the use of the prototypes and semistructured exit interviews, which was qualitatively analyzed to identify perspectives related to the usefulness and usability of the e-consultation system. Results: Both general pediatricians (n=12) and adolescent medicine specialists (n=12) perceived the usefulness of e-consultations for AYA patients, with more varied perceptions of potential usefulness for generalist and adolescent medicine clinicians. General pediatricians and adolescent medicine specialists discussed ways to maximize the usability of e-consultations for AYAs, primarily by improving efficiency (eg, reducing documentation, emphasizing critical information, using autopopulated data fields, and balancing specificity and efficiency through text prompts) and reducing the potential for errors (eg, prompting a review of autopopulated data fields, requiring physician contact information, and prompting explicit discussion of patient communication and confidentiality expectations). Through iterative design, patient history documentation was streamlined, whereas documentation of communication and confidentiality expectations were enhanced. Conclusions: Through an iterative user-centered design process, we identified user perspectives to guide the refinement of an e-consultation system based on general pediatrician and adolescent medicine specialist feedback on usefulness and usability related to the care of AYAs. Qualitative analysis of this feedback revealed both opportunities and risks related to confidentiality, communication, and the use of tailored documentation prompts that should be considered in the development and use of e-consultations with AYAs. %M 34383665 %R 10.2196/25568 %U https://humanfactors.jmir.org/2021/3/e25568 %U https://doi.org/10.2196/25568 %U http://www.ncbi.nlm.nih.gov/pubmed/34383665 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 8 %N 3 %P e25046 %T Barriers to the Use of Clinical Decision Support for the Evaluation of Pulmonary Embolism: Qualitative Interview Study %A Richardson,Safiya %A Dauber-Decker,Katherine L %A McGinn,Thomas %A Barnaby,Douglas P %A Cattamanchi,Adithya %A Pekmezaris,Renee %+ Donald and Barbara Zucker School of Medicine at Hofstra/Northwell, 600 Community Drive, Suite 403, Manhasset, NY, 11030, United States, 1 5166001411, srichard12@northwell.edu %K medical informatics %K pulmonary embolism %K electronic health records %K quality improvement %K clinical decision support systems %D 2021 %7 4.8.2021 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Clinicians often disregard potentially beneficial clinical decision support (CDS). Objective: In this study, we sought to explore the psychological and behavioral barriers to the use of a CDS tool. Methods: We conducted a qualitative study involving emergency medicine physicians and physician assistants. A semistructured interview guide was created based on the Capability, Opportunity, and Motivation-Behavior model. Interviews focused on the barriers to the use of a CDS tool built based on Wells’ criteria for pulmonary embolism to assist clinicians in establishing pretest probability of pulmonary embolism before imaging. Results: Interviews were conducted with 12 clinicians. Six barriers were identified, including (1) Bayesian reasoning, (2) fear of missing a pulmonary embolism, (3) time pressure or cognitive load, (4) gestalt includes Wells’ criteria, (5) missed risk factors, and (6) social pressure. Conclusions: Clinicians highlighted several important psychological and behavioral barriers to CDS use. Addressing these barriers will be paramount in developing CDS that can meet its potential to transform clinical care. %M 34346901 %R 10.2196/25046 %U https://humanfactors.jmir.org/2021/3/e25046 %U https://doi.org/10.2196/25046 %U http://www.ncbi.nlm.nih.gov/pubmed/34346901 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 5 %N 7 %P e27992 %T Clinical Utility and Functionality of an Artificial Intelligence–Based App to Predict Mortality in COVID-19: Mixed Methods Analysis %A Abdulaal,Ahmed %A Patel,Aatish %A Al-Hindawi,Ahmed %A Charani,Esmita %A Alqahtani,Saleh A %A Davies,Gary W %A Mughal,Nabeela %A Moore,Luke Stephen Prockter %+ National Institute for Health Research Health Protection Research Unit in Healthcare Associated Infections and Antimicrobial Resistance, Imperial College London, Commonwealth Building 8th Floor, Du Cane Road, London, W12 0NN, United Kingdom, 44 2033158273, l.moore@imperial.ac.uk %K app %K artificial intelligence %K coronavirus %K COVID-19 %K development %K function %K graphical user interface %K machine learning %K model %K mortality %K neural network %K prediction %K usability %K utility %D 2021 %7 28.7.2021 %9 Original Paper %J JMIR Form Res %G English %X Background: The artificial neural network (ANN) is an increasingly important tool in the context of solving complex medical classification problems. However, one of the principal challenges in leveraging artificial intelligence technology in the health care setting has been the relative inability to translate models into clinician workflow. Objective: Here we demonstrate the development of a COVID-19 outcome prediction app that utilizes an ANN and assesses its usability in the clinical setting. Methods: Usability assessment was conducted using the app, followed by a semistructured end-user interview. Usability was specified by effectiveness, efficiency, and satisfaction measures. These data were reported with descriptive statistics. The end-user interview data were analyzed using the thematic framework method, which allowed for the development of themes from the interview narratives. In total, 31 National Health Service physicians at a West London teaching hospital, including foundation physicians, senior house officers, registrars, and consultants, were included in this study. Results: All participants were able to complete the assessment, with a mean time to complete separate patient vignettes of 59.35 (SD 10.35) seconds. The mean system usability scale score was 91.94 (SD 8.54), which corresponds to a qualitative rating of “excellent.” The clinicians found the app intuitive and easy to use, with the majority describing its predictions as a useful adjunct to their clinical practice. The main concern was related to the use of the app in isolation rather than in conjunction with other clinical parameters. However, most clinicians speculated that the app could positively reinforce or validate their clinical decision-making. Conclusions: Translating artificial intelligence technologies into the clinical setting remains an important but challenging task. We demonstrate the effectiveness, efficiency, and system usability of a web-based app designed to predict the outcomes of patients with COVID-19 from an ANN. %M 34115603 %R 10.2196/27992 %U https://formative.jmir.org/2021/7/e27992 %U https://doi.org/10.2196/27992 %U http://www.ncbi.nlm.nih.gov/pubmed/34115603 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 8 %N 3 %P e24927 %T User Evaluation of the Swedish Patient Accessible Electronic Health Record: System Usability Scale %A Hägglund,Maria %A Scandurra,Isabella %+ Healthcare Sciences and e-Health, Department of Women's and Children's Health, Uppsala University, Dag Hammarskjölds väg 14B, 1 tr, Uppsala, 752 37, Sweden, 46 0729999381, maria.hagglund@kbh.uu.se %K usability %K system usability scale %K evaluation %K patient accessible electronic health records %K open notes %K patient portals %D 2021 %7 27.7.2021 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Transparency is increasingly called for in health care, especially, when it comes to patients’ access to their electronic health records. In Sweden, the e-service Journalen is a national patient accessible electronic health record (PAEHR), accessible online via the national patient portal. User characteristics and perceived benefits of using a PAEHR influence behavioral intention for use and adoption, but poor usability that increases the effort expectancy can have a negative impact. It is, therefore, of interest to explore how users of the PAEHR Journalen perceive its usability and usefulness. Objective: The aim of this study was to explore how the users of the Swedish PAEHR experience the usability of the system and to identify differences in these experiences based on the level of transparency of the region. Methods: A survey study was conducted to elicit opinions and experiences of patients using Journalen. The data were collected from June to October 2016. The questionnaire included questions regarding the usability of the system from the System Usability Scale (SUS). The SUS analysis was the focus of this paper. Analysis was performed on different levels: nationally looking at the whole data set and breaking it down by focusing on 2 different regions to explore differences in experienced usability based on the level of transparency. Results: During the survey period, 423,141 users logged into Journalen, of which 2587 unique users completed the survey (response rate 0.61%). The total mean score for all respondents to the SUS items was 79.81 (SD 14.25), which corresponds to a system with good usability. To further explore whether the level of transparency in a region would affect the user’s experience of the usability of the system, we analyzed the 2 regions with the most respondents: Region Uppsala (the first to launch, with a high level of transparency), and Region Skåne (an early implementer, with a low level of transparency at the time of the survey). Of the participants who responded to at least 1 SUS statement, 520 stated that they had received care in Region Skåne, whereas 331 participants had received care in Region Uppsala. Uppsala’s mean SUS score was 80.71 (SD 13.41), compared with Skåne’s mean of 79.37 (SD 13.78). Conclusions: The Swedish national PAEHR Journalen has a reasonably good usability (mean SUS score 79.81, SD 14.25); however, further research into more specific usability areas are needed to ensure usefulness and ease of use in the future. A somewhat higher SUS score for the region with high transparency compared with the region with low transparency could indicate a relationship between the perceived usability of a PAEHR and the level of transparency offered, but further research on the relationship between transparency and usability is required. %M 34313596 %R 10.2196/24927 %U https://humanfactors.jmir.org/2021/3/e24927 %U https://doi.org/10.2196/24927 %U http://www.ncbi.nlm.nih.gov/pubmed/34313596 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 7 %P e24022 %T A Tool for Evaluating Medication Alerting Systems: Development and Initial Assessment %A Zheng,Wu Yi %A Van Dort,Bethany %A Marcilly,Romaric %A Day,Richard %A Burke,Rosemary %A Shakib,Sepehr %A Ku,Young %A Reid-Anderson,Hannah %A Baysari,Melissa %+ Black Dog Institute, Hospital Rd, Prince of Wales Hospital, Randwick, NSW, 2031, Australia, 61 422510718, wuyi.zheng@unsw.edu.au %K medication alerts %K decision support %K human factors %K assessment tool %K usability flaws %D 2021 %7 16.7.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: It is well known that recommendations from electronic medication alerts are seldom accepted or acted on by users. Key factors affecting the effectiveness of medication alerts include system usability and alert design. Thus, human factors principles that apply knowledge of human capabilities and limitations are increasingly used in the design of health technology to improve the usability of systems. Objective: This study aims to evaluate a newly developed evidence-based self-assessment tool that allows the valid and reliable evaluation of computerized medication alerting systems. This tool was developed to be used by hospital staff with detailed knowledge of their hospital’s computerized provider order entry system and alerts to identify and address potential system deficiencies. In this initial assessment, we aim to determine whether the items in the tool can measure compliance of medication alerting systems with human factors principles of design, the tool can be consistently used by multiple users to assess the same system, and the items are easy to understand and perceived to be useful for assessing medication alerting systems. Methods: The Tool for Evaluating Medication Alerting Systems (TEMAS) was developed based on human factors design principles and consisted of 66 items. In total, 18 staff members recruited across 6 hospitals used the TEMAS to assess their medication alerting systems. Data collected from participant assessments were used to evaluate the validity, reliability, and usability of the TEMAS. Validity was assessed by comparing the results of the TEMAS with those of prior in-house evaluations. Reliability was measured using Krippendorff α to determine agreement among assessors. A 7-item survey was used to determine usability. Results: The participants reported mostly negative (n=8) and neutral (n=7) perceptions of alerts in their medication alerting system. However, the validity of the TEMAS could not be directly tested, as participants were unaware of any results from prior in-house evaluations. The reliability of the TEMAS, as measured by Krippendorff α, was low to moderate (range 0.26-0.46); however, participant feedback suggests that individuals’ knowledge of the system varied according to their professional background. In terms of usability, 61% (11/18) of participants reported that the TEMAS items were generally easy to understand; however, participants suggested the revision of 22 items to improve clarity. Conclusions: This initial assessment of the TEMAS allowed the identification of its components that required modification to improve usability and usefulness. It also revealed that for the TEMAS to be effective in facilitating a comprehensive assessment of a medication alerting system, it should be completed by a multidisciplinary team of hospital staff from both clinical and technical backgrounds to maximize their knowledge of systems. %M 34269680 %R 10.2196/24022 %U https://medinform.jmir.org/2021/7/e24022 %U https://doi.org/10.2196/24022 %U http://www.ncbi.nlm.nih.gov/pubmed/34269680 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 5 %N 7 %P e22110 %T Evaluating Simplified Web Interfaces of Risk Models for Clinical Use: Pilot Survey Study %A Beaubien,Louis %A Conrad,Colin %A Music,Janet %A Toze,Sandra %+ Rowe School of Business, Faculty of Management, Dalhousie University, Kenneth C Rowe Management Building, 6100 University Ave, Halifax, NS, B3H 4R2, Canada, 1 902 494 7080, louis.beaubien@dal.ca %K risk model %K electronic records %K user interface %K technology acceptance %D 2021 %7 16.7.2021 %9 Original Paper %J JMIR Form Res %G English %X Background: In this pilot study, we investigated sociotechnical factors that affect intention to use a simplified web model to support clinical decision making. Objective: We investigated factors that are known to affect technology adoption using the unified theory of acceptance and use of technology (UTAUT2) model. The goal was to pilot and test a tool to better support complex clinical assessments. Methods: Based on the results of a previously published work, we developed a web-based mobile user interface, WebModel, to allow users to work with regression equations and their predictions to evaluate the impact of various characteristics or treatments on key outcomes (eg, survival time) for chronic obstructive pulmonary disease. The WebModel provides a way to combat information overload and more easily compare treatment options. It limits the number of web forms presented to a user to between 1 and 20, rather than the dozens of detailed calculations typically required. The WebModel uses responsive design and can be used on multiple devices. To test the WebModel, we designed a questionnaire to probe the efficacy of the WebModel and assess the usability and usefulness of the system. The study was live for one month, and participants had access to it over that time. The questionnaire was administered online, and data from 674 clinical users who had access to the WebModel were captured. SPSS and R were used for statistical analysis. Results: The regression model developed from UTAUT2 constructs was a fit. Specifically, five of the seven factors were significant positive coefficients in the regression: performance expectancy (β=.2730; t=7.994; P<.001), effort expectancy (β=.1473; t=3.870; P=.001), facilitating conditions (β=.1644; t=3.849; P<.001), hedonic motivation (β=.2321; t=3.991; P<.001), and habit (β=.2943; t=12.732). Social influence was not a significant factor, while price value had a significant negative influence on intention to use the WebModel. Conclusions: Our results indicate that multiple influences impact positive response to the system, many of which relate to the efficiency of the interface to provide clear information. Although we found that the price value was a negative factor, it is possible this was due to the removal of health workers from purchasing decisions. Given that this was a pilot test, and that the system was not used in a clinical setting, we could not examine factors related to actual workflow, patient safety, or social influence. This study shows that the concept of a simplified WebModel could be effective and efficient in reducing information overload in complex clinical decision making. We recommend further study to test this in a clinical setting and gather qualitative data from users regarding the value of the tool in practice. %M 34269692 %R 10.2196/22110 %U https://formative.jmir.org/2021/7/e22110 %U https://doi.org/10.2196/22110 %U http://www.ncbi.nlm.nih.gov/pubmed/34269692 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 8 %N 3 %P e29690 %T Clinician Perspectives on Telemedicine: Observational Cross-sectional Study %A Alkureishi,Maria Alcocer %A Choo,Zi-Yi %A Lenti,Gena %A Castaneda,Jason %A Zhu,Mengqi %A Nunes,Kenneth %A Weyer,George %A Oyler,Julie %A Shah,Sachin %A Lee,Wei Wei %+ Department of Academic Medicine, University of Chicago, 5841 S Maryland Av, MC 6082 Rm C124, Chicago, IL, 60637, United States, 1 773 834 8927, malkureishi@peds.bsd.uchicago.edu %K telemedicine %K clinician perspective %K patient-centered care %K burnout %K trainee %K outpatient %K workflow %K virtual health %K training %K human factors %D 2021 %7 9.7.2021 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Since the COVID-19 pandemic onset, telemedicine has increased exponentially across numerous outpatient departments and specialties. Qualitative studies examining clinician telemedicine perspectives during the pandemic identified challenges with physical examination, workflow concerns, burnout, and reduced personal connection with patients. However, these studies only included a relatively small number of physicians or were limited to a single specialty, and few assessed perspectives on integrating trainees into workflows, an important area to address to support the clinical learning environment. As telemedicine use continues, it is necessary to understand a range of clinician perspectives. Objective: This study aims to survey pediatric and adult medicine clinicians at the University of Chicago Medical Center to understand their telemedicine benefits and barriers, workflow impacts, and training and support needs. Methods: In July 2020, we conducted an observational cross-sectional study of University of Chicago Medical Center faculty and advanced practice providers in the Department of Medicine (DOM) and Department of Pediatrics (DOP). Results: The overall response rate was 39% (200/517; DOM: 135/325, 42%; DOP: 65/192, 34%); most respondents were physicians (DOM: 100/135, 74%; DOP: 51/65, 79%). One-third took longer to prepare for (65/200, 33%) and conduct (62/200, 32%) video visits compared to in-person visits. Male clinicians reported conducting a higher percentage of telemedicine visits by video than their female counterparts (P=.02), with no differences in the number of half-days per week providing direct outpatient care or supervising trainees. Further, clinicians who conducted a higher percentage of their telemedicine by video were less likely to feel overwhelmed (P=.02), with no difference in reported burnout. Female clinicians were “more overwhelmed” with video visits compared to males (41/130, 32% vs 12/64, 19%; P=.05). Clinicians 50 years or older were “less overwhelmed” than those younger than 50 years (30/85, 35% vs 23/113, 20%; P=.02). Those who received more video visit training modalities (eg, a document and webinar on technical issues) were less likely to feel overwhelmed by the conversion to video visits (P=.007) or burnt out (P=.009). In addition, those reporting a higher ability to technically navigate a video visit were also less likely to feel overwhelmed by video visits (P=.02) or burnt out (P=.001). The top telemedicine barriers were patient-related: lack of technology access, lack of skill, and reluctance. Training needs to be focused on integrating learners into workflows. Open-ended responses highlighted a need for increased support staff. Overall, more than half “enjoyed conducting video visits” (119/200, 60%) and wanted to continue using video visits in the future (150/200, 75%). Conclusions: Despite positive telemedicine experiences, more support to facilitate video visits for patients and clinicians is needed. Further, clinicians need additional training on trainee education and integration into workflows. Further work is needed to better understand why gender and age differences exist. In conclusion, interventions to address clinician and patient barriers, and enhance clinician training are needed to support telemedicine’s durability. %M 34184994 %R 10.2196/29690 %U https://humanfactors.jmir.org/2021/3/e29690 %U https://doi.org/10.2196/29690 %U http://www.ncbi.nlm.nih.gov/pubmed/34184994 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 7 %P e26817 %T Frustration With Technology and its Relation to Emotional Exhaustion Among Health Care Workers: Cross-sectional Observational Study %A Tawfik,Daniel S %A Sinha,Amrita %A Bayati,Mohsen %A Adair,Kathryn C %A Shanafelt,Tait D %A Sexton,J Bryan %A Profit,Jochen %+ Department of Pediatrics, Stanford University School of Medicine, 770 Welch Road, Suite 435, Palo Alto, CA, 94304, United States, 1 650 723 9902, dtawfik@stanford.edu %K frustration with technology %K emotional exhaustion %K professional burnout %K work-life integration %K biomedical technology %K work-life balance %K user-centered design %K electronic health records %K medical informatics applications %K health information systems %D 2021 %7 6.7.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: New technology adoption is common in health care, but it may elicit frustration if end users are not sufficiently considered in their design or trained in their use. These frustrations may contribute to burnout. Objective: This study aimed to evaluate and quantify health care workers’ frustration with technology and its relationship with emotional exhaustion, after controlling for measures of work-life integration that may indicate excessive job demands. Methods: This was a cross-sectional, observational study of health care workers across 31 Michigan hospitals. We used the Safety, Communication, Operational Reliability, and Engagement (SCORE) survey to measure work-life integration and emotional exhaustion among the survey respondents. We used mixed-effects hierarchical linear regression to evaluate the relationship among frustration with technology, other components of work-life integration, and emotional exhaustion, with adjustment for unit and health care worker characteristics. Results: Of 15,505 respondents, 5065 (32.7%) reported that they experienced frustration with technology on at least 3-5 days per week. Frustration with technology was associated with higher scores for the composite Emotional Exhaustion scale (r=0.35, P<.001) and each individual item on the Emotional Exhaustion scale (r=0.29-0.36, P<.001 for all). Each 10-point increase in the frustration with technology score was associated with a 1.2-point increase (95% CI 1.1-1.4) in emotional exhaustion (both measured on 100-point scales), after adjustment for other work-life integration items and unit and health care worker characteristics. Conclusions: This study found that frustration with technology and several other markers of work-life integration are independently associated with emotional exhaustion among health care workers. Frustration with technology is common but not ubiquitous among health care workers, and it is one of several work-life integration factors associated with emotional exhaustion. Minimizing frustration with health care technology may be an effective approach in reducing burnout among health care workers. %M 34255674 %R 10.2196/26817 %U https://www.jmir.org/2021/7/e26817 %U https://doi.org/10.2196/26817 %U http://www.ncbi.nlm.nih.gov/pubmed/34255674 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 7 %P e28729 %T The Association Between Using a Mobile Version of an Electronic Health Record and the Well-Being of Nurses: Cross-sectional Survey Study %A Heponiemi,Tarja %A Kaihlanen,Anu-Marja %A Gluschkoff,Kia %A Saranto,Kaija %A Nissinen,Sari %A Laukka,Elina %A Vehko,Tuulikki %+ Department of Public Health and Welfare, Finnish Institute for Health and Welfare, PO Box 30, Helsinki, 00271, Finland, 358 295247434, tarja.heponiemi@thl.fi %K stress related to information systems %K time pressure %K usability %K stress %K health and social care %D 2021 %7 6.7.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: Mobile devices such as tablets and smartphones are increasingly being used in health care in many developed countries. Nurses form the largest group in health care that uses electronic health records (EHRs) and their mobile versions. Mobile devices are suggested to promote nurses’ workflow, constant updating of patient information, and improve the communication within the health care team. However, little is known about their effect on nurses’ well-being. Objective: This study aimed to examine the association between using a mobile version of the EHR and nurses’ perceived time pressure, stress related to information systems, and self-rated stress. Moreover, we examined whether mobile device use modifies the associations of EHR usability (ease of use and technical quality), experience in using EHRs, and number of systems in daily use with these well-being indicators. Methods: This was a cross-sectional population-based survey study among 3610 Finnish registered nurses gathered in 2020. The aforesaid associations were examined using analyses of covariance and logistic regression adjusted for age, gender, and employment sector (hospital, primary care, social service, and other). Results: Nurses who used the mobile version of their EHR had higher levels of time pressure (F1,3537=14.96, P<.001) and stress related to information systems (F1,3537=6.11, P=.01), compared with those who did not use mobile versions. Moreover, the interactions of mobile device use with experience in using EHRs (F1,3581=14.93, P<.001), ease of use (F1,3577=10.16, P=.001), and technical quality (F1,3577=6.45, P=.01) were significant for stress related to information systems. Inexperience in using EHRs, low levels of ease of use, and technical quality were associated with higher stress related to information systems and this association was more pronounced among those who used mobile devices. That is, the highest levels of stress related to information systems were perceived among those who used mobile devices as well as among inexperienced EHR users or those who perceived usability problems in their EHRs. Conclusions: According to our results, it seems that at present mobile device use is not beneficial for the nurses’ well-being. In addition, mobile device use seems to intensify the negative effects of usability issues related to EHRs. In particular, inexperienced users of EHRs seem to be at a disadvantage when using mobile devices. Thus, we suggest that EHRs and their mobile versions should be improved such that they would be easier to use and would better support the nurses’ workflow (eg, improvements to problems related to small display, user interface, and data entry). Moreover, additional training on EHRs, their mobile versions, and workflow related to these should be provided to nurses. %M 34255704 %R 10.2196/28729 %U https://medinform.jmir.org/2021/7/e28729 %U https://doi.org/10.2196/28729 %U http://www.ncbi.nlm.nih.gov/pubmed/34255704 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 8 %N 2 %P e25957 %T Sex Differences in Electronic Health Record Navigation Strategies: Secondary Data Analysis %A Seifer,Daniel R %A Mcgrath,Karess %A Scholl,Gretchen %A Mohan,Vishnu %A Gold,Jeffrey A %+ Division of Pulmonary and Critical Care Medicine, Oregon Health & Science University, 3181 SW Sam Jackson Park Rd, Portland, OR, 97239, United States, 1 5034181496, goldje@ohsu.edu %K eye tracking %K electronic health record %K gaze %K usability %K sex differences %D 2021 %7 24.6.2021 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Use of electronic health records (EHRs) has increased dramatically over the past decade. Their widespread adoption has been plagued with numerous complaints about usability, with subsequent impacts on patient safety and provider well-being. Data in other fields suggest biological sex impacts basic patterns of navigation in electronic media. Objective: This study aimed to determine whether biological sex impacted physicians’ navigational strategies while using EHRs. Methods: This is a secondary analysis of a prior study where physicians were given verbal and written signout, and then, while being monitored with an eye tracker, were asked to review a simulated record in our institution’s EHR system, which contained 14 patient safety items. Afterward, the number of safety items recognized was recorded. Results: A total of 93 physicians (female: n=46, male: n=47) participated in the study. Two gaze patterns were identified: one characterized more so by saccadic (“scanning”) eye movements and the other characterized more so by longer fixations (“staring”). Female physicians were more likely to use the scanning pattern; they had a shorter mean fixation duration (P=.005), traveled more distance per minute of screen time (P=.03), had more saccades per minute of screen time (P=.02), and had longer periods of saccadic movement (P=.03). The average proportion of time spent staring compared to scanning (the Gaze Index [GI]) across all participants was approximately 3:1. Females were more likely than males to have a GI value <3.0 (P=.003). At the extremes, males were more likely to have a GI value >5, while females were more likely to have a GI value <1. Differences in navigational strategy had no impact on task performance. Conclusions: Females and males demonstrate fundamentally different navigational strategies while navigating the EHR. This has potentially significant impacts for usability testing in EHR training and design. Further studies are needed to determine if the detected differences in gaze patterns produce meaningful differences in cognitive load while using EHRs. %M 34184995 %R 10.2196/25957 %U https://humanfactors.jmir.org/2021/2/e25957/ %U https://doi.org/10.2196/25957 %U http://www.ncbi.nlm.nih.gov/pubmed/34184995 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 6 %P e25741 %T A Web Interface for Antibiotic Prescription Recommendations in Primary Care: User-Centered Design Approach %A Madar,Ronni %A Ugon,Adrien %A Ivanković,Damir %A Tsopra,Rosy %+ Inserm, Université de Paris, Sorbonne Université, Centre de Recherche des Cordeliers, Information Sciences to support Personalized Medicine, 15 rue de l'école de médecine, Paris, 75006, France, 33 56092167, rosy.tsopra@nhs.net %K clinical decision support system %K visualization %K usability %K clinical practice guidelines %K antibiotic %K primary care %D 2021 %7 11.6.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: Antibiotic misuse is a serious public health problem worldwide. National health authorities release clinical practice guidelines (CPGs) to guide general practitioners (GPs) in their choice of antibiotics. However, despite the large-scale dissemination of CPGs, GPs continue to prescribe antibiotics that are not recommended as first-line treatments. This nonadherence to recommendations may be due to GPs misunderstanding the CPGs. A web interface displaying antibiotic prescription recommendations and their justifications could help to improve the comprehensibility and readability of CPGs, thereby increasing the adoption of recommendations regarding antibiotic treatment. Objective: This study aims to design and evaluate a web interface for antibiotic prescription displaying both the recommended antibiotics and their justifications in the form of antibiotic properties. Methods: A web interface was designed according to the same principles as e-commerce interfaces and was assessed by 117 GPs. These GPs were asked to answer 17 questions relating to the usefulness, user-friendliness, and comprehensibility and readability of the interface, and their satisfaction with it. Responses were recorded on a 4-point Likert scale (ranging from “absolutely disagree” to “absolutely agree”). At the end of the evaluation, the GPs were allowed to provide optional, additional free comments. Results: The antibiotic prescription web interface consists of three main sections: a clinical summary section, a filter section, and a recommended antibiotics section. The majority of GPs appreciated the clinical summary (90/117, 76.9%) and filter (98/117, 83.8%) sections, whereas 48.7% (57/117) of them reported difficulty reading some of the icons in the recommended antibiotics section. Overall, 82.9% (97/117) of GPs found the display of drug properties useful, and 65.8% (77/117) reported that the web interface improved their understanding of CPG recommendations. Conclusions: The web interface displaying antibiotic recommendations and their properties can help doctors understand the rationale underlying CPG recommendations regarding antibiotic treatment, but further improvements are required before its implementation into a clinical decision support system. %M 34114958 %R 10.2196/25741 %U https://www.jmir.org/2021/6/e25741 %U https://doi.org/10.2196/25741 %U http://www.ncbi.nlm.nih.gov/pubmed/34114958 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 8 %N 2 %P e25787 %T Comparing Single-Page, Multipage, and Conversational Digital Forms in Health Care: Usability Study %A Iftikhar,Aleeha %A Bond,Raymond R %A McGilligan,Victoria %A Leslie,Stephen J %A Rjoob,Khaled %A Knoery,Charles %A Quigg,Ciara %A Campbell,Ryan %A Boyd,Kyle %A McShane,Anne %A Peace,Aaron %+ Computing Engineering and Build Environment, Ulster University, Shore Rd, Newtownabbey, Jordanstown, BT37 0QB, United Kingdom, 44 07496635353, iftikhar-a1@ulster.ac.uk %K digital forms %K health care %K usability evaluation %K single-page form %K multipage form %K conversational forms %D 2021 %7 26.5.2021 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Even in the era of digital technology, several hospitals still rely on paper-based forms for data entry for patient admission, triage, drug prescriptions, and procedures. Paper-based forms can be quick and convenient to complete but often at the expense of data quality, completeness, sustainability, and automated data analytics. Digital forms can improve data quality by assisting the user when deciding on the appropriate response to certain data inputs (eg, classifying symptoms). Greater data quality via digital form completion not only helps with auditing, service improvement, and patient record keeping but also helps with novel data science and machine learning research. Although digital forms are becoming more prevalent in health care, there is a lack of empirical best practices and guidelines for their design. The study-based hospital had a definite plan to abolish the paper form; hence, it was not necessary to compare the digital forms with the paper form. Objective: This study aims to assess the usability of three different interactive forms: a single-page digital form (in which all data input is required on one web page), a multipage digital form, and a conversational digital form (a chatbot). Methods: The three digital forms were developed as candidates to replace the current paper-based form used to record patient referrals to an interventional cardiology department (Cath-Lab) at Altnagelvin Hospital. We recorded usability data in a counterbalanced usability test (60 usability tests: 20 subjects×3 form usability tests). The usability data included task completion times, System Usability Scale (SUS) scores, User Experience Questionnaire data, and data from a postexperiment questionnaire. Results: We found that the single-page form outperformed the other two digital forms in almost all usability metrics. The mean SUS score for the single-page form was 76 (SD 15.8; P=.01) when compared with the multipage form, which had a mean score of 67 (SD 17), and the conversational form attained the lowest scores in usability testing and was the least preferred choice of users, with a mean score of 57 (SD 24). An SUS score of >68 was considered above average. The single-page form achieved the least task completion time compared with the other two digital form styles. Conclusions: In conclusion, the digital single-page form outperformed the other two forms in almost all usability metrics; it had the least task completion time compared with those of the other two digital forms. Moreover, on answering the open-ended question from the final customized postexperiment questionnaire, the single-page form was the preferred choice. %M 34037531 %R 10.2196/25787 %U https://humanfactors.jmir.org/2021/2/e25787 %U https://doi.org/10.2196/25787 %U http://www.ncbi.nlm.nih.gov/pubmed/34037531 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 8 %N 2 %P e16491 %T Augmenting Critical Care Patient Monitoring Using Wearable Technology: Review of Usability and Human Factors %A Andrade,Evismar %A Quinlan,Leo %A Harte,Richard %A Byrne,Dara %A Fallon,Enda %A Kelly,Martina %A Casey,Siobhan %A Kirrane,Frank %A O'Connor,Paul %A O'Hora,Denis %A Scully,Michael %A Laffey,John %A Pladys,Patrick %A Beuchée,Alain %A ÓLaighin,Gearoid %+ Human Movement Laboratory, CÚRAM Centre for Research in Medical Devices, National University of Ireland, Galway, University Road, Galway, Ireland, 353 9149 ext 3710, leo.quinlan@nuigalway.ie %K patient monitor %K physiologic monitor %K human factors %K ergonomics %K usability %K user experience %K wearable %K mobile phone %K critical care %D 2021 %7 25.5.2021 %9 Review %J JMIR Hum Factors %G English %X Background: Continuous monitoring of the vital signs of critical care patients is an essential component of critical care medicine. For this task, clinicians use a patient monitor (PM), which conveys patient vital sign data through a screen and an auditory alarm system. Some limitations with PMs have been identified in the literature, such as the need for visual contact with the PM screen, which could result in reduced focus on the patient in specific scenarios, and the amount of noise generated by the PM alarm system. With the advancement of material science and electronic technology, wearable devices have emerged as a potential solution for these problems. This review presents the findings of several studies that focused on the usability and human factors of wearable devices designed for use in critical care patient monitoring. Objective: The aim of this study is to review the current state of the art in wearable devices intended for use by clinicians to monitor vital signs of critical care patients in hospital settings, with a focus on the usability and human factors of the devices. Methods: A comprehensive literature search of relevant databases was conducted, and 20 studies were identified and critically reviewed by the authors. Results: We identified 3 types of wearable devices: tactile, head-mounted, and smartwatch displays. In most cases, these devices were intended for use by anesthesiologists, but nurses and surgeons were also identified as potentially important users of wearable technology in critical care medicine. Although the studies investigating tactile displays revealed their potential to improve clinical monitoring, usability problems related to comfort need to be overcome before they can be considered suitable for use in clinical practice. Only a few studies investigated the usability and human factors of tactile displays by conducting user testing involving critical care professionals. The studies of head-mounted displays (HMDs) revealed that these devices could be useful in critical care medicine, particularly from an ergonomics point of view. By reducing the amount of time the user spends averting their gaze from the patient to a separate screen, HMDs enable clinicians to improve their patient focus and reduce the potential of repetitive strain injury. Conclusions: Researchers and designers of new wearable devices for use in critical care medicine should strive to achieve not only enhanced performance but also enhanced user experience for their users, especially in terms of comfort and ease of use. These aspects of wearable displays must be extensively tested with the intended end users in a setting that properly reflects the intended context of use before their adoption can be considered in clinical settings. %M 34032574 %R 10.2196/16491 %U https://humanfactors.jmir.org/2021/2/e16491 %U https://doi.org/10.2196/16491 %U http://www.ncbi.nlm.nih.gov/pubmed/34032574 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 10 %N 5 %P e27205 %T Usability Testing of a Digital Assessment Routing Tool: Protocol for an Iterative Convergent Mixed Methods Study %A Lowe,Cabella %A Hanuman Sing,Harry %A Browne,Mitchell %A Alwashmi,Meshari F %A Marsh,William %A Morrissey,Dylan %+ Centre for Sports & Exercise Medicine, William Harvey Research Institute, Queen Mary University of London, Bancroft Road, London, E1 4DG, United Kingdom, 44 7976315105, c.lowe@qmul.ac.uk %K mHealth %K mobile health %K eHealth %K digital health %K digital technology %K musculoskeletal injury %K musculoskeletal conditions %K triage %K physiotherapy triage %K usability %K acceptability %D 2021 %7 18.5.2021 %9 Protocol %J JMIR Res Protoc %G English %X Background: Musculoskeletal conditions account for 16% of global disability, resulting in a negative effect on millions of patients and an increasing burden on health care utilization. Digital technologies that improve health care outcomes and efficiency are considered a priority; however, innovations are often inadequately developed and poorly adopted. Further, they are rarely tested with sufficient rigor in clinical trials—the gold standard for clinical proof of efficacy. We have developed a new musculoskeletal Digital Assessment Routing Tool (DART) that allows users to self-assess and be directed to the right care. DART requires usability testing in preparation for clinical trials. Objective: This study will use the iterative convergent mixed methods design to assess and mitigate all serious usability issues to optimize user experience and adoption. Using this methodology, we will provide justifiable confidence to progress to full-scale randomized controlled trials when DART is integrated into clinical management pathways. This study protocol will provide a blueprint for future usability studies of mobile health solutions. Methods: We will collect qualitative and quantitative data from 20-30 participants aged 18 years and older for 4 months. The exact number of participants recruited will be dependent on the number of iterative cycles required to reach the study end points. Building on previous internal testing and stakeholder involvement, quantitative data collection is defined by the constructs within the ISO 9241-210-2019 standard and the system usability scale, providing a usability score for DART. Guided by the participant responses to quantitative questioning, the researcher will focus the qualitative data collection on specific usability problems. These will then be graded to provide the rationale for further DART system improvements throughout the iterative cycles. Results: This study received approval from the Queen Mary University of London Ethics of Research Committee (QMREC2018/48/048) on June 4, 2020. At manuscript submission, study recruitment was on-going, with data collection to be completed and results published in 2021. Conclusions: This study will provide evidence concerning mobile health DART system usability and acceptance determining system improvements required to support user adoption and minimize suboptimal system usability as a potential confounder within subsequent noninferiority clinical trials. Success should produce a safe effective system with excellent usability, facilitating quicker and easier patient access to appropriate care while reducing the burden on primary and secondary care musculoskeletal services. This deliberately rigorous approach to mobile health innovation could be used as a guide for other developers of similar apps. International Registered Report Identifier (IRRID): DERR1-10.2196/27205 %M 34003135 %R 10.2196/27205 %U https://www.researchprotocols.org/2021/5/e27205 %U https://doi.org/10.2196/27205 %U http://www.ncbi.nlm.nih.gov/pubmed/34003135 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 4 %P e24014 %T Physician Stress During Electronic Health Record Inbox Work: In Situ Measurement With Wearable Sensors %A Akbar,Fatema %A Mark,Gloria %A Prausnitz,Stephanie %A Warton,E Margaret %A East,Jeffrey A %A Moeller,Mark F %A Reed,Mary E %A Lieu,Tracy A %+ Division of Research, Kaiser Permanente Northern California, 2000 Broadway, Oakland, CA, 94611, United States, 1 510 891 3407, tracy.lieu@kp.org %K electronic health records %K stress %K wearables %K HRV %K inbox %K EHR alerts %K after-hours work %K electronic mail %K physician well-being %K Inbasket %D 2021 %7 28.4.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: Increased work through electronic health record (EHR) messaging is frequently cited as a factor of physician burnout. However, studies to date have relied on anecdotal or self-reported measures, which limit the ability to match EHR use patterns with continuous stress patterns throughout the day. Objective: The aim of this study is to collect EHR use and physiologic stress data through unobtrusive means that provide objective and continuous measures, cluster distinct patterns of EHR inbox work, identify physicians’ daily physiologic stress patterns, and evaluate the association between EHR inbox work patterns and physician physiologic stress. Methods: Physicians were recruited from 5 medical centers. Participants (N=47) were given wrist-worn devices (Garmin Vivosmart 3) with heart rate sensors to wear for 7 days. The devices measured physiological stress throughout the day based on heart rate variability (HRV). Perceived stress was also measured with self-reports through experience sampling and a one-time survey. From the EHR system logs, the time attributed to different activities was quantified. By using a clustering algorithm, distinct inbox work patterns were identified and their associated stress measures were compared. The effects of EHR use on physician stress were examined using a generalized linear mixed effects model. Results: Physicians spent an average of 1.08 hours doing EHR inbox work out of an average total EHR time of 3.5 hours. Patient messages accounted for most of the inbox work time (mean 37%, SD 11%). A total of 3 patterns of inbox work emerged: inbox work mostly outside work hours, inbox work mostly during work hours, and inbox work extending after hours that were mostly contiguous to work hours. Across these 3 groups, physiologic stress patterns showed 3 periods in which stress increased: in the first hour of work, early in the afternoon, and in the evening. Physicians in group 1 had the longest average stress duration during work hours (80 out of 243 min of valid HRV data; P=.02), as measured by physiological sensors. Inbox work duration, the rate of EHR window switching (moving from one screen to another), the proportion of inbox work done outside of work hours, inbox work batching, and the day of the week were each independently associated with daily stress duration (marginal R2=15%). Individual-level random effects were significant and explained most of the variation in stress (conditional R2=98%). Conclusions: This study is among the first to demonstrate associations between electronic inbox work and physiological stress. We identified 3 potentially modifiable factors associated with stress: EHR window switching, inbox work duration, and inbox work outside work hours. Organizations seeking to reduce physician stress may consider system-based changes to reduce EHR window switching or inbox work duration or the incorporation of inbox management time into work hours. %M 33908888 %R 10.2196/24014 %U https://medinform.jmir.org/2021/4/e24014 %U https://doi.org/10.2196/24014 %U http://www.ncbi.nlm.nih.gov/pubmed/33908888 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 4 %P e24179 %T Beyond Notes: Why It Is Time to Abandon an Outdated Documentation Paradigm %A Steinkamp,Jackson %A Kantrowitz,Jacob %A Sharma,Abhinav %A Bala,Wasif %+ Department of Family and Community Medicine, University of Toronto, 500 University Avenue, 5th Floor, Toronto, ON, M5G 1V7, Canada, 1 7789387714, abhinavarun@gmail.com %K electronic medical records %K health informatics %K information chaos %K medical documentation %K clinicians %K medical notes %K electronic medical notes %K medical team %D 2021 %7 20.4.2021 %9 Viewpoint %J J Med Internet Res %G English %X Clinicians spend a substantial part of their workday reviewing and writing electronic medical notes. Here we describe how the current, widely accepted paradigm for electronic medical notes represents a poor organizational framework for both the individual clinician and the broader medical team. As described in this viewpoint, the medical chart—including notes, labs, and imaging results—can be reconceptualized as a dynamic, fully collaborative workspace organized by topic rather than time, writer, or data type. This revised framework enables a more accurate and complete assessment of the current state of the patient and easy historical review, saving clinicians substantial time on both data input and retrieval. Collectively, this approach has the potential to improve health care delivery effectiveness and efficiency. %M 33877053 %R 10.2196/24179 %U https://www.jmir.org/2021/4/e24179 %U https://doi.org/10.2196/24179 %U http://www.ncbi.nlm.nih.gov/pubmed/33877053 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 4 %P e25657 %T Usability of Electronic Health Record–Generated Discharge Summaries: Heuristic Evaluation %A Tremoulet,Patrice D %A Shah,Priyanka D %A Acosta,Alisha A %A Grant,Christian W %A Kurtz,Jon T %A Mounas,Peter %A Kirchhoff,Michael %A Wade,Elizabeth %+ Department of Psychology, Rowan University, 201 Mullica Hill Rd, Robinson Hall Room 115K, Glassboro, NJ, 08028, United States, 1 8562564500 ext 53777, tremoulet@rowan.edu %K discharge summary %K usability %K electronic health record (EHR) %K care coordination %K elderly patients %K patient safety %K heuristic evaluation %K human factors %D 2021 %7 15.4.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: Obtaining accurate clinical information about recent acute care visits is extremely important for outpatient providers. However, documents used to communicate this information are often difficult to use. This puts patients at risk of adverse events. Elderly patients who are seen by more providers and have more care transitions are especially vulnerable. Objective: This study aimed to (1) identify the information about elderly patients’ recent acute care visits needed to coordinate their care, (2) use this information to assess discharge summaries, and (3) provide recommendations to help improve the quality of electronic health record (EHR)–generated discharge summaries, thereby increasing patient safety. Methods: A literature review, clinician interviews, and a survey of outpatient providers were used to identify and categorize data needed to coordinate care for recently discharged elderly patients. Based upon those data, 2 guidelines for creating useful discharge summaries were created. The new guidelines, along with 17 previously developed medical documentation usability heuristics, were applied to assess 4 simulated elderly patient discharge summaries. Results: The initial research effort yielded a list of 29 items that should always be included in elderly patient discharge summaries and a list of 7 “helpful, but not always necessary” items. Evaluation of 4 deidentified elderly patient discharge summaries revealed that none of the documents contained all 36 necessary items; between 14 and 18 were missing. The documents each had several other issues, and they differed significantly in organization, layout, and formatting. Conclusions: Variations in content and structure of discharge summaries in the United States make them unnecessarily difficult to use. Standardization would benefit both patients, by lowering the risk of care transition–related adverse events, and outpatient providers, by helping reduce frustration that can contribute to burnout. In the short term, acute care providers can help improve the quality of their discharge summaries by working with EHR vendors to follow recommendations based upon this study. Meanwhile, additional human factors work should determine the most effective way to organize and present information in discharge summaries, to facilitate effective standardization. %M 33856353 %R 10.2196/25657 %U https://www.jmir.org/2021/4/e25657 %U https://doi.org/10.2196/25657 %U http://www.ncbi.nlm.nih.gov/pubmed/33856353 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 8 %N 2 %P e26390 %T Impact of Individual, Organizational, and Technological Factors on the Implementation of an Online Portal to Support a Clinical Pathway Addressing Psycho-Oncology Care: Mixed Methods Study %A Masya,Lindy %A Shepherd,Heather L %A Butow,Phyllis %A Geerligs,Liesbeth %A Allison,Karen C %A Dolan,Colette %A Prest,Gabrielle %A , %A Shaw,Joanne %+ Psycho-Oncology Co-operative Research Group, School of Psychology, The University of Sydney, Griffith Taylor Building (A19), Sydney, NSW 2006, Australia, 61 2 86270828, heather.shepherd@sydney.edu.au %K decision support systems %K clinical decision making %K psycho-oncology %K health informatics %K clinical pathways %K health services research %D 2021 %7 14.4.2021 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Clinical pathways (CPs) can improve patient outcomes but can be complex to implement. Technologies, such as clinical decision support (CDS) tools, can facilitate their use, but require end-user testing in clinical settings. Objective: This study applied the Technology Acceptance Model to evaluate the individual, organizational, and technological contexts impacting application of a portal to facilitate a CP for anxiety and depression (the ADAPT Portal) in a metropolitan cancer service. The ADAPT Portal triggers patient screening on patient reported outcomes, alerts staff to high scores, recommends evidence-based management, and triggers review and rescreening at set intervals. Methods: Quantitative and qualitative data on portal activity, data accuracy, and health service staff perspectives were collected. Quantitative data were analyzed descriptively, and thematic analysis was applied to qualitative data. Results: Overall, 15 (100% of those invited) health service staff agreed to be interviewed. During the pilot, 73 users (36 health service staff members and 37 patients) were registered on the ADAPT Portal. Of the 37 patients registered, 16 (43%) completed screening at least once, with seven screening positive and triaged appropriately. In total, 34 support requests were lodged, resulting in 17 portal enhancements (technical issues). Health service staff considered the ADAPT Portal easy to use and useful; however, some deemed it unnecessary or burdensome (individual issues), particularly in a busy cancer service (organizational issues). Conclusions: User testing of a CDS to facilitate screening and assessment of anxiety and depression in cancer patients highlighted some technological issues in implementing the ADAPT CDS, resulting in 17 enhancements. Our results highlight the importance of obtaining health service staff feedback when piloting specialized CDS tools and addressing contextual factors when implementing them. %M 33851926 %R 10.2196/26390 %U https://humanfactors.jmir.org/2021/2/e26390 %U https://doi.org/10.2196/26390 %U http://www.ncbi.nlm.nih.gov/pubmed/33851926 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 5 %N 4 %P e18764 %T User Perspectives on Barriers and Facilitators to the Implementation of Electronic Health Records in Behavioral Hospitals: Qualitative Study %A Jung,Se Young %A Hwang,Hee %A Lee,Keehyuck %A Lee,Donghyun %A Yoo,Sooyoung %A Lim,Kahyun %A Lee,Ho-Young %A Kim,Eric %+ Department of Digital Healthcare, Seoul National University Bundang Hospital, Dolma-ro 172, Seongnam, Republic of Korea, 82 317870114, chrisruga@naver.com %K electronic health records %K mental health care %K qualitative study %K mobile phone %D 2021 %7 8.4.2021 %9 Original Paper %J JMIR Form Res %G English %X Background: Despite the rapid adoption of electronic health records (EHRs) resulting from the reimbursement program of the US government, EHR adoption in behavioral hospitals is still slow, and there remains a lack of evidence regarding barriers and facilitators to the implementation of mental health care EHRs. Objective: The aim of this study is to analyze the experience of mental health professionals to explore the perceived barriers, facilitators, and critical ideas influencing the implementation and usability of a mental health care EHR. Methods: In this phenomenological qualitative study, we interviewed physicians, nurses, pharmacists, mental health clinicians, and administrative professionals separately at 4 behavioral hospitals in the United States. We conducted semistructured interviews (N=43) from behavioral hospitals involved in the adoption of the mental health care EHR. Purposeful sampling was used to maximize the diversity. Transcripts were coded and analyzed for emergent domains. An exploratory data analysis was conducted. Results: Content analyses revealed 7 barriers and 4 facilitators. The most important barriers to implementing the mental health care EHR were the low levels of computer proficiency among nurses, complexity of the system, alert fatigue, and resistance because of legacy systems. This led to poor usability, low acceptability, and distrust toward the system. The major facilitators to implementing the mental health care EHR were well-executed training programs, improved productivity, better quality of care, and the good usability of the mental health care EHR. Conclusions: Health care professionals expected to enhance their work productivity and interprofessional collaboration by introducing the mental health care EHR. Routine education for end users is an essential starting point for the successful implementation of mental health care EHR electronic decision support. When adopting the mental health care EHR, managers need to focus on common practices in behavioral hospitals, such as documenting structured data in their organizations and adopting a seamless workflow of mental health care into the system. %M 33830061 %R 10.2196/18764 %U https://formative.jmir.org/2021/4/e18764 %U https://doi.org/10.2196/18764 %U http://www.ncbi.nlm.nih.gov/pubmed/33830061 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 10 %N 3 %P e25576 %T The Implementation Science for Genomic Health Translation (INSIGHT) Study in Epilepsy: Protocol for a Learning Health Care System %A Feofanova,Elena Valeryevna %A Zhang,Guo-Qiang %A Lhatoo,Samden %A Metcalf,Ginger A %A Boerwinkle,Eric %A Venner,Eric %+ Human Genetics Center, Department of Epidemiology, Human Genetics and Environmental Sciences, School of Public Health, The University of Texas Health Science Center at Houston, 1200 Pressler St, RAS E-435, Houston, TX, 77030, United States, 1 713 500 982, elena.v.feofanova@uth.tmc.edu %K genomic medicine %K electronic health record %K implementation %K genetics %K prototype %K decision support %D 2021 %7 26.3.2021 %9 Protocol %J JMIR Res Protoc %G English %X Background: Genomic medicine is poised to improve care for common complex diseases such as epilepsy, but additional clinical informatics and implementation science research is needed for it to become a part of the standard of care. Epilepsy is an exemplary complex neurological disorder for which DNA diagnostics have shown to be advantageous for patient care. Objective: We designed the Implementation Science for Genomic Health Translation (INSIGHT) study to leverage the fact that both the clinic and testing laboratory control the development and customization of their respective electronic health records and clinical reporting platforms. Through INSIGHT, we can rapidly prototype and benchmark novel approaches to incorporating clinical genomics into patient care. Of particular interest are clinical decision support tools that take advantage of domain knowledge from clinical genomics and can be rapidly adjusted based on feedback from clinicians. Methods: Building on previously developed evidence and infrastructure components, our model includes the following: establishment of an intervention-ready genomic knowledge base for patient care, creation of a health informatics platform and linking it to a clinical genomics reporting system, and scaling and evaluation of INSIGHT following established implementation science principles. Results: INSIGHT was approved by the Institutional Review Board at the University of Texas Health Science Center at Houston on May 15, 2020, and is designed as a 2-year proof-of-concept study beginning in December 2021. By design, 120 patients from the Texas Comprehensive Epilepsy Program are to be enrolled to test the INSIGHT workflow. Initial results are expected in the first half of 2023. Conclusions: INSIGHT’s domain-specific, practical but generalizable approach may help catalyze a pathway to accelerate translation of genomic knowledge into impactful interventions in patient care. International Registered Report Identifier (IRRID): PRR1-10.2196/25576 %M 33769305 %R 10.2196/25576 %U https://www.researchprotocols.org/2021/3/e25576 %U https://doi.org/10.2196/25576 %U http://www.ncbi.nlm.nih.gov/pubmed/33769305 %0 Journal Article %@ 2369-1999 %I JMIR Publications %V 7 %N 1 %P e24062 %T A Mobile App to Improve Symptom Control and Information Exchange Among Specialists and Local Health Workers Treating Tanzanian Cancer Patients: Human-Centered Design Approach %A Morse,Robert S %A Lambden,Kaley %A Quinn,Erin %A Ngoma,Twalib %A Mushi,Beatrice %A Ho,Yun Xian %A Ngoma,Mamsau %A Mahuna,Habiba %A Sagan,Sarah B %A Mmari,Joshua %A Miesfeldt,Susan %+ Maine Medical Center, Suite 111, 100 Campus Drive, Scarborough, ME, 04074, United States, 1 207 396 7787, MIESFS@mmc.org %K mobile health %K mHealth %K user-centered design %K palliative care %K pain %K cancer %K sub-Saharan Africa %K mobile phone %D 2021 %7 23.3.2021 %9 Original Paper %J JMIR Cancer %G English %X Background: Improving access to end-of-life symptom control interventions among cancer patients is a public health priority in Tanzania, and innovative community-based solutions are needed. Mobile health technology holds promise; however, existing resources are limited, and outpatient access to palliative care specialists is poor. A mobile platform that extends palliative care specialist access via shared care with community-based local health workers (LHWs) and provides remote support for pain and other symptom management can address this care gap. Objective: The aim of this study is to design and develop mobile-Palliative Care Link (mPCL), a web and mobile app to support outpatient symptom assessment and care coordination and control, with a focus on pain. Methods: A human-centered iterative design framework was used to develop the mPCL prototype for use by Tanzanian palliative care specialists (physicians and nurses trained in palliative care), poor-prognosis cancer patients and their lay caregivers (patients and caregivers), and LHWs. Central to mPCL is the validated African Palliative Care Outcome Scale (POS), which was adapted for automated, twice-weekly collection of quality of life–focused patient and caregiver responses and timely review, reaction, and tracking by specialists and LHWs. Prototype usability testing sessions were conducted in person with 21 key informants representing target end users. Sessions consisted of direct observations and qualitative and quantitative feedback on app ease of use and recommendations for improvement. Results were applied to optimize the prototype for subsequent real-world testing. Early pilot testing was conducted by deploying the app among 10 patients and caregivers, randomized to mPCL use versus phone-contact POS collection, and then gathering specialist and study team feedback to further optimize the prototype for a broader randomized field study to examine the app’s effectiveness in symptom control among cancer patients. Results: mPCL functionalities include the ability to create and update a synoptic clinical record, regular real-time symptom assessment, patient or caregiver and care team communication and care coordination, symptom-focused educational resources, and ready access to emergency phone contact with a care team member. Results from the usability and pilot testing demonstrated that all users were able to successfully navigate the app, and feedback suggests that mPCL has clinical utility. User-informed recommendations included further improvement in app navigation, simplification of patient and caregiver components and language, and delineation of user roles. Conclusions: We designed, built, and tested a usable, functional mobile app prototype that supports outpatient palliative care for Tanzanian patients with cancer. mPCL is expressly designed to facilitate coordinated care via customized interfaces supporting core users—patients or caregivers, LHWs, and members of the palliative care team—and their respective roles. Future work is needed to demonstrate the effectiveness and sustainability of mPCL to remotely support the symptom control needs of Tanzanian cancer patients, particularly in harder-to-reach areas. %M 33755022 %R 10.2196/24062 %U https://cancer.jmir.org/2021/1/e24062 %U https://doi.org/10.2196/24062 %U http://www.ncbi.nlm.nih.gov/pubmed/33755022 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 3 %P e23951 %T Commitment Levels of Health Care Providers in Using the District Health Information System and the Associated Factors for Decision Making in Resource-Limited Settings: Cross-sectional Survey Study %A Kanfe,Shuma G %A Endehabtu,Berhanu F %A Ahmed,Mohammedjud H %A Mengestie,Nebyu D %A Tilahun,Binyam %+ Health Informatics, Mettu University, Metu Zuria, Ethiopia, 251 0935054730, shumagosha33@gmail.com %K commitment %K district health information system %K decision making %K performance monitoring %K health facilities %K information use %D 2021 %7 4.3.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: Changing the culture of information use, which is one of the transformation agendas of the Ministry of Health of Ethiopia, cannot become real unless health care providers are committed to using locally collected data for evidence-based decision making. The commitment of health care providers has paramount influence on district health information system 2 (DHIS2) data utilization for decision making. Evidence is limited on health care providers’ level of commitment to using DHIS2 data in Ethiopia. Therefore, this study aims to fill this evidence gap. Objective: This study aimed to assess the levels of commitment of health care providers and the factors influencing their commitment levels in using DHIS2 data for decision making at public health care facilities in the Ilu Aba Bora zone of the Oromia national regional state, Ethiopia in 2020. Methods: The cross-sectional quantitative study supplemented by qualitative methods was conducted from February 26, 2020 to April 17, 2020. A total of 264 participants were approached. SPSS version 20 software was used for data entry and analysis. Descriptive and analytical statistics, including bivariable and multivariable analyses, were performed. Thematic analysis was conducted for the qualitative data. Results: Of the 264 respondents, 121 (45.8%, 95% CI 40.0%-52.8%) respondents showed high commitment levels to use DHIS2 data. The variables associated with the level of commitment to use DHIS2 data were found to be provision of feedback for DHIS2 data use (adjusted odds ratio [AOR] 1.85, 95% CI 1.02-3.33), regular supervision and managerial support (AOR 2.84, 95% CI 1.50-5.37), information use culture (AOR 1.92, 95% CI 1.03-3.59), motivation to use DHIS2 data (AOR 1.80, 95% CI 1.00-3.25), health needs (AOR 3.96, 95% CI 2.11-7.41), and competency in DHIS2 tasks (AOR 2.41, 95% CI 1.27-4.55). Conclusions: In general, less than half of the study participants showed high commitment levels to use DHIS2 data for decision making in health care. Providing regular supportive supervision and feedback and increasing the motivation and competency of the health care providers in performing DHIS2 data tasks will help in promoting their levels of commitment that can result in the cultural transformation of data use for evidence-based decision making in health care. %M 33661133 %R 10.2196/23951 %U https://medinform.jmir.org/2021/3/e23951 %U https://doi.org/10.2196/23951 %U http://www.ncbi.nlm.nih.gov/pubmed/33661133 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 2 %P e21615 %T A Novel Patient Values Tab for the Electronic Health Record: A User-Centered Design Approach %A Desai,Anjali Varma %A Michael,Chelsea L %A Kuperman,Gilad J %A Jordan,Gregory %A Mittelstaedt,Haley %A Epstein,Andrew S %A Connor,MaryAnn %A B Villar,Rika Paula %A Bernal,Camila %A Kramer,Dana %A Davis,Mary Elizabeth %A Chen,Yuxiao %A Malisse,Catherine %A Markose,Gigi %A Nelson,Judith E %+ Memorial Sloan Kettering Cancer Center, 1275 York Avenue, New York, NY, 10065, United States, 1 917 865 2495, Desaia2@mskcc.org %K electronic health record %K health informatics %K supportive care %K palliative care %K oncology %D 2021 %7 17.2.2021 %9 Viewpoint %J J Med Internet Res %G English %X Background: The COVID-19 pandemic has shined a harsh light on a critical deficiency in our health care system: our inability to access important information about patients’ values, goals, and preferences in the electronic health record (EHR). At Memorial Sloan Kettering Cancer Center (MSK), we have integrated and systematized health-related values discussions led by oncology nurses for newly diagnosed cancer patients as part of routine comprehensive cancer care. Such conversations include not only the patient’s wishes for care at the end of life but also more holistic personal values, including sources of strength, concerns, hopes, and their definition of an acceptable quality of life. In addition, health care providers use a structured template to document their discussions of patient goals of care. Objective: To provide ready access to key information about the patient as a person with individual values, goals, and preferences, we undertook the creation of the Patient Values Tab in our center’s EHR to display this information in a single, central location. Here, we describe the interprofessional, interdisciplinary, iterative process and user-centered design methodology that we applied to build this novel functionality as well as our initial implementation experience and plans for evaluation. Methods: We first convened a working group of experts from multiple departments, including medical oncology, health informatics, information systems, nursing informatics, nursing education, and supportive care, and a user experience designer. We conducted in-depth, semistructured, audiorecorded interviews of over 100 key stakeholders. The working group sought consensus on the tab’s main content, homing in on high-priority areas identified by the stakeholders. The core content was mapped to various EHR data sources. We established a set of high-level design principles to guide our process. Our user experience designer then created wireframes of the tab design. The designer conducted usability testing with physicians, nurses, and other health professionals. Data validation testing was conducted. Results: We have already deployed the Patient Values Tab to a pilot sample of users in the MSK Gastrointestinal Medical Oncology Service, including physicians, advanced practice providers, nurses, and administrative staff. We have early evidence of the positive impact of this EHR innovation. Audit logs show increasing use. Many of the initial user comments have been enthusiastically positive, while others have provided constructive suggestions for additional tab refinements with respect to format and content. Conclusions: It is our challenge and obligation to enrich the EHR with information about the patient as a person. Realization of this capability is a pressing public health need requiring the collaboration of technological experts with a broad range of clinical leaders, users, patients, and families to achieve solutions that are both principled and practical. Our new Patient Values Tab represents a step forward in this important direction. %M 33595448 %R 10.2196/21615 %U http://www.jmir.org/2021/2/e21615/ %U https://doi.org/10.2196/21615 %U http://www.ncbi.nlm.nih.gov/pubmed/33595448 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 2 %P e22939 %T A Bespoke Electronic Health Record for Epilepsy Care (EpiToMe): Development and Qualitative Evaluation %A Tao,Shiqiang %A Lhatoo,Samden %A Hampson,Johnson %A Cui,Licong %A Zhang,Guo-Qiang %+ Department of Neurology, The University of Texas Health Science Center at Houston, 1133 John Freeman Blvd, JJL 430, Houston, TX, 77030, United States, 1 7135007117, guo-qiang.zhang@uth.tmc.edu %K specialty-specific EHR %K physician-centered design %K clinical workflow %K patient care management %K clinical care documentation %K physician burnout %K interoperability %D 2021 %7 12.2.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: While electronic health records (EHR) bring various benefits to health care, EHR systems are often criticized as cumbersome to use, failing to fulfill the promise of improved health care delivery with little more than a means of meeting regulatory and billing requirements. EHR has also been recognized as one of the contributing factors for physician burnout. Objective: Specialty-specific EHR systems have been suggested as an alternative approach that can potentially address challenges associated with general-purpose EHRs. We introduce the Epilepsy Tracking and optimized Management engine (EpiToMe), an exemplar bespoke EHR system for epilepsy care. EpiToMe uses an agile, physician-centered development strategy to optimize clinical workflow and patient care documentation. We present the design and implementation of EpiToMe and report the initial feedback on its utility for physician burnout. Methods: Using collaborative, asynchronous data capturing interfaces anchored to a domain ontology, EpiToMe distributes reporting and documentation workload among technicians, clinical fellows, and attending physicians. Results of documentation are transmitted to the parent EHR to meet patient care requirements with a push of a button. An HL7 (version 2.3) messaging engine exchanges information between EpiToMe and the parent EHR to optimize clinical workflow tasks without redundant data entry. EpiToMe also provides live, interactive patient tracking interfaces to ease the burden of care management. Results: Since February 2019, 15,417 electroencephalogram reports, 2635 Epilepsy Monitoring Unit daily reports, and 1369 Epilepsy Monitoring Unit phase reports have been completed in EpiToMe for 6593 unique patients. A 10-question survey was completed by 11 (among 16 invited) senior clinical attending physicians. Consensus was found that EpiToMe eased the burden of care documentation for patient management, a contributing factor to physician burnout. Conclusions: EpiToMe offers an exemplar bespoke EHR system developed using a physician-centered design and latest advancements in information technology. The bespoke approach has the potential to ease the burden of care management in epilepsy. This approach is applicable to other clinical specialties. %M 33576745 %R 10.2196/22939 %U http://www.jmir.org/2021/2/e22939/ %U https://doi.org/10.2196/22939 %U http://www.ncbi.nlm.nih.gov/pubmed/33576745 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 8 %N 1 %P e21884 %T Perceptual Gaps Between Clinicians and Technologists on Health Information Technology-Related Errors in Hospitals: Observational Study %A Ndabu,Theophile %A Mulgund,Pavankumar %A Sharman,Raj %A Singh,Ranjit %+ Department of Management Science and Systems, School of Management, State University of New York at Buffalo, 203 Alfiero Center, Buffalo, NY, 14260, United States, 1 7166453271, tnndabu@buffalo.edu %K patient safety %K medical errors %K health information technology %K sociotechnical framework %K patient harm %D 2021 %7 5.2.2021 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Health information technology (HIT) has been widely adopted in hospital settings, contributing to improved patient safety. However, many types of medical errors attributable to information technology (IT) have negatively impacted patient safety. The continued occurrence of many errors is a reminder that HIT software testing and validation is not adequate in ensuring errorless software functioning within the health care organization. Objective: This pilot study aims to classify technology-related medical errors in a hospital setting using an expanded version of the sociotechnical framework to understand the significant differences in the perceptions of clinical and technology stakeholders regarding the potential causes of these errors. The paper also provides some recommendations to prevent future errors. Methods: Medical errors were collected from previous studies identified in leading health databases. From the main list, we selected errors that occurred in hospital settings. Semistructured interviews with 5 medical and 6 IT professionals were conducted to map the events on different dimensions of the expanded sociotechnical framework. Results: Of the 2319 identified publications, 36 were included in the review. Of the 67 errors collected, 12 occurred in hospital settings. The classification showed the “gulf” that exists between IT and medical professionals in their perspectives on the underlying causes of medical errors. IT experts consider technology as the source of most errors and suggest solutions that are mostly technical. However, clinicians assigned the source of errors within the people, process, and contextual dimensions. For example, for the error “Copied and pasted charting in the wrong window: Before, you could not easily get into someone else’s chart accidentally...because you would have to pull the chart and open it,” medical experts highlighted contextual issues, including the number of patients a health care provider sees in a short time frame, unfamiliarity with a new electronic medical record system, nurse transitions around the time of error, and confusion due to patients having the same name. They emphasized process controls, including failure modes, as a potential fix. Technology experts, in contrast, discussed the lack of notification, poor user interface, and lack of end-user training as critical factors for this error. Conclusions: Knowledge of the dimensions of the sociotechnical framework and their interplay with other dimensions can guide the choice of ways to address medical errors. These findings lead us to conclude that designers need not only a high degree of HIT know-how but also a strong understanding of the medical processes and contextual factors. Although software development teams have historically included clinicians as business analysts or subject matter experts to bridge the gap, development teams will be better served by more immersive exposure to clinical environments, leading to better software design and implementation, and ultimately to enhanced patient safety. %M 33544089 %R 10.2196/21884 %U http://humanfactors.jmir.org/2021/1/e21884/ %U https://doi.org/10.2196/21884 %U http://www.ncbi.nlm.nih.gov/pubmed/33544089 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 1 %P e19194 %T A Mobile App to Support Clinical Diagnosis of Upper Respiratory Problems (eHealthResp): Co-Design Approach %A Moura,João %A Almeida,Ana Margarida Pisco %A Roque,Fátima %A Figueiras,Adolfo %A Herdeiro,Maria Teresa %+ Department of Communication and Art/DigiMedia, University of Aveiro, campus universitário de santiago, Aveiro, 3810-193, Portugal, 351 234 370389 ext 2370, marga@ua.pt %K mHealth %K Clinical Decision Support Systems %K respiratory system %K diagnose %K interface %K mobile phone %D 2021 %7 28.1.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: The misuse of antibiotics is a global public health issue that fosters bacterial resistance and jeopardizes generational health. The development of validated tools such as web-based courses and mobile apps to enhance clinical decisions in upper respiratory infections is of great importance in reducing the incorrect use of antibiotics in these situations. Objective: The aim of this study was to design and prevalidate the interface of a mobile app to assist and provide clinical support in the diagnosis of upper respiratory problems. We aimed to assess the adequacy and usability of the interface of the tool in the belief that it could be beneficial to health care delivery in the clinical decision setting. Methods: Using a co-design approach that brought together professionals in interface design and experts in pharmacology and pharmacoepidemiology, the mobile app interface was evaluated through peer review sessions held by interface design professionals on a heuristic survey. The reviewers accessed a high-fidelity interactive mock-up of the interface and filled in a questionnaire to assess the dimensions of layout and visual design and navigation and tasks. The resulting feedback of this evaluation supported the redesign of the primary interface, which was assessed for the second time by 2 of the previously mentioned reviewers. Results: With 4 as the highest score, the interface scored a mean of 3.16 (SD 0.45; median of the means 3.2) for layout and visual design and a mean of 3.43 (SD 0.33; median of the means 3.51) for navigation and tasks, reflecting an overall positive evaluation. The open-ended commentaries allowed us to better understand specific recommendations of the reviewers. Throughout this section, approximately 0.98 comments per parameter were registered, reflecting a high level of effectiveness of the chosen parameters in identifying potential problems. The resultant beta version of the interface, addressing the majority of the detected problems, was further assessed by 2 of the previous reviewers, validating the new design. Future tests with physicians and pharmacists will help assess credibility and user experience dimensions. Conclusions: Our study revealed that the designed interface is easy to interpret and use. Peer reviewers raised important issues that could be easily fixed and positively reassessed. As a result, the study enabled us to produce a new tool for interface usability assessment and a set of recommendations for developing mobile interfaces for clinical decision support systems in the scope of upper respiratory problems. %M 33507153 %R 10.2196/19194 %U http://www.jmir.org/2021/1/e19194/ %U https://doi.org/10.2196/19194 %U http://www.ncbi.nlm.nih.gov/pubmed/33507153 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 22 %N 12 %P e23382 %T Perceived Electronic Health Record Usability as a Predictor of Task Load and Burnout Among US Physicians: Mediation Analysis %A Melnick,Edward R %A Harry,Elizabeth %A Sinsky,Christine A %A Dyrbye,Liselotte N %A Wang,Hanhan %A Trockel,Mickey Todd %A West,Colin P %A Shanafelt,Tait %+ Department of Emergency Medicine, Yale University School of Medicine, 464 Congress Ave, Suite 260, New Haven, CT, 06519, United States, 1 (203) 785 5174, edward.melnick@yale.edu %K electronic health record %K EHR %K usability %K cognitive load %K System Usability Scale %K physician task load %K NASA Task Load Index %K physician %K burnout %K stress %D 2020 %7 22.12.2020 %9 Original Paper %J J Med Internet Res %G English %X Background: Electronic health record (EHR) usability and physician task load both contribute to physician professional burnout. The association between perceived EHR usability and workload has not previously been studied at a national level. Better understanding these interactions could give further information as to the drivers of extraneous task load. Objective: This study aimed to determine the relationship between physician-perceived EHR usability and workload by specialty and evaluate for associations with professional burnout. Methods: A secondary analysis of a cross-sectional survey of US physicians from all specialties was conducted from October 2017 to March 2018. Among the 1250 physicians invited to respond to the subsurvey analyzed here, 848 (67.8%) completed it. EHR usability was assessed with the System Usability Scale (SUS; range: 0-100). Provider task load (PTL) was assessed using the mental demand, physical demand, temporal demand, and effort required subscales of the National Aeronautics and Space Administration Task Load Index (range: 0-400). Burnout was measured using the Maslach Burnout Inventory. Results: The mean scores were 46.1 (SD 22.1) for SUS and 262.5 (SD 71.7) for PTL. On multivariable analysis adjusting for age, gender, relationship status, medical specialty, practice setting, hours worked per week, and number of nights on call per week, physician-rated EHR usability was associated with PTL, with each 1-point increase in SUS score (indicating more favorable) associated with a 0.57-point decrease in PTL score (P<.001). On mediation analysis, higher SUS score was associated with lower PTL score, which was associated with lower odds of burnout. Conclusions: A strong association was observed between EHR usability and workload among US physicians, with more favorable usability associated with less workload. Both outcomes were associated with the odds of burnout, with task load acting as a mediator between EHR usability and burnout. Improving EHR usability while decreasing task load has the potential to allow practicing physicians more working memory for medical decision making and patient communication. %M 33289493 %R 10.2196/23382 %U http://www.jmir.org/2020/12/e23382/ %U https://doi.org/10.2196/23382 %U http://www.ncbi.nlm.nih.gov/pubmed/33289493 %0 Journal Article %@ 2291-5222 %I JMIR Publications %V 8 %N 12 %P e23622 %T Effective Use of Mobile Electronic Medical Records by Medical Interns in Real Clinical Settings: Mixed Methods Study %A Kim,SuJin %A Ku,Seulji %A Kim,Taerim %A Cha,Won Chul %A Jung,Kwang Yul %+ Department of Emergency Medicine, Ewha Womans University Seoul Hospital, 260 Gonghang-daero, Gangseo-gu, Seoul, 07804, Republic of Korea, 82 10 9183 0491, mdyul414@gmail.com %K mobile health %K mobile EMR %K intern %K workflow %K mHealth %K electronic medical record %K electronic health record %K EHR %K EMR %K efficiency %D 2020 %7 15.12.2020 %9 Original Paper %J JMIR Mhealth Uhealth %G English %X Background: In South Korea, most graduated medical students undertake a 1-year internship before beginning residency and specialization. Interns usually work in a tertiary hospital and rotate between different, randomly assigned departments to be exposed to different medical specialties. Their jobs are mostly simple and repetitive but are still essential for the patient care process. However, owing to the lack of experience and overwhelming workload, interns at tertiary hospitals in South Korea are usually inefficient, often delaying the entire clinical process. Health care providers have widely adopted mobile electronic medical records (mEMRs) as they have been shown to improve workflow efficiency. Objective: This study investigates the association between the frequency of mEMR usage and the clinical task completion interval time among interns in a tertiary hospital. Methods: This mixed methods study was conducted at the Samsung Medical Center, Seoul, South Korea. Interns who worked at the Samsung Medical Center from March 2018 to February 2019 were included. The hospital electronic medical record (EMR) system known as DARWIN (Data Analysis and Research Window for Integrated kNowledge) was launched with PC and mobile. Both versions are actively used in hospitals by personnel in various positions. We collected the log data from the mEMR server and the intern clinical task time-series data from the EMR server. Interns can manage the process of identifying patients, assigning the clinical task, finishing the requested clinical intern tasks, etc, through the use of the mEMR system. We compared the clinical task completion interval among 4 groups of interns divided by the mEMR frequency quantile. Then, System Usability Score (SUS) questionnaires and semistructured interviews were conducted. Results: The regular mEMR users were defined as those who logged in more than once a day on average and used the mEMR until the level after login. Among a total of 87 interns, 84 used the mEMR to verify the requested clinical tasks. The most frequently used item was “Intern task list.” Analysis of the 4 intern groups revealed an inverse relationship between the median time of the task completion interval and the frequency of mEMR use. Correlation analysis showed that the intern task completion time interval had a significant inverse relationship with the individual frequency of mEMR usage (coefficient=-0.27; 95% CI -0.46 to -0.04; P=.02). In the additional survey, the mean SUS value was 81.67, which supported the results of the data analysis. Conclusions: Our findings suggest that frequent mEMR use is associated with improved work efficiency in hospital interns with good usability of the mEMR. Such finding supports the idea that the use of mEMR improves the effectiveness and workflow efficiency of interns working in hospitals and, more generally, in the context of health care. %M 33320098 %R 10.2196/23622 %U http://mhealth.jmir.org/2020/12/e23622/ %U https://doi.org/10.2196/23622 %U http://www.ncbi.nlm.nih.gov/pubmed/33320098 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 7 %N 4 %P e21698 %T Health Care Professionals’ Experience of a Digital Tool for Patient Exchange, Anamnesis, and Triage in Primary Care: Qualitative Study %A Eldh,Ann Catrine %A Sverker,Annette %A Bendtsen,Preben %A Nilsson,Evalill %+ Department of Health, Medicine and Caring Sciences, Linköping University, HMV, 511-13, Campus US, Linköping University, Linköping, 581 83, Sweden, 46 13286634, ann.catrine.eldh@liu.se %K communication %K content analysis %K eHealth %K telemedicine %K digital technology %K interviews %K primary health care %K qualitative research %D 2020 %7 14.12.2020 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Despite a growing body of knowledge about eHealth innovations, there is still limited understanding of the implementation of such tools in everyday primary care. Objective: The objective of our study was to describe health care staff’s experience with a digital communication system intended for patient-staff encounters via a digital route in primary care. Methods: In this qualitative study we conducted 21 individual interviews with staff at 5 primary care centers in Sweden that had used a digital communication system for 6 months. The interviews were guided by narrative queries, transcribed verbatim, and subjected to content analysis. Results: While the digital communication system was easy to grasp, it was nevertheless complex to use, affecting both staffing and routines for communicating with patients, and documenting contacts. Templates strengthened equivalent procedures for patients but dictated a certain level of health and digital literacy for accuracy. Although patients expected a chat to be synchronous, asynchronous communication was extended over time. The system for digital communication benefited assessments and enabled more efficient use of resources, such as staff. On the other hand, telephone contact was faster and better for certain purposes, especially when the patient’s voice itself provided data. However, many primary care patients, particularly younger ones, expected digital routes for contact. To match preferences for communicating to a place and time that suited patients was significant; staff were willing to accept some nuisance from a suboptimal service—at least for a while—if it procured patient satisfaction. A team effort, including engaged managers, scaffolded the implementation process, whereas being subjected to a trial without likely success erected barriers. Conclusions: A digital communication system introduced in regular primary care involved complexity beyond merely learning how to manage the tool. Rather, it affected routines and required that both the team and the context were addressed. Further knowledge is needed about what factors facilitate implementation, and how. This study suggested including ethical perspectives on eHealth tools, providing an important but novel aspect of implementation. %M 33315014 %R 10.2196/21698 %U http://humanfactors.jmir.org/2020/4/e21698/ %U https://doi.org/10.2196/21698 %U http://www.ncbi.nlm.nih.gov/pubmed/33315014 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 7 %N 4 %P e23188 %T Health Care Staff’s Experiences of Engagement When Introducing a Digital Decision Support System for Wound Management: Qualitative Study %A Wickström,Hanna %A Tuvesson,Hanna %A Öien,Rut %A Midlöv,Patrik %A Fagerström,Cecilia %+ Department of Clinical Sciences Malmö, Center for Primary Health Care Research, Lund University, Box 50332, Malmö, 202 13, Sweden, 46 702728294, hanna.wickstrom@med.lu.se %K decision support systems, clinical %K eHealth %K staff engagement %K leg ulcer %K telemedicine %K mobile phone %D 2020 %7 9.12.2020 %9 Original Paper %J JMIR Hum Factors %G English %X Background: eHealth solutions such as digital decision support systems (DDSSs) have the potential to assist collaboration between health care staff to improve matters for specific patient groups. Patients with hard-to-heal ulcers have long healing times because of a lack of guidelines for structured diagnosis, treatment, and follow-up. Multidisciplinary collaboration in wound management teams is essential. A DDSS could offer a way of aiding improvement within wound management. The introduction of eHealth solutions into health care is complicated, and the engagement of the staff seems crucial. Factors influencing and affecting engagement need to be understood and considered for the introduction of a DDSS to succeed. Objective: This study aims to describe health care staff’s experiences of engagement and barriers to and influencers of engagement when introducing a DDSS for wound management. Methods: This study uses a qualitative approach. Interviews were conducted with 11 health care staff within primary (n=4), community (n=6), and specialist (n=1) care during the start-up of the introduction of a DDSS for wound management. The interviews focused on the staff’s experiences of engagement. Content analysis by Burnard was used in the data analysis process. Results: A total of 4 categories emerged describing the participants’ experiences of engagement: a personal liaison, a professional commitment, an extended togetherness, and an awareness and understanding of the circumstances. Conclusions: This study identifies barriers to and influencers of engagement, reinforcing that staff experience engagement through feeling a personal liaison and a professional commitment to make things better for their patients. In addition, engagement is nourished by sharing with coworkers and by active support and understanding from leadership. %M 33295295 %R 10.2196/23188 %U http://humanfactors.jmir.org/2020/4/e23188/ %U https://doi.org/10.2196/23188 %U http://www.ncbi.nlm.nih.gov/pubmed/33295295 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 8 %N 11 %P e23623 %T The Associations of Electronic Health Record Usability and User Age With Stress and Cognitive Failures Among Finnish Registered Nurses: Cross-Sectional Study %A Kaihlanen,Anu-Marja %A Gluschkoff,Kia %A Hyppönen,Hannele %A Kaipio,Johanna %A Puttonen,Sampsa %A Vehko,Tuulikki %A Saranto,Kaija %A Karhe,Liisa %A Heponiemi,Tarja %+ Finnish Institute for Health and Welfare, P.O. Box 30, Helsinki, FI-00271, Finland, 358 295246033, anu.kaihlanen@thl.fi %K electronic health records %K usability %K stress %K cognitive failure %K nurse %D 2020 %7 18.11.2020 %9 Original Paper %J JMIR Med Inform %G English %X Background: Electronic health records (EHRs) are expected to provide many clinical and organizational benefits. Simultaneously, the end users may face unintended consequences, such as stress and increased cognitive workload, due to poor EHR usability. However, whether the effects of usability depend on end user characteristics, such as career stage or age, remains poorly understood. Objective: The objective of this study was to examine the associations of EHR usability and user age with stress related to information systems and cognitive failures among registered nurses. Methods: A cross-sectional survey design was employed in Finland in 2017. A total of 3383 registered nurses responded to the nationwide electronic survey. Multiple linear regression was used to examine the associations of EHR usability (eg, how easily information can be found and a patient’s care can be documented) and user age with stress related to information systems and cognitive failures. Interaction effects of EHR usability and age were also tested. Models were adjusted for gender and employment sector. Results: Poor EHR usability was associated with higher levels of stress related to information systems (β=.38; P<.001). The strength of the association did not depend on user age. Poor EHR usability was also associated with higher levels of cognitive failures (β=.28; P<.001). There was a significant interaction effect between age and EHR usability for cognitive failures (β=.04; P<.001). Young nurses who found the EHR difficult to use reported the most cognitive failures. Conclusions: Information system stress due to poor EHR usability afflicts younger and older nurses alike. However, younger nurses starting their careers may be more cognitively burdened if they find EHR systems difficult to use compared to older nurses. Adequate support in using the EHRs may be particularly important to young registered nurses, who have a lot to learn and adopt in their early years of practice. %M 33206050 %R 10.2196/23623 %U http://medinform.jmir.org/2020/11/e23623/ %U https://doi.org/10.2196/23623 %U http://www.ncbi.nlm.nih.gov/pubmed/33206050 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 7 %N 4 %P e19769 %T A Simulated Graphical Interface for Integrating Patient-Generated Health Data From Smartwatches With Electronic Health Records: Usability Study %A Alpert,Jordan M %A Kota,Naga S Prabhakar %A Ranka,Sanjay %A Mendoza,Tonatiuh V %A Solberg,Laurence M %A Rashidi,Parisa %A Manini,Todd M %+ Department of Advertising, College of Journalism and Communications, University of Florida, 2913 Weimer Hall, Gainesville, FL, 32611, United States, 1 3523920453, jordan.alpert@ufl.edu %K wearable %K smartwatch %K mobile %K mHealth %K user-centered design %K electronic health records %D 2020 %7 30.10.2020 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Wearable technology, such as smartwatches, can capture valuable patient-generated data and help inform patient care. Electronic health records provide logical and practical platforms for including such data, but it is necessary to evaluate the way the data are presented and visualized. Objective: The aim of this study is to evaluate a graphical interface that displays patients’ health data from smartwatches, mimicking the integration within the environment of electronic health records. Methods: A total of 12 health care professionals evaluated a simulated interface using a usability scale questionnaire, testing the clarity of the interface, colors, usefulness of information, navigation, and readability of text. Results: The interface was positively received, with 14 out of the 16 questions generating a score of 5 or greater among at least 75% of participants (9/12). On an 8-point Likert scale, the highest rated features of the interface were quick turnaround times (mean score 7.1), readability of the text (mean score 6.8), and use of terminology/abbreviations (mean score 6.75). Conclusions: Collaborating with health care professionals to develop and refine a graphical interface for visualizing patients’ health data from smartwatches revealed that the key elements of the interface were acceptable. The implementation of such data from smartwatches and other mobile devices within electronic health records should consider the opinions of key stakeholders as the development of this platform progresses. %M 33124988 %R 10.2196/19769 %U http://humanfactors.jmir.org/2020/4/e19769/ %U https://doi.org/10.2196/19769 %U http://www.ncbi.nlm.nih.gov/pubmed/33124988 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 7 %N 4 %P e18484 %T Characterizing and Visualizing Display and Task Fragmentation in the Electronic Health Record: Mixed Methods Design %A Senathirajah,Yalini %A Kaufman,David R %A Cato,Kenrick D %A Borycki,Elizabeth M %A Fawcett,Jaime Allen %A Kushniruk,Andre W %+ Department of Biomedical Informatics, School of Medicine, University of Pittsburgh, Fifth Floor Rm 515, 5607 Baum Blvd, Pittsburgh, PA, 15206, United States, 1 412 648 9214, yalini@pitt.edu %K electronic health record %K electronic medical record %K medical informatics %K information technology %K data visualization %K user computer interface %D 2020 %7 21.10.2020 %9 Original Paper %J JMIR Hum Factors %G English %X Background: The complexity of health care data and workflow presents challenges to the study of usability in electronic health records (EHRs). Display fragmentation refers to the distribution of relevant data across different screens or otherwise far apart, requiring complex navigation for the user’s workflow. Task and information fragmentation also contribute to cognitive burden. Objective: This study aims to define and analyze some of the main sources of fragmentation in EHR user interfaces (UIs); discuss relevant theoretical, historical, and practical considerations; and use granular microanalytic methods and visualization techniques to help us understand the nature of fragmentation and opportunities for EHR optimization or redesign. Methods: Sunburst visualizations capture the EHR navigation structure, showing levels and sublevels of the navigation tree, allowing calculation of a new measure, the Display Fragmentation Index. Time belt visualizations present the sequences of subtasks and allow calculation of proportion per instance, a measure that quantifies task fragmentation. These measures can be used separately or in conjunction to compare EHRs as well as tasks and subtasks in workflows and identify opportunities for reductions in steps and fragmentation. We present an example use of the methods for comparison of 2 different EHR interfaces (commercial and composable) in which subjects apprehend the same patient case. Results: Screen transitions were substantially reduced for the composable interface (from 43 to 14), whereas clicks (including scrolling) remained similar. Conclusions: These methods can aid in our understanding of UI needs under complex conditions and tasks to optimize EHR workflows and redesign. %M 33084580 %R 10.2196/18484 %U https://humanfactors.jmir.org/2020/4/e18484 %U https://doi.org/10.2196/18484 %U http://www.ncbi.nlm.nih.gov/pubmed/33084580 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 22 %N 10 %P e17003 %T Impact of Electronic Health Record Interface Design on Unsafe Prescribing of Ciclosporin, Tacrolimus, and Diltiazem: Cohort Study in English National Health Service Primary Care %A MacKenna,Brian %A Bacon,Sebastian %A Walker,Alex J %A Curtis,Helen J %A Croker,Richard %A Goldacre,Ben %+ The DataLab, Nuffield Department of Primary Care Health Sciences, University of Oxford, Radcliffe Observatory Quarter, Woodstock Road, Oxford, OX2 6GG, United Kingdom, 44 1865289313, ben.goldacre@phc.ox.ac.uk %K prescribing %K primary care %K electronic health records %K clinical software %K branded prescribing %K diltiazem %K tacrolimus %K ciclosporin %D 2020 %7 16.10.2020 %9 Original Paper %J J Med Internet Res %G English %X Background: In England, national safety guidance recommends that ciclosporin, tacrolimus, and diltiazem are prescribed by brand name due to their narrow therapeutic windows and, in the case of tacrolimus, to reduce the chance of organ transplantation rejection. Various small studies have shown that changes to electronic health record (EHR) system interfaces can affect prescribing choices. Objective: Our objectives were to assess variation by EHR systems in breach of safety guidance around prescribing of ciclosporin, tacrolimus, and diltiazem, and to conduct user-interface research into the causes of such breaches. Methods: We carried out a retrospective cohort study using prescribing data in English primary care. Participants were English general practices and their respective EHR systems. The main outcome measures were (1) the variation in ratio of safety breaches to adherent prescribing in all practices and (2) the description of observations of EHR system usage. Results: A total of 2,575,411 prescriptions were issued in 2018 for ciclosporin, tacrolimus, and diltiazem (over 60 mg); of these, 316,119 prescriptions breached NHS guidance (12.27%). Breaches were most common among users of the EMIS EHR system (breaches in 18.81% of ciclosporin and tacrolimus prescriptions and in 17.99% of diltiazem prescriptions), but breaches were observed in all EHR systems. Conclusions: Design choices in EHR systems strongly influence safe prescribing of ciclosporin, tacrolimus, and diltiazem, and breaches are prevalent in general practices in England. We recommend that all EHR vendors review their systems to increase safe prescribing of these medicines in line with national guidance. Almost all clinical practice is now mediated through an EHR system; further quantitative research into the effect of EHR system design on clinical practice is long overdue. %M 33064085 %R 10.2196/17003 %U https://www.jmir.org/2020/10/e17003 %U https://doi.org/10.2196/17003 %U http://www.ncbi.nlm.nih.gov/pubmed/33064085 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 22 %N 7 %P e19274 %T The Influence of Electronic Health Record Use on Physician Burnout: Cross-Sectional Survey %A Tajirian,Tania %A Stergiopoulos,Vicky %A Strudwick,Gillian %A Sequeira,Lydia %A Sanches,Marcos %A Kemp,Jessica %A Ramamoorthi,Karishini %A Zhang,Timothy %A Jankowicz,Damian %+ Centre for Addiction and Mental Health, 6168F, 100 Stokes St., Toronto, ON, M6J1H4, Canada, 1 4165358501 ext 30515, Tania.Tajirian@camh.ca %K electronic health record %K physician %K burnout %K psychiatry %K medical informatics %D 2020 %7 15.7.2020 %9 Original Paper %J J Med Internet Res %G English %X Background: Physician burnout has a direct impact on the delivery of high-quality health care, with health information technology tools such as electronic health records (EHRs) adding to the burden of practice inefficiencies. Objective: The aim of this study was to determine the extent of burnout among physicians and learners (residents and fellows); identify significant EHR-related contributors of physician burnout; and explore the differences between physicians and learners with regard to EHR-related factors such as time spent in EHR, documentation styles, proficiency, training, and perceived usefulness. In addition, the study aimed to address gaps in the EHR-related burnout research methodologies by determining physicians’ patterns of EHR use through usage logs. Methods: This study used a cross-sectional survey methodology and a review of administrative data for back-end log measures of survey respondents’ EHR use, which was conducted at a large Canadian academic mental health hospital. Chi-square and Fisher exact tests were used to examine the association of EHR-related factors with general physician burnout. The survey was sent out to 474 individuals between May and June 2019, including physicians (n=407), residents (n=53), and fellows (n=14), and we measured physician burnout and perceptions of EHR stressors (along with demographic and practice characteristics). Results: Our survey included 208 respondents, including physicians (n=176) and learners (n=32). The response rate was 43.2% for physicians (full-time: 156/208, 75.0%; part-time: 20/199, 10.1%), and 48% (32/67) for learners. A total of 25.6% (45/176) of practicing physicians and 19% (6/32) of learners reported having one or more symptoms of burnout, and 74.5% (155/208) of all respondents who reported burnout symptoms identified the EHR as a contributor. Lower satisfaction and higher frustration with the EHRs were significantly associated with perceptions of EHR contributing toward burnout. Physicians’ and learners’ experiences with the EHR, gathered through open-ended survey responses, identified challenges around the intuitiveness and usability of the technology as well as workflow issues. Metrics gathered from back-end usage logs demonstrated a 13.6-min overestimation in time spent on EHRs per patient and a 5.63-hour overestimation of after-hours EHR time, when compared with self-reported survey data. Conclusions: This study suggests that the use of EHRs is a perceived contributor to physician burnout. There should be a focus on combating physician burnout by reducing the unnecessary administrative burdens of EHRs through efficient implementation of systems and effective postimplementation strategies. %M 32673234 %R 10.2196/19274 %U https://www.jmir.org/2020/7/e19274 %U https://doi.org/10.2196/19274 %U http://www.ncbi.nlm.nih.gov/pubmed/32673234 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 7 %N 3 %P e19713 %T Patients’ Utilization and Perception of an Artificial Intelligence–Based Symptom Assessment and Advice Technology in a British Primary Care Waiting Room: Exploratory Pilot Study %A Miller,Stephen %A Gilbert,Stephen %A Virani,Vishaal %A Wicks,Paul %+ Ada Health GmbH, Adalbertstraße 20, Berlin, , Germany, 49 3060031987, science@ada.com %K human-centered design %K innovative %K health care apps %K eHealth %K symptom checker %K primary care %K general practice %K app %K usability %K acceptability %K utility %D 2020 %7 10.7.2020 %9 Original Paper %J JMIR Hum Factors %G English %X Background: When someone needs to know whether and when to seek medical attention, there are a range of options to consider. Each will have consequences for the individual (primarily considering trust, convenience, usefulness, and opportunity costs) and for the wider health system (affecting clinical throughput, cost, and system efficiency). Digital symptom assessment technologies that leverage artificial intelligence may help patients navigate to the right type of care with the correct degree of urgency. However, a recent review highlighted a gap in the literature on the real-world usability of these technologies. Objective: We sought to explore the usability, acceptability, and utility of one such symptom assessment technology, Ada, in a primary care setting. Methods: Patients with a new complaint attending a primary care clinic in South London were invited to use a custom version of the Ada symptom assessment mobile app. This exploratory pilot study was conducted between November 2017 and January 2018 in a practice with 20,000 registered patients. Participants were asked to complete an Ada self-assessment about their presenting complaint on a study smartphone, with assistance provided if required. Perceptions on the app and its utility were collected through a self-completed study questionnaire following completion of the Ada self-assessment. Results: Over a 3-month period, 523 patients participated. Most were female (n=325, 62.1%), mean age 39.79 years (SD 17.7 years), with a larger proportion (413/506, 81.6%) of working-age individuals (aged 15-64) than the general population (66.0%). Participants rated Ada’s ease of use highly, with most (511/522, 97.8%) reporting it was very or quite easy. Most would use Ada again (443/503, 88.1%) and agreed they would recommend it to a friend or relative (444/520, 85.3%). We identified a number of age-related trends among respondents, with a directional trend for more young respondents to report Ada had provided helpful advice (50/54, 93%, 18-24-year olds reported helpful) than older respondents (19/32, 59%, adults aged 70+ reported helpful). We found no sex differences on any of the usability questions fielded. While most respondents reported that using the symptom checker would not have made a difference in their care-seeking behavior (425/494, 86.0%), a sizable minority (63/494, 12.8%) reported they would have used lower-intensity care such as self-care, pharmacy, or delaying their appointment. The proportion was higher for patients aged 18-24 (11/50, 22%) than aged 70+ (0/28, 0%). Conclusions: In this exploratory pilot study, the digital symptom checker was rated as highly usable and acceptable by patients in a primary care setting. Further research is needed to confirm whether the app might appropriately direct patients to timely care, and understand how this might save resources for the health system. More work is also needed to ensure the benefits accrue equally to older age groups. %M 32540836 %R 10.2196/19713 %U https://humanfactors.jmir.org/2020/3/e19713 %U https://doi.org/10.2196/19713 %U http://www.ncbi.nlm.nih.gov/pubmed/32540836 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 7 %N 3 %P e15052 %T Novel Interface Designs for Patient Monitoring Applications in Critical Care Medicine: Human Factors Review %A Andrade,Evismar %A Quinlan,Leo %A Harte,Richard %A Byrne,Dara %A Fallon,Enda %A Kelly,Martina %A Casey,Siobhan %A Kirrane,Frank %A O'Connor,Paul %A O'Hora,Denis %A Scully,Michael %A Laffey,John %A Pladys,Patrick %A Beuchée,Alain %A ÓLaighin,Gearóid %+ Human Movement Laboratory, CÚRAM Centre for Research in Medical Devices, National University of Ireland, Galway, Alice Perry Engineering Building, University Road, Galway, Ireland, 353 91493710 ext 3710, leo.quinlan@nuigalway.ie %K interface design %K usability %K situation awareness %K graphical display %K satisfaction %K response time %K accuracy %K anesthesiology %K critical care %K performance %K ecological display %D 2020 %7 3.7.2020 %9 Review %J JMIR Hum Factors %G English %X Background: The patient monitor (PM) is one of the most commonly used medical devices in hospitals worldwide. PMs are used to monitor patients’ vital signs in a wide variety of patient care settings, especially in critical care settings, such as intensive care units. An interesting observation is that the design of PMs has not significantly changed over the past 2 decades, with the layout and structure of PMs more or less unchanged, with incremental changes in design being made rather than transformational changes. Thus, we believe it well-timed to review the design of novel PM interfaces, with particular reference to usability and human factors. Objective: This paper aims to review innovations in PM design proposed by researchers and explore how clinicians responded to these design changes. Methods: A literature search of relevant databases, following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines, identified 16 related studies. A detailed description of the interface design and an analysis of each novel PM were carried out, including a detailed analysis of the structure of the different user interfaces, to inform future PM design. The test methodologies used to evaluate the different designs are also presented. Results: Most of the studies included in this review identified some level of improvement in the clinician’s performance when using a novel display in comparison with the traditional PM. For instance, from the 16 reviewed studies, 12 studies identified an improvement in the detection and response times, and 10 studies identified an improvement in the accuracy or treatment efficiency. This indicates that novel displays have the potential to improve the clinical performance of nurses and doctors. However, the outcomes of some of these studies are weakened because of methodological deficiencies. These deficiencies are discussed in detail in this study. Conclusions: More careful study design is warranted to investigate the user experience and usability of future novel PMs for real time vital sign monitoring, to establish whether or not they could be used successfully in critical care. A series of recommendations on how future novel PM designs and evaluations can be enhanced are provided. %M 32618574 %R 10.2196/15052 %U https://humanfactors.jmir.org/2020/3/e15052 %U https://doi.org/10.2196/15052 %U http://www.ncbi.nlm.nih.gov/pubmed/32618574 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 7 %N 2 %P e16036 %T Comparison of the Effects of Automated and Manual Record Keeping on Anesthetists’ Monitoring Performance: Randomized Controlled Simulation Study %A Tse,Man-Kei %A Li,Simon Y W %A Chiu,Tsz Hin %A Lau,Chung Wai %A Lam,Ka Man %A Cheng,Chun Pong Benny %+ Department of Applied Psychology, Lingnan University, WYL-306, WYL Building, 8 Castle Peak Road, Tuen Mun, Hong Kong, , China (Hong Kong), 852 26167129, simonli2@ln.edu.hk %K anesthesia information management system %K automated record keeping %K vigilance %K situation awareness %K mental workload %D 2020 %7 16.6.2020 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Anesthesia information management systems (AIMSs) automatically import real-time vital signs from physiological monitors to anesthetic records, replacing part of anesthetists’ traditional manual record keeping. However, only a handful of studies have examined the effects of AIMSs on anesthetists’ monitoring performance. Objective: This study aimed to compare the effects of AIMS use and manual record keeping on anesthetists’ monitoring performance, using a full-scale high-fidelity simulation. Methods: This simulation study was a randomized controlled trial with a parallel group design that compared the effects of two record-keeping methods (AIMS vs manual) on anesthetists’ monitoring performance. Twenty anesthetists at a tertiary hospital in Hong Kong were randomly assigned to either the AIMS or manual condition, and they participated in a 45-minute scenario in a high-fidelity simulation environment. Participants took over a case involving general anesthesia for below-knee amputation surgery and performed record keeping. The three primary outcomes were participants’ (1) vigilance detection accuracy (%), (2) situation awareness accuracy (%), and (3) subjective mental workload (0-100). Results: With regard to the primary outcomes, there was no significant difference in participants’ vigilance detection accuracy (AIMS, 56.7% vs manual, 56.7%; P=.50), and subjective mental workload was significantly lower in the AIMS condition than in the manual condition (AIMS, 34.2 vs manual, 46.7; P=.02). However, the result for situation awareness accuracy was inconclusive as the study did not have enough power to detect a difference between the two conditions. Conclusions: Our findings suggest that it is promising for AIMS use to become a mainstay of anesthesia record keeping. AIMSs are effective in reducing anesthetists’ workload and improving the quality of their anesthetic record keeping, without compromising vigilance. %M 32543440 %R 10.2196/16036 %U http://humanfactors.jmir.org/2020/2/e16036/ %U https://doi.org/10.2196/16036 %U http://www.ncbi.nlm.nih.gov/pubmed/32543440 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 22 %N 4 %P e13188 %T Developing an Intranet-Based Lymphedema Dashboard for Breast Cancer Multidisciplinary Teams: Design Research Study %A Janssen,Anna %A Donnelly,Candice %A Kay,Judy %A Thiem,Peter %A Saavedra,Aldo %A Pathmanathan,Nirmala %A Elder,Elisabeth %A Dinh,Phuong %A Kabir,Masrura %A Jackson,Kirsten %A Harnett,Paul %A Shaw,Tim %+ Research in Implementation Science and eHealth Group, Faculty of Health Sciences, The University of Sydney, Charles Perkins Centre (D17), Sydney, 2006, Australia, 61 9036 9406, anna.janssen@sydney.edu.au %K eHealth %K clinical informatics %K human-centered design %K data visualization %D 2020 %7 21.4.2020 %9 Original Paper %J J Med Internet Res %G English %X Background: A large quantity of data is collected during the delivery of cancer care. However, once collected, these data are difficult for health professionals to access to support clinical decision making and performance review. There is a need for innovative tools that make clinical data more accessible to support health professionals in these activities. One approach for providing health professionals with access to clinical data is to create the infrastructure and interface for a clinical dashboard to make data accessible in a timely and relevant manner. Objective: This study aimed to develop and evaluate 2 prototype dashboards for displaying data on the identification and management of lymphedema. Methods: The study used a co-design framework to develop 2 prototype dashboards for use by health professionals delivering breast cancer care. The key feature of these dashboards was an approach for visualizing lymphedema patient cohort and individual patient data. This project began with 2 focus group sessions conducted with members of a breast cancer multidisciplinary team (n=33) and a breast cancer consumer (n=1) to establish clinically relevant and appropriate data for presentation and the visualization requirements for a dashboard. A series of fortnightly meetings over 6 months with an Advisory Committee (n=10) occurred to inform and refine the development of a static mock-up dashboard. This mock-up was then presented to representatives of the multidisciplinary team (n=3) to get preliminary feedback about the design and use of such dashboards. Feedback from these presentations was reviewed and used to inform the development of the interactive prototypes. A structured evaluation was conducted on the prototypes, using Think Aloud Protocol and semistructured interviews with representatives of the multidisciplinary team (n=5). Results: Lymphedema was selected as a clinically relevant area for the prototype dashboards. A qualitative evaluation is reported for 5 health professionals. These participants were selected from 3 specialties: surgery (n=1), radiation oncology (n=2), and occupational therapy (n=2). Participants were able to complete the majority of tasks on the dashboard. Semistructured interview themes were categorized into engagement or enthusiasm for the dashboard, user experience, and data quality and completeness. Conclusions: Findings from this study constitute the first report of a co-design process for creating a lymphedema dashboard for breast cancer health professionals. Health professionals are interested in the use of data visualization tools to make routinely collected clinical data more accessible. To be used effectively, dashboards need to be reliable and sourced from accurate and comprehensive data sets. While the co-design process used to develop the visualization tool proved effective for designing an individual patient dashboard, the complexity and accessibility of the data required for a cohort dashboard remained a challenge. %M 32314968 %R 10.2196/13188 %U https://www.jmir.org/2020/4/e13188 %U https://doi.org/10.2196/13188 %U http://www.ncbi.nlm.nih.gov/pubmed/32314968 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 22 %N 3 %P e14855 %T A Late Attempt to Involve End Users in the Design of Medication-Related Alerts: Survey Study %A Baysari,Melissa Therese %A Zheng,Wu Yi %A Van Dort,Bethany %A Reid-Anderson,Hannah %A Gronski,Mihaela %A Kenny,Eliza %+ Faculty of Health Sciences, The University of Sydney, Charles Perkins Centre D17, Sydney, 2006, Australia, 61 28627 9245, melissa.baysari@sydney.edu.au %K alert fatigue %K alerting %K medication alert systems %K clinical decision support %K hospital information systems %D 2020 %7 13.3.2020 %9 Original Paper %J J Med Internet Res %G English %X Background: When users of electronic medical records (EMRs) are presented with large numbers of irrelevant computerized alerts, they experience alert fatigue, begin to ignore alert information, and override alerts without processing or heeding alert recommendations. Anecdotally, doctors at our study site were dissatisfied with the medication-related alerts being generated, both in terms of volume being experienced and clinical relevance. Objective: This study aimed to involve end users in the redesign of medication-related alerts in a hospital EMR, 4 years post implementation. Methods: This work was undertaken at a private not-for-profit teaching hospital in Sydney, Australia. Since EMR implementation in 2015, the organization elected to implement all medication-related alert types available in the system for prescribers: allergy and intolerance alerts, therapeutic duplication alerts, pregnancy alerts, and drug-drug interaction alerts. The EMR included no medication administration alerts for nurses. To obtain feedback on current alerts and suggestions for redesign, a Web-based survey was distributed to all doctors and nurses at the site via hospital mailing lists. Results: Despite a general dissatisfaction with alerts, very few end users completed the survey. In total, only 3.37% (36/1066) of doctors and 14.5% (60/411) of nurses took part. Approximately 90% (30/33) of doctors who responded held the view that too many alerts were triggered in the EMR. Doctors suggested that most alerts be removed and that alerts be more specific and less sensitive. In contrast, 97% (58/60) of the nurse respondents indicated that they would like to receive medication administration alerts in the EMR. Most nurses indicated that they would like to receive all the alert types available at all severity levels. Conclusions: Attempting to engage with end users several years post implementation was challenging. Involving users so late in the implementation process may lead to clinicians viewing the provision of feedback to be futile. Seeking user feedback on usefulness, volume, and design of alerts is extremely valuable; however, we suggest this is undertaken early, preferably before system implementation. %M 32167479 %R 10.2196/14855 %U https://www.jmir.org/2020/3/e14855 %U https://doi.org/10.2196/14855 %U http://www.ncbi.nlm.nih.gov/pubmed/32167479 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 7 %N 4 %P e13917 %T Building a Semantic Health Data Warehouse in the Context of Clinical Trials: Development and Usability Study %A Lelong,Romain %A Soualmia,Lina F %A Grosjean,Julien %A Taalba,Mehdi %A Darmoni,Stéfan J %+ Department of Biomedical Informatics, Rouen University Hospital, D2IM, Porte 21, Cour Leschevin, Hôpital Charles-Nicolle, 1 Rue de Germont, Rouen, 76000, France, 33 232 888 898, romain.lelong@gmail.com %K data warehousing %K search engine %K semantics %K clinical trial %K patient selection %D 2019 %7 20.12.2019 %9 Original Paper %J JMIR Med Inform %G English %X Background: The huge amount of clinical, administrative, and demographic data recorded and maintained by hospitals can be consistently aggregated into health data warehouses with a uniform data model. In 2017, Rouen University Hospital (RUH) initiated the design of a semantic health data warehouse enabling both semantic description and retrieval of health information. Objective: This study aimed to present a proof of concept of this semantic health data warehouse, based on the data of 250,000 patients from RUH, and to assess its ability to assist health professionals in prescreening eligible patients in a clinical trials context. Methods: The semantic health data warehouse relies on 3 distinct semantic layers: (1) a terminology and ontology portal, (2) a semantic annotator, and (3) a semantic search engine and NoSQL (not only structured query language) layer to enhance data access performances. The system adopts an entity-centered vision that provides generic search capabilities able to express data requirements in terms of the whole set of interconnected conceptual entities that compose health information. Results: We assessed the ability of the system to assist the search for 95 inclusion and exclusion criteria originating from 5 randomly chosen clinical trials from RUH. The system succeeded in fully automating 39% (29/74) of the criteria and was efficiently used as a prescreening tool for 73% (54/74) of them. Furthermore, the targeted sources of information and the search engine–related or data-related limitations that could explain the results for each criterion were also observed. Conclusions: The entity-centered vision contrasts with the usual patient-centered vision adopted by existing systems. It enables more genericity in the information retrieval process. It also allows to fully exploit the semantic description of health information. Despite their semantic annotation, searching within clinical narratives remained the major challenge of the system. A finer annotation of the clinical texts and the addition of specific functionalities would significantly improve the results. The semantic aspect of the system combined with its generic entity-centered vision enables the processing of a large range of clinical questions. However, an important part of health information remains in clinical narratives, and we are currently investigating novel approaches (deep learning) to enhance the semantic annotation of those unstructured data. %M 31859675 %R 10.2196/13917 %U http://medinform.jmir.org/2019/4/e13917/ %U https://doi.org/10.2196/13917 %U http://www.ncbi.nlm.nih.gov/pubmed/31859675 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 21 %N 11 %P e11346 %T Investigating Software Requirements for Systems Supporting Task-Shifted Interventions: Usability Study %A Van de Ven,Pepijn %A Araya,Ricardo %A P de Paula Couto,Maria Clara %A Henrique,Maiara Garcia %A Meere,Damien %A Vilela Mendes,Ana %A Peters,Tim J %A Seabra,Antônio %A Franzin,Renato M %A Carvalho Pereda,Paula %A Scazufca,Marcia %+ Health Research Institute, HIST Cluster, University of Limerick, Main Building D2043, Limerick, Ireland, 353 61 202925, pepijn.vandeven@ul.ie %K task-shifting %K community health workers %K depression %K medical informatics %D 2019 %7 12.11.2019 %9 Original Paper %J J Med Internet Res %G English %X Background: There is a considerable shortfall in specialized health care professionals worldwide to deliver health services, and this shortfall is especially pronounced in low-middle-income countries. This has led to the implementation of task-shifted interventions, in which specific tasks are moved away from highly qualified health workers to health workers with less training. The World Health Organization (WHO) has published recommendations for such interventions, but guidelines for software and systems supporting such interventions are not included. Objective: The objective of this study was to formulate a number of software requirements for computer systems supporting task-shifted interventions. As the treatment of mental health problems is generally considered to be a task for highly trained health care professionals, it poses interesting case studies for task-shifted interventions. Therefore, we illustrated the use of the identified software requirements in a mobile system created for a task-shifted depression intervention to be provided to older adults in deprived areas of São Paulo, Brazil. Methods: Using a set of recommendations based on the WHO’s guidance documentation for task-shifted interventions, we identified 9 software requirements that aim to support health workers in management and supervision, training, good relationship with other health workers, and community embeddedness of the intervention. These 9 software requirements were used to implement a system for the provision of a psychosocial depression intervention with mobile Android interfaces to structure interventions and collect data, and Web interfaces for supervision and support of the health care workers delivering the intervention. The system was tested in a 2-arm pilot study with 33 patients and 11 health workers. In all, 8 of these 11 health workers participated in a usability study subsequent to the pilot. Results: The qualitative and quantitative feedback obtained with the System Usability Scale suggest that the system was deemed to have a usability of between OK and Good. Nevertheless, some participants’ responses indicated that they felt they needed technical assistance to use the system. This was reinforced by answers obtained with perceived usefulness and ease of use questionnaires, which indicated some users felt that they had issues around correct use of the system and perceived ability to become skillful at using the system. Conclusions: Overall, these high-level requirements adequately captured the functionality required to enable the health workers to provide the intervention successfully. Nevertheless, the analysis of results indicated that some improvements were required for the system to be useable in a task-shifted intervention. The most important of these were better access to a training environment, access for supervisors to metadata such as duration of sessions or exercises to identify issues, and a more robust and human-error–proof approach to the availability of patient data on the mobile devices used during the intervention. %M 31714246 %R 10.2196/11346 %U https://www.jmir.org/2019/11/e11346 %U https://doi.org/10.2196/11346 %U http://www.ncbi.nlm.nih.gov/pubmed/31714246 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 7 %N 4 %P e13466 %T Usability Factors Associated With Physicians’ Distress and Information System–Related Stress: Cross-Sectional Survey %A Heponiemi,Tarja %A Kujala,Sari %A Vainiomäki,Suvi %A Vehko,Tuulikki %A Lääveri,Tinja %A Vänskä,Jukka %A Ketola,Eeva %A Puttonen,Sampsa %A Hyppönen,Hannele %+ National Institute for Health and Welfare, PoBox 30, Helsinki, 00271, Finland, 358 295247434, tarja.heponiemi@thl.fi %K health information systems %K physicians %K electronic health records %K computers, digital %D 2019 %7 5.11.2019 %9 Original Paper %J JMIR Med Inform %G English %X Background: Constantly changing and difficult-to-use information systems have arisen as a significant source of stress in physicians’ work. Physicians have reported several usability problems, system failures, and a lack of integration between the systems and have experienced that systems poorly support the documentation and retrieval of patient data. This stress has kept rising in the 21st century, and it seems that it may also affect physicians’ well-being. Objective: This study aimed to examine the associations of (1) usability variables (perceived benefits, technical problems, support for feedback, and user-friendliness), (2) the number of systems in daily use, (3) experience of using information systems, and (4) participation in information systems development work with physicians’ distress and levels of stress related to information systems (SRIS) levels. Methods: A cross-sectional survey was conducted among 4018 Finnish physicians (64.82%, 2572 out of 3968 women) aged between 24 and 64 years (mean 46.8 years) in 2017. The analyses of covariance were used to examine the association of independent variables with SRIS and distress (using the General Health Questionnaire) adjusted for age, gender, employment sector, specialization status, and the electronic health record system in use. Results: High levels of technical problems and a high number of systems in daily use were associated with high levels of SRIS, whereas high levels of user-friendliness, perceived benefits, and support for feedback were associated with low levels of SRIS. Moreover, high levels of technical problems were associated with high levels of psychological distress, whereas high levels of user-friendliness were associated with low distress levels. Those who considered themselves experienced users of information systems had low levels of both SRIS and distress. Conclusions: It seems that by investing in user-friendly systems with better technical quality and good support for feedback that professionals perceive as being beneficial would improve the work-related well-being and overall well-being of physicians. Moreover, improving physicians’ skills related to information systems by giving them training could help to lessen the stress that results from poorly functioning information systems and improve physicians’ well-being. %M 31687938 %R 10.2196/13466 %U http://medinform.jmir.org/2019/4/e13466/ %U https://doi.org/10.2196/13466 %U http://www.ncbi.nlm.nih.gov/pubmed/31687938 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 6 %N 3 %P e13779 %T Factors Associated With Electronic Health Record Usage Among Primary Care Physicians After Hours: Retrospective Cohort Study %A Attipoe,Selasi %A Huang,Yungui %A Schweikhart,Sharon %A Rust,Steve %A Hoffman,Jeffrey %A Lin,Simon %+ College of Public Health, The Ohio State University, 250 Cunz Hall, 1841 Neil Avenue, Columbus, OH, 43210, United States, 1 410 908 2472, selasni@hotmail.com %K electronic health records %K health information technology %K primary care physicians %D 2019 %7 30.9.2019 %9 Original Paper %J JMIR Hum Factors %G English %X Background: There is limited published data on variation in physician usage of electronic health records (EHRs), particularly after hours. Research in this area could provide insight into the effects of EHR-related workload on physicians. Objective: This study sought to examine factors associated with after-hours EHR usage among primary care physicians. Methods: Electronic health records usage information was collected from primary care pediatricians in a large United States hospital. Inclusion criteria consisted solely of being a primary care physician who started employment with the hospital before the study period, so all eligible primary care physicians were included without sampling. Mixed effects statistical modeling was used to investigate the effects of age, gender, workload, normal-hour usage, week to week variation, and provider-to-provider variation on the after-hour usage of EHRs. Results: There were a total of 3498 weekly records obtained on 50 physicians, of whom 22% were male and 78% were female. Overall, more EHR usage during normal work hours was associated with decreased usage after hours. The more work relative value units generated by physicians, the more time they spent interacting with EHRs after hours (β=.04, P<.001) and overall (ie, during normal hours and after hours) (β=.24, P<.001). Gender was associated with total usage time, with females spending more time than males (P=.03). However, this association was not observed with after-hours EHR usage. provider-to-provider variation was the largest and most dominant source of variation in after-hour EHR usage, which accounted for 52% of variance of total EHR usage. Conclusion: The present study found that there is a considerable amount of variability in EHR use among primary care physicians, which suggested that many factors influence after-hours EHR usage by physicians. However, provider-to-provider variation was the largest and most dominant source of variation in after-hours EHR usage. While the results are intuitive, future studies should consider the effect of EHR use variations on workload efficiency. %M 31573912 %R 10.2196/13779 %U https://humanfactors.jmir.org/2019/3/e13779 %U https://doi.org/10.2196/13779 %U http://www.ncbi.nlm.nih.gov/pubmed/31573912 %0 Journal Article %@ 2561-9128 %I JMIR Publications %V 2 %N 2 %P e13559 %T Development and Implementation of the Portable Operating Room Tracker App With Vital Signs Streaming Infrastructure: Operational Feasibility Study %A Görges,Matthias %A West,Nicholas C %A Petersen,Christian L %A Ansermino,J Mark %+ Department of Anesthesiology, Pharmacology & Therapeutics, The University of British Columbia, 950 W 28th Avenue, Rm V3-324, Vancouver, BC, V5Z 4H4, Canada, 1 604 875 2000 ext 5616, mgorges@bcchr.ca %K communication systems %K patient monitoring %K user-centered design %K human factors %K anesthesia %D 2019 %7 05.08.2019 %9 Original Paper %J JMIR Perioper Med %G English %X Background: In the perioperative environment, a multidisciplinary clinical team continually observes and evaluates patient information. However, data availability may be restricted to certain locations, cognitive workload may be high, and team communication may be constrained by availability and priorities. We developed the remote Portable Operating Room Tracker app (the telePORT app) to improve information exchange and communication between anesthesia team members. The telePORT app combines a real-time feed of waveforms and vital signs from the operating rooms with messaging, help request, and reminder features. Objective: The aim of this paper is to describe the development of the app and the back-end infrastructure required to extract monitoring data, facilitate data exchange and ensure privacy and safety, which includes results from clinical feasibility testing. Methods: telePORT’s client user interface was developed using user-centered design principles and workflow observations. The server architecture involves network-based data extraction and data processing. Baseline user workload was assessed using step counters and communication logs. Clinical feasibility testing analyzed device usage over 11 months. Results: telePORT was more commonly used for help requests (approximately 4.5/day) than messaging between team members (approximately 1/day). Passive operating room monitoring was frequently utilized (34% of screen visits). Intermittent loss of wireless connectivity was a major barrier to adoption (decline of 0.3%/day). Conclusions: The underlying server infrastructure was repurposed for real-time streaming of vital signs and their collection for research and quality improvement. Day-to-day activities of the anesthesia team can be supported by a mobile app that integrates real-time data from all operating rooms. %M 33393912 %R 10.2196/13559 %U http://periop.jmir.org/2019/2/e13559/ %U https://doi.org/10.2196/13559 %U http://www.ncbi.nlm.nih.gov/pubmed/33393912 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 6 %N 3 %P e13812 %T Understanding the Situated Roles of Electronic Medical Record Systems to Enable Redesign: Mixed Methods Study %A Helou,Samar %A Abou-Khalil,Victoria %A Yamamoto,Goshiro %A Kondoh,Eiji %A Tamura,Hiroshi %A Hiragi,Shusuke %A Sugiyama,Osamu %A Okamoto,Kazuya %A Nambu,Masayuki %A Kuroda,Tomohiro %+ Department of Social Informatics, Graduate School of Informatics, Kyoto University, Yoshida-honmachi, Sakyo-ku, Kyoto, 606-8501, Japan, 81 5055346505, helou.samar@gmail.com %K computerized medical record systems %K physicians' offices %K design %K role %K prenatal care %K Japan %K observational study %D 2019 %7 09.07.2019 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Redesigning electronic medical record (EMR) systems is needed to improve their usability and usefulness. Similar to other artifacts, EMR systems can evolve with time and exhibit situated roles. Situated roles refer to the ways in which a system is appropriated by its users, that is, the unintended ways the users engage with, relate to, and perceive the system in its context of use. These situated roles are usually unknown to the designers as they emerge and evolve as a response by the users to a contextual need or constraint. Understanding the system’s situated roles can expose the unarticulated needs of the users and enable redesign opportunities. Objective: This study aimed to find EMR redesign opportunities by understanding the situated roles of EMR systems in prenatal care settings. Methods: We conducted a field-based observational study at a Japanese prenatal care clinic. We observed 3 obstetricians and 6 midwives providing prenatal care to 37 pregnant women. We looked at how the EMR system is used during the checkups. We analyzed the observational data following a thematic analysis approach and identified the situated roles of the EMR system. Finally, we administered a survey to 5 obstetricians and 10 midwives to validate our results and understand the attitudes of the prenatal care staff regarding the situated roles of the EMR system. Results: We identified 10 distinct situated roles that EMR systems play in prenatal care settings. Among them, 4 roles were regarded as favorable as most users wanted to experience them more frequently, and 4 roles were regarded as unfavorable as most users wanted to experience them less frequently; 2 ambivalent roles highlighted the providers’ reluctance to document sensitive psychosocial information in the EMR and their use of the EMR system as an accomplice to pause communication during the checkups. To improve the usability and usefulness of EMR systems, designers can amplify the favorable roles and minimize the unfavorable roles. Our results also showed that obstetricians and midwives may have different experiences, wants, and priorities regarding the use of the EMR system. Conclusions: Currently, EMR systems are mainly viewed as tools that support the clinical workflow. Redesigning EMR systems is needed to amplify their roles as communication support tools. Our results provided multiple EMR redesign opportunities to improve the usability and usefulness of EMR systems in prenatal care. Designers can use the results to guide their EMR redesign activities and align them with the users’ wants and priorities. The biggest challenge is to redesign EMR systems in a way that amplifies their favorable roles for all the stakeholders concurrently. %M 31290398 %R 10.2196/13812 %U http://humanfactors.jmir.org/2019/3/e13812/ %U https://doi.org/10.2196/13812 %U http://www.ncbi.nlm.nih.gov/pubmed/31290398 %0 Journal Article %@ 2291-5222 %I JMIR Publications %V 7 %N 7 %P e13194 %T Applying a User-Centered Approach to Building a Mobile Personal Health Record App: Development and Usability Study %A Zhou,Leming %A DeAlmeida,Dilhari %A Parmanto,Bambang %+ Department of Health Information Management, University of Pittsburgh, 6021 Forbes Tower, 3600 Forbes Avenue at Meyran Avenue, Pittsburgh, PA, 15260, United States, 1 412 383 6653, lmzhou@gmail.com %K mobile app %K personal health record %K needs assessment %D 2019 %7 05.07.2019 %9 Original Paper %J JMIR Mhealth Uhealth %G English %X Background: A personal health record (PHR) system encourages patients to engage with their own health care by giving them the ability to manage and keep track of their own health data. Of the numerous PHR systems available in the market, many are Web-based patient portals and a few are mobile apps. They have mainly been created by hospitals and electronic health record (EHR) vendors. One major limitation of these hospital-created PHR systems is that patients can only view specific health data extracted from their EHR. Patients do not have the freedom to add important personal health data they collect in their daily lives into their PHR. Therefore, there is an information gap between clinical visits. Objective: The aim of this study was to develop and evaluate a new mobile PHR app that can be easily used to manage various types of personal health data to fill the information gap. Methods: A user-centered approach was used to guide the development and evaluation of the new mobile PHR app. There were three steps in this study: needs assessment, app design and development, and conducting a usability study. First, a large-scale questionnaire study was conducted with the general population to gain an understanding of their needs and expectations with regard to a mobile PHR app. A mobile PHR app for personal medical data tracking and management was then created based on the results of the questionnaire study. End users were actively involved in all stages of the app development. Finally, a usability study was performed with participants to evaluate the usability of the mobile PHR app, which involved asking participants to finish a set of tasks and to respond to a usability questionnaire. Results: In the questionnaire study for needs assessment, there were 609 participants in total. The answers from these participants revealed that they wanted to manage various types of personal health data in a mobile PHR app. Participants also reported some features they desired to have in the app. On the basis of the needs assessment findings, a new mobile PHR app (PittPHR) was created with 6 major modules: health records, history, trackers, contacts, appointments, and resources. This app allows users to customize the trackers according to their needs. In the usability study, there were 15 participants. The usability study participants expressed satisfaction with the app and provided comments and suggestions for further development. Conclusions: This new mobile PHR app provides options for users to manage a wide range of personal health data conveniently in one place. The app fills the information gap between clinical visits. The study results indicated that this new mobile PHR app meets the need of users and that users welcome this app. %M 31278732 %R 10.2196/13194 %U https://mhealth.jmir.org/2019/7/e13194/ %U https://doi.org/10.2196/13194 %U http://www.ncbi.nlm.nih.gov/pubmed/31278732 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 6 %N 3 %P e12695 %T Evaluating the Feasibility of a Software Prototype Supporting the Management of Multimorbid Seniors: Mixed Methods Study in General Practices %A Kersting,Christine %A Weltermann,Birgitta %+ Institute for General Medicine, University Hospital Essen, University of Duisburg-Essen, Hufelandstr. 55, Essen, 45147, Germany, 49 201 723 8038, christine.kersting@uk-essen.de %K patient care management %K primary health care %K clinical decision support systems %K electronic health record %K reminder system %K health information technology %D 2019 %7 04.07.2019 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Longitudinal, patient-centered care represents a challenge for general practices. Decision support and reminder systems can offer targeted support. Objective: The objective of this study was to follow a user-oriented, stepwise approach to develop an add-on for German electronic health record (EHR) systems, which aims to support longitudinal care management of multimorbid seniors, using a flag system displaying patient-centered information relevant for comprehensive health care management. This study evaluated the prototype’s feasibility from both a technical and users’ perspective. Methods: The study was conducted with 18 general practitioners (GPs) and practice assistants (PAs) from 9 general practices using a mixed methods approach. In all practices, 1 GP and 1 PA tested the software each for 4 multimorbid seniors selected from the practice patient data. Technical feasibility was evaluated by documenting all technical problems. To evaluate the feasibility from the users’ perspective, participants’ responses during the software test were documented. In addition, they completed a self-administered questionnaire, including the validated System Usability Scale (SUS). Data were merged by transforming qualitative data into quantitative data. Analyses were performed using univariate statistics in IBM SPSS statistics. Results: From a technical perspective, the new software was easy to install and worked without problems. Difficulties during the installation occurred in practices lacking a 64-bit system or a current version of Microsoft .NET. As EHRs used in German practices do not provide an interface to extract the data needed, additional software was required. Incomplete flags for some laboratory data occurred, although this function was implemented in our software as shown in previous tests. From the users’ perspective, the new add-on provided a better overview of relevant patient information, reminded more comprehensively about upcoming examinations, and better supported guideline-based care when compared with their individual practice strategies. A total of 14 out of 18 participants (78%) were interested in using the software long-term. Furthermore, 8 of 9 GPs were willing to pay 5 to 25 Euros (mean 14.75, SD 5.93) monthly for its use. The usability was rated as 75% (43%-95%). Conclusions: The new EHR add-on was well accepted and achieved a good usability rating measured by the validated SUS. In perspective, the legally consolidated, standardized interface to German EHRs will facilitate the technical integration. In view of the high feasibility, we plan to study the software’s effectiveness in everyday primary care. Trial Registration: German Clinical Trials Register DRKS00008777; https://www.drks.de/drks_web/navigate.do? navigationId=trial.HTML&TRIAL_ID=DRKS00008777 %M 31274115 %R 10.2196/12695 %U https://humanfactors.jmir.org/2019/3/e12695/ %U https://doi.org/10.2196/12695 %U http://www.ncbi.nlm.nih.gov/pubmed/31274115 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 7 %N 3 %P e13627 %T Descriptive Usability Study of CirrODS: Clinical Decision and Workflow Support Tool for Management of Patients With Cirrhosis %A Garvin,Jennifer Hornung %A Ducom,Julie %A Matheny,Michael %A Miller,Anne %A Westerman,Dax %A Reale,Carrie %A Slagle,Jason %A Kelly,Natalie %A Beebe,Russ %A Koola,Jejo %A Groessl,Erik J %A Patterson,Emily S %A Weinger,Matthew %A Perkins,Amy M %A Ho,Samuel B %+ Health Information Management and Systems, The Ohio State University, Rm 543 Atwell Hall, 453 West 10th Ave, Columbus, OH,, United States, 1 215 620 3390, jennifer.garvin@hsc.utah.edu %K clinical decision support %K human factors engineering %K liver cirrhosis %K interview %D 2019 %7 03.07.2019 %9 Original Paper %J JMIR Med Inform %G English %X Background: There are gaps in delivering evidence-based care for patients with chronic liver disease and cirrhosis. Objective: Our objective was to use interactive user-centered design methods to develop the Cirrhosis Order Set and Clinical Decision Support (CirrODS) tool in order to improve clinical decision-making and workflow. Methods: Two work groups were convened with clinicians, user experience designers, human factors and health services researchers, and information technologists to create user interface designs. CirrODS prototypes underwent several rounds of formative design. Physicians (n=20) at three hospitals were provided with clinical scenarios of patients with cirrhosis, and the admission orders made with and without the CirrODS tool were compared. The physicians rated their experience using CirrODS and provided comments, which we coded into categories and themes. We assessed the safety, usability, and quality of CirrODS using qualitative and quantitative methods. Results: We created an interactive CirrODS prototype that displays an alert when existing electronic data indicate a patient is at risk for cirrhosis. The tool consists of two primary frames, presenting relevant patient data and allowing recommended evidence-based tests and treatments to be ordered and categorized. Physicians viewed the tool positively and suggested that it would be most useful at the time of admission. When using the tool, the clinicians placed fewer orders than they placed when not using the tool, but more of the orders placed were considered to be high priority when the tool was used than when it was not used. The physicians’ ratings of CirrODS indicated above average usability. Conclusions: We developed a novel Web-based combined clinical decision-making and workflow support tool to alert and assist clinicians caring for patients with cirrhosis. Further studies are underway to assess the impact on quality of care for patients with cirrhosis in actual practice. %M 31271153 %R 10.2196/13627 %U https://medinform.jmir.org/2019/3/e13627/ %U https://doi.org/10.2196/13627 %U http://www.ncbi.nlm.nih.gov/pubmed/31271153 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 21 %N 5 %P e12875 %T Developing the National Usability-Focused Health Information System Scale for Physicians: Validation Study %A Hyppönen,Hannele %A Kaipio,Johanna %A Heponiemi,Tarja %A Lääveri,Tinja %A Aalto,Anna-Mari %A Vänskä,Jukka %A Elovainio,Marko %+ National Institute for Health and Welfare, PO Box 30, Helsinki, 00271, Finland, 358 295247434, tarja.heponiemi@thl.fi %K physicians %K health information systems %K questionnaire %K validation studies %D 2019 %7 16.05.2019 %9 Original Paper %J J Med Internet Res %G English %X Background: Problems in the usability of health information systems (HISs) are well acknowledged, but research still lacks a validated questionnaire for measuring and monitoring different dimensions of usability of HISs. Such questionnaires are needed not only for research but also for developing usability of HISs from the viewpoint of end-user experiences. Objective: This study aimed to develop and test the validity of the questionnaire measuring the National Usability-Focused HIS-Scale (NuHISS) among a nationally representative sample of Finnish physicians. Methods: We utilized 2 cross-sectional data collected from a random sample of Finnish physicians in 2014 (N=3781; of which 2340 [61.9%] were women) and 2017 (N=4018; of which 2604 [64.8%] were women). Exploratory and confirmatory factor analyses (structural equation modeling [SEM]) were applied to test the structural validity of the NuHISS. As the concurrent validity measure, we used the self-reported overall quality of the electronic health record system (school grade) provided by the participants using marginal structural models. Results: The exploratory factor analyses with Varimax rotation suggested that the 7-factor solution did offer a good fit to the data in both samples (C2=2136.14 in 2014 and C2=2109.83 in 2017, both P<.001). Moreover, structural equation modelling analyses, using comparative fit index (CFI), Tucker-Lewis Index (TLI), Normed Fit Index (NFI), root mean squared error of approximation (RMSEA), and Standardized Root Mean square Residual (SRMR), showed that the 7-factor solution provided an acceptable fit in both samples (CFI=0.92/0.91, TLI=0.92/0.91, NFI=0.92/0.91, RMSEA=0.048/0.049, and SRMR=0.040/0.039). In addition, concurrent validity of this solution was shown to be acceptable. Ease of use, but also all other dimensions, was especially associated with overall quality reports independent of measured confounders. The 7-factor solution included dimensions of technical quality, information quality, feedback, ease of use, benefits, internal collaboration, and cross-organizational collaboration. Conclusions: NuHISS provides a useful tool for measuring usability of HISs among physicians and offers a valid measure for monitoring the long-term development of HISs on a large scale. The relative importance of items needs to be assessed against national electronic health policy goals and complemented with items that have remained outside the NuHISS from the questionnaire when appropriate. %M 31099336 %R 10.2196/12875 %U https://www.jmir.org/2019/5/e12875/ %U https://doi.org/10.2196/12875 %U http://www.ncbi.nlm.nih.gov/pubmed/31099336 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 21 %N 5 %P e12426 %T Design Choices and Trade-Offs in Health Care Blockchain Implementations: Systematic Review %A O'Donoghue,Odhran %A Vazirani,Anuraag A %A Brindley,David %A Meinert,Edward %+ Healthcare Translation Research Group, Department of Paediatrics, University of Oxford, Level 2, Children's Hospital, John Radcliffe Hospital, Oxford, OX3 9DU, United Kingdom, 44 7824446808, edward.meinert@paediatrics.ox.ac.uk %K blockchain %K interoperability %K distributed ledger technology %K scalability %K health information exchange %D 2019 %7 10.05.2019 %9 Review %J J Med Internet Res %G English %X Background: A blockchain is a list of records that uses cryptography to make stored data immutable; their use has recently been proposed for electronic medical record (EMR) systems. This paper details a systematic review of trade-offs in blockchain technologies that are relevant to EMRs. Trade-offs are defined as “a compromise between two desirable but incompatible features.” Objective: This review’s primary research question was: “What are the trade-offs involved in different blockchain designs that are relevant to the creation of blockchain-based electronic medical records systems?” Methods: Seven databases were systematically searched for relevant articles using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA). Papers published from January 1, 2017 to June 15, 2018 were selected. Quality assessments of papers were performed using the Risk Of Bias In Non-randomized Studies—of Interventions (ROBINS-I) tool and the Critical Assessment Skills Programme (CASP) tool. Database searches identified 2885 articles, of which 15 were ultimately included for analysis. Results: A total of 17 trade-offs were identified impacting the design, development, and implementation of blockchain systems; these trade-offs are organized into themes, including business, application, data, and technology architecture. Conclusions: The key findings concluded the following: (1) multiple trade-offs can be managed adaptively to improve EMR utility; (2) multiple trade-offs involve improving the security of blockchain systems at the cost of other features, meaning EMR efficacy highly depends on data protection standards; and (3) multiple trade-offs result in improved blockchain scalability. Consideration of these trade-offs will be important to the specific environment in which electronic medical records are being developed. This review also uses its findings to suggest useful design choices for a hypothetical National Health Service blockchain. International Registered Report Identifier (IRRID): RR2-10.2196/10994 %M 31094344 %R 10.2196/12426 %U https://www.jmir.org/2019/5/e12426/ %U https://doi.org/10.2196/12426 %U http://www.ncbi.nlm.nih.gov/pubmed/31094344 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 6 %N 2 %P e12553 %T Evaluation of an Anesthesia Dashboard Functional Model Based on a Manufacturer-Independent Communication Standard: Comparative Feasibility Study %A Ohligs,Marian %A Pereira,Carina %A Voigt,Verena %A Koeny,Marcus %A Janß,Armin %A Rossaint,Rolf %A Czaplik,Michael %+ Department of Anesthesiology, Faculty of Medicine, Rheinisch-Westfälische Technische Hochschule Aachen University, Pauwelsstrasse 30, Aachen, 52074, Germany, 49 241 80 ext 83136, mohligs@ukaachen.de %K operating room %K anesthesia %K interconnection %K networking %K human-computer interaction %K process optimization %K intelligent alarms %K decision-support systems %K 11073 SDC %K service-oriented device connectivity %D 2019 %7 01.05.2019 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Current anesthesia workspaces consist of several technical devices, such as patient monitors, anesthesia machines, among others. Commonly, they are produced by different manufacturers; thus, they differ in terms of their modus operandi, user interface, and representation of alarms. Merging the information from these devices using a single joint protocol and displaying it in a single graphical user interface could lead to a general improvement in perioperative management. For this purpose, the recently approved and published Institute of Electrical and Electronics Engineers 11073 service-oriented device connectivity standard was implemented. Objective: This paper aims to develop and then evaluate an anesthesia workstation (ANWS) functional model in terms of usability, fulfillment of clinical requirements, and expected improvements in patient safety. Methods: To compare the self-developed ANWS with the conventional system, a pilot observational study was conducted at the University Hospital Aachen, Germany. A total of 5 anesthesiologists were asked to perform different tasks using the ANWS and then the conventional setup. For evaluation purposes, response times were measured and an interaction-centered usability test with an eye-tracking system was carried out. Finally, the subjects were asked to fill in a questionnaire in order to measure user satisfaction. Results: Response times were significantly higher when using the ANWS, but decreased considerably after one repetition. Furthermore, usability was rated as excellent (≥95) according to the System Usability Scale score, and the majority of clinical requirements were met. Conclusions: In general, the results were highly encouraging, considering that the ANWS was only a functional model, as well as the lack of training of the participants. However, further studies are necessary to improve the universal user interface and the interplay of the various networked devices. %M 31042150 %R 10.2196/12553 %U http://humanfactors.jmir.org/2019/2/e12553/ %U https://doi.org/10.2196/12553 %U http://www.ncbi.nlm.nih.gov/pubmed/31042150 %0 Journal Article %@ 2291-5222 %I JMIR Publications %V 7 %N 4 %P e13226 %T A Digital Cognitive Aid for Anesthesia to Support Intraoperative Crisis Management: Results of the User-Centered Design Process %A Schild,Stefanie %A Sedlmayr,Brita %A Schumacher,Ann-Kathrin %A Sedlmayr,Martin %A Prokosch,Hans-Ulrich %A St.Pierre,Michael %A , %+ Department of Medical Informatics, Biometrics and Epidemiology, Chair of Medical Informatics, Friedrich-Alexander University Erlangen-Nürnberg, Wetterkreuz 13, Erlangen, 91058, Germany, 49 9131 85 26785, stefanie.schild@fau.de %K anesthesiology %K checklist %K crew resource management, healthcare %K emergency treatment %K ergonomics %K human factors %K practice guideline %K reference books, medical %K resuscitation %K user-computer interface %D 2019 %7 29.04.2019 %9 Original Paper %J JMIR Mhealth Uhealth %G English %X Background: Stressful situations during intraoperative emergencies have negative impact on human cognitive functions. Consequently, task performance may decrease and patient safety may be compromised. Cognitive aids can counteract these effects and support anesthesiologists in their crisis management. The Professional Association of German Anesthesiologists set up a project to develop a comprehensive set of digital cognitive aids for intraoperative emergencies. A parallel development for several software platforms and stationary and mobile devices will accommodate the inhomogeneity of the information technology infrastructure within German anesthesia departments. Objective: This paper aimed to provide a detailed overview of how the task of developing a digital cognitive aid for intraoperative crisis management in anesthesia was addressed that meets user requirements and is highly user-friendly. Methods: A user-centered design (UCD) process was conducted to identify, specify, and supplement the requirements for a digital cognitive aid. The study covered 4 aspects: analysis of the context of use, specification of user requirements, development of design solutions, and evaluation of design solutions. Three prototypes were developed and evaluated by end users of the application. Following each evaluation, the new requirements were prioritized and used for redesign. For the first and third prototype, the System Usability Scale (SUS) score was determined. The second prototype was evaluated with an extensive Web-based questionnaire. The evaluation of the third prototype included a think-aloud protocol. Results: The chosen methods enabled a comprehensive collection of requirements and helped to improve the design of the application. The first prototype achieved an average SUS score of 74 (SD 12), indicating good usability. The second prototype included the following main revisions: 2-column layout, initial selection of patient type (infant, adult, or parturient), 4 offered search options, and the option to check off completed action steps. Its evaluation identified the following major revision points: add quick selection for resuscitation checklists, design the top bar and tabs slightly larger, and add more pictograms to the text. The third prototype achieved an average SUS score of 77 (SD 15). The evaluation of the think-aloud protocol revealed a good intuitiveness of the application and identified a missing home button as the main issue. Conclusions: Anesthesiology—as an acute medical field—is particularly characterized by its high demands on decision making and action in dynamic, or time-critical situations. The integration of usability aspects is essential for everyday and emergency suitability. The UCD process allowed us to develop a prototypical digital cognitive aid, exhibiting high usability and user satisfaction in the demanding environment of anesthesiological emergencies. Both aspects are essential to increase the acceptance of the application in later stages. The study approach, combining different methods for determining user requirements, may be useful for other implementation projects in a highly demanding environment. %M 31033445 %R 10.2196/13226 %U http://mhealth.jmir.org/2019/4/e13226/ %U https://doi.org/10.2196/13226 %U http://www.ncbi.nlm.nih.gov/pubmed/31033445 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 6 %N 2 %P e12155 %T Anesthesiology Control Tower—Feasibility Assessment to Support Translation (ACTFAST): Mixed-Methods Study of a Novel Telemedicine-Based Support System for the Operating Room %A Murray-Torres,Teresa %A Casarella,Aparna %A Bollini,Mara %A Wallace,Frances %A Avidan,Michael S %A Politi,Mary C %+ Department of Anesthesiology, Washington University School of Medicine in St. Louis, Washington University in St. Louis, 660 S Euclid Avenue, Box 8054, St. Louis, MO,, United States, 1 314 226 8880, murrayt@wustl.edu %K clinical decision support systems %K usability %K anesthesiology %K telemedicine %D 2019 %7 23.04.2019 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Despite efforts to improve patient outcomes, major morbidity and mortality remain common after surgery. Health information technologies that provide decision support for clinicians might improve perioperative and postoperative patient care. Evaluating the usability of these technologies and barriers to their implementation can facilitate their acceptance within health systems. Objective: This manuscript describes usability testing and refinement of an innovative telemedicine-based clinical support system, the Anesthesiology Control Tower (ACT). It also reports stakeholders’ perceptions of the barriers and facilitators to implementation of the intervention. Methods: Three phases of testing were conducted in an iterative manner. Phase 1 testing employed a think-aloud protocol analysis to identify surface-level usability problems with individual software components of the ACT and its structure. Phase 2 testing involved an extended qualitative and quantitative real-world usability analysis. Phase 3 sought to identify major barriers and facilitators to implementation of the ACT through semistructured interviews with key stakeholders. Results: Phase 1 and phase 2 usability testing sessions identified numerous usability problems with the software components of the ACT. The ACT platform was revised in seven iterations in response to these usability concerns. Initial satisfaction with the ACT, as measured by standardized instruments, was below commonly accepted cutoffs for these measures. Satisfaction improved to acceptable levels over the course of revision and testing. A number of barriers to implementation were also identified and addressed during the refinement of the ACT intervention. Conclusions: The ACT model can improve the standard of perioperative anesthesia care. Through our thorough and iterative usability testing process and stakeholder assessment of barriers and facilitators, we enhanced the acceptability of this novel technology and improved our ability to implement this innovation into routine practice. International Registered Report Identifier (IRRID): RR2-10.1186/s40814-018-0233-4 %M 31012859 %R 10.2196/12155 %U http://humanfactors.jmir.org/2019/2/e12155/ %U https://doi.org/10.2196/12155 %U http://www.ncbi.nlm.nih.gov/pubmed/31012859 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 6 %N 2 %P e12469 %T Interruptive Versus Noninterruptive Clinical Decision Support: Usability Study %A Blecker,Saul %A Pandya,Rishi %A Stork,Susan %A Mann,Devin %A Kuperman,Gilad %A Shelley,Donna %A Austrian,Jonathan S %+ Department of Population Health, New York University School of Medicine, 227 E 30th St, 6th Floor, New York, NY, 10016, United States, 1 6465012513, saul.blecker@nyumc.org %K clinical decision support %K hospital %K electronic health records %D 2019 %7 17.04.2019 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Clinical decision support (CDS) has been shown to improve compliance with evidence-based care, but its impact is often diminished because of issues such as poor usability, insufficient integration into workflow, and alert fatigue. Noninterruptive CDS may be less subject to alert fatigue, but there has been little assessment of its usability. Objective: This study aimed to study the usability of interruptive and noninterruptive versions of a CDS. Methods: We conducted a usability study of a CDS tool that recommended prescribing an angiotensin-converting enzyme inhibitor for inpatients with heart failure. We developed 2 versions of the CDS: an interruptive alert triggered at order entry and a noninterruptive alert listed in the sidebar of the electronic health record screen. Inpatient providers were recruited and randomly assigned to use the interruptive alert followed by the noninterruptive alert or vice versa in a laboratory setting. We asked providers to “think aloud” while using the CDS and then conducted a brief semistructured interview about usability. We used a constant comparative analysis informed by the CDS Five Rights framework to analyze usability testing. Results: A total of 12 providers participated in usability testing. Providers noted that the interruptive alert was readily noticed but generally impeded workflow. The noninterruptive alert was felt to be less annoying but had lower visibility, which might reduce engagement. Provider role seemed to influence preferences; for instance, some providers who had more global responsibility for patients seemed to prefer the noninterruptive alert, whereas more task-oriented providers generally preferred the interruptive alert. Conclusions: Providers expressed trade-offs between impeding workflow and improving visibility with interruptive and noninterruptive versions of a CDS. In addition, 2 potential approaches to effective CDS may include targeting alerts by provider role or supplementing a noninterruptive alert with an occasional, well-timed interruptive alert. %M 30994460 %R 10.2196/12469 %U http://humanfactors.jmir.org/2019/2/e12469/ %U https://doi.org/10.2196/12469 %U http://www.ncbi.nlm.nih.gov/pubmed/30994460 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 6 %N 2 %P e12471 %T Live Usability Testing of Two Complex Clinical Decision Support Tools: Observational Study %A Richardson,Safiya %A Feldstein,David %A McGinn,Thomas %A Park,Linda S %A Khan,Sundas %A Hess,Rachel %A Smith,Paul D %A Mishuris,Rebecca Grochow %A McCullagh,Lauren %A Mann,Devin %+ Donald and Barbara Zucker School of Medicine at Hofstra/Northwell, 500 Hofstra University, Hempstead, NY, 11549, United States, 1 404 695 7883, srichard12@northwell.edu %K usability %K usability testing %K user experience %K clinical decision support %K health informatics %K provider adoption %K workflow %K live usability %K clinical prediction rules %D 2019 %7 15.04.2019 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Potential of the electronic health records (EHR) and clinical decision support (CDS) systems to improve the practice of medicine has been tempered by poor design and the resulting burden they place on providers. CDS is rarely tested in the real clinical environment. As a result, many tools are hard to use, placing strain on providers and resulting in low adoption rates. The existing CDS usability literature relies primarily on expert opinion and provider feedback via survey. This is the first study to evaluate CDS usability and the provider-computer-patient interaction with complex CDS in the real clinical environment. Objective: This study aimed to further understand the barriers and facilitators of meaningful CDS usage within a real clinical context. Methods: This qualitative observational study was conducted with 3 primary care providers during 6 patient care sessions. In patients with the chief complaint of sore throat, a CDS tool built with the Centor Score was used to stratify the risk of group A Streptococcus pharyngitis. In patients with a chief complaint of cough or upper respiratory tract infection, a CDS tool built with the Heckerling Rule was used to stratify the risk of pneumonia. During usability testing, all human-computer interactions, including audio and continuous screen capture, were recorded using the Camtasia software. Participants’ comments and interactions with the tool during clinical sessions and participant comments during a postsession brief interview were placed into coding categories and analyzed for generalizable themes. Results: In the 6 encounters observed, primary care providers toggled between addressing either the computer or the patient during the visit. Minimal time was spent listening to the patient without engaging the EHR. Participants mostly used the CDS tool with the patient, asking questions to populate the calculator and discussing the results of the risk assessment; they reported the ability to do this as the major benefit of the tool. All providers were interrupted during their use of the CDS tool by the need to refer to other sections of the chart. In half of the visits, patients’ clinical symptoms challenged the applicability of the tool to calculate the risk of bacterial infection. Primary care providers rarely used the incorporated incentives for CDS usage, including progress notes and patient instructions. Conclusions: Live usability testing of these CDS tools generated insights about their role in the patient-provider interaction. CDS may contribute to the interaction by being simultaneously viewed by the provider and patient. CDS can improve usability and lessen the strain it places on providers by being short, flexible, and customizable to unique provider workflow. A useful component of CDS is being as widely applicable as possible and ensuring that its functions represent the fastest way to perform a particular task. %M 30985283 %R 10.2196/12471 %U http://humanfactors.jmir.org/2019/2/e12471/ %U https://doi.org/10.2196/12471 %U http://www.ncbi.nlm.nih.gov/pubmed/30985283 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 6 %N 1 %P e13121 %T Computerized Clinical Decision Support System for Emergency Department–Initiated Buprenorphine for Opioid Use Disorder: User-Centered Design %A Ray,Jessica M %A Ahmed,Osama M %A Solad,Yauheni %A Maleska,Matthew %A Martel,Shara %A Jeffery,Molly M %A Platts-Mills,Timothy F %A Hess,Erik P %A D’Onofrio,Gail %A Melnick,Edward R %+ Yale School of Medicine, 464 Congress Ave, Suite 260, New Haven, CT, 06519, United States, 1 9176081760, edward.melnick@yale.edu %K user-centered design %K decision support systems, clinical %K opioid-related disorders %K opiate substitution treatment %K health information technology %D 2019 %7 27.02.2019 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Emergency departments (EDs) frequently care for individuals with opioid use disorder (OUD). Buprenorphine (BUP) is an effective treatment option for patients with OUD that can safely be initiated in the ED. At present, BUP is rarely initiated as a part of routine ED care. Clinical decision support (CDS) could accelerate adoption of ED-initiated BUP into routine emergency care. Objective: This study aimed to design and formatively evaluate a user-centered decision support tool for ED initiation of BUP for patients with OUD. Methods: User-centered design with iterative prototype development was used. Initial observations and interviews identified workflows and information needs. The design team and key stakeholders reviewed prototype designs to ensure accuracy. A total of 5 prototypes were evaluated and iteratively refined based on input from 26 attending and resident physicians. Results: Early feedback identified concerns with the initial CDS design: an alert with several screens. The timing of the alert led to quick dismissal without using the tool. User feedback on subsequent iterations informed the development of a flexible tool to support clinicians with varied levels of experience with the intervention by providing both one-click options for direct activation of care pathways and user-activated support for critical decision points. The final design resolved challenging navigation issues through targeted placement, color, and design of the decision support modules and care pathways. In final testing, users expressed that the tool could be easily learned without training and was reasonable for use during routine emergency care. Conclusions: A user-centered design process helped designers to better understand users’ needs for a Web-based clinical decision tool to support ED initiation of BUP for OUD. The process identified varying needs across user experience and familiarity with the protocol, leading to a flexible design supporting both direct care pathways and user-initiated decision support. %M 30810531 %R 10.2196/13121 %U http://humanfactors.jmir.org/2019/1/e13121/ %U https://doi.org/10.2196/13121 %U http://www.ncbi.nlm.nih.gov/pubmed/30810531 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 6 %N 1 %P e9317 %T The Usability of Electronic Medical Record Systems Implemented in Sub-Saharan Africa: A Literature Review of the Evidence %A Kavuma,Michael %+ Department of Tele-Health, College of Health Sciences, University of KwaZulu-Natal, Nelson R Mandela School of Medicine, 719 Umbilo Road 4001, Congella 4013, Durban,, South Africa, 27 782632775, mikavs@gmail.com %K review %K computer systems %K delivery of health care %K sub-Saharan Africa %D 2019 %7 25.02.2019 %9 Review %J JMIR Hum Factors %G English %X Background: Electronic medical record (EMR) systems hold the exciting promise of accurate, real-time access to patient health care data and great potential to improve the quality of patient care through decision support to clinicians. This review evaluated the usability of EMR systems implemented in sub-Saharan Africa based on a usability evaluation criterion developed by the Healthcare Information and Management Systems Society (HIMSS). Objective: This review aimed to evaluate EMR system implementations in sub-Saharan Africa against a well-defined evaluation methodology and assess their usability based on a defined set of metrics. In addition, the review aimed to identify the extent to which usability has been an enabling or hindering factor in the implementation of EMR systems in sub-Saharan Africa. Methods: Five key metrics for evaluating EMR system usability were developed based on the methodology proposed by HIMSS. These were efficiency, effectiveness, ease of learning, cognitive load, and user satisfaction. A 5-point rating system was developed for the review. EMR systems in 19 reviewed publications were scored based on this rating system. It awarded 5 points per metric to any EMR system that was identified as excellent, 4 points for good, 3 points for fair, 2 points for poor, and 1 point for bad. In addition, each of the 5 key metrics carried a maximum weighted score of 20. The percentage scores for each metric were then computed from the weighted scores from which the final overall usability score was derived. Results: In possibly contributing to the usability of implemented EMR systems, ease of learning obtained the highest percentage score of 71% (SD 1.09) followed by cognitive load in second place with a score of 68% (SD 1.62). Effectiveness followed closely in third place at 67% (SD 1.47) and efficiency was in fourth place at 64% (SD 1.04). User satisfaction came in last at 63% (SD 1.70). The overall usability score for all systems was calculated to be 66%. Conclusions: The usability of EMR systems implemented in sub-Saharan Africa has been good with ease of learning possibly being the biggest positive contributor to this rating. Cognitive load and effectiveness have also possibly positively influenced the usability of EMR systems, whereas efficiency and user satisfaction have perhaps contributed least to positively influencing EMR system usability. %M 30801251 %R 10.2196/humanfactors.9317 %U http://humanfactors.jmir.org/2019/1/e9317/ %U https://doi.org/10.2196/humanfactors.9317 %U http://www.ncbi.nlm.nih.gov/pubmed/30801251 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 6 %N 1 %P e10245 %T Improving Provider Adoption With Adaptive Clinical Decision Support Surveillance: An Observational Study %A Khan,Sundas %A Richardson,Safiya %A Liu,Andrew %A Mechery,Vinodh %A McCullagh,Lauren %A Schachter,Andy %A Pardo,Salvatore %A McGinn,Thomas %+ Donald and Barbara Zucker School of Medicine at Hofstra/Northwell, 600 Community Drive, Manhasset, NY, 11030, United States, 1 516 600 1419, skhan31@northwell.edu %K pulmonary embolism %K clinical decision support %K evidence-based medicine %D 2019 %7 20.02.2019 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Successful clinical decision support (CDS) tools can help use evidence-based medicine to effectively improve patient outcomes. However, the impact of these tools has been limited by low provider adoption due to overtriggering, leading to alert fatigue. We developed a tracking mechanism for monitoring trigger (percent of total visits for which the tool triggers) and adoption (percent of completed tools) rates of a complex CDS tool based on the Wells criteria for pulmonary embolism (PE). Objective: We aimed to monitor and evaluate the adoption and trigger rates of the tool and assess whether ongoing tool modifications would improve adoption rates. Methods: As part of a larger clinical trial, a CDS tool was developed using the Wells criteria to calculate pretest probability for PE at 2 tertiary centers’ emergency departments (EDs). The tool had multiple triggers: any order for D-dimer, computed tomography (CT) of the chest with intravenous contrast, CT pulmonary angiography (CTPA), ventilation-perfusion scan, or lower extremity Doppler ultrasound. A tracking dashboard was developed using Tableau to monitor real-time trigger and adoption rates. Based on initial low provider adoption rates of the tool, we conducted small focus groups with key ED providers to elicit barriers to tool use. We identified overtriggering of the tool for non-PE-related evaluations and inability to order CT testing for intermediate-risk patients. Thus, the tool was modified to allow CT testing for the intermediate-risk group and not to trigger for CT chest with intravenous contrast orders. A dialogue box, “Are you considering PE for this patient?” was added before the tool triggered to account for CTPAs ordered for aortic dissection evaluation. Results: In the ED of tertiary center 1, 95,295 patients visited during the academic year. The tool triggered for an average of 509 patients per month (average trigger rate 2036/30,234, 6.73%) before the modifications, reducing to 423 patients per month (average trigger rate 1629/31,361, 5.22%). In the ED of tertiary center 2, 88,956 patients visited during the academic year, with the tool triggering for about 473 patients per month (average trigger rate 1892/29,706, 6.37%) before the modifications and for about 400 per month (average trigger rate 1534/30,006, 5.12%) afterward. The modifications resulted in a significant 4.5- and 3-fold increase in provider adoption rates in tertiary centers 1 and 2, respectively. The modifications increased the average monthly adoption rate from 23.20/360 (6.5%) tools to 81.60/280.20 (29.3%) tools and 46.60/318.80 (14.7%) tools to 111.20/263.40 (42.6%) tools in centers 1 and 2, respectively. Conclusions: Close postimplementation monitoring of CDS tools may help improve provider adoption. Adaptive modifications based on user feedback may increase targeted CDS with lower trigger rates, reducing alert fatigue and increasing provider adoption. Iterative improvements and a postimplementation monitoring dashboard can significantly improve adoption rates. %M 30785410 %R 10.2196/10245 %U http://humanfactors.jmir.org/2019/1/e10245/ %U https://doi.org/10.2196/10245 %U http://www.ncbi.nlm.nih.gov/pubmed/30785410 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 6 %N 1 %P e11846 %T Nurses’ Perceptions of a Care Plan Information Technology Solution With Hundreds of Clinical Practice Guidelines in Adult Intensive Care Units: Survey Study %A Sowan,Azizeh Khaled %A Leibas,Meghan %A Tarriela,Albert %A Reed,Charles %+ School of Nursing, University of Texas Health at San Antonio, 7703 Floyd Curl Drive - MC 7975, San Antonio, TX, 78229, United States, 1 210 567 5799, sowan@uthscsa.edu %K usability %K patient care planning %K evidence-based practice %K nursing %K documentation %K information technology %K survey %D 2019 %7 12.02.2019 %9 Original Paper %J JMIR Hum Factors %G English %X Background: The integration of clinical practice guidelines (CPGs) into the nursing care plan and documentation systems aims to translate evidence into practice, improve safety and quality of care, and standardize care processes. Objective: This study aimed to evaluate nurses’ perceptions of the usability of a nursing care plan solution that includes 234 CPGs. Methods: A total of 100 nurses from 4 adult intensive care units (ICUs) responded to a survey measuring nurses’ perceptions of system usability. The survey included 37 rated items and 3 open-ended questions. Results: Nurses’ perceptions were favorable with more than 60.0% (60/100) in agreement on 12 features of the system and negative to moderate with 20.0% (20/100), to 59.0% (59/100) in agreement on 19 features. The majority of the nurses (80/100, 80.0% to 90/100, 90.0%) agreed on 4 missing safety features within the system. More than half of the nurses believed they would benefit from refresher classes on system use. Overall satisfaction with the system was just above average (54/100, 54.0%). Common positive themes from the narrative data were related to the system serving as a reminder for complete documentation and individualizing patient care. Common negative aspects were related to duplicate charting, difficulty locating CPGs, missing unit-specific CPGs, irrelevancy of information, and lack of perceived system value on patient outcomes. No relationship was found between years of system use or ICU experience and satisfaction with the system (P=.10 to P=.25). Conclusions: Care plan systems in ICUs should be easy to navigate; support efficient documentation; present relevant, unit-specific, and easy-to-find information; endorse interdisciplinary communication; and improve safety and quality of care. %M 30747713 %R 10.2196/11846 %U http://humanfactors.jmir.org/2019/1/e11846/ %U https://doi.org/10.2196/11846 %U http://www.ncbi.nlm.nih.gov/pubmed/30747713 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 20 %N 10 %P e11027 %T Improving the Understanding of Test Results by Substituting (Not Adding) Goal Ranges: Web-Based Between-Subjects Experiment %A Scherer,Aaron M %A Witteman,Holly O %A Solomon,Jacob %A Exe,Nicole L %A Fagerlin,Angela %A Zikmund-Fisher,Brian J %+ Department of Internal Medicine, University of Iowa, 200 Hawkins Drive, Iowa City, IA, 52242, United States, 1 3193356707, aaron-scherer@uiowa.edu %K decision making, education of patients, electronic health record, computer graphics, clinical laboratory information systems %D 2018 %7 19.10.2018 %9 Original Paper %J J Med Internet Res %G English %X Background: Most displays of laboratory test results include a standard reference range. For some patients (eg, those with chronic conditions), however, getting a result within the standard range may be unachievable, inappropriate, or even harmful. Objective: The objective of our study was to test the impact of including clinically appropriate goal ranges outside the standard range in the visual displays of laboratory test results. Methods: Participants (N=6776) from a demographically diverse Web-based panel viewed hypothetical glycated hemoglobin (HbA1c) test results (HbA1c either 6.2% or 8.2%) as part of a type 2 diabetes management scenario. Test result visual displays included either a standard range (4.5%-5.7%) only, a goal range (6.5%-7.5%) added to the standard range, or the goal range only. The results were displayed in 1 of the following 3 display formats: (1) a table; (2) a simple, two-colored number line (simple line); or (3) a number line with diagnostic categories indicated via colored blocks (block line). Primary outcome measures were comprehension of and negative reactions to test results. Results: While goal range information did not influence the understanding of HbA1c=8.2% results, the goal range only display produced higher levels of comprehension and decreased negative reactions to HbA1c=6.2% test results compared with the no goal range and goal range added conditions. Goal range information was less helpful in the block line condition versus the other formats. Conclusions: Replacing the standard range with a clinically appropriate goal range could help patients better understand how their test results relate to their personal targets. %M 30341053 %R 10.2196/11027 %U http://www.jmir.org/2018/10/e11027/ %U https://doi.org/10.2196/11027 %U http://www.ncbi.nlm.nih.gov/pubmed/30341053 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 5 %N 4 %P e11826 %T Data Visualizations to Support Health Practitioners’ Provision of Personalized Care for Patients With Cancer and Multiple Chronic Conditions: User-Centered Design Study %A Backonja,Uba %A Haynes,Sarah C %A Kim,Katherine K %+ Betty Irene Moore School of Nursing, University of California Davis, 2450 48th Street, Suite 2600, Sacramento, CA, 95817, United States, 1 510 761 5461, kathykim@ucdavis.edu %K cancer care facilities %K informatics %K patient-centered care %K patient-generated health data %K precision medicine %K visualization %D 2018 %7 16.10.2018 %9 Original Paper %J JMIR Hum Factors %G English %X Background: There exists a challenge of understanding and integrating various types of data collected to support the health of individuals with multiple chronic conditions engaging in cancer care. Data visualization has the potential to address this challenge and support personalized cancer care. Objective: The aim of the study was to assess the health care practitioners’ perceptions of and feedback regarding visualizations developed to support the care of individuals with multiple chronic conditions engaging in cancer care. Methods: Medical doctors (n=4) and registered nurses (n=4) providing cancer care at an academic medical center in the western United States provided feedback on visualization mock-ups. Mock-up designs were guided by current health informatics and visualization literature and the Munzner Nested Model for Visualization Design. User-centered design methods, a mock patient persona, and a scenario were used to elicit insights from participants. Directed content analysis was used to identify themes from session transcripts. Means and SDs were calculated for health care practitioners’ rankings of overview visualizations. Results: Themes identified were data elements, supportive elements, confusing elements, interpretation, and use of visualization. Overall, participants found the visualizations useful and with the potential to provide personalized care. Use of color, reference lines, and familiar visual presentations (calendars, line graphs) were noted as helpful in interpreting data. Conclusions: Visualizations guided by a framework and literature can support health care practitioners’ understanding of data for individuals with multiple chronic conditions engaged in cancer care. This understanding has the potential to support the provision of personalized care. %M 30327290 %R 10.2196/11826 %U http://humanfactors.jmir.org/2018/4/e11826/ %U https://doi.org/10.2196/11826 %U http://www.ncbi.nlm.nih.gov/pubmed/30327290 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 5 %N 4 %P e28 %T Applying Persuasive Design Techniques to Influence Data-Entry Behaviors in Primary Care: Repeated Measures Evaluation Using Statistical Process Control %A St-Maurice,Justin %A Burns,Catherine %A Wolting,Justin %+ Systems Design Engineering, University of Waterloo, 200 University Avenue West, Waterloo, ON, N2L 3G1, Canada, 1 5198884567, jstmaurice@conestogac.on.ca %K data collection %K data entry %K data accuracy %K health care %K persuasive design %K persuasive systems design framework %K user interface %D 2018 %7 11.10.2018 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Persuasive design is an approach that seeks to change the behaviors of users. In primary care, clinician behaviors and attitudes are important precursors to structured data entry, and there is an impact on overall data quality. We hypothesized that persuasive design changes data-entry behaviors in clinicians and thus improves data quality. Objective: The objective of this study was to use persuasive design principles to change clinician data-entry behaviors in a primary care environment and to increase data quality of data held in a family health team’s reporting system. Methods: We used the persuasive systems design framework to describe the persuasion context. Afterward, we designed and implemented new features into a summary screen that leveraged several persuasive design principles. We tested the influence of the new features by measuring its impact on 3 data quality measures (same-day entry, record completeness, and data validity). We also measured the impacts of the new features with a paired pre-post t test and generated XmR charts to contextualize the results. Survey responses were also collected from users. Results: A total of 53 users used the updated system that incorporated the new features over the course of 8 weeks. Based on a pre-post analysis, the new summary screen successfully encouraged users to enter more of their data on the same day as their encounter. On average, the percentage of same-day entries rose by 10.3% for each user (P<.001). During the first month of the postimplementation period, users compensated by sacrificing aspects of data completeness before returning to normal in the second month. Improvements to record validity were marginal over the study period (P=.05). Statistical process control techniques allowed us to study the XmR charts to contextualize our results and understand trends throughout the study period. Conclusions: By conducting a detailed systems analysis and introducing new persuasive design elements into a data-entry system, we demonstrated that it was possible to change data-entry behavior and influence data quality in a reporting system. The results show that using persuasive design concepts may be effective in influencing data-entry behaviors in clinicians. There may be opportunities to continue improving this approach, and further work is required to perfect and test additional designs. Persuasive design is a viable approach to encourage clinician user change and could support better data capture in the field of medical informatics. %M 30309836 %R 10.2196/humanfactors.9029 %U http://humanfactors.jmir.org/2018/4/e28/ %U https://doi.org/10.2196/humanfactors.9029 %U http://www.ncbi.nlm.nih.gov/pubmed/30309836 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 6 %N 3 %P e10184 %T Emergency Physician Use of the Alberta Netcare Portal, a Province-Wide Interoperable Electronic Health Record: Multi-Method Observational Study %A Graham,Timothy AD %A Ballermann,Mark %A Lang,Eddy %A Bullard,Michael J %A Parsons,Denise %A Mercuur,Gabriella %A San Agustin,Pat %A Ali,Samina %+ Alberta Health Services, 5th Floor, 44 Capital Boulevard, 10044-108 Street, Edmonton, AB, T5J 3S7, Canada, 1 780 431 2549, tim.graham@ahs.ca %K ambulatory care facilities %K cross-sectional studies %K electronic health records %K utilization %K hospital emergency service %K statistics and numerical data %K health information exchange %K humans %K information dissemination %K medical record linkage %K program evaluation %D 2018 %7 25.9.2018 %9 Original Paper %J JMIR Med Inform %G English %X Background: The adoption and use of an electronic health record (EHR) can facilitate real-time access to key health information and support improved outcomes. Many Canadian provinces use interoperable EHRs (iEHRs) to facilitate health information exchange, but the clinical use and utility of iEHRs has not been well described. Objective: The aim of this study was to describe the use of a provincial iEHR known as the Alberta Netcare Portal (ANP) in 4 urban Alberta emergency departments. The secondary objectives were to characterize the time spent using the respective electronic tools and identify the aspects that were perceived as most useful by emergency department physicians. Methods: In this study, we have included 4 emergency departments, 2 using paper-based ordering (University of Alberta Hospital [UAH] and Grey Nuns Community Hospital [GNCH]) and 2 using a commercial vendor clinical information system (Peter Lougheed Centre [PLC] and Foothills Medical Centre [FMC]). Structured clinical observations of ANP use and system audit logs analysis were compared at the 4 sites from October 2014 to March 2016. Results: Observers followed 142 physicians for a total of 566 hours over 376 occasions. The median percentage of observed time spent using ANP was 8.5% at UAH (interquartile range, IQR, 3.7%-13.3%), 4.4% at GNCH (IQR 2.4%-4.4%), 4.6% at FMC (IQR 2.4%-7.6%), and 5.1% at PLC (IQR 3.0%-7.7%). By combining administrative and access audit data, the median number of ANP screens (ie, results and reports displayed on a screen) accessed per patient visit were 20 at UAH (IQR 6-67), 9 at GNCH (IQR 4-29), 7 at FMC (IQR 2-18), and 5 at PLC (IQR 2-14). When compared with the structured clinical observations, the statistical analysis of screen access data showed that ANP was used more at UAH than the other sites. Conclusions: This study shows that the iEHR is well utilized at the 4 sites studied, and the usage patterns implied clinical value. Use of the ANP was highest in a paper-based academic center and lower in the centers using a commercial emergency department clinical information system. More study about the clinical impacts of using iEHRs in the Canadian context including longer term impacts on quality of practice and safety are required. %M 30274967 %R 10.2196/10184 %U http://medinform.jmir.org/2018/3/e10184/ %U https://doi.org/10.2196/10184 %U http://www.ncbi.nlm.nih.gov/pubmed/30274967 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 5 %N 3 %P e10070 %T Software for Administering the National Cancer Institute’s Patient-Reported Outcomes Version of the Common Terminology Criteria for Adverse Events: Usability Study %A Schoen,Martin W %A Basch,Ethan %A Hudson,Lori L %A Chung,Arlene E %A Mendoza,Tito R %A Mitchell,Sandra A %A St. Germain,Diane %A Baumgartner,Paul %A Sit,Laura %A Rogak,Lauren J %A Shouery,Marwan %A Shalley,Eve %A Reeve,Bryce B %A Fawzy,Maria R %A Bhavsar,Nrupen A %A Cleeland,Charles %A Schrag,Deborah %A Dueck,Amylou C %A Abernethy,Amy P %+ Division of Hematology/Oncology, Department of Medicine, University of North Carolina School of Medicine, 170 Manning Drive, Chapel Hill, NC,, United States, 1 919 843 3055, ebasch@med.unc.edu %K usability %K patient-reported outcomes %K symptoms %K adverse events %K PRO-CTCAE %K cancer clinical trials %D 2018 %7 16.07.2018 %9 Original Paper %J JMIR Hum Factors %G English %X Background: The US National Cancer Institute (NCI) developed software to gather symptomatic adverse events directly from patients participating in clinical trials. The software administers surveys to patients using items from the Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE) through Web-based or automated telephone interfaces and facilitates the management of survey administration and the resultant data by professionals (clinicians and research associates). Objective: The purpose of this study was to iteratively evaluate and improve the usability of the PRO-CTCAE software. Methods: Heuristic evaluation of the software functionality was followed by semiscripted, think-aloud protocols in two consecutive rounds of usability testing among patients with cancer, clinicians, and research associates at 3 cancer centers. We conducted testing with patients both in clinics and at home (remotely) for both Web-based and telephone interfaces. Furthermore, we refined the software between rounds and retested. Results: Heuristic evaluation identified deviations from the best practices across 10 standardized categories, which informed initial software improvement. Subsequently, we conducted user-based testing among 169 patients and 47 professionals. Software modifications between rounds addressed identified issues, including difficulty using radio buttons, absence of survey progress indicators, and login problems (for patients) as well as scheduling of patient surveys (for professionals). The initial System Usability Scale (SUS) score for the patient Web-based interface was 86 and 82 (P=.22) before and after modifications, respectively, whereas the task completion score was 4.47, which improved to 4.58 (P=.39) after modifications. Following modifications for professional users, the SUS scores improved from 71 to 75 (P=.47), and the mean task performance improved significantly (4.40 vs 4.02; P=.001). Conclusions: Software modifications, informed by rigorous assessment, rendered a usable system, which is currently used in multiple NCI-sponsored multicenter cancer clinical trials. Trial Registration: ClinicalTrials.gov NCT01031641; https://clinicaltrials.gov/ct2/show/NCT01031641 (Archived by WebCite at http://www.webcitation.org/708hTjlTl) %M 30012546 %R 10.2196/10070 %U http://humanfactors.jmir.org/2018/3/e10070/ %U https://doi.org/10.2196/10070 %U http://www.ncbi.nlm.nih.gov/pubmed/30012546 %0 Journal Article %@ 2371-4379 %I JMIR Publications %V 3 %N 3 %P e10431 %T Design and Development of a Context-Aware Knowledge-Based Module for Identifying Relevant Information and Information Gaps in Patients With Type 1 Diabetes Self-Collected Health Data %A Giordanengo,Alain %A Øzturk,Pinar %A Hansen,Anne Helen %A Årsand,Eirik %A Grøttland,Astrid %A Hartvigsen,Gunnar %+ Norwegian Centre for E-health Research, University Hospital of North Norway, Postboks 35, Tromsø, 9038, Norway, 47 91715718, alain.giordanengo@ehealthresearch.no %K context aware %K knowledge-based system %K diabetes %K self-collected health data %K information gaps %D 2018 %7 11.07.2018 %9 Original Paper %J JMIR Diabetes %G English %X Background: Patients with diabetes use an increasing number of self-management tools in their daily life. However, health institutions rarely use the data generated by these services mainly due to (1) the lack of data reliability, and (2) medical workers spending too much time extracting relevant information from the vast amount of data produced. This work is part of the FullFlow project, which focuses on self-collected health data sharing directly between patients’ tools and EHRs. Objective: The main objective is to design and implement a prototype for extracting relevant information and documenting information gaps from self-collected health data by patients with type 1 diabetes using a context-aware approach. The module should permit (1) clinicians to assess the reliability of the data and to identify issues to discuss with their patients, and (2) patients to understand the implication their lifestyle has on their disease. Methods: The identification of context and the design of the system relied on (1) 2 workshops in which the main author participated, 1 patient with type 1 diabetes, and 1 clinician, and (2) a co-design session involving 5 patients with type 1 diabetes and 4 clinicians including 2 endocrinologists and 2 diabetes nurses. The software implementation followed a hybrid agile and waterfall approach. The testing relied on load, and black and white box methods. Results: We created a context-aware knowledge-based module able to (1) detect potential errors, and information gaps from the self-collected health data, (2) pinpoint relevant data and potential causes of noticeable medical events, and (3) recommend actions to follow to improve the reliability of the data issues and medical issues to be discussed with clinicians. The module uses a reasoning engine following a hypothesize-and-test strategy built on a knowledge base and using contextual information. The knowledge base contains hypotheses, rules, and plans we defined with the input of medical experts. We identified a large set of contextual information: emotional state (eg, preferences, mood) of patients and medical workers, their relationship, their metadata (eg, age, medical specialty), the time and location of usage of the system, patient-collected data (eg, blood glucose, basal-bolus insulin), patients’ goals and medical standards (eg, insulin sensitivity factor, in range values). Demonstrating the usage of the system revealed that (1) participants perceived the system as useful and relevant for consultation, and (2) the system uses less than 30 milliseconds to treat new cases. Conclusions: Using a knowledge-based system to identify anomalies concerning the reliability of patients’ self-collected health data to provide information on potential information gaps and to propose relevant medical subjects to discuss or actions to follow could ease the introduction of self-collected health data into consultation. Combining this reasoning engine and the system of the FullFlow project could improve the diagnostic process in health care. %M 30291097 %R 10.2196/10431 %U http://diabetes.jmir.org/2018/3/e10431/ %U https://doi.org/10.2196/10431 %U http://www.ncbi.nlm.nih.gov/pubmed/30291097 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 20 %N 7 %P e10413 %T A Decade of Veteran Voices: Examining Patient Portal Enhancements Through the Lens of User-Centered Design %A Nazi,Kim M %A Turvey,Carolyn L %A Klein,Dawn M %A Hogan,Timothy P %+ Veterans and Consumers Health Informatics Office, Office of Connected Care, Veterans Health Administration, US Department of Veterans Affairs, 44 Mountain View Drive, Coxsackie, NY, 12051, United States, 1 518 703 6330, kmnazi@gmail.com %K patient portal %K user-centered design %K eHealth %K veteran %D 2018 %7 10.07.2018 %9 Original Paper %J J Med Internet Res %G English %X Background: Health care systems have entered a new era focused on patient engagement. Patient portals linked to electronic health records are recognized as a promising multifaceted tool to help achieve patient engagement goals. Achieving significant growth in adoption and use requires agile evaluation methods to complement periodic formal research efforts. Objective: This paper describes one of the implementation strategies that the Department of Veterans Affairs (VA) has used to foster the adoption and sustained use of its patient portal, My HealtheVet, over the last decade: an ongoing focus on user-centered design (UCD). This strategy entails understanding the users and their tasks and goals and optimizing portal design and functionality accordingly. Using a case study approach, we present a comparison of early user demographics and preferences with more recent data and several examples to illustrate how a UCD can serve as an effective implementation strategy for a patient portal within a large integrated health care system. Methods: VA has employed a customer experience analytics (CXA) survey on its patient portal since 2007 to enable ongoing direct user feedback. In a continuous cycle, a random sample of site visitors is invited to participate in the Web-based survey. CXA model questions are used to track and trend satisfaction, while custom questions collect data about users’ characteristics, needs, and preferences. In this case study, we performed analyses of descriptive statistics comparing user characteristics and preferences from FY2008 (wherein “FY” means “fiscal year”) to FY2017 and user trends regarding satisfaction with and utilization of specific portal functions over the last decade, as well as qualitative content analysis of user’s open-ended survey comments. Results: User feedback has guided the development of enhancements to core components of the My HealtheVet portal including available features, content, interface design, prospective functional design, and related policies. Ten-year data regarding user characteristics and portal utilization demonstrate trends toward greater patient engagement and satisfaction. Administration of a continuous voluntary Web-based survey is an efficient and effective way to capture veterans’ voices about who they are, how they use the patient portal, needed system improvements, and desired additional services. Conclusions: Leveraging “voice-of-the-customer” techniques as part of patient portal implementation can ensure that such systems meet users’ needs in ways that are agile and most effective. Through this strategy, VA has fostered significant adoption and use of My HealtheVet to engage patients in managing their health. %M 29991468 %R 10.2196/10413 %U http://www.jmir.org/2018/7/e10413/ %U https://doi.org/10.2196/10413 %U http://www.ncbi.nlm.nih.gov/pubmed/29991468 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 5 %N 2 %P e22 %T The Impact of Visualization Dashboards on Quality of Care and Clinician Satisfaction: Integrative Literature Review %A Khairat,Saif Sherif %A Dukkipati,Aniesha %A Lauria,Heather Alico %A Bice,Thomas %A Travers,Debbie %A Carson,Shannon S %+ University of North Carolina at Chapel Hill, 428 Carrington Hall, Chapel Hill, NC, 27514, United States, 1 919843514, saif@unc.edu %K intensive care unit %K visualization, Dashboard %K cognitive load %K information overload %K usability %K user interface design %K health information technology %K electronic health record %D 2018 %7 31.05.2018 %9 Review %J JMIR Hum Factors %G English %X Background: Intensive Care Units (ICUs) in the United States admit more than 5.7 million people each year. The ICU level of care helps people with life-threatening illness or injuries and involves close, constant attention by a team of specially-trained health care providers. Delay between condition onset and implementation of necessary interventions can dramatically impact the prognosis of patients with life-threatening diagnoses. Evidence supports a connection between information overload and medical errors. A tool that improves display and retrieval of key clinical information has great potential to benefit patient outcomes. The purpose of this review is to synthesize research on the use of visualization dashboards in health care. Objective: The purpose of conducting this literature review is to synthesize previous research on the use of dashboards visualizing electronic health record information for health care providers. A review of the existing literature on this subject can be used to identify gaps in prior research and to inform further research efforts on this topic. Ultimately, this evidence can be used to guide the development, testing, and implementation of a new solution to optimize the visualization of clinical information, reduce clinician cognitive overload, and improve patient outcomes. Methods: Articles were included if they addressed the development, testing, implementation, or use of a visualization dashboard solution in a health care setting. An initial search was conducted of literature on dashboards only in the intensive care unit setting, but there were not many articles found that met the inclusion criteria. A secondary follow-up search was conducted to broaden the results to any health care setting. The initial and follow-up searches returned a total of 17 articles that were analyzed for this literature review. Results: Visualization dashboard solutions decrease time spent on data gathering, difficulty of data gathering process, cognitive load, time to task completion, errors, and improve situation awareness, compliance with evidence-based safety guidelines, usability, and navigation. Conclusions: Researchers can build on the findings, strengths, and limitations of the work identified in this literature review to bolster development, testing, and implementation of novel visualization dashboard solutions. Due to the relatively few studies conducted in this area, there is plenty of room for researchers to test their solutions and add significantly to the field of knowledge on this subject. %R 10.2196/humanfactors.9328 %U http://humanfactors.jmir.org/2018/2/e22/ %U https://doi.org/10.2196/humanfactors.9328 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 5 %N 2 %P e14 %T A Novel Information Retrieval Tool to Find Hospital Care Team Members: Development and Usability Study %A Morawski,Kyle %A Monsen,Craig %A Takhar,Sukhjit %A Landman,Adam %+ Atrius Health, 133 Brookline Ave, Boston, MA,, United States, 1 617 421 1000, kyle_morawski@atriushealth.org %K communication %K patient care team %D 2018 %7 16.04.2018 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Hospital communication among members of a patient’s care team is a central part of clinical workflow and consumes a large amount of a health care provider’s time. Oftentimes the complexity of hospital care leads to difficulty in finding the appropriate contact, which can lead to inefficiencies and frustration. Squire is a Web-based information retrieval app created to improve the speed and efficiency in reaching the appropriate team member during the care of a hospitalized patient. Objective: The objective of the study was to design and develop Squire and to evaluate the usage, usability, and perceived effect of the app on finding the correct contact within a hospital. Methods: We used a mixed-methods design using a before-after survey methodology combined with one-on-one interviews to understand the perceived effect of Squire. The study took place at an academic medical center with internal medicine resident physicians. We surveyed residents on demographics, as well as time and efficiency of hospital communication before and after the use of Squire. After using Squire, participants were also asked to evaluate Squire’s Net Promoter Score (NPS). A subset of voluntary participants participated in one-on-one interviews and completed the System Usability Scale (SUS). We performed descriptive statistics on participant characteristics, app usage data, and responses to surveys. Survey results were compared before and after Squire adoption using the Wilcoxon rank-sum test and a general linear model. Interview data were analyzed using content analysis with a qualitative description approach to review and categorize feedback from participants. Results: There was a 67.9% (74/109) response rate to the pre-Squire survey and 89.9% (98/109) response rate to the post-Squire survey. At baseline, there was an average of 22.2 (95% CI 18.4-26.0) minutes/day spent searching for the right contact, and this decreased to 16.3 (95% CI 13.9-18.7) minutes/day after Squire was launched (P=.01). There were favorable usability scores, with an average SUS of 84.7, and a marginal NPS of +6.1. Overall, the use of Squire included 22,283 page views, most commonly to contact the admissions office or portable chest x-ray technician. Interviews highlighted common benefits of Squire, including decreased perceived time spent on hold with operators and improvement in connecting with the appropriate contact in specialized, complex departments. Future opportunities were also identified to improve Squire including adding a two-way communication between physician and nursing staff and providing offline access. Conclusions: Squire decreased the perceived time required to find an appropriate contact and had a favorable usability score; however, the NPS was marginal and several opportunities were identified to improve Squire for future use. %M 29661752 %R 10.2196/humanfactors.6781 %U http://humanfactors.jmir.org/2018/2/e14/ %U https://doi.org/10.2196/humanfactors.6781 %U http://www.ncbi.nlm.nih.gov/pubmed/29661752 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 5 %N 1 %P e13 %T Evaluating the Usability and Usefulness of a Mobile App for Atrial Fibrillation Using Qualitative Methods: Exploratory Pilot Study %A Hirschey,Jaclyn %A Bane,Sunetra %A Mansour,Moussa %A Sperber,Jodi %A Agboola,Stephen %A Kvedar,Joseph %A Jethwani,Kamal %+ Partners Connected Health, Partners Healthcare, 25 New Chardon Street, Suite 300, Boston, MA, 02114, United States, 1 617 724 2724, jhirschey@mgh.harvard.edu %K nonvalvular atrial fibrillation %K medication adherence %K patient self-care %K mobile application %K exploratory research %K pilot study %K usability study %K acceptability study %K qualitative methods %D 2018 %7 15.03.2018 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Atrial fibrillation (AFib) is the most common form of heart arrhythmia and a potent risk factor for stroke. Nonvitamin K antagonist oral anticoagulants (NOACs) are routinely prescribed to manage AFib stroke risk; however, nonadherence to treatment is a concern. Additional tools that support self-care and medication adherence may benefit patients with AFib. Objective: The aim of this study was to evaluate the perceived usability and usefulness of a mobile app designed to support self-care and treatment adherence for AFib patients who are prescribed NOACs. Methods: A mobile app to support AFib patients was previously developed based on early stage interview and usability test data from clinicians and patients. An exploratory pilot study consisting of naturalistic app use, surveys, and semistructured interviews was then conducted to examine patients’ perceptions and everyday use of the app. Results: A total of 12 individuals with an existing diagnosis of nonvalvular AFib completed the 4-week study. The average age of participants was 59 years. All participants somewhat or strongly agreed that the app was easy to use, and 92% (11/12) reported being satisfied or very satisfied with the app. Participant feedback identified changes that may improve app usability and usefulness for patients with AFib. Areas of usability improvement were organized by three themes: app navigation, clarity of app instructions and design intent, and software bugs. Perceptions of app usefulness were grouped by three key variables: core needs of the patient segment, patient workflow while managing AFib, and the app’s ability to support the patient’s evolving needs. Conclusions: The results of this study suggest that mobile tools that target self-care and treatment adherence may be helpful to AFib patients, particularly those who are newly diagnosed. Additionally, participant feedback provided insight into the varied needs and health experiences of AFib patients, which may improve the design and targeting of the intervention. Pilot studies that qualitatively examine patient perceptions of usability and usefulness are a valuable and often underutilized method for assessing the real-world acceptability of an intervention. Additional research evaluating the AFib Connect mobile app over a longer period, and including a larger, more diverse sample of AFib patients, will be helpful for understanding whether the app is perceived more broadly to be useful and effective in supporting patient self-care and medication adherence. %M 29549073 %R 10.2196/humanfactors.8004 %U http://humanfactors.jmir.org/2018/1/e13/ %U https://doi.org/10.2196/humanfactors.8004 %U http://www.ncbi.nlm.nih.gov/pubmed/29549073 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 5 %N 1 %P e7 %T How Health Care Professionals Evaluate a Digital Intervention to Improve Medication Adherence: Qualitative Exploratory Study %A Thomson,Karen %A Brouwers,Corline %A Damman,Olga C %A de Bruijne,Martine C %A Timmermans,Danielle RM %A Melles,Marijke %+ Department of Public and Occupational Health and Amsterdam Public Health Research Institute, VU University Medical Center, Van der Boechorststraat 7, Amsterdam, 1081 BT, Netherlands, 31 204448263, c.brouwers@vumc.nl %K medication adherence %K eHealth %K shared decision making %K self-management %K patient engagement %D 2018 %7 20.02.2018 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Medication nonadherence poses a serious and a hard-to-tackle problem for many chronic diseases. Electronic health (eHealth) apps that foster patient engagement and shared decision making (SDM) may be a novel approach to improve medication adherence. Objective: The aim of this study was to investigate the perspective of health care professionals regarding a newly developed digital app aimed to improve medication adherence. Familial hypercholesterolemia (FH) was chosen as a case example. Methods: A Web-based prototype of the eHealth app—MIK—was codesigned with patients and health care professionals. After user tests with patients, we performed semistructured interviews and user tests with 12 physicians from 6 different hospitals to examine how the functionalities offered by MIK could assist physicians in their consultation and how they could be integrated into daily clinical practice. Qualitative thematic analysis was used to identify themes that covered the physicians’ evaluations. Results: On the basis of the interview data, 3 themes were identified, which were (1) perceived impact on patient-physician collaboration; (2) perceived impact on the patient’s understanding and self-management regarding medication adherence; and (3) perceived impact on clinical decisions and workflow. Conclusions: The eHealth app MIK seems to have the potential to improve the consultation between the patient and the physician in terms of collaboration and patient engagement. The impact of eHealth apps based on the concept of SDM for improving medication-taking behavior and clinical outcomes is yet to be evaluated. Insights will be useful for further development of eHealth apps aimed at improving self-management by means of patient engagement and SDM. %M 29463494 %R 10.2196/humanfactors.8948 %U http://humanfactors.jmir.org/2018/1/e7/ %U https://doi.org/10.2196/humanfactors.8948 %U http://www.ncbi.nlm.nih.gov/pubmed/29463494 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 5 %N 1 %P e4 %T Reducing Misses and Near Misses Related to Multitasking on the Electronic Health Record: Observational Study and Qualitative Analysis %A Ratanawongsa,Neda %A Matta,George Y %A Bohsali,Fuad B %A Chisolm,Margaret S %+ Division of General Internal Medicine, Department of Medicine, University of California, San Francisco, 1001 Potrero Avenue, Box 1364, San Francisco, CA, 94143, United States, 1 416 206 3188, neda.ratanawongsa@ucsf.edu %K electronic health records %K physician-patient relations %K patient safety %D 2018 %7 06.02.2018 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Clinicians’ use of electronic health record (EHR) systems while multitasking may increase the risk of making errors, but silent EHR system use may lower patient satisfaction. Delaying EHR system use until after patient visits may increase clinicians’ EHR workload, stress, and burnout. Objective: We aimed to describe the perspectives of clinicians, educators, administrators, and researchers about misses and near misses that they felt were related to clinician multitasking while using EHR systems. Methods: This observational study was a thematic analysis of perspectives elicited from 63 continuing medical education (CME) participants during 2 workshops and 1 interactive lecture about challenges and strategies for relationship-centered communication during clinician EHR system use. The workshop elicited reflection about memorable times when multitasking EHR use was associated with “misses” (errors that were not caught at the time) or “near misses” (mistakes that were caught before leading to errors). We conducted qualitative analysis using an editing analysis style to identify codes and then select representative themes and quotes. Results: All workshop participants shared stories of misses or near misses in EHR system ordering and documentation or patient-clinician communication, wondering about “misses we don’t even know about.” Risk factors included the computer’s position, EHR system usability, note content and style, information overload, problematic workflows, systems issues, and provider and patient communication behaviors and expectations. Strategies to reduce multitasking EHR system misses included clinician transparency when needing silent EHR system use (eg, for prescribing), narrating EHR system use, patient activation during EHR system use, adapting visit organization and workflow, improving EHR system design, and improving team support and systems. Conclusions: CME participants shared numerous stories of errors and near misses in EHR tasks and communication that they felt related to EHR multitasking. However, they brainstormed diverse strategies for using EHR systems safely while preserving patient relationships. %M 29410388 %R 10.2196/humanfactors.9371 %U http://humanfactors.jmir.org/2018/1/e4/ %U https://doi.org/10.2196/humanfactors.9371 %U http://www.ncbi.nlm.nih.gov/pubmed/29410388 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 5 %N 1 %P e2 %T The Perceived Ease of Use and Usefulness of Loop: Evaluation and Content Analysis of a Web-Based Clinical Collaboration System %A Kurahashi,Allison M %A Stinson,Jennifer N %A van Wyk,Margaret %A Luca,Stephanie %A Jamieson,Trevor %A Weinstein,Peter %A Cafazzo,Joseph A %A Lokuge,Bhadra %A Cohen,Eyal %A Rapoport,Adam %A Husain,Amna %+ The Temmy Latner Centre for Palliative Care, Sinai Health System, 60 Murray Street, 4th Floor, Box 13, Toronto, ON, M5T 3L9, Canada, 1 416 586 4800 ext 7886, amna.husain@sinaihealthsystem.ca %K patient-centered care %K patient participation %K chronic disease %K communication %K internet communication tools %K usability testing %K interdisciplinary communication %K health communication %K continuity of patient care %K patient care team %K inventions %D 2018 %7 09.01.2018 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Patients with complex health care needs require the expertise of many health care providers. Communication, collaboration, and patient-centered care positively impact care quality and patient outcomes. Few technologies exist that facilitate collaboration between providers across settings of care and also engage the patient. We developed a Web-based clinical collaboration system, Loop, to address this gap. The likelihood of a technological system’s uptake is associated with its perceived ease of use and perceived usefulness. We engaged stakeholders in the conceptualization and development of Loop in an effort to maximize its intuitiveness and utility. Objective: This study aimed to report end users’ perceptions about the ease of use and usefulness of Loop captured during usability tests of Loop. Methods: Participants represented three user types (patients, caregivers, and health care providers) recruited from three populations (adults with cancer, adolescents and young adults with cancer, and children with medical complexity). We conducted usability testing over three iterative cycles of testing and development in both laboratory-based and off-site environments. We performed a content analysis of usability testing transcripts to summarize and describe participant perceptions about the ease of use and usefulness of Loop. Results: Participants enjoyed testing Loop and were able to use the core functions—composing, posting, and reading messages—with little difficulty. They had difficulty interpreting certain visual cues and design elements or the purpose of some features. This difficulty negatively impacted perceived ease of use but was primarily limited to auxiliary features. Participants predicted that Loop could improve the efficiency and effectiveness of communication between care team members; however, this perceived usefulness could be compromised by disruptions to personal workflow such as additional time or task requirements. Conclusions: Loop was perceived to have value as a collaboration system; however, usability testing findings indicate that some design and functional elements need to be addressed to improve ease of use. Additionally, participant concerns highlight the need to consider how a system can be implemented so as to minimize impact on workflow and optimize usefulness. %M 29317386 %R 10.2196/humanfactors.7882 %U http://humanfactors.jmir.org/2018/1/e2/ %U https://doi.org/10.2196/humanfactors.7882 %U http://www.ncbi.nlm.nih.gov/pubmed/29317386 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 4 %N 4 %P e27 %T Workarounds Emerging From Electronic Health Record System Usage: Consequences for Patient Safety, Effectiveness of Care, and Efficiency of Care %A Blijleven,Vincent %A Koelemeijer,Kitty %A Wetzels,Marijntje %A Jaspers,Monique %+ Center for Marketing & Supply Chain Management, Nyenrode Business University, Straatweg 25, Breukelen, 3621 BG, Netherlands, 31 630023248, vincentblijleven@gmail.com %K electronic health records %K qualitative research %K physicians %K nurses %K patient safety %K quality of health care %K efficiency %K workflow %D 2017 %7 05.10.2017 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Health care providers resort to informal temporary practices known as workarounds for handling exceptions to normal workflow unintendedly imposed by electronic health record systems (EHRs). Although workarounds may seem favorable at first sight, they are generally suboptimal and may jeopardize patient safety, effectiveness of care, and efficiency of care. Objective: Research into the scope and impact of EHR workarounds on patient care processes is scarce. This paper provides insight into the effects of EHR workarounds on organizational workflows and outcomes of care services by identifying EHR workarounds and determining their rationales, scope, and impact on health care providers’ workflows, patient safety, effectiveness of care, and efficiency of care. Knowing the rationale of a workaround provides valuable clues about the source of origin of each workaround and how each workaround could most effectively be resolved. Knowing the scope and impact a workaround has on EHR-related safety, effectiveness, and efficiency provides insight into how to address related concerns. Methods: Direct observations and follow-up semistructured interviews with 31 physicians, 13 nurses, and 3 clerks and qualitative bottom-up coding techniques was used to identify, analyze, and classify EHR workarounds. The research was conducted within 3 specialties and settings at a large university hospital. Rationales were associated with work system components (persons, technology and tools, tasks, organization, and physical environment) of the Systems Engineering Initiative for Patient Safety (SEIPS) framework to reveal their source of origin as well as to determine the scope and the impact of each EHR workaround from a structure-process-outcome perspective. Results: A total of 15 rationales for EHR workarounds were identified of which 5 were associated with persons, 4 with technology and tools, 4 with the organization, and 2 with the tasks. Three of these 15 rationales for EHR workarounds have not been identified in prior research: data migration policy, enforced data entry, and task interference. Conclusions: EHR workaround rationales associated with different SEIPS work system components demand a different approach to be resolved. Persons-related workarounds may most effectively be resolved through personal training, organization-related workarounds through reviewing organizational policy and regulations, tasks-related workarounds through process redesign, and technology- and tools-related workarounds through EHR redesign efforts. Furthermore, insights gained from knowing a workaround’s degree of influence as well as impact on patient safety, effectiveness of care, and efficiency of care can inform design and redesign of EHRs to further align EHR design with work contexts, subsequently leading to better organization and (safe) provision of care. In doing so, a research team in collaboration with all stakeholders could use the SEIPS framework to reflect on the current and potential future configurations of the work system to prevent unfavorable workarounds from occurring and how a redesign of the EHR would impact interactions between the work system components. %M 28982645 %R 10.2196/humanfactors.7978 %U http://humanfactors.jmir.org/2017/4/e27/ %U https://doi.org/10.2196/humanfactors.7978 %U http://www.ncbi.nlm.nih.gov/pubmed/28982645 %0 Journal Article %@ 2369-1999 %I JMIR Publications %V 3 %N 2 %P e9 %T Development, Feasibility, and Small-Scale Implementation of a Web-Based Prognostic Tool—Surveillance, Epidemiology, and End Results Cancer Survival Calculator %A Henton,Michelle %A Gaglio,Bridget %A Cynkin,Laurie %A Feuer,Eric J %A Rabin,Borsika A %+ Clinical Effectiveness and Decision Science, Patient-Centered Outcomes Research Institute, 1828 L St. NW, Ste. 900, Washington, DC, 20036, United States, 1 202 370 9410, mhenton@pcori.org %K clinical decision-making %K communication %K neoplasms %K patient care team %K Internet %D 2017 %7 20.07.2017 %9 Original Paper %J JMIR Cancer %G English %X Background: Population datasets and the Internet are playing an ever-growing role in the way cancer information is made available to providers, patients, and their caregivers. The Surveillance, Epidemiology, and End Results Cancer Survival Calculator (SEER*CSC) is a Web-based cancer prognostic tool that uses SEER data, a large population dataset, to provide physicians with highly valid, evidence-based prognostic estimates for increasing shared decision-making and improving patient-provider communication of complex health information. Objective: The aim of this study was to develop, test, and implement SEER*CSC. Methods: An iterative approach was used to develop the SEER*CSC. Based on input from cancer patient advocacy groups and physicians, an initial version of the tool was developed. Next, providers from 4 health care delivery systems were recruited to do formal usability testing of SEER*CSC. A revised version of SEER*CSC was then implemented in two health care delivery sites using a real-world clinical implementation approach, and usage data were collected. Post-implementation follow-up interviews were conducted with site champions. Finally, patients from two cancer advocacy groups participated in usability testing. Results: Overall feedback of SEER*CSC from both providers and patients was positive, with providers noting that the tool was professional and reliable, and patients finding it to be informational and helpful to use when discussing their diagnosis with their provider. However, use during the small-scale implementation was low. Reasons for low usage included time to enter data, not having treatment options in the tool, and the tool not being incorporated into the electronic health record (EHR). Patients found the language in its current version to be too complex. Conclusions: The implementation and usability results showed that participants were enthusiastic about the use and features of SEER*CSC, but sustained implementation in a real-world clinical setting faced significant challenges. As a result of these findings, SEER*CSC is being redesigned with more accessible language for a public facing release. Meta-tools, which put different tools in context of each other, are needed to assist in understanding the strengths and limitations of various tools and their place in the clinical decision-making pathway. The continued development and eventual release of prognostic tools should include feedback from multidisciplinary health care teams, various stakeholder groups, patients, and caregivers. %M 28729232 %R 10.2196/cancer.7120 %U http://cancer.jmir.org/2017/2/e9/ %U https://doi.org/10.2196/cancer.7120 %U http://www.ncbi.nlm.nih.gov/pubmed/28729232 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 19 %N 5 %P e174 %T Patient-Centered Decision Support: Formative Usability Evaluation of Integrated Clinical Decision Support With a Patient Decision Aid for Minor Head Injury in the Emergency Department %A Melnick,Edward R %A Hess,Erik P %A Guo,George %A Breslin,Maggie %A Lopez,Kevin %A Pavlo,Anthony J %A Abujarad,Fuad %A Powsner,Seth M %A Post,Lori A %+ Department of Emergency Medicine, Yale School of Medicine, 464 Congress Ave, Suite 260, New Haven, CT,, United States, 1 203 737 6454, edward.melnick@yale.edu %K clinical decision support %K decision aids %K head injury, minor %K medical informatics %K spiral computed tomography %K health services overuse %K patient-centered outcomes research %D 2017 %7 19.05.2017 %9 Original Paper %J J Med Internet Res %G English %X Background: The Canadian Computed Tomography (CT) Head Rule, a clinical decision rule designed to safely reduce imaging in minor head injury, has been rigorously validated and implemented, and yet expected decreases in CT were unsuccessful. Recent work has identified empathic care as a key component in decreasing CT overuse. Health information technology can hinder the clinician-patient relationship. Patient-centered decision tools to support the clinician-patient relationship are needed to promote evidence-based decisions. Objective: Our objective is to formatively evaluate an electronic tool that not only helps clinicians at the bedside to determine the need for CT use based on the Canadian CT Head Rule but also promotes evidence-based conversations between patients and clinicians regarding patient-specific risk and patients’ specific concerns. Methods: User-centered design with practice-based and participatory decision aid development was used to design, develop, and evaluate patient-centered decision support regarding CT use in minor head injury in the emergency department. User experience and user interface (UX/UI) development involved successive iterations with incremental refinement in 4 phases: (1) initial prototype development, (2) usability assessment, (3) field testing, and (4) beta testing. This qualitative approach involved input from patients, emergency care clinicians, health services researchers, designers, and clinical informaticists at every stage. Results: The Concussion or Brain Bleed app is the product of 16 successive iterative revisions in accordance with UX/UI industry design standards. This useful and usable final product integrates clinical decision support with a patient decision aid. It promotes shared use by emergency clinicians and patients at the point of care within the emergency department context. This tablet computer app facilitates evidence-based conversations regarding CT in minor head injury. It is adaptable to individual clinician practice styles. The resultant tool includes a patient injury evaluator based on the Canadian CT Head Rule and provides patient specific risks using pictographs with natural frequencies and cues for discussion about patient concerns. Conclusions: This tool was designed to align evidence-based practices about CT in minor head injury patients. It establishes trust, empowers active participation, and addresses patient concerns and uncertainty about their condition. We hypothesize that, when implemented, the Concussion or Brain Bleed app will support—not hinder—the clinician-patient relationship, safely reduce CT use, and improve the patient experience of care. %M 28526667 %R 10.2196/jmir.7846 %U http://www.jmir.org/2017/5/e174/ %U https://doi.org/10.2196/jmir.7846 %U http://www.ncbi.nlm.nih.gov/pubmed/28526667 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 4 %N 2 %P e14 %T Iterative User Interface Design for Automated Sequential Organ Failure Assessment Score Calculator in Sepsis Detection %A Aakre,Christopher Ansel %A Kitson,Jaben E %A Li,Man %A Herasevich,Vitaly %+ Mayo Clinic, Department of Medicine, Division of General Internal Medicine, 200 First St SW, Rochester, MN, 55905, United States, 1 507 538 0621, aakre.christopher@mayo.edu %K automation %K organ dysfunction scores %K software design %K user-computer interface %D 2017 %7 18.05.2017 %9 Original Paper %J JMIR Hum Factors %G English %X Background: The new sepsis definition has increased the need for frequent sequential organ failure assessment (SOFA) score recalculation and the clerical burden of information retrieval makes this score ideal for automated calculation. Objective: The aim of this study was to (1) estimate the clerical workload of manual SOFA score calculation through a time-motion analysis and (2) describe a user-centered design process for an electronic medical record (EMR) integrated, automated SOFA score calculator with subsequent usability evaluation study. Methods: First, we performed a time-motion analysis by recording time-to-task-completion for the manual calculation of 35 baseline and 35 current SOFA scores by 14 internal medicine residents over a 2-month period. Next, we used an agile development process to create a user interface for a previously developed automated SOFA score calculator. The final user interface usability was evaluated by clinician end users with the Computer Systems Usability Questionnaire. Results: The overall mean (standard deviation, SD) time-to-complete manual SOFA score calculation time was 61.6 s (33). Among the 24% (12/50) usability survey respondents, our user-centered user interface design process resulted in >75% favorability of survey items in the domains of system usability, information quality, and interface quality. Conclusions: Early stakeholder engagement in our agile design process resulted in a user interface for an automated SOFA score calculator that reduced clinician workload and met clinicians’ needs at the point of care. Emerging interoperable platforms may facilitate dissemination of similarly useful clinical score calculators and decision support algorithms as “apps.” A user-centered design process and usability evaluation should be considered during creation of these tools. %M 28526675 %R 10.2196/humanfactors.7567 %U http://humanfactors.jmir.org/2017/2/e14/ %U https://doi.org/10.2196/humanfactors.7567 %U http://www.ncbi.nlm.nih.gov/pubmed/28526675 %0 Journal Article %@ 2291-5222 %I JMIR Publications %V 4 %N 4 %P e131 %T Design and Testing of the Safety Agenda Mobile App for Managing Health Care Managers’ Patient Safety Responsibilities %A Mira,José Joaquín %A Carrillo,Irene %A Fernandez,Cesar %A Vicente,Maria Asuncion %A Guilabert,Mercedes %+ Alicante-Sant Joan Health District, Consellería Sanitat, Hospital-Plá Health Center c/ Hermanos López Osaba s/n, Alicante, 03013, Spain, 34 606433599, jose.mira@umh.es %K patient safety %K mobile apps %K administrators %K health service %D 2016 %7 08.12.2016 %9 Original Paper %J JMIR Mhealth Uhealth %G English %X Background: Adverse events are a reality in clinical practice. Reducing the prevalence of preventable adverse events by stemming their causes requires health managers’ engagement. Objective: The objective of our study was to develop an app for mobile phones and tablets that would provide managers with an overview of their responsibilities in matters of patient safety and would help them manage interventions that are expected to be carried out throughout the year. Methods: The Safety Agenda Mobile App (SAMA) was designed based on standardized regulations and reviews of studies about health managers’ roles in patient safety. A total of 7 managers used a beta version of SAMA for 2 months and then they assessed and proposed improvements in its design. Their experience permitted redesigning SAMA, improving functions and navigation. A total of 74 Spanish health managers tried out the revised version of SAMA. After 4 months, their assessment was requested in a voluntary and anonymous manner. Results: SAMA is an iOS app that includes 37 predefined tasks that are the responsibility of health managers. Health managers can adapt these tasks to their schedule, add new ones, and share them with their team. SAMA menus are structured in 4 main areas: information, registry, task list, and settings. Of the 74 users who tested SAMA, 64 (86%) users provided a positive assessment of SAMA characteristics and utility. Over an 11-month period, 238 users downloaded SAMA. This mobile app has obtained the AppSaludable (HealthyApp) Quality Seal. Conclusions: SAMA includes a set of activities that are expected to be carried out by health managers in matters of patient safety and contributes toward improving the awareness of their responsibilities in matters of safety. %M 27932315 %R 10.2196/mhealth.5796 %U http://mhealth.jmir.org/2016/4/e131/ %U https://doi.org/10.2196/mhealth.5796 %U http://www.ncbi.nlm.nih.gov/pubmed/27932315 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 4 %N 4 %P e36 %T Increasing Complexity in Rule-Based Clinical Decision Support: The Symptom Assessment and Management Intervention %A Lobach,David F %A Johns,Ellis B %A Halpenny,Barbara %A Saunders,Toni-Ann %A Brzozowski,Jane %A Del Fiol,Guilherme %A Berry,Donna L %A Braun,Ilana M %A Finn,Kathleen %A Wolfe,Joanne %A Abrahm,Janet L %A Cooley,Mary E %+ School of Medicine, Department of Community & Family Medicine, Duke University Medical Center, Duke University, Box 3886, Durham, NC, 22710, United States, 1 919 438 2346, David.Lobach@klesishealthcare.com %K rule-based clinical decision support %K clinical algorithms %K Web services %K software as a service %K symptom management %K patient-reported outcomes %K lung cancer %D 2016 %7 08.11.2016 %9 Original Paper %J JMIR Med Inform %G English %X Background: Management of uncontrolled symptoms is an important component of quality cancer care. Clinical guidelines are available for optimal symptom management, but are not often integrated into the front lines of care. The use of clinical decision support (CDS) at the point-of-care is an innovative way to incorporate guideline-based symptom management into routine cancer care. Objective: The objective of this study was to develop and evaluate a rule-based CDS system to enable management of multiple symptoms in lung cancer patients at the point-of-care. Methods: This study was conducted in three phases involving a formative evaluation, a system evaluation, and a contextual evaluation of clinical use. In Phase 1, we conducted iterative usability testing of user interface prototypes with patients and health care providers (HCPs) in two thoracic oncology clinics. In Phase 2, we programmed complex algorithms derived from clinical practice guidelines into a rules engine that used Web services to communicate with the end-user application. Unit testing of algorithms was conducted using a stack-traversal tree-spanning methodology to identify all possible permutations of pathways through each algorithm, to validate accuracy. In Phase 3, we evaluated clinical use of the system among patients and HCPs in the two clinics via observations, structured interviews, and questionnaires. Results: In Phase 1, 13 patients and 5 HCPs engaged in two rounds of formative testing, and suggested improvements leading to revisions until overall usability scores met a priori benchmarks. In Phase 2, symptom management algorithms contained between 29 and 1425 decision nodes, resulting in 19 to 3194 unique pathways per algorithm. Unit testing required 240 person-hours, and integration testing required 40 person-hours. In Phase 3, both patients and HCPs found the system usable and acceptable, and offered suggestions for improvements. Conclusions: A rule-based CDS system for complex symptom management was systematically developed and tested. The complexity of the algorithms required extensive development and innovative testing. The Web service-based approach allowed remote access to CDS knowledge, and could enable scaling and sharing of this knowledge to accelerate availability, and reduce duplication of effort. Patients and HCPs found the system to be usable and useful. %M 27826132 %R 10.2196/medinform.5728 %U http://medinform.jmir.org/2016/4/e36/ %U https://doi.org/10.2196/medinform.5728 %U http://www.ncbi.nlm.nih.gov/pubmed/27826132 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 4 %N 4 %P e35 %T Natural Language Processing–Enabled and Conventional Data Capture Methods for Input to Electronic Health Records: A Comparative Usability Study %A Kaufman,David R %A Sheehan,Barbara %A Stetson,Peter %A Bhatt,Ashish R %A Field,Adele I %A Patel,Chirag %A Maisel,James Mark %+ Department of Biomedical Informatics, Arizona State University, 13212 East Shea, Scottsdale, AZ, 85260, United States, 1 4808840250, dave.kaufman@asu.edu %K electronic health records %K natural language processing %K medical transcription %K user-computer interface %D 2016 %7 28.10.2016 %9 Original Paper %J JMIR Med Inform %G English %X Background: The process of documentation in electronic health records (EHRs) is known to be time consuming, inefficient, and cumbersome. The use of dictation coupled with manual transcription has become an increasingly common practice. In recent years, natural language processing (NLP)–enabled data capture has become a viable alternative for data entry. It enables the clinician to maintain control of the process and potentially reduce the documentation burden. The question remains how this NLP-enabled workflow will impact EHR usability and whether it can meet the structured data and other EHR requirements while enhancing the user’s experience. Objective: The objective of this study is evaluate the comparative effectiveness of an NLP-enabled data capture method using dictation and data extraction from transcribed documents (NLP Entry) in terms of documentation time, documentation quality, and usability versus standard EHR keyboard-and-mouse data entry. Methods: This formative study investigated the results of using 4 combinations of NLP Entry and Standard Entry methods (“protocols”) of EHR data capture. We compared a novel dictation-based protocol using MediSapien NLP (NLP-NLP) for structured data capture against a standard structured data capture protocol (Standard-Standard) as well as 2 novel hybrid protocols (NLP-Standard and Standard-NLP). The 31 participants included neurologists, cardiologists, and nephrologists. Participants generated 4 consultation or admission notes using 4 documentation protocols. We recorded the time on task, documentation quality (using the Physician Documentation Quality Instrument, PDQI-9), and usability of the documentation processes. Results: A total of 118 notes were documented across the 3 subject areas. The NLP-NLP protocol required a median of 5.2 minutes per cardiology note, 7.3 minutes per nephrology note, and 8.5 minutes per neurology note compared with 16.9, 20.7, and 21.2 minutes, respectively, using the Standard-Standard protocol and 13.8, 21.3, and 18.7 minutes using the Standard-NLP protocol (1 of 2 hybrid methods). Using 8 out of 9 characteristics measured by the PDQI-9 instrument, the NLP-NLP protocol received a median quality score sum of 24.5; the Standard-Standard protocol received a median sum of 29; and the Standard-NLP protocol received a median sum of 29.5. The mean total score of the usability measure was 36.7 when the participants used the NLP-NLP protocol compared with 30.3 when they used the Standard-Standard protocol. Conclusions: In this study, the feasibility of an approach to EHR data capture involving the application of NLP to transcribed dictation was demonstrated. This novel dictation-based approach has the potential to reduce the time required for documentation and improve usability while maintaining documentation quality. Future research will evaluate the NLP-based EHR data capture approach in a clinical setting. It is reasonable to assert that EHRs will increasingly use NLP-enabled data entry tools such as MediSapien NLP because they hold promise for enhancing the documentation process and end-user experience. %M 27793791 %R 10.2196/medinform.5544 %U http://medinform.jmir.org/2016/4/e35/ %U https://doi.org/10.2196/medinform.5544 %U http://www.ncbi.nlm.nih.gov/pubmed/27793791 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 3 %N 2 %P e26 %T Integrating Patient-Generated Health Data Into Clinical Care Settings or Clinical Decision-Making: Lessons Learned From Project HealthDesign %A Cohen,Deborah J %A Keller,Sara R %A Hayes,Gillian R %A Dorr,David A %A Ash,Joan S %A Sittig,Dean F %+ Department of Family Medicine, Oregon Health & Science University, 3181 SW Sam Jackson Park Road, FM, Portland, OR, 97239, United States, 1 503 494 7840, cohendj@ohsu.edu %K mobile applications %K chronic disease, self-management %K doctor-patient relations %D 2016 %7 19.10.2016 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Patient-generated health data (PGHD) are health-related data created or recorded by patients to inform their self-care and understanding about their own health. PGHD is different from other patient-reported outcome data because the collection of data is patient-driven, not practice- or research-driven. Technical applications for assisting patients to collect PGHD supports self-management activities such as healthy eating and exercise and can be important for preventing and managing disease. Technological innovations (eg, activity trackers) are making it more common for people to collect PGHD, but little is known about how PGHD might be used in outpatient clinics. Objective: The objective of our study was to examine the experiences of health care professionals who use PGHD in outpatient clinics. Methods: We conducted an evaluation of Project HealthDesign Round 2 to synthesize findings from 5 studies funded to test tools designed to help patients collect PGHD and share these data with members of their health care team. We conducted semistructured interviews with 13 Project HealthDesign study team members and 12 health care professionals that participated in these studies. We used an immersion-crystallization approach to analyze data. Our findings provide important information related to health care professionals’ attitudes toward and experiences with using PGHD in a clinical setting. Results: Health care professionals identified 3 main benefits of PGHD accessibility in clinical settings: (1) deeper insight into a patient’s condition; (2) more accurate patient information, particularly when of clinical relevance; and (3) insight into a patient’s health between clinic visits, enabling revision of care plans for improved health goal achievement, while avoiding unnecessary clinic visits. Study participants also identified 3 areas of consideration when implementing collection and use of PGHD data in clinics: (1) developing practice workflows and protocols related to PGHD collection and use; (2) data storage, accessibility at the point of care, and privacy concerns; and (3) ease of using PGHD data. Conclusions: PGHD provides value to both patients and health care professionals. However, more research is needed to understand the benefit of using PGHD in clinical care and to identify the strategies and clinic workflow needs for optimizing these tools. %M 27760726 %R 10.2196/humanfactors.5919 %U http://humanfactors.jmir.org/2016/2/e26/ %U https://doi.org/10.2196/humanfactors.5919 %U http://www.ncbi.nlm.nih.gov/pubmed/27760726 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 3 %N 2 %P e19 %T Do Gender-Specific and High-Resolution Three Dimensional Body Charts Facilitate the Communication of Pain for Women? A Quantitative and Qualitative Study %A Egsgaard,Line Lindhardt %A Christensen,Trine Søby %A Petersen,Ida Munk %A Brønnum,Dorthe Scavenius %A Boudreau,Shellie Ann %+ Faculty of Medicine, SMI®, Department of Health Science and Technology, Aalborg University, Fredrik Bajers Vej 7D3, Aalborg Øst, 9220, Denmark, 45 99409829, sboudreau@hst.aau.dk %K mHealth %K app %K android %K pain measurement %K chronic pain %K three dimensional pain drawing %K digital communication %D 2016 %7 20.07.2016 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Chronic pain is more prevalent among women; however, the majority of standardized pain drawings are often collected using male-like androgynous body representations. Objective: The purpose of this study was to assess whether gender-specific and high-resolution three-dimensional (3D) body charts facilitate the communication of pain for women. Methods: Using mixed-methods and a cross-over design, female patients with chronic pain were asked to provide detailed drawings of their current pain on masculine and feminine two-dimensional (2D) body schemas (N=41, Part I) or on female 2D and 3D high-resolution body schemas (N=41, Part II) on a computer tablet. The consistency of the drawings between body charts were assessed by intraclass correlation coefficient (ICC) and Bland-Altman plots. Semistructured interviews and a preference questionnaire were then used to obtain qualitative and quantitative responses of the drawing experience. Results: The consistency between body charts were high (Part I: ICC=0.980, Part II: ICC=0.994). The preference ratio for the masculine to feminine body schemas were 6:35 and 18:23 for the 2D to 3D female body charts. Patients reported that the 3D body chart enabled a more accurate expression of their pain due to the detailed contours of the musculature and bone structure, however, patients also reported the 3D body chart was too human and believed that skin-like appearance limited ‘deep pain’ expressions. Conclusions: Providing gender-specific body charts may facilitate the communication of pain and the level of detail (2D vs 3D body charts) should be used according to patients’ needs. %M 27440737 %R 10.2196/humanfactors.5693 %U http://humanfactors.jmir.org/2016/2/e19/ %U https://doi.org/10.2196/humanfactors.5693 %U http://www.ncbi.nlm.nih.gov/pubmed/27440737 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 3 %N 1 %P e17 %T Usability Testing and Adaptation of the Pediatric Cardiovascular Risk Reduction Clinical Decision Support Tool %A Williams,Pamela A %A Furberg,Robert D %A Bagwell,Jacqueline E %A LaBresh,Kenneth A %+ RTI International, Faculty of Health, Social Policy, Health and Economics Research Unit, 3040 East Cornwallis Road, Research Triangle Park, North Carolina, 27709-2194, United States, 1 919-316-3936, pamwilliams@rti.org %K adaptation %K cardiovascular diseases %K clinical decision support %K decision aids %K guidelines %K mHealth %K pediatrics %K risk factors %K usability %D 2016 %7 21.06.2016 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Cardiovascular disease (CVD) is 1 of the leading causes of death, years of life lost, and disability-adjusted years of life lost worldwide. CVD prevention for children and teens is needed, as CVD risk factors and behaviors beginning in youth contribute to CVD development. In 2012, the National Heart, Lung, and Blood Institute released their “Integrated Guidelines for Cardiovascular Health and Risk Reduction in Children and Adolescents” for clinicians, describing CVD risk factors they should address with patients at primary care preventative visits. However, uptake of new guidelines is slow. Clinical decision support (CDS) tools can improve guideline uptake. In this paper, we describe our process of testing and adapting a CDS tool to help clinicians evaluate patient risk, recommend behaviors to prevent development of risk, and complete complex calculations to determine appropriate interventions as recommended by the guidelines, using a user-centered design approach. Objective: The objective of the study was to assess the usability of a pediatric CVD risk factor tool by clinicians. Methods: The tool was tested using one-on-one in-person testing and a “think aloud” approach with 5 clinicians and by using the tool in clinical practice along with formal usability metrics with 14 pediatricians. Thematic analysis of the data from the in-person testing and clinical practice testing identified suggestions for change in 3 major areas: user experience, content refinement, and technical deployment. Descriptive statistical techniques were employed to summarize users’ overall experience with the tool. Results: Data from testers showed that general reactions toward the CDS tool were positive. Clinical practice testers suggested revisions to make the application more user-friendly, especially for clinicians using the application on the iPhone, and called for refining recommendations to be more succinct and better tailored to the patient. Tester feedback was incorporated into the design when feasible, including streamlining data entry during clinical visits, reducing the volume of results displayed, and highlighting critical results. Conclusions: This study found support for the usability of our pediatric CVD risk factor tool. Insights shared about this tool may be applicable for designing other mHealth applications and CDS tools. The usability of decision support tools in clinical practice depends critically on receiving (ie, through an accessible device) and adapting the tool to meet the needs of clinicians in the practice setting. %M 27328761 %R 10.2196/humanfactors.5440 %U http://humanfactors.jmir.org/2016/1/e17/ %U https://doi.org/10.2196/humanfactors.5440 %U http://www.ncbi.nlm.nih.gov/pubmed/27328761 %0 Journal Article %@ 2291-5222 %I JMIR Publications %V 4 %N 2 %P e70 %T Mobile Electronic Medical Records Promote Workflow: Physicians’ Perspective From a Survey %A Duhm,Julian %A Fleischmann,Robert %A Schmidt,Sein %A Hupperts,Hagen %A Brandt,Stephan A %+ Vision and Motor System Research Group, Department of Neurology, Charité - Universitätsmedizin Berlin, Charitéplatz 1, Berlin, 10117, Germany, 49 30 450 560045, robert.fleischmann@charite.de %K tablet PC %K electronic health record %K usability %K health services %K inpatient care %D 2016 %7 06.06.2016 %9 Original Paper %J JMIR Mhealth Uhealth %G English %X Background: As a result of demographic changes, physicians are required to deliver needed services with limited resources. Research suggests that tablet PCs with access to patient data may streamline clinical workflow. A recent study found tablets with mobile electronic medical records (EMRs) can facilitate data retrieval and produce time savings across the clinical routine within hospital settings. However, the reasons for these time savings, including details on how tablets were being used, remain unclear. The same applies to physicians’ perceptions of this tool within an inpatient setting. Objective: This study examined physicians’ perception of tablets with EMRs in an inpatient setting. The rationale was to identify both subjective and objective factors that impacted the successful implementation and use of tablets running an EMR. Methods: We developed a 57-item survey questionnaire designed to examine users’ perception of and attitude toward tablets, which was administered to 14 participating physicians following 7 weeks of tablet use. Five participants volunteered to participate in a second study that investigated physicians’ patterns of tablet use within the EMR environment by digitally tracking and storing usage behavior. Statistical analyses of questionnaire results included mean values with their bootstrapped 95% confidence intervals and multivariate analysis of variance to identify predictors of tablet use. Results: Physicians reported high degrees of satisfaction with the tablets. There was a general consensus among physicians that tablet use streamlined clinical workflow through optimized data retrieval (rated 0.69, 0.23-1.15 points better than control) and improved communication with patients and other physicians (rated 0.85, 0.54-1.15 and 0.77, 0.38-1.15 points better than control, respectively). Age (F3,11=3.54, P=.04), occupational group (F1,11=7.17, P=.04), and attitude toward novel technologies (F1,11=10.54, P=.02) predicted physicians’ satisfaction with the devices and their motivation regarding their further use. Tracking data yielded that only a few of the available functions were used frequently. Conclusions: Although tablet PCs were consistently perceived as beneficial, several factors contributed to the fact that their full potential was not fully exploited. Training in functionality and providing a reliable infrastructure might foster successful tablet implementation. %M 27268720 %R 10.2196/mhealth.5464 %U http://mhealth.jmir.org/2016/2/e70/ %U https://doi.org/10.2196/mhealth.5464 %U http://www.ncbi.nlm.nih.gov/pubmed/27268720 %0 Journal Article %@ 2292-9495 %I Gunther Eysenbach %V 3 %N 1 %P e15 %T How Regrouping Alerts in Computerized Physician Order Entry Layout Influences Physicians’ Prescription Behavior: Results of a Crossover Randomized Trial %A Wipfli,Rolf %A Ehrler,Frederic %A Bediang,Georges %A Bétrancourt,Mireille %A Lovis,Christian %+ Division of Medical Information Sciences, Department of Radiology and Medical Informatics, University Hospitals of Geneva, Rue Gabrielle-Perret-Gentil 4, Geneva,, Switzerland, 41 22 372 8697, frederic.ehrler@hcuge.ch %K medical order entry systems %K clinical decision support systems %K adverse drug reaction reporting systems %K User-Computer Interface %K eye tracking %D 2016 %7 02.06.2016 %9 Original Paper %J JMIR Human Factors %G English %X Background: As demonstrated in several publications, low positive predictive value alerts in computerized physician order entry (CPOE) induce fatigue and may interrupt physicians unnecessarily during prescription of medication. Although it is difficult to increase the consideration of medical alerts by physician through an improvement of their predictive value, another approach consists to act on the way they are presented. The interruption management model inspired us to propose an alternative alert display strategy of regrouping the alerts in the screen layout, as a possible solution for reducing the interruption in physicians’ workflow. Objective: In this study, we compared 2 CPOE designs based on a particular alert presentation strategy: one design involved regrouping the alerts in a single place on the screen, and in the other, the alerts were located next to the triggering information. Our objective was to evaluate experimentally whether the new design led to fewer interruptions in workflow and if it affected alert handling. Methods: The 2 CPOE designs were compared in a controlled crossover randomized trial. All interactions with the system and eye movements were stored for quantitative analysis. Results: The study involved a group of 22 users consisting of physicians and medical students who solved medical scenarios containing prescription tasks. Scenario completion time was shorter when the alerts were regrouped (mean 117.29 seconds, SD 36.68) than when disseminated on the screen (mean 145.58 seconds, SD 75.07; P=.045). Eye tracking revealed that physicians fixated longer on alerts in the classic design (mean 119.71 seconds, SD 76.77) than in the centralized alert design (mean 70.58 seconds, SD 33.53; P=.001). Visual switches between prescription and alert areas, indicating interruption, were reduced with centralized alerts (mean 41.29, SD 21.26) compared with the classic design (mean 57.81, SD 35.97; P=.04). Prescription behavior (ie, prescription changes after alerting), however, did not change significantly between the 2 strategies of display. The After-Scenario Questionnaire (ASQ) that was filled out after each scenario showed that overall satisfaction was significantly rated lower when alerts were regrouped (mean 4.37, SD 1.23) than when displayed next to the triggering information (mean 5.32, SD 0.94; P=.02). Conclusions: Centralization of alerts in a table might be a way to motivate physicians to manage alerts more actively, in a meaningful way, rather than just being interrupted by them. Our study could not provide clear recommendations yet, but provides objective data through a cognitive psychological approach. Future tests should work on standardized scenarios that would enable to not only measure physicians’ behavior (visual fixations and handling of alerts) but also validate those actions using clinical criteria. %M 27255612 %R 10.2196/humanfactors.5320 %U http://humanfactors.jmir.org/2016/1/e15/ %U https://doi.org/10.2196/humanfactors.5320 %U http://www.ncbi.nlm.nih.gov/pubmed/27255612 %0 Journal Article %@ 2291-9694 %I Gunther Eysenbach %V 4 %N 2 %P e15 %T Facilitating Secure Sharing of Personal Health Data in the Cloud %A Thilakanathan,Danan %A Calvo,Rafael A %A Chen,Shiping %A Nepal,Surya %A Glozier,Nick %+ Software Engineering Lab, School of Electrical and Information Engineering, The University of Sydney, Level 5, Bldg J03 (Electrical Engineering Building), Maze Crescent, Sydney, 2006, Australia, 61 02 9351 8171, Danan.Thilakanathan@sydney.edu.au %K self care %K telemedicine %K privacy %K computer security %K information dissemination %D 2016 %7 27.05.2016 %9 Original Paper %J JMIR Med Inform %G English %X Background: Internet-based applications are providing new ways of promoting health and reducing the cost of care. Although data can be kept encrypted in servers, the user does not have the ability to decide whom the data are shared with. Technically this is linked to the problem of who owns the data encryption keys required to decrypt the data. Currently, cloud service providers, rather than users, have full rights to the key. In practical terms this makes the users lose full control over their data. Trust and uptake of these applications can be increased by allowing patients to feel in control of their data, generally stored in cloud-based services. Objective: This paper addresses this security challenge by providing the user a way of controlling encryption keys independently of the cloud service provider. We provide a secure and usable system that enables a patient to share health information with doctors and specialists. Methods: We contribute a secure protocol for patients to share their data with doctors and others on the cloud while keeping complete ownership. We developed a simple, stereotypical health application and carried out security tests, performance tests, and usability tests with both students and doctors (N=15). Results: We developed the health application as an app for Android mobile phones. We carried out the usability tests on potential participants and medical professionals. Of 20 participants, 14 (70%) either agreed or strongly agreed that they felt safer using our system. Using mixed methods, we show that participants agreed that privacy and security of health data are important and that our system addresses these issues. Conclusions: We presented a security protocol that enables patients to securely share their eHealth data with doctors and nurses and developed a secure and usable system that enables patients to share mental health information with doctors. %M 27234691 %R 10.2196/medinform.4756 %U http://medinform.jmir.org/2016/2/e15/ %U https://doi.org/10.2196/medinform.4756 %U http://www.ncbi.nlm.nih.gov/pubmed/27234691 %0 Journal Article %@ 2292-9495 %I Gunther Eysenbach %V 3 %N 1 %P e7 %T Evaluating the Usability and Perceived Impact of an Electronic Medical Record Toolkit for Atrial Fibrillation Management in Primary Care: A Mixed-Methods Study Incorporating Human Factors Design %A Tran,Kim %A Leblanc,Kori %A Valentinis,Alissia %A Kavanagh,Doug %A Zahr,Nina %A Ivers,Noah M %+ OpenLab, University Health Network, 200 Elizabeth Street, Toronto, ON, M5G2C4, Canada, 1 6476789375, kim.tran2@uhn.ca %K primary health care, atrial fibrillation, electronic health records, mixed-methods research, evidence-based medicine %D 2016 %7 17.02.2016 %9 Original Paper %J JMIR Human Factors %G English %X Background: Atrial fibrillation (AF) is a common and preventable cause of stroke. Barriers to reducing stroke risk through appropriate prescribing have been identified at the system, provider, and patient levels. To ensure a multifaceted initiative to address these barriers is effective, it is essential to incorporate user-centered design to ensure all intervention components are optimized for users. Objective: To test the usability of an electronic medical record (EMR) toolkit for AF in primary care with the goal of further refining the intervention to meet the needs of primary care clinicians. Methods: An EMR-based toolkit for AF was created and optimized through usability testing and iterative redesign incorporating a human factors approach. A mixed-methods pilot study consisting of observations, semi-structured interviews, and surveys was conducted to examine usability and perceived impact on patient care and workflow. Results: A total of 14 clinicians (13 family physicians and 1 nurse practitioner) participated in the study. Nine iterations of the toolkit were created in response to feedback from clinicians and the research team; interface-related changes were made, additional AF-related resources were added, and functionality issues were fixed to make the toolkit more effective. After improvements were made, clinicians expressed that the toolkit improved accessibility to AF-related information and resources, served as a reminder for guideline-concordant AF management, and was easy to use. Most clinicians intended to continue using the toolkit for patient care. With respect to impact on care, clinicians believed the toolkit increased the thoroughness of their assessments for patients with AF and improved the quality of AF-related care received by their patients. Conclusions: The positive feedback surrounding the EMR toolkit for AF and its perceived impact on patient care can be attributed to the adoption of a user-centered design that merged clinically important information about AF management with user needs. This study demonstrates the utility of a human factors approach to piloting and refining an intervention prior to wide-scale implementation. %M 27026394 %R 10.2196/humanfactors.4289 %U http://humanfactors.jmir.org/2016/1/e7/ %U https://doi.org/10.2196/humanfactors.4289 %U http://www.ncbi.nlm.nih.gov/pubmed/27026394 %0 Journal Article %@ 2292-9495 %I Gunther Eysenbach %V 3 %N 1 %P e9 %T How Does Learnability of Primary Care Resident Physicians Increase After Seven Months of Using an Electronic Health Record? A Longitudinal Study %A Clarke,Martina A %A Belden,Jeffery L %A Kim,Min Soon %+ Department of Health Management and Informatics, University of Missouri, CE728 Clinical Support & Education, DC006.00, 5 Hospital Drive, Columbia, MO, , United States, 1 573 884 0115, kimms@health.missouri.edu %K primary care, physicians, usability, electronic health records, computerized physician order entry, user-computer interface %D 2016 %7 15.02.2016 %9 Original Paper %J JMIR Human Factors %G English %X Background: Electronic health records (EHRs) with poor usability present steep learning curves for new resident physicians, who are already overwhelmed in learning a new specialty. This may lead to error-prone use of EHRs in medical practice by new resident physicians. Objective: The study goal was to determine learnability gaps between expert and novice primary care resident physician groups by comparing performance measures when using EHRs. Methods: We compared performance measures after two rounds of learnability tests (November 12, 2013 to December 19, 2013; February 12, 2014 to April 22, 2014). In Rounds 1 and 2, 10 novice and 6 expert physicians, and 8 novice and 4 expert physicians participated, respectively. Laboratory-based learnability tests using video analyses were conducted to analyze learnability gaps between novice and expert physicians. Physicians completed 19 tasks, using a think-aloud strategy, based on an artificial but typical patient visit note. We used quantitative performance measures (percent task success, time-on-task, mouse activities), a system usability scale (SUS), and qualitative narrative feedback during the participant debriefing session. Results: There was a 6-percentage-point increase in novice physicians’ task success rate (Round 1: 92%, 95% CI 87-99; Round 2: 98%, 95% CI 95-100) and a 7-percentage-point increase in expert physicians’ task success rate (Round 1: 90%, 95% CI 83-97; Round 2: 97%, 95% CI 93-100); a 10% decrease in novice physicians’ time-on-task (Round 1: 44s, 95% CI 32-62; Round 2: 40s, 95% CI 27-59) and 21% decrease in expert physicians’ time-on-task (Round 1: 39s, 95% CI 29-51; Round 2: 31s, 95% CI 22-42); a 20% decrease in novice physicians mouse clicks (Round 1: 8 clicks, 95% CI 6-13; Round 2: 7 clicks, 95% CI 4-12) and 39% decrease in expert physicians’ mouse clicks (Round 1: 8 clicks, 95% CI 5-11; Round 2: 3 clicks, 95% CI 1-10); a 14% increase in novice mouse movements (Round 1: 9247 pixels, 95% CI 6404-13,353; Round 2: 7991 pixels, 95% CI 5350-11,936) and 14% decrease in expert physicians’ mouse movements (Round 1: 7325 pixels, 95% CI 5237-10,247; Round 2: 6329 pixels, 95% CI 4299-9317). The SUS measure of overall usability demonstrated only minimal change in the novice group (Round 1: 69, high marginal; Round 2: 68, high marginal) and no change in the expert group (74; high marginal for both rounds). Conclusions: This study found differences in novice and expert physicians’ performance, demonstrating that physicians’ proficiency increased with EHR experience. Our study may serve as a guideline to improve current EHR training programs. Future directions include identifying usability issues faced by physicians when using EHRs, through a more granular task analysis to recognize subtle usability issues that would otherwise be overlooked. %M 27025237 %R 10.2196/humanfactors.4601 %U http://humanfactors.jmir.org/2016/1/e9/ %U https://doi.org/10.2196/humanfactors.4601 %U http://www.ncbi.nlm.nih.gov/pubmed/27025237 %0 Journal Article %@ 2292-9495 %I Gunther Eysenbach %V 2 %N 2 %P e15 %T Assessing the Usability of Six Data Entry Mobile Interfaces for Caregivers: A Randomized Trial %A Ehrler,Frederic %A Haller,Guy %A Sarrey,Evelyne %A Walesa,Magali %A Wipfli,Rolf %A Lovis,Christian %+ Division of Medical Information Sciences, Department of medical imaging and medical information sciences, University Hospitals of Geneva, Rue Gabrielle-Perret-Gentil 4, Geneva, 1211, Switzerland, 41 223728697, frederic.ehrler@hcuge.ch %K data collection %K mobile applications %K computers, handheld %K user-computer interface %K vital signs %K patient safety %D 2015 %7 15.12.2015 %9 Original Paper %J JMIR Human Factors %G English %X Background: There is an increased demand in hospitals for tools, such as dedicated mobile device apps, that enable the recording of clinical information in an electronic format at the patient’s bedside. Although the human-machine interface design on mobile devices strongly influences the accuracy and effectiveness of data recording, there is still a lack of evidence as to which interface design offers the best guarantee for ease of use and quality of recording. Therefore, interfaces need to be assessed both for usability and reliability because recording errors can seriously impact the overall level of quality of the data and affect the care provided. Objective: In this randomized crossover trial, we formally compared 6 handheld device interfaces for both speed of data entry and accuracy of recorded information. Three types of numerical data commonly recorded at the patient’s bedside were used to evaluate the interfaces. Methods: In total, 150 health care professionals from the University Hospitals of Geneva volunteered to record a series of randomly generated data on each of the 6 interfaces provided on a smartphone. The interfaces were presented in a randomized order as part of fully automated data entry scenarios. During the data entry process, accuracy and effectiveness were automatically recorded by the software. Results: Various types of errors occurred, which ranged from 0.7% for the most reliable design to 18.5% for the least reliable one. The length of time needed for data recording ranged from 2.81 sec to 14.68 sec, depending on the interface. The numeric keyboard interface delivered the best performance for pulse data entry with a mean time of 3.08 sec (SD 0.06) and an accuracy of 99.3%. Conclusions: Our study highlights the critical impact the choice of an interface can have on the quality of recorded data. Selecting an interface should be driven less by the needs of specific end-user groups or the necessity to facilitate the developer’s task (eg, by opting for default solutions provided by commercial platforms) than by the level of speed and accuracy an interface can provide for recording information. An important effort must be made to properly validate mobile device interfaces intended for use in the clinical setting. In this regard, our study identified the numeric keyboard, among the proposed designs, as the most accurate interface for entering specific numerical values. This is an important step toward providing clearer guidelines on which interface to choose for the appropriate use of handheld device interfaces in the health care setting. %M 27025648 %R 10.2196/humanfactors.4093 %U http://humanfactors.jmir.org/2015/2/e15/ %U https://doi.org/10.2196/humanfactors.4093 %U http://www.ncbi.nlm.nih.gov/pubmed/27025648 %0 Journal Article %@ 2292-9495 %I Gunther Eysenbach %V 2 %N 2 %P e14 %T Usability Testing of a Complex Clinical Decision Support Tool in the Emergency Department: Lessons Learned %A Press,Anne %A McCullagh,Lauren %A Khan,Sundas %A Schachter,Andy %A Pardo,Salvatore %A McGinn,Thomas %+ Hofstra North Shore-LIJ School of Medicine, Department of Medicine, 4th Floor, 300 Community Drive, Manhasset, NY, 11030, United States, 1 267 979 7940, apress@nshs.edu %K clinical decision support %K emergency department %K usability testing %K clinical prediction rules %K Wells criteria %K pulmonary embolism %D 2015 %7 10.09.2015 %9 Original Paper %J JMIR Human Factors %G English %X Background: As the electronic health record (EHR) becomes the preferred documentation tool across medical practices, health care organizations are pushing for clinical decision support systems (CDSS) to help bring clinical decision support (CDS) tools to the forefront of patient-physician interactions. A CDSS is integrated into the EHR and allows physicians to easily utilize CDS tools. However, often CDSS are integrated into the EHR without an initial phase of usability testing, resulting in poor adoption rates. Usability testing is important because it evaluates a CDSS by testing it on actual users. This paper outlines the usability phase of a study, which will test the impact of integration of the Wells CDSS for pulmonary embolism (PE) diagnosis into a large urban emergency department, where workflow is often chaotic and high stakes decisions are frequently made. We hypothesize that conducting usability testing prior to integration of the Wells score into an emergency room EHR will result in increased adoption rates by physicians. Objective: The objective of the study was to conduct usability testing for the integration of the Wells clinical prediction rule into a tertiary care center’s emergency department EHR. Methods: We conducted usability testing of a CDS tool in the emergency department EHR. The CDS tool consisted of the Wells rule for PE in the form of a calculator and was triggered off computed tomography (CT) orders or patients’ chief complaint. The study was conducted at a tertiary hospital in Queens, New York. There were seven residents that were recruited and participated in two phases of usability testing. The usability testing employed a “think aloud” method and “near-live” clinical simulation, where care providers interacted with standardized patients enacting a clinical scenario. Both phases were audiotaped, video-taped, and had screen-capture software activated for onscreen recordings. Results: Phase I: Data from the “think-aloud” phase of the study showed an overall positive outlook on the Wells tool in assessing a patient for a PE diagnosis. Subjects described the tool as “well-organized” and “better than clinical judgment”. Changes were made to improve tool placement into the EHR to make it optimal for decision-making, auto-populating boxes, and minimizing click fatigue. Phase II: After incorporating the changes noted in Phase 1, the participants noted tool improvements. There was less toggling between screens, they had all the clinical information required to complete the tool, and were able to complete the patient visit efficiently. However, an optimal location for triggering the tool remained controversial. Conclusions: This study successfully combined “think-aloud” protocol analysis with “near-live” clinical simulations in a usability evaluation of a CDS tool that will be implemented into the emergency room environment. Both methods proved useful in the assessment of the CDS tool and allowed us to refine tool usability and workflow. %M 27025540 %R 10.2196/humanfactors.4537 %U http://humanfactors.jmir.org/2015/2/e14/ %U https://doi.org/10.2196/humanfactors.4537 %U http://www.ncbi.nlm.nih.gov/pubmed/27025540