Abstract

Introduction

Despite the widespread adoption of patient simulation training and the evidence to support its value in medical education, ^1,2 current patient simulators typically lack the ability of trainees to identify important internal landmarks and anatomy. However, current technological advancements are providing new modalities for simulation-based training to address this deficiency. One such promising approach is Augmented Reality (AR) which is created by combining real and virtual data with real time interactivity and 3-dimensional registration.³ An Augmented Reality environment consists of 3-D images or animations that can be displayed in the same field of view as real objects, providing an enhanced but realistic view. AR differs from Virtual Reality (VR) in that the user interaction provides a more realistic environment compared to a virtual one.⁴ This results from the fact that some of the viewed objects are real and some are virtual, leading to the closely related term “mixed reality” Using this approach, the trainee wears virtual reality goggles that project internal structures onto a mannequin. The goggles contain visual sensors that detect markers that are placed on the mannequin. This provides an input by which the internal anatomy is constantly displayed within the field of view of the mannequin, creating an ongoing illusion combined with a real object i.e. the mannequin. An object such as a needle can also be tagged with a marker that then can be sensed by the goggles, allowing a virtual needle to be generated with the environment. The procedure is performed as it would in routine mannequin based simulation, but with the addition of the AR generated by the goggles which allows the trainee to place the needle within the mannequin while viewing its interaction with the internal anatomy in real time. The technology also alerts the trainees in real-time with respect to complications (e.g., bleeding) or procedural errors.

Preliminary studies involving AR suggest that this technology may be effective in teaching a number of different clinical procedural skills.^5,6 However, a recent review of available AR applications by Barsom et al. revealed that although use of this technology is gaining interest, data are still lacking to support its’ overall applicability and effectiveness.⁴ These authors however, suggest that if there is sufficient evidence to support its’ integration into medical training, AR could soon play a large role in medical education. This review highlighted that categories of current AR applications were limited to the disciplines of laparoscopy, neurosurgery and echocardiography. Only one report of use of AR in central venous catheter (CVC) insertion was identified. This study performed by Jeon et al. used ultra-sound enhanced imaging to assist in the AR imaging. The primary limitation of this study was that the ultrasound model used was a vascular phantom, and thus lacked the capacity to project true human anatomy.⁷

Previous development of AR technology has relied on what is termed 3^rd person point of view (POV) in which the virtual anatomical images and procedure (performed by a trainer) are projected onto a remote screen that can be viewed by the trainees. While this approach facilitates group learning, the passive nature precludes individual “hands-on” participation. To obviate this limitation, we have developed a 1st person POV approach in which the individual trainee is able to visualize a stereoscopic virtual image of the anatomy in real-time through use of the augmented reality glasses or goggles. This approach thus enables seamless integration into individualized rather than group medical training. Although early iterations of AR goggles were limited by issues related to a lack of depth perception and slow refresh rates, recent advances in optical see-through AR glasses and goggles have largely overcome these early problems.

For the purposes of this study, we evaluated the usability and feasibility of a 1st person POV augmented reality application on the ability of participants with varying levels of clinical experience to correctly place a CVC in the right internal jugular vein (RIJ) of an AR capable mannequin.

Methods

This project was deemed exempt by the University of Michigan Medical School’s Institutional Review Board.

Prototype development

Using medical textbooks, the extant literature and expert opinion, graphic artists create 3-dimensional computer image models of the anatomical structures e.g., clavicle, sternocleidomastoid muscle, jugular vein, etc. The 3D image models are then manipulated using Unity^® Game Engine and Vuforia^® to render, scale and register the computer images virtual graphics on the mannequin. Given that CVC placement requires insertion of a flexible object (needle); rigid-body tracking cannot be used. Instead, flexible instruments are augmented with small infrared reflective markers which are detected by an infrared-seeing camera mounted at a fixed pose relative to the mannequin. (Figure 1) Video streams captured by the camera are processed to develop a statistical model representing the needle and its insertion into the mannequin. In each camera frame, pixels belonging to the infrared reflective markers are extracted by simple thresholding (on the basis of pixel intensities, infrared markers being brighter than other objects). Thus, by running an optical flow algorithm on the current and previous frames’ feature sets; an estimate of the motion of the needle is calculated. The AR glasses used in this study were Epson Moverio BT-200^® Smart Glasses (Epson America, Inc., Long Beach, CA) that allowed for 1^st person POV stereoscopic imaging.

Open in a separate window

Figure 1

The trainee’s view of the internal anatomy as seen through the augmented reality glasses (1^st person POV).

Participants

The study population included third and fourth-year medical students attending the University of Michigan’s Medical School (UMMS) together with residents, fellows, faculty and nurse anesthetists from the Department of Anesthesiology. Participants were identified through UMMS e-mail listings and face-to-face recruitment. Participants were guaranteed that performance in this project would have no effect on clinical evaluations.

Medical student participants and those with limited experience with CVC placement watched a brief video explaining the 1st person POV AR technology and what the testing would entail. All other participants received the same information verbally by two of the authors (LR, AT) based on their level of experience. Following the video or introduction, participants were then individually guided through an identical sequence of events using the AR trainer. The AR glasses were adjusted and tested to ensure comfort and visual accuracy. Participants were asked to locate the clavicle and the two heads of the sternocleidomastoid muscle by palpation. Once identified correctly, the internal anatomy was projected through the glasses and simultaneously on the accompanying television screen. The triangle formed by the sternocleidomastoid muscle and clavicle was highlighted, revealing the apex of the triangle as the target for needle insertion. Following identification of the apex, the internal jugular vein and internal carotid artery appeared, allowing for appreciation of their position relative to the previously identified anatomy. Each participant was then asked to practice needle insertion using the AR-adapted needle and with the anatomy in view. Once the participant felt confident with their technique, they were then asked to insert the needle without the anatomy in view. If the needle was inserted correctly into the vein, the needle turned red to simulate blood return and the anatomy was revealed to confirm correct placement. If the carotid artery was inadvertently punctured, the anatomy would be revealed with an expanding hematoma. Participants not being tested were able to watch from a 3^rd person POV on a television screen (Figure 2).

Open in a separate window

Figure 2

Trainee attempting central venous catheter placement on the mannequin. The trainee’s view of the projected internal anatomy is seen on the remote TV screen (3^rd person POV).

Two trained raters (LR, AT) independently documented performance via an assessment (see Document, Supplemental Digital Content 1, participant assessment) checklist created in an online survey software tool (Qualtrics, Provo, UT). The checklist items reflected each step of the procedure and were dichotomous (yes/no) indicating that the participant had either correctly performed each task or had not. The primary tasks that were evaluated for completion were correct identification of anatomy, correct needle insertion angle and direction, and maintenance of the needle within the vein.⁸ Total time from participant start to finish as well as time to needle insertion and advancement in the RIJ vein without AR assistance were recorded.

At the conclusion of the procedure, participants completed a survey (see Document, Supplemental Digital Content 2, participant evaluation) to elicit information about their level of training, prior experience with CVC line placement, satisfaction with the AR technology, and perceptions of AR as a potential teaching tool for medical skills training. The participants were also surveyed to determine their likes and dislikes, potential barriers to successful adoption of AR technology, and any suggestions for improvement.

Results

A total of 40 participants with varied levels of experience were included. Demographics of the study sample are described in Table 1. Prior experience and training with central line placement and training are shown in Table 2. Less than half of the participants had received prior simulation-based training in CVC insertion. None of the medical students had prior experience with central line placement. All of those who had previously placed a CVC had only done so in the internal jugular location.

Table 2

Experience with CVC Training and Insertion

Experience	n/N (%)
Prior CVC training (Yes)	27/40 (67.5)
If prior CVC training:
With simulator-Yes?	12/27 (44.4)
Simulator type:
Partial mannequin with ultrasound	10/12 (83.3)
Multiple types	2/12 (16.7)
Prior CVC insertions:
None	24/40 (60.0)
1–5	5/40 (12.5)
6–10	0/40 (0.0)
11–20	1/40 (2.5)
>20	10/40 (25.0)
Location of prior CVC insertions:
Internal jugular	16/16 (100.0)

Open in a separate window

CVC = Central venous catheterization

Overall, participants reported that the AR technology was realistic, easy to use, and would be a useful adjunct to current training (Table 3). Comparisons of the perceptions of the AR technology between individuals with and without prior CVC training revealed very similar findings. For example, compared with participants with no prior CVC training, those with prior CVC experience reported that the AR technology: helped them identify the anatomical landmarks (84.6% vs 88.9%, respectively); was fun to use (76.9% vs 77.8%); would improve their skills (76.9% vs 66.7%); promoted learning (84.6% vs 85.2%), and; would be useful in their medical skills training (84.6% vs 85.2%). Of those with prior training in CVC placement 59.3% reported that the AR technology was superior compared to other methods of simulated procedural skills training.

Evaluation of CVC insertion milestones adjusted for experience reveals that all participants were able to correctly identify the required anatomy and place the needle (Table 4). However, those with no prior training were significantly more likely to inadvertently puncture the carotid artery during the testing phase. The results in this table describe achievements by the participants without using the AR projection followed by confirmation of achievement with the AR technology.

Table 4

Competency achievements by experience

	No Prior CVC Training (n = 13)	Prior CVC Training (n = 27)	Overall (N = 40)
Correct identification of:
Clavicle	13 (100.0)	27 (100.0)	40 (100.0)
Heads of Sternocleidomastoid	13 (100.0)	26 (96.3)	39 (97.5)
Apex of triangle	12 (92.3)	26 (96.3)	38 (95.0)
Correct:
Needle insertion site	13 (100.0)	24 (88.9)	37 (92.5)
Insertion direction	11 (84.6)	26 (96.3)	37 (92.5)
Insertion angle	4 (30.8)	13 (48.1)	17 (42.5)
Vein cannulation	10 (76.9)	26 (96.3)	36 (90.0)
Needle remained stable in vein	9 (69.2)	25 (92.6)	34 (85.0)
Carotid puncture	4 (30.8)	1 (3.7)*	5 (12.5)
Times (secs) from:
Identification of anatomy to vein cannulation	152.3 ± 78.2	141.1 ± 41.6	144.9 ± 55.7
Needle insertion to vein cannulation	48.88 ±49.6	37.42 ±17.9	41.24 ± 31.9

Open in a separate window

CVC = Central venous catheterization

Data are n (%),

^*P< 0.05 vs no prior CVC training

Open-ended comments regarding the participants’ positive and negative impressions of the augmented reality prototype and its potential as a training tool for medical procedures were recorded and sorted by themes using standard qualitative techniques (e.g., proofreading, identifying repetitions, indigenous categories, and similarities/differences).⁹ Positive comments were primarily related to the ability to visualize the internal structures and anatomy in 3 dimensions (26/39 responses = 69.2%). Seven (17.9%) participants combined positive remarks regarding the interactive nature of the prototype and the observation that it was a fun and novel way to learn a procedure. A sample of positive comments is provided verbatim.

“Ability to see overlying anatomical structures as the procedure is in process”

“As a total beginner when it comes to CVC insertion, it was really helpful to visualize the internal anatomy. This was a helpful way to understand internal anatomy as it relates to the procedure. I think this kind of simulation is helpful for a beginner like me.”

Negative comments were largely related to bugs inherent in any prototype. In this case, the majority of negative comments addressed issues related to the fit of the goggles (particularly for those who wore eyeglasses (9/36 responses =25.0%) and the stability of the image or need to keep the head relatively still (16/36 = 44.4%). Examples of some of the negative comments are included.

“Wearing the glasses over glasses (that is, the ones I need to see) was clunky and awkward.”

“The initial anatomy of the SCM/clavicle seemed to be a bit off from what I had palpated on the mannequin. However, it lined up later. Keeping head in one specific position was somewhat difficult.”

Comments regarding perceived barriers to implementation of AR as a potential training tool were primarily related to the potential cost of implementation (10/31 responses = 32.3%) and bugs related to the alignment of images and comfort of the glasses (9/31 = 29.0%). Verbatim comments related to perceived barriers to implementation included.

“I question the cost/utility ratio. You might be able to achieve the same thing with a printed picture lying over the mannequin.”

“Different size people and individuals with corrected vision - the glasses are limiting in those people.”

Discussion

Results of this study demonstrated the usability and feasibility of a 1^st person POV AR application in simulated CVC insertion training. Participants with a variety of levels of experience found the AR technology to be realistic and the features consistent with promotion of learning and skills facilitation. Furthermore, there were no significant differences in the identification of landmarks and CVC placement between experienced and inexperienced trainees, supporting the evidence that AR can be a useful tool for providers of all levels of experience. While initial results from this pilot study are encouraging, the authors recognize that there is still much work to be done to improve this technology prior to implementation as a potentially useful procedural training tool for broad levels of medical training.

Research into the potential uses of AR is quickly advancing. There is existing evidence to support its beneficial impact in many aspects of clinical medicine and medical training such as laparoscopic surgery, neurosurgery and cardiac pulmonary resuscitation (CPR).^4,10–17 One study, for example found that pediatric medical workers performed better after training on a virtual-reality enhanced child mannequin¹⁸ and another showed that AR technology for CPR was deemed realistic and useful.¹⁹ Researchers have found that trainees prefer AR teaching devices over VR devices due to the addition of tactile feedback.⁵ A report from the National League of Nursing identified augmented reality as one of six emerging technologies that are expected to significantly impact learning, teaching and research in the near future.²⁰

A potential advantage of AR over VR is the ability to allow the trainee to visualize the relevant internal structures in a physical anatomic model, rather than a screen based model. Whereas most standard mannequin simulators allow for recognition of external anatomical landmarks, AR technology also allows the trainee to identify the internal structures that are pertinent to the procedure being performed. In this study the AR glasses guided the trainees to the correct point of needle insertion and alerted them to the relative positioning of the internal jugular and carotid artery. The prototype used in this study was also unique in that it provided a 1^st person POV of the internal anatomical landmarks. The 1^st person POV provides the operator with the full tactile experience of the procedure, compared to passively watching, or 3^rd person POV. Previous work has largely utilized 3^rd Person POV wherein the internal landmarks are projected on a TV screen. Use of a 1^st person approach better simulates the real clinical situation in that it allows the trainee to focus attention on the area of interest and not on a remote screen. To our knowledge, this is the first study to have evaluated 1^st person POV AR technology for medical training.

CVC’s are inserted in more than 5 million patients each year, with an estimated complication rate as high as 15%.²¹ Deliberate practice by residents in simulated settings has shown improved performance in CVC insertion both in simulation-based assessment as well as during placement in actual patients.^22,23 The ability to identify the location of internal landmarks, as well as visualizing normal and pathological states of the body during training would help trainees understand the relationship between their actions and what is occurring in the body.

Although this study was a pilot of a new prototype, it is important to recognize the potential limitations of the technology and how it might be incorporated into medical training curricula.²⁴ The primary complaints of the prototype used in our study related to the constraints of the 3-D AR glasses such as fit (particularly for those wearing eye glasses), the requirement to keep the head still, and, for some, a feeling of unnatural head positioning. While we acknowledge that the adjustment of the glasses and alignment of images may have slightly delayed the time to completion of the procedure, the data suggest that the ability of the trainees to correctly identify the salient structures and place the needle correctly was not compromised by these adjustments. However, updated models of the hardware continue to be in development with the full expectation that these “bugs” will be eliminated in the future. Participants perceived the cost of AR development as a potential barrier to implementation of AR technology. Raque et al. demonstrated that using simulation-based education for surgery residents increased the cost of training each resident, with only minimal impact in training and a decrease in faculty time spent on instruction.²⁵ Thus, for advanced tools such as AR to make a broad impact, they will have to be cost-effective. Although it is likely, as with any burgeoning technology, that developmental and production costs will decrease over time, it will be important to fully evaluate the AR technology against standard simulation methodology to ensure efficacy and cost-benefit.

As a pilot, this study is limited by its small size and focus on usability and feasibility. Future studies comparing the AR technology with standard simulation training models will thus be important. Although internal jugular placement carries other potential risks such as hemothorax, for the purposes of this evaluation, only carotid artery puncture and multiple attempts were included as complications.

In summary, this pilot study demonstrates the usability and feasibility of 1^st person POV AR technology as a potentially promising training tool for central line placement. Of note was the reported unique ability of this approach to provide visualization of internal anatomical landmarks and meaningful hands-on experience. Further studies comparing AR technology with standard simulation trainers for a variety of procedural skills will thus be important prior to implementation into medical simulation-education curricula.

Supplementary Material

Contributor Information

Lauryn R. Rochlen, Department of Anesthesiology, University of Michigan, Ann Arbor, MI.

Robert Levine, Emergency Care Center, Jackson Memorial Hospital, Miami, FL; ArchieMD, Inc., Boca Raton, FL.

Alan R. Tait, Department of Anesthesiology, University of Michigan, Ann Arbor, MI; Center for Bioethics and Social Sciences in Medicine, University of Michigan, Ann Arbor, MI.

References