abstract

Exploring Mixed Reality in Specialized Surgical Environments

Authors:

Danilo Gasques Rodrigues,

Steven R. Rick,

Preetham Suresh,

Nadir WeibelAuthors Info & Claims

CHI EA '17: Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems

Pages 2591 - 2598

https://doi.org/10.1145/3027063.3053273

Published: 06 May 2017 Publication History

Abstract

Recent technology advances in both Virtual Reality and Augmented Reality are creating an opportunity for a paradigm shift in the design of human-computer interaction systems. Delving into the Reality-Virtuality Continuum, we find Mixed Reality - systems designed to augment the physical world with virtual entities that embody characteristics of real world objects. In the medical field, Mixed Reality systems can overlay real-time and spatially accurate results onto a patient's body without the need for external screens. The complexity of these systems previously required specialized prototypes, but newly available commercial products like the Microsoft HoloLens make the technology more available. Through a combination of literature review, expert analysis, and prototyping we explore the use of Mixed Reality in healthcare. From the experience of prototyping Patiently and HoloSim, two applications for augmenting medical training, we outline considerations for the future design and development of virtual interfaces grounded in reality.

References

[1]

Ronald T Azuma. 1997. A survey of augmented reality. Presence: Teleoperators and virtual environments 6, 4 (1997), 355--385.

[2]

Michael Bajura, Henry Fuchs, and Ryutarou Ohbuchi. Merging Virtual Objects with the Real World: Seeing Ultrasound Imagery Within the Patient. In Proceedings SIGGRAPH '92, the 19th Annual Conference on Computer Graphics and Interactive Techniques (1992). ACM, 203--210.

Digital Library

[3]

Reinhold Behringer, Johannes Christian, Andreas Holzinger, and Steve Wilkinson. Some usability issues of augmented and mixed reality for e-health applications in the medical domain. In Symposium of the Austrian HCI and Usability Engineering Group (2007). Springer, 255--266.

[4]

Mark Billinghurst, Adrian Clark, and Gun Lee. 2015. A survey of augmented reality. Foundations and Trends in Human-Computer Interaction 8, 2--3 (2015), 73-- 272.

Digital Library

[5]

Henry Chen, Austin S Lee, Mark Swift, and John C Tang. 2015. 3D Collaboration Method over HoloLens and Skype End Points. In Proceedings of the 3rd International Workshop on Immersive Media Experiences. ACM, 27--30.

Digital Library

[6]

Steven Feiner, Blair MacIntyre, Marcus Haupt, and Eliot Solomon. 1993b. Windows on the world: 2D windows for 3D augmented reality. In Proceedings of UIST'93, the 6th annual ACM symposium on User interface software and technology. ACM, 145--155.

Digital Library

[7]

Steven Feiner, Blair Macintyre, and Dorée Seligmann. 1993a. Knowledge-based Augmented Reality. Commun. ACM 36, 7 (July 1993), 53--62.

Digital Library

[8]

Denis Kalkofen, Erick Mendez, and Dieter Schmalstieg. 2009. Comprehensible visualization for augmented reality. IEEE transactions on visualization and computer graphics 15, 2 (2009), 193--204.

Digital Library

[9]

Xin Kang, Mahdi Azizian, Emmanuel Wilson, Kyle Wu, Aaron D Martin, Timothy D Kane, Craig A Peters, Kevin Cleary, and Raj Shekhar. 2014. Stereoscopic augmented reality for laparoscopic surgery. Surgical endoscopy 28, 7 (2014), 2227--2235.

[10]

Marta Kersten-Oertel, Pierre Jannin, and D Louis Collins. 2012. DVV: a taxonomy for mixed reality visualization in image guided surgery. IEEE Transactions on Visualization and Computer Graphics 18, 2 (2012), 332--352.

Digital Library

[11]

Fatimah Lateef and others. 2010. Simulation-based learning: Just like the real thing. Journal of emergencies, trauma, and shock 3, 4 (2010), 348.

[12]

Mark A Livingston, William F Garrett, Gentaro Hirota, Mary C Whitton, Etta D Pisano, Henry Fuchs, and others. 1996. Technologies for augmented reality systems: realizing ultrasound-guided needle biopsies. In Proceedings of SIGGRAPH '96, 23rd annual conference on computer graphics and interactive techniques. ACM, 439--446.

[13]

Paul Milgram, Shumin Zhai, David Drascic, and Julius Grodski. 1993. Applications of augmented reality for human-robot communication. In Intelligent Robots and Systems93, IROS'93. Proceedings of the 1993 IEEE/RSJ International Conference on, Vol. 3. IEEE, 1467--1472.

[14]

Nobutaka Mitsuhashi, Kaori Fujieda, Takuro Tamura, Shoko Kawamoto, Toshihisa Takagi, and Kousaku Okubo. 2009. BodyParts3D: 3D structure database for anatomical concepts. Nucleic acids research 37, suppl 1 (2009), D782--D785.

[15]

Soh Nishimoto, Maki Tonooka, Kazutoshi Fujita, Yohei Sotsuka, Toshihiro Fujiwara, Kenichiro Kawai, and Masao Kakibuchi. 2016. An augmented reality system in lymphatico-venous anastomosis surgery. Journal of surgical case reports 2016, 5 (2016).

[16]

Timothy M Rankin, Marvin J Slepian, and David G Armstrong. 2015. Augmented Reality in Surgery. In Technological Advances in Surgery, Trauma and Critical Care. Springer, 59--71.

[17]

Tobias Sielhorst, Tobias Obst, Rainer Burgkart, Robert Riener, and Nassir Navab. 2004. An augmented reality delivery simulator for medical training. In International Workshop on Augmented Environments for Medical Imaging-MICCAI Satellite Workshop, Vol. 141. 11--20.

[18]

Ivan E. Sutherland. The Ultimate Display. In Proceedings of the IFIP Congress (1965). 506--508.

[19]

J Edward Swan and Joseph L Gabbard. 2005. Survey of user-based experimentation in augmented reality. In Proceedings of 1st International Conference on Virtual Reality. 1--9.

Cited By

Zhao YZhou YJin DDong TWang GYing FShen QCao J(2024)Towards Equitable CPR: An Interactive System for Female CPR TrainingExtended Abstracts of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613905.3651092(1-5)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3651092
Uhl JGutierrez RRegal GSchrom-Feiertag HSchuster BTscheligi M(2024)Choosing the Right Reality: A Comparative Analysis of Tangibility in Immersive Trauma SimulationsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3641912(1-17)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3641912
Antonelli MLucignani MParrillo CGrassi FFigà Talamanca LRossi Espagnet MGandolfo CSecinaro APasquini LDe Benedictis APlacidi EDe Palma LMarras CMarasi ANapolitano A(2023)Magnetic resonance imaging based neurosurgical planning on hololens 2: A feasibility study in a paediatric hospitalDIGITAL HEALTH10.1177/205520762312140669Online publication date: 16-Nov-2023
https://doi.org/10.1177/20552076231214066
Show More Cited By

Index Terms

Exploring Mixed Reality in Specialized Surgical Environments
1. Human-centered computing

Recommendations

Adding Force Feedback to Mixed Reality Experiences and Games using Electrical Muscle Stimulation
CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems

We present a mobile system that enhances mixed reality experiences and games with force feedback by means of electrical muscle stimulation (EMS). The benefit of our approach is that it adds physical forces while keeping the users' hands free to interact ...
Merging environments for shared spaces in mixed reality
VRST '18: Proceedings of the 24th ACM Symposium on Virtual Reality Software and Technology

In virtual reality a real walking interface limits the extent of a virtual environment to our local walkable space. As local spaces are specific to each user, sharing a virtual environment with others for collaborative work or games becomes complicated. ...
HoloLens-based Mixed Reality Experiences for Subjects with Alzheimer's Disease
CHItaly '17: Proceedings of the 12th Biannual Conference on Italian SIGCHI Chapter

HoloLens technology enables mixed reality experiences that integrate holographic objects into the real world in which the mixed reality head-mounted device is used. Since HoloLens' launch in March 2016, some mixed-reality applications of this technology ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI EA '17: Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems

May 2017

3954 pages

ISBN:9781450346566

DOI:10.1145/3027063

General Chairs:
Gloria Mark
University of California Irvine
,
Susan Fussell
Cornell University
,
Program Chairs:
Cliff Lampe
University of Michigan
,
m.c. schraefel
University of Southampton
,
Juan Pablo Hourcade
University of Iowa
,
Caroline Appert
Université Paris-Sud
,
Daniel Wigdor
University of Toronto

Copyright © 2017 Owner/Author.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 May 2017

Check for updates

Author Tags

Qualifiers

Abstract

Conference

CHI '17

Sponsor:

SIGCHI

CHI '17: CHI Conference on Human Factors in Computing Systems

May 6 - 11, 2017

Colorado, Denver, USA

Acceptance Rates

CHI EA '17 Paper Acceptance Rate 1,000 of 5,000 submissions, 20%;

Overall Acceptance Rate 6,164 of 23,696 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

18
Total Citations
View Citations
1,298
Total Downloads

Downloads (Last 12 months)53
Downloads (Last 6 weeks)6

Reflects downloads up to 04 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zhao YZhou YJin DDong TWang GYing FShen QCao J(2024)Towards Equitable CPR: An Interactive System for Female CPR TrainingExtended Abstracts of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613905.3651092(1-5)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3651092
Uhl JGutierrez RRegal GSchrom-Feiertag HSchuster BTscheligi M(2024)Choosing the Right Reality: A Comparative Analysis of Tangibility in Immersive Trauma SimulationsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3641912(1-17)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3641912
Antonelli MLucignani MParrillo CGrassi FFigà Talamanca LRossi Espagnet MGandolfo CSecinaro APasquini LDe Benedictis APlacidi EDe Palma LMarras CMarasi ANapolitano A(2023)Magnetic resonance imaging based neurosurgical planning on hololens 2: A feasibility study in a paediatric hospitalDIGITAL HEALTH10.1177/205520762312140669Online publication date: 16-Nov-2023
https://doi.org/10.1177/20552076231214066
Uhl JSchrom-Feiertag HRegal GGallhuber KTscheligi M(2023)Tangible Immersive Trauma Simulation: Is Mixed Reality the next level of medical skills training?Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581292(1-17)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581292
Woodward JRuiz J(2023)Analytic Review of Using Augmented Reality for Situational AwarenessIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2022.314158529:4(2166-2183)Online publication date: 1-Apr-2023
https://dl.acm.org/doi/10.1109/TVCG.2022.3141585
Kabuye ELeDuc PCagan J(2023)A mixed reality system combining augmented reality, 3D bio-printed physical environments and inertial measurement unit sensors for task planningVirtual Reality10.1007/s10055-023-00777-027:3(1845-1858)Online publication date: 2-Mar-2023
https://dl.acm.org/doi/10.1007/s10055-023-00777-0
Ye WHe NWang J(2022)Research on Augmented Reality Technology in the Training of Pre-flight Safety Inspect Process2022 IEEE Asia-Pacific Conference on Image Processing, Electronics and Computers (IPEC)10.1109/IPEC54454.2022.9777423(68-71)Online publication date: 14-Apr-2022
https://doi.org/10.1109/IPEC54454.2022.9777423
Cai HDing RHuang WZhan YYang R(2021)Fast Calibration with OTS for AR-based Surgical Navigation2021 27th International Conference on Mechatronics and Machine Vision in Practice (M2VIP)10.1109/M2VIP49856.2021.9665005(738-743)Online publication date: 26-Nov-2021
https://doi.org/10.1109/M2VIP49856.2021.9665005
Ye WYu QHe NPang L(2021)Augmented reality for assist apron operation safety supervisor in aircraft inspection processes2021 IEEE International Conference on Power Electronics, Computer Applications (ICPECA)10.1109/ICPECA51329.2021.9362661(273-276)Online publication date: 22-Jan-2021
https://doi.org/10.1109/ICPECA51329.2021.9362661
Gomes Ataide EFritzsche HFilax MChittamuri DPotluri LFriebe M(2020)ENT Endoscopic Surgery and Mixed RealityBiomedical and Clinical Engineering for Healthcare Advancement10.4018/978-1-7998-0326-3.ch002(17-29)Online publication date: 2020
https://doi.org/10.4018/978-1-7998-0326-3.ch002
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents