Article

Visual support for medical communication by using projector-based augmented reality and thermal markers

Authors:

Jeremy Bluteau,

Itaru Kitahara,

Yoshinari Kameda,

Kiyoshi Kogure, and

Yuichi OhtaAuthors Info & Claims

ICAT '05: Proceedings of the 2005 international conference on Augmented tele-existence

December 2005

Pages 98 - 105

https://doi.org/10.1145/1152399.1152418

Published: 05 December 2005 Publication History

Abstract

This paper presents a system that allows patients and physicians to experience better communication during medical consultations using Augmented Reality (AR) technology. The AR system can superimpose augmentations (i.e., human body components) onto the real patient's body, and such annotated information serves to form the cornerstone for collaborative work between the two actors. We focus on the advantages of projector-based technology and ARToolKit. Our technique, based on thermal markers (i.e., using human body temperature as a source of information) is used for tracking the location of pain in the patient through the projected augmentations. The second aim of using thermal markers is to protect the patient's privacy. The required calibration method between thermal-camera and projector is also presented. The system's feasibility is demonstrated through development of a complete application.

References

[1]

{{Azu97} R. T. Azuma: "A Survey of Augmented Reality." Presence: Teleoperators and Virtual Environments 6, 4 (August 1997), 355--385.

[2]

{Ash03} M. Ashdown and P. Robinson: "Experiences Implementing and Using Personal Projected Displays." In IEEE International Workshop on Projector-Camera Systems, (Procams 2003).

[3]

{Ash03a} M. Ashdown and R. Sukthankar. "Robust calibration of camera-projector system for multi-planar displays." Technical Report HP Labs HPL-2003-24, January 2003.

[4]

{Baj92} M. Bajura, H. Fuchs, and R. Ohbuchi: "Merging Virtual Objects with the Real World: Seeing Ultrasound Imagery within the Patient." In Proceedings of SIGGRAPH '92 (Chicago, IL, July 26--31, 1992). In Computer Graphics 26, #2 (July 1992): 203--210.

Digital Library

[5]

{Bil99} M. Billinghurst, H. Kato, S. Weghorst, and T. A. Furness. "A mixed-reality 3D conferencing application." Technical report, Human Interface Technology Laboratory, University of Washington, 1999.

[6]

{Fis04} J. Fischer, M. Neff, D. Freudenstein, and D. Bartz. "Medical Augmented Reality based on Commercial Image Guided Surgery." In Eurographics Symposium on Virtual Environments (EGVE), June 2004.

Digital Library

[7]

{Fuc98} H. Fuchs, M. Livingston, R. Raskar, D. Colucci, K. Keller, A. State, J. Crawford, P. Rademacher, S. Drake and A. Mayer: "Augmented reality visualization for laparoscopic surgery." In Proceedings of Medical Image Computing and Computer-Assisted Intervention '98 (MICCAI '98), volume 1496 of LNCS, pages 934--943, October, 1998.

Digital Library

[8]

{Goe03} G. Goebbels, K. Troche, M. Braun, A. Ivanovic, A. Grab, K. von Lübtow, H. F. Zeilhofer, R. Sader, F. Thieringer, K. Albrecht, K. Praxmarer, E. Keeve, N. Hanssen, Z. Krol and F. Hasenbrink: "Development of an Augmented Reality System for intra-operative navigation in maxillo-facial surgery." In 2. Jahrestagung des Gesellschaft für Computer-und Roboterassistierte Chirurgie (CURAC), November 2003.

[9]

{Gra03} R. Grasset and J.-D. Gascuel. "Réalité Augmentée et Environnement Collaboratif: Un Tour d'Horizon." In: AFIG '03 (Actes des 16èmes journéees de l'AFIG), December 2003.

[10]

{Ina00} M. Inami, N. Kawakami, D. Sekiguchi, Y. Ya-nagida, T. Maeda and S. Tachi, Visuo-Haptic Display Using Head-Mounted Projector, Proceedings of IEEE Virtual Reality 2000, pp.233--240, 2000.

Digital Library

[11]

{Joh05} K. Johnsen, R. Dickerson, A. Raij, B. Lok, J. Jackson, M. Shin, J. Hernandez, A. Stevens and D. S. Lind: "Experiences in Using Immersive Virtual Characters to Educate Medical Communication Skills." In Proceedings of IEEE Virtual Reality 2005 (VR 2005).

Digital Library

[12]

{Kit05} I. Kitahara, M. Ito, H. Noma, K. Kogure and N. Hagita, "Thermal-ID: A Personal Identification Method Using Body Temperature." Adjunct Proc. of The 3rd International Conference on Pervasive Computing (Pervasive 2005) Advances in Pervasive Computing, pp. 69--72.

[13]

{Mal02} P. Malbezin, W. Piekarski and B. Thomas: "Measuring ARToolkit accuracy in long-distance tracking experiments." In: 1st Int'l AR Toolkit Workshop. (2002).

[14]

{QR} http://www.denso-wave.com/qrcode/index-e.html

[15]

{Ras01} R. Raskar, G. Welch, K. L. Low and D. Bandyopadhyay: "Shader Lamps: Animating Real Objects with Image Based Illumination," Eurographics Workshop on Rendering, June 2001.

Digital Library

[16]

{Sie04} T. Sielhorst, T. Obst, R. Burgkart, R. Riener and N. Navab. "An augmented reality delivery simulator for medical training." In International Workshop on Augmented Environments for Medical Imaging - MICCAI Satellite Workshop, 2004.

[17]

{Stat96} A. State, M. Livingston, G. Hirota, W. Garrett, M. Whitton, H. Fuchs and E. Pisano "Technologies for Augmented-Reality Systems: realizing Ultrasound-Guided Needle Biopsies." In Proceedings of SIGGRAPH '96 (New Orleans, LA, August 4--9, 1996). In Computer Graphics Proceedings, Annual Conference Series, 1996, ACM SIGGRAPH, 439--446.

Digital Library

[18]

{Tac03} S. Tachi, "Telexistence and Retro-reflective Projection Technology (RPT)," Proceedings of the 5th Virtual Reality International Conference (VRIC2003) pp. 69/1--69/9, Laval Virtual 2003, France, May 13--18, 2003.

[19]

{Tak02} M. Takemura and Y. Ohta, "Diminishing Head-Mounted Display for Shared Mixed Reality," IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR 2002), pp. 149--156. Oct. 2002.

Digital Library

Cited By

Caltanissetta FDreifus GHart AColosimo B(2022)In-situ monitoring of Material Extrusion processes via thermal videoimaging with application to Big Area Additive Manufacturing (BAAM)Additive Manufacturing10.1016/j.addma.2022.10299558(102995)Online publication date: Oct-2022
https://doi.org/10.1016/j.addma.2022.102995
Asao TKobayashi TKotani KSuzuki SObama KSumii ANishigori T(2021)Development of a Drawing Application for Communication Support during Endoscopic Surgery and Its Evaluation Using Steering LawApplied Sciences10.3390/app1110450511:10(4505)Online publication date: 14-May-2021
https://doi.org/10.3390/app11104505
Orlosky JKim PKiyokawa KMashita TRatsamee PUranishi YTakemura H(2017)VisMerge: Light Adaptive Vision Augmentation via Spectral and Temporal Fusion of Non-visible Light2017 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)10.1109/ISMAR.2017.19(22-31)Online publication date: Oct-2017
https://doi.org/10.1109/ISMAR.2017.19
Show More Cited By

Visual support for medical communication by using projector-based augmented reality and thermal markers
1. Computing methodologies
  1. Computer graphics
    1. Graphics systems and interfaces
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction paradigms

Recommendations

Interactive Augmented Live Virtual Reality Streaming: A Health Care Application
ICMHI '18: Proceedings of the 2nd International Conference on Medical and Health Informatics

Virtual Reality (VR) technology has been around for decades; however, we have probably only begun to realize the practical applications until recent years. With increase of computing power, decrease of cost and physical dimension, a new class of ...
Read More
Haptics in Augmented Reality
ICMCS '99: Proceedings of the IEEE International Conference on Multimedia Computing and Systems - Volume 2

An augmented reality system merges synthetic sensory information into a user's perception of a three-dimensional environment. An important performance goal for an augmented reality system is that the user perceives a single seamless environment. In most ...
Read More
Extending Virtual Reality Display Wall Environments Using Augmented Reality
SUI '19: Symposium on Spatial User Interaction

Two major form factors for virtual reality are head-mounted displays and large display environments such as CAVE®and the LCD-based successor CAVE2®. Each of these has distinct advantages and limitations based on how they’re used. This work explores ...
Read More

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

ICAT '05: Proceedings of the 2005 international conference on Augmented tele-existence

December 2005

291 pages

ISBN:0473106574

DOI:10.1145/1152399

Copyright © 2005 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 December 2005

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

ICAT05

ICAT05: The International Conference on Augmented Tele-Existence

December 5 - 8, 2005

Christchurch, New Zealand

Acceptance Rates

ICAT '05 Paper Acceptance Rate 48 of 48 submissions, 100%;

Overall Acceptance Rate 48 of 48 submissions, 100%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
647
Total Downloads

Downloads (Last 12 months)7
Downloads (Last 6 weeks)0

Other Metrics

View Author Metrics

Citations

Cited By

Caltanissetta FDreifus GHart AColosimo B(2022)In-situ monitoring of Material Extrusion processes via thermal videoimaging with application to Big Area Additive Manufacturing (BAAM)Additive Manufacturing10.1016/j.addma.2022.10299558(102995)Online publication date: Oct-2022
https://doi.org/10.1016/j.addma.2022.102995
Asao TKobayashi TKotani KSuzuki SObama KSumii ANishigori T(2021)Development of a Drawing Application for Communication Support during Endoscopic Surgery and Its Evaluation Using Steering LawApplied Sciences10.3390/app1110450511:10(4505)Online publication date: 14-May-2021
https://doi.org/10.3390/app11104505
Orlosky JKim PKiyokawa KMashita TRatsamee PUranishi YTakemura H(2017)VisMerge: Light Adaptive Vision Augmentation via Spectral and Temporal Fusion of Non-visible Light2017 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)10.1109/ISMAR.2017.19(22-31)Online publication date: Oct-2017
https://doi.org/10.1109/ISMAR.2017.19
Cameron Lowell Palmer Bjorn Olav Haugen Tegnander EEik-Nes STorp HKiss G(2015)Mobile 3D augmented-reality system for ultrasound applications2015 IEEE International Ultrasonics Symposium (IUS)10.1109/ULTSYM.2015.0488(1-4)Online publication date: Oct-2015
https://doi.org/10.1109/ULTSYM.2015.0488
Punpongsanon PIwai DSato K(2015)Projection-based visualization of tangential deformation of nonrigid surface by deformation estimation using infrared textureVirtual Reality10.1007/s10055-014-0256-y19:1(45-56)Online publication date: 1-Mar-2015
https://dl.acm.org/doi/10.1007/s10055-014-0256-y
Kiss GPalmer CTorp H(2015)Patient Adapted Augmented Reality System for Real-Time Echocardiographic ApplicationsAugmented Environments for Computer-Assisted Interventions10.1007/978-3-319-24601-7_15(145-154)Online publication date: 10-Dec-2015
https://doi.org/10.1007/978-3-319-24601-7_15
Kiss GStorve SHaugen BTorp H(2014)Augmented reality based tools for echocardiographic acquisitions2014 IEEE International Ultrasonics Symposium10.1109/ULTSYM.2014.0171(695-698)Online publication date: Sep-2014
https://doi.org/10.1109/ULTSYM.2014.0171
Kasahara MTakano KLi K(2014)A Personalized Learning System with an AR Augmented Reality Browser for Ecosystem FieldworkProceedings of the 2014 IEEE 28th International Conference on Advanced Information Networking and Applications10.1109/AINA.2014.16(89-97)Online publication date: 13-May-2014
https://dl.acm.org/doi/10.1109/AINA.2014.16
Ford SViola IBruckner STorp HKiss GManocha DWinslett M(2012)HeartPadProceedings of the Workshop at SIGGRAPH Asia10.1145/2425296.2425326(169-176)Online publication date: 26-Nov-2012
https://dl.acm.org/doi/10.1145/2425296.2425326
Peiris RFernando OCheok A(2011)A dynamic AR marker for a paper based temperature sensorProceedings of the Second international conference on Ambient Intelligence10.1007/978-3-642-25167-2_24(195-199)Online publication date: 16-Nov-2011
https://dl.acm.org/doi/10.1007/978-3-642-25167-2_24

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents