Abstract
Commercial availability of three-dimensional (3D) augmented reality (AR) devices has increased interest in using this novel technology for visualizing neuroimaging data. Here, a technical workflow and algorithm for importing 3D surface-based segmentations derived from magnetic resonance imaging data into a head-mounted AR device is presented and illustrated on selected examples: the pial cortical surface of the human brain, fMRI BOLD maps, reconstructed white matter tracts, and a brain network of functional connectivity.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Azuma RT: A survey of augmented reality. Presence Teleop Virt 6(4):355–385, 1997
Milgram P, Kishino F: A taxonomy of mixed reality visual displays. IEIC Transactions on Information and Systems, E77D:12, 1321–29,1994
Abe Y, Sato S, Kato K et al.: A novel 3D guidance system using augmented reality for percutaneous vertebroplasty. J Neurosurg Spine 19(4):492–501, 2013
Blackwell M, Morgan F, DiGioia, 3rd AM: Augmented reality and its future in orthopaedics. Clin Orthop Relat Res 354:111–122, 1998
Kerner KF, et al.: Augmented reality for teaching endotracheal intubation: MR imaging to create anatomically correct models. AMIA Annu Symp Proc p. 888,2003
Nicolau S et al.: Augmented reality in laparoscopic surgical oncology. Surg Oncol 20(3):189–201, 2011
Fritz J et al.: Augmented reality visualization using image overlay technology for MR-guided interventions: cadaveric bone biopsy at 1.5 T. Invest Radiol 48(6):464–470, 2013
Volonte F et al.: Augmented reality to the rescue of the minimally invasive surgeon. The usefulness of the interposition of stereoscopic images in the Da Vinci robotic console. Int J Med Robot 9(3):e34–e38, 2013
Markman A et al.: Augmented reality three-dimensional object visualization and recognition with axially distributed sensing. Opt Lett 41(2):297–300, 2016
Chinnock C: Virtual reality in surgery and medicine. Hosp Technol Ser 13(18):1–48, 1994
Ota D et al.: Virtual reality in surgical education. Comput Biol Med 25(2):127–137, 1995
Olofsson J et al.: Advanced 3D-visualization, including virtual reality, distributed by PCs, in brain research, clinical radiology and education. Stud Health Technol Inform 50:357–358, 1998
Webb G et al.: Virtual reality and interactive 3D as effective tools for medical training. Stud Health Technol Inform 94:392–394, 2003
Farber M et al.: Virtual reality simulator for the training of lumbar punctures. Methods Inf Med 48(5):493–501, 2009
Clarke DB et al.: Virtual reality simulator: demonstrated use in neurosurgical oncology. Surg Innov 20(2):190–197, 2013
Mi SH et al.: A 3D virtual reality simulator for training of minimally invasive surgery. Conf Proc IEEE Eng Med Biol Soc 2014:349–352, 2014
Khavari R et al.: Functional magnetic resonance imaging with concurrent urodynamic testing identifies brain structures involved in micturition cycle in patients with multiple sclerosis. J Urol 197:438–444, 2016
Shy M et al.: Functional magnetic resonance imaging during urodynamic testing identifies brain structures initiating micturition. J Urol 192(4):1149–1154, 2014
Bidgood, Jr WD, Horii SC: Introduction to the ACR-NEMA DICOM standard. Radiographics 12(2):345–355, 1992
John NW et al.: MedX3D: standards enabled desktop medical 3D. Stud Health Technol Inform 132:189–194, 2008
Cox RW: AFNI: what a long strange trip it’s been. Neuroimage 62(2):743–747, 2012
Larobina M, Murino L: Medical image file formats. J Digit Imaging 27(2):200–206, 2014
Schneider CA, Rasband WS, Eliceiri KW: NIH image to ImageJ: 25 years of image analysis. Nat Methods 9(7):671–675, 2012
Xie S et al.: DiffusionKit: a light one-stop solution for diffusion MRI data analysis. J Neurosci Methods 273:107–119, 2016
Karmonik C et al.: Music listening modulates functional connectivity and information flow in the human brain. Brain Connect, 2016. doi: 10.1089/brain.2016.0428
Berlage T: Augmented-reality communication for diagnostic tasks in cardiology. IEEE Trans Inf Technol Biomed 2(3):169–173, 1998
Sato Y et al.: Image guidance of breast cancer surgery using 3-D ultrasound images and augmented reality visualization. IEEE Trans Med Imaging 17(5):681–693, 1998
Kawamata T et al.: Endoscopic augmented reality navigation system for endonasal transsphenoidal surgery to treat pituitary tumors: technical note. Neurosurgery 50(6):1393–1397, 2002
Paul P, Fleig O, Jannin P: Augmented virtuality based on stereoscopic reconstruction in multimodal image-guided neurosurgery: methods and performance evaluation. IEEE Trans Med Imaging 24(11):1500–1511, 2005
Lukosch S, Billinghurst M, Alem L et al.: The effect of view independence in a collaborative AR system. Computer supported cooperative work. J Collab Comput 24(6):563–589, 2015
Lukosch S, Billinghurst M, Alem L et al.: Collaboration in augmented reality. Computer supported cooperative work. J Collab Comput 24(6):515–525, 2015
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
(MP4 81.0 mb)
(MP4 87.9 mb)
Rights and permissions
About this article
Cite this article
Karmonik, C., Boone, T.B. & Khavari, R. Workflow for Visualization of Neuroimaging Data with an Augmented Reality Device. J Digit Imaging 31, 26–31 (2018). https://doi.org/10.1007/s10278-017-9991-4
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10278-017-9991-4