Abstract
Localizing oneself during endoscopic procedures can be problematic due to the lack of distinguishable textures and landmarks, as well as difficulties due to the endoscopic device such as a limited field of view and challenging lighting conditions. Expert knowledge shaped by years of experience is required for localization within the human body during endoscopic procedures. In this work, we present a deep learning method based on anatomy recognition, that constructs a surgical path in an unsupervised manner from surgical videos, modelling relative location and variations due to different viewing angles. At inference time, the model can map unseen video frames on the path and estimate the viewing angle, aiming to provide guidance, for instance, to reach a particular destination. We test the method on a dataset consisting of surgical videos of pituitary surgery, i.e. transsphenoidal adenomectomy, as well as on a synthetic dataset. An online tool that lets researchers upload their surgical videos to obtain anatomy detections and the weights of the trained YOLOv7 model are available at: https://surgicalvision.bmic.ethz.ch.
This work funded by the SNSF (Project IZKSZ3_218786).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Härtl, R., Lam, K.S., Wang, J., Korge, A., Kandziora, F., Audigé, L.: Worldwide survey on the use of navigation in spine surgery. World neurosurgery 79(1), 162–172 (1 2013)
Orringer, D.A., Golby, A., Jolesz, F.: Neuronavigation in the surgical management of brain tumors: current and future trends. Expert review of medical devices 9(5), 491–500 (9 2012)
Iversen, D.H., Wein, W., Lindseth, F., Unsgård, G., Reinertsen, I.: Automatic Intraoperative Correction of Brain Shift for Accurate Neuronavigation. World neurosurgery 120, e1071–e1078 (12 2018)
Berkmann, S., Schlaffer, S., Nimsky, C., Fahlbusch, R., Buchfelder, M.: Intraoperative high-field MRI for transsphenoidal reoperations of nonfunctioning pituitary adenoma. Journal of neurosurgery 121(5), 1166–1175 (11 2014)
Staartjes, V.E., Volokitin, A., Regli, L., Konukoglu, E., Serra, C.: Machine Vision for Real-Time Intraoperative Anatomic Guidance: A Proof-of-Concept Study in Endoscopic Pituitary Surgery. Operative neurosurgery (Hagerstown, Md.) 21(4), 242–247 (10 2021)
Burkhardt, J.K., Serra, C., Neidert, M.C., Woernle, C.M., Fierstra, J., Regli, L., Bozinov, O.: High-frequency intra-operative ultrasound-guided surgery of superficial intra-cerebral lesions via a single-burr-hole approach. Ultrasound in medicine & biology 40(7), 1469–1475 (2014)
Ulrich, N.H., Burkhardt, J.K., Serra, C., Bernays, R.L., Bozinov, O.: Resection of pediatric intracerebral tumors with the aid of intraoperative real-time 3-D ultrasound. Child’s nervous system : ChNS : official journal of the International Society for Pediatric Neurosurgery 28(1), 101–109 (1 2012)
Stummer, W., Stepp, H., Wiestler, O.D., Pichlmeier, U.: Randomized, Prospective Double-Blinded Study Comparing 3 Different Doses of 5-Aminolevulinic Acid for Fluorescence-Guided Resections of Malignant Gliomas. Neurosurgery 81(2), 230–239 (8 2017)
Hadjipanayis, C.G., Widhalm, G., Stummer, W.: What is the Surgical Benefit of Utilizing 5-Aminolevulinic Acid for Fluorescence-Guided Surgery of Malignant Gliomas? Neurosurgery 77(5), 663–673 (8 2015)
Hervey-Jumper, S.L., Li, J., Lau, D., Molinaro, A.M., Perry, D.W., Meng, L., Berger, M.S.: Awake craniotomy to maximize glioma resection: methods and technical nuances over a 27-year period. Journal of neurosurgery 123(2), 325–339 (8 2015)
De Witt Hamer, P.C., Robles, S.G., Zwinderman, A.H., Duffau, H., Berger, M.S.: Impact of intraoperative stimulation brain mapping on glioma surgery outcome: a meta-analysis. Journal of clinical oncology : official journal of the American Society of Clinical Oncology 30(20), 2559–2565 (7 2012)
Sanai, N., Mirzadeh, Z., Berger, M.S.: Functional outcome after language mapping for glioma resection. The New England journal of medicine 358(1), 18–27 (1 2008)
Staartjes, V.E., Stumpo, V., Kernbach, J.M., Klukowska, A.M., Gadjradj, P.S., Schröder, M.L., Veeravagu, A., Stienen, M.N., van Niftrik, C.H., Serra, C., Regli, L.: Machine learning in neurosurgery: a global survey. Acta Neurochirurgica 162(12), 3081–3091 (12 2020)
Grasa, O.G., Civera, J., Montiel, J.M.: EKF monocular SLAM with relocalization for laparoscopic sequences. Proceedings - IEEE International Conference on Robotics and Automation pp. 4816–4821 (2011)
Ozyoruk, K.B., Gokceler, G.I., Bobrow, T.L., Coskun, G., Incetan, K., Almalioglu, Y., Mahmood, F., Curto, E., Perdigoto, L., Oliveira, M., Sahin, H., Araujo, H., Alexandrino, H., Durr, N.J., Gilbert, H.B., Turan, M.: EndoSLAM dataset and an unsupervised monocular visual odometry and depth estimation approach for endoscopic videos. Medical Image Analysis 71, 102058 (7 2021)
Mahmoud, N., Cirauqui, I., Hostettler, A., Doignon, C., Soler, L., Marescaux, J., Montiel, J.M.: ORBSLAM-based Endoscope Tracking and 3D Reconstruction. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 10170 LNCS, 72–83 (8 2016)
Leonard, S., Sinha, A., Reiter, A., Ishii, M., Gallia, G.L., Taylor, R.H., Hager, G.D.: Evaluation and Stability Analysis of Video-Based Navigation System for Functional Endoscopic Sinus Surgery on In Vivo Clinical Data. IEEE Transactions on Medical Imaging 37(10), 2185–2195 (10 2018)
Sarwin, G., Carretta, A., Staartjes, V., Zoli, M., Mazzatenta, D., Regli, L., Serra, C., Konukoglu, E.: Live image-based neurosurgical guidance and roadmap generation using unsupervised embedding. Information Processing in Medical Imaging. IPMI 2023. Lecture Notes in Computer Science, 13939, 107–118 (06 2023)
Das, A., Khan, D.Z., Williams, S.C., Hanrahan, J.G., Borg, A., Dorward, N.L., Bano, S., Marcus, H.J. and Stoyanov, D.: A Multi-task Network for Anatomy Identification in Endoscopic Pituitary Surgery. Medical Image Computing and Computer Assisted Intervention - MICCAI 2023. MICCAI 2023. Lecture Notes in Computer Science, 14228, 472–482 (10 2023)
Wang, C.Y., Bochkovskiy, A., Liao, H.Y.M.: YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors (7 2022)
Community, B. O. (2018). Blender - a 3D modelling and rendering package. Stichting Blender Foundation, Amsterdam. Retrieved from http://www.blender.org
Loshchilov, I., Hutter, F.: Decoupled Weight Decay Regularization. 7th International Conference on Learning Representations, ICLR 2019 (11 2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Ethics declarations
Disclosure of Interests
The authors have no competing interests to declare that are relevant to the content of this article.
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Sarwin, G. et al. (2024). Vision-Based Neurosurgical Guidance: Unsupervised Localization and Camera-Pose Prediction. In: Linguraru, M.G., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2024. MICCAI 2024. Lecture Notes in Computer Science, vol 15006. Springer, Cham. https://doi.org/10.1007/978-3-031-72089-5_69
Download citation
DOI: https://doi.org/10.1007/978-3-031-72089-5_69
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-72088-8
Online ISBN: 978-3-031-72089-5
eBook Packages: Computer ScienceComputer Science (R0)
