Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
SpringerLink
Log in

Real-time deformable SLAM with geometrically adapted template for dynamic monocular laparoscopic scenes

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

Intraoperative reconstruction of endoscopic scenes is a key technology for surgical navigation systems. The accuracy and efficiency of 3D reconstruction directly determine the effectiveness of navigation systems in a variety of clinical applications. While current deformable SLAM algorithms can meet real-time requirements, their underlying reliance on regular templates still makes it challenging to efficiently capture abrupt geometric features within scenes, such as organ contours and surgical margins.

Methods

We propose a novel real-time monocular deformable SLAM algorithm with geometrically adapted template. To ensure real-time performance, the proposed algorithm consists of two threads: a deformation mapping thread updates the template at keyframe rate and a deformation tracking thread estimates the camera pose and the deformation at frame rate. To capture geometric features more efficiently, the algorithm first detects salient edge features using a pre-trained contour detection network and then constructs the template through a triangulation method with guidance of the salient features.

Results

We thoroughly evaluated this method on Mandala and Hamlyn datasets in terms of accuracy and performance. The results demonstrated that the proposed method achieves better accuracy with 0.75–7.95% improvement and achieves consistent effectiveness in data association compared with the closest method.

Conclusion

This study verified an adaptive template does improve the performance of reconstruction of dynamic laparoscopic Scenes with abrupt geometric features. However, further exploration is needed for applications in laparoscopic surgery with incisal margins caused by surgical instruments. This research serves as a crucial step toward enhanced automatic computer-assisted navigation in laparoscopic surgery. Code is available at https://github.com/Tang257/SLAM-with-geometrically-adapted-template.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Acidi B, Ghallab M, Cotin S, Vibert E, Golse N (2023) Augmented reality in liver surgery, where we stand in 2023. J Visc Surg. https://doi.org/10.1016/j.jviscsurg.2023.01.008

    Article  PubMed  Google Scholar 

  2. Arbelaez P, Maire M, Fowlkes C, Malik J (2010) Contour detection and hierarchical image segmentation. IEEE Trans Pattern Anal Mach Intell 33(5):898–916. https://doi.org/10.1109/TPAMI.2010.161

    Article  Google Scholar 

  3. Chew LP (1987) Constrained delaunay triangulations. In: Proceedings of the third annual symposium on Computational geometry, pp 215–222. https://doi.org/10.1145/41958.41981

  4. Chhatkuli A, Pizarro D, Bartoli A, Collins T (2016) A stable analytical framework for isometric shape-from-template by surface integration. IEEE Trans Pattern Anal Mach Intell 39(5):833–850

    Article  PubMed  Google Scholar 

  5. Ferguson Z, Schneider T, Kaufman D, Panozzo D (2023) In-timestep remeshing for contacting elastodynamics. ACM Trans Graph (TOG) 42(4):1–15. https://doi.org/10.1145/3592428

    Article  Google Scholar 

  6. Gómez-Rodríguez JJ, Lamarca J, Morlana J, Tardós JD, Montiel JM (2021) Sd-defslam: semi-direct monocular slam for deformable and intracorporeal scenes. In: IEEE int conf robot autom. IEEE, pp 5170–5177. https://doi.org/10.1109/ICRA48506.2021.9561512

  7. Innmann M, Zollhöfer M, Nießner M, Theobalt C, Stamminger M (2016) Volumedeform: real-time volumetric non-rigid reconstruction. In: Comput vis ECCV. Springer, pp 362–379. https://doi.org/10.1007/978-3-319-46484-8_22

  8. Lamarca J, Montiel JMM (2018) Camera tracking for slam in deformable maps. In: Proceedings of the European conference on computer vision (ECCV) workshops

  9. Lamarca J, Parashar S, Bartoli A, Montiel J (2020) Defslam: tracking and mapping of deforming scenes from monocular sequences. IEEE Trans Robot 37(1):291–303. https://doi.org/10.1109/TRO.2020.3020739

    Article  Google Scholar 

  10. Lin B, Johnson A, Qian X, Sanchez J, Sun Y (2013) Simultaneous tracking, 3d reconstruction and deforming point detection for stereoscope guided surgery. In: International workshop on medical imaging and virtual reality. Springer, pp 35–44. https://doi.org/10.1007/978-3-642-40843-4_5

  11. Lloyd S (1982) Least squares quantization in PCM. IEEE Trans Inf Theory 28(2):129–137. https://doi.org/10.1109/TIT.1982.1056489

    Article  Google Scholar 

  12. Ma J, Ren X, Tsviatkou VY, Kanapelka VK (2022) A novel fully parallel skeletonization algorithm. Pattern Anal Appl. https://doi.org/10.1007/s10044-021-01039-y

  13. Marr D (1982) Vision: a computational investigation into the human representation and processing of visual information. Freeman and Company, New York

    Google Scholar 

  14. Mountney P, Stoyanov D, Yang GZ (2010) Three-dimensional tissue deformation recovery and tracking. IEEE Signal Process Mag 27(4):14–24. https://doi.org/10.1109/MSP.2010.936728

  15. Parashar S, Pizarro D, Bartoli A (2017) Isometric non-rigid shape-from-motion with Riemannian geometry solved in linear time. IEEE Trans Pattern Anal Mach Intell 40(10):2442–2454. https://doi.org/10.1109/TPAMI.2017.2760301

  16. Pu M, Huang Y, Guan Q, Ling H (2021) Rindnet: edge detection for discontinuity in reflectance, illumination, normal and depth. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 6879–6888

  17. Slavcheva M, Baust M, Ilic S (2020) Variational level set evolution for non-rigid 3d reconstruction from a single depth camera. IEEE Trans Pattern Anal Mach Intell 43(8):2838–2850. https://doi.org/10.1109/TPAMI.2020.2976065

  18. Song J, Wang J, Zhao L, Huang S, Dissanayake G (2018) Mis-slam: real-time large-scale dense deformable slam system in minimal invasive surgery based on heterogeneous computing. IEEE Robot Autom Lett 3(4):4068–4075. https://doi.org/10.1109/LRA.2018.2856519

    Article  Google Scholar 

  19. Stoyanov D, Mylonas GP, Deligianni F, Darzi A, Yang GZ (2005) Soft-tissue motion tracking and structure estimation for robotic assisted mis procedures. In: Med image comput comput assist interv. Springer, pp 139–146. https://doi.org/10.1007/11566489

  20. Tan W, Liu H, Dong Z, Zhang G, Bao H (2013) Robust monocular slam in dynamic environments. In: 2013 IEEE international symposium on mixed and augmented reality (ISMAR). IEEE, pp 209–218. https://doi.org/10.1109/ISMAR.2013.6671781

  21. Zhang P, Luo H, Zhu W, Yang J, Zeng N, Fan Y, Wen S, Xiang N, Jia F, Fang C (2020) Real-time navigation for laparoscopic hepatectomy using image fusion of preoperative 3d surgical plan and intraoperative indocyanine green fluorescence imaging. Surg Endosc 34:3449–3459. https://doi.org/10.1007/s00464-019-07121-1

    Article  PubMed  Google Scholar 

Download references

Funding

This work was supported by National Key R &D Program of China (Grant No. 2023YFB4705700), NSFC Key Project U23A20391, NSFC General Project (62072452, 82272132), Shenzhen Science and Technology Program (JCYJ20200109115627045,JCYJ20220818101401003,JCYJ20200109114244249), the Regional Joint Fund of Guangdong under Grant 2021B1515120011, and the Regional Joint Fund of Guangdong (Guangdong-Hong Kong-Macao Research Team Project) under Grant 2021B1515130003.

Author information

Authors and Affiliations

  1. Department of Computer Science, Sichuan University, Chengdu, 610065, China

    Xuanshuang Tang & Ziliang Feng

  2. Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Nanshan, Shenzhen, 518055, China

    Xuanshuang Tang, Yinling Qian & Qiong Wang

  3. Guangdong Provincial Clinical and Engineering Center of Digital Medicine, Guangzhou, 510280, China

    Haisu Tao & Jian Yang

  4. Department of Hepatobiliary Surgery, Zhujiang Hospital, Southern Medical University, Guangzhou, 510280, China

    Haisu Tao & Jian Yang

Authors
  1. Xuanshuang Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Haisu Tao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yinling Qian
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jian Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ziliang Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Qiong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Yinling Qian or Qiong Wang.

Ethics declarations

Conflict of interest

The authors declare that they have no Conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tang, X., Tao, H., Qian, Y. et al. Real-time deformable SLAM with geometrically adapted template for dynamic monocular laparoscopic scenes. Int J CARS (2024). https://doi.org/10.1007/s11548-024-03174-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11548-024-03174-3

Keywords

Search

Navigation