Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
Springer Nature Link
Log in

A General Computationally-Efficient 3D Reconstruction Pipeline for Multiple Images with Point Clouds

  • Conference paper
  • First Online:
Medical Image Computing and Computer Assisted Intervention – MICCAI 2023 Workshops (MICCAI 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14393))

  • 1368 Accesses

Abstract

Histology images are the golden standard for medical diagnostic analysis. However, 2D images can lose some critical information, such as the spatial structure of blood vessels. Therefore, it is necessary to perform 3D reconstruction for the histology images. At the same time, due to the differences between institutions and hospitals, a general 3D reconstruction method is needed. In this work, we propose a 3D reconstruction pipeline that is compatible with Whole Slide Imaging (WSI) and can also be applied to other imaging modalities such as CT images, MRI images, and immunohistochemistry images. Through semantic segmentation, point cloud construction and registration, and 3D rendering, we can reconstruct serialized images into 3D models. By optimizing the pipeline workflow, we can significantly reduce the computation workload required for the 3D reconstruction of high-resolution images and thus save time. In clinical practice, our method helps pathologists triage and evaluate tumor tissues with real-time 3D visualization.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 74.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Kiemen, A.L., Braxton, A.M., Grahn, M.P., et al.: CODA: quantitative 3D reconstruction of large tissues at cellular resolution. Nat. Meth. 19, 1490–1499 (2022). https://doi.org/10.1038/s41592-022-01650-9

    Article  Google Scholar 

  2. Ma, R., Wang, R., Pizer, S., Rosenman, J., McGill, S.K., Frahm, J.-M.: Real-time 3D reconstruction of colonoscopic surfaces for determining missing regions. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11768, pp. 573–582. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32254-0_64

    Chapter  Google Scholar 

  3. Xie, W., et al.: Prostate cancer risk stratification via nondestructive 3D pathology with deep learning-assisted gland analysis. Cancer Res. 82(2), 334–345 (2022). https://doi.org/10.1158/0008-5472.CAN-21-2843

    Article  MathSciNet  Google Scholar 

  4. Chen, C., et al.: Region proposal network with graph prior and IoU-balance loss for landmark detection in 3D ultrasound. In: 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI), pp. 1–5 (2020). https://doi.org/10.1109/ISBI45749.2020.9098368

  5. Wiskin, J., et al.: Full wave 3D inverse scattering transmission ultrasound tomography: breast and whole body imaging. In: 2019 IEEE International Ultrasonics Symposium (IUS), pp. 951–958 (2019). https://doi.org/10.1109/ULTSYM.2019.8925778

  6. Kamencay, P., Zachariasova, M., Hudec, R., Benco, M., Radil, R.: 3D image reconstruction from 2D CT slices. In: 2014 3DTV-Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON), pp. 1–4 (2014). https://doi.org/10.1109/3DTV.2014.6874742

  7. Kermi, A., Djennelbaroud, H.C., Khadir, M.T.: A deep learning-based 3D CNN for automated Covid-19 lung lesions segmentation from 3D chest CT scans. In: 2022 5th International Symposium on Informatics and its Applications (ISIA), pp. 1–5 (2022). https://doi.org/10.1109/ISIA55826.2022.9993505

  8. Ueda, D., et al.: Deep learning for MR angiography: automated detection of cerebral aneurysms. Radiology 290(1), 187–194 (2019). pMID: 30351253. https://doi.org/10.1148/radiol.2018180901

  9. Tang, H., Hsung, T.C., Lam, W.Y., Cheng, L.Y.Y., Pow, E.H.: On 2D–3D image feature detections for image-to-geometry registration in virtual dental model. In: 2020 IEEE International Conference on Visual Communications and Image Processing (VCIP), pp. 140–143 (2020). https://doi.org/10.1109/VCIP49819.2020.9301774

  10. Zhang, L.z., Shen, K.: A volumetric measurement algorithm of defects in 3D CT image based on spatial intuitionistic fuzzy c-means. In: 2021 IEEE Far East NDT New Technology & Application Forum (FENDT), pp. 78–82 (2021). https://doi.org/10.1109/FENDT54151.2021.9749668

  11. Leonardi, V., Vidal, V., Mari, J.L., Daniel, M.: 3D reconstruction from CT-scan volume dataset application to kidney modeling. In: Proceedings of the 27th Spring Conference on Computer Graphics, SCCG 2011, pp. 111–120. Association for Computing Machinery, New York (2011). https://doi.org/10.1145/2461217.2461239

  12. Valanarasu, J.M.J., Oza, P., Hacihaliloglu, I., Patel, V.M.: Medical transformer: gated axial-attention for medical image segmentation. In: de Bruijne, M., Cattin, P.C., Cotin, S., Padoy, N., Speidel, S., Zheng, Y., Essert, C. (eds.) MICCAI 2021. LNCS, vol. 12901, pp. 36–46. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87193-2_4

    Chapter  Google Scholar 

  13. Wang, H., Zhu, Y., Green, B., Adam, H., Yuille, A., Chen, L.-C.: Axial-DeepLab: stand-alone axial-attention for panoptic segmentation. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12349, pp. 108–126. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58548-8_7

    Chapter  Google Scholar 

  14. Bankhead, P., et al.: QuPath: open source software for digital pathology image analysis. Sci. Rep. 7(1), 16878 (2017). https://doi.org/10.1038/s41598-017-17204-5

    Article  Google Scholar 

  15. Chauhan, I., Rawat, A., Chauhan, M., Garg, R.: Fusion of low-cost UAV point cloud with TLS point cloud for complete 3D visualisation of a building. In: 2021 IEEE International India Geoscience and Remote Sensing Symposium (InGARSS), pp. 234–237 (2021). https://doi.org/10.1109/InGARSS51564.2021.9792104

  16. Chen, M., Miao, Y., Gong, Y., Mao, X.: Convolutional neural network powered identification of the location and orientation of human body via human form point cloud. In: 2021 15th European Conference on Antennas and Propagation (EuCAP), pp. 1–5 (2021). https://doi.org/10.23919/EuCAP51087.2021.9410980

  17. Wen, Z., Yan, Y., Cui, H.: Study on segmentation of 3D human body based on point cloud data. In: 2012 Second International Conference on Intelligent System Design and Engineering Application, pp. 657–660 (2012). https://doi.org/10.1109/ISdea.2012.676

  18. Arun, K.S., Huang, T.S., Blostein, S.D.: Least-squares fitting of two 3-D point sets. IEEE Trans. Pattern Anal. Mach. Intell. (PAMI) 9(5), 698–700 (1987). https://doi.org/10.1109/TPAMI.1987.4767965

    Article  Google Scholar 

  19. Zhou, Q.Y., Park, J., Koltun, V.: Open3D: a modern library for 3D data processing. arXiv:1801.09847 (2018)

  20. Alsaid, B., et al.: Coexistence of adrenergic and cholinergic nerves in the inferior hypogastric plexus: anatomical and immunohistochemical study with 3D reconstruction in human male fetus. J. Anat. 214(5), 645–654 (2009). https://doi.org/10.1111/j.1469-7580.2009.01071.x. https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1469-7580.2009.01071.x

  21. Karam, I., Droupy, S., Abd-Alsamad, I., Uhl, J.F., Benoît, G., Delmas, V.: Innervation of the female human urethral sphincter: 3D reconstruction of immunohistochemical studies in the fetus. Eur. Urol. 47(5), 627–634 (2005). https://doi.org/10.1016/j.eururo.2005.01.001. https://www.sciencedirect.com/science/article/pii/S0302283805000060

Download references

Author information

Authors and Affiliations

  1. School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University, Shanghai, China

    Qingyang Wu & Jing Ke

  2. Department of Computer Science, Johns Hopkins University, Baltimore, USA

    Yiqing Shen

  3. School of Computer Science and Engineering, University of New South Wales, Sydney, Australia

    Jing Ke

Authors
  1. Qingyang Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yiqing Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jing Ke
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qingyang Wu .

Editor information

Editors and Affiliations

  1. University of Central Arkansas, Conway, AR, USA

    M. Emre Celebi

  2. Amazon Development Center U.S. Inc., Seattle, WA, USA

    Md Sirajus Salekin

  3. Korea University, Seoul, Korea (Republic of)

    Hyunwoo Kim

  4. University Hospital Bonn, Bonn, Germany

    Shadi Albarqouni

  5. Instituto Superior Técnico, Lisboa, Portugal

    Catarina Barata

  6. Memorial Sloan Kettering Cancer Center, New Yrok, NY, USA

    Allan Halpern

  7. Medical University of Vienna, Vienna, Austria

    Philipp Tschandl

  8. Kenko AI, Barcelona, Spain

    Marc Combalia

  9. Google (United States), Palo Alto, CA, USA

    Yuan Liu

  10. National Institutes of Health, Bethesda, MD, USA

    Ghada Zamzmi

  11. Amazon, USA, Cambridge, MA, USA

    Joshua Levy

  12. Amazon (United States), Fairfax, VI, USA

    Huzefa Rangwala

  13. German Cancer Research Center, Germany, Heidelberg, Germany

    Annika Reinke

  14. Amazon (United States), Baltimore, WA, USA

    Diya Wynn

  15. Vanderbilt University, Brentwood, TN, USA

    Bennett Landman

  16. Korea University, Seoul, Korea (Republic of)

    Won-Ki Jeong

  17. Johns Hopkins University, Baltimore, MD, USA

    Yiqing Shen

  18. University of Surrey, Guildford, UK

    Zhongying Deng

  19. University of Pennsylvania, Philadelphia, PA, USA

    Spyridon Bakas

  20. University of British Columbia, Vancouver, Canada

    Xiaoxiao Li

  21. Imperial College London, London, UK

    Chen Qin

  22. Nvidia, Munich, Germany

    Nicola Rieke

  23. Nvidia Corporation, Bethesda, MD, USA

    Holger Roth

  24. NVIDIA Corporation, Santa Clara, CA, USA

    Daguang Xu

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wu, Q., Shen, Y., Ke, J. (2023). A General Computationally-Efficient 3D Reconstruction Pipeline for Multiple Images with Point Clouds. In: Celebi, M.E., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2023 Workshops . MICCAI 2023. Lecture Notes in Computer Science, vol 14393. Springer, Cham. https://doi.org/10.1007/978-3-031-47401-9_19

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-47401-9_19

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-47400-2

  • Online ISBN: 978-3-031-47401-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation