Abstract
In this work we address the problem of landmark-based segmentation for anatomical structures. We propose HybridGNet, an encoder-decoder neural architecture which combines standard convolutions for image feature encoding, with graph convolutional neural networks to decode plausible representations of anatomical structures. We benchmark the proposed architecture considering other standard landmark and pixel-based models for anatomical segmentation in chest x-ray images, and found that HybridGNet is more robust to image occlusions. We also show that it can be used to construct landmark-based segmentations from pixel level annotations. Our experimental results suggest that Hybrid-Net produces accurate and anatomically plausible landmark-based segmentations, by naturally incorporating shape constraints within the decoding process via spectral convolutions.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Alvén, J., Kahl, F., Landgren, M., Larsson, V., Ulén, J.: Shape-aware multi-atlas segmentation. In: 2016 23rd International Conference on Pattern Recognition (Icpr), pp. 1101–1106. IEEE (2016)
Alvén, J., Kahl, F., Landgren, M., Larsson, V., Ulén, J., Enqvist, O.: Shape-aware label fusion for multi-atlas frameworks. Pattern Recogn. Lett. 124, 109–117 (2019)
Bhalodia, R., Elhabian, S.Y., Kavan, L., Whitaker, R.T.: DeepSSM: a deep learning framework for statistical shape modeling from raw images. In: Reuter, M., Wachinger, C., Lombaert, H., Paniagua, B., Lüthi, M., Egger, B. (eds.) ShapeMI 2018. LNCS, vol. 11167, pp. 244–257. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-04747-4_23
Boussaid, H., Kokkinos, I., Paragios, N.: Discriminative learning of deformable contour models. In: 2014 IEEE 11th International Symposium on Biomedical Imaging (ISBI), pp. 624–628. IEEE (2014)
Bronstein, M.M., Bruna, J., LeCun, Y., Szlam, A., Vandergheynst, P.: Geometric deep learning: going beyond euclidean data. IEEE Signal Process. Mag. 34(4), 18–42 (2017)
Bruna, J., Zaremba, W., Szlam, A., LeCun, Y.: Spectral networks and locally connected networks on graphs. arXiv preprint arXiv:1312.6203 (2013)
Cootes, T.F., Edwards, G.J., Taylor, C.J.: Active appearance models. In: Burkhardt, H., Neumann, B. (eds.) ECCV 1998. LNCS, vol. 1407, pp. 484–498. Springer, Heidelberg (1998). https://doi.org/10.1007/BFb0054760
Cootes, T.F., Taylor, C.J., Cooper, D.H., Graham, J.: Training models of shape from sets of examples. In: BMVC92, pp. 9–18. Springer, Heidelberg (1992). https://doi.org/10.1007/978-1-4471-3201-1_2
Defferrard, M., Bresson, X., Vandergheynst, P.: Convolutional neural networks on graphs with fast localized spectral filtering. arXiv preprint arXiv:1606.09375 (2016)
Foti, S., Foti, S., et al.: Intraoperative liver surface completion with graph convolutional VAE. In: Sudre, C.H., et al. (eds.) UNSURE/GRAIL -2020. LNCS, vol. 12443, pp. 198–207. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-60365-6_19
Frangi, A.F., Niessen, W.J., Rueckert, D., Schnabel, J.A.: Automatic 3D ASM construction via atlas-based landmarking and volumetric elastic registration. In: Insana, M.F., Leahy, R.M. (eds.) IPMI 2001. LNCS, vol. 2082, pp. 78–91. Springer, Heidelberg (2001). https://doi.org/10.1007/3-540-45729-1_7
Gilmer, J., Schoenholz, S.S., Riley, P.F., Vinyals, O., Dahl, G.E.: Neural message passing for quantum chemistry. In: International Conference on Machine Learning, pp. 1263–1272. PMLR (2017)
van Ginneken, B., Stegmann, M., Loog, M.: Segmentation of anatomical structures in chest radiographs using supervised methods: a comparative study on a public database. Med. Image Anal. 10(1), 19–40 (2006)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition (2015)
Heimann, T., Meinzer, H.P.: Statistical shape models for 3d medical image segmentation: a review. Med. Image Anal. 13(4), 543–563 (2009)
Heitz, G., Rohlfing, T., Maurer Jr, C.R.: Automatic generation of shape models using nonrigid registration with a single segmented template mesh. In: VMV, pp. 73–80 (2004)
Jurdia, R.E., Petitjean, C., Honeine, P., Cheplygina, V., Abdallah, F.: High-level prior-based loss functions for medical image segmentation: A survey. arXiv preprint arXiv:2011.08018 (2020)
Kingma, D.P., Welling, M.: Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 (2013)
Kipf, T.N., Welling, M.: Variational graph auto-encoders. arXiv preprint arXiv:1611.07308 (2016)
Larrazabal, A.J., Martinez, C., Ferrante, E.: Anatomical priors for image segmentation via post-processing with denoising autoencoders. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11769, pp. 585–593. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32226-7_65
Marstal, K., Berendsen, F., Staring, M., Klein, S.: Simpleelastix: a user-friendly, multi-lingual library for medical image registration. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 134–142 (2016)
Milletari, F., Navab, N., Ahmadi, S.A.: V-net: fully convolutional neural networks for volumetric medical image segmentation. In: 2016 Fourth International Conference on 3D Vision (3DV), pp. 565–571. IEEE (2016)
Milletari, F., Rothberg, A., Jia, J., Sofka, M.: Integrating statistical prior knowledge into convolutional neural networks. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10433, pp. 161–168. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66182-7_19
Oktay, O., et al.: Anatomically constrained neural networks (acnns): application to cardiac image enhancement and segmentation. IEEE Trans. Med. Imaging 37(2), 384–395 (2017)
Paulsen, R., Larsen, R., Nielsen, C., Laugesen, S., Ersbøll, B.: Building and testing a statistical shape model of the human ear canal. In: Dohi, T., Kikinis, R. (eds.) MICCAI 2002. LNCS, vol. 2489, pp. 373–380. Springer, Heidelberg (2002). https://doi.org/10.1007/3-540-45787-9_47
Ranjan, A., Bolkart, T., Sanyal, S., Black, M.J.: Generating 3d faces using convolutional mesh autoencoders. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 704–720 (2018)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Shakeri, M., et al.: Sub-cortical brain structure segmentation using f-cnn’s. In: 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI), pp. 269–272. IEEE (2016)
l Shiraishi, J., et al.: Development of a digital image database for chest radiographs with and without a lung nodule: receiver operating characteristic analysis of radiologists’ detection of pulmonary nodules. Am. J. Roentgenol. 174(1), 71–74 (2000)
Shuman, D.I., Narang, S.K., Frossard, P., Ortega, A., Vandergheynst, P.: The emerging field of signal processing on graphs: extending high-dimensional data analysis to networks and other irregular domains. IEEE Signal Process. Mag. 30(3), 83–98 (2013)
Sozou, P.D., Cootes, T.F., Taylor, C.J., Di Mauro, E., Lanitis, A.: Non-linear point distribution modelling using a multi-layer perceptron. Image Vision Comput. 15(6), 457–463 (1997)
Zitova, B., Flusser, J.: Image registration methods: a survey. Image Vision Comput. 21(11), 977–1000 (2003)
Acknowledgment
This work was supported by grants from ANPCyT (PICT 2018-3907 and 3384), UNL (CAI+D 50220140100-084LI, 50620190100-145LI and 115LI) and The Royal Society (IES/R2/202165). We gratefully acknowledge the support of NVIDIA Corporation with the donation of the Titan Xp used for this research.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Gaggion, N., Mansilla, L., Milone, D.H., Ferrante, E. (2021). Hybrid Graph Convolutional Neural Networks for Landmark-Based Anatomical Segmentation. In: de Bruijne, M., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2021. MICCAI 2021. Lecture Notes in Computer Science(), vol 12901. Springer, Cham. https://doi.org/10.1007/978-3-030-87193-2_57
Download citation
DOI: https://doi.org/10.1007/978-3-030-87193-2_57
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87192-5
Online ISBN: 978-3-030-87193-2
eBook Packages: Computer ScienceComputer Science (R0)
