research-article

Weakly-supervised Cerebrovascular Segmentation Network with Shape Prior and Model Indicator

Authors:

Xiaodong YueAuthors Info & Claims

ICMR '22: Proceedings of the 2022 International Conference on Multimedia Retrieval

Pages 668 - 676

https://doi.org/10.1145/3512527.3531377

Published: 27 June 2022 Publication History

Abstract

Labeling cerebral vessels requires domain knowledge in neurology and could be extremely laborious, and there is a scarcity of public annotated cerebrovascular datasets. Traditional machine learning or statistical models could yield decent results on thick vessels with high contrast while having poor performance on those regions of low contrast. In our work, we employ a statistic model as noisy labels and propose a Transformer-based architecture which utilizes Hessian shape prior as soft supervision. It enhances the learning ability of the network to tubular structures, so that the model can make more accurate predictions on refined cerebrovascular segmentation. Furthermore, to combat the overfitting towards noisy labels as model training, we introduce an effective label extension strategy that only calls for a few manual strokes on one sample. These supplementary labels are not used for supervision but only as an indicator to tell where the model keeps the most generalization capability, so as to further guide the model selection in validation. Our experiments are carried out on a public TOF-MRA dataset from MIDAS data platform, and the results demonstrate that our method shows superior performance on cerebrovascular segmentation which achieves Dice of 0.831±0.040 in the dataset.

Supplementary Material

MP4 File (ICMR22-SS258.mp4)

The presentation Video of "Weakly Supervised Cerebrovascular Segmentation Network with Shape Prior and Model Indicator" for ICMR22 special session--Weakly Supervised Learning for Medical Image Analysis

Download
226.99 MB

References

[1]

Luca Antiga. 2007. Generalizing vesselness with respect to dimensionality and shape. The Insight Journal 3 (2007), 1--14.

[2]

Devansh Arpit, Stanislaw Jastrzebski, Nicolas Ballas, David Krueger, Emmanuel Bengio, Maxinder S Kanwal, Tegan Maharaj, Asja Fischer, Aaron Courville, Yoshua Bengio, et al. 2017. A closer look at memorization in deep networks. In International Conference on Machine Learning. PMLR, 233--242.

[3]

Yutong Bai, Jieru Mei, Alan L Yuille, and Cihang Xie. 2021. Are Transformers more robust than CNNs? Advances in Neural Information Processing Systems 34 (2021).

[4]

Elizabeth Bullitt, Donglin Zeng, Guido Gerig, Stephen Aylward, Sarang Joshi, J Keith Smith, Weili Lin, and Matthew G Ewend. 2005. Vessel tortuosity and brain tumor malignancy: a blinded study1. Academic radiology 12, 10 (2005), 1232--1240.

[5]

Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. 2020. End-to-end object detection with transformers. In European Conference on Computer Vision. Springer, 213--229.

Digital Library

[6]

Suheyla Cetin and Gozde Unal. 2015. A higher-order tensor vessel tractography for segmentation of vascular structures. IEEE transactions on medical imaging 34, 10 (2015), 2172--2185.

[7]

Jieneng Chen, Yongyi Lu, Qihang Yu, Xiangde Luo, Ehsan Adeli, Yan Wang, Le Lu, Alan L Yuille, and Yuyin Zhou. 2021. Transunet: Transformers make strong encoders for medical image segmentation. arXiv preprint arXiv:2102.04306 (2021).

[8]

Yufei Chen, Xiaodong Yue, Richard Yi Da Xu, and Hamido Fujita. 2017. Region scalable active contour model with global constraint. Knowledge-Based Systems 120 (2017), 57--73.

Digital Library

[9]

Özgün Çiçek, Ahmed Abdulkadir, Soeren S Lienkamp, Thomas Brox, and Olaf Ronneberger. 2016. 3D U-Net: learning dense volumetric segmentation from sparse annotation. In International conference on medical image computing and computer-assisted intervention. Springer, 424--432.

[10]

GBD 2019 Stroke Collaborators et al . 2021. Global, regional, and national burden of stroke and its risk factors, 1990--2019: a systematic analysis for the Global Burden of Disease Study 2019. The Lancet. Neurology 20, 10 (2021), 795.

[11]

Vien Ngoc Dang, Francesco Galati, Rosa Cortese, Giuseppe Di Giacomo, Viola Marconetto, Prateek Mathur, Karim Lekadir, Marco Lorenzi, Ferran Prados, and Maria A Zuluaga. 2022. Vessel-CAPTCHA: an efficient learning framework for vessel annotation and segmentation. Medical Image Analysis 75 (2022), 102263.

[12]

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. 2021. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. ICLR (2021).

[13]

Shengyu Fan, Yueyan Bian, Hao Chen, Yan Kang, Qi Yang, and Tao Tan. 2020. Unsupervised cerebrovascular segmentation of TOF-MRA images based on deep neural network and hidden Markov random field model. Frontiers in neuroinformatics 13 (2020), 77.

[14]

Alejandro F Frangi, Wiro J Niessen, Koen L Vincken, and Max A Viergever. 1998. Multiscale vessel enhancement filtering. In International conference on medical image computing and computer-assisted intervention. Springer, 130--137.

[15]

Xin Gao, Yoshikazu Uchiyama, Xiangrong Zhou, Takeshi Hara, Takahiko Asano, and Hiroshi Fujita. 2011. A fast and fully automatic method for cerebrovascular segmentation on time-of-flight (TOF) MRA image. Journal of digital imaging 24, 4 (2011), 609--625.

[16]

George D Giannoglou, Yiannis S Chatzizisis, Vassilis Koutkias, Ioannis Kompatsiaris, Maria Papadogiorgaki, Vasileios Mezaris, Eirini Parissi, Panagiotis Diamantopoulos, Michael G Strintzis, Nicos Maglaveras, et al. 2007. A novel active contour model for fully automated segmentation of intravascular ultrasound images: in vivo validation in human coronary arteries. Computers in Biology and Medicine 37, 9 (2007), 1292--1302.

Digital Library

[17]

Bo Han, Quanming Yao, Xingrui Yu, Gang Niu, Miao Xu, Weihua Hu, Ivor Tsang, and Masashi Sugiyama. 2018. Co-teaching: Robust training of deep neural networks with extremely noisy labels. Advances in neural information processing systems 31 (2018).

[18]

Kai Han, Yunhe Wang, Hanting Chen, Xinghao Chen, Jianyuan Guo, Zhenhua Liu, Yehui Tang, An Xiao, Chunjing Xu, Yixing Xu, Zhaohui Yang, Yiman Zhang, and Dacheng Tao. 2022. A Survey on Vision Transformer. IEEE Transactions on Pattern Analysis and Machine Intelligence (2022), 1--1. https://doi.org/10.1109/TPAMI.2022.3152247

[19]

M Sabry Hassouna, Aly A Farag, Stephen Hushek, and Thomas Moriarty. 2006. Cerebrovascular segmentation from TOF using stochastic models. Medical image analysis 10, 1 (2006), 2--18.

[20]

Ali Hatamizadeh, Yucheng Tang, Vishwesh Nath, Dong Yang, Andriy Myronenko, Bennett Landman, Holger R Roth, and Daguang Xu. 2022. Unetr: Transformers for 3d medical image segmentation. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 574--584.

[21]

Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. 2017. Mask r-cnn. In Proceedings of the IEEE international conference on computer vision. 2961--2969.

[22]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770--778.

[23]

Adam Hilbert, Vince I Madai, Ela M Akay, Orhun U Aydin, Jonas Behland, Jan Sobesky, Ivana Galinovic, Ahmed A Khalil, Abdel A Taha, Jens Wuerfel, et al. 2020. BRAVE-NET: fully automated arterial brain vessel segmentation in patients with cerebrovascular disease. Frontiers in artificial intelligence 3 (2020), 78.

[24]

Kai Hu, Guang-Yu Tang, Da-Peng Xiong, and Quan Qiu. 2012. A novel image segmentation algorithm based on Hidden Markov Random Field model and Finite Mixture Model parameter estimation. In 2012 International Conference on Wavelet Analysis and Pattern Recognition. IEEE, 1--6.

[25]

Kai Hu, Zhenzhen Zhang, Xiaorui Niu, Yuan Zhang, Chunhong Cao, Fen Xiao, and Xieping Gao. 2018. Retinal vessel segmentation of color fundus images using multiscale convolutional neural network with an improved cross-entropy loss function. Neurocomputing 309 (2018), 179--191.

Digital Library

[26]

Fabian Isensee, Marianne Schell, Irada Pflueger, Gianluca Brugnara, David Bonekamp, Ulf Neuberger, Antje Wick, Heinz-Peter Schlemmer, Sabine Heiland, Wolfgang Wick, et al. 2019. Automated brain extraction of multisequence MRI using artificial neural networks. Human brain mapping 40, 17 (2019), 4952--4964.

[27]

Diederik P. Kingma and Jimmy Lei Ba. 2015. Adam: A method for stochastic optimization. 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings. Computationally efficient; Convergence properties; Gradient-based optimization; Lower order moments; Objective functions; Online convex optimizations; Stochastic optimization methods; Stochastic optimizations;.

[28]

Artur Klepaczko, Piotr Szczypi'ski, Andreas Deistung, Jürgen R Reichenbach, and Andrzej Materka. 2016. Simulation of MR angiography imaging for validation of cerebral arteries segmentation algorithms. Computer methods and programs in biomedicine 137 (2016), 293--309.

[29]

Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems 25 (2012), 1097--1105.

[30]

Na Li, Shoujun Zhou, Zonghan Wu, Baochang Zhang, and Gang Zhao. 2020. Statistical modeling and knowledge-based segmentation of cerebral artery based on TOF-MRA and MR-T1. Computer methods and programs in biomedicine 186 (2020), 105110.

[31]

Amy Lin, Sapna Rawal, Ronit Agid, and Daniel M Mandell. 2018. Cerebrovascular imaging: which test is best? Neurosurgery 83, 1 (2018), 5--18.

[32]

Fanchao Lin, Chuanbin Liu, Hongtao Xie, Zheng-Jun Zha, and Yongdong Zhang. 2019. Semantic-embedding and shape-aware u-net for ultrasound eyeball segmentation. In 2019 IEEE International Conference on Multimedia and Expo (ICME). IEEE, 892--897.

[33]

Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. 2017. Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision. 2980--2988.

[34]

Jonathan Long, Evan Shelhamer, and Trevor Darrell. 2015. Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition. 3431--3440.

[35]

Matthew Michael McCormick, Xiaoxiao Liu, Luis Ibanez, Julien Jomier, and Charles Marion. 2014. ITK: enabling reproducible research and open science. Frontiers in neuroinformatics 8 (2014), 13.

[36]

Fausto Milletari, Nassir Navab, and Seyed-Ahmad Ahmadi. 2016. V-net: Fully convolutional neural networks for volumetric medical image segmentation. In 2016 fourth international conference on 3D vision (3DV). IEEE, 565--571.

[37]

Sara Moccia, Elena De Momi, Sara El Hadji, and Leonardo S Mattos. 2018. Blood vessel segmentation algorithms-review of methods, datasets and evaluation metrics. Computer methods and programs in biomedicine 158 (2018), 71--91.

[38]

Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention. Springer, 234--241.

[39]

Yoshinobu Sato, Shin Nakajima, Hideki Atsumi, Thomas Koller, Guido Gerig, Shigeyuki Yoshida, and Ron Kikinis. 1997. 3D multi-scale line filter for segmentation and visualization of curvilinear structures in medical images. In CVRMed- MRCAS'97. Springer, 213--222.

[40]

Giles Tetteh, Velizar Efremov, Nils D Forkert, Matthias Schneider, Jan Kirschke, Bruno Weber, Claus Zimmer, Marie Piraud, and Bjoern H Menze. 2020. Deepvesselnet: Vessel segmentation, centerline prediction, and bifurcation detection in 3-d angiographic volumes. Frontiers in Neuroscience 14 (2020), 1285.

[41]

Tim, Jerman, Franjo, Pernu, Bo?tjan, Likar, iga, and "piclin. 2016. Enhancement of Vascular Structures in 3D and 2D Angiographic Images. IEEE Transactions on Medical Imaging (2016).

[42]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems. 5998--6008.

[43]

Arvind Vepa, Andrew Choi, Noor Nakhaei, Wonjun Lee, Noah Stier, Andrew Vu, Greyson Jenkins, Xiaoyan Yang, Manjot Shergill, Moira Desphy, et al. 2022. Weakly-Supervised Convolutional Neural Networks for Vessel Segmentation in Cerebral Angiography. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 585--594.

Cited By

Zhou LWu HLuo GZhou H(2024)Deep learning-based 3D cerebrovascular segmentation workflow on bright and black blood sequences magnetic resonance angiographyInsights into Imaging10.1186/s13244-024-01657-015:1Online publication date: 22-Mar-2024
https://doi.org/10.1186/s13244-024-01657-0
Liu XQu LXie ZZhao JShi YSong Z(2024)Towards more precise automatic analysis: a systematic review of deep learning-based multi-organ segmentationBioMedical Engineering OnLine10.1186/s12938-024-01238-823:1Online publication date: 8-Jun-2024
https://doi.org/10.1186/s12938-024-01238-8
Huang HCui YShi MYu S(2024)Semi-Supervised Cerebrovascular Segmentation Using TOF-MRA Images Based on Label Refinement and Consistency Regularization2024 IEEE International Symposium on Biomedical Imaging (ISBI)10.1109/ISBI56570.2024.10635575(1-5)Online publication date: 27-May-2024
https://doi.org/10.1109/ISBI56570.2024.10635575
Show More Cited By

Index Terms

Weakly-supervised Cerebrovascular Segmentation Network with Shape Prior and Model Indicator
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
        Image segmentation
  2. Machine learning
    1. Learning paradigms
      1. Unsupervised learning

Recommendations

Weakly Supervised Random Forest for Multi-Label Image Clustering and Segmentation
ICMR '15: Proceedings of the 5th ACM on International Conference on Multimedia Retrieval

Clustering is a useful statistical tool in data mining and computer vision. Supervised information is introduced to improve the clustering performance. However, labeling each piece of data accurately is extremely expensive when the amount of data is ...
Learning pseudo labels for semi-and-weakly supervised semantic segmentation
Highlights
- We improve the semi-and-weakly supervised semantic segmentation via learning high-quality pseudo labels.
Abstract
In this paper, we aim to tackle semi-and-weakly supervised semantic segmentation (SWSSS), where many image-level classification labels and a few pixel-level annotations are available. We believe the most crucial point for solving SWSSS ...
Weakly supervised semantic segmentation by iteratively refining optimal segmentation with deep cues guidance
Abstract
Weakly supervised semantic segmentation under image-level label supervision has undergone impressive improvements over the past years. These approaches can significantly reduce human annotation efforts, although they remain inferior to fully ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ICMR '22: Proceedings of the 2022 International Conference on Multimedia Retrieval

June 2022

714 pages

ISBN:9781450392389

DOI:10.1145/3512527

General Chairs:
Vincent Oria
New Jersey Institute of Technology, USA
,
Maria Luisa Sapino
Università degli Studi di Torino, Italy
,
Shin'ichi Satoh
National Institute of Informatics, Japan
,
Brigitte Kerhervé
Université du Québec à Montréal, Canada
,
Program Chairs:
Wen-Huang Cheng
National Yang Ming Chao Tung University, Taiwan
,
Ichiro Ide
Nagoya University, Japan
,
Vivek Singh
Rutgers University, USA

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 June 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Conference

ICMR '22

Sponsor:

SIGMM

ICMR '22: International Conference on Multimedia Retrieval

June 27 - 30, 2022

NJ, Newark, USA

Acceptance Rates

Overall Acceptance Rate 254 of 830 submissions, 31%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
261
Total Downloads

Downloads (Last 12 months)101
Downloads (Last 6 weeks)9

Reflects downloads up to 02 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zhou LWu HLuo GZhou H(2024)Deep learning-based 3D cerebrovascular segmentation workflow on bright and black blood sequences magnetic resonance angiographyInsights into Imaging10.1186/s13244-024-01657-015:1Online publication date: 22-Mar-2024
https://doi.org/10.1186/s13244-024-01657-0
Liu XQu LXie ZZhao JShi YSong Z(2024)Towards more precise automatic analysis: a systematic review of deep learning-based multi-organ segmentationBioMedical Engineering OnLine10.1186/s12938-024-01238-823:1Online publication date: 8-Jun-2024
https://doi.org/10.1186/s12938-024-01238-8
Huang HCui YShi MYu S(2024)Semi-Supervised Cerebrovascular Segmentation Using TOF-MRA Images Based on Label Refinement and Consistency Regularization2024 IEEE International Symposium on Biomedical Imaging (ISBI)10.1109/ISBI56570.2024.10635575(1-5)Online publication date: 27-May-2024
https://doi.org/10.1109/ISBI56570.2024.10635575
Jeong SLee SSeok CLee ELee J(2023)Lightweight Deep Learning Model for Real-Time Colorectal Polyp SegmentationElectronics10.3390/electronics1209196212:9(1962)Online publication date: 23-Apr-2023
https://doi.org/10.3390/electronics12091962
Hamed ESalem MBadr NTolba M(2023)An Efficient Combination of Convolutional Neural Network and LightGBM Algorithm for Lung Cancer Histopathology ClassificationDiagnostics10.3390/diagnostics1315246913:15(2469)Online publication date: 25-Jul-2023
https://doi.org/10.3390/diagnostics13152469
Chen CChen YSong SWang JNing HXiao REl Saddik AMei TCucchiara RBertini MTobon Vallejo DAtrey PHossain M(2023)Cerebrovascular Segmentation in TOF-MRA with Topology Regularization Adversarial ModelProceedings of the 31st ACM International Conference on Multimedia10.1145/3581783.3611718(4250-4259)Online publication date: 26-Oct-2023
https://dl.acm.org/doi/10.1145/3581783.3611718
Chen CZhou KWang ZZhang QXiao R(2023)All answers are in the images: A review of deep learning for cerebrovascular segmentationComputerized Medical Imaging and Graphics10.1016/j.compmedimag.2023.102229107(102229)Online publication date: Jul-2023
https://doi.org/10.1016/j.compmedimag.2023.102229
Chen CWang HChen YYin ZYang XNing HZhang QLi WXiao RZhao J(2023)Understanding the brain with attention: A survey of transformers in brain sciencesBrain‐X10.1002/brx2.291:3Online publication date: 12-Oct-2023
https://doi.org/10.1002/brx2.29

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents