Overview of Deep Learning Based Cardiac MR Image Segmentation Methods
Pages 503 - 509
Abstract
Cardiac image segmentation has great significance of making correct diagnoses of cardiac diseases, and providing useful information for clinical treatment and surgery. Among cardiac images, magnetic resonance imaging (MRI) is the most common modality. With the rapid development of deep learning, it is widely used in various fields such as computer vision, natural language processing and visual recognition. In the meantime, a large number of deep learning based methods have been proposed for Cardiac MR image segmentation. In this paper, we attempt to provide a survey on such methods published in peer-reviewed journals and conferences between 2017 and 2021, including their motivations and segmentation strategies. We then explore the advantages, disadvantages and the challenges such as generalization ability, interpretability and so on. In addition, several publicly available cardiac image datasets and evaluation metrics are given. This paper will offer useful guidelines to study MR image segmentation for researchers and practitioners working in related domains.
References
[1]
Maurits A. Allessie, Jannie Ausma, and Ulrich Schotten. 2002. Electrical, contractile and structural remodeling during atrial fibrillation.Cardiovascular research 542 (2002), 230–46.
[2]
Andreas Boldt, Ulrike Wetzel, Joerg Lauschke, J Weigl, Jf Gummert, Gerd Hindricks, Hans Kottkamp, and Stefan Dhein. 2004. Fibrosis in left atrial tissue of patients with atrial fibrillation with and without underlying mitral valve disease. Heart 90(2004), 400–405.
[3]
Patrick M. Boyle, Sohail Zahid, and Natalia A. Trayanova. 2016. Towards personalized computational modelling of the fibrotic substrate for atrial arrhythmia. Europace: European pacing, arrhythmias, and cardiac electrophysiology : journal of the working groups on cardiac pacing, arrhythmias, and cardiac cellular electrophysiology of the European Society of Cardiology 18 suppl 4 (2016), iv136–iv145.
[4]
Bin Cai, Erkang Cheng, Pengpeng Liang, Chi Xiong, Zhiyong Sun, Qiang Zhang, and Bo Song. 2021. Ghost-Light-3dnet: Efficient Network For Heart Segmentation. In 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI). IEEE, 1704–1708.
[5]
Chen Chen, Wenjia Bai, Rhodri H Davies, Anish N Bhuva, Charlotte H Manisty, Joao B Augusto, James C Moon, Nay Aung, Aaron M Lee, Mihir M Sanghvi, 2020. Improving the generalizability of convolutional neural network-based segmentation on CMR images. Frontiers in cardiovascular medicine 7 (2020), 105.
[6]
Chen Chen, Wenjia Bai, and Daniel Rueckert. 2018. Multi-task learning for left atrial segmentation on GE-MRI. In International workshop on statistical atlases and computational models of the heart. Springer, 292–301.
[7]
Chen Chen, Chen Qin, Huaqi Qiu, Giacomo Tarroni, Jinming Duan, Wenjia Bai, and Daniel Rueckert. 2020. Deep learning for cardiac image segmentation: a review. Frontiers in Cardiovascular Medicine 7 (2020), 25.
[8]
Jun Chen, Guang Yang, Habib Khan, Heye Zhang, Yanping Zhang, Shu Zhao, Raad Mohiaddin, Tom Wong, David Firmin, and Jennifer Keegan. 2021. JAS-GAN: Generative Adversarial Network based joint atrium and scar segmentation on unbalanced Atrial targets. IEEE Journal of Biomedical and Health Informatics (2021).
[9]
Shusil Dangi, Cristian A Linte, and Ziv Yaniv. 2019. A distance map regularized CNN for cardiac cine MR image segmentation. Medical physics 46, 12 (2019), 5637–5651.
[10]
Tarun Kanti Ghosh, Md. Kamrul Hasan, Shidhartho Roy, Md. Ashraful Alam, Eklas Hossain, and Mohiudding Ahmad. 2021. Multi-Class Probabilistic Atlas-Based Whole Heart Segmentation Method in Cardiac CT and MRI. IEEE Access 9(2021), 66948–66964.
[11]
Evgin Goceri. 2019. Challenges and recent solutions for image segmentation in the era of deep learning. (2019), 1–6.
[12]
Ian J. Goodfellow, Yoshua Bengio, and Aaron C. Courville. 2015. Deep Learning. Nature 521(2015), 436–444.
[13]
Hayit Greenspan, Bram Van Ginneken, and Ronald M Summers. 2016. Guest editorial deep learning in medical imaging: Overview and future promise of an exciting new technique. IEEE transactions on medical imaging 35, 5 (2016), 1153–1159.
[14]
Marija Habijan, Irena Galić, Hrvoje Leventić, and Krešimir Romić. 2021. Whole Heart Segmentation Using 3D FM-Pre-ResNet Encoder–Decoder Based Architecture with Variational Autoencoder Regularization. Applied Sciences 11, 9 (2021), 3912.
[15]
Fabian Isensee, Paul F. Jaeger, Peter M. Full, Ivo Wolf, Sandy Engelhardt, and Klaus Maier-Hein. 2017. Automatic Cardiac Disease Assessment on cine-MRI via Time-Series Segmentation and Domain Specific Features. ArXiv abs/1707.00587(2017).
[16]
Haochuan Jiang, Chengjia Wang, Agisilaos Chartsias, and Sotirios A Tsaftaris. 2020. Max-Fusion U-Net for Multi-Modal Pathology Segmentation with Attention and Dynamic Resampling. (2020), 68–81.
[17]
Bedia Kara, Alaaddin Nayman, Ibrahim Guler, Enes Elvin Gul, Mustafa Koplay, and Yahya Paksoy. 2016. Quantitative Assessment of Left Ventricular Function and Myocardial Mass: A Comparison of Coronary CT Angiography with Cardiac MRI and Echocardiography. Polish Journal of Radiology 81 (2016), 95–102.
[18]
Rashed Karim, Aruna Arujuna, Richard James Housden, Jaspal Gill, Hannah Cliffe, Kavir Matharu, Jaswinder Gill, Christopher Aldo Rindaldi, Mark O’Neill, Daniel Rueckert, 2014. A method to standardize quantification of left atrial scar from delayed-enhancement MR images. IEEE journal of translational engineering in health and medicine 2 (2014), 1–15.
[19]
Rashed Karim, Lauren-Emma Blake, Jiro Inoue, Qian Tao, Shuman Jia, R. James Housden, Pranav Bhagirath, Jean-Luc Duval, Marta Varela, Jonathan M. Behar, Lo?c Cadour, Rob J. van der Geest, Hubert Cochet, Maria Drangova, Maxime Sermesant, Reza Razavi, Oleg Aslanidi, Ronak Rajani, and Kawal Rhode. 2018. Algorithms for left atrial wall segmentation and thickness-Evaluation on an open-source CT and MRI image database. Medical Image Analysis 50 (2018), 36–53.
[20]
Rashed Karim, Richard James Housden, Mayuragoban Balasubramaniam, Zhuotong Chen, D. Perry, Ayesha Uddin, Yosra Al-Beyatti, Ebrahim Palkhi, Prince Acheampong, Samantha Obom, Anja Hennemuth, Yingli Lu, Wenjia Bai, Wenzhe Shi, Yi Gao, Heinz-Otto Peitgen, Perry E. Radau, Reza Razavi, Allen R. Tannenbaum, Daniel Rueckert, Joshua E. Cates, Tobias Schaeffter, Dana C. Peters, Rob Macleod, and Kawal S. Rhode. 2013. Evaluation of current algorithms for segmentation of scar tissue from late Gadolinium enhancement cardiovascular magnetic resonance of the left atrium: an open-access grand challenge. Journal of Cardiovascular Magnetic Resonance 15 (2013), 105–105.
[21]
Mahendra Khened, Varghese Alex Kollerathu, and Ganapathy Krishnamurthi. 2019. Fully convolutional multi-scale residual DenseNets for cardiac segmentation and automated cardiac diagnosis using ensemble of classifiers. Medical image analysis 51 (2019), 21–45.
[22]
Fanwei Kong, Nathan M. Wilson, and Shawn C. Shadden. 2021. A Deep-Learning Approach For Direct Whole-Heart Mesh Reconstruction. Medical image analysis 74 (2021), 102222.
[23]
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems 25 (2012), 1097–1105.
[24]
Caizi Li, Qianqian Tong, Xiangyun Liao, Weixin Si, Shu Chen, Qiong Wang, and Zhiyong Yuan. 2019. APCP-NET: Aggregated Parallel Cross-Scale Pyramid Network for CMR Segmentation. 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019) (2019), 784–788.
[25]
Lei Li, Xin Weng, Julia A Schnabel, and Xiahai Zhuang. 2020. Joint left atrial segmentation and scar quantification based on a DNN with spatial encoding and shape attention. In International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 118–127.
[26]
Lei Li, Fuping Wu, Guang Yang, Lingchao Xu, Tom Wong, Raad Mohiaddin, David Firmin, Jennifer Keegan, and Xiahai Zhuang. 2020. Atrial scar quantification via multi-scale CNN in the graph-cuts framework. Medical image analysis 60 (2020), 101595.
[27]
Fangzhou Liao, Xi Chen, Xiaolin Hu, and Sen Song. 2019. Estimation of the Volume of the Left Ventricle From MRI Images Using Deep Neural Networks. IEEE Transactions on Cybernetics 49 (2019), 495–504.
[28]
Xiangyun Liao, Yinling Qian, Yilong Chen, Xueying Xiong, Qiong Wang, and Pheng-Ann Heng. 2020. MMTLNet: Multi-Modality Transfer Learning Network with adversarial training for 3D whole heart segmentation. Computerized medical imaging and graphics : the official journal of the Computerized Medical Imaging Society 85 (2020), 101785.
[29]
Geert Litjens, Thijs Kooi, Babak Ehteshami Bejnordi, Arnaud Arindra Adiyoso Setio, Francesco Ciompi, Mohsen Ghafoorian, Jeroen Awm Van Der Laak, Bram Van Ginneken, and Clara I Sánchez. 2017. A survey on deep learning in medical image analysis. Medical image analysis 42 (2017), 60–88.
[30]
Jiang Liu, Cheng Jin, J. Feng, Yu Du, Jiwen Lu, and Jie Zhou. 2018. A Vessel-Focused 3D Convolutional Network for Automatic Segmentation and Classification of Coronary Artery Plaques in Cardiac CTA. In STACOM@MICCAI.
[31]
Jun Ma. 2020. Cascaded Framework for Myocardial Infarction Segmentation from Delayed-Enhancement Cardiac MRI.
[32]
Pier-Giorgio Masci, Anna Giulia Pavon, Olivier Muller, Juan-Fernando Iglesias, Gabriella Vincenti, Pierre Monney, Brahim Harbaoui, Eric Eeckhout, and Juerg Schwitter. 2018. Relationship between CMR-derived parameters of ischemia/reperfusion injury and the timing of CMR after reperfused ST-segment elevation myocardial infarction. Journal of Cardiovascular Magnetic Resonance 20, 1 (2018), 1–10.
[33]
Marta Nuñez-Garcia, Xiahai Zhuang, Gerard Sanroma, Lei Li, Lingchao Xu, Constantine Butakoff, and Oscar Camara. 2018. Left atrial segmentation combining multi-atlas whole heart labeling and shape-based atlas selection. In International Workshop on Statistical Atlases and Computational Models of the Heart. Springer, 302–310.
[34]
Nathan Painchaud, Youssef Skandarani, Thierry Judge, Olivier Bernard, Alain Lalande, and Pierre-Marc Jodoin. 2019. Cardiac MRI Segmentation with Strong Anatomical Guarantees. ArXiv abs/1907.02865(2019).
[35]
Bhrigu R. Parmar, Tyler R. Jarrett, Eugene G. Kholmovski, Nan Hu, Dennis L. Parker, Rob Macleod, Nassir F. Marrouche, and Ravi Ranjan. 2015. Poor scar formation after ablation is associated with atrial fibrillation recurrence. Journal of Interventional Cardiac Electrophysiology 44 (2015), 247–256.
[36]
Christian Payer, Darko Štern, Horst Bischof, and Martin Urschler. 2017. Multi-label whole heart segmentation using CNNs and anatomical label configurations. In International Workshop on Statistical Atlases and Computational Models of the Heart. Springer, 190–198.
[37]
Peng Peng, Karim Lekadir, Ali Gooya, Ling Shao, Steffen Erhard Petersen, and Alejandro F. Frangi. 2015. A review of heart chamber segmentation for structural and functional analysis using cardiac magnetic resonance imaging. Magma (New York, N.y.) 29 (2015), 155–195.
[38]
Mark A Peterzan, Oliver J. Rider, and Lisa J Anderson. 2016. The Role of Cardiovascular Magnetic Resonance Imaging in Heart Failure.Cardiac failure review 22 (2016), 115–122.
[39]
Caroline Petitjean, Maria A. Zuluaga, Wenjia Bai, Jean-Nicolas Dacher, Damien Grosgeorge, Jérôme Caudron, Su Ruan, Ismail Ben Ayed, Manuel Jorge Cardoso, Hsiang-Chou Chen, Daniel Jimenez-Carretero, María J. Ledesma-Carbayo, Christos Davatzikos, Jimit Doshi, Guray Erus, Oskar Maier, Cyrus M. S. Nambakhsh, Yangming Ou, Sébastien Ourselin, Chun-Wei Peng, Nicholas S. Peters, Terry M. Peters, Martin Rajchl, Daniel Rueckert, Andrés Santos, Wenzhe Shi, Ching-Wei Wang, Haiyan Wang, and Jing Yuan. 2015. Right ventricle segmentation from cardiac MRI: A collation study. Medical image analysis 191 (2015), 187–202.
[40]
Mihaela Pop, Maxime Sermesant, Jichao Zhao, Shuo Li, Kristin McLeod, Alistair Young, Kawal Rhode, and Tommaso Mansi. 2019. Statistical Atlases and Computational Models of the Heart. Atrial Segmentation and LV Quantification Challenges: 9th International Workshop, STACOM 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 16, 2018, Revised Selected Papers. 11395 (2019).
[41]
Perry Radau, Yingli Lu, Kim Connelly, G Paul, AJWG Dick, and Graham Wright. 2009. Evaluation framework for algorithms segmenting short axis cardiac MRI. The MIDAS Journal-Cardiac MR Left Ventricle Segmentation Challenge 49 (2009).
[42]
Alec Radford, Luke Metz, and Soumith Chintala. 2016. Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks. CoRR abs/1511.06434(2016).
[43]
Daniele Ravanelli, Elena Costanza dal Piaz, Maurizio Centonze, Giulia Casagranda, Massimiliano Marini, Maurizio Del Greco, Rashed Karim, Kawal S. Rhode, and Aldo Valentini. 2014. A Novel Skeleton Based Quantification and 3-D Volumetric Visualization of Left Atrium Fibrosis Using Late Gadolinium Enhancement Magnetic Resonance Imaging. IEEE Transactions on Medical Imaging 33 (2014), 566–576.
[44]
Marc-Michel Rohé, Maxime Sermesant, and Xavier Pennec. 2017. Automatic Multi-Atlas Segmentation of Myocardium with SVF-Net. In STACOM@MICCAI.
[45]
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention. Springer, 234–241.
[46]
José Domingo Sagastagoitia, Yolanda Sáez, Marta Vacas, Idoia Narváez, Jesús Pablo Sáez de Lafuente, Enrique Molinero, Antonio Escobar, Manuel Lafita, and José Antonio Iriarte. 2007. Acute versus Chronic Myocardial Ischemia: A Differential Biological Profile Study. Pathophysiology of Haemostasis and Thrombosis 36 (2007), 91–97.
[47]
Jörg Sander, Bob D de Vos, Jelmer M Wolterink, and Ivana Išgum. 2019. Towards increased trustworthiness of deep learning segmentation methods on cardiac MRI. In Medical Imaging 2019: Image Processing, Vol. 10949. International Society for Optics and Photonics, 1094919.
[48]
Jo Schlemper, Ozan Oktay, Wenjia Bai, Daniel Coelho de Castro, Jinming Duan, Chen Qin, Joseph V. Hajnal, and Daniel Rueckert. 2018. Cardiac MR Segmentation from Undersampled k-space Using Deep Latent Representation Learning. In MICCAI.
[49]
Christodoulos Stefanadis, John Dernellis, and Pavlos K. Toutouzas. 2001. A clinical appraisal of left atrial function.European heart journal 221 (2001), 22–36.
[50]
Avan Suinesiaputra, Brett R. Cowan, Ahmed O. Al-Agamy, Mustafa A. Alattar, Nicholas Ayache, Ahmed S. Fahmy, Ayman Mahmoud Khalifa, Pau Medrano-Gracia, Marie-Pierre Jolly, Alan H. Kadish, Daniel C. Lee, Ján Margeta, S. Warfield, and Alistair A. Young. 2014. A collaborative resource to build consensus for automated left ventricular segmentation of cardiac MR images. Medical image analysis 181 (2014), 50–62.
[51]
Phi Vu Tran. 2016. A fully convolutional neural network for cardiac segmentation in short-axis MRI. arXiv preprint arXiv:1604.00494(2016).
[52]
Chengjia Wang, Thomas J. MacGillivray, Gillian Macnaught, Guang Yang, and David E. Newby. 2018. A Two-Stage U-Net Model for 3D Multi-class Segmentation on Full-Resolution Cardiac Data. In STACOM@MICCAI.
[53]
Hinrich B Winther, Christian Hundt, Bertil Schmidt, Christoph Czerner, Johann Bauersachs, Frank Wacker, and Jens Vogel-Claussen. 2017. ν-net: Deep Learning for Generalized Biventricular Cardiac Mass and Function Parameters. arXiv preprint arXiv:1706.04397(2017).
[54]
Qing Xia, Yuxin Yao, Zhiqiang Hu, and Aimin Hao. 2018. Automatic 3D atrial segmentation from GE-MRIs using volumetric fully convolutional networks. In International Workshop on Statistical Atlases and Computational Models of the Heart. Springer, 211–220.
[55]
Zhaohan Xiong, Vadim V. Fedorov, Xiaohang Fu, Elizabeth Cheng, Rob Macleod, and Jichao Zhao. 2019. Fully Automatic Left Atrium Segmentation From Late Gadolinium Enhanced Magnetic Resonance Imaging Using a Dual Fully Convolutional Neural Network. IEEE Transactions on Medical Imaging 38 (2019), 515–524.
[56]
Zhaohan Xiong, Qing Xia, Zhiqiang Hu, Ning Huang, Cheng Bian, Yefeng Zheng, Sulaiman Vesal, Nishant Ravikumar, Andreas Maier, Xin Yang, 2021. A global benchmark of algorithms for segmenting the left atrium from late gadolinium-enhanced cardiac magnetic resonance imaging. Medical Image Analysis 67 (2021), 101832.
[57]
Lequan Yu, Shujun Wang, Xiaomeng Li, Chi-Wing Fu, and Pheng-Ann Heng. 2019. Uncertainty-aware self-ensembling model for semi-supervised 3D left atrium segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 605–613.
[58]
Qian Yue, Xinzhe Luo, Qing Ye, Lingchao Xu, and Xiahai Zhuang. 2019. Cardiac Segmentation from LGE MRI Using Deep Neural Network Incorporating Shape and Spatial Priors. In MICCAI.
[59]
Jianpeng Zhang, Yutong Xie, Zhibin Liao, Johan Verjans, and Yong Xia. 2020. EfficientSeg: A Simple But Efficient Solution to Myocardial Pathology Segmentation Challenge. In Myocardial Pathology Segmentation Combining Multi-Sequence CMR Challenge. Springer, 17–25.
[60]
Zhen Zhang, Chenyu Liu, Wangbin Ding, Sihan Wang, Chenhao Pei, Mingjing Yang, and Liqin Huang. 2020. Multi-modality Pathology Segmentation Framework: Application to Cardiac Magnetic Resonance Images. (2020), 37–48.
[61]
Qiao Zheng, Hervé Delingette, Nicolas Duchateau, and Nicholas Ayache. 2018. 3D Consistent Biventricular Myocardial Segmentation Using Deep Learning for Mesh Generation. ArXiv abs/1803.11080(2018).
[62]
Tongxue Zhou, Su Ruan, and Stéphane Canu. 2019. A review: Deep learning for medical image segmentation using multi-modality fusion. Array 3–4(2019), 100004.
[63]
Xiahai Zhuang. 2013. Challenges and methodologies of fully automatic whole heart segmentation: areview. Journal of healthcare engineering 43 (2013), 371–408.
[64]
Clément Zotti, Zhiming Luo, Alain Lalande, and Pierre-Marc Jodoin. 2019. Convolutional Neural Network With Shape Prior Applied to Cardiac MRI Segmentation. IEEE Journal of Biomedical and Health Informatics 23 (2019), 1119–1128.
- Overview of Deep Learning Based Cardiac MR Image Segmentation Methods
Recommendations
DUDA: Deep Unsupervised Domain Adaptation Learning for Multi-sequence Cardiac MR Image Segmentation
Pattern Recognition and Computer VisionAbstractAutomatic segmentation of ventricles and myocardium in late gadolinium-enhanced (LGE) cardiac images is an important assessment for the clinical diagnosis of myocardial infarction. The most segmentation methods rely on the annotation of ventricles ...
Refined Deep Layer Aggregation for Multi-Disease, Multi-View & Multi-Center Cardiac MR Segmentation
Statistical Atlases and Computational Models of the Heart. Multi-Disease, Multi-View, and Multi-Center Right Ventricular Segmentation in Cardiac MRI ChallengeAbstractEfficient and accurate segmentation of the heart is important for analysis of cardiac magnetic resonance imaging (MRI). Although many convolutional neural networks (CNNs) have been proposed to address cardiac segmentation in cine MRI, the task is ...
Automated Segmentation Based on Deep Learning of the MR Vessel Wall Imaging
ICBDC '21: Proceedings of the 6th International Conference on Big Data and ComputingThe rupture of unstable or vulnerable atherosclerotic plaque is the major cause of ischemic stroke. Manual analysis of the vessel wall and plaque is labor-intensive and experience-dependent. The purpose of this study is to develop an automatic method to ...
Comments
Information & Contributors
Information
Published In
December 2021
541 pages
Copyright © 2021 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 11 April 2022
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
WI-IAT '21
Sponsor:
WI-IAT '21: IEEE/WIC/ACM International Conference on Web Intelligence
December 14 - 17, 2021
VIC, Melbourne, Australia
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 143Total Downloads
- Downloads (Last 12 months)37
- Downloads (Last 6 weeks)2
Reflects downloads up to 09 Aug 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format