Explainable Deep Learning for Medical Image Segmentation With Learnable Class Activation Mapping
Abstract
Medical image segmentation is crucial for facilitating pathology assessment, ensuring reliable diagnosis and monitoring disease progression. Deep-learning models have been extensively applied in automating medical image analysis to reduce human effort. However, the non-transparency of deep-learning models limits their clinical practicality due to the unaffordably high risk of misdiagnosis resulted from the misleading model output. In this paper, we propose a explainability metric as part of the loss function. The proposed explainability metric comes from Class Activation Map(CAM) with learnable weights such that the model can be optimized to achieve desirable balance between segmentation performance and explainability. Experiments found that the proposed model visibly heightened Dice score from to, Jaccard similarity from to and Recall from to respectively compared with U-net. In addition, results make clear that the drawn model outdistances the conventional U-net in terms of explainability performance.
References
[1]
Vijay Badrinarayanan, Alex Kendall, and Roberto Cipolla. 2017. Segnet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE transactions on pattern analysis and machine intelligence 39, 12 (2017), 2481–2495.
[2]
Diogo V Carvalho, Eduardo M Pereira, and Jaime S Cardoso. 2019. Machine learning interpretability: A survey on methods and metrics. Electronics 8, 8 (2019), 832.
[3]
Liang-Chieh Chen, Yi Yang, Jiang Wang, Wei Xu, and Alan L Yuille. 2016. Attention to scale: Scale-aware semantic image segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition. 3640–3649.
[4]
Liang-Chieh Chen, Yukun Zhu, George Papandreou, Florian Schroff, and Hartwig Adam. 2018. Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation. In Computer Vision – ECCV 2018: 15th European Conference, Munich, Germany, September 8–14, 2018, Proceedings, Part VII (Munich, Germany). Springer-Verlag, Berlin, Heidelberg, 833–851. https://doi.org/10.1007/978-3-030-01234-2_49
[5]
Noel C. F. Codella, Veronica Rotemberg, Philipp Tschandl, M. Emre Celebi, Stephen W. Dusza, David A. Gutman, Brian Helba, Aadi Kalloo, Konstantinos Liopyris, Michael A. Marchetti, Harald Kittler, and Allan Halpern. 2019. Skin Lesion Analysis Toward Melanoma Detection 2018: A Challenge Hosted by the International Skin Imaging Collaboration (ISIC). CoRR abs/1902.03368 (2019). arXiv:1902.03368http://arxiv.org/abs/1902.03368
[6]
Alexey Dosovitskiy and Thomas Brox. 2015. Inverting Visual Representations with Convolutional Networks. https://doi.org/10.48550/ARXIV.1506.02753
[7]
Jun Fu, Jing Liu, Haijie Tian, Yong Li, Yongjun Bao, Zhiwei Fang, and Hanqing Lu. 2019. Dual Attention Network for Scene Segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
[8]
Geert, Litjens, Thijs, Kooi, Babak, Ehteshami, Bejnordi, Arnaud, Arindra, and Adiyoso. 2017. A survey on deep learning in medical image analysis. Medical Image Analysis (2017).
[9]
Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. 2017. Mask r-cnn. In Proceedings of the IEEE international conference on computer vision. 2961–2969.
[10]
Andreas Holzinger, Chris Biemann, Constantinos S Pattichis, and Douglas B Kell. 2017. What do we need to build explainable AI systems for the medical domain?arXiv preprint arXiv:1712.09923 (2017).
[11]
Jie Hu, Li Shen, and Gang Sun. 2017. Squeeze-and-Excitation Networks. CoRR abs/1709.01507 (2017). arXiv:1709.01507http://arxiv.org/abs/1709.01507
[12]
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2017. ImageNet Classification with Deep Convolutional Neural Networks. Commun. ACM 60, 6 (may 2017), 84–90. https://doi.org/10.1145/3065386
[13]
Geert Litjens, Thijs Kooi, Babak Ehteshami Bejnordi, Arnaud Arindra Adiyoso Setio, Francesco Ciompi, Mohsen Ghafoorian, Jeroen Awm Van Der Laak, Bram Van Ginneken, and Clara I Sánchez. 2017. A survey on deep learning in medical image analysis. Medical image analysis 42 (2017), 60–88.
[14]
Jonathan Long, Evan Shelhamer, and Trevor Darrell. 2015. Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition. 3431–3440.
[15]
Jiasen Lu, Caiming Xiong, Devi Parikh, and Richard Socher. 2017. Knowing when to look: Adaptive attention via a visual sentinel for image captioning. In Proceedings of the IEEE conference on computer vision and pattern recognition. 375–383.
[16]
Aravindh Mahendran and Andrea Vedaldi. 2014. Understanding Deep Image Representations by Inverting Them. CoRR abs/1412.0035 (2014). arXiv:1412.0035http://arxiv.org/abs/1412.0035
[17]
O. Ronneberger, P. Fischer, and T. Brox. 2015. U-Net: Convolutional Networks for Biomedical Image Segmentation. Springer International Publishing (2015).
[18]
Abhijit Guha Roy, Nassir Navab, and Christian Wachinger. 2018. Concurrent spatial and channel ‘squeeze & excitation’in fully convolutional networks. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2018: 21st International Conference, Granada, Spain, September 16-20, 2018, Proceedings, Part I. Springer, 421–429.
[19]
Wojciech Samek, Thomas Wiegand, and Klaus-Robert Müller. 2017. Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Models. ITU Journal: ICT Discoveries - Special Issue 1 - The Impact of Artificial Intelligence (AI) on Communication Networks and Services 1 (10 2017), 1–10.
[20]
Jo Schlemper, Ozan Oktay, Michiel Schaap, Mattias Heinrich, Bernhard Kainz, Ben Glocker, and Daniel Rueckert. 2018. Attention Gated Networks: Learning to Leverage Salient Regions in Medical Images. https://doi.org/10.48550/ARXIV.1808.08114
[21]
Jo Schlemper, Ozan Oktay, Michiel Schaap, Mattias Heinrich, Bernhard Kainz, Ben Glocker, and Daniel Rueckert. 2019. Attention gated networks: Learning to leverage salient regions in medical images. Medical image analysis 53 (2019), 197–207.
[22]
Yi Wang, Zijun Deng, Xiaowei Hu, Lei Zhu, Xin Yang, Xuemiao Xu, Pheng-Ann Heng, and Dong Ni. 2018. Deep attentional features for prostate segmentation in ultrasound. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2018: 21st International Conference, Granada, Spain, September 16-20, 2018, Proceedings, Part IV 11. Springer, 523–530.
[23]
MD Zeiler and R. Fergus. 2014. Visualizing and Understanding Convolutional Networks. Springer, Cham (2014).
[24]
Matthew D. Zeiler and Rob Fergus. 2013. Visualizing and Understanding Convolutional Networks. CoRR abs/1311.2901 (2013). arXiv:1311.2901http://arxiv.org/abs/1311.2901
[25]
Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. 2014. Object Detectors Emerge in Deep Scene CNNs. https://doi.org/10.48550/ARXIV.1412.6856
[26]
B. Zhou, A. Khosla, A. Lapedriza, A. Oliva, and A. Torralba. 2016. Learning Deep Features for Discriminative Localization. CVPR (2016).
[27]
Bolei Zhou, Agata Lapedriza, Jianxiong Xiao, Antonio Torralba, and Aude Oliva. 2014. Learning Deep Features for Scene Recognition using Places Database. In Advances in Neural Information Processing Systems, Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K.Q. Weinberger (Eds.). Vol. 27. Curran Associates, Inc.https://proceedings.neurips.cc/paper/2014/file/3fe94a002317b5f9259f82690aeea4cd-Paper.pdf
Index Terms
- Explainable Deep Learning for Medical Image Segmentation With Learnable Class Activation Mapping
Recommendations
Deep Learning for Multi-task Medical Image Segmentation in Multiple Modalities
Medical Image Computing and Computer-Assisted Intervention – MICCAI 2016AbstractAutomatic segmentation of medical images is an important task for many clinical applications. In practice, a wide range of anatomical structures are visualised using different imaging modalities. In this paper, we investigate whether a single ...
Hybrid dilation and attention residual U-Net for medical image segmentation
AbstractMedical image segmentation is a typical task in medical image processing and critical foundation in medical image analysis. U-Net is well-liked in medical image segmentation, but it doesn't fully explore useful features of the channel ...
Graphical abstractDisplay Omitted
Highlights- A medical image segmentation method is proposed based on U-Net.
- A novel channel ...
Stacking Feature Maps of Multi-scaled Medical Images in U-Net for 3D Head and Neck Tumor Segmentation
Head and Neck Tumor Segmentation and Outcome PredictionAbstractMachine learning, especially deep learning, has achieved state-of-the-art performance on various computer vision tasks. For computer vision tasks in the medical domain, it remains as challenging tasks since medical data is heterogeneous, multi-...
Comments
Information & Contributors
Information
Published In
March 2023
598 pages
ISBN:9781450399449
DOI:10.1145/3590003
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 29 May 2023
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
CACML 2023
CACML 2023: 2023 2nd Asia Conference on Algorithms, Computing and Machine Learning
March 17 - 19, 2023
Shanghai, China
Acceptance Rates
CACML '23 Paper Acceptance Rate 93 of 241 submissions, 39%;
Overall Acceptance Rate 93 of 241 submissions, 39%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 84Total Downloads
- Downloads (Last 12 months)64
- Downloads (Last 6 weeks)5
Reflects downloads up to 13 Sep 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format