Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
research-article

An Explainable Deep Learning Ensemble Model for Robust Diagnosis of Diabetic Retinopathy Grading

Published: 26 October 2021 Publication History

Abstract

Diabetic retinopathy (DR) is one of the most common causes of vision loss in people who have diabetes for a prolonged period. Convolutional neural networks (CNNs) have become increasingly popular for computer-aided DR diagnosis using retinal fundus images. While these CNNs are highly reliable, their lack of sufficient explainability prevents them from being widely used in medical practice. In this article, we propose a novel explainable deep learning ensemble model where weights from different models are fused into a single model to extract salient features from various retinal lesions found on fundus images. The extracted features are then fed to a custom classifier for the final diagnosis of DR severity level. The model is trained on an APTOS dataset containing retinal fundus images of various DR grades using a cyclical learning rates strategy with an automatic learning rate finder for decaying the learning rate to improve model accuracy. We develop an explainability approach by leveraging gradient-weighted class activation mapping and shapely adaptive explanations to highlight the areas of fundus images that are most indicative of different DR stages. This allows ophthalmologists to view our model's decision in a way that they can understand. Evaluation results using three different datasets (APTOS, MESSIDOR, IDRiD) show the effectiveness of our model, achieving superior classification rates with a high degree of precision (0.970), sensitivity (0.980), and AUC (0.978). We believe that the proposed model, which jointly offers state-of-the-art diagnosis performance and explainability, will address the black-box nature of deep CNN models in robust detection of DR grading.

References

[1]
Q. L. Jeany, T. Welchowski, M. Schmid, J. Letow, I. Wolpers, F. Pascual-Camps, F. G. Holz, and R. P. Finger. 2020. Prevalence, incidence and future projection of diabetic eye disease in Europe: A systematic review and meta-analysis. European Journal of Epidemiology 35, 1 (2020), 11–23.
[2]
N. Cheung, T. Y. Wong, and P. Mitchell. 2010. Diabetic retinopathy. Lancet 376, 9735 (2010), 124–136.
[3]
L. Carson, Y. Darvin, G. Margaret, and L. Tony. 2017. Automated detection of diabetic retinopathy using deep learning. In Proceedings of AMIA Joint Summits on Translational Science 2017, 147–155.
[4]
W. L. Alyoubi, W. M. Shalash, and M. F. Abulkhair. 2020. Diabetic retinopathy detection through deep learning techniques: A review. Informatics in Medicine Unlocked 20 (2020), 100377.
[5]
J. Krause, V. Gulshan, E. Rahimy, P. Karth, K. Widner, G. S. Corrado, L. Peng, and D. R. Webster. 2017. Grader variability and the importance of reference standards for evaluating machine learning models for diabetic retinopathy, 2017, CoRR, abs/1710.01711.
[6]
M. A. Rahman, M. S. Hossain, N. A. Alrajeh, and B. B. Gupta. 2021. A multimodal, multimedia point-of-care deep learning framework for COVID-19 diagnosis. ACM Trans. Multimedia Comput. Commun. 17, 1s, Article 18 (2021), 24 pages. DOI:
[7]
H. Pratt, F. Coenen, D. M.Broadbent, S. P. Harding, and Y. Zheng. 2016. Convolutional neural networks for diabetic retinopathy. Procedia Computer Science 90, 2016, 200–205.
[8]
A. Singh, A. Dhillon, N. Kumar, M. S. Hossain, G. Muhammad, and M. Kumar. 2021. eDiaPredict: An ensemble-based framework for diabetes prediction. ACM Trans. Multimedia Comput. Commun. Appl. 17, 2s, Article 66 (2021), 26 pages.
[9]
S. Wan, Y. Liang, and Y. Zhang. 2018. Deep convolutional neural networks for diabetic retinopathy detection by image classification. Computers & Electrical Engineering 72, (2018), 274–282.
[10]
B. T. Polyak and A. B. Juditsky. 1992. Acceleration of stochastic approximation by averaging. SIAM Journal on Control and Optimization 30, 4 (1992), 838–855.
[11]
L. N. Smith. 2017. Cyclical learning rates for training neural networks. In Proceedings of the IEEE Winter Conference on Applications of Computer Vision 464–472.
[12]
APTOS. 2019. APTOS 2019 blindness detection. https://www.kaggle.com/c/aptos2019-blindness-detectionLast. accessed October 25, 2020.
[13]
E. Decencire, X. Zhang, G. Cazuguel, B. Lay, B. Cochener, C. Trone, P. Gain, R. Ordonez, P. Massin, A. Erginay, B. Charton, and J.-C. Klein. 2014. Feedback on a publicly distributed database: The Messidor database. Image Analysis & Stereology 33, 3 (2014), 231–234.
[14]
Prasanna Porwal, Samiksha Pachade, Manesh Kokare, Girish Deshmukh, Jaemin Son, Woong Bae, Lihong Liu, Jianzong Wang, Xinhui Liu, Liangxin Gao, TianBo Wu, Jing Xiao, Fengyan Wang, Baocai Yin, Yunzhi Wang, Gopichandh Danala, Linsheng He, Yoon Ho Choi, Yeong Chan Lee, Sang-Hyuk Jung, Zhongyu Li, Xiaodan Sui, Junyan Wu, Xiaolong Li, Ting Zhou, Janos Toth, Agnes Baran, Avinash Kori, Sai Saketh Chennamsetty, Mohammed Safwan, Varghese Alex, Xingzheng Lyu, Li Cheng, Qinhao Chu, Pengcheng Li, Xin Ji, Sanyuan Zhang, Yaxin Shen, Ling Dai, Oindrila Saha, Rachana Sathish, Tânia Melo, Teresa Araújo, Balazs Harangi, Bin Sheng, Ruogu Fang, Debdoot Sheet, Andras Hajdu, Yuanjie Zheng, Ana Maria Mendonça, Shaoting Zhang, Aurélio Campilho, Bin Zheng, Dinggang Shen, Luca Giancardo, Gwenolé Quellec, and Fabrice Mériaudeau. 2020. IDRiD: diabetic retinopathy – Segmentation and grading challenge. Med. Image Anal. 59 (2020), 101561.
[15]
M. S. Hossain, S. U. Amin, G. Muhammad, and M. Al Sulaiman. 2019. Applying deep learning for epilepsy seizure detection and brain mapping visualization. ACM Trans. Multimedia Comput. Commun. 15, 1s 10 (2019), 17 pages.
[16]
G. Muhammad, M. S. Hossain, and N. Kumar. 2021. EEG-Based pathology detection for home health monitoring. IEEE Journal on Selected Areas in Communications 39, 2 (2021), 603–610. DOI:https://doi.org/10.1109/JSAC.2020.3020654
[17]
R. Priya and P. Aruna. 2012. SVM and neural-network-based diagnosis of diabetic retinopathy. International Journal of Computer Applications 41, 1 (2012), 6–12.
[18]
S. S. Rahim, V. Palade, J. Shuttleworth, and C. Jayne. 2014. Automatic screening and classification of diabetic retinopathy fundus images. In Proceedings of the International Conference on Communications in Computer and Information Science. 113–122.
[19]
P. P. Conde, J. de la Calleja, A. Benitez, and M. A. Medina. 2012. Image-based classification of diabetic retinopathy using machine learning. In Proceedings of the 12th International Conference on Intelligent Systems Design and Applications (ISDA). 826–830.
[20]
A. Colomer, J. Igual, and V. Naranjo. 2020. Detection of early signs of diabetic retinopathy based on textural and morphological information in fundus images. Sensors 2020, 20, 1005. DOI:
[21]
M. S. Hossain, G. Muhammad, and N. Guizani. 2020. Explainable AI and mass surveillance system-based healthcare framework to combat COVID-I9 like pandemics. IEEE Network 34, 4 (2020), 126–132.
[22]
M. S. Hossain, M. Al-Hammadi, and G. Muhammad. 2019. Automatic fruit classification using deep learning for industrial applications. IEEE Transactions on Industrial Informatics 15, 2 (2019), 1027–1034.
[23]
R. Rajalakshmi, S. Arulmalar, M. Usha, V. Prathiba, K. S. Kareemuddin, R. M. Anjana, and V. Mohan. 2015. Validation of smartphone based retinal photography for diabetic retinopathy screening. PLoS One 10, 9 (2015).
[24]
V. Gulshan. 2016. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA 316, 22 (2016), 2402–2410.
[25]
C. Szegedy, C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna. 2016. Rethinking the inception architecture for computer vision. In Proc.of IEEE Conference on Computer Vision and Pattern Recognition 2016, 2818–2826.
[26]
Diabetic Retinopathy Detection. https://www.kaggle.com/c/diabetic-retinopathy-detection/data, Last accessed October 30, 2020.
[27]
Messidor-2 DR Grades. https://www.kaggle.com/google-brain/messidor2-dr-grades, Last accessed October 30, 2020.
[28]
R. Gargey and T. Leng. 2017. Automated identification of diabetic retinopathy using deep learning. Ophthalmology 124, 7 (2017), 962–969.
[29]
Y.-H. Li, N.-N. Yeh, S.-J. Chen, and Y.-C. Chung. 2019. Computer-assisted diagnosis for diabetic retinopathy based on fundus images using deep convolutional neural network. Mobile Information Systems 2019, Article ID 6142839. https://doi.org/10.1155/2019/6142839
[30]
M. S. Hossain. 2017. Cloud-supported cyber–physical localization framework for patients monitoring. IEEE Systems Journal 11, 1 (2017), 118–127.
[31]
B. Tymchenko, P. Marchenko, and D. Spodarets. 2020. Deep learning approach to diabetic retinopathy detection. arXiv:2003.02261 [cs.LG], 2020.
[32]
M. T. Hagos and S. Kant. 2019. Transfer learning based detection of diabetic retinopathy from small dataset. CoRR, abs/1905.07203.
[33]
R. Sarki, S. Michalska, K. Ahmed, H. Wang, and Y. Zhang. 2019. Convolutional neural networks for mild diabetic retinopathy detection: An experimental study. bioRxiv, 2019.
[34]
V. S. Tseng, C.-L. Chen, C.-M. Liang, M.-C. Tai, J.-T. Liu, P.-Y. Wu, M.-S. Deng, Y.-W. Lee, T.-Y. Huang, and Y.-H. Chen. 2020. Leveraging multimodal deep learning architecture with retina lesion information to detect diabetic retinopathy. Translational Vision Science & Technology (TVST) 9, 2 (2020), 41. https://doi.org/10.1167/tvst.9.2.41
[35]
R. E. Hacisoftaoglu, M. Karakaya, and A. B. Sallam. 2020. Deep learning frameworks for diabetic retinopathy detection with smartphone-based retinal imaging systems. Pattern Recognition Letters 135, (2020), 409–417.
[36]
C. Mahiba and A. Jayachandran. 2019. Severity analysis of diabetic retinopathy in retinal images using hybrid structure descriptor and modified CNNs. Measurement 135 (2019), 762–767.
[37]
M. Mateen, J. Wen, N. Nasrullah, S. Sun, and S. Hayat. 2020. Exudate detection for diabetic retinopathy using pretrained convolutional neural networks. Complexity 2020.
[38]
M. Chetoui and M. A. Akhloufi. 2020. Explainable diabetic retinopathy using efficientNET. In Proceedings of the 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC). 2020. DOI:
[39]
A. Kind and G. Azzopardi. 2019. An explainable AI-based computer aided detection system for diabetic retinopathy using retinal fundus images. CAIP (1) 2019, 457–468.
[40]
S. Stolte and R. Fang. 2020. A survey on medical image analysis in diabetic retinopathy. Med Image Anal 64 101742.
[41]
J. Deng, W. Dong, R. Socher, L. Li, K. Li, and L. Fei-Fei. 2009. ImageNet: A large-scale hierarchical image database. International Conference on Computer Vision and Pattern Recognition (CVPR), 248–255.
[42]
Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. 2015. Going deeper with convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1–9.
[43]
B. Zhou, A. Khosla, A. Lapedriza, A. Oliva, and A. Torralba. 2016. Learning deep features for discriminative localization.In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2016, 2921–2929. DOI:
[44]
R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra. 2019. Grad-CAM: Visual explanations from deep networks via gradient-based localization. International Journal of Computer Vision 128, 2 (2019), 336–359.
[45]
S. Lundberg and S. I. Lee. 2017. A unified approach to interpreting model predictions. In Proceedings of Neural Information Processing Systems (NIPS). 4768–4777.
[46]
B. Kenstler. Cyclical Learning Rates Implementation. https://github.com/bckenstler/CLR.

Cited By

View all
  • (2024)Bio-Inspired Watermarking Method for Authentication of Fundus Images in Computer-Aided Diagnosis of RetinopathyMathematics10.3390/math1205073412:5(734)Online publication date: 29-Feb-2024
  • (2024)Reviewing CAM-Based Deep Explainable Methods in HealthcareApplied Sciences10.3390/app1410412414:10(4124)Online publication date: 13-May-2024
  • (2024)Smart grading of diabetic retinopathy: an intelligent recommendation-based fine-tuned EfficientNetB0 frameworkFrontiers in Artificial Intelligence10.3389/frai.2024.13961607Online publication date: 16-Apr-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Multimedia Computing, Communications, and Applications
ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 17, Issue 3s
October 2021
324 pages
ISSN:1551-6857
EISSN:1551-6865
DOI:10.1145/3492435
Issue’s Table of Contents

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 October 2021
Accepted: 01 June 2021
Revised: 01 May 2021
Received: 01 December 2020
Published in TOMM Volume 17, Issue 3s

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Explainable deep CNN
  2. diabetic retinopathy diagnosis
  3. ensemble model
  4. transfer learning
  5. retinal fundus images

Qualifiers

  • Research-article
  • Refereed

Funding Sources

  • Taif University Researchers

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)220
  • Downloads (Last 6 weeks)18
Reflects downloads up to 09 Nov 2024

Other Metrics

Citations

Cited By

View all
  • (2024)Bio-Inspired Watermarking Method for Authentication of Fundus Images in Computer-Aided Diagnosis of RetinopathyMathematics10.3390/math1205073412:5(734)Online publication date: 29-Feb-2024
  • (2024)Reviewing CAM-Based Deep Explainable Methods in HealthcareApplied Sciences10.3390/app1410412414:10(4124)Online publication date: 13-May-2024
  • (2024)Smart grading of diabetic retinopathy: an intelligent recommendation-based fine-tuned EfficientNetB0 frameworkFrontiers in Artificial Intelligence10.3389/frai.2024.13961607Online publication date: 16-Apr-2024
  • (2024)An efficient explainable deep neural network classifier for diabetic retinopathy detectionInternational Journal of Computers and Applications10.1080/1206212X.2024.238934246:9(795-810)Online publication date: 23-Aug-2024
  • (2024)Diabetic retinopathy screening through artificial intelligence algorithms: A systematic reviewSurvey of Ophthalmology10.1016/j.survophthal.2024.05.008Online publication date: Jun-2024
  • (2024)A multi-resolution convolutional attention network for efficient diabetic retinopathy classificationComputers and Electrical Engineering10.1016/j.compeleceng.2024.109243117(109243)Online publication date: Jul-2024
  • (2024)A comprehensive review of explainable AI for disease diagnosisArray10.1016/j.array.2024.10034522(100345)Online publication date: Jul-2024
  • (2024)Advances in retinal microaneurysms detection, segmentation and datasets for the diagnosis of diabetic retinopathy: a systematic literature reviewMultimedia Tools and Applications10.1007/s11042-023-18089-583:30(74897-74935)Online publication date: 13-Feb-2024
  • (2024)Automated machine learning with interpretation: A systematic review of methodologies and applications in healthcareMedicine Advances10.1002/med4.752:3(205-237)Online publication date: 27-Aug-2024
  • (2023)Designing the Architecture of a Convolutional Neural Network Automatically for Diabetic Retinopathy DiagnosisMathematics10.3390/math1102030711:2(307)Online publication date: 6-Jan-2023
  • Show More Cited By

View Options

Get Access

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Full Text

View this article in Full Text.

Full Text

HTML Format

View this article in HTML Format.

HTML Format

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media