Abstract
Diabetic Retinopathy (DR) is a primary cause of blindness, necessitating early detection and diagnosis. This paper focuses on referable DR classification to enhance the applicability of the proposed method in clinical practice. We develop an advanced cross-learning DR classification method leveraging transfer learning and cross-attention mechanisms. The proposed method employs the Swin U-Net architecture to segment lesion maps from DR fundus images. The Swin U-Net segmentation model, enriched with DR lesion insights, is transferred to generate a lesion map. Both the fundus image and its segmented lesion map are used as complementary inputs for the classification model. A cross-attention mechanism is deployed to improve the model’s ability to capture fine-grained details from the input pairs. Our experiments, utilizing two public datasets, FGADR and EyePACS, demonstrate a superior accuracy of 94.6%, surpassing current state-of-the-art methods by 4.4%. To this end, we aim for the proposed method to be seamlessly integrated into clinical workflows, enhancing accuracy and efficiency in identifying referable DR.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
A.Chandranath, et al.: Detecting severity of diabetic retinopathy from fundus images using ensembled transformers. arXiv (2023)
A.Sebastian, et al.: A survey on deep-learning-based diabetic retinopathy classification. Diagnostics 13(3) (2023)
B.Usharani, et al.: Deep learning for the detection and classification of diabetic retinopathy with an improved activation function. Healthcare 11(1) (2022)
C.F.Chen, et al.: Crossvit: Cross-attention multi-scale vision transformer for image classification. ICCV (2021)
D.Ling, et al.: A deep learning system for detecting diabetic retinopathy across the disease spectrum. Nature communications 12(1) (2021)
EyePACS: Diabetic retinopathy detection (2015)
G.IJ, et al.: Explaining and harnessing adversarial examples. arXiv (2014)
H.Gao, et al.: Densely connected convolutional networks. Proceedings of the IEEE conference on computer vision and pattern recognition (2017)
J.Long, et al.: Fully convolutional networks for semantic segmentation. CVPR (2015)
K.He, et al.: Deep residual learning for image recognition. CVPR (2016)
L.Xiaoling, et al.: Mvdrnet: Multi-view diabetic retinopathy detection by combining dcnns and attention mechanisms. Pattern Recognition (2021)
M.Cheena, et al.: Using deep learning architectures for detection and classification of diabetic retinopathy. Sensors 23(12) (2023)
M.Chetoui, Moulay, A.: Explainable end-to-end deep learning for diabetic retinopathy detection across multiple datasets. Journal of Medical Imaging (2020)
M.Sadria, et al.: Adversarial training improves model interpretability in single-cell rna-seq analysis. Bioinformatics Advances (2023)
O.Petit, et al.: U-net transformer: Self and cross attention for medical image segmentation. MICCAI pp. 267–276 (2021)
S.Cheng, et al.: Lesion-aware contrastive learning for diabetic retinopathy diagnosis. MICCAI (2023)
S.Ioffe, et al.: Batch normalization: Accelerating deep network training by reducing internal covariate shift. ICML pp. 448–456 (2015)
S.Sushmita, et al.: Mv-swin-t: Mammogram classification with multi-view swin transformer. arXiv (2024)
Wang, S., et al.: Diabetic retinopathy diagnosis using multichannel generative adversarial network with semisupervision. IEEE Trans. Autom. Sci. Eng. 18(2), 574–585 (2021)
S.Zhao, et al.: Mitigating the accuracy-robustness trade-off via multi-teacher adversarial distillation. arXiv (2023)
V.Boreiko, et al.: Visual explanations for the detection of diabetic retinopathy from retinal fundus images. MICCAI (2022)
Gulshan, V., et al.: Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA 316(22), 2402–2410 (2016)
W.Chen, et al.: High-resolution swin transformer for automatic medical image segmentation. Sensors (2023)
Z.Dai, et al.: Coatnet: Marrying convolution and attention for all data sizes. NeurIPS (2021)
Z.Liu, et al.: Swin transformer: Hierarchical vision transformer using shifted windows. ICCV (2021)
Z.Liu, et al.: Swin transformer: Hierarchical vision transformer using shifted windows. ICCV (2021)
Z.Liu, et al.: A convnet for the 2020s. CVPR (2022)
Yi, Z., et al.: A benchmark for studying diabetic retinopathy: segmentation, grading, and transferability. IEEE Trans. Med. Imaging 40(3), 818–828 (2020)
Acknowledgments
This work was supported in part by IITP grant funded by the Korean government (MSIT) under the ICT Creative Consilience program (RS-2020-II201821, 30%), Development of Brain Disease (Stroke) Prediction Model based on Fundus Image Analysis (RS-2024-00459512, 30%), Artificial Intelligence Graduate School Program (RS-2019-II190421, 20%), and Artificial Intelligence Innovation Hub (RS-2021-II212068, 20%).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Ethics declarations
Disclosure of Interests
The authors have no competing interests to declare that are relevant to the content of this article.
Rights and permissions
Copyright information
© 2025 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Mok, D., Bum, J., Tai, L.D., Choo, H. (2025). Cross Feature Fusion of Fundus Image and Generated Lesion Map for Referable Diabetic Retinopathy Classification. In: Cho, M., Laptev, I., Tran, D., Yao, A., Zha, H. (eds) Computer Vision – ACCV 2024. ACCV 2024. Lecture Notes in Computer Science, vol 15473. Springer, Singapore. https://doi.org/10.1007/978-981-96-0901-7_3
Download citation
DOI: https://doi.org/10.1007/978-981-96-0901-7_3
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-96-0900-0
Online ISBN: 978-981-96-0901-7
eBook Packages: Computer ScienceComputer Science (R0)