Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
SpringerLink
Log in

BCT-OFD: bridging CNN and transformer via online feature distillation for COVID-19 image recognition

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

Computer-aided systems can assist radiologists in improving the efficiency of COVID-19 diagnosis. Current work adopts complex structures and the correlations between convolutional neural network (CNN) and Transformer haven’t been explored. We propose a novel model named bridging CNN and Transformer via online feature distillation (BCT-OFD). First, lightweight Mpvit-tiny and MobileNetV3-small are chosen as the teacher and student networks, respectively. Sufficient pathological knowledge is smoothly transferred from the teacher to the student using OFD. Then, a adaptive feature fusion module is designed to efficiently fuse the heterogeneous CNN and Transformer features. The implicit correlations between the two networks are fully mined to generate more discriminative fused features. And coordinate attention is adopted to make further feature refinement. The accuracy on three public avaliable datasets reach 97.76%, 98.12% and 96.96%, respectively. It validates that BCT-OFD outperforms state-of-the-art baselines in terms of effectiveness and generalization ability. Notably, BCT-OFD is relatively more lightweight and easier to deploy on those resource-constrained devices, making it the bridge that links theory to application as well as narrowing the gap between them. This study provides an innovative approach in the field of COVID-19 image recognition, offering valuable insights for further improvements in performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data Availbility

The COV dataset is freely available at https://github.com/UCSD-AI4H/COVID-CT. The SAR dataset is freely available at: https://www.kaggle.com/datasets/plameneduardo/sarscov2-ctscan-dataset. The RAD dataset is freely available at: https://www.kaggle.com/datasets/tawsifurrahman/covid19-radiography-database?select=COVID-19_Radiography_Dataset. Online query website: https://covid19.who.int/table. Our code will be uploaded to https://github.com/xiaoli-10086/BCT-OFD.

References

  1. Kufel J, Bargiel K, Koźlik M, Czogalik Ł, Dudek P, Jaworski A, Cebula M, Gruszczyńska K (2022) Application of artificial intelligence in diagnosing COVID-19 disease symptoms on chest X-rays: a systematic review. Int J Med Sci 19:1743–1752

    Article  Google Scholar 

  2. Treesatayapun C. (2023) Optimal interventional policy based on discrete-time fuzzy rules equivalent model utilizing with COVID-19 pandemic data. Int J Machine Learn Cyber 1–10.

  3. Toro CA, Ortiz ÁM, García-Pedrero M-S, Gonzalo-Martín C (2022) Automatic detection of pneumonia in chest X-ray images using textural features. Comput Biol Med 145:105466–105466

    Article  Google Scholar 

  4. Yang H, Wang L, Xu Y et al (2023) CovidViT: a novel neural network with self-attention mechanism to detect Covid-19 through X-ray images. Int J Mach Learn Cybern 14(3):973–987

    Article  Google Scholar 

  5. “Machine Learing with Linear Regression Model For COVID-19 Prediction.” International Research Journal of Modernization in Engineering Technology and Science (2022): n. pag.

  6. Jaiswal A, Gianchandani N, Singh D, Kumar V, Kaur M (2020) Classification of the COVID-19 infected patients using DenseNet201 based deep transfer learning. J Biomol Struct Dyn 39:5682–5689

    Article  Google Scholar 

  7. Aslani S, Jacob J (2023) Utilisation of deep learning for COVID-19 diagnosis. Clin Radiol 78(2):150–157

    Article  Google Scholar 

  8. Shome, Debaditya, Tejaswini Kar, Sachi Nandan Mohanty, Prayag Tiwari, Khan Muhammad, Abdullah Abdulaziz Altameem, Yazhou Zhang and Abdul Khader Jilani Saudagar. (2021) “COVID-Transformer: Interpretable COVID-19 Detection Using Vision Transformer for Healthcare.” International Journal of Environmental Research and Public Health 18: n. pag.

  9. Perera, Shehan, Srikar Adhikari and Alper Yilmaz. (2021) “Pocformer: A Lightweight Transformer Architecture For Detection Of Covid-19 Using Point Of Care Ultrasound.” 2021 IEEE International Conference on Image Processing (ICIP) 195–199.

  10. Krishnan, Koushik Sivarama and Karthik Sivarama Krishnan. “Vision (2021): Transformer based COVID-19 Detection using Chest X-rays.” 2021 6th International Conference on Signal Processing, Computing and Control (ISPCC) 644–648.

  11. Angelov, Plamen P. and Eduardo A. Soares. “Explainable-BY-design approach for COVID-19 classification VIA CT-SCAN.” medRxiv (2020).

  12. Panwar, Harsh, P. K. Gupta, Mohammad Khubeb Siddiqui, Rubén Morales-Menéndez, Prakhar Bhardwaj and Vaishnavi Singh. (2020) “A deep learning and grad-CAM based color visualization approach for fast detection of COVID-19 cases using chest X-ray and CT-Scan images.” Chaos, Solitons, and Fractals 140: 110190-110190.

  13. Sen, Shibaprasad, Soumyajit Saha, Somnath Chatterjee, Seyed Hesamoddin Mirjalili and Ram Sarkar. (2021) “A bi-stage feature selection approach for COVID-19 prediction using chest CT images.” Applied Intelligence (Dordrecht, Netherlands) 51: 8985-9000.

  14. Wang, Linda, Zhong Qiu Lin and Alexander Wong. (2020) “COVID-Net: a tailored deep convolutional neural network design for detection of COVID-19 cases from chest X-ray images.” Scientific Reports 10: n. pag.

  15. Huang, Ling, Su Ruan and Thierry Denoeux. “Covid-19 Classification with Deep (2021): Neural Network and Belief Functions.” The Fifth International Conference on Biological Information and Biomedical Engineering n. pag.

  16. Ozturk, Tulin, Muhammed Talo, Eylul Azra Yildirim, Ulas Baran Baloglu, Özal Yıldırım and U. Rajendra Acharya. (2020) “Automated detection of COVID-19 cases using deep neural networks with X-ray images.” Computers in Biology and Medicine 121: 103792–103792.

  17. Rebuffi, Sylvestre-Alvise, (2017) Hakan Bilen and Andrea Vedaldi. “Learning multiple visual domains with residual adapters.” NIPS.

  18. Rebuffi, Sylvestre-Alvise, Hakan Bilen and Andrea Vedaldi. (2018) “Efficient Parametrization of Multi-domain Deep Neural Networks.” 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition: 8119–8127.

  19. Liu Q, Dou Qi, Lequan Yu, Heng P-A (2020) MS-net: Multi-site network for improving prostate segmentation with heterogeneous MRI data. IEEE Trans Med Imaging 39:2713–2724

    Article  Google Scholar 

  20. Wang Z, Liu Q, Dou Qi (2020) Contrastive cross-site learning with redesigned Net for COVID-19 CT classification. IEEE J Biomed Health Inform 24:2806–2813

    Article  Google Scholar 

  21. Heidarian, Shahin, Parnian Afshar, Nastaran Enshaei, Farnoosh Naderkhani, Anastasia Oikonomou, Seyed Farokh Atashzar, Faranak Babaki Fard, Kaveh Samimi, Konstantinos N. Plataniotis, Arash Mohammadi and Moezedin Javad (2020) Rafiee. “COVID-Fact: A Fully-Automated Capsule Network-Based Framework for Identification of COVID-19 Cases from Chest CT Scans.” Frontiers in Artificial Intelligence 4: n. pag.

  22. Zhou Z-H (2018) A brief introduction to weakly supervised learning. Natl Sci Rev 5:44–53

    Article  Google Scholar 

  23. Ardakani, Ali Abbasian, Alireza Rajabzadeh Kanafi, U. Rajendra Acharya, Nazanin Khadem and Afshin Mohammadi. (2020) “Application of deep learning technique to manage COVID-19 in routine clinical practice using CT images: Results of 10 convolutional neural networks.” Computers in Biology and Medicine 121: 103795–103795.

  24. Zhang H, Liang W, Li C, Xiong Q, Shi H, Lang Hu, Li G (2022) DCML: Deep contrastive mutual learning for COVID-19 recognition. Biomed Signal Process Control 77:103770–103770

    Article  Google Scholar 

  25. Yang, Xingyi, Jinyu Zhao, Yichen Zhang, Xuehai He and Pengtao Xie. (2020) “COVID-CT-Dataset: A CT Scan Dataset about COVID-19.”ArXiv abs/2003.13865: n. pag.

  26. Soares, Eduardo A., Plamen P. Angelov, Sarah Biaso, Michele Higa Froes and Daniel Kanda Abe. (2020) “SARS-CoV-2 CT-scan dataset:A large dataset of real patients CT scans for SARS-CoV-2 identification.” medRxiv: n. pag.

  27. Rahman T, Khandakar A, Qiblawey Y et al (2021) Exploring the effect of image enhancement techniques on COVID-19 detection using chest X-ray images. Comput Biol Med 132:104319

    Article  Google Scholar 

  28. Liu, Zhuang, Hanzi Mao, Chaozheng Wu, Christoph Feichtenhofer, Trevor Darrell and Saining Xie. (2022) “A ConvNet for the 2020s.” 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR): 11966–11976.

  29. Liu, Ze, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin and Baining Guo. (2021) “Swin Transformer: Hierarchical Vision Transformer using Shifted Windows.” 2021 IEEE/CVF International Conference on Computer Vision (ICCV): 9992–10002.

  30. Banerjee, Avinandan, Rajdeep Bhattacharya, Vikrant Bhateja, Pawan Kumar Singh, Aime’ Lay-Ekuakille and Ram Sarkar. (2021) “COFE-Net: An ensemble strategy for Computer-Aided Detection for COVID-19.” Measurement 187 110289–110289.

  31. Garg M, Dhiman G (2021) A novel content-based image retrieval approach for classification using GLCM features and texture fused LBP variants. Neural Comput Appl 33:1311–1328

    Article  Google Scholar 

  32. Howard, Andrew G., Mark Sandler, Grace Chu, Liang-Chieh Chen, Bo Chen, Mingxing Tan, Weijun Wang, Yukun Zhu, Ruoming Pang, Vijay Vasudevan, Quoc V. Le and Hartwig Adam. (2019) “Searching for MobileNetV3.” 2019 IEEE/CVF International Conference on Computer Vision (ICCV): 1314–1324.

  33. Lee, Youngwan, Jonghee Kim, Jeffrey Willette and Sung Ju Hwang. “MPViT: Multi-Path Vision Transformer for Dense Prediction.” 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2021): 7277–7286.

  34. Liu, Chengeng and Qingshan Yin. “Automatic Diagnosis of COVID-19 Using a tailored Transformer-Like Network.” Journal of Physics: Conference Series 2010 (2021): n. pag.

  35. Liang, Shuang. “A hybrid deep learning framework for Covid-19 detection via 3D Chest CT Images.” ArXiv abs/2107.03904 (2021): n. pag.

  36. Park, Sangjoon, Gwanghyun Kim, Yujin Oh, Joon Beom Seo, Sang Min Lee, Jin Hwan Kim, Sungjun Moon, Jae-Kwang Lim and J. C. Ye. (2021) “Vision Transformer for COVID-19 CXR Diagnosis using Chest X-ray Feature Corpus.” ArXiv abs/2103.07055: n. pag.

  37. Cao, Kai, Tao Deng, Chuanlin Zhang, Limeng Lu and Lin Li. (2022) “A CNN-transformer fusion network for COVID-19 CXR image classification.” PLOS ONE 17.

  38. Fan X, Feng X, Dong Y, Hou H (2022) COVID-19 CT image recognition algorithm based on transformer and CNN. Displays 72:102150–102150

    Article  Google Scholar 

  39. Hinton, Geoffrey E., Oriol Vinyals and Jeffrey Dean. (2015) “Distilling the Knowledge in a Neural Network.” ArXiv abs/1503.02531: n. pag.

  40. Zhang D, Yu Y, Chen F, et al. Decomposing Logits Distillation for Incremental Named Entity Recognition[C]//Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2023: 1919–1923.

  41. Zhang L, Song J, Gao A, et al. (2019) Be your own teacher: Improve the performance of convolutional neural networks via self distillation//Proceedings of the IEEE/CVF international conference on computer vision 3713–3722.

  42. Zou P, Teng Y, Niu T. (2022) Multi-scale Feature Extraction and Fusion for Online knowledge distillation//International Conference on Artificial Neural Networks. Cham: Springer Nature Switzerland: 126–138.

  43. Li, Jingxing, Zhang Yang and Yifan Yu. (2021) “A Medical AI Diagnosis Platform Based on Vision Transformer for Coronavirus.” 2021 IEEE International Conference on Computer Science, Electronic Information Engineering and Intelligent Control Technology (CEI): 246–252.

  44. Jin Y, Wang J, Lin D. (2023) Multi-Level Logit Distillation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.: 24276–24285.

  45. Gonçalves, Tiago, Isabel Rio-Torto, Luís Filipe Teixeira and Jaime dos Santos Cardoso. (2022) “A survey on attention mechanisms for medical applications: are we moving toward better Algorithms?” IEEE Access 10: 98909–98935.

  46. Wan D, Lu R, Shen S et al (2023) Mixed local channel attention for object detection. Eng Appl Artif Intell 123:106442

    Article  Google Scholar 

  47. Guo, Meng-Hao, Tianhan Xu, Jiangjiang Liu, Zheng-Ning Liu, Peng-Tao Jiang, Tai-Jiang Mu, Song-Hai Zhang, Ralph Robert Martin, Ming-Ming Cheng and Shimin Hu. “Attention mechanisms in computer vision: A survey.” Computational Visual Media 8 (2021): 331 - 368.

  48. Wang Q, Wu B, Zhu P, et al. (2020) ECA-Net: Efficient channel attention for deep convolutional neural network//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition.: 11534–11542.

  49. Woo S, Park J, Lee J Y, et al. Cbam: convolutional block attention module[C]//proceedings of the European conference on computer vision (ECCV). 2018: 3–19.

  50. Liu J, Qiao H, Yang L et al (2023) Improved lightweight YOLOv4 foreign object detection method for conveyor belts combined with CBAM. Appl Sci 13(14):8465

    Article  Google Scholar 

  51. Hou, Qibin, Daquan Zhou and Jiashi Feng. (2021) “Coordinate attention for efficient mobile network design.” IEEE/CVF Conference on computer vision and pattern recognition (CVPR): 13708–13717.

  52. Ambita, Ara Abigail E., Eujene Nikka V. Boquio and Prospero C. Naval. (2021) “COViT-GAN: vision transformer for COVID-19 detection in CT Scan imageswith Self-Attention GAN for data augmentation.” International conference on artificial neural Networks.

  53. Han Z, Wei B, Hong Y, Li T, Cong J, Zhu X, Wei H-L, Zhang W (2020) Accurate screening of COVID-19 using attention-based deep 3D multiple instance learning. IEEE Trans Med Imaging 39:2584–2594

    Article  Google Scholar 

  54. Poličar PG, Stražar M, Zupan B (2023) Embedding to reference t-SNE space addresses batch effects in single-cell classification. Mach Learn 112(2):721–740

    Article  MathSciNet  Google Scholar 

  55. Suara S, Jha A, Sinha P, et al. (2023) Is Grad-CAM Explainable in Medical Images?. arXiv preprint arXiv:2307.10506.

  56. Preechakul, Konpat, Sira Sriswasdi, Boonserm Kijsirikul and Ekapol Chuangsuwanich. (2022) “Improved image classification explainability with high-accuracy heatmaps.” iScience 25.

  57. Al-Waisy AS, Al-Fahdawi S, Mohammed MA et al (2023) COVID-CheXNet: hybrid deep learning framework for identifying COVID-19 virus in chest X-rays images. Soft Comput 27(5):2657–2672

    Article  Google Scholar 

  58. Han, Yan, Greg Holste, Ying Ding, Ahmed Tewfik, Yifan Peng and Zhangyang Wang. (2022) “Radiomics-Guided Global-Local Transformer for Weakly Supervised Pathology Localization in Chest X-Rays.” IEEE transactions on medical imaging PP.

Download references

Acknowledgements

We should gratitude the authors for collecting and organizing the three datasets[25–27]. The authors also would like to thank the editor and the reviewers for their helpful suggestions. This research was partly funded by the National Natural Science Foundation of China (Grant Nos. 62361027 and 62161011), the Key Research and Development Plan of Jiangxi Provincial Science and Technology Department (Key Project) (Grant No. 20223BBE51036), the Humanity and Social Science Fund of Ministry of Education of China (Grant No. 23YJA870005), the Natural Science Foundation of Jiangxi Provincial Department of Science and Technology (Grant No. 20232BAB202004), the Humanity and Social Science Foundation of the Jiangxi Province (Grant No. 22TQ01), the Training Plan for Academic and Technical Leaders of Major Disciplines of Jiangxi Province (Grant No. 20204BCJL23035), the Science and Technology Projects of Jiangxi Provincial Department of Education (Grant nos. GJJ200628 and GJJ2200639), the Humanity and Social Science Foundation of Jiangxi University (Grant Nos. TQ22102 and TQ21203), and the Graduate Innovation Foundation Project of Jiangxi Province (Grant No. YC2022-s497).

Author information

Authors and Affiliations

  1. School of Software, East China Jiaotong University, Nanchang, China

    Hongbin Zhang, Lang Hu, Weinan Liang, Zhijie Li, Meng Yuan & Xiong Li

  2. School of Information Engineering, East China Jiaotong University, Nanchang, China

    Yiyuan Ye

  3. School of Information Science and Technology, Nantong University, Nantong, China

    Zelin Wang

  4. School of Interpreting and Translation Studies, Guangdong University of Foreign Studies, Guangzhou, China

    Yafeng Ren

Authors
  1. Hongbin Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lang Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Weinan Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhijie Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Meng Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yiyuan Ye
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Zelin Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Yafeng Ren
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Xiong Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

HZ: Conceptualization, Validation, Investigation, Writing—Review and Editing, Supervision, Project Administration, Funding Acquisition. LH: Software, Validation, Methodology, Formal Analysis, Investigation, Resources, Visualization, Data Curation, Writing—Original Draft. WL: Resources, Formal Analysis. ZL: Software, Validation, Methodology. MY: Conceptualization, Data Curation. YY: Validation, Investigation. ZW: Conceptualization. YR: Methodology. XL: Formal Analysis.

Corresponding author

Correspondence to Hongbin Zhang.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, H., Hu, L., Liang, W. et al. BCT-OFD: bridging CNN and transformer via online feature distillation for COVID-19 image recognition. Int. J. Mach. Learn. & Cyber. 15, 2347–2366 (2024). https://doi.org/10.1007/s13042-023-02034-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-023-02034-x

Keywords

Search

Navigation