Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content

Advertisement

Springer Nature Link
Log in

DeepCraftFuse: visual and deeply-learnable features work better together for esophageal cancer detection in patients with Barrett’s esophagus

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Limitations in computer-assisted diagnosis include lack of labeled data and inability to model the relation between what experts see and what computers learn. Even though artificial intelligence and machine learning have demonstrated remarkable performances in medical image computing, their accountability and transparency level must be improved to transfer this success into clinical practice. The reliability of machine learning decisions must be explained and interpreted, especially for supporting the medical diagnosis. While deep learning techniques are broad so that unseen information might help learn patterns of interest, human insights to describe objects of interest help in decision-making. This paper proposes a novel approach, DeepCraftFuse, to address the challenge of combining information provided by deep networks with visual-based features to significantly enhance the correct identification of cancerous tissues in patients affected with Barrett’s esophagus (BE). We demonstrate that DeepCraftFuse outperforms state-of-the-art techniques on private and public datasets, reaching results of around 95% when distinguishing patients affected by BE that is either positive or negative to esophageal cancer.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data availability

The data that support this research are available on request from the authors. The data are not publicly available due to privacy or ethical restrictions.

Notes

  1. One physician manually annotated the dataset.

  2. https://endovissub-barrett.grand-challenge.org.

  3. We used such a zooming rate upper boundary to avoid missing important details of the esophagus area.

  4. The images to be mirrored were chosen randomly. Apart from the number of images generated artificially, one-third accounts for rotation, one-third for mirroring, and the remaining stands for zoomed images.

References

  1. Lagergren J, Lagergren P (2010) Oesophageal cancer. BMJ 341

  2. Dent J (2011) Barrett’s esophagus: a historical perspective, an update on core practicalities and predictions on future evolutions of management. J Gastroenterol Hepatol 26:11–30

    Article  Google Scholar 

  3. Sharma P, Bergman JJGHM, Goda K, Kato M et al (2016) Development and validation of a classification system to identify high-grade dysplasia and esophageal adenocarcinoma in barrett’s esophagus using narrow-band imaging. Gastroenterology 150(3):591–598

    Article  Google Scholar 

  4. Phoa KN, Pouw RE, Bisschops R, Pech O, Ragunath K, Weusten BLAM et al (2016) Multimodality endoscopic eradication for neoplastic Barrett oesophagus: results of an European Multicentre Study (EURO-II). Gut 65(4):555–562

    Article  Google Scholar 

  5. Shaheen NJ, Sharma P, Overhold BF et al (2009) Radiofrequency ablation in Barrett’s esophagus with dysplasia. N Engl J Med 360(22):2277–2288

    Article  Google Scholar 

  6. Johnson MH, Eastone JA, Horwhat JD et al (2005) Cryoablation of Barrett’s esophagus: a pilot study. Gastrointest Endosc 62:842–848

    Article  Google Scholar 

  7. Ghatwary N, Zolgharni M, Ye X (2019) Early esophageal adenocarcinoma detection using deep learning methods. Int J Comput Assist Radiol Surg 14(4):611–621

    Article  Google Scholar 

  8. Ghatwary N, Ye X, Zolgharni M (2019) Esophageal abnormality detection using densenet based faster R-CNN with gabor features. IEEE Access 7:84374–84385

    Article  Google Scholar 

  9. van der Putten J, de Groof J, Struyvenberg M, Boers T, Fockens K, Curvers W, Schoon E, Bergman J, van der Sommen F, de With PHN (2020) Multi-stage domain-specific pretraining for improved detection and localization of Barrett’s neoplasia: A comprehensive clinically validated study. Artif Intell Med 107:101914

    Article  Google Scholar 

  10. Ebigbo A, Mendel R, Probst A, Manzeneder J, de Souza LA, Papa JP, Palm C, Messmann H (2019) Computer-aided diagnosis using deep learning in the evaluation of early oesophageal adenocarcinoma. Gut 68(7):1143–1145. https://doi.org/10.1136/gutjnl-2018-317573

    Article  Google Scholar 

  11. Ebigbo A, Mendel R, Probst A, Manzeneder J, Prinz F, Souza LA Jr, Papa J, Palm C, Messmann H (2020) Real-time use of artificial intelligence in the evaluation of cancer in Barrett’s oesophagus. Gut 69(4):615–616. https://doi.org/10.1136/gutjnl-2019-319460

    Article  Google Scholar 

  12. Souza LA Jr, Mendel R, Strasser S, Ebigbo A, Probst A, Messmann H, Papa JP, Palm C (2021) Convolutional neural networks for the evaluation of cancer in Barrett’s esophagus: explainable AI to lighten up the black-box. Comput Biol Med 135:104578

    Article  Google Scholar 

  13. Souza LA Jr, Passos LA, Mendel R, Ebigbo A, Probst A, Messmann H, Palm C, Papa JP (2020) Assisting Barrett’s esophagus identification using endoscopic data augmentation based on generative adversarial networks. Comput Biol Med 126:104029

    Article  Google Scholar 

  14. Souza Jr, LA, Passos LA, Mendel R, Ebigbo A, Probst A, Messmann H, Palm C, Papa JP (2021) Fine-tuning generative adversarial networks using metaheuristics. In: Bildverarbeitung Für die Medizin (BVM), pp 205–210

  15. Souza Jr., LA, Ebigbo A, Probst A, Messmann H, Papa JP, Mendel R, Palm C (2018) Barrett’s esophagus identification using color co-occurrence matrices. In: Conference on graphics, patterns and images (SIBGRAPI), pp 166–173 . https://doi.org/10.1109/SIBGRAPI.2018.00028

  16. Souza LA Jr, Afonso LCS, Ebigbo A, Probst A, Messmann H, Mendel R, Hook C, Palm C, Papa JP (2019) Learning visual representations with optimum-path forest and its applications to Barrett’s esophagus and adenocarcinoma diagnosis. Neural Comput Appl 32:759–775

    Article  Google Scholar 

  17. Souza Jr. LA, Hook C, Papa JP, Palm (2017) Barrett’s esophagus analysis using SURF features. In: Bildverarbeitung Für die Medizin (BVM), pp 141–146

  18. Mendel R, Ebigbo A, Probst A, Messmann H, Palm C (2017) Barrett’s esophagus analysis using convolutional neural networks. In: Bildverarbeitung Für die Medizin (BVM) pp 80–85

  19. Horie Y, Yoshio T, Aoyama K, Yoshimizu S, Horiuchi Y, Ishiyama A, Hirasawa T, Tsuchida T, Ozawa T, Ishihara S (2019) Diagnostic outcomes of esophageal cancer by artificial intelligence using convolutional neural networks. Gastrointest Endosc 89(1):25–32

    Article  Google Scholar 

  20. Hassan AR, Haque MA (2015) Computer-aided gastrointestinal hemorrhage detection in wireless capsule endoscopy videos. Comput Biol Med 122:341–353

    Google Scholar 

  21. Souza LA Jr, Palm C, Mendel R, Hook C, Ebigbo A, Probst A, Messmann H, Weber S, Papa JP (2018) A survey on Barrett’s esophagus analysis using machine learning. Comput Biol Med 96:203–213. https://doi.org/10.1016/j.compbiomed.2018.03.014

    Article  Google Scholar 

  22. Souza Jr. LA, Afonso LCS, Palm C, Papa JP (2017) Barrett’s esophagus identification using optimum-path forest. In: Conference on graphics, patterns and images (SIBGRAPI), pp 308–314

  23. Passos LA, Souza LA Jr, Mendel R, Ebigbo A, Probst A, Messmann H, Palm C, Papa JP (2019) Barrett’s esophagus analysis using infinity restricted Boltzmann machines. J Vis Commun Image R 59:475–485. https://doi.org/10.1016/j.jvcir.2019.01.043

    Article  Google Scholar 

  24. Bay H, Ess A, Tuytelaars T, Van Gool L (2008) Speeded-up robust features (SURF). Comput Vis Image Underst 110(3):346–359. https://doi.org/10.1016/j.cviu.2007.09.014

    Article  Google Scholar 

  25. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110. https://doi.org/10.1023/B:VISI.0000029664.99615.94

    Article  Google Scholar 

  26. Papa JP, Falcão AX, Suzuki CTN (2009) Supervised pattern classification based on optimum-path forest. Int J Imaging Syst Technol 19(2):120–131. https://doi.org/10.1002/ima.20188

    Article  Google Scholar 

  27. Papa JP, Falcão AX, Albuquerque VHC, Tavares JMRS (2012) Efficient supervised optimum-path forest classification for large datasets. Pattern Recognit 45(1):512–520. https://doi.org/10.1016/j.patcog.2011.07.013

    Article  Google Scholar 

  28. Peng X, Gao X, Li X (2018) On better training the infinite restricted Boltzmann machines. Mach Learn 107(6):943–968. https://doi.org/10.1007/s10994-018-5696-2

    Article  MathSciNet  Google Scholar 

  29. de Groof AJ, Struyvenberg MR, van der Putten J, van der Sommen F, Fockens KN, Curvers WL, Zinger S, Pouw RE, Coron E, Baldaque-Silva F, Pech O, Weusten B, Meining A, Neuhaus H, Bisschops R, Dent J, Schoon EJ, de With PH, Bergman JJ (2020) Deep-learning system detects neoplasia in patients with Barrett’s esophagus with higher accuracy than endoscopists in a multistep training and validation study with benchmarking. Gastroenterology 158(4):915–9294. https://doi.org/10.1053/j.gastro.2019.11.030

    Article  Google Scholar 

  30. van der Putten J, Struyvenberg M, de Groof J, Scheeve T, Curvers W, Schoon E, Bergman JJGHM, de With PHN, van der Sommen F (2020) Deep principal dimension encoding for the classification of early neoplasia in Barrett’s esophagus with volumetric laser endomicroscopy. Comput Med Imagimg Graph 80:101701. https://doi.org/10.1016/j.compmedimag.2020.101701

    Article  Google Scholar 

  31. van der Putten J, de Groof J, Struyvenberg M, Boers T, Fockens K, Curvers W, Schoon E, Bergman J, van der Sommen F, de With PHN (2020) Multi-stage domain-specific pretraining for improved detection and localization of Barrett’s neoplasia: a comprehensive clinically validated study. Artif Intell Med 107:101914. https://doi.org/10.1016/j.artmed.2020.101914

    Article  Google Scholar 

  32. Xie Y, Gao G, Chen XA (2019) Outlining the design space of explainable intelligent systems for medical diagnosis. CoRR arXiv:1902.06019

  33. Alcantarilla P, Nuevo J, Bartoli A (2013) Fast explicit diffusion for accelerated features in nonlinear scale spaces. In: British machine vision conference (BMVC), pp 13–11311

  34. Wilcoxon F (1945) Individual comparisons by ranking methods. Biom Bull 1(6):80–83

    Article  Google Scholar 

  35. He K, Zhang X, Ren S, Sun J (2015) Deep residual learning for image recognition. CoRR arXiv:1512.03385

  36. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp 770–778 . https://doi.org/10.1109/CVPR.2016.90

  37. Liu S, Deng W (2015) Very deep convolutional neural network based image classification using small training sample size. In: 2015 3rd IAPR Asian conference on pattern recognition (ACPR), pp 730–734 . https://doi.org/10.1109/ACPR.2015.7486599

  38. Huang G, Liu Z, Weinberger KQ (2016) Densely connected convolutional networks. CoRR arxiv:1608.06993

  39. van der Sommen F, Zinger S, Curvers WL et al (2016) Computer-aided detection of early neoplastic lesions in Barret’s esophagus. Endoscopy 48(7):617–624

    Article  Google Scholar 

  40. Riel SV, van der Sommen F, Zinger S, Schoon EJ, de With PHN (2018) Automatic detection of early esophageal cancer with CNNs using transfer learning. In: IEEE international conference on image processing (ICIP), pp 1383–1387

  41. Ohmori M, Ishihara R, Aoyama K, Nakagawa K, Iwagami H, Matsuura N, Shichijo S, Yamamoto K, Nagaike K, Nakahara M, Inoue T, Aoi K, Okada H, Tada T (2019) Endoscopic detection and differentiation of esophageal lesions using a deep neural network. Gastrointest Endosc 91(2):301–3091

    Article  Google Scholar 

  42. Hou W, Wang L, Cai S, Lin Z, Yu R, Qin J (2021) Early neoplasia identification in Barrett’s esophagus via attentive hierarchical aggregation and self-distillation. Med Image Anal 72:102092

    Article  Google Scholar 

  43. Gehrung M, Crispin-Ortuzar M, Berman A, O’Donovan M, Fitzgerald R, Markowetz F (2021) Triage-driven diagnosis of Barrett’s esophagus for early detection of esophageal adenocarcinoma using deep learning. Nat Med 27(5):833–841

    Article  Google Scholar 

Download references

Acknowledgements

The authors are greateful to Capes/Alexander von Humboldt Foundation Grant Number BEX 0581-16-0, CNPq Grants 306166/2014-3 and 307066/2017-7, FAPESP Grants 2013/07375-0, 2014/12236-1, 2016/19403-6, 2017/04847-9, and 2019/08605-5.

Author information

Authors and Affiliations

  1. Department of Informatics, Federal University of Espírito Santo, Vitória, Brazil

    Luis A. Souza Jr. & André G. C. Pacheco

  2. Regensburg Medical Image Computing (ReMIC), Ostbayerische Technische Hochschule Regensburg (OTH Regensburg), Regensburg, Germany

    Luis A. Souza Jr., Robert Mendel & Christoph Palm

  3. Department of Computing, São Paulo State University, Bauru, Brazil

    Leandro A. Passos, Marcos C. S. Santana & João Paulo Papa

  4. Department of Gastroenterology, University Hospital Augsburg, Augsburg, Germany

    Alanna Ebigbo, Andreas Probst & Helmut Messmann

Authors
  1. Luis A. Souza Jr.
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. André G. C. Pacheco
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Leandro A. Passos
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Marcos C. S. Santana
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Robert Mendel
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Alanna Ebigbo
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Andreas Probst
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Helmut Messmann
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Christoph Palm
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. João Paulo Papa
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Luis A. Souza Jr..

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Souza Jr., L.A., Pacheco, A.G.C., Passos, L.A. et al. DeepCraftFuse: visual and deeply-learnable features work better together for esophageal cancer detection in patients with Barrett’s esophagus. Neural Comput & Applic 36, 10445–10459 (2024). https://doi.org/10.1007/s00521-024-09615-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-024-09615-z

Keywords

Search

Navigation