Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
SpringerLink
Log in

PTC-CapsNet: capsule network for papillary thyroid carcinoma pathological images classification

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Automatic classification of pathological images is an important task in the thyroid carcinoma histopathological analysis. Currently, histological diagnosis has become a leading field in medical imaging computing, which can reduce the burden of pathologists and the misdiagnosis rate. At present, works on designing convocational neural networks(CNNs) for automatically classifying pathological images are increasing rapidly. However, classification of thyroid carcinoma pathological images remains challenging due to the small differences between benign and malignant observed from pathological images. The existing CNN methods cannot classify the pathological images well. Inspired by the diagnosis process observed by the pathologists and the reasoning ability of CapsNet, we propose a novel capsule network named PTC-CapsNet based on semantic features derived from multi-magnification pathological images for papillary thyroid carcinoma (PTC) pathological images classification. The proposed method is more in line with the diagnostic decision and visual cognitive of pathologist. It can deal with spatial hierarchies between complex pathological features excellently. A new loss function named penalty binary cross-entropy is designed to reduce the false negative rate. Furthermore, an eye movement dataset is established by pathologists to evaluate the performance of models subjectively, while a new metric also is proposed to verify the performance of these methods objectively. The experimental results illustrate the superiority of our method on our PTC pathological images dataset and the BreakHis dataset.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data Availability

The data are available from the corresponding author on reasonable request.

References

  1. Sung H, Ferlay J, Siegel RL, Laversanne M, Soerjomataram I, Jemal A, Bray F (2021) Global cancer statistics 2020: Globocan estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin 71(3):209–249

    Article  Google Scholar 

  2. American Cancer Society ( 2021) Cancer facts & figures 2021. American Cancer Society, Atlanta

  3. Deng Y, Li H, Wang M et al (2020) Global burden of thyroid cancer from 1990 to 2017. JAMA Network Open 3(6):208759–208759

    Article  Google Scholar 

  4. La Vecchia C, Malvezzi M, Bosetti C, Garavello W, Bertuccio P, Levi F, Negri E (2015) Thyroid cancer mortality and incidence: a global overview. Int J Cancer 136(3):2187–2195

    Article  Google Scholar 

  5. Tsujikawa T, Thibault G, Azimi V, Sivagnanam S, Banik G, Means C, Kawashima R, Clayburgh DR, Gray JW, Coussens LM, Chang YH (2019) Tumor immune microenvironment characteristics of papillary thyroid carcinoma are associated with histopathological aggressiveness and BRAF mutation status. Head Neck 41(8):2636–2646

    Article  Google Scholar 

  6. Hamilton SR, Aaltonen LA (2010) Pathology and genetics of tumours of the digestive system. Histopathology 38(6):585–585

    Google Scholar 

  7. Rubin R (2007) Rubin’s Pathology. Clinicopathologic Foundations of Medicine. Lippincott Williams & Wilkins, Philadelphia

    Google Scholar 

  8. Chankong T, Theera-Umpon N, Auephanwiriyakul S (2014) Automatic cervical cell segmentation and classification in Pap smears. Comput Methods Prog Biomed 113(2):539–556

    Article  Google Scholar 

  9. Guo P, Banerjee K, Joe Stanley R, Long R, Antani S, Thoma G, Zuna R, Frazier SR, Moss RH, Stoecker WV (2016) Nuclei-based features for uterine cervical cancer histology image analysis with fusion-based classification. IEEE J Biomed Health Informat 20(6):1595–1607

    Article  Google Scholar 

  10. Anant M, George L (2016) Image analysis and machine learning in digital pathology: challenges and opportunities. Med Image Anal 33:170–175

    Article  Google Scholar 

  11. Chen H, Han X, Fan X, Lou X, Liu H, Huang J, Yao J (2019) Rectified cross-entropy and upper transition loss for weakly supervised whole slide image classifier. In: International conference on medical image computing and computer-assisted intervention, pp 351–359

  12. Murtaza G, Shuib L, Mujtaba G, Raza G (2019) Breast cancer multi-classification through deep neural network and hierarchical classification approach. Multimed Tools Appl 79(21):15481–15511

    Google Scholar 

  13. Yu C, Chen H, Li Y, Peng Y, Li J, Yang F (2019) Breast cancer classification in pathological images based on hybrid features. Multimed Tools Appl 78(15):21325–21345

    Article  Google Scholar 

  14. Bayramoglu N, Kannala J, Heikkil J (2016) Deep learning for magnification independent breast cancer histopathology image classification. In: 2016 23rd international conference on pattern recognition (ICPR), pp 2440–2445

  15. Hou L, Samaras D, Kurc TM, Gao Y, Davis JE, Saltz JH (2016) Patch-based convolutional neural network for whole slide tissue image classification. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp 2424–2433

  16. Zhu X, Yao J, Zhu F, Huang J (2017) WSISA: making survival prediction from whole slide histopathological images. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR), pp 6855–6863

  17. Li M, Wu L, Wiliem A, Zhao K, Zhang T, Lovell BC (2019) Deep instance-level hard negative mining model for histopathology images. In: International conference on medical image computing and computer-assisted intervention, pp 514–522

  18. Yang H, Kim J-Y, Kim H, Adhikari SP (2020) Guided soft attention network for classification of breast cancer histopathology images. IEEE Trans Med Imaging 39(5):1306–1315

    Article  Google Scholar 

  19. Mercan C, Aksoy S, Mercan E, Shapiro LG, Weaver DL, Elmore JG (2018) Multi-instance multi-label learning for multi-class classification of whole slide breast histopathology images. IEEE Trans Med Imaging 37(1):316–325

    Article  Google Scholar 

  20. Huo X, Sun G, Tian S, Wang Y, Yu L, Long J, Zhang W, Li A (2024) HiFuse: hierarchical multi-scale feature fusion network for medical image classification. Biomed Signal Process Control 87:105534

    Article  Google Scholar 

  21. Zhou Y, Zhang C, Gao S (2022) Breast cancer classification from histopathological images using resolution adaptive network. IEEE Access 10:35977–35991

    Article  Google Scholar 

  22. Burcak KC, Baykan OK, Uguz H (2021) New deep convolutional neural network model for classifying breast cancer histopathological images and the hyperparameter optimization of the proposed model. J Supercomput 77(1):973–989

    Article  Google Scholar 

  23. Alruwaili M, Gouda W (2022) Automated breast cancer detection models based on transfer learning. J Supercomput 22(3):876

    Google Scholar 

  24. Arooj S, Atta-ur-Rahman, Zubair M, Khan MF, Alissa K, Khan MA, Mosavi A (2022) Breast cancer detection and classification empowered with transfer learning. Front Public Health 10:924432

  25. Vanda A, Carla S (2022) Quantum transfer learning for breast cancer detection. Quant Mach Intell 4(1):1–5

    Google Scholar 

  26. Saini M, Susan S (2023) VGGIN-Net: deep transfer network for imbalanced breast cancer dataset. IEEE/ACM Trans Comput Biol Bioinforma 20(1):752–762

    Article  Google Scholar 

  27. Liu Z, Lin Y, Cao Y, Hu H, Wei Y, Zhang Z, Lin S, Guo B (2021) Swin transformer: hierarchical vision transformer using shifted windows. In: Proceedings of the IEEE/CVF international conference on computer vision (ICCV), pp 10012–10022

  28. Afshar P, Mohammadi A, Plataniotis KN (2018) Brain tumor type classification via capsule networks. In: 2018 25th IEEE international conference on image processing (ICIP), pp 3129–3133

  29. Huang W, Zhou F (2020) DA-CapsNet: dual attention mechanism capsule network. Sci Rep 10(1):11383–11383

    Article  MathSciNet  Google Scholar 

  30. Sabour S, Frosst N, Hinton GE (2017) Dynamic routing between capsules. Adv Neural Inf Process Syst 30:3856–3866

    Google Scholar 

  31. Mobiny A, Nguyen HV (2018) Fast CapsNet for lung cancer screening. In: International conference on medical image computing and computer-assisted intervention, pp 741–749

  32. Afshar P, Plataniotis KN, Mohammadi A (2019) Capsule networks for brain tumor classification based on MRI images and coarse tumor boundaries. In: ICASSP 2019 - 2019 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp 1368–1372

  33. Toraman S, Alakus TB, Turkoglu I (2020) Convolutional capsnet: a novel artificial neural network approach to detect COVID-19 disease from X-ray images using capsule networks. Chaos, Solitons Fractals 140(C):110–122

    MathSciNet  Google Scholar 

  34. Chen L-C, Papandreou G, Kokkinos I, Murphy K, Yuille AL (2018) Deeplab: semantic image segmentation with deep convolutional nets, Atrous convolution, and fully connected CRFs. IEEE Trans Pattern Anal Mach Intell 40(4):834–848

    Article  Google Scholar 

  35. Rojas R (1996) Neural networks: a systematical introduction. Springer, Berlin

    Book  Google Scholar 

  36. Xie Y, Xia Y, Zhang J, Song Y, Feng D, Fulham M, Cai W (2019) Knowledge-based collaborative deep learning for benign-malignant lung nodule classification on chest CT. IEEE Trans Med Imaging 38(4):991–1004

    Article  Google Scholar 

  37. Yarbus AL (2013) Eye movements and vision. Springer, Berlin

    Google Scholar 

  38. De Boer PT, Kroese DP, Mannor S, Rubinstein RY (2005) A tutorial on the cross-entropy method. Ann Oper Res 134(1):19–67

    Article  MathSciNet  Google Scholar 

  39. Tang Y (2013) Deep learning using linear support vector machines. In: International conference on machine learning

  40. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp 770–778

  41. Simonyan K, Zisserman A (2016) Very deep convolutional networks for large-scale image recognition. In: International conference of learning representation, pp 115–121

  42. Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp 2818–2826

  43. Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR), pp 2261–2269

  44. Zoph B, Vasudevan V, Shlens J, Le QV (2018) Learning transferable architectures for scalable image recognition. In: 2018 IEEE/CVF conference on computer vision and pattern recognition, pp 8697–8710

  45. Xie J, Liu R, Luttrell J, Zhang C (2019) Deep learning based analysis of histopathological images of breast cancer. Front Genet 10(80)

  46. Spanhol FA, Oliveira LS, Petitjean C, Heutte L (2016) A dataset for breast cancer histopathological image classification. IEEE Trans Biomed Eng 63(7):1455–1462

    Article  Google Scholar 

  47. Du B, Qi Q, Zheng H et al (2018) Breast cancer histopathological image classification via deep active learning and confidence boosting. In: International conference on artificial neural networks (ICANN 2018), pp 109–116

  48. Gandomkar Z, Brennan PC, Mello-Thoms C (2018) A framework for distinguishing benign from malignant breast histopathological images using deep residual networks. In: 14th International workshop on breast imaging (IWBI 2018), vol 10718

  49. Han Z, Wei B, Zheng Y, Yin Y, Li K, Li S (2017) Breast cancer multi-classification from histopathological images with structured deep learning model. Sci Rep 7(1):4172–4172

    Article  Google Scholar 

  50. Nawaz M, Sewissy AA, Soliman THA (2018) Multi-class breast cancer classification using deep learning convolutional neural network. Adv Theory Simul 9(6):316–332

    Google Scholar 

  51. Alom MZ, Yakopcic C, Taha TM, Asari VK (2019) Breast cancer classification from histopathological images with inception recurrent residual convolutional neural network. J Digit Imaging 32(4):605–607

    Article  Google Scholar 

  52. Aloyayri A, Krzyzak A (2020) Breast cancer classification from histopathological images using transfer learning and deep neural networks. In: International conference on artificial intelligence and soft computing, pp 491–502

  53. Kumar A, Singh SK, Saxena S, Lakshmanan K, Sangaiah AK, Chauhan H, Shrivastava S, Singh RK (2020) Deep feature learning for histopathological image classification of canine mammary tumors and human breast cancer. Inf Sci Int J 508:405–421

    Google Scholar 

  54. Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D (2017) Grad-CAM: visual explanations from deep networks via gradient-based localization. In: 2017 IEEE international conference on computer vision (ICCV), pp 618–626

Download references

Acknowledgements

This research was funded in part by the National Natural Science Foundation of China (Grant No: 62076190, 41831072), in part by The Key Industry Innovation Chain of Shaanxi Province (Grant No: 2022ZDLGY01-11), in part by The Key Industry chain technology research project of Xi’an (Grant No: 23ZDCYJSGG0022-2023), in part by The Youth Open Project of National Space Science Data Center (Grant No: NSSDC2302005).

Author information

Authors and Affiliations

  1. School of Electronic Engineering, Xidian University, No. 2 South Taibai Road, Xi’an, 710071, Shaanxi, China

    Bing Han, Yiyuan Han, Haoran Li & Xinbo Gao

  2. Chongqing Key Laboratory of Image Cognition, Chongqing University of Posts and Telecommunications, NO.2, Chongwen Road, Chongqing, 400065, Shaanxi, China

    Xinbo Gao

Authors
  1. Bing Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yiyuan Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Haoran Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xinbo Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bing Han.

Ethics declarations

Competing interests

The authors have no competing interests to declare that are relevant to the content of this article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Han, B., Han, Y., Li, H. et al. PTC-CapsNet: capsule network for papillary thyroid carcinoma pathological images classification. Multimed Tools Appl (2024). https://doi.org/10.1007/s11042-024-18985-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11042-024-18985-4

Keywords

Search

Navigation