research-article

Multiple organ-specific cancers classification from PET/CT images using deep learning

Authors:

Yongxiong Wang,

Zhe WangAuthors Info & Claims

Multimedia Tools and Applications, Volume 81, Issue 12

Pages 16133 - 16154

https://doi.org/10.1007/s11042-022-12055-3

Published: 01 May 2022 Publication History

Abstract

As the number of cancer cases increases and the popularity of positron emission tomography/computed tomography (PET/CT), an automated cancer screening system that can assist radiologists is desired. The existing methods based on PET/CT images are mostly limited to one specific organ. In this paper, a method based on deep learning is proposed that can classify multiple organ-specific cancer to assist radiologists. In the classification model, we introduce a modality fusion module to fuse PET images, CT images, and the segmentation result of multi-organs. The segmentation result of organs is used as the attention map which can force the network to learn organ-related features from the whole-body PET/CT image. Since the low-dose computed tomography (LDCT) images are widely used in PET/CT, a grayscale transformation strategy and a double-level V-net are proposed to segment multiple organs in LDCT. The proposed grayscale transformation strategy solves insufficient annotated data, and the double-level V-net strengthens the context information of images. The proposed method can classify PET/CT images into six screening classes (health, esophageal cancer, gastric cancer, liver cancer, pancreatic cancer, and lung cancer). The experimental results demonstrate that the F-score of the classifier reaches 82.3%, indicating that it can assists radiologists in screening cancers.

References

[1]

Ahmed HAG, Al Ramlawy S, and Khalil MEstimation of the most appropriate threshold for lesion delineation and volumetric quantification in 18F-FDG PETArab J Nucl Sci Appl201952169-180https://doi.org/10.21608/ajnsa.2019.3449.1080

[2]

Arnal J, Chillarón M, Parcero E, et al.A parallel fuzzy algorithm for real-time medical image enhancementInt J Fuzzy Syst2020222599-2612https://doi.org/10.1007/s40815-020-00953-3

[3]

Bi L, Kim J, Feng D, Fulham M (2014) Multi-stage thresholded region classification for whole-body PET-CT lymphoma studies. In: International conference on medical image computing and computer-assisted intervention., pp 569–576

[4]

Bi L, Kim J, Kumar A, et al.Automatic detection and classification of regions of FDG uptake in whole-body PET-CT lymphoma studiesComput Med Imaging Graph2017603-10https://doi.org/10.1016/j.compmedimag.2016.11.008

[5]

Cao Z, Sun C, Wang W, et al.Multi-modality fusion learning for the automatic diagnosis of optic neuropathyPattern Recognit Lett202114258-64https://doi.org/10.1016/j.patrec.2020.12.009

[6]

Chauvie S, Bertone E, Bergesio F, et al.Automatic liver detection and standardised uptake value evaluation in whole-body positron emission tomography/computed tomography scansComput Methods Programs Biomed201815647-52https://doi.org/10.1016/j.cmpb.2017.12.026

[7]

Christ PF, Elshaer MEA, Ettlinger F et al (2016) Automatic liver and lesion segmentation in CT using cascaded fully convolutional neural networks and 3D conditional random fields. In: International conference on medical image computing and computer-assisted intervention., pp 415–423

[8]

Conze P-H, Kavur AE, Cornec-Le Gall E et al (2021) Abdominal multi-organ segmentation with cascaded convolutional and adversarial deep networks. Artif Intell Med :102109.

[9]

Dolz J, Ayed I Ben, Desrosiers C (2018) Dense multi-path U-Net for ischemic stroke lesion segmentation in multiple image modalities. In: International MICCAI brainlesion workshop., pp 271–282

[10]

Elia I, Schmieder R, Christen S, Fendt S-M (2015) Organ-specific cancer metabolism and its potential for therapy. In: Metabolic control. Springer, pp 321–353

[11]

Eslami M, Tabarestani S, Adjouadi M (2020) Joint low dose CT denoising and kidney segmentation. In: 2020 IEEE 17th international symposium on biomedical imaging workshops (ISBI Workshops)., pp 1–4

[12]

Fu X, Bi L, Kumar A et al (2021) Multimodal spatial attention module for targeting multimodal PET-CT lung tumor segmentation. IEEE J Biomed Heal Inf.

[13]

Gibson E, Giganti F, Hu Y, et al.Automatic multi-organ segmentation on abdominal CT with dense v-networksIEEE Trans Med Imaging2018371822-1834https://doi.org/10.1109/TMI.2018.2806309

[14]

Han M, Yao G, Zhang W et al (2019) Segmentation of CT thoracic organs by multi-resolution VB-nets. In: SegTHOR@ ISBI

[15]

He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition. IEEE Computer Society, pp 770–778

[16]

Hellwig D, Graeter TP, Ukena D, et al.18F-FDG PET for mediastinal staging of lung cancer: which SUV threshold makes sense?J Nucl Med2007481761-1766https://doi.org/10.2967/jnumed.107.044362

[17]

Hofman MS and Hicks RJHow we read oncologic FDG PET/CTCancer Imaging2016161-14https://doi.org/10.1186/s40644-016-0091-3

[18]

Hu S, Hoffman EA, and Reinhardt JMAutomatic lung segmentation for accurate quantitation of volumetric X-ray CT imagesIEEE Trans Med Imaging200120490-498https://doi.org/10.1109/42.929615

[19]

Ioffe S, Szegedy C (2015) Batch normalization: Accelerating deep network training by reducing internal covariate shift. In: International conference on machine learning, pp 448–456

[20]

Isensee F, Jaeger PF, Kohl SAA, et al.nnU-Net: a self-configuring method for deep learning-based biomedical image segmentationNat Methods202118203-211https://doi.org/10.1038/s41592-020-01008-z

[21]

Ishikawa S, Hiraka T, Kirii K, et al.Relationship between standard uptake values of positron emission tomography/computed tomography and salivary metabolites in oral cancer: a pilot studyJ Clin Med202093958https://doi.org/10.3390/jcm9123958

[22]

Jinnai S, Yamazaki N, Hirano Y, et al.The development of a skin cancer classification system for pigmented skin lesions using deep learningBiomolecules2020101123https://doi.org/10.3390/biom10081123

[23]

Kawauchi K, Furuya S, Hirata K, et al.A convolutional neural network-based system to classify patients using FDG PET/CT examinationsBMC Cancer2020201-10https://doi.org/10.1186/s12885-020-6694-x

[24]

Kwee TC, Cheng G, Lam MGEH, et al.SUV max of 2.5 should not be embraced as a magic threshold for separating benign from malignant lesionsEur J Nucl Med Mol Imaging2013401475-1477https://doi.org/10.1007/s00259-013-2484-x

[25]

Landman B, Xu Z, Igelsias JE et al (2015) MICCAI multi-atlas labeling beyond the cranial vault–workshop and challenge

[26]

Lei Y, He X, Yao J, et al.Breast tumor segmentation in 3D automatic breast ultrasound using Mask scoring R-CNNMed Phys202148204-214https://doi.org/10.1002/mp.14569

[27]

Li L, Zhao X, Lu W, and Tan SDeep learning for variational multimodality tumor segmentation in PET/CTNeurocomputing2020392277-295https://doi.org/10.1016/j.neucom.2018.10.099

[28]

Li Y, Zhao J, Lv Z, and Li JMedical image fusion method by deep learningInt J Cogn Comput Eng2021221-29https://doi.org/10.1016/j.ijcce.2020.12.004

[29]

Liao X, Li K, and Yin JSeparable data hiding in encrypted image based on compressive sensing and discrete fourier transformMultimed Tools Appl20177620739-20753https://doi.org/10.1007/s11042-016-3971-4

[30]

Liao X, Yin J, Guo S, et al.Medical JPEG image steganography based on preserving inter-block dependenciesComput Electr Eng201867320-329https://doi.org/10.1016/j.compeleceng.2017.08.020

[31]

Liu Z, Wang S, Dong D, et al.The applications of radiomics in precision diagnosis and treatment of oncology: opportunities and challengesTheranostics201991303-1322https://doi.org/10.7150/thno.30309

[32]

Miller KD, Goding Sauer A, Ortiz AP, et al.Cancer statistics for hispanics/latinos, 2018CA Cancer J Clin201868425-445https://doi.org/10.3322/caac.21494

[33]

Milletari F, Navab N, Ahmadi S-A (2016) V-net: Fully convolutional neural networks for volumetric medical image segmentation. In: 2016 fourth international conference on 3D vision (3DV)., pp 565–571

[34]

Nabi HA and Zubeldia JM Clinical applications of (18)F-FDG in oncology J Nucl Med Technol 2002 30 1-3

[35]

Ning X, Duan P, Li W, and Zhang SReal-time 3D face alignment using an encoder-decoder network with an efficient deconvolution layerIEEE Signal Process Lett2020271944-1948https://doi.org/10.1109/LSP.2020.3032277

[36]

Ning X, Gong K, Li W et al (2020) Feature refinement and filter network for person re-identification. IEEE Trans Circ Syst Video Technol 1.

[37]

Ning X, Gong K, Li W, Zhang L (2020) JWSAA: Joint weak saliency and attention aware for person re-identification. Neurocomputing.

[38]

Park JY, Han S, Ka HI, et al.Silent mating-type information regulation 2 homolog 1 overexpression is an important strategy for the survival of adapted suspension tumor cellsCancer Sci20191102773https://doi.org/10.1111/cas.14147

[39]

Radosavovic I, Kosaraju RP, Girshick R et al (2020) Designing network design spaces. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 10428–10436

[40]

Razzak MI, Naz S, Zaib A (2018) Deep learning for medical image processing: Overview, challenges and the future. Classif BioApps :323–350.

[41]

Ronneberger O, Fischer P, Brox T (2015) U-net: Convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer-assisted intervention., pp 234–241

[42]

Roth GA, Abate D, Abate KH, et al.Global, regional, and national age-sex-specific mortality for 282 causes of death in 195 countries and territories, 1980–2017: a systematic analysis for the Global Burden of Disease Study 2017Lancet20183921736-1788https://doi.org/10.1016/S0140-6736(18)32203-7

[43]

Roth HR, Shen C, Oda H et al (2018) A multi-scale pyramid of 3D fully convolutional networks for abdominal multi-organ segmentation. In: International conference on medical image computing and computer-assisted intervention., pp 417–425

[44]

Shalini G, Remya RS, Gayathri G et al (2011) Organ specific cancers-recent advances in diagnosis and treatment. Cancer Sci Ther :S17.

[45]

Shankar A, Saini D, Dubey A, et al.Feasibility of lung cancer screening in developing countries: challenges, opportunities and way forwardTransl lung cancer Res20198S106https://doi.org/10.21037/tlcr.2019.03.03

[46]

Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:14091556

[47]

Sudre CH, Li W, Vercauteren T et al (2017) Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations. In: Cardoso M. et al (eds) Deep learning in medical image analysis and multimodal learning for clinical decision support. Springer, Cham, pp 240–248

[48]

Tan M, Le Q (2019) Efficientnet: Rethinking model scaling for convolutional neural networks. In: International conference on machine learning, pp 6105–6114

[49]

Tang S, Huang G, Liu J, et al.Usefulness of 18F-FDG PET, combined FDG-PET/CT and EUS in diagnosing primary pancreatic carcinoma: a meta-analysisEur J Radiol201178142-150https://doi.org/10.1016/j.ejrad.2009.09.026

[50]

Trivizakis E, Manikis GC, Nikiforaki K, et al.Extending 2-D convolutional neural networks to 3-D for advancing deep learning cancer classification with application to MRI liver tumor differentiationIEEE J Biomed Heal Informatics201923923-930https://doi.org/10.1109/JBHI.2018.2886276

[51]

Tu Z and Bai XAuto-context and its application to high-level vision tasks and 3d brain image segmentationIEEE Trans Pattern Anal Mach Intell2010321744-1757https://doi.org/10.1109/TPAMI.2009.186

[52]

Wang H, Zhou Z, Li Y, et al.Comparison of machine learning methods for classifying mediastinal lymph node metastasis of non-small cell lung cancer from 18 F-FDG PET/CT imagesEJNMMI Res2017711https://doi.org/10.1186/s13550-017-0260-9

[53]

Wang T, Lei Y, Akin-Akintayo OO et al (2021) Prostate and tumor segmentation on PET/CT using Dual Mask R-CNN. In: Medical imaging 2021: Biomedical applications in molecular, structural, and functional imaging., p 116000S

[54]

Wolz R, Chu C, Misawa K, et al.Automated abdominal multi-organ segmentation with subject-specific atlas generationIEEE Trans Med Imaging2013321723-1730https://doi.org/10.1109/TMI.2013.2265805

[55]

Xu X, Zhou F, Liu B, et al.Efficient multiple organ localization in CT image using 3D region proposal networkIEEE Trans Med Imaging2019381885-1898https://doi.org/10.1109/TMI.2019.2894854

[56]

Xu Z, Zeng S, Gong Z, and Yan YExosome-based immunotherapy: a promising approach for cancer treatmentMol Cancer202019160https://doi.org/10.1186/s12943-020-01278-3

[57]

Xu Z, Lee CP, Heinrich MP, et al.Evaluation of six registration methods for the human abdomen on clinically acquired CTIEEE Trans Biomed Eng2016631563-1572https://doi.org/10.1109/TBME.2016.2574816

[58]

Zaidi H and El Naqa IPET-guided delineation of radiation therapy treatment volumes: a survey of image segmentation techniquesEur J Nucl Med Mol Imaging2010372165-2187https://doi.org/10.1007/s00259-010-1423-3

[59]

Zhang J, Wang W, Lu C, et al.Lightweight deep network for traffic sign classificationAnn Telecommun202075369-379https://doi.org/10.1007/s12243-019-00731-9

[60]

Zhang L, Li X, Hu Q (2012) Automatic rib segmentation in chest CT volume data. In: Proceedings - 2012 international conference on biomedical engineering and biotechnology., pp 750–753

[61]

Zhang Z, Ren J, Tao X, et al.Automatic segmentation of pulmonary lobes on low-dose computed tomography using deep learningAnn Transl Med20219291https://doi.org/10.21037/atm-20-5060

Cited By

Peng YLuan PTu HLi XZhou P(2023)Pulmonary fissure segmentation in CT images based on ODoS filter and shape featuresMultimedia Tools and Applications10.1007/s11042-023-14931-y82:22(34959-34980)Online publication date: 1-Sep-2023
https://dl.acm.org/doi/10.1007/s11042-023-14931-y
Nithiyaraj ESelvaraj A(2022)CTSC-Net: an effectual CT slice classification network to categorize organ and non-organ slices from a 3-D CT imageNeural Computing and Applications10.1007/s00521-022-07701-834:24(22141-22156)Online publication date: 1-Dec-2022
https://dl.acm.org/doi/10.1007/s00521-022-07701-8

Index Terms

Multiple organ-specific cancers classification from PET/CT images using deep learning
1. Applied computing
  1. Life and medical sciences
2. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
  2. Machine learning
    1. Learning paradigms
      1. Supervised learning
    2. Machine learning approaches
      1. Neural networks

Index terms have been assigned to the content through auto-classification.

Recommendations

A PET/CT directed, 3D ultrasound-guided biopsy system for prostate cancer
MICCAI'11: Proceedings of the 2011 international conference on Prostate cancer imaging: image analysis and image-guided interventions

Prostate cancer affects 1 in 6 men in the USA. Systematic transrectal ultrasound (TRUS)-guided biopsy is the standard method for a definitive diagnosis of prostate cancer. However, this "blind" biopsy approach can miss at least 20% of prostate cancers. ...
Deep PET/CT Fusion with Dempster-Shafer Theory for Lymphoma Segmentation
Machine Learning in Medical Imaging
Abstract
Lymphoma detection and segmentation from whole-body Positron Emission Tomography/Computed Tomography (PET/CT) volumes are crucial for surgical indication and radiotherapy. Designing automatic segmentation methods capable of effectively exploiting ...
Thoracic cavity definition for 3D PET/CT analysis and visualization

X-ray computed tomography (CT) and positron emission tomography (PET) serve as the standard imaging modalities for lung-cancer management. CT gives anatomical details on diagnostic regions of interest (ROIs), while PET gives highly specific functional ...

Comments

Information & Contributors

Information

Published In

cover image Multimedia Tools and Applications

Multimedia Tools and Applications Volume 81, Issue 12

May 2022

1382 pages

ISSN:1380-7501

Issue’s Table of Contents

© The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2022.

Publisher

Kluwer Academic Publishers

United States

Publication History

Published: 01 May 2022

Accepted: 04 January 2022

Revision received: 30 May 2021

Received: 04 February 2021

Author Tags

Qualifiers

Research-article

Funding Sources

National Natural Science Foundation of China

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 12 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Peng YLuan PTu HLi XZhou P(2023)Pulmonary fissure segmentation in CT images based on ODoS filter and shape featuresMultimedia Tools and Applications10.1007/s11042-023-14931-y82:22(34959-34980)Online publication date: 1-Sep-2023
https://dl.acm.org/doi/10.1007/s11042-023-14931-y
Nithiyaraj ESelvaraj A(2022)CTSC-Net: an effectual CT slice classification network to categorize organ and non-organ slices from a 3-D CT imageNeural Computing and Applications10.1007/s00521-022-07701-834:24(22141-22156)Online publication date: 1-Dec-2022
https://dl.acm.org/doi/10.1007/s00521-022-07701-8

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents