Abstract
Purpose
To develop a deep learning–based computer-aided diagnosis (CAD) system for use in the CT diagnosis of cervical lymph node metastasis (LNM) in patients with thyroid cancer.
Methods
A total of 995 axial CT images that included benign (n = 647) and malignant (n = 348) lymph nodes were collected from 202 patients with thyroid cancer who underwent CT for surgical planning between July 2017 and January 2018. The datasets were randomly split into training (79.0%), validation (10.5%), and test (10.5%) datasets. Eight deep convolutional neural network (CNN) models were used to classify the images into metastatic or benign lymph nodes. Pretrained networks were used on the ImageNet and the best-performing algorithm was selected. Class-specific discriminative regions were visualized with attention heatmap using a global average pooling method.
Results
The area under the ROC curve (AUROC) for the tested algorithms ranged from 0.909 to 0.953. The sensitivity, specificity, and accuracy of the best-performing algorithm were all 90.4%, respectively. Attention heatmap highlighted important subregions for further clinical review.
Conclusion
A deep learning–based CAD system could accurately classify cervical LNM in patients with thyroid cancer on preoperative CT with an AUROC of 0.953. Whether this approach has clinical utility will require evaluation in a clinical setting.
Key Points
• A deep learning–based CAD system could accurately classify cervical lymph node metastasis. The AUROC for the eight tested algorithms ranged from 0.909 to 0.953.
• Of the eight models, the ResNet50 algorithm was the best-performing model for the validation dataset with 0.953 AUROC. The sensitivity, specificity, and accuracy of the ResNet50 model were all 90.4%, respectively, in the test dataset.
• Based on its high accuracy of 90.4%, we consider that this model may be useful in a clinical setting to detect LNM on preoperative CT in patients with thyroid cancer.
Similar content being viewed by others
Abbreviations
- AUROC:
-
Area under the receiver operating characteristic curve
- CAD:
-
Computer-aided diagnosis
- CAM:
-
Class activation mapping
- CNN:
-
Convolutional neural network
- FNA:
-
Fine needle aspiration
- LNM:
-
Lymph node metastasis
- PTC:
-
Papillary thyroid carcinoma
- US:
-
Ultrasonography
References
Davies L, Welch HG (2014) Current thyroid cancer trends in the United States. JAMA Otolaryngol Head Neck Surg 140:317–322
Li N, Du XL, Reitzel LR, Xu L, Sturgis EM (2013) Impact of enhanced detection on the increase in thyroid cancer incidence in the United States: review of incidence trends by socioeconomic status within the surveillance, epidemiology, and end results registry, 1980-2008. Thyroid 23:103–110
Haugen BR, Alexander EK, Bible KC et al (2016) 2015 American Thyroid Association management guidelines for adult patients with thyroid nodules and differentiated thyroid Cancer: the American Thyroid Association guidelines task force on thyroid nodules and differentiated thyroid cancer. Thyroid 26:1–133
Al-Saif O, Farrar WB, Bloomston M, Porter K, Ringel MD, Kloos RT (2010) Long-term efficacy of lymph node reoperation for persistent papillary thyroid cancer. J Clin Endocrinol Metab 95:2187–2194
Stack BC Jr, Ferris RL, Goldenberg D et al (2012) American Thyroid Association consensus review and statement regarding the anatomy, terminology, and rationale for lateral neck dissection in differentiated thyroid cancer. Thyroid 22:501–508
Durante C, Montesano T, Torlontano M et al (2013) Papillary thyroid cancer: time course of recurrences during postsurgery surveillance. J Clin Endocrinol Metab 98:636–642
Tufano RP, Clayman G, Heller KS et al (2015) Management of recurrent/persistent nodal disease in patients with differentiated thyroid cancer: a critical review of the risks and benefits of surgical intervention versus active surveillance. Thyroid 25:15–27
Yeh MW, Bauer AJ, Bernet VA et al (2015) American Thyroid Association statement on preoperative imaging for thyroid cancer surgery. Thyroid 25:3–14
Wong SC, Gatt A, Stamatescu V, McDonnell MD (2016) Understanding data augmentation for classification: when to warp? https://arXiv.org/abs/1609.08764. Accessed 19 Nov 2018
Shin JH, Baek JH, Chung J et al (2016) Ultrasonography diagnosis and imaging-based management of thyroid nodules: revised Korean Society of Thyroid Radiology consensus statement and recommendations. Korean J Radiol 17:370–395
Suh CH, Baek JH, Choi YJ, Lee JH (2017) Performance of CT in the preoperative diagnosis of cervical lymph node metastasis in patients with papillary thyroid cancer: a systematic review and meta-analysis. AJNR Am J Neuroradiol 38:154–161
Lee Y, Kim JH, Baek JH et al (2018) Value of CT added to ultrasonography for the diagnosis of lymph node metastasis in patients with thyroid cancer. Head Neck. https://doi.org/10.1002/hed.25202
Shen D, Wu G, Suk HI (2017) Deep learning in medical image analysis. Annu Rev Biomed Eng 19:221–248
Miotto R, Wang F, Wang S, Jiang X, Dudley JT (2017) Deep learning for healthcare: review, opportunities and challenges. Brief Bioinform
LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521:436
Choi YJ, Baek JH, Park HS et al (2017) A computer-aided diagnosis system using artificial intelligence for the diagnosis and characterization of thyroid nodules on ultrasound: initial clinical assessment. Thyroid 27:546–552
Wu MH, Chen CN, Chen KY et al (2016) Quantitative analysis of echogenicity for patients with thyroid nodules. Sci Rep 6:35632
Jeong EY, Kim HL, Ha EJ et al (2018) Computer-aided diagnosis system for thyroid nodules on ultrasonography: diagnostic performance and reproducibility based on the experience level of operators. Eur Radiol
Lee JH, Baek JH, Kim JH et al (2018) Deep learning-based computer-aided diagnosis system for localization and diagnosis of metastatic lymph nodes on ultrasound: a pilot study. Thyroid 28:1332–1338
Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. https://arXiv.org/abs/1409.1556. Accessed 19 Nov 2018
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognitionProceedings of the IEEE conference on computer vision and. Pattern Recogn:770–778
Chollet F (2017) Xception: deep learning with depthwise separable convolutions. https://arXiv.org/abs/1610.02357. Accessed 19 Nov 2018
Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. https://arXiv.org/abs/1512.00567. Accessed 19 Nov 2018
Szegedy C, Ioffe S, Vanhoucke V, Alemi AA (2016) Inception-v4, inception-resnet and the impact of residual connections on learning. https://arXiv.org/abs/1602.07261. Accessed 19 Nov 2018
Iandola F, MoskewiczM,Karayev S, Girshick R, Darrell T, Keutzer K (2014) Densenet: Implementing efficient convnet descriptor pyramids. https://arXiv.org/abs/1404.1869. Accessed 19 Nov 2018
Russakovsky O, Deng J, Su H et al (2015) Imagenet large scale visual recognition challenge. Int J Comput Vis 115:211–252
Abadi M, Barham P, Chen J et al (2016) Tensorflow: a system for large-scale machine learning. https://arXiv.org/abs/1605.08695. Accessed 19 Nov 2018
Mulla MG, Knoefel WT, Gilbert J, McGregor A, Schulte KM (2012) Lateral cervical lymph node metastases in papillary thyroid cancer: a systematic review of imaging-guided and prophylactic removal of the lateral compartment. Clin Endocrinol 77:126–131
Moon HJ, Kim EK, Yoon JH, Kwak JY (2012) Differences in the diagnostic performances of staging US for thyroid malignancy according to experience. Ultrasound Med Biol 38:568–573
Mulla M, Schulte KM (2012) Central cervical lymph node metastases in papillary thyroid cancer: a systematic review of imaging-guided and prophylactic removal of the central compartment. Clin Endocrinol 76:131–136
Jeong HS, Baek CH, Son YI et al (2006) Integrated 18F-FDG PET/CT for the initial evaluation of cervical node level of patients with papillary thyroid carcinoma: comparison with ultrasound and contrast-enhanced CT. Clin Endocrinol 65:402–407
Ahn JE, Lee JH, Yi JS et al (2008) Diagnostic accuracy of CT and ultrasonography for evaluating metastatic cervical lymph nodes in patients with thyroid cancer. World J Surg 32:1552–1558
Lee DW, Ji YB, Sung ES et al (2013) Roles of ultrasonography and computed tomography in the surgical management of cervical lymph node metastases in papillary thyroid carcinoma. Eur J Surg Oncol 39:191–196
Ha EJ, Baek JH, Na DG (2017) Risk stratification of thyroid nodules on ultrasonography: current status and perspectives. Thyroid 27:1463–1468
Esteva A, Kuprel B, Novoa RA et al (2017) Dermatologist-level classification of skin cancer with deep neural networks. Nature 542:115
Lee HJ, Yoon DY, Seo YL et al (2018) Intraobserver and interobserver variability in ultrasound measurements of thyroid nodules. J Ultrasound Med 37:173–178
Funding
The authors state that this work was supported by the National Research Foundation of Korea (no. 2017R1C1B5016217).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Guarantor
The scientific guarantor of this publication is Eun Ju Ha.
Conflict of interest
The authors of this manuscript declare no relationships with any companies, whose products or services may be related to the subject matter of the article.
Statistics and biometry
No complex statistical methods were necessary for this paper.
Informed consent
Written informed consent was obtained from all patients before they underwent US.
Ethical approval
This study was approved by our institutional review board.
Methodology
• retrospective
• case-control study
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
ESM 1
(DOC 11717 kb)
Rights and permissions
About this article
Cite this article
Lee, J.H., Ha, E.J. & Kim, J.H. Application of deep learning to the diagnosis of cervical lymph node metastasis from thyroid cancer with CT. Eur Radiol 29, 5452–5457 (2019). https://doi.org/10.1007/s00330-019-06098-8
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00330-019-06098-8