Abstract
The accurate localization of nodules in ultrasound images can convey crucial information to support a reliable diagnosis. However, this is usually challenging due to low contrast and image artifacts, especially in thyroid ultrasound images where nodules are relatively small in most cases. To address these problems, in this paper, we propose a joint-training convolutional neural network (CNN) for thyroid nodule localization in ultrasound images. Considering the advantage of the faster region-based CNN (Faster R-CNN) in detecting natural targets, we adopt it as the basic framework. To boost the representative power and noise suppression capability of the network, the attention mechanism module is embedded for adaptive feature refinement along the channel and spatial dimensions. Furthermore, in the training process, we annotate the training set in a novel way, called joint-training annotation, by exploiting the fake foreground (FFG) area around the nodule as a spatial prior constraint to improve the sensitivity to small nodules. Ablation experiments are conducted to verify the effectiveness of our proposed method. The experimental results show that our method outperforms others by a mean average precision (mAP) of 0.93 and achieves an intersection over union (IoU) of 0.9, indicating that the localization results agree well with the ground truth. Furthermore, extended experiments on breast nodule datasets are also conducted to verify the generalizability of the proposed approach. Above all, the proposed algorithm is of considerable significance for accurate thyroid nodule localization in ultrasound images and can be generalized to other types of nodules, thereby providing trustworthy assistance for clinical diagnosis.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Haugen BR, Alexander EK, Bible KC: 2015 American Thyroid Association management guidelines for adult patients with thyroid nodules and differentiated thyroid cancer: the American Thyroid Association Guidelines Task Force on thyroid nodules and differentiated thyroid cancer. Thyroid 26(1):1–133, 2016
Gharib H, Papini E, Paschke R: Medical guidelines for clinical practice for the diagnosis and management of thyroid nodules. Endocr Pract 12(1):63-102, 2006
Kloos RT , Eng C , Evans DB: Medullary thyroid cancer: management guidelines of the American Thyroid Association. Thyroid 19(6):565-612, 2009
Haugen BR, Alexander EK, Bible KC: The American Thyroid Association (ATA) guidelines task force on thyroid nodules and differentiated thyroid cancer. Thyroid 26(1):1-133, 2016
Gautherie, Michel: Thermobiological assessment of benign and malignant breast diseases. Am J Obstet Gynecol 147(8):861-869, 1983
Gharib H, Papini E, Paschke R: American Association of Clinical Endocrinologists, Associazione Medici Endocrinologi, and European Thyroid Association medical guidelines for clinical practice for the diagnosis and management of thyroid nodules: executive summary of recommendations. J Endocrinol Investig 33:287, 2010
Yap MH, Edirisinghe E, Bez H: Processed images in human perception: a case study in ultrasound breast imaging. Eur J Radiol 73(3): 682–687, 2010
Calas MJG, Almeida RMVR, Gutfilen B, Pereira WCA: Intraobserver interpretation of breast ultrasonography following the bi-rads classification. Eur J Radiol 74(4):525-528, 2010
Yap, Moi Hoon, Edirisinghe E, Bez H: Processed images in human perception: a case study in ultrasound breast imaging. Eur J Radiol 73(11):682–687, 2010
Chang RF, Wu WJ, Moon WK, Chen DR: Improvement in breast nodule discrimination by support vector machines and speckle-emphasis texture analysis. Ultrasound Med Biol 29(5):679-686, 2003
Noble JA, Boukerroui D: Ultrasound image segmentation: a survey. IEEE Trans Med Imaging 25(8): 987-1010, 2006
Lee H, Chen YPP: Image based computer aided diagnosis system for cancer detection. Expert Syst Appl 42(2):5356-5365. 2015
LeCun Y, Bengio Y, Hinton G: Deep Learning. Nature 521(5): 436-444, 2015
Shin HC, Roth HR, Gao M, Lu L, Xu Z, Nogues I: Deep convolutional neural networks for computer-aided detection: cnn architectures, dataset characteristics and transfer learning. IEEE Trans Med Imaging 35(5):1285–1298, 2016
Litjens G, Kooi T, Bejnordi BE, Setio A, Ciompi F, Ghafoorian M: A survey on deep learning in medical image analysis. Med Image Anal 42(12):60-88, 2017
Hussain MA, Amir-Khalili A, Hamarneh G, Abugharbieh R: Segmentation-free kidney localization and volume estimation using aggregated orthogonal decision cnns. In: International Conference on Medical Image Computing & Computer-assisted Intervention. Springer, Cham, 2017, pp 612–620
Chen H, Ni D, Qin J, Li SL, Yang X, Wang TF: Standard plane localization in fetal ultrasound via domain transferred deep neural networks. IEEE J Biomed Health Inform 19(4):1627–1636, 2015
Ren S, He K, Girshick R, Sun J: Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell 39(6): 1137–1149, 2017
Redmon J, Divvala S, Girshick R, Farhadi A: You Only Look Once: Unified, Real-Time Object Detection. In: 2016 IEEE International Conference on Computer Vision and Pattern Recognition (CVPR). 2016, pp 779-788
Fang J, Cheng J: Target detection and recognition based on improved Faster R-CNN. J Image Signal Process 8(1):43-50, 2019
Szegedy C, Ioffe S, Vanhoucke V, Alemi A: Inception-v4, inception-resnet and the impact of residual connections on learning. arXiv:1602.07261, 2016
Wang X, Girshick R, Mulam H, He K: Non-Local Neural Networks. arXiv:1711.07971, 2017
Wei SE, Ramakrishna V, Kanade T, Sheikh Y: Convolutional Pose Machines. arXiv:1602.00134, 2016
Neubeck A, Van Gool L: Efficient non-maximum suppression. In: 18th International Conference on Pattern Recognition (ICPR). 2006, pp 850-855
Girshic R, Donahue J, Darrell T: Rich feature hierarchies for accurate object detection and semantic segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2014, pp 580-587
Ribli D, Horváth A, Unger Z, Pollner P, Péter, Csabai I: Detecting and classifying lesions in mammograms with deep learning. Sci Rep 8:4165, 2018
Chi J, Walia E, Babyn P, Wang J, Eramian M: Thyroid nodule classification in ultrasound images by fine-tuning deep convolutional neural network. J Digit Imaging 30(3):477-486, 2017
Ma J, Wu F, Jiang T, Zhu J, Kong D: Cascade convolutional neural networks for automatic detection of thyroid nodules in ultrasound images. Med Phys 44(5):1678-1691, 2017
Woo S, Park J, Lee JY: CBAM: Convolutional Block Attention Module. arXiv:1807.06521, 2018
Pizer SM, Amburn EP, Austin JD, Cromartie R, Zuiderveld K: Adaptive histogram equalization and its variations. Comput Vis Graphics Image Process 39(9):355-368, 1987
Ploquin M, Basarab A, Kouamé D: Resolution enhancement in medical ultrasound imaging. J Med Imaging 2(1):017001, 2015
He K, Zhang X, Ren S, Sun J: Deep residual learning for image recognition. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2016, pp 770-778
Kingma D, Ba J: Adam: A method for stochastic optimization. arXiv:1412.6980, 2015
Abadi M, Barham P, Chen J, Chen Z, Davis A, Dean J: TensorFlow: a system for large-scale machine learning. In: Conference on Operating Systems Design and Implementation. 2016, pp 265-283
Funding
This work was funded by the National Natural Science Foundation of China (61871135, 81830058, 81627804) and the Science and Technology Commission of Shanghai Municipality (18511102904, 17411953400).
Author information
Authors and Affiliations
Corresponding authors
Ethics declarations
Conflict of Interest
The authors declare that they have no conflict of interest.
Informed Consent
Informed consent was obtained from all individual participants included in the study.
Ethical Approval
This article does not contain any studies with human participants performed by any of the authors.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Liu, R., Zhou, S., Guo, Y. et al. Nodule Localization in Thyroid Ultrasound Images with a Joint-Training Convolutional Neural Network. J Digit Imaging 33, 1266–1279 (2020). https://doi.org/10.1007/s10278-020-00366-6
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10278-020-00366-6