Abstract
Elastography ultrasound (EUS) imaging has shown its effectiveness for diagnosis of tumors by providing additional information about tissue stiffness to the conventional B-mode ultrasound (BUS). However, due to the lack of EUS devices and experienced sonologists, EUS is not widely used, especially in rural areas. It is still a challenging task to improve the performance of the single-modal BUS-based computer-aided diagnosis (CAD) for tumors. In this work, we propose a novel transfer learning (TL)–based deep neural network (DNN) algorithm, named CW-PM-DNN, for the BUS-based CAD by transferring diagnosis knowledge from EUS during model training. CW-PM-DNN integrates both the feature-level and classifier-level knowledge transfer into a unified framework. In the feature-level TL, a bichannel DNN is learned by the cross-weight-based multimodal DL (MDL-CW) algorithm to transfer informative features from EUS to BUS. In the classifier-level TL, a projective model (PM)–based classifier is then embedded to the pretrained bichannel DNN to implement the parameter transfer in the classifier model at the second stage. The back-propagation procedure is then applied to optimize the whole CW-PM-DNN to further improve its performance. Experimental results on two bimodal ultrasound tumor datasets demonstrate that the proposed CW-PM-DNN achieves the best classification accuracy, sensitivity, and specificity of 89.02 ± 1.54%, 88.37 ± 4.72%, and 89.63 ± 4.06%, respectively, for the breast ultrasound dataset, and the corresponding values of 80.57 ± 3.41%, 76.67 ± 3.85%, and 83.94 ± 3.95%, respectively, for the prostate ultrasound dataset. The proposed two-stage TL-based CW-PM-DNN algorithm outperforms all the compared algorithms. It is also proved that the performance of the BUS-based CAD can be significantly improved by transferring the knowledge of EUS. It suggests that CW-PM-DNN has the potential for more applications in the field of medical image–based CAD.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Mitterberger M, Horningera W, Aignerb F, Pinggeraa G, Steppanb I, Rehdera P, et al. Ultrasound of the prostate. Cancer Imaging. 2010;10:40–8.
Hernaez R, Lazo M, Bonekamp S, Kamel I, Brancati FL, Guallar E, et al. Diagnostic accuracy and reliability of ultrasonography for the detection of fatty liver: a meta-analysis. Hepatology. 2011;54:1082–90.
Huang Q, Zhang F, Li X. Machine learning in ultrasound computer-aided diagnostic systems: a survey. Biomed Res Int. 2018;2018:1–10.
Liu S, Wang Y, Yang X, Lei B, Liu L, Li S, et al. Deep learning in medical ultrasound analysis: a review. Engineering. 2019;5:261–75.
Sigrist R, Liau J, Kaffas A, Chammas M, Willmann J. Ultrasound elastography: review of techniques and clinical applications. Theranostics. 2017;7:1303–29.
Gatos I, Tsantis S, Spiliopoulos S. A new computer aided diagnosis system for evaluation of chronic liver disease with ultrasound shear wave elastography imaging. Med Phys. 2016;43:1428–36.
Zhang Q, Xiao Y, Dai W, Suo J, Wang C, Shi J, et al. Deep learning based classification of breast tumors with shear-wave elastography. Ultrasonics. 2016;72:150–7.
Ara S, Alam F, Rahman M, Akhter S, Awwal R, Hasan M. Bimodal multiparameter-based approach for benign-malignant classification of breast tumors. Ultrasound Med Biol. 2015;41:2022–38.
Zhang Q, Song S, Xiao Y, Chen S, Shi J, Zhen H. Dual-modal artificially intelligent diagnosis of breast tumors on both shear-wave elastography and B-mode ultrasound using deep polynomial networks. Med Eng Phys. 2019;64:1–6.
Robicquet A, Ramsundar B, Kuleshov V, Depristo M, Chou K, Cui C, et al. A guide to deep learning in healthcare. Nat Med. 2019;25:24–9.
Shi J, Xue Z, Dai Y, Peng B, Dong Y, Zhang Q, et al. Cascaded multi-column RVFL+ classifier for single-modal neuroimaging-based diagnosis of Parkinson’s disease. IEEE Trans Biomed Eng. 2019;66:2362–71.
Pan S, Yang Q. A survey on transfer learning. IEEE Trans Knowl Data Eng. 2010;22:1345–59.
Weiss K, Khoshgoftaar T, Wang D. A survey of transfer learning. J Big Data. 2016;3:9.
Cheplygina V, Bruijne M, Pluim J. Not-so-supervised a survey of semi-supervised, multi-instance, and transfer learning in medical image analysis. Med Image Anal. 2019;54:280–96.
Chao C, Jiang B. Parameter transfer extreme learning machine based on projective model. In: International Joint Conference on Neural Network; 2018. p. 1–8.
Huang G, Zhu Q, Siew C. Extreme learning machine: theory and applications. Neurocomputing. 2006;70:489–501.
Huang G, Huang G, Song S, You K. Trends in extreme learning machines: a review. Neural Netw. 2015;61:32–48.
Xia J, Chen H, Li Q, Zhou M, Chen L, Cai Z, et al. Ultrasound-based differentiation of malignant and benign thyroid nodules: an extreme learning machine approach. Comput Methods Prog Biomed. 2017;147:37–49.
Salaken S, Khosravi A, Nguyen T, Nahavandi S. Extreme learning machine based transfer learning algorithms: a survey. Neurocomputing. 2017;267:516–24.
Zhou S, Shi J, Zhu J, Cai Y, Wang R. Shearlet-based texture feature extraction for classification of breast tumor in ultrasound image. Biomed Signal Process Control. 2013;8:688–96.
Shi J, Zheng X, Wu J, Gong B, Zhang Q, Ying S. Quaternion grassmann average network for learning representation of histopathological image. Pattern Recogn. 2019;89:67–76.
Schmidhuber J. Deep learning in neural networks: an overview. Neural Netw. 2015;61:85–117.
Tang J, Deng C, Huang G. Extreme learning machine for multilayer perceptron. IEEE Trans Neur Net Learn Syst. 2015;27:809–21.
Tan C, Sun F, Kong T, Zhang W, Yang C, Liu C. A survey on deep transfer learning. In: International Conference on Artificial Neural Networks; 2018. p. 270–9.
Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. In: Computer Vision and Pattern Recognition; 2014.
Krizhevsky A, Sutskever I, Hinton G. ImageNet classification with deep convolutional neural networks. Neural Inf Process Syst. 2012;141:1097–105.
Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, et al. Going deeper with convolutions. In: Computer Vision and Pattern Recognition; 2015. p. 1–9.
Long M, Cao Y, Wang J, Jordan M. Learning transferable features with deep adaptation networks. In: International Conference on Machine Learning; 2015. p. 97–105.
Rastegar S, Soleymani M, Rabiee H, Shojaee S. MDL-CW: a multimodal deep learning framework with cross weights. In: Computer Vision and Pattern Recognition; 2016. p. 2601–9.
Ramachandram D, Taylor G. Deep multimodal learning: a survey on recent advances and trends. IEEE Signal Proc Mag. 2017;34:96–108.
Guo W, Wang J, Wang S. Deep multimodal representation learning: a survey. IEEE Access. 2019;7:63373–94.
Tommasi T, Orabona F, Caputo B. Learning categories from few examples with multi model knowledge transfer. IEEE Trans Pattern Anal Mach Intell. 2014;36:928–41.
Ding C, Zhou D, He X, Zha H. R1-PCA: rotational invariant L1-norm principal component analysis for robust subspace factorization. In: International Conference on Machine Learning; 2006. p. 281–8.
Nemat H, Fehri H, Ahmadinejad N, Frangi A, Gooya A. Classification of breast lesions in ultrasonography using sparse logistic regression and morphology-based texture features. Med Phys. 2018;45:4112–24.
Huang Q, Huang Y, Luo Y, Yuan F, Li X. Segmentation of breast ultrasound image with semantic classification of superpixels. Med Image Anal. 2020;61:101657.
Zhang Q, Xiao Y, Suo J, Shi J, Yu J, Guo Y, et al. Sonoelastomics for breast tumor classification: a radiomics approach with clustering-based feature selection on sonoelastography. Ultrasound Med Biol. 2017;43:1058–69.
Livni R, Shalevshwartz S, Shamir O. An algorithm for training polynomial networks. arXiv: Learning. 2013.
Li W, Duan L, Xu D, Tsang I. Learning with augmented features for supervised and semi-supervised heterogeneous domain adaptation. IEEE Trans Pattern Anal Mach Intell. 2014;36:1134–48.
Zhang L, Zhang D. Robust visual knowledge transfer via extreme learning machine based domain adaptation. IEEE Trans Image Process. 2016;25:4959–73.
Zhang P. A new learning paradigm for random vector functional-link network: RVFL+. Neural Netw. 2020;122:94–105.
Sohn K, Shang W, Lee H. Improved multimodal deep learning with variation of information. In: Neural Information Processing Systems; 2014. p. 2141–9.
Li Y, Meng F, Shi J. Learning using privileged information improves neuroimaging-based CAD of Alzheimer’s disease: a comparative study. Med Biol Eng Comput. 2019;57(7):1605–16.
Andrew G, Arora R, Bilmes J, Livescu K. Deep canonical correlation analysis. In: International Conference on Machine Learning; 2013. p. 1247–55.
Huang Q, Zhang F, Li X. Machine learning in ultrasound computer-aided diagnostic systems: a survey. Biomed Res Int. 2018;2018:1–10.
Huang Q, Chen Y, Liu L, Tao D, Li X. On combining biclustering mining and adaBoost for breast tumor classification. IEEE Trans Knowl Data Eng. 2020;32(4):728–38.
Huang Q, Hu B, Zhang F. Evolutionary optimized fuzzy reasoning with mined diagnostic patterns for classification of breast tumors in ultrasound. Inf Sci. 2019;502:525–36.
Funding
This work is supported by the National Natural Science Foundation of China (81830058, 81627804), the Science and Technology Commission of Shanghai Municipality (17411953400, 18010500600, and18411967400), the 111 Project (D20031), and the Nanjing Science and Technology Planning Project (201803027).
Author information
Authors and Affiliations
Corresponding authors
Ethics declarations
Conflict of Interest
The authors declare that they have no conflict of interest.
Ethical Approval
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
Informed Consent
Informed consent was obtained from all individual participants included in the study.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Fei, X., Shen, L., Ying, S. et al. Parameter Transfer Deep Neural Network for Single-Modal B-Mode Ultrasound-Based Computer-Aided Diagnosis. Cogn Comput 12, 1252–1264 (2020). https://doi.org/10.1007/s12559-020-09761-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12559-020-09761-1