Rapid Segmentation and Diagnosis of Breast Tumor Ultrasound Images at the Sonographer Level Using Deep Learning
Abstract
:1. Introduction
- -
- We introduce a two-stage convolutional neural network framework that integrates deep-learning-based image segmentation and classification tasks, thereby reducing the time to diagnosis and improving recognition accuracy.
- -
- With the integration of the attention mechanism in the tumor recognition method, the breast tumor region can be located and information regarding the size and shape of its main region can be obtained. The influence of the ultrasonic background on the classification can be reduced, and the diagnostic effect of senior ultrasound doctors can be achieved.
- -
- We validate the efficiency of the proposed method on different datasets.
- -
- We develop an intelligent auxiliary diagnosis system for breast ultrasound images, which can realize end-to-end recognition, provide auxiliary diagnosis schemes for ultrasound doctors and improve diagnosis efficiency.
2. Materials and Methods
2.1. Breast Ultrasound Image Datasets Are Used for the Development of Multiple Convolutional Neural Networks
2.2. Data Processing
2.3. Method Overview
2.4. Evaluation Indicators
- (1)
- MIoU and Dice are used to evaluate the first deep convolutional neural network.
- (2)
- The most commonly used indicators in a medical evaluation for the second-degree convolutional neural network include accuracy (ACC), sensitivity, specificity and the area under the curve. The sensitivity is also called the true positive rate (TPR), which is the probability that a patient is classified as malignant, and the specificity is called the true negative rate (TNR), which is the probability that a person who does not actually have the disease is classified as benign. The AUC is the area bounded by the axis under the receiver operator characteristic curve (ROC).
3. Result
3.1. Model Performance Evaluation
3.2. External Data Validation
3.3. Ablation Experiment
3.4. Design of Intelligent Auxiliary Diagnosis System for Breast Ultrasound Imaging
3.5. Clinical Trial Design
4. Conclusions
5. Discussion
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Siegel, R.L.; Miller, K.D.; Wagle, N.S.; Jemal, A. Cancer statistics, 2023. CA A Cancer J. Clin. 2023, 73, 17–48. [Google Scholar] [CrossRef] [PubMed]
- World Health Organization. World Cancer Report, 2022; IARC: International Agency for Research on Cancer: Lyon, France, 2022. [Google Scholar]
- Kim, M.S.; Yun, L.B.; Jang, M. Clinical Applications of Automated Breast Ultrasound: Screening for Breast Cancer. J. Korean Soc. Radiol. 2019, 80, 32–46. [Google Scholar] [CrossRef]
- Kriaucioniene, V.; Petkeviciene, J. Predictors and Trend in Attendance for Breast Cancer Screening in Lithuania. Int. J. Environ. Res. Public Health 2019, 16, 4535. [Google Scholar] [CrossRef] [PubMed]
- Das, M. Supplemental ultrasonography for breast cancer screening. Lancet Oncol. 2019, 20, e244. [Google Scholar] [CrossRef] [PubMed]
- Hooley, R.J.; Butler, R. Modern Challenges in Assessing Breast Cancer Screening Strategies: A Call for Added Resources. Radiology 2023, 306, e230145. [Google Scholar] [CrossRef]
- Xing, B.; Chen, X.; Wang, Y.; Li, S.; Liang, Y.K.; Wang, D. Evaluating breast ultrasound S-detect image analysis for small focal breast lesions. Front. Oncol. 2022, 12, 1030624. [Google Scholar] [CrossRef]
- Marini, T.J.; Castaneda, B.; Iyer, R.; Baran, T.M.; Nemer, O.; Dozier, A.M.; Parker, K.J.; Zhao, Y.; Serratelli, W.; Matos, G.; et al. Breast Ultrasound Volume Sweep Imaging: A New Horizon in Expanding Imaging Access for Breast Cancer Detection. J. Ultrasound Med. Off. J. Am. Inst. Ultrasound Med. 2022, 42, 817–832. [Google Scholar] [CrossRef]
- Nicosia, L.; Pesapane, F.; Bozzini, A.C.; Latronico, A.; Rotili, A.; Ferrari, F.; Signorelli, G.; Raimondi, S.; Vignati, S.; Gaeta, A.; et al. Prediction of the Malignancy of a Breast Lesion Detected on Breast Ultrasound: Radiomics Applied to Clinical Practice. Cancers 2023, 15, 964. [Google Scholar] [CrossRef]
- Choi, E.J.; Lee, E.H.; Kim, Y.M.; Chang, Y.W.; Lee, J.H.; Park, Y.M.; Kim, K.W.; Kim, Y.J.; Jun, J.K.; Hong, S. Interobserver agreement in breast ultrasound categorization in the Mammography and Ultrasonography Study for Breast Cancer Screening Effectiveness (MUST-BE) trial: Results of a preliminary study. Ultrasonography 2019, 38, 172. [Google Scholar] [CrossRef]
- Apantaku, L. Breast cancer diagnosis and screening. Am. Fam. Physician 2000, 62, 605–606. [Google Scholar]
- Romeo, V.; Cuocolo, R.; Apolito, R.; Stanzione, A.; Ventimiglia, A.; Vitale, A.; Verde, F.; Accurso, A.; Amitrano, M.; Insabato, L.; et al. Clinical value of radiomics and machine learning in breast ultrasound: A multicenter study for differential diagnosis of benign and malignant lesions. Eur. Radiol. 2021, 31, 9511–9519. [Google Scholar] [CrossRef] [PubMed]
- Zhang, S.; Liao, M.; Wang, J.; Zhu, Y.; Zhang, Y.; Zhang, J.; Zheng, R.; Lv, L.; Zhu, D.; Chen, H.; et al. Fully automatic tumor segmentation of breast ultrasound images with deep learning. J. Appl. Clin. Med. Phys. 2022, 24, e13863. [Google Scholar] [CrossRef] [PubMed]
- Mahant, S.S.; Varma, A.R.; Varma, A. Artificial Intelligence in Breast Ultrasound: The Emerging Future of Modern Medicine. Cureus 2022, 14, e28945. [Google Scholar] [CrossRef]
- Podda, A.S.; Balia, R.; Barra, S.; Carta, S.; Fenu, G.; Piano, L. Fully-automated deep learning pipeline for segmentation and classification of breast ultrasound images. J. Comput. Sci. 2022, 63, 101816. [Google Scholar] [CrossRef]
- Qu, X.; Lu, H.; Tang, W.; Wang, S.; Zheng, D.; Hou, Y.; Jiang, J. A VGG attention vision transformer network for benign and malignant classification of breast ultrasound images. Med. Phys. 2022, 49, 5787–5798. [Google Scholar] [CrossRef] [PubMed]
- Mishra, A.K.; Roy, P.; Bandyopadhyay, S.; Das, S.K. CR-SSL: A closely related self-supervised learning based approach for improving breast ultrasound tumor segmentation. Int. J. Imaging Syst. Technol. 2021, 32, 1209–1220. [Google Scholar] [CrossRef]
- Mishra, A.K.; Roy, P.; Bandyopadhyay, S.; Das, S.K. Achieving highly efficient breast ultrasound tumor classification with deep convolutional neural networks. Int. J. Inf. Technol. 2022, 14, 3311–3320. [Google Scholar] [CrossRef]
- Huang, Y.; Han, L.; Dou, H.; Luo, H.; Yuan, Z.; Liu, Q.; Zhang, J.; Yin, G. Two-stage CNNs for computerized BI-RADS categorization in breast ultrasound images. Biomed. Eng. Online 2019, 18, 8. [Google Scholar] [CrossRef]
- Chen, D.; Liu, Y.; Tao, Y.; Liu, W. A level set segmentation algorithm for breast B-ultrasound images combined with morphological automatic initialization. Electron. Compon. Inf. Technol. 2022, 6, 22–24. [Google Scholar]
- Wu, Y.; Lou, L.; Xu, B.; Huang, J.; Zhao, L. Intelligent classification diagnosis of ultrasound images of breast tumors based on transfer learning. Chin. Med. Imaging Technol. 2019, 35, 357–360. [Google Scholar]
- Du, Z.; Gong, X.; Luo, J.; Zhang, J.; Yang, F. Classification of confusing and difficult samples in breast ultrasound images. Chin. J. Image Graph. 2020, 25, 7. [Google Scholar]
- Yang, Z.; Wang, J.; Xin, C. Automatic segmentation and classification of MR Images using DCE-MRI combined with improved convolutional neural network. J. Chongqing Univ. Technol. 2020, 34, 147–157. [Google Scholar]
- Yu, F.; Yi, L.; Luo, X.; Li, L.; Yi, S. A deep learn-based BI-RADS classification method for breast ultrasound images. J. Yunnan Univ. 2023, 17, 815–824. [Google Scholar]
- Ren, P.; Sun, W.; Luo, C.; Hussain, A. Clustering-Oriented Multiple Convolutional Neural Networks for Single Image Super-Resolution. Cogn. Comput. 2018, 10, 165–178. [Google Scholar] [CrossRef]
- Hussain, S.; Xi, X.; Ullah, I.; Inam, S.A.; Naz, F.; Shaheed, K.; Ali, S.A.; Tian, C. A Discriminative Level Set Method with Deep Supervision for Breast Tumor Segmentation. Comput. Biol. Med. 2022, 149, 105995. [Google Scholar] [CrossRef] [PubMed]
- Kabir, S.M.; Bhuiyan, M.I. Correlated-Weighted Statistically Modeled Contourlet and Curvelet Coefficient Image-Based Breast Tumor Classification Using Deep Learning. Diagnostics 2022, 13, 69. [Google Scholar] [CrossRef] [PubMed]
- Cheng, X.; Tan, L.; Ming, F. Feature Fusion Based on Convolutional Neural Network for Breast Cancer Auxiliary Diagnosis. Math. Probl. Eng. 2021, 2021, 7010438. [Google Scholar] [CrossRef]
- Lecun, Y.; Bengio, Y.; Hinton, G. Deep learning. Nature 2015, 521, 436–444. [Google Scholar] [CrossRef] [PubMed]
- Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A.N.; Kaiser, Ł.; Polosukhin, I. Attention Is All You Need. Comput. Sci. 2017, 30, 7. [Google Scholar]
- Zhong, Y. Attention Mechanisms with UNet3+ on Brain Tumor MRI Segmentation. In Proceedings of the 2021 International Conference on Biological Engineering and Medical Science (ICBioMed 2021), Online, 9–15 December 2021; AIP Publishing: Long Island, NY, USA, 2021; pp. 209–215. [Google Scholar]
Diagnosis | The Images Used as Training and Validation Datasets | The Images Used as Test Datasets | ||
---|---|---|---|---|
Number of Patients (n) | Number of Images (n) | Number of Patients (n) | Number of Images (n) | |
benign | 649 | 1288 | 56 | 132 |
malignancy | 482 | 769 | 44 | 132 |
Model | MIoU | Mdice |
---|---|---|
U-net | 0.82 | 0.84 |
Fast-RCNN | 0.83 | 0.85 |
Deeplab V3 | 0.85 | 0.87 |
Ours | 0.89 | 0.92 |
Model | Accuracy | Sensitivity | Specificity | AUC |
---|---|---|---|---|
U-net + D-CNN | 97% | 97.7% | 96.4% | 0.96 |
D-CNN | 89% | 86.4% | 91.5% | 0.87 |
Medical Seniority | Accuracy of Benign and Malignant | Time |
---|---|---|
1 year | 71% | 60 min |
3 years | 83% | 45 min |
20 years | 98% | 42 min |
Medical Seniority | Accuracy of Benign and Malignant | Time |
---|---|---|
1 year | 85% | 40 min |
3 years | 92% | 34 min |
20 years | 98% | 16 min |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Yang, L.; Zhang, B.; Ren, F.; Gu, J.; Gao, J.; Wu, J.; Li, D.; Jia, H.; Li, G.; Zong, J.; et al. Rapid Segmentation and Diagnosis of Breast Tumor Ultrasound Images at the Sonographer Level Using Deep Learning. Bioengineering 2023, 10, 1220. https://doi.org/10.3390/bioengineering10101220
Yang L, Zhang B, Ren F, Gu J, Gao J, Wu J, Li D, Jia H, Li G, Zong J, et al. Rapid Segmentation and Diagnosis of Breast Tumor Ultrasound Images at the Sonographer Level Using Deep Learning. Bioengineering. 2023; 10(10):1220. https://doi.org/10.3390/bioengineering10101220
Chicago/Turabian StyleYang, Lei, Baichuan Zhang, Fei Ren, Jianwen Gu, Jiao Gao, Jihua Wu, Dan Li, Huaping Jia, Guangling Li, Jing Zong, and et al. 2023. "Rapid Segmentation and Diagnosis of Breast Tumor Ultrasound Images at the Sonographer Level Using Deep Learning" Bioengineering 10, no. 10: 1220. https://doi.org/10.3390/bioengineering10101220