MFS-Net: Multi-Stage Feature Fusion and Shape Fitting Network for Ultrasound Image Segmentation
Authors: 
Lei Lyu, Tongze Liu, Chen Pang, Jingping QiaoAuthors Info & Claims
Lei Lyu, Tongze Liu, Chen Pang, Jingping QiaoAuthors Info & ClaimsPages 35 - 43
Abstract
Most ultrasound image segmentation methods employ a joint training strategy, where the model learns the edge and core regions indiscriminately and uniformly. However, they often overlook the distinct shapes of the edges and the sizes of the core regions. In this paper, we propose an innovative Multi-Stage Feature Fusion and Shape Fitting Network for Ultrasound Image Segmentation (MFS-Net) that includes a core segmentation stage, an edge enhancement stage, and a fusion correction stage. The first two stages focus on learning the spatial features of the core region and the texture features of the edge region, respectively. After two stages of learning, the core and edge features of the labeled area are then fused in the third stage. In the third stage, we proposed a shape correction loss function to correct the shape of the labeled regions. To validate the superiority of our proposed MFS-Net, we conducted extensive experiments on our self-constructed Parathyroid Segmentation Dataset (PUS) and three public datasets. The results demonstrate substantial improvements over mainstream methods across all four datasets.
References
[1]
Walid Al-Dhabyani, Mohammed Gomaa, Hussien Khaled, and Aly Fahmy. 2020. Dataset of breast ultrasound images. Data in brief, Vol. 28 (2020), 104863.
[2]
Ahmed Beji, Ahmed Ghazi Blaiech, Mourad Said, Asma Ben Abdallah, and Mohamed Hédi Bedoui. 2023. An innovative medical image synthesis based on dual GAN deep neural networks for improved segmentation quality. Applied Intelligence, Vol. 53, 3 (2023), 3381--3397.
[3]
Yuri Boykov, Vladimir Kolmogorov, Daniel Cremers, and Andrew Delong. 2006. An integral solution to surface evolution PDEs via geo-cuts. In Computer Vision--ECCV 2006: 9th European Conference on Computer Vision, Graz, Austria, May 7--13, 2006, Proceedings, Part III 9. Springer, 409--422.
[4]
Jieneng Chen, Yongyi Lu, Qihang Yu, Xiangde Luo, Ehsan Adeli, Yan Wang, Le Lu, Alan L Yuille, and Yuyin Zhou. 2021. Transunet: Transformers make strong encoders for medical image segmentation. arXiv preprint arXiv:2102.04306 (2021).
[5]
Yaxin Chen, Benteng Ma, and Yong Xia. 2023. Adaptive decoder-block selection with filter reweighting for medical image segmentation. Biomedical Signal Processing and Control, Vol. 85 (2023), 105033.
[6]
Haifan Gong, Jiaxin Chen, Guanqi Chen, Haofeng Li, Guanbin Li, and Fei Chen. 2023. Thyroid region prior guided attention for ultrasound segmentation of thyroid nodules. Computers in Biology and Medicine, Vol. 155 (2023), 106389.
[7]
Ahmed Iqbal and Muhammad Sharif. 2023. BTS-ST: Swin transformer network for segmentation and classification of multimodality breast cancer images. Knowledge-Based Systems, Vol. 267 (2023), 110393.
[8]
Konstantinos Kamnitsas, Enzo Ferrante, Sarah Parisot, Christian Ledig, Aditya V Nori, Antonio Criminisi, Daniel Rueckert, and Ben Glocker. 2016. DeepMedic for brain tumor segmentation. In Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: Second International Workshop, BrainLes 2016, with the Challenges on BRATS, ISLES and mTOP 2016, Held in Conjunction with MICCAI 2016, Athens, Greece, October 17, 2016, Revised Selected Papers 2. Springer, 138--149.
[9]
Davood Karimi and Septimiu E Salcudean. 2019. Reducing the hausdorff distance in medical image segmentation with convolutional neural networks. IEEE Transactions on medical imaging, Vol. 39, 2 (2019), 499--513.
[10]
Hoel Kervadec, Jihene Bouchtiba, Christian Desrosiers, Eric Granger, Jose Dolz, and Ismail Ben Ayed. 2019. Boundary loss for highly unbalanced segmentation. In International conference on medical imaging with deep learning. PMLR, 285--296.
[11]
Tao Lei, Rui Sun, Xiaogang Du, Huazhu Fu, Changqing Zhang, and Asoke K. Nandi. 2023. SGU-Net: Shape-Guided Ultralight Network for Abdominal Image Segmentation. IEEE Journal of Biomedical and Health Informatics, Vol. 27, 3 (2023), 1431--1442. https://doi.org/10.1109/JBHI.2023.3238183
[12]
Karim Lekadir, Alfiia Galimzianova, Àngels Betriu, Maria del Mar Vila, Laura Igual, D. Rubin, Elvira Fernández, Petia Radeva, and Sandy Napel. 2017. A Convolutional Neural Network for Automatic Characterization of Plaque Composition in Carotid Ultrasound. IEEE Journal of Biomedical and Health Informatics, Vol. 21 (2017), 48--55.
[13]
Xiaoqiang Li, Yuanchen Wu, and Songmin Dai. 2023. Semi-supervised medical imaging segmentation with soft pseudo-label fusion. Applied Intelligence, Vol. 53, 18 (2023), 20753--20765.
[14]
Yuhang Liu, Han Wang, Zugang Chen, Kehan Huangliang, and Haixian Zhang. 2022. TransUNet: Redesigning the skip connection to enhance features in medical image segmentation. Knowledge-Based Systems, Vol. 256 (2022), 109859.
[15]
Zhaoshan Liu and Lei Shen. 2022. Medical image analysis based on transformer: A Review. arXiv preprint arXiv:2208.06643 (2022).
[16]
Jonathan Long, Evan Shelhamer, and Trevor Darrell. 2015. Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition. 3431--3440.
[17]
Yuting Lu, Kun Wang, Wei Zhang, Jin Xie, Sheng Huang, Dan Yang, and Xiaohong Zhang. 2023. Learning contextual representations with copula function for medical image segmentation. Biomedical Signal Processing and Control, Vol. 85 (2023), 104900.
[18]
Ozan Oktay, Jo Schlemper, Loic Le Folgoc, Matthew Lee, Mattias Heinrich, Kazunari Misawa, Kensaku Mori, Steven McDonagh, Nils Y Hammerla, Bernhard Kainz, et al. 2018. Attention u-net: Learning where to look for the pancreas. arXiv preprint arXiv:1804.03999 (2018).
[19]
Tao Peng, Daqiang Xu, Caiyin Tang, Jing Zhao, Yuntian Shen, Cong Yang, and Jing Cai. 2023. Automatic coarse-to-refinement-based ultrasound prostate segmentation using optimal polyline segment tracking method and deep learning. Applied Intelligence, Vol. 53, 18 (2023), 21390--21406.
[20]
Esther Puyol-Antón, Baldeep S Sidhu, Justin Gould, Bradley Porter, Mark K Elliott, Vishal Mehta, Christopher A Rinaldi, and Andrew P King. 2022. A multimodal deep learning model for cardiac resynchronisation therapy response prediction. Medical Image Analysis, Vol. 79 (2022), 102465.
[21]
Uma M Reddy, Roy A Filly, and Joshua A Copel. 2008. Prenatal imaging: ultrasonography and magnetic resonance imaging. Obstetrics and gynecology, Vol. 112, 1 (2008), 145.
[22]
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention--MICCAI 2015: 18th International Conference, Munich, Germany, October 5--9, 2015, Proceedings, Part III 18. Springer, 234--241.
[23]
Hans Sagan. 2012. Introduction to the Calculus of Variations. Courier Corporation.
[24]
Zahra Sobhaninia, Nader Karimi, Pejman Khadivi, and Shadrokh Samavi. 2023. Brain tumor segmentation by cascaded multiscale multitask learning framework based on feature aggregation. Biomedical Signal Processing and Control, Vol. 85 (2023), 104834.
[25]
Fenghe Tang, Lingtao Wang, Chunping Ning, Min Xian, and Jianrui Ding. 2023. Carnegie Mellon University-Net: A Strong ConvMixer-based Medical Ultrasound Image Segmentation Network. In 2023 IEEE 20th International Symposium on Biomedical Imaging (ISBI). 1--5. https://doi.org/10.1109/ISBI53787.2023.10230609
[26]
Guoyu Tong and Huiyan Jiang. 2023. A hard segmentation network guided by soft segmentation for tumor segmentation on PET/CT images. Biomedical Signal Processing and Control, Vol. 85 (2023), 104918.
[27]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems, Vol. 30 (2017).
[28]
Risheng Wang, Tao Lei, Ruixia Cui, Bingtao Zhang, Hongying Meng, and Asoke K Nandi. 2022. Medical image segmentation using deep learning: A survey. IET Image Processing, Vol. 16, 5 (2022), 1243--1267.
[29]
Zixuan Wang, Ruofan Wu, Yanran Xu, Yi Liu, Ruimei Chai, and He Ma. 2023. A two-stage CNN method for MRI image segmentation of prostate with lesion. Biomedical Signal Processing and Control, Vol. 82 (2023), 104610.
[30]
Huisi Wu, Shihuai Chen, Guilian Chen, Wei Wang, Baiying Lei, and Zhenkun Wen. 2022. FAT-Net: Feature adaptive transformers for automated skin lesion segmentation. Medical image analysis, Vol. 76 (2022), 102327.
[31]
Haiying Xia, Yang Lan, Shuxiang Song, and Haisheng Li. 2021. A multi-scale segmentation-to-classification network for tiny microaneurysm detection in fundus images. Knowledge-Based Systems, Vol. 226 (2021), 107140.
[32]
Zengqiang Yan, Xin Yang, and Kwang-Ting Cheng. 2018. A three-stage deep learning model for accurate retinal vessel segmentation. IEEE journal of Biomedical and Health Informatics, Vol. 23, 4 (2018), 1427--1436.
[33]
Qinghan Yang, Chong Geng, Ruyue Chen, Chen Pang, Run Han, Lei Lyu, and Yuang Zhang. 2022. DMU-Net: Dual-route mirroring U-Net with mutual learning for malignant thyroid nodule segmentation. Biomedical Signal Processing and Control, Vol. 77 (2022), 103805.
[34]
Tian-Tian Zhang, Hua Shu, Kam-Yiu Lam, Chi-Yin Chow, and Ao Li. 2023. Feature decomposition and enhancement for unsupervised medical ultrasound image denoising and instance segmentation. Applied Intelligence, Vol. 53, 8 (2023), 9548--9561.
[35]
Yue Zhao, Pengcheng Li, Chenqiang Gao, Yang Liu, Qiaoyi Chen, Feng Yang, and Deyu Meng. 2020. TSASNet: Tooth segmentation on dental panoramic X-ray images by Two-Stage Attention Segmentation Network. Knowledge-Based Systems, Vol. 206 (2020), 106338.
[36]
Yawu Zhao, Yuanyuan Zhang, Lin Liu, Ziyi Han, Fanyun Meng, and Yulin Zhang. [n. d.]. Transefusionnet: Deep Fusion Network for Colorectal Polyp Segmentation. Available at SSRN 4379840 ([n. d.]).
[37]
Z Zhou, MMR Siddiquee, N Tajbakhsh, and J UNet Liang. [n. d.]. A nested U-Net architecture for medical image segmentation (2018). arXiv preprint arXiv:1807.10165 ([n. d.]).
[38]
Hongwei Zhu, Peng Li, Haoran Xie, Xuefeng Yan, Dong Liang, Dapeng Chen, Mingqiang Wei, and Jing Qin. 2022. I can find you! Boundary-guided separated attention network for camouflaged object detection. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 36. 3608--3616.
Index Terms
- MFS-Net: Multi-Stage Feature Fusion and Shape Fitting Network for Ultrasound Image Segmentation
Recommendations
Ultrasound Image Segmentation of Breast Tumors Based on Swin-transformerv2
ICIT '22: Proceedings of the 2022 10th International Conference on Information Technology: IoT and Smart CityBreast cancer is one of the most common cancers with a high mortality rate. Early detection is essential to reduce the risk of mortality and morbidity for breast cancer. The diagnosis of breast cancer relies heavily on the segmentation accuracy of the ...
Ultrasound Image Segmentation Algorithm of Thyroid Nodules Based on Improved U-Net Network
CCRIS '22: Proceedings of the 2022 3rd International Conference on Control, Robotics and Intelligent SystemThyroid nodule is a common clinical case, and there are different cancer risks according to the benign and malignant grades of the nodules. The shape, edge, and size of thyroid nodules are all important basis for judging the nature of thyroid nodules in ...
SThy-Net: a feature fusion-enhanced dense-branched modules network for small thyroid nodule classification from ultrasound images
AbstractDeep learning studies of thyroid nodule classification from ultrasound (US) images have focused mainly on nodules with diameters > 1 cm. However, small thyroid nodules measuring cm, especially nodules with high-risk stratification, are ...
Comments
Information & Contributors
Information
Published In
October 2024
85 pages
ISBN:9798400711954
DOI:10.1145/3688868
- Program Chairs:
- Xuequan Lu,
- Wenxi Yue,
- Imran Razzak,
- Kun Hu,
- Jinglei Lv,
- Sen Zhang,
- Junhui Hou,
- Zhiyong Wang,
- Jiebo Luo,
- Wei Xiang
Copyright © 2024 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 31 October 2024
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- Taishan Scholar Program of Shandong Province of China
- Natural Science Foundation of Shandong Province
- National Natural Science Foundation of China
Conference
MM '24
Sponsor:
MM '24: The 32nd ACM International Conference on Multimedia
October 28 - November 1, 2024
Melbourne VIC, Australia
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 51Total Downloads
- Downloads (Last 12 months)51
- Downloads (Last 6 weeks)7
Reflects downloads up to 14 Jan 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in