DualU-Net Mixed with Convolution and Transformers
Article No.: 54, Pages 1 - 6
Abstract
As the basis of quantitative analysis of lung diseases, lung segmentation has a great impact on the accuracy of subsequent work. Most of the current lung segmentation models focus on CNN. The inherent limitations of convolution make such networks unable to utilize the global context information of the lung, which is very important for lung segmentation. To solve this problem, this paper proposes a DualU-Net with convolutions and Transformers hybrid, which complements the global features of the lung through Transformers. DualU-Net consists of Local Down-sampling, Global Down-sampling, Invertible Feature Fusion Module, Bilinear-based Up-sampling Mechanism. Local Down-sampling extracts the local detail features of the lung through the convolution-based residual structure, and Global Down-sampling extracts the global long-range dependence through Transformers. The two down-samplings extract the local and global features of the lung simultaneously and independently in a parallel manner. Then, the local and global features are fused by the Invertible Feature Fusion Module. Finally, the resolution of the image is gradually restored by the Bilinear-based Up-sampling Mechanism to obtain the final segmentation result. We conduct experiments on two public datasets. The experimental results show that global features can indeed help the network to perform better segmentation, and DualU-Net can have a relatively balanced segmentation effect on various datasets.
References
[1]
Ben-Haim, T., Sofer, R.M., Ben-Arie, G., Shelef, I. and Raviv, T.R., 2022, October. A deep ensemble learning approach to lung CT segmentation for Covid-19 severity assessment. In 2022 IEEE International Conference on Image Processing (ICIP) (pp. 151-155). IEEE.
[2]
Salama, W.M. and Aly, M.H., 2021, December. Lung CT Image Segmentation: A Generalized Framework Based on U-Net Architecture and Preprocessing Models. In 2021 31st International Conference on Computer Theory and Applications (ICCTA) (pp. 141-146). IEEE.
[3]
Raj, A.N.J., Zhu, H., Khan, A., Zhuang, Z., Yang, Z., Mahesh, V.G. and Karthik, G., 2021. ADID-UNET—a segmentation model for COVID-19 infection from lung CT scans. PeerJ Computer Science, 7, p.e349.
[4]
Adarsh, R., Amarnageswarao, G., Pandeeswari, R. and Deivalakshmi, S., 2020, September. Inception block based residual auto encoder for lung segmentation. In 2020 4th International Conference on Computer, Communication and Signal Processing (ICCCSP) (pp. 1-5). IEEE.
[5]
Guo, L., Wang, C., Li, F., He, H. and Li, F., 2020. Binary surface smoothing for abnormal lung segmentation. Computers & Graphics, 89, pp.68-76.
[6]
Zhou, T., Canu, S. and Ruan, S., 2021. Automatic COVID‐19 CT segmentation using U‐Net integrated spatial and channel attention mechanism. International Journal of Imaging Systems and Technology, 31(1), pp.16-27.
[7]
Yan, Q., Wang, B., Gong, D., Luo, C., Zhao, W., Shen, J., Shi, Q., Jin, S., Zhang, L. and You, Z., 2020. COVID-19 chest CT image segmentation–a deep convolutional neural network solution. arXiv preprint arXiv:2004.10987.
[8]
Gerard, S. E., Herrmann, J., Kaczka, D. W., Musch, G., Fernandez-Bustamante, A., & Reinhardt, J. M. (2020). Multi-resolution convolutional neural networks for fully automated segmentation of acutely injured lungs in multiple species. Medical image analysis, 60, 101592.
[9]
Ronneberger, O., Fischer, P. and Brox, T., 2015. U-net: Convolutional networks for biomedical image segmentation. In Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18 (pp. 234-241). Springer International Publishing.
[10]
Zhou, Z., Rahman Siddiquee, M.M., Tajbakhsh, N. and Liang, J., 2018. Unet++: A nested u-net architecture for medical image segmentation. In Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support: 4th International Workshop, DLMIA 2018, and 8th International Workshop, ML-CDS 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 20, 2018, Proceedings 4 (pp. 3-11). Springer International Publishing.
[11]
Xiao, X., Lian, S., Luo, Z., & Li, S. 2018, October. Weighted res-unet for high-quality retina vessel segmentation. In 2018 9th international conference on information technology in medicine and education (ITME) (pp. 327-331). IEEE.
[12]
Alom, M. Z., Hasan, M., Yakopcic, C., Taha, T. M., & Asari, V. K. 2018. Recurrent residual convolutional neural network based on u-net (r2u-net) for medical image segmentation. arXiv preprint arXiv:1802.06955.
[13]
Oktay, O., Schlemper, J., Folgoc, L. L., Lee, M., Heinrich, M., Misawa, K., ... & Rueckert, D. 2018. Attention u-net: Learning where to look for the pancreas. arXiv preprint arXiv:1804.03999.
[14]
Iglovikov, V. and Shvets, A., 2018. Ternausnet: U-net with vgg11 encoder pre-trained on imagenet for image segmentation. ar Xiv preprint ar Xiv:1801.05746.
[15]
Isensee, F., Jäger, P. F., Kohl, S. A., Petersen, J., & Maier-Hein, K. H. (2019). Automated design of deep learning methods for biomedical image segmentation. ar Xiv preprint ar Xiv:1904.08128.
[16]
Jha, D., Riegler, M.A., Johansen, D., Halvorsen, P. and Johansen, H.D., 2020, July. Doubleu-net: A deep convolutional neural network for medical image segmentation. In 2020 IEEE 33rd International symposium on computer-based medical systems (CBMS) (pp. 558-564). IEEE.
[17]
Lou, A., Guan, S., & Loew, M. (2021, February). DC-UNet: rethinking the U-Net architecture with dual channel efficient CNN for medical image segmentation. In Medical Imaging 2021: Image Processing (Vol. 11596, pp. 758-768). SPIE.
[18]
Valanarasu, J. M. J., Sindagi, V. A., Hacihaliloglu, I., & Patel, V. M. 2021. Kiu-net: Overcomplete convolutional architectures for biomedical image and volumetric segmentation. IEEE Transactions on Medical Imaging, 41(4), 965-976.
[19]
Jiang, S. and Li, J., 2022. TransCUNet: UNet cross fused transformer for medical image segmentation. Computers in Biology and Medicine, 150, p.106207.
[20]
Dinh, L., Sohl-Dickstein, J. and Bengio, S., 2016. Density estimation using real nvp. arXiv preprint arXiv:1605.08803.
[21]
Wang, H., Cao, P., Wang, J., & Zaiane, O. R. 2022, June. Uctransnet: rethinking the skip connections in u-net from a channel-wise perspective with transformer. In Proceedings of the AAAI conference on artificial intelligence (Vol. 36, No. 3, pp. 2441-2449).
[22]
Cao, H., Wang, Y., Chen, J., Jiang, D., Zhang, X., Tian, Q. and Wang, M., 2022, October. Swin-unet: Unet-like pure transformer for medical image segmentation. In European conference on computer vision (pp. 205-218). Cham: Springer Nature Switzerland.
[23]
Wenxuan, W., Chen, C., Meng, D., Hong, Y., Sen, Z. and Jiangyun, L., 2021. Transbts: Multimodal brain tumor segmentation using transformer. In International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer (pp. 109-119).
[24]
Chen, J., Lu, Y., Yu, Q., Luo, X., Adeli, E., Wang, Y., ... & Zhou, Y. 2021. Transunet: Transformers make strong encoders for medical image segmentation. arXiv preprint arXiv:2102.04306.
[25]
Gao, Y., Zhou, M. and Metaxas, D.N., 2021. UTNet: a hybrid transformer architecture for medical image segmentation. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part III 24 (pp. 61-71). Springer International Publishing.
[26]
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., ... & Guo, B. (2021). Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision (pp. 10012-10022).
[27]
Hu, J., Shen, L., & Sun, G. (2018). Squeeze-and-excitation networks. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 7132-7141).
[28]
Zhang, K., Liu, X., Shen, J., Li, Z., Sang, Y., Wu, X., Zha, Y., Liang, W., Wang, C., Wang, K. and Ye, L., 2020. Clinically applicable AI system for accurate diagnosis, quantitative measurements, and prognosis of COVID-19 pneumonia using computed tomography. Cell, 181(6), pp.1423-1433.
Recommendations
Lung CT Image Segmentation via Dilated U-Net Model and Multi-scale Gray Correlation-Based Approach
AbstractLung segmentation is a prerequisite for lung cancer diagnosis with computer-aided diagnosis systems. However, correct lung segmentation is a challenging task due to image noise, diseases, different lung nodule presences, unique morphological ...
An Efficient Variant of Fully-Convolutional Network for Segmenting Lung Fields from Chest Radiographs
Automatic analysis of chest radiographs using computer-aided diagnosis (CAD) systems is pivotal to perform mass screening and detect early signs of various abnormalities in patients. In a chest radiographic CAD system, segmentation of lung fields is a ...
Vision Transformers for Lung Segmentation on CXR Images
AbstractAccurate segmentation of the lungs in CXR images is the basis for an automated CXR image analysis system. It helps radiologists in detecting lung areas, subtle signs of disease and improving the diagnosis process for patients. However, precise ...
Comments
Information & Contributors
Information
Published In
January 2024
506 pages
ISBN:9798400718199
DOI:10.1145/3653804
Copyright © 2024 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 01 June 2024
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
CVDL 2024
CVDL 2024: The International Conference on Computer Vision and Deep Learning
January 19 - 21, 2024
Changsha, China
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 11Total Downloads
- Downloads (Last 12 months)11
- Downloads (Last 6 weeks)3
Reflects downloads up to 12 Jan 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format