FAFuse: A Four-Axis Fusion framework of CNN and Transformer for medical image segmentation
Computers in Biology and Medicine, 2023•Elsevier
Medical image segmentation is crucial for accurate diagnosis and treatment in the medical
field. In recent years, convolutional neural networks (CNNs) and Transformers have been
frequently adopted as network architectures in medical image segmentation. The
convolution operation is limited in modeling long-range dependencies because it can only
extract local information through the limited receptive field. In comparison, Transformers
demonstrate excellent capability in modeling long-range dependencies but are less effective …
field. In recent years, convolutional neural networks (CNNs) and Transformers have been
frequently adopted as network architectures in medical image segmentation. The
convolution operation is limited in modeling long-range dependencies because it can only
extract local information through the limited receptive field. In comparison, Transformers
demonstrate excellent capability in modeling long-range dependencies but are less effective …
Abstract
Medical image segmentation is crucial for accurate diagnosis and treatment in the medical field. In recent years, convolutional neural networks (CNNs) and Transformers have been frequently adopted as network architectures in medical image segmentation. The convolution operation is limited in modeling long-range dependencies because it can only extract local information through the limited receptive field. In comparison, Transformers demonstrate excellent capability in modeling long-range dependencies but are less effective in capturing local information. Hence, effectively modeling long-range dependencies while preserving local information is essential for accurate medical image segmentation. In this paper, we propose a four-axis fusion framework called FAFuse, which can exploit the advantages of CNN and Transformer. As the core component of our FAFuse, a Four-Axis Fusion module (FAF) is proposed to efficiently fuse global and local information. FAF combines Four-Axis attention (height, width, main diagonal, and counter diagonal axial attention), a multi-scale convolution, and a residual structure with a depth-separable convolution and a Hadamard product. Furthermore, we also introduce deep supervision to enhance gradient flow and improve overall performance. Our approach achieves state-of-the-art segmentation accuracy on three publicly available medical image segmentation datasets. The code is available at https://github.com/cczu-xiao/FAFuse.
Elsevier