Cited By
View all- Jian BPan JGhahremani MRueckert DWachinger CWiestler B(2024)Mamba? Catch The Hype Or Rethink What Really Helps for Image RegistrationBiomedical Image Registration10.1007/978-3-031-73480-9_7(86-97)Online publication date: 6-Oct-2024
Non-iterative Coarse-to-Fine Transformer Networks for Joint Affine and Deformable Image Registration
Pages 750 - 760
Abstract
Image registration is a fundamental requirement for medical image analysis. Deep registration methods based on deep learning have been widely recognized for their capabilities to perform fast end-to-end registration. Many deep registration methods achieved state-of-the-art performance by performing coarse-to-fine registration, where multiple registration steps were iterated with cascaded networks. Recently, Non-Iterative Coarse-to-finE (NICE) registration methods have been proposed to perform coarse-to-fine registration in a single network and showed advantages in both registration accuracy and runtime. However, existing NICE registration methods mainly focus on deformable registration, while affine registration, a common prerequisite, is still reliant on time-consuming traditional optimization-based methods or extra affine registration networks. In addition, existing NICE registration methods are limited by the intrinsic locality of convolution operations. Transformers may address this limitation for their capabilities to capture long-range dependency, but the benefits of using transformers for NICE registration have not been explored. In this study, we propose a Non-Iterative Coarse-to-finE Transformer network (NICE-Trans) for image registration. Our NICE-Trans is the first deep registration method that (i) performs joint affine and deformable coarse-to-fine registration within a single network, and (ii) embeds transformers into a NICE registration framework to model long-range relevance between images. Extensive experiments with seven public datasets show that our NICE-Trans outperforms state-of-the-art registration methods on both registration accuracy and runtime.
References
[1]
Sotiras A, Davatzikos C, and Paragios N Deformable medical image registration: a survey IEEE Trans. Med. Imaging 2013 32 7 1153-1190
[2]
Meng, M., Liu, S.: High-quality panorama stitching based on asymmetric bidirectional optical flow. In: International Conference on Computational Intelligence and Applications (ICCIA), pp. 118–122 (2020)
[3]
Avants BB, Epstein CL, Grossman M, and Gee JC Symmetric diffeomorphic image registration with cross-correlation: evaluating automated labeling of elderly and neurodegenerative brain Med. Image Anal. 2008 12 1 26-41
[4]
Modat M et al. Fast free-form deformation using graphics processing units Comput. Meth. Programs Biomed. 2010 98 3 278-284
[5]
Haskins G, Kruger U, and Yan P Deep learning in medical image registration: a survey Mach. Vis. Appl. 2020 31 8
[6]
Xiao H et al. A review of deep learning-based three-dimensional medical image registration methods Quant. Imaging Med. Surg. 2021 11 12 4895-4916
[7]
Balakrishnan G et al. Voxelmorph: a learning framework for deformable medical image registration IEEE Trans. Med. Imaging 2019 38 8 1788-1800
[8]
Dalca AV et al. Unsupervised learning of probabilistic diffeomorphic registration for images and surfaces Med. Image Anal. 2019 57 226-236
[9]
Meng M et al. Enhancing medical image registration via appearance adjustment networks Neuroimage 2022 259
[10]
De Vos BD et al. A deep learning framework for unsupervised affine and deformable image registration Med. Image Anal. 2019 52 128-143
[11]
Hering A, van Ginneken B, Heldmann S, et al. Shen D et al. mlVIRNET: multilevel variational image registration network MICCAI 2019 2019 Cham Springer 257-265
[12]
Zhao, S., et al.: Recursive cascaded networks for unsupervised medical image registration. In: IEEE International Conference on Computer Vision, pp. 10600–10610 (2019)
[13]
Mok TCW, Chung ACS, et al. Martel AL et al. Large deformation diffeomorphic image registration with Laplacian pyramid networks MICCAI 2020 2020 Cham Springer 211-221
[14]
Shu Y, et al., et al. deBruijne M, et al., et al. Medical image registration based on uncoupled learning and accumulative enhancement MICCAI 2021 2021 Cham Springer 3-13
[15]
Hu B, Zhou S, Xiong Z, and Wu F Recursive decomposition network for deformable image registration IEEE J. Biomed. Health Inform. 2022 26 10 5130-5141
[16]
Kang M et al. Dual-stream pyramid registration network Med. Image Anal. 2022 78
[17]
Lv J et al. Joint progressive and coarse-to-fine registration of brain MRI via deformation field integration and non-rigid feature fusion IEEE Trans. Med. Imaging 2022 41 10 2788-2802
[18]
Meng M, Bi L, Feng D, Kim J, et al. Wang L et al. Non-iterative coarse-to-fine registration based on single-pass deep cumulative learning MICCAI 2022 2022 Cham Springer 88-97
[19]
Meng, M., Bi, L., Feng, D. Kim, J.: Brain Tumor Sequence Registration with Non-iterative Coarse-to-fine Networks and Dual Deep Supervision. arXiv preprint arXiv:2211.07876 (2022)
[20]
Dosovitskiy, A., et al.: An image is worth 16×16 words: transformers for image recognition at scale. In: International Conference on Learning Representations (2021)
[21]
Chen J et al. Transmorph: transformer for unsupervised medical image registration Med. Image Anal. 2022 82
[22]
Zhu Y, Lu S, et al. Wang L et al. Swin-voxelmorph: a symmetric unsupervised learning model for deformable medical image registration using swin transformer MICCAI 2022 2022 Cham Springer 78-87
[23]
Shi J, et al., et al. Wang L, et al., et al. Xmorpher: full transformer for deformable medical image registration via cross attention MICCAI 2022 2022 Cham Springer 217-226
[24]
Liu, Z., et al.: Swin transformer: hierarchical vision transformer using shifted windows. In: IEEE/CVF International Conference on Computer Vision, pp. 10012–10022 (2021)
[25]
Kuang D, Schmah T, et al. Suk HI et al. Faim–a convnet method for unsupervised 3d medical image registration MLMI 2019 2019 Cham Springer 646-654
[26]
Ashburner J A fast diffeomorphic image registration algorithm Neuroimage 2007 38 1 95-113
[27]
Mueller SG et al. Ways toward an early diagnosis in Alzheimer’s disease: the Alzheimer’s Disease Neuroimaging Initiative (ADNI) Alzheimers Dement. 2005 1 1 55-66
[28]
Martino D et al. The autism brain imaging data exchange: towards a large-scale evaluation of the intrinsic brain architecture in autism Mol. Psychiatry 2014 19 6 659-667
[29]
ADHD-200 consortium.: the ADHD-200 consortium: a model to advance the translational potential of neuroimaging in clinical neuroscience. Front. Syst. Neurosci. 6, 62 (2012)
[30]
The Information eXtraction from Images (IXI) dataset. https://brain-development.org/ixi-dataset/. Accessed 31 Oct 2022
[31]
Klein A and Tourville J 101 labeled brain images and a consistent human cortical labeling protocol Front. Neurosci. 2012 6 171
[32]
Fischl, B.: FreeSurfer. Neuroimage 62(2), 774–781 (2012)
[33]
Shattuck DW et al. Construction of a 3D probabilistic atlas of human cortical structures Neuroimage 2008 39 3 1064-1080
[34]
McCormick M et al. ITK: enabling reproducible research and open science Front. Neuroinform. 2014 8 13
[35]
Jenkinson M and Smith S A global optimisation method for robust affine registration of brain images Med. Image Anal. 2001 5 2 143-156
[36]
Zhao S et al. Unsupervised 3D end-to-end medical image registration with volume tweening network IEEE J. Biomed. Health Inform. 2019 24 5 1394-1404
[37]
Baheti, B., et al.: The brain tumor sequence registration challenge: establishing correspondence between pre-operative and follow-up MRI scans of diffuse glioma patients. arXiv preprint arXiv:2112.06979 (2021)
Recommendations
Brain Tumor Sequence Registration with Non-iterative Coarse-To-Fine Networks and Dual Deep Supervision
Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain InjuriesAbstractIn this study, we focus on brain tumor sequence registration between pre-operative and follow-up Magnetic Resonance Imaging (MRI) scans of brain glioma patients, in the context of Brain Tumor Sequence Registration challenge (BraTS-Reg 2022). Brain ...
TAHIR: Transformer-Based Affine Histological Image Registration
Pattern Recognition, Computer Vision, and Image Processing. ICPR 2022 International Workshops and ChallengesAbstractIn medical practice it is often necessary to jointly analyze differently stained histological sections. However, when slides are being prepared tissues are subjected to deformations and registration is highly required. Although the transformation ...
A Transformer-Based Network for Deformable Medical Image Registration
Artificial IntelligenceAbstractDeformable medical image registration plays an important role in clinical diagnosis and treatment. Recently, the deep learning (DL) based image registration methods have been widely investigated and showed excellent performance in computational ...
Comments
Information & Contributors
Information
Published In
Oct 2023
831 pages
ISBN:978-3-031-43998-8
DOI:10.1007/978-3-031-43999-5
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2023.
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 08 October 2023
Author Tags
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 09 Nov 2024
Other Metrics
Citations
Cited By
View all- Jian BPan JGhahremani MRueckert DWachinger CWiestler B(2024)Mamba? Catch The Hype Or Rethink What Really Helps for Image RegistrationBiomedical Image Registration10.1007/978-3-031-73480-9_7(86-97)Online publication date: 6-Oct-2024
View Options
View options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in