Abstract
Deformable image registration is a fundamental problem in medical image analysis. During the last years, several methods based on deep convolutional neural networks (CNN) proved to be highly accurate to perform this task. These models achieved state-of-the-art accuracy while drastically reducing the required computational time, but mainly focusing on images of specific organs and modalities. To date, no work has reported on how these models adapt across different domains. In this work, we ask the question: can we use CNN-based registration models to spatially align images coming from a domain different than the one/s used at training time? We explore the adaptability of CNN-based image registration to different organs/modalities. We employ a fully convolutional architecture trained following an unsupervised approach. We consider a simple transfer learning strategy to study the generalisation of such model to unseen target domains, and devise a one-shot learning scheme taking advantage of the unsupervised nature of the proposed method. Evaluation on two publicly available datasets of X-Ray lung images and cardiac cine magnetic resonance sequences is provided. Our experiments suggest that models learned in different domains can be transferred at the expense of a decrease in performance, and that one-shot learning in the context of unsupervised CNN-based registration is a valid alternative to achieve consistent registration performance when only a pair of images from the target domain is available.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
A complete list of configuration parameters for Elastix can be found in http://elastix.bigr.nl/wiki/index.php/Parameter_file_database.
- 2.
publicly available at https://github.com/mshunshin/SegNetCMR.
- 3.
Fine-tuning for 50 iterations takes only 0.5 s on GPU. When training from scratch/fine-tuning until convergence, we update the model for 3000 iterations leading to about 30 s per registration case.
- 4.
We used Python and Tensorflow for implementation. Experiments were run in a machine with CPU Intel Core i7-7700, 64GB of RAM and NVidia Titan XP GPU. In order to encourage reproducible research, the project source code and Elastix parameter files can be downloaded from: https://gitlab.com/eferrante/.
- 5.
The CNN-based models take 0.06s on GPU and 0.08 s on CPU to register a pair of images, while Elastix 2.47s. In all the experiments we used Adam optimization, with LR = 1e-4 and \(\lambda _1 = \lambda _2\)=1e-6.
- 6.
Elastix parameters were chosen by grid search using the training data and are available online in our project website.
- 7.
We experimented with fine-tuning the model in whole or in part, but we found that fine-tuning the complete model achieved better results in general.
References
Balakrishnan, G., et al.: An unsupervised learning model for deformable medical image registration. Accepted at CVPR 2018 (2018)
Ganin, Y., Lempitsky, V.: Unsupervised domain adaptation by backpropagation. ICML (2015)
Guerrero, R., et al.: White matter hyperintensity and stroke lesion segmentation and differentiation using cnns. NeuroImage: Clin. 17, 918–934 (2018)
Jaderberg, M., et al.: Spatial transformer networks. In: NIPS, pp. 2017–2025 (2015)
Kawaguchi, T., Harada, Y., Nagata, R., Miyake, H.: Image registration methods for contralateral subtraction of chest radiographs. In: IEE BMEI (2010)
Klein, S., Staring, M., Murphy, K., Viergever, M.A., Pluim, J.P.: Elastix: a toolbox for intensity-based medical image registration. IEEE TMI 29(1), 196–205 (2010)
Li, H., Fan, Y.: Non-rigid image registration using self-supervised fully convolutional networks without training data. Accepted at ISBI 2018 (2018)
Paragios, N.: (hyper)-graphical models in biomedical image analysis. Med. Image Anal. 33, 102–106 (2016)
Phatak, N.S.: Strain measurement in the left ventricle during systole with deformable image registration. Med. Image Anal. 13(2), 354–361 (2009)
Radau, P., Lu, Y., Connelly, K., Paul, G., et al.: Evaluation framework for algorithms segmenting short axis cardiac MRI. MIDAS J. 49 (2009)
Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: MICCAI, pp. 234–241. Springer, Cham (2015)
Salakhutdinov, R., Tenenbaum, J., Torralba, A.: One-shot learning with a hierarchical nonparametric Bayesian model. In: ICML Workshop Proceedings (2012)
Shiraishi, J.: Development of a digital image database for chest radiographs with and without a lung nodule: receiver operating characteristic analysis of radiologists’ detection of pulmonary nodules. Am. J. Roentgenol. 174(1), 71–74 (2000)
Sokooti, Hessam, de Vos, Bob, Berendsen, Floris, Lelieveldt, Boudewijn P.F., Išgum, Ivana, Staring, Marius: Nonrigid Image Registration Using Multi-scale 3D Convolutional Neural Networks. In: Descoteaux, Maxime, Maier-Hein, Lena, Franz, Alfred, Jannin, Pierre, Collins, D.Louis, Duchesne, Simon (eds.) MICCAI 2017. LNCS, vol. 10433, pp. 232–239. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66182-7_27
Thirion, J.P.: Image matching as a diffusion process: an analogy with Maxwell’s demons. Med. Image Anal. 2(3), 243–260 (1998)
de Vos, Bob D., Berendsen, Floris F., Viergever, Max A., Staring, Marius, Išgum, Ivana: End-to-End Unsupervised Deformable Image Registration with a Convolutional Neural Network. In: Cardoso, M.Jorge, Arbel, Tal, Carneiro, Gustavo, Syeda-Mahmood, Tanveer, Tavares, João Manuel R.S., Moradi, Mehdi, Bradley, Andrew, Greenspan, Hayit, Papa, João Paulo, Madabhushi, Anant, Nascimento, Jacinto C., Cardoso, Jaime S., Belagiannis, Vasileios, Lu, Zhi (eds.) DLMIA/ML-CDS -2017. LNCS, vol. 10553, pp. 204–212. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-67558-9_24
Acknowledgements
EF is beneficiary of an AXA Research Grant. We thank NVIDIA Corporation for the donation of the Titan X GPU used for this project.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Ferrante, E., Oktay, O., Glocker, B., Milone, D.H. (2018). On the Adaptability of Unsupervised CNN-Based Deformable Image Registration to Unseen Image Domains. In: Shi, Y., Suk, HI., Liu, M. (eds) Machine Learning in Medical Imaging. MLMI 2018. Lecture Notes in Computer Science(), vol 11046. Springer, Cham. https://doi.org/10.1007/978-3-030-00919-9_34
Download citation
DOI: https://doi.org/10.1007/978-3-030-00919-9_34
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-00918-2
Online ISBN: 978-3-030-00919-9
eBook Packages: Computer ScienceComputer Science (R0)