Abstract
This paper presents a novel approach, termed Temporal Latent Residual Network (TLRN), to predict a sequence of deformation fields in time-series image registration. The challenge of registering time-series images often lies in the occurrence of large motions, especially when images differ significantly from a reference (e.g., the start of a cardiac cycle compared to the peak stretching phase). To achieve accurate and robust registration results, we leverage the nature of motion continuity and exploit the temporal smoothness in consecutive image frames. Our proposed TLRN highlights a temporal residual network with residual blocks carefully designed in latent deformation spaces, which are parameterized by time-sequential initial velocity fields. We treat a sequence of residual blocks over time as a dynamic training system, where each block is designed to learn the residual function between desired deformation features and current input accumulated from previous time frames. We validate the effectivenss of TLRN on both synthetic data and real-world cine cardiac magnetic resonance (CMR) image videos. Our experimental results shows that TLRN is able to achieve substantially improved registration accuracy compared to the state-of-the-art. Our code is publicly available at https://github.com/nellie689/TLRN.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Arsigny, V., Commowick, O., Pennec, X., Ayache, N.: A log-euclidean framework for statistics on diffeomorphisms. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 924–931. Springer (2006)
Balakrishnan, G., Zhao, A., Sabuncu, M.R., Guttag, J., Dalca, A.V.: An unsupervised learning model for deformable medical image registration. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 9252–9260 (2018)
Balakrishnan, G., Zhao, A., Sabuncu, M.R., Guttag, J., Dalca, A.V.: Voxelmorph: a learning framework for deformable medical image registration. IEEE transactions on medical imaging 38(8), 1788–1800 (2019)
Beg, M.F., Miller, M.I., Trouvé, A., Younes, L.: Computing large deformation metric mappings via geodesic flows of diffeomorphisms. International journal of computer vision 61(2), 139–157 (2005)
Chen, J., Frey, E.C., He, Y., Segars, W.P., Li, Y., Du, Y.: Transmorph: Transformer for unsupervised medical image registration. Medical image analysis 82, 102615 (2022)
Csapo, I., Holland, C.M., Guttmann, C.R.: Image registration framework for large-scale longitudinal mri data sets: strategy and validation. Magnetic Resonance Imaging 25(6), 889–893 (2007)
De Craene, M., Piella, G., Camara, O., Duchateau, N., Silva, E., Doltra, A., D’hooge, J., Brugada, J., Sitges, M., Frangi, A.F.: Temporal diffeomorphic free-form deformation: Application to motion and strain estimation from 3d echocardiography. Medical image analysis 16(2), 427–450 (2012)
Dice, L.R.: Measures of the amount of ecologic association between species. Ecology 26(3), 297–302 (1945)
Geng, X., Christensen, G.E., Gu, H., Ross, T.J., Yang, Y.: Implicit reference-based group-wise image registration and its application to structural and functional mri. Neuroimage 47(4), 1341–1351 (2009)
Ghanem, B., Zhang, T., Ahuja, N.: Robust video registration applied to field-sports video analysis. In: IEEE International conference on acoustics, speech, and signal processing (ICASSP). vol. 2 (2012)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 770–778 (2016)
Hinkle, J., Womble, D., Yoon, H.J.: Diffeomorphic autoencoders for lddmm atlas building (2018)
Hong, Y., Golland, P., Zhang, M.: Fast geodesic regression for population-based image analysis. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 317–325. Springer (2017)
Huttenlocher, D.P., Klanderman, G.A., Rucklidge, W.J.: Comparing images using the hausdorff distance. IEEE Transactions on pattern analysis and machine intelligence 15(9), 850–863 (1993)
Joshi, A., Hong, Y.: Diffeomorphic image registration using lipschitz continuous residual networks. In: International Conference on Medical Imaging with Deep Learning. pp. 605–617. PMLR (2022)
Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Krebs, J., Delingette, H., Ayache, N., Mansi, T.: Learning a generative motion model from image sequences based on a latent motion matrix. IEEE Transactions on Medical Imaging 40(5), 1405–1416 (2021)
Krebs, J., Mansi, T., Ayache, N., Delingette, H.: Probabilistic motion modeling from medical image sequences: application to cardiac cine-mri. In: Statistical Atlases and Computational Models of the Heart. Multi-Sequence CMR Segmentation, CRT-EPiggy and LV Full Quantification Challenges: 10th International Workshop, STACOM 2019, Held in Conjunction with MICCAI 2019, Shenzhen, China, October 13, 2019, Revised Selected Papers 10. pp. 176–185. Springer (2020)
Ledesma-Carbayo, M.J., Kybic, J., Desco, M., Santos, A., Suhling, M., Hunziker, P., Unser, M.: Spatio-temporal nonrigid registration for ultrasound cardiac motion estimation. IEEE transactions on medical imaging 24(9), 1113–1126 (2005)
Liao, R., Turk, E.A., Zhang, M., Luo, J., Grant, P.E., Adalsteinsson, E., Golland, P.: Temporal registration in in-utero volumetric mri time series. In: Medical Image Computing and Computer-Assisted Intervention-MICCAI 2016: 19th International Conference, Athens, Greece, October 17-21, 2016, Proceedings, Part III 19. pp. 54–62. Springer (2016)
Maas, A.L., Hannun, A.Y., Ng, A.Y., et al.: Rectifier nonlinearities improve neural network acoustic models. In: Proc. icml. vol. 30, p. 3. Atlanta, GA (2013)
Metz, C.T., Klein, S., Schaap, M., van Walsum, T., Niessen, W.J.: Nonrigid registration of dynamic medical imaging data using nd+ t b-splines and a groupwise optimization approach. Medical image analysis 15(2), 238–249 (2011)
Morais, P., Heyde, B., Barbosa, D., Queirós, S., Claus, P., D’hooge, J.: Cardiac motion and deformation estimation from tagged mri sequences using a temporal coherent image registration framework. In: Functional Imaging and Modeling of the Heart: 7th International Conference, FIMH 2013, London, UK, June 20-22, 2013. Proceedings 7. pp. 316–324. Springer (2013)
Niethammer, M., Huang, Y., Vialard, F.X.: Geodesic regression for image time-series. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2011: 14th International Conference, Toronto, Canada, September 18-22, 2011, Proceedings, Part II 14. pp. 655–662. Springer (2011)
Perperidis, D., Mohiaddin, R.H., Rueckert, D.: Spatio-temporal free-form registration of cardiac mr image sequences. Medical image analysis 9(5), 441–456 (2005)
Qiao, M., Wang, S., Qiu, H., De Marvao, A., O’Regan, D.P., Rueckert, D., Bai, W.: Cheart: A conditional spatio-temporal generative model for cardiac anatomy. IEEE transactions on medical imaging (2023)
Qin, C., Wang, S., Chen, C., Bai, W., Rueckert, D.: Generative myocardial motion tracking via latent space exploration with biomechanics-informed prior. Medical Image Analysis 83, 102682 (2023)
Reinhardt, J.M., Ding, K., Cao, K., Christensen, G.E., Hoffman, E.A., Bodas, S.V.: Registration-based estimates of local lung tissue expansion compared to xenon ct measures of specific ventilation. Medical image analysis 12(6), 752–763 (2008)
Singh, M., Thompson, R., Basu, A., Rieger, J., Mandal, M.: Image based temporal registration of mri data for medical visualization. In: 2006 International Conference on Image Processing. pp. 1169–1172. IEEE (2006)
Vercauteren, T., Pennec, X., Perchant, A., Ayache, N.: Symmetric log-domain diffeomorphic registration: A demons-based approach. In: International conference on medical image computing and computer-assisted intervention. pp. 754–761. Springer (2008)
Wang, J., Zhang, M.: Deepflash: An efficient network for learning-based medical image registration. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 4444–4452 (2020)
Wu, N., Zhang, M.: Neurepdiff: Neural operators to predict geodesics in deformation spaces. In: International Conference on Information Processing in Medical Imaging. pp. 588–600. Springer (2023)
Xing, J., Wu, N., Bilchick, K., Epstein, F., Zhang, M.: Multimodal learning to improve cardiac late mechanical activation detection from cine mr images. arXiv preprint arXiv:2402.18507 (2024)
Yang, Z., Dan, T., Yang, Y.: Multi-temporal remote sensing image registration using deep convolutional features. Ieee Access 6, 38544–38555 (2018)
Acknowledgments
This work was supported by NSF CAREER Grant 2239977 and NIH 1R21EB032597.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Ethics declarations
Disclosure of Interests
The authors have no competing interests to declare that are relevant to the content of this article.
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Wu, N., Xing, J., Zhang, M. (2024). TLRN: Temporal Latent Residual Networks for Large Deformation Image Registration. In: Linguraru, M.G., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2024. MICCAI 2024. Lecture Notes in Computer Science, vol 15002. Springer, Cham. https://doi.org/10.1007/978-3-031-72069-7_68
Download citation
DOI: https://doi.org/10.1007/978-3-031-72069-7_68
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-72068-0
Online ISBN: 978-3-031-72069-7
eBook Packages: Computer ScienceComputer Science (R0)
