Abstract
Computed Tomography (CT) and Magnetic Resonance Imaging (MRI) are two of the most informative modalities in spinal diagnostics and treatment planning. CT is useful when analysing bony structures, while MRI gives information about the soft tissue. Thus, fusing the information of both modalities can be very beneficial. Registration is the first step for this fusion. While the soft tissues around the vertebra are deformable, each vertebral body is constrained to move rigidly. We propose a weakly-supervised deep learning framework that preserves the rigidity and the volume of each vertebra while maximizing the accuracy of the registration. To achieve this goal, we introduce anatomy-aware losses for training the network. We specifically design these losses to depend only on the CT label maps since automatic vertebra segmentation in CT gives more accurate results contrary to MRI. We evaluate our method on an in-house dataset of 167 patients. Our results show that adding the anatomy-aware losses increases the plausibility of the inferred transformation while keeping the accuracy untouched.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Arsigny, V., Commowick, O., Pennec, X., Ayache, N.: A log-euclidean framework for statistics on diffeomorphisms. In: Larsen, R., Nielsen, M., Sporring, J. (eds.) MICCAI 2006. LNCS, vol. 4190, pp. 924–931. Springer, Heidelberg (2006). https://doi.org/10.1007/11866565_113
Balakrishnan, G., Zhao, A., Sabuncu, M.R., Guttag, J., Dalca, A.V.: A log-Euclidean framework for statistics on diffeomorphisms. IEEE Trans. Med. Imaging 38(8), 1788–1800 (2019)
Bukas, C., et al.: Patient-specific virtual spine straightening and vertebra inpainting: an automatic framework for osteoplasty planning. In: de Bruijne, M., Cattin, P.C., Cotin, S., Padoy, N., Speidel, S., Zheng, Y., Essert, C. (eds.) MICCAI 2021. LNCS, vol. 12904, pp. 529–539. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87202-1_51
De Vos, B.D., Berendsen, F.F., Viergever, M.A., Sokooti, H., Staring, M., Išgum, I.: A deep learning framework for unsupervised affine and deformable image registration. Med. Image Anal. 52, 128–143, (2019)
Fu, Y., et al.: Deformable MR-CBCT prostate registration using biomechanically constrained deep learning networks. Med. Phys. 48(1), 253–263 (2021)
Gill, S., et al.: Biomechanically constrained groupwise ultrasound to CT registration of the lumbar spine. Med. Image Anal. 16(3), 662–674 (2012)
Haber, E., Modersitzki, J.: Intensity gradient based registration and fusion of multi-modal images. In: Larsen, R., Nielsen, M., Sporring, J. (eds.) MICCAI 2006. LNCS, vol. 4191, pp. 726–733. Springer, Heidelberg (2006). https://doi.org/10.1007/11866763_89
Haskins, G., Kruger, U., Yan, P.: Deep learning in medical image registration: a survey. Mach. Vis. Appl. 31(1), 1–18 (2020)
Heinrich, M.P., Jenkinson, M., Papież, B.W., Brady, S.M., Schnabel, J.A.: Towards realtime multimodal fusion for image-guided interventions using self-similarities. In: Mori, K., Sakuma, I., Sato, Y., Barillot, C., Navab, N. (eds.) MICCAI 2013. LNCS, vol. 8149, pp. 187–194. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-40811-3_24
Kim, G.U., Chang, M.C., Kim, T.U., Lee, G.W.: Diagnostic modality in spine disease: a review. Asian Spine Journal 14(6), 910 (2020)
Kim, J., Matuszak, M.M., Saitou, K., Balter, J.M.: Distance-preserving rigidity penalty on deformable image registration of multiple skeletal components in the neck. Med. Phys. 40(12), (2013)
Little, J.A., Hill, D.L., Hawkes, D.J.: Deformations incorporating rigid structures. Comput. Vis. Image Underst. 66(2), 223–232 (1997)
McKenzie, E.M., Santhanam, A., Ruan, D., O’Connor, D., Cao, M., Sheng, K.: Multimodality image registration in the head-and-neck using a deep learning-derived synthetic ct as a bridge. Med. Phys. 47(3), 1094–1104 (2020)
Mok, Tony C. W.., Chung, Albert C. S..: Conditional deformable image registration with convolutional neural network. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12904, pp. 35–45. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87202-1_4
Momin, S., et al.: CT-MRI pelvic deformable registration via deep learning. In: Medical Imaging 2021: Image-Guided Procedures, Robotic Interventions, and Modeling. vol. 11598, p. 1159818. International Society for Optics and Photonics (2021)
Parizel, P., et al.: Trauma of the spine and spinal cord: imaging strategies. Eur. Spine J. 19(1), 8–17 (2010)
Pluim, J.P.W., Maintz, J.B.A., Viergever, M.A.: Image registration by maximization of combined mutual information and gradient information. In: Delp, S.L., DiGoia, A.M., Jaramaz, B. (eds.) MICCAI 2000. LNCS, vol. 1935, pp. 452–461. Springer, Heidelberg (2000). https://doi.org/10.1007/978-3-540-40899-4_46
Rohlfing, T., Maurer, C.R., Bluemke, D.A., Jacobs, M.A.: Volume-preserving nonrigid registration of MR breast images using free-form deformation with an incompressibility constraint. IEEE Trans. Med. Imaging 22(6), 730–741 (2003)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Sekuboyina, A., Rempfler, M., Kukačka, J., Tetteh, G., Valentinitsch, A., Kirschke, J.S., Menze, B.H.: Btrfly Net: vertebrae labelling with energy-based adversarial learning of local spine prior. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11073, pp. 649–657. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00937-3_74
Shah, L.M., Salzman, K.L.: Imaging of spinal metastatic disease. Int. J. Surg. Oncol. 2011 (2011)
Sorkine-Hornung, O., Rabinovich, M.: Least-squares rigid motion using svd (2017)
Staring, M., Klein, S., Pluim, J.P.: A rigidity penalty term for nonrigid registration. Med. Phys. 34(11), 4098–4108 (2007)
Tins, B.: Technical aspects of ct imaging of the spine. Insights Imaging 1(5), 349–359 (2010)
Wells, W.M., III., Viola, P., Atsumi, H., Nakajima, S., Kikinis, R.: Multi-modal volume registration by maximization of mutual information. Med. Image Anal. 1(1), 35–51 (1996)
Acknowledgements
This work was partially supported by the Bavarian State Ministry of Economics in the frame of the BayVFP Funding Line Digitalization - Information and Communication Technology, grant DIK0127/01 (“DigiBiop”).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Jian, B. et al. (2022). Weakly-Supervised Biomechanically-Constrained CT/MRI Registration of the Spine. In: Wang, L., Dou, Q., Fletcher, P.T., Speidel, S., Li, S. (eds) Medical Image Computing and Computer Assisted Intervention – MICCAI 2022. MICCAI 2022. Lecture Notes in Computer Science, vol 13436. Springer, Cham. https://doi.org/10.1007/978-3-031-16446-0_22
Download citation
DOI: https://doi.org/10.1007/978-3-031-16446-0_22
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-16445-3
Online ISBN: 978-3-031-16446-0
eBook Packages: Computer ScienceComputer Science (R0)
