Abstract
The accurate reconstruction of accelerated Magnetic Resonance Imaging (MRI) brings significant clinical benefits, including improved diagnostic accuracy and reduced examination costs. Traditional cardiac MRI requires repetitive acquisitions over multiple heartbeats, leading to longer acquisition times. Deep learning-based MRI reconstruction methods have made significant progress in accelerating MRI. However, existing methods suffer from the following limitations: (1) Due to the involvement of multiple complex time-series data and image information in the process of heart reconstruction, exploring the nonlinear dependencies between temporal contexts is challenging. (2) Most of the research has neglected weight sharing in iterative frameworks, which prevents better capturing of long-range/non-local information in the data, thus restricting the improvement of model performance. In this paper, we propose a novel Multi-level Temporal Information Sharing Transformer to enhance cardiac MRI reconstruction. Based on the Transformer’s multi-level encoder and decoder architecture, we perform multi-level temporal information feature aggregation across multiple adjacent views, establishing nonlinear dependencies between features and effectively learning crucial information between adjacent cardiac temporal frames. We also incorporate cross-view attention for temporal information interaction and fusion to enhance contextual understanding between adjacent views. Additionally, in the reconstruction process, we introduce a training approach of feature reuse to update weights, enhancing feature fusion in important regions and calculating global feature dependencies with fewer computations. Numerous experiments have indicated that this method is significantly superior to state-of-the-art techniques, thereby holding the potential for widespread application in the clinical field.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Aggarwal, H.K., Mani, M.P., Jacob, M.: MoDL: model-based deep learning architecture for inverse problems. IEEE Trans. Med. Imaging 38(2), 394–405 (2018)
Ahmed, A.H., Zhou, R., Yang, Y., Nagpal, P., Salerno, M., Jacob, M.: Free-breathing and ungated dynamic MRI using navigator-less spiral storm. IEEE Trans. Med. Imaging 39(12), 3933–3943 (2020)
Carreira, J., Zisserman, A.: Quo vadis, action recognition? A new model and the kinetics dataset. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6299–6308 (2017)
Guo, X., Guo, X., Lu, Y.: SSAN: separable self-attention network for video representation learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12618–12627 (2021)
Hara, K., Kataoka, H., Satoh, Y.: Learning spatio-temporal features with 3D residual networks for action recognition. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 3154–3160 (2017)
Ho, J., Kalchbrenner, N., Weissenborn, D., Salimans, T.: Axial attention in multidimensional transformers. arXiv preprint arXiv:1912.12180 (2019)
Jung, H., Ye, J.C., Kim, E.Y.: Improved k-t blast and k-t sense using FOCUSS. Phys. Med. Biol. 52(11), 3201 (2007)
Liang, J., et al.: VRT: a video restoration transformer. arXiv preprint arXiv:2201.12288 (2022)
Liang, J., Cao, J., Sun, G., Zhang, K., Van Gool, L., Timofte, R.: SwinIR: image restoration using swin transformer. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1833–1844 (2021)
Lin, J., et al.: Flow-guided sparse transformer for video deblurring. arXiv preprint arXiv:2201.01893 (2022)
Lingala, S.G., Hu, Y., DiBella, E., Jacob, M.: Accelerated dynamic MRI exploiting sparsity and low-rank structure: kt SLR. IEEE Trans. Med. Imaging 30(5), 1042–1054 (2011)
Liu, R., et al.: FuseFormer: Fusing fine-grained information in transformers for video inpainting. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 14040–14049 (2021)
Liu, Z., et al.: Swin transformer: hierarchical vision transformer using shifted windows. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10012–10022 (2021)
Lyu, J., et al.: Region-focused multi-view transformer-based generative adversarial network for cardiac cine MRI reconstruction. Med. Image Anal. 85, 102760 (2023)
Lyu, J., Sui, B., Wang, C., Tian, Y., Dou, Q., Qin, J.: DuDoCAF: dual-domain cross-attention fusion with recurrent transformer for fast multi-contrast MR Imaging. In: Wang, L., Dou, Q., Fletcher, P.T., Speidel, S., Li, S. (eds.) MICCAI 2022. LNCS, vol. 13436, pp. 474–484. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-16446-0_45
Murugesan, B., Vijaya Raghavan, S., Sarveswaran, K., Ram, K., Sivaprakasam, M.: Recon-GLGAN: a global-local context based generative adversarial network for MRI reconstruction. In: Knoll, F., Maier, A., Rueckert, D., Ye, J.C. (eds.) MLMIR 2019. LNCS, vol. 11905, pp. 3–15. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-33843-5_1
Otazo, R., Candes, E., Sodickson, D.K.: Low-rank plus sparse matrix decomposition for accelerated dynamic MRI with separation of background and dynamic components. Magn. Reson. Med. 73(3), 1125–1136 (2015)
Piergiovanni, A., Kuo, W., Angelova, A.: Rethinking video viTs: sparse video tubes for joint image and video learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2214–2224 (2023)
Qin, C., Schlemper, J., Caballero, J., Price, A.N., Hajnal, J.V., Rueckert, D.: Convolutional recurrent neural networks for dynamic MR Image reconstruction. IEEE Trans. Med. Imaging 38(1), 280–290 (2018)
Ramanarayanan, S., Murugesan, B., Ram, K., Sivaprakasam, M.: DC-WCNN: a deep cascade of wavelet based convolutional neural networks for MR Image reconstruction. In: 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI), pp. 1069–1073. IEEE (2020)
Schlemper, J., Caballero, J., Hajnal, J.V., Price, A., Rueckert, D.: A deep cascade of convolutional neural networks for MR image reconstruction. In: Niethammer, M., et al. (eds.) IPMI 2017. LNCS, vol. 10265, pp. 647–658. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-59050-9_51
Simonyan, K., Zisserman, A.: Two-stream convolutional networks for action recognition in videos. In: Advances in Neural Information Processing Systems, vol. 27 (2014)
Touvron, H., Cord, M., Sablayrolles, A., Synnaeve, G., Jégou, H.: Going deeper with image transformers. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 32–42 (2021)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Wang, C., et al.: CMR\(\times \)Recon: an open cardiac MRI dataset for the competition of accelerated image reconstruction. arXiv preprint arXiv:2309.10836 (2023)
Wang, Y., et al.: End-to-end video instance segmentation with transformers. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8741–8750 (2021)
Xing, Z., Yu, L., Wan, L., Han, T., Zhu, L.: NestedFormer: nested modality-aware transformer for brain tumor segmentation. In: Wang, L., Dou, Q., Fletcher, P.T., Speidel, S., Li, S. (eds.) MICCAI 2022. LNCS, vol. 13435, pp. 140–150. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-16443-9_14
Xing, Z., Dai, Q., Hu, H., Chen, J., Wu, Z., Jiang, Y.G.: SVFormer: semi-supervised video transformer for action recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 18816–18826 (2023)
Yan, S., et al.: Multiview transformers for video recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3333–3343 (2022)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Wang, G., Lyu, J., Wang, F., Wang, C., Qin, J. (2024). Multi-level Temporal Information Sharing Transformer-Based Feature Reuse Network for Cardiac MRI Reconstruction. In: Camara, O., et al. Statistical Atlases and Computational Models of the Heart. Regular and CMRxRecon Challenge Papers. STACOM 2023. Lecture Notes in Computer Science, vol 14507. Springer, Cham. https://doi.org/10.1007/978-3-031-52448-6_39
Download citation
DOI: https://doi.org/10.1007/978-3-031-52448-6_39
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-52447-9
Online ISBN: 978-3-031-52448-6
eBook Packages: Computer ScienceComputer Science (R0)