A Deep Learning Model for Multi-Domain MRI Synthesis Using Generative Adversarial Networks
Abstract
In recent years, Magnetic Resonance Imaging (MRI) has emerged as a prevalent medical imaging technique, offering comprehensive anatomical and functional information. However, the MRI data acquisition process presents several challenges, including time-consuming procedures, prone motion artifacts, and hardware constraints. To address these limitations, this study proposes a novel method that leverages the power of generative adversarial networks (GANs) to generate multi-domain MRI images from a single input MRI image. Within this framework, two primary generator architectures, namely ResUnet and StarGANs generators, were incorporated. Furthermore, the networks were trained on multiple datasets, thereby augmenting the available data, and enabling the generation of images with diverse contrasts obtained from different datasets, given an input image from another dataset. Experimental evaluations conducted on the IXI and BraTS2020 datasets substantiate the efficacy of the proposed method compared to an existing method, as assessed through metrics such as Structural Similarity Index (SSIM), Peak Signal-to-Noise Ratio (PSNR) and Normalized Mean Absolute Error (NMAE). The synthesized images resulting from this method hold substantial potential as invaluable resources for medical professionals engaged in research, education, and clinical applications. Future research gears towards expanding experiments to larger datasets and encompassing the proposed approach to 3D images, enhancing medical diagnostics within practical applications.
References
[1]
Ali, H., Biswas, M.R., Mohsen, F., Shah U., Alamgir A., Mousa O., Shah Z. (2022). The role of generative adversarial networks in brain MRI: a scoping review. Insights Imaging, 13, 98.
[2]
Avants, B.B., Tustison, N., Song, G., (2009). Advanced normalization tools (ANTS). Insight Journal, 2(365), 1–35.
[3]
Bharti, S.K., Inani, H., Gupta, R.K. (2022). Smart photo editor using generative adversarial network: a machine learning approach. In: 2022 IEEE 2nd International Symposium on Sustainable Energy, Signal Processing and Cyber Security (ISSSC), pp. 1–6.
[4]
Chen W., Wu S., Wang S., Li Z., Yang J., Yao H., Song X. (2023). Compound attention and neighbor matching network for multi-contrast MRI super-resolution. arXiv preprint arXiv:2307.02148.
[5]
Choi, Y., Choi, M., Kim, M., Ha, J.-W., Kim, S., Choo, J. (2018). StarGAN: unified generative adversarial networks for multi-domain image-to-image translation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8789–8797.
[6]
Choi, Y., Uh, Y., Yoo, J., Ha, J.-W. (2020). StarGAN v2: diverse image synthesis for multiple domains. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8188–8197.
[7]
Dai, X., Lei, Y., Fu, Y., Curran, W.J., Liu, T., Mao, H., Yang, X. (2020). Multimodal MRI synthesis using unified generative adversarial networks. Medical Physics, 47(12), 6343–6354.
[8]
Geethanath, S., Vaughan Jr., J.T. (2019). Accessible magnetic resonance imaging: a review. Journal of Magnetic Resonance Imaging, 49(7), 65–77.
[9]
Grigas, O., Maskeliūnas, R., Damaševičius, R. (2023). Improving structural MRI preprocessing with hybrid transformer GANs. Life, 13(9), 1893.
[10]
Hatamizadeh, A., Nath, V., Tang, Y., Yang, D., Roth, H.R., Xu, D. (2021). Swin UNETR: swin transformers for semantic segmentation of brain tumors in MRI images. In: International MICCAI Brainlesion Workshop. Springer, Cham, pp. 272–284.
[11]
Hien, N.L.H., Hoang, H.M., Tien, N.V., Hieu, N.V. (2021). Keyphrase extraction model: a new design and application on tourism information. Informatica, 45(4), 563–569.
[12]
Hien, N.L.H., Tien, T.Q., Hieu, N.V. (2020). Web crawler: design and implementation for extracting article-like contents. Cybernetics and Physics, 9(3), 144–151.
[13]
Hien, N.L.H., Van Huy, L., Van Hieu, N. (2021). Artwork style transfer model using deep learning approach. Cybernetics and Physics, 10, 127–137.
[14]
Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., Courville, A. (2017). Improved training of wasserstein GANs. arXiv preprint arXiv:1704.00028.
[15]
Katti, G., Ara, S.A., Shireen, A. (2011). Magnetic resonance imaging (MRI)– a review. International Journal of Dental Clinics, 3(1), 65–70.
[16]
Kingma, D.P., Ba, J.L. (2014). Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980.
[17]
Li, H., Paetzold, J.C., Sekuboyina, A., Kofler, F., Zhang, J., Kirschke, J.S., Wiestler, B., Menze, B. (2019). DiamondGAN: unified multi-modal generative adversarial networks for MRI sequences synthesis. In: Medical Image Computing and Computer Assisted Intervention–MICCAI 2019: 22nd International Conference, Shenzhen, China, October 13–17, 2019, Proceedings, Part IV 22, pp. 795–803. Springer.
[18]
Liang, Z.-P., Lauterbur, P.C. (2000). Principles of Magnetic Resonance Imaging. SPIE Optical Engineering Press, Belllingham, WA.
[19]
Liu, Y., Lei, Y., Wang, Y., Shafai-Erfani, G., Wang, T., Tian, S., Patel, P., Jani, A.B., McDonald, M., Curran, W.J., Liu, T., Zhou, J., Yang, X. (2019). Evaluation of a deep learning-based pelvic synthetic CT generation technique for MRI-based prostate proton treatment planning. Physics in Medicine & Biology, 64(20), 205022.
[20]
Massa, H.A., Johnson, J.M., McMillan, A.B. (2020). Comparison of deep learning synthesis of synthetic CTs using clinical MRI inputs. Physics in Medicine & Biology, 65(23), 23N03.
[21]
Nelson, S., Menon, R. (2022). Bijective-constrained cycle-consistent deep learning for optics-free imaging and classification. Optica, 9(1), 26–31.
[22]
Shen, L., Zhu, W., Wang, X., Xing, L., Pauly, J.M., Turkbey, B., Harmon, S.A., Sanford, T.H., Mehralivand, S., Choyke, P.L., Wood, B., Xu, D. (2020). Multi-domain image completion for random missing input data. IEEE Transactions on Medical Imaging, 40(4), 1113–1122.
[23]
Sohail, M., Riaz, M.N., Wu, J., Long, C., Li, S. (2019). Unpaired multi-contrast MR image synthesis using generative adversarial networks. In: International Workshop on Simulation and Synthesis in Medical Imaging. Springer, Cham, pp. 22–31.
[24]
Wang, T., Lei, Y., Curran, W.J., Liu, T., Yang, X. (2021). Contrast-enhanced MRI synthesis from non-contrast MRI using attention CycleGAN. In: Proceedings od SPIE 11600, Medical Imaging 2021: Biomedical Applications in Molecular, Structural, and Functional Imaging, pp. 388–393.
[25]
Wei, X., Gong, B., Liu, Z., Lu, W., Wang, L. (2018). Improving the improved training of wasserstein gans: a consistency term and its dual effect. arXiv preprint arXiv:1803.01541.
[26]
Xiang, L., Li, Y., Lin, W., Wang, Q., Shen, D. (2018). Unpaired deep cross-modality synthesis with fast training. In: Stoyanov, D. et al. (Eds.), Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, DLMIA ML-CDS 2018. Lecture Notes in Computer Science, Vol. 11045. Springer, Cham, pp. 155–164.
[27]
Zhang, J., Chi, Y., Lyu, J., Yang, W., Tian, Y. (2023). Dual arbitrary scale super-resolution for multi-contrast MRI. arXiv preprint arXiv:2307.02334.
[28]
Zhang, X., He, X., Guo, J., Ettehadi, N., Aw, N., Semanek, D., Posner, J., Laine, A., Wang, Y. (2022). PTNet3d: a 3D high-resolution longitudinal infant brain MRI synthesizer based on transformers. IEEE Transactions on Medical Imaging, 41(10), 2925–2940.
[29]
Zhu, J.-Y., Park, T., Isola, P., Efros, A.A. (2017). Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2223–2232.
Recommendations
Training data independent image registration using generative adversarial networks and domain adaptation
Highlights- A dataset independent deep learning based approach for image registration is described.
AbstractMedical image registration is an important task in automated analysis of multimodal images and temporal data involving multiple patient visits. Conventional approaches, although useful for different image types, are time consuming. Of ...
Review of MRI-based Brain Tumor Image Segmentation Using Deep Learning Methods
Brain tumor segmentation is an important task in medical image processing. Early diagnosis of brain tumors plays an important role in improving treatment possibilities and increases the survival rate of the patients. Manual segmentation of the brain ...
Medical Image Synthesis for Data Augmentation and Anonymization Using Generative Adversarial Networks
Simulation and Synthesis in Medical ImagingAbstractData diversity is critical to success when training deep learning models. Medical imaging data sets are often imbalanced as pathologic findings are generally rare, which introduces significant challenges when training deep learning models. In this ...
Comments
Information & Contributors
Information
Published In
© 2024 Vilnius University.
Open access article under the CC BY license.
Publisher
IOS Press
Netherlands
Publication History
Published: 29 April 2024
Author Tags
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 09 Nov 2024
Other Metrics
Citations
View Options
View options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in