Abstract
To obtain high-quality positron emission tomography (PET) images while minimizing radiation exposure, various methods have been proposed for reconstructing standard-dose PET (SPET) images from low-dose PET (LPET) sinograms directly. However, current methods often neglect boundaries during sinogram-to-image reconstruction, resulting in high-frequency distortion in the frequency domain and diminished or fuzzy edges in the reconstructed images. Furthermore, the convolutional architectures, which are commonly used, lack the ability to model long-range non-local interactions, potentially leading to inaccurate representations of global structures. To alleviate these problems, in this paper, we propose a transformer-based model that unites triple domains of sinogram, image, and frequency for direct PET reconstruction, namely TriDo-Former. Specifically, the TriDo-Former consists of two cascaded networks, i.e., a sinogram enhancement transformer (SE-Former) for denoising the input LPET sinograms and a spatial-spectral reconstruction transformer (SSR-Former) for reconstructing SPET images from the denoised sinograms. Different from the vanilla transformer that splits an image into 2D patches, based specifically on the PET imaging mechanism, our SE-Former divides the sinogram into 1D projection view angles to maintain its inner-structure while denoising, preventing the noise in the sinogram from prorogating into the image domain. Moreover, to mitigate high-frequency distortion and improve reconstruction details, we integrate global frequency parsers (GFPs) into SSR-Former. The GFP serves as a learnable frequency filter that globally adjusts the frequency components in the frequency domain, enforcing the network to restore high-frequency details resembling real SPET images. Validations on a clinical dataset demonstrate that our TriDo-Former outperforms the state-of-the-art methods qualitatively and quantitatively.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Chen, W.: Clinical applications of PET in brain tumors. J. Nucl. Med. 48(9), 1468–1481 (2007)
Wang, Y., Ma, G., An, L., et al.: Semi-supervised tripled dictionary learning for standard-dose PET image prediction using low-dose PET and multimodal MRI. IEEE Trans. Biomed. Eng. 64(3), 569–579 (2016)
Zhou, T., Fu, H., Chen, G., et al.: Hi-net: hybrid-fusion network for multi-modal MR image synthesis. IEEE Trans. Med. Imaging 39(9), 2772–2781 (2020)
Li, Y., Zhou, T., He, K., et al.: Multi-scale transformer network with edge-aware pre-training for cross-modality MR image synthesis. IEEE Trans. Med. Imaging (2023)
Wang, K., et al.: Tripled-uncertainty guided mean teacher model for semi-supervised medical image segmentation. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12902, pp. 450–460. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87196-3_42
Zhan, B., Xiao, J., Cao, C., et al.: Multi-constraint generative adversarial network for dose prediction in radiotherapy. Med. Image Anal. 77, 102339 (2022)
Wang, Y., Zhang, P., Ma, g., et al: Predicting standard-dose PET image from low- dose PET and multimodal MR images using mapping-based sparse representation. Phys. Med. Biol. 61(2), 791–812 (2016)
Spuhler, K., Serrano-Sosa, M., Cattell, R., et al.: Full-count PET recovery from low-count image using a dilated convolutional neural network. Med. Phys. 47(10), 4928–4938 (2020)
Wang, Y., Yu, B., Wang, L., et al.: 3D conditional generative adversarial networks for high-quality PET image estimation at low dose. Neuroimage 174, 550–562 (2018)
Wang, Y., Zhou, L., Yu, B., et al.: 3D auto-context-based locality adaptive multi-modality GANs for PET synthesis. IEEE Trans. Med. Imaging 38(6), 1328–1339 (2018)
Wang, Y., Zhou, L., Wang, L., et al.: Locality adaptive multi-modality GANs for high-quality PET image synthesis. In: Frangi, A., et al. (eds.) MICCAI 2018, vol. 11070, pp. 329–337. Springer, Cham (2018)
Luo, Y., Wang, Y., Zu, C., et al.: 3D Transformer-GAN for high-quality PET reconstruction. In: de Bruijne, M., et al. (eds.) MICCAI 2021, vol. 12906, pp. 276–285. Springer, Cham (2021)
Luo, Y., Zhou, L., Zhan, B., et al.: Adaptive rectification based adversarial network with spectrum constraint for high-quality PET image synthesis. Med. Image Anal. 77, 102335 (2022)
Fei, Y., Zu, C., Jiao, Z., et al.: Classification-aided high-quality PET image synthesis via bidirectional contrastive GAN with shared information maximization. In: Wang, L., et al. (eds.) MICCAI 2022, vol. 13436, pp. 527–537. Springer, Cham (2022)
Zeng, P., Zhou, L., Zu, C., et al.: 3D CVT-GAN: a 3D convolutional vision transformer-GAN for PET reconstruction. In: Wang, L., et al. (eds.) MICCAI 2022, vol. 13436, pp. 516–526. Springer, Cham (2022)
Jiang, C., Pan, Y., Cui, Z., et al: Reconstruction of standard-dose PET from low-dose PET via dual-frequency supervision and global aggregation module. In: Proceedings of the19th International Symposium on Biomedical Imaging Conference, pp. 1–5 (2022)
Cui, J., Jiao, Z., Wei, Z., et al.: CT-only radiotherapy: an exploratory study for automatic dose prediction on rectal cancer patients via deep adversarial network. Front. Oncol. 12, 875661 (2022)
Li, H., Peng, X., Zeng, J., et al.: Explainable attention guided adversarial deep network for 3D radiotherapy dose distribution prediction. Knowl. Based Syst. 241, 108324 (2022)
Häggström, I., Schmidtlein, C.R., et al.: DeepPET: A deep encoder-decoder network for directly solving the PET image reconstruction inverse problem. Med. Image Anal. 54, 253–262 (2019)
Wang, B., Liu, H.: FBP-Net for direct reconstruction of dynamic PET images. Phys. Med. Biol. 65(23), 235008 (2020)
Ma, R., Hu, J., Sari, H., et al.: An encoder-decoder network for direct image reconstruction on sinograms of a long axial field of view PET. Eur. J. Nucl. Med. Mol. Imaging 49(13), 4464–4477 (2022)
Whiteley, W., Luk, W.K., et al.: DirectPET: full-size neural network PET reconstruction from sinogram data. J. Med. Imaging 7(3), 32503 (2020)
Liu, Z., Ye, H., and Liu, H: Deep-learning-based framework for PET image reconstruction from sinogram domain. Appl. Sci. 12(16), 8118 (2022)
Xue, H., Zhang, Q., Zou, S., et al.: LCPR-Net: low-count PET image reconstruction using the domain transform and cycle-consistent generative adversarial networks. Quant. Imaging Med. Surg. 11(2), 749 (2021)
Feng, Q., Liu, H.: Rethinking PET image reconstruction: ultra-low-dose, sinogram and deep learning. In: Martel, A.L., et al. (eds.) MICCAI 2020, vol. 12267, pp. 783–792. Springer, Cham (2020)
Liu, Z., Chen, H., Liu, H.: Deep learning based framework for direct reconstruction of PET images. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11766, pp. 48–56. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32248-9_6
Hu, R., Liu, H: TransEM: Residual swin-transformer based regularized PET image reconstruction. In: Wang, L., et al (eds.) MICCAI 2022, vol. 13434, pp. 184–193. Springer, Cham (2022)
Dosovitskiy, A., Beyer, L., Kolesnikov, A., et al.: An image is worth 16 × 16 words: transformers for image recognition at scale. In: Proceedings of the IEEE/CVF International Con-ference on Computer Vision. IEEE, Venice (2020)
Zhang, Z., Yu, L., Liang, X., et al.: TransCT: dual-path transformer for low dose computed tomography. In: de Bruijne, M., et al. (eds.) MICCAI 2021, vol. 12906, pp. 55–64. Springer, Cham (2021)
Zheng, H., Lin, Z., Zhou, Q., et al.: Multi-transSP: Multimodal transformer for survival prediction of nasopharyngeal carcinoma patients. In: Wang, L., et al. (eds.) MICCAI 2022, vol. 13437, pp. 234–243. Springer, Cham (2022)
Liu, Z., Lin, Y., Cao, Y., et al: Swin transformer: hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10012–10022. IEEE, Montreal (2021)
Hudson, H., Larkin, R.: Accelerated image reconstruction using ordered subsets of projection data. IEEE Trans. Med. Imaging 13, 601–609 (1994)
Zhang, K., Zuo, W., Chen, Y., Meng, D., Zhang, L.: Beyond a gaussian denoiser: residual learning of deep CNN for image denoising. IEEE Trans. Image Process. 26(7), 3142-3155. (2017)
Acknowledgement
This work is supported by the National Natural Science Foundation of China (NSFC 62071314), Sichuan Science and Technology Program 2023YFG0263, 2023NSFSC0497, 22YYJCYJ0086, and Opening Foundation of Agile and Intelligent Computing Key Laboratory of Sichuan Province.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Cui, J. et al. (2023). TriDo-Former: A Triple-Domain Transformer for Direct PET Reconstruction from Low-Dose Sinograms. In: Greenspan, H., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2023. MICCAI 2023. Lecture Notes in Computer Science, vol 14229. Springer, Cham. https://doi.org/10.1007/978-3-031-43999-5_18
Download citation
DOI: https://doi.org/10.1007/978-3-031-43999-5_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-43998-8
Online ISBN: 978-3-031-43999-5
eBook Packages: Computer ScienceComputer Science (R0)
