Abstract
Isocitrate dehydrogenase (IDH) status is an important factor for the diagnosis of gliomas reported in the 2016 World Health Organization classification scheme for gliomas. There is a strong relationship between IDH mutation status and prognosis. The preoperative prediction of IDH status is necessary for appropriate treatment planning. However, existing methods cannot predict IDH status accurately before the operation. In this paper, we propose a radiomics associated modality attention network to predict IDH mutation status on multi-modality MRI images. Our method first predicts the importance of each modality for the classification task and calculates weights, then uses weighted images for prediction. We also present a light-weight and high-performance self-attention network for gliomas tumor classification to solve the overfitting problem. Additionally, we associate radiomics features for computation of modality attention and classification to enhance the classification accuracy. Our method achieves a 0.7246 F1 Score on our private dataset provided by the First Affiliated Hospital of Zhengzhou University (FHZU), which is better than state-of-the-art methods.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Ostrom, Q.T., Gittleman, H., Xu, J., et al.: CBTRUS statistical report: primary brain and other central nervous system tumors diagnosed in the united states in 2009–2013. Neuro Oncol 18(suppl_5), v1–v75 (2016)
De Vleeschouwer, S. (ed.): Glioblastoma [Internet]. Codon Publications, Brisbane (2017)
Louis, D.N., Perry, A., Reifenberger, G., et al.: The 2016 World Health Organization classification of tumors of the central nervous system: a summary. Acta Neuropathol. 131(6), 803–820 (2016)
Yan, H., Parsons, D.W., Jin, G., et al.: IDH1 and IDH2 mutations in gliomas. N. Engl. J. Med. 360(8), 765–773 (2009)
Broen, M.P.G., Smits, M., Wijnenga, M.M.J., et al.: The T2-FLAIR mismatch sign as an imaging marker for non-enhancing IDH-mutant, 1p/19q-intact lower-grade glioma: a validation study. Neuro Oncol. 20(10), 1393–1399 (2018)
Choi, Y.S., et al. Fully automated hybrid approach to predict the IDH mutation status of gliomas via deep learning and radiomics. Neuro-oncology (2020).
Liu, S., Shah, Z., Sav, A., Russo, C., Berkovsky, S., Qian, Y., Coiera, E., Ieva, A.: Isocitrate dehydrogenase (IDH) status prediction in histopathology images of gliomas using deep learning. Sci. Rep. 10, 7733 (2020)
Choi, K.S., Hong Choi, S., Jeong, B.: Prediction of IDH genotype in gliomas with dynamic susceptibility contrast perfusion MR imaging using an explainable recurrent neural network. Neuro Oncol. 21(9), 1197–1209 (2019)
Zhao, H., Jia, J., Koltun, V.: Exploring self-attention for image recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2020).
Jang, K., Russo, C., Di Ieva, A.: Radiomics in gliomas: clinical implications of computational modeling and fractal-based analysis. Neuroradiology 62(7), 771–790 (2020)
Zhang, X., et al.: IDH mutation status prediction by modality-self attention network. In: Chen, Y.W., Tanaka, S., Howlett, R.J., Jain, L.C. (eds.) Innovation in Medicine and Healthcare. Smart Innovation, Systems and Technologies, vol. 242, pp.51–57. Springer, Singapore (2021)
Çiçek, Ö., et al.: 3D U-Net: learning dense volumetric segmentation from sparse annotation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, Cham (2016)
He, K., et al. Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016)
Li, C., Sun, H., Liu, Z., et al.: Learning Cross-Modal Deep Representations for Multi-Modal MR Image Segmentation. Springer, Cham (2019)
Zhang, G., Shen, X., Luo, Y., et al.: Cross-Modal Self-Attention Distillation for Prostate Cancer Segmentation (2020)
Kullback, S., Leibler, R.A.: On information and sufficiency. Ann. Math. Stat. 22(1), 79–86 (1951)
Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2018)
Tibshirani, R.: Regression shrinkage and selection via the lasso. J. R. Stat. Soc. Ser. B (Methodol.) 58(1), 267–288 (1996)
van Griethuysen, J.J.M., Fedorov, A., Parmar, C., Hosny, A., Aucoin, N., Narayan, V., Beets-Tan, R.G.H., Fillon-Robin, J.C., Pieper, S., Aerts, H.J.W.L.: Computational radiomics system to decode the radiographic phenotype. Can. Res. 77(21), e104–e107 (2017). https://doi.org/10.1158/0008-5472.CAN-17-0339
Lowekamp, B.C., Chen, D.T., Ibáñez, L., Blezek, D.: The Design of SimpleITK. Front. Neuroinform. 7, 45 (2013). https://doi.org/10.3389/fninf.2013.00045
Yaniv, Z., Lowekamp, B.C., Johnson, H.J., Beare, R.: SimpleITK image-analysis notebooks: a collaborative environment for education and reproducible research. J. Digit Imaging (2017). https://doi.org/10.1007/s10278-017-0037-8
Acknowledgements
This work is supported in part by the Grant-in-Aid for Scientific Research from the Japanese Ministry for Education, Science, Culture and Sports (MEXT) under the Grant No. 20KK0234, No. 20K21821. Authors would like to thank Ms. Swathi ANANDA and Rahul JAIN of Ritsumeikan University for their English proof and corrections. Funding was provided by the Zhejiang Lab Program (No. 2020ND8AD01).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Zhang, X., Shi, X., Iwamoto, Y. et al. IDH mutation status prediction by a radiomics associated modality attention network. Vis Comput 39, 2367–2379 (2023). https://doi.org/10.1007/s00371-022-02452-y
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00371-022-02452-y