Abstract
Hepatocellular carcinoma (HCC) is the most common primary liver cancer which accounts for a high mortality rate in clinical, and the most effective treatment for HCC is surgical resection. However, patients with HCC are still at a huge risk of recurrence after tumor resection. In this light, preoperative early recurrence prediction methods are necessary to guide physicians to develop an individualized preoperative treatment plan and postoperative follow-up, thus prolonging the survival time of patients. Nevertheless, existing methods based on clinical data neglect information on the image modality; existing methods based on radiomics are limited by the ability of its predefined features compared with deep learning methods; and existing methods based on CT scans are constrained by the inability to capture the details of images compared with MRI. With these observations, we propose a deep learning transformer-based model on multi-modality MRI to tackle the preoperative early recurrence prediction task of HCC. Enlightened by the vigorous capacity of context modeling of the transformer architecture, our proposed model exploits it to dig out the inter-modality correlations, and the performance significantly improves. Our experimental results reveal that our transformer-based model can achieve better performance than other state-of-the-art existing methods.
G. Zhan and F. Wang—First authors.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Elsayes, K.M., Kielar, A.Z., Agrons, M.M.: Liver imaging reporting and data system: an expert consensus statement. J. Hepatocel. Carcinoma 4, 29–39 (2017)
Zhu, R.X., Seto, W.K., Lai, C.L.: Epidemiology of hepatocellular carcinoma in the Asia-Pacific region. Gut Liver 10, 332–339 (2016)
Thomas, M.B., Zhu, A.X.: Hepatocellular carcinoma: the need for progress. J. Clin. Oncol. 23, 2892–2899 (2005)
association, E.: EASL clinical practice guidelines: management of hepatocellular carcinoma. J. Hepatol. 69, 182–236 (2018)
Marrero, J.A., et al.: Diagnosis, staging and management of hepatocellular carcinoma: 2018 practice guidance by the American Association for the study of liver diseases. Hepatology 68, 723–750 (2018)
Bray, F., Ferlay, J., Soerjomataram, I., Siegel, R.L., Torre, L.A., Jemal, A.: Global cancer statistics 2018: Globocan estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J. Clin. 68, 394–424 (2018)
Cheng, Z., Yang, P., Qu, S.: Risk factors and management for early and late intrahepatic recurrence of solitary hepatocellular carcinoma after curative resection. HPB 17, 422–427 (2015)
Liu1, C., Yang, H., Feng, Y.: A k-nearest neighbor model to predict early recurrence of hepatocellular carcinoma after resection. J. Clin. Translat. Hepatol. 10, 600–607 (2022)
Gillies, R., Kinahan, P.E., Hricak, H.: Radiomics: images are more than pictures, they are data. radiology. Radiology 278, 563–577 (2015)
Zhao, Y., Wu, J., Zhang, Q.: Radiomics analysis based on multiparametric MRI for predicting early recurrence in hepatocellular carcinoma after partial hepatectomy. J. Magn. Reson. Imaging 53, 1066–1079 (2021)
Litjens, G., Kooi, T., Bejnordi, B.E.: A survey on deep learning in medical image analysis. Med. Image Anal. 42, 60–88 (2017)
Krizhevsky, A., Sutskever, I., HintonImagenet, G.E.: ImageNet classification with deep convolutional neural networks. Communications ACM 60, 84–90 (2017)
Kolesnikov, A., Beyer, L., Zhai, X., Puigcerver, J., Yung, J., Gelly, S., Houlsby, N.: Big Transfer (BiT): general visual representation learning. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12350, pp. 491–507. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58558-7_29
Wang, W., et al.: Phase attention model for prediction of early recurrence of hepatocellular carcinoma with multi-phase CT images and clinical data. Front. Radiol. 8 (2022)
Burre, M., et al.: MRI angiography is superior to helical CT for detection of HCC prior to liver transplantation: an explant correlation. Hepatology 38, 1034–1042 (2003)
Armbruster, M., et al.: Measuring HCC tumor size in MRI-the sequence matters! Diagnostics 11 (2002)
Vaswan, A., et al.: Attention is all you need. In: 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA (2017)
Lee, Y., et al.: Benign versus malignant soft-tissue tumors: differentiation with 3t magnetic resonance image textural analysis including diffusion-weighted imaging. Investig. Magn. Resonance Imaging 25, 118–128 (2021)
Chartampilas, E., Rafailidis, V., Georgopoulou, V., Kalarakis, G., Hatzidakis, A., Prassopoulos, P.: Current imaging diagnosis of hepatocellular carcinoma. Cancers 14, 3997 (2022)
He, K., Zhang, X., Ren, S.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Li, F.-F.: ImageNet: a large-scale hierarchical image database. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009)
Dosovitskiy, A., et al.: An image is worth 16x16 words: transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020)
Xing, H., Zhang, W.G., Cescon, M.: Defining and predicting early recurrence after liver resection of hepatocellular carcinoma: a multi-institutional study. HPB 22, 677–689 (2020)
Manjon, J.V.: MRI preprocessing. In: Imaging Biomarkers, pp. 53–63 (2017)
Wang, W., et al.: Deep learning-based radiomics models for early recurrence prediction of hepatocellular carcinoma with multi-phase CT images and clinical data. In: 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 4881–4884 (2019)
Acknowledgments
This work was supported in part by the Grant in Aid for Scientific Research from the Japanese Ministry for Education, Science, Culture and Sports (MEXT) under the Grant Nos. 20KK0234, 21H03470, and 20K21821, and in part by the Natural Science Foundation of Zhejiang Province (LZ22F020012), in part by Major Scientific Research Project of Zhejiang Lab (2020ND8AD01), and in part by the National Natural Science Foundation of China (82071988), the Key Research and Development Program of Zhejiang Province (2019C03064), the Program Co-sponsored by Province and Ministry (No. WKJ-ZJ-1926) and the Special Fund for Basic Scientific Research Business Expenses of Zhejiang University (No. 2021FZZX003-02-17).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Zhan, G. et al. (2023). A Transformer-Based Model for Preoperative Early Recurrence Prediction of Hepatocellular Carcinoma with Muti-modality MRI. In: Zheng, Y., Keleş, H.Y., Koniusz, P. (eds) Computer Vision – ACCV 2022 Workshops. ACCV 2022. Lecture Notes in Computer Science, vol 13848. Springer, Cham. https://doi.org/10.1007/978-3-031-27066-6_13
Download citation
DOI: https://doi.org/10.1007/978-3-031-27066-6_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-27065-9
Online ISBN: 978-3-031-27066-6
eBook Packages: Computer ScienceComputer Science (R0)