Abstract
The interpretation of prostate MRI suffers from low agreement across radiologists due to the subtle differences between cancer and normal tissue. Image registration addresses this issue by accurately mapping the ground-truth cancer labels from surgical histopathology images onto MRI. Cancer labels achieved by image registration can be used to improve radiologists’ interpretation of MRI by training deep learning models for early detection of prostate cancer. A major limitation of current automated registration approaches is that they require manual prostate segmentations, which is a time-consuming task, prone to errors. This paper presents a weakly supervised approach for affine and deformable registration of MRI and histopathology images without requiring prostate segmentations. We used manual prostate segmentations and mono-modal synthetic image pairs to train our registration networks to align prostate boundaries and local prostate features. Although prostate segmentations were used during the training of the network, such segmentations were not needed when registering unseen images at inference time. We trained and validated our registration network with 135 and 10 patients from an internal cohort, respectively. We tested the performance of our method using 16 patients from the internal cohort and 22 patients from an external cohort. The results show that our weakly supervised method has achieved significantly higher registration accuracy than a state-of-the-art method run without prostate segmentations. Our deep learning framework will ease the registration of MRI and histopathology images by obviating the need for prostate segmentations.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
American Cancer Society. Facts & Figures 2021. American Cancer Society, Atlanta, GA (2021)
Baris Turkbey, L., Peter Choyke, L.: Multiparametric MRI and prostate cancer diagnosis and risk stratification. Curr. Opin. Urol. 22(4), 310–315 (2012)
Ahmed, H.U., et al.: Diagnostic accuracy of multi-parametric MRI and TRUS biopsy in prostate cancer (PROMIS): a paired validating confirmatory study. The Lancet 389(10071), 815–822 (2017)
Lovegrove, C.E., et al.: The role of pathology correlation approach in prostate cancer index lesion detection and quantitative analysis with multiparametric MRI. In: NIH (2016)
Bhattacharya, I., et al.: CorrSigNet: learning CORRelated prostate cancer SIGnatures from radiology and pathology images for improved computer aided diagnosis. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12262, pp. 315–325. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59713-9_31
Seetharaman, A., et al.: Automated detection of aggressive and indolent prostate cancer on magnetic resonance imaging. Med. Phys. (2021)
Saha, A., Hosseinzadeh, M., Huisman, H.: End-to-end prostate cancer detection in bpMRI via 3D CNNs: effects of attention mechanisms, clinical priori and decoupled false positive reduction. Med. Image Anal. 102155 (2021)
Chappelow, J., et al.: Elastic registration of multimodal prostate MRI and histology via multiattribute combined mutual information. Med. Phys. 38(4), 2005–2018 (2011)
Reynolds, H.M., Williams, S., Zhang, A., Chakravorty, R., Rawlinson, D., Ong, C.S., et al.: Development of a registration framework to validate MRI with histology for prostate focal therapy. Med. Phys. 42(12), 7078–7089 (2015)
Wu, H.H., et al.: A system using patient-specific 3D-printed molds to spatially align in vivo MRI with ex vivo MRI and whole-mount histopathology for prostate cancer research. J. Magnet. Reson. Imaging 49(1) (2019)
Rusu, M., et al.: Registration of presurgical MRI and histopathology images from radical prostatectomy via RAPSODI. Med. Phys. 47(9), 4177–4188 (2020)
Shao, W., et al.: Prosregnet: a deep learning framework for registration of MRI and histopathology images of the prostate. Med. Image Anal. 68, 101919 (2021)
Sood, R.R., et al.: 3D registration of pre-surgical prostate MRI and histopathology images via super-resolution volume reconstruction. Med. Image Anal. 69, 101957 (2021)
Hu, Y., et al.: Label-driven weakly-supervised learning for multimodal deformable image registration. In: 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), pp. 1070–1074. IEEE (2018)
Balakrishnan, G., et al.: Voxelmorph: a learning framework for deformable medical image registration. IEEE Trans. Med. Imaging 38(8), 1788–1800 (2019)
de Vos, B.D., Berendsen, F.F., Viergever, M.A., Sokooti, H., Staring, M., Igum, I.: A deep learning framework for unsupervised affine and deformable image registration. Med. Image Anal. 52, 128–143 (2019)
Choyke, P., Turkbey, B., Pinto, P., Merino, M., Wood, B.: Data from PROSTATE-MRI (2016)
Nyúl, L.G., Udupa, J.K., Zhang, X.: New variants of a method of MRI scale standardization. IEEE Trans. Med. Imaging 19(2), 143–150 (2000)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv.org (2015)
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255. IEEE (2009)
Donato, G., Belongie, S.: Approximate thin plate spline mappings. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2352, pp. 21–31. Springer, Heidelberg (2002). https://doi.org/10.1007/3-540-47977-5_2
Kingma, D., Ba, J.: Adam: a method for stochastic optimization. arXiv.org (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Shao, W. et al. (2021). Weakly Supervised Registration of Prostate MRI and Histopathology Images. In: de Bruijne, M., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2021. MICCAI 2021. Lecture Notes in Computer Science(), vol 12904. Springer, Cham. https://doi.org/10.1007/978-3-030-87202-1_10
Download citation
DOI: https://doi.org/10.1007/978-3-030-87202-1_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87201-4
Online ISBN: 978-3-030-87202-1
eBook Packages: Computer ScienceComputer Science (R0)
