Abstract
We propose Orthogonal-Nets consisting of a large number of ensembles of 2D encoder-decoder convolutional neural networks. The Orthogonal-Nets takes 2D slices of the image from axial, sagittal, and coronal views of the 3D brain volume and predicts the probability for the tumor segmentation region. The predicted probability distributions from all three views are averaged to generate a 3D probability distribution map that is subsequently used to predict the tumor regions for the 3D images. In this work, we propose a two-stage Orthogonal-Nets. Stage-I predicts the brain tumor labels for the whole 3D image using the axial, sagittal, and coronal views. The labels from the first stage are then used to crop only the tumor region. Multiple Orthogonal-Nets were then trained in stage-II, which takes only the cropped region as input. The two-stage strategy substantially reduces the computational burden on the stage-II networks and thus many Orthogonal-Nets can be used in stage-II. We used one Orthogonal-Net for stage-I and 28 Orthogonal-Nets for stage-II. The mean dice score on the testing datasets was 0.8660, 0.8776, 0.9118 for enhancing tumor, core tumor, and whole tumor respectively.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Menze, B.H., et al.: The multimodal brain tumor image segmentation benchmark (BRATS). IEEE Trans. Med. Imaging (2015)
Bakas, S., et al.: Advancing the cancer genome atlas glioma MRI collections with expert segmentation labels and radiomic features. Sci. Data 4, 170117 (2017)
Bakas, S., et al.: Identifying the best machine learning algorithms for brain tumor segmentation, progression assessment, and overall survival prediction in the BRATS challenge. arXiv preprint arXiv:1811.02629 (2018)
Bakas, S., et al.: Segmentation labels and radiomic features for the pre-operative scans of the TCGA-LGG collection. The Cancer Imaging Archive 286 (2017)
Baid, U., et al.: The RSNA-ASNR-MICCAI BraTS 2021 benchmark on brain tumor segmentation and radiogenomic classification. arXiv preprint arXiv:2107.02314 (2021)
Bakas, S., et al.: Segmentation labels and radiomic features for the pre-operative scans of the TCGA-GBM collection. The Cancer Imaging Archive (2017)
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255. IEEE (2009)
Ballestar, L.M., Vilaplana, V.: MRI brain tumor segmentation and uncertainty estimation using 3D-UNet architectures. arXiv preprint arXiv:2012.15294 (2020)
Hu, J., Wang, H., Wang, J., Wang, Y., He, F., Zhang, J.: SA-Net: a scale-attention network for medical image segmentation. PLoS ONE 16, e0247388 (2021)
Wang, Y., et al.: Modality-pairing learning for brain tumor segmentation. arXiv preprint arXiv:2010.09277 (2020)
Jia, H., Cai, W., Huang, H., Xia, Y.: H2NF-Net for Brain Tumor Segmentation using Multimodal MR Imaging: 2nd Place Solution to BraTS Challenge 2020 Segmentation Task (2020)
Isensee, F., Jäger, P.F., Full, P.M., Vollmuth, P., Maier-Hein, K.H.: Nnu-net for brain tumor segmentation. In: Crimi, A., Bakas, Spyridon (eds.) Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: 6th International Workshop, BrainLes 2020, Held in Conjunction with MICCAI 2020, Lima, Peru, October 4, 2020, Revised Selected Papers, Part II, pp. 118–132. Springer International Publishing, Cham (2021). https://doi.org/10.1007/978-3-030-72087-2_11
Li, S., Sui, X., Luo, X., Xu, X., Liu, Y., Goh, R.S.M.: Medical image segmentation using squeeze-and-expansion transformers. arXiv preprint arXiv:2105.09511 (2021)
Zhao, Y.-X., Zhang, Y.-M., Liu, C.-L.: Bag of tricks for 3D MRI brain tumor segmentation. In: Crimi, A., Bakas, S. (eds.) Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: 5th International Workshop, BrainLes 2019, Held in Conjunction with MICCAI 2019, Shenzhen, China, October 17, 2019, Revised Selected Papers, Part I, pp. 210–220. Springer International Publishing, Cham (2020). https://doi.org/10.1007/978-3-030-46640-4_20
Kamnitsas, K., et al.: Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Med. Image Anal. 36, 61–78 (2017)
Pawar, K., Chen, N.Z., Shah, J., Egan, G.F.: An ensemble of 2D convolutional neural network for 3D brain tumor segmentation. In: Crimi, A., Bakas, S. (eds.) Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: 5th International Workshop, BrainLes 2019, Held in Conjunction with MICCAI 2019, Shenzhen, China, October 17, 2019, Revised Selected Papers, Part I, pp. 359–367. Springer International Publishing, Cham (2020). https://doi.org/10.1007/978-3-030-46640-4_34
Pawar, K., Chen, Z., Shah, N.J., Egan, G.: Residual encoder and convolutional decoder neural network for glioma segmentation. In: Crimi, A., Bakas, S., Kuijf, H., Menze, B., Reyes, M. (eds.) Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries BrainLes 2017 Third International Workshop, BrainLes 2017, Held in Conjunction with MICCAI 2017, Quebec City, QC, Canada, September 14, 2017, Revised Selected Papers. LNCS, vol. 10670, pp. 263–273. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-75238-9_23
McKinley, R., Meier, R., Wiest, R.: Ensembles of densely-connected CNNs with label-uncertainty for brain tumor segmentation. In: Crimi, A., Bakas, S., Kuijf, H., Keyvan, F., Reyes, M., van Walsum, T. (eds.) Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: 4th International Workshop, BrainLes 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 16, 2018, Revised Selected Papers, Part II, pp. 456–465. Springer International Publishing, Cham (2019). https://doi.org/10.1007/978-3-030-11726-9_40
Isensee, F., Jaeger, P.F., Kohl, S.A., Petersen, J., Maier-Hein, K.H.: nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nat. Methods 18, 203–211 (2021)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Zhou, Z., Siddiquee, M.M.R., Tajbakhsh, N., Liang, J.: Unet++: a nested u-net architecture for medical image segmentation. Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, pp. 3–11. Springer (2018)
Li, R., et al.: Multiattention network for semantic segmentation of fine-resolution remote sensing images. IEEE Transactions on Geoscience and Remote Sensing (2021)
Zhao, H., Shi, J., Qi, X., Wang, X., Jia, J.: Pyramid scene parsing network. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2881–2890 (2017)
Khosravan, N., Mortazi, A., Wallace, M., Bagci, U.: Pan: projective adversarial network for medical image segmentation. In: Shen, D., Liu, T., Peters, T.M., Staib, L.H., Essert, C., Zhou, S., Yap, P.-T., Khan, A. (eds.) Medical Image Computing and Computer Assisted Intervention – MICCAI 2019: 22nd International Conference, Shenzhen, China, October 13–17, 2019, Proceedings, Part VI, pp. 68–76. Springer International Publishing, Cham (2019). https://doi.org/10.1007/978-3-030-32226-7_8
Lin, T.-Y., Dollár, P., Girshick, R., He, K., Hariharan, B., Belongie, S.: Feature pyramid networks for object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2117–2125 (2017)
Chaurasia, A., Culurciello, E.: Linknet: Exploiting encoder representations for efficient semantic segmentation. In: 2017 IEEE Visual Communications and Image Processing (VCIP), pp. 1–4. IEEE (2017)
He, K.M., Zhang, X.Y., Ren, S.Q., Sun, J.: Deep residual learning for image recognition. Proc. CVPR IEEE, 770–778 (2016)
Szegedy, C., Ioffe, S., Vanhoucke, V., Alemi, A.A.: Inception-v4, inception-resnet and the impact of residual connections on learning. In: Thirty-First AAAI Conference on Artificial Intelligence (2017)
Tan, M., Le, Q.: Efficientnet: rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114. PMLR (2019)
Henry, T., et al.: Brain tumor segmentation with self-ensembled, deeply-supervised 3D U-net neural networks: a BraTS 2020 challenge solution. arXiv preprint arXiv:2011.01045 (2020)
Pawar, K., Chen, Z., Shah, N.J., Egan, G.F.: Suppressing motion artefacts in MRI using an Inception‐ResNet network with motion simulation augmentation. NMR Biomed. e4225 (2019)
Pawar, K., Egan, G.F., Chen, Z.: Domain knowledge augmentation of parallel MR image reconstruction using deep learning. Comput. Med. Imaging Graph. 101968 (2021)
Pawar, K., Chen, Z., Shah, N.J., Egan, G.F.: A Deep learning framework for transforming image reconstruction into pixel classification. IEEE Access 7, 177690–177702 (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Pawar, K., Zhong, S., Goonatillake, D.S., Egan, G., Chen, Z. (2022). Orthogonal-Nets: A Large Ensemble of 2D Neural Networks for 3D Brain Tumor Segmentation. In: Crimi, A., Bakas, S. (eds) Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. BrainLes 2021. Lecture Notes in Computer Science, vol 12963. Springer, Cham. https://doi.org/10.1007/978-3-031-09002-8_5
Download citation
DOI: https://doi.org/10.1007/978-3-031-09002-8_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-09001-1
Online ISBN: 978-3-031-09002-8
eBook Packages: Computer ScienceComputer Science (R0)
