Abstract
Deep learning-based interactive segmentation has attracted research interest recently since it can smartly utilize user interactions to refine a coarse automatic segmentation to get higher accuracy for clinical use. Current methods usually transform user clicks to geodesic distance hint maps as guidance, then concatenate them with the raw image and coarse segmentation, and feed them into a refinement network. Such methods are insufficient in refining error region, which is a key capability required for interactive segmentation. In this paper, we propose Error Attention Interactive network with Matting and Fusion to auto-extract guide information of mis-segmentation region from two branches and transfer it into main segmentor. We first design Region Matting to obtain foreground and background mattings from coarse segmentation. And then we adopt the features extracted by two branches trained on above mattings as guidance. Attention-Fusion is further proposed to transfer the guidance to main segmentor effectively based on attention mechanism and feature concatenation. Experimental results on BraTS 2015 and our Neuroblastoma datasets have shown that our method significantly outperforms state-of-the-art methods, with the advantage of fewer interactions.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Xu, N., Price, B., Cohen, S., Yang, J., Huang, T.S.: Deep interactive object selection. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 373–381. IEEE (2016)
Wang, G., Zuluaga, M.A., Li, W., et al.: DeepIGeoS: a deep interactive geodesic framework for medical image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 41(7), 1559–1572 (2018)
Lei, W., Wang, H., Gu, R., Zhang, S., Zhang, S., Wang, G.: DeepIGeoS-V2: deep interactive segmentation of multiple organs from head and neck images with lightweight CNNs. In: Zhou, L., et al. (eds.) LABELS/HAL-MICCAI/CuRIOUS -2019. LNCS, vol. 11851, pp. 61–69. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-33642-4_7
Zhao, F., Xie, X.: An overview of interactive medical image segmentation. In: Annals of the BMVA, pp: 1–22 (2013)
Haider, S. A., et al.: Single-click, semi-automatic lung nodule contouring using hierarchical conditional random fields. In: ISBI, (2015)
Xu, C., Prince, J.L.: Snakes, shapes, and gradient vector flow. TIP 7(3), 359–369 (1998)
Boykov, Y., Jolly, M.P.: Interactive graph cuts for optimal boundary region segmentation of objects in N-D images. In: ICCV, pp: 105–112 (2001)
Rother, C., Kolmogorov, V., Blake, A.: “GrabCut”: interactive foreground extraction using iterated graph cuts. ACM Trans. Graph. 23(3), 309–314 (2004)
Criminisi, A., Sharp, T., Blake, A.: GeoS: geodesic image segmentation. In: ECCV, pp: 99–112 (2008)
Grady, L.: Random walks for image segmentation. PAMI 28(11), 1768–1783 (2006)
Menze, B.H., Jakab, A., Bauer, S., et al.: The multimodal brain tumor image segmentation benchmark (BRATS). IEEE Trans. Med. Imaging 34(10), 1993–2024 (2014)
Zhou, B., Chen, L., Wang, Z.: Interactive deep editing framework for medical image segmentation. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11766, pp. 329–337. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32248-9_37
Price, B.L., Morse, B., Cohen, S.: Geodesic graph cut for interactive image segmentation. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, CA, pp. 3161–3168. IEEE (2010)
Xie, Y., Lu, H., Zhang, J., Shen, C., Xia, Y.: Deep segmentation-emendation model for gland instance segmentation. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11764, pp. 469–477. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32239-7_52
Maaten, L.V.D., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9(Nov), pp. 2579–2605 (2008)
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Hu, W. et al. (2020). Error Attention Interactive Segmentation of Medical Image Through Matting and Fusion. In: Liu, M., Yan, P., Lian, C., Cao, X. (eds) Machine Learning in Medical Imaging. MLMI 2020. Lecture Notes in Computer Science(), vol 12436. Springer, Cham. https://doi.org/10.1007/978-3-030-59861-7_2
Download citation
DOI: https://doi.org/10.1007/978-3-030-59861-7_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-59860-0
Online ISBN: 978-3-030-59861-7
eBook Packages: Computer ScienceComputer Science (R0)