Abstract
Recently, weakly supervised nuclei segmentation methods using only points are gaining attention, as they can ease the tedious labeling process. However, most methods often fail to separate adjacent nuclei and are particularly sensitive to point annotations that deviate from the center of nuclei, resulting in lower accuracy. In this study, we propose a novel weakly supervised method to effectively distinguish adjacent nuclei and maintain robustness regardless of point label deviation. We detect and segment nuclei by combining a binary segmentation module, an offset regression module, and a center detection module to determine foreground pixels, delineate boundaries and identify instances. In training, we first generate pseudo binary masks using geodesic distance-based Voronoi diagrams and k-means clustering. Next, segmentation predictions are used to repeatedly generate pseudo offset maps that indicate the most likely nuclei center. Finally, an Expectation Maximization (EM) based process iteratively refines initial point labels based on the offset map predictions to fine-tune our framework. Experimental results show that our model consistently outperforms state-of-the-art methods on public datasets regardless of the point annotation accuracy.
S. Nam and J. Jeong—Equal contribution.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Alsubaie, N., Sirinukunwattana, K., Raza, S.E.A., Snead, D., Rajpoot, N.: A bottom-up approach for tumour differentiation in whole slide images of lung adenocarcinoma. In: Medical Imaging 2018: Digital Pathology, vol. 10581, pp. 104–113. SPIE (2018)
Cheng, B., et al.: Panoptic-DeepLab: a simple, strong, and fast baseline for bottom-up panoptic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12475–12485 (2020)
Criminisi, A., Sharp, T., Blake, A.: GeoS: geodesic image segmentation. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008. LNCS, vol. 5302, pp. 99–112. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-88682-2_9
Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm. J. Royal Stat. Soc.: Series B (Methodological) 39(1), 1–22 (1977)
Dong, M., et al.: Towards neuron segmentation from macaque brain images: a weakly supervised approach. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12265, pp. 194–203. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59722-1_19
Graham, S., et al.: Hover-Net: simultaneous segmentation and classification of nuclei in multi-tissue histology images. Med. Image Anal. 58, 101563 (2019)
Hartigan, J.A., Wong, M.A.: Algorithm as 136: a k-means clustering algorithm. J. Royal Stat. Soc. Ser. c (Applied Statistics) 28(1), 100–108 (1979)
He, H., et al.: CDNet: centripetal direction network for nuclear instance segmentation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4026–4035 (2021)
Kendall, A., Gal, Y., Cipolla, R.: Multi-task learning using uncertainty to weigh losses for scene geometry and semantics. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7482–7491 (2018)
Khoreva, A., Benenson, R., Hosang, J., Hein, M., Schiele, B.: Simple does it: weakly supervised instance and semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 876–885 (2017)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Kumar, N., et al.: A multi-organ nucleus segmentation challenge. IEEE Trans. Med. Imaging 39(5), 1380–1391 (2019)
Kumar, N., Verma, R., Sharma, S., Bhargava, S., Vahadane, A., Sethi, A.: A dataset and a technique for generalized nuclear segmentation for computational pathology. IEEE Trans. Med. Imaging 36(7), 1550–1560 (2017)
Law, H., Deng, J.: CornerNet: detecting objects as paired keypoints. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 734–750 (2018)
Lin, D., Dai, J., Jia, J., He, K., Sun, J.: ScribbleSup: scribble-supervised convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3159–3167 (2016)
Lin, T.Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017)
Liu, W., He, Q., He, X.: Weakly supervised nuclei segmentation via instance learning. In: 2022 IEEE 19th International Symposium on Biomedical Imaging (ISBI), pp. 1–5. IEEE (2022)
Lu, C., et al.: Nuclear shape and orientation features from h &e images predict survival in early-stage estrogen receptor-positive breast cancers. Lab. Investig. 98(11), 1438–1448 (2018)
Neven, D., Brabandere, B.D., Proesmans, M., Gool, L.V.: Instance segmentation by jointly optimizing spatial embeddings and clustering bandwidth. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8837–8845 (2019)
Nishimura, K., Ker, D.F.E., Bise, R.: Weakly supervised cell instance segmentation by propagating from detection response. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 649–657. Springer (2019)
Qu, H., et al.: Weakly supervised deep nuclei segmentation using points annotation in histopathology images. In: International Conference on Medical Imaging with Deep Learning, pp. 390–400. PMLR (2019)
Qu, H., Yi, J., Huang, Q., Wu, P., Metaxas, D.: Nuclei segmentation using mixed points and masks selected from uncertainty. In: 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI), pp. 973–976. IEEE (2020)
Tian, K., et al.: Weakly-supervised nucleus segmentation based on point annotations: a coarse-to-fine self-stimulated learning strategy. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12265, pp. 299–308. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59722-1_29
Toivanen, P.J.: New geodosic distance transforms for gray-scale images. Pattern Recogn. Lett. 17(5), 437–450 (1996)
Uhrig, J., Rehder, E., Fröhlich, B., Franke, U., Brox, T.: Box2Pix: single-shot instance segmentation by assigning pixels to object boxes. In: 2018 IEEE Intelligent Vehicles Symposium (IV), pp. 292–299. IEEE (2018)
Vu, Q.D., Graham, S., et al.: Methods for segmentation and classification of digital microscopy tissue images. Front. Bioeng. Biotech., p. 53 (2019)
Wang, G., et al.: DeepiGeoS: a deep interactive geodesic framework for medical image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 41(7), 1559–1572 (2019). https://doi.org/10.1109/TPAMI.2018.2840695
Yoo, I., Yoo, D., Paeng, K.: PseudoEdgeNet: nuclei segmentation only with point annotations. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11764, pp. 731–739. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32239-7_81
Zhou, X., Wang, D., Krähenbühl, P.: Objects as points. arXiv preprint arXiv:1904.07850 (2019)
Acknowledgment
This work was supported by IITP grant funded by the Korean government (MSIT) (No.2021-0-02068, Artificial Intelligence Innovation Hub), the National Research Foundation of Korea (NRF) grant funded by the Korean Government (MSIT) (No. 2019R1C1C1008727), Smart Health Care Program funded by the Korean National Police Agency (220222M01), DGIST R &D program of the Ministry of Science and ICT of KOREA (21-DPIC-08)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Nam, S., Jeong, J., Luna, M., Chikontwe, P., Park, S.H. (2023). PROnet: Point Refinement Using Shape-Guided Offset Map for Nuclei Instance Segmentation. In: Greenspan, H., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2023. MICCAI 2023. Lecture Notes in Computer Science, vol 14220. Springer, Cham. https://doi.org/10.1007/978-3-031-43907-0_51
Download citation
DOI: https://doi.org/10.1007/978-3-031-43907-0_51
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-43906-3
Online ISBN: 978-3-031-43907-0
eBook Packages: Computer ScienceComputer Science (R0)
