Abstract
Learning meaningful local and global information remains a challenge in point cloud segmentation tasks. When utilizing local information, prior studies indiscriminately aggregates neighbor information from different classes to update query points, potentially compromising the distinctive feature of query points. In parallel, inaccurate modeling of long-distance contextual dependencies when utilizing global information can also impact model performance. To address these issues, we propose GSTran, a novel transformer network tailored for the segmentation task. The proposed network mainly consists of two principal components: a local geometric transformer and a global semantic transformer. In the local geometric transformer module, we explicitly calculate the geometric disparity within the local region. This enables amplifying the affinity with geometrically similar neighbor points while suppressing the association with other neighbors. In the global semantic transformer module, we design a multi-head voting strategy. This strategy evaluates semantic similarity across the entire spatial range, facilitating the precise capture of contextual dependencies. Experiments on ShapeNetPart and S3DIS benchmarks demonstrate the effectiveness of the proposed method, showing its superiority over other algorithms. The code is available at https://github.com/LAB123-tech/GSTran.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bazazian, D., Casas, J.R., Ruiz-Hidalgo, J.: Fast and robust edge extraction in unorganized point clouds. In: 2015 International Conference on Digital Image Computing: Techniques and applications, pp. 1–8. IEEE (2015)
Chen, J., Kakillioglu, B., Velipasalar, S.: Background-aware 3-D point cloud segmentation with dynamic point feature aggregation. IEEE Trans. Geosci. Remote Sens. 60, 1–12 (2022)
Du, Z., Ye, H., Cao, F.: A novel local-global graph convolutional method for point cloud semantic segmentation. IEEE Trans. Neural Netw. Learn. Syst. (2022)
Fan, S., Dong, Q., Zhu, F., Lv, Y., Ye, P., Wang, F.Y.: SCF-net: learning spatial contextual features for large-scale point cloud segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14504–14513 (2021)
Feng, M., Zhang, L., Lin, X., Gilani, S.Z., Mian, A.: Point attention network for semantic segmentation of 3D point clouds. Pattern Recogn. 107, 107446 (2020)
Guo, B., Deng, L., Wang, R., Guo, W., Ng, A.H.M., Bai, W.: MCTNet: multiscale cross-attention based transformer network for semantic segmentation of large-scale point cloud. IEEE Trans. Geosci. Remote Sens. (2023)
Guo, M.-H., Cai, J.-X., Liu, Z.-N., Mu, T.-J., Martin, R.R., Hu, S.-M.: PCT: point cloud transformer. Comput. Vis. Media 7(2), 187–199 (2021). https://doi.org/10.1007/s41095-021-0229-5
Li, Y., Duan, Y.: Multi-scale network with attentional multi-resolution fusion for point cloud semantic segmentation. In: 2022 26th International Conference on Pattern Recognition, pp. 3980–3986. IEEE (2022)
Li, Z., et al.: Geodesic self-attention for 3d point clouds. Adv. Neural. Inf. Process. Syst. 35, 6190–6203 (2022)
Liu, M., et al.: PartSLIP: low-shot part segmentation for 3d point clouds via pretrained image-language models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 21736–21746 (2023)
Lu, D., Xie, Q., Gao, K., Xu, L., Li, J.: 3DCTN: 3D convolution-transformer network for point cloud classification. IEEE Trans. Intell. Transp. Syst. 23(12), 24854–24865 (2022)
Ma, X., Qin, C., You, H., Ran, H., Fu, Y.: Rethinking network design and local geometry in point cloud: a simple residual MLP framework. In: International Conference on Learning Representations (2021)
Mei, G., Riz, L., Wang, Y., Poiesi, F.: Geometrically-driven aggregation for zero-shot 3D point cloud understanding. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2024)
Mei, G., et al.: Unsupervised point cloud representation learning by clustering and neural rendering. Int. J. Comput. Vision 1–19 (2024)
Park, J., Lee, S., Kim, S., Xiong, Y., Kim, H.J.: Self-positioning point-based transformer for point cloud understanding. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 21814–21823 (2023)
Qi, C.R., Yi, L., Su, H., Guibas, L.J.: PointNet++: deep hierarchical feature learning on point sets in a metric space. Adv. Neural Inf. Process. Syst. 30 (2017)
Qiu, S., Anwar, S., Barnes, N.: Geometric back-projection network for point cloud classification. IEEE Trans. Multimedia 24, 1943–1955 (2021)
Qiu, S., Anwar, S., Barnes, N.: Semantic segmentation for real point cloud scenes via bilateral augmentation and adaptive fusion. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1757–1767 (2021)
Ran, H., Liu, J., Wang, C.: Surface representation for point clouds. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 18942–18952 (2022)
Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Song, H., Feng, H.Y.: A progressive point cloud simplification algorithm with preserved sharp edge data. Int. J. Adv. Manuf. Technol. 45, 583–592 (2009)
Srivastava, S., Sharma, G.: Exploiting local geometry for feature and graph construction for better 3d point cloud processing with graph neural networks. In: 2021 IEEE INternational Conference on robotics and Automation, pp. 12903–12909. IEEE (2021)
Tang, L., Zhan, Y., Chen, Z., Yu, B., Tao, D.: Contrastive boundary learning for point cloud segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8489–8499 (2022)
Wang, C., Ning, X., Sun, L., Zhang, L., Li, W., Bai, X.: Learning discriminative features by covering local geometric space for point cloud analysis. IEEE Trans. Geosci. Remote Sens. 60, 1–15 (2022)
Wu, C., Zheng, J., Pfrommer, J., Beyerer, J.: Attention-based point cloud edge sampling. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5333–5343 (2023)
Yan, X., Zheng, C., Li, Z., Wang, S., Cui, S.: PointASNL: Robust point clouds processing using nonlocal neural networks with adaptive sampling. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5589–5598 (2020)
Yu, X., Rao, Y., Wang, Z., Liu, Z., Lu, J., Zhou, J.: PoinTr: diverse point cloud completion with geometry-aware transformers. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 12498–12507 (2021)
Zhang, H., Wang, C., Yu, L., Tian, S., Ning, X., Rodrigues, J.: PointGT: a method for point-cloud classification and segmentation based on local geometric transformation. IEEE Trans. Multimed. (2024)
Zhao, H., Jiang, L., Fu, C.W., Jia, J.: PointWeb: enhancing local neighborhood features for point cloud processing. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5565–5573 (2019)
Zhao, H., Jiang, L., Jia, J., Torr, P.H., Koltun, V.: Point transformer. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 16259–16268 (2021)
Zhou, W., et al.: GTNet: graph transformer network for 3D point cloud classification and semantic segmentation. arXiv preprint arXiv:2305.15213 (2023)
Zhu, H., Yang, H., Wu, X., Huang, D., Zhang, S., et.al.: PonderV2: pave the way for 3D foundataion model with a universal pre-training paradigm. arXiv preprint arXiv:2310.08586 (2023)
Acknowledgements
This work was supported in part by the National Natural Science Foundation of China under Grants 62132006, 62441203, 62311530101 and 62271237, Natural Science Foundation of Jiangxi Province of China under Grants 20223AEI91002, the PNRR project FAIR- Future AI Research (PE00000013), under the NRRP MUR program funded by the NextGenerationEU and Postgraduate Innovation Special Fund of Jiangxi Province under Grant YC2023-B184.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
A Appendix
A Appendix
Experiments on robustness, outdoor point cloud scenarios, and others are included in the supplementary material. For more details, please refer to the links below. Either of the following two links can be chosen.
Link 1: Google Drive.
https://drive.google.com/file/d/1rS36mBizZS4yHw4tcuOc5JAYDYLr1SUk/view?usp=sharing
Link 2: Baidu Drive. Password: 1234
https://pan.baidu.com/s/1T3hOOrgMKvwmQOvGTzaeVQ
Password: 1234
Rights and permissions
Copyright information
© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Li, A., Lv, C., Mei, G., Zuo, Y., Zhang, J., Fang, Y. (2025). GSTran: Joint Geometric and Semantic Coherence for Point Cloud Segmentation. In: Antonacopoulos, A., Chaudhuri, S., Chellappa, R., Liu, CL., Bhattacharya, S., Pal, U. (eds) Pattern Recognition. ICPR 2024. Lecture Notes in Computer Science, vol 15318. Springer, Cham. https://doi.org/10.1007/978-3-031-78456-9_29
Download citation
DOI: https://doi.org/10.1007/978-3-031-78456-9_29
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-78455-2
Online ISBN: 978-3-031-78456-9
eBook Packages: Computer ScienceComputer Science (R0)
