research-article

BEACon: a boundary embedded attentional convolution network for point cloud instance segmentation

Authors:

Nadia Magnenat ThalmannAuthors Info & Claims

The Visual Computer, Volume 38, Issue 7

Pages 2303 - 2313

https://doi.org/10.1007/s00371-021-02112-7

Published: 01 July 2022 Publication History

Abstract

Motivated by how humans perceive geometry and color to recognize objects, we propose a boundary embedded attentional convolution (BEACon) network for point cloud instance segmentation. At the core of BEACon, we introduce the attentional weight in the convolution layer to adjust the neighboring features, with the weight being adapted to the relationship between geometry and color changes. As a result, BEACon makes use of both geometry and color information, takes instance boundary as an important feature, and thus learns a more discriminative feature representation in the neighborhood. Experimental results show that BEACon outperforms the state-of-the-art by a large margin. Ablation studies are also provided to prove the large benefit of incorporating both geometry and color into attention weight for instance segmentation.

References

[1]

Haoran L, Fazhi H, and Yilin C Learning dynamic simultaneous clustering and classification via automatic differential evolution and firework algorithm Appl. Soft Comput. J. 2020 96 106593

[2]

Zhang S and He F DRCDN: learning deep residual convolutional dehazing networks Vis. Comput. 2020 36 1797-1808

[3]

Li, Y., Bu, R., Sun, M., Chen, B.: PointCNN: Convolution On X-Transformed Points. In: Advances in Neural Information Processing Systems (2018)

[4]

Qi, C.R., Yi, L., Su, H., Guibas, L.J.: PointNet++: Deep hierarchical feature learning on point sets in a metric space. In: Advances in Neural Information Processing Systems (2017)

[5]

Rethage, D., Wald, J., Sturm, J, Navab, N., Tombari, F.: Fully-convolutional point networks for large-scale point clouds. In: ECCV (2018)

[6]

Wu, W., Qi, Z., Fuxin, L.: PointConv: Deep convolutional networks on 3D point clouds. In: CVPR (2019)

[7]

Xu, Y., Fan, T., Xu, M., Zeng, L., Qiao, Y.: SpiderCNN: Deep learning on point sets with parameterized convolutional filters. In: ECCV (2018)

[8]

Li H, He F, Liang Y, and Quan Q A dividing-based many-objective evolutionary algorithm for large-scale feature selection Soft Comput. 2020 24 6851-6870

[9]

Liu, Y., Fan, B., Xiang, S., Pan, C.: Relation-shape convolutional neural network for point cloud analysis. In: CVPR (2019)

[10]

Wang, L., Huang, Y., Hou, Y., Zhang, S., Shan, J.: Graph attention convolution for point cloud semantic segmentation. In: CVPR (2019)

[11]

Zhao, H., Jiang, L., Fu, C.-W., Jia, J.: PointWeb: Enhancing local neighborhood features for point cloud processing. In: CVPR (2019)

[12]

Landrieu L and Obozinski G Cut Pursuit: fast algorithms to learn piecewise constant functions on general weighted graphs SIAM J. Imaging Sci. Soc. Ind. Appl. Math. 2017 10 4 1724-1766

[13]

Armeni, I., Sener, O., Zamir, A.R., Jiang, H., Brilakis, I., Fischer, M., Savarese, S.: 3D semantic parsing of large-scale indoor spaces. In: CVPR, pp. 1534–1543 (2016)

[14]

Mo, K., Zhu, S., Chang, A.X., Yi, L., Tripathi, S., Guibas, L.J., Su, H.: PartNet: A large-scale Benchmark for fine-grained and hierarchical part-level 3d object understanding. In: CVPR (2019)

[15]

Maturana, D., Scherer, S.: “VoxNet: A 3D convolutional neural network for real-time object recognition. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (2015)

[16]

Wu, Z., Song, S., Khosla, A., Yu, F., Zhang, L., Tang, X., Xiao, J.: 3D ShapeNets: A deep representation for volumetric shapes. In: CVPR, vol. 07-12-June, pp. 1912–1920 (2015)

[17]

Graham, B., Engelcke, M., Van Der Maaten, L.: 3D semantic segmentation with submanifold sparse convolutional networks. In: CVPR (2018)

[18]

Choy, C., Gwak, J., Savarese, S.: 4D Spatio-temporal ConvNets: Minkowski convolutional neural networks. In: CVPR (2019)

[19]

Boulch A, Guerry J, Le Saux B, and Audebert N SnapNet: 3D point cloud semantic labeling with 2D deep segmentation networks Comput. Gr. 2018 71 189-198

[20]

Qi, C.R., Su, H., Mo, K., Guibas, L.J.: PointNet: Deep learning on point sets for 3d classification and segmentation. In: CVPR, pp. 601–610 (2017)

[21]

Ye, X., Li, J., Huang, H., Du, L., Zhang, X.: 3D Recurrent neural networks with context fusion for point cloud semantic segmentation. In: ECCV, pp. 403–417 (2018)

[22]

Liu, S., Xie, S., Chen, Z., Tu, Z.: Attentional ShapeContextNet for point cloud recognition. In: CVPR, pp. 4606–4615 (2018)

[23]

Wang, Y., Bronstein, M.M., Solomon, J.M., Sun, Y., Liu, Z., Sarma, S.E.: Dynamic graph CNN for learning on point clouds. In: ACM Trans. Graph. 1, 1, Article, vol. 1, No. 1, p. 13 (2019)

[24]

Landrieu, L., Simonovsky, M.: Large-scale point cloud semantic segmentation with superpoint graphs. In: CVPR (2018)

[25]

Landrieu, L., Boussaha, M.: Point cloud oversegmentation with graph-structured deep metric learning. In: CVPR (2019)

[26]

Sun Y, Miao Y, Chen J, and Pajarola R PGCNet: patch graph convolutional network for point cloud segmentation of indoor scenes Vis. Comput. 2020 36 2407-2418

[27]

Wang, C., Samari, B., Siddiqi, K.: Local spectral graph convolution for point set feature learning. In: ECCV (2018)

[28]

Li H and Sun Z A structural-constraint 3D point clouds segmentation adversarial method Vis. Comput. 2020 37 325

[29]

Lei, H., Akhtar, N., Mian, A.: Octree guided CNN with Spherical Kernels for 3D Point Clouds. In: CVPR (2019)

[30]

Komarichev, A., Zhong, Z., Hua, J.: A-CNN: Annularly convolutional neural networks on point clouds. In: CVPR (2019)

[31]

Thomas, H., Qi, C.R., Deschaud, J.-E., Marcotegui, B., Goulette, F., Guibas, L.J.: “KPConv: flexible and deformable convolution for point clouds. In: ICCV (2019)

[32]

Wang, W., Yu, R., Huang, Q., Neumann, U.: SGPN: Similarity group proposal network for 3D point cloud instance segmentation. In: CVPR (2018)

[33]

Wang, X., Liu, S., Shen, X., Shen, C., Jia, J.: Associatively segmenting instances and semantics in point clouds. In: CVPR (2019)

[34]

Pham, Q.-H., Thanh Nguyen, D., Hua Gemma Roig, B.-S., Yeung, S.-K.: JSIS3D: Joint semantic-instance segmentation of 3D point clouds with multi-task pointwise networks and multi-value conditional random fields. In: CVPR (2019)

[35]

Hou, J., Dai, A., Nießner, M.: 3D-SIS: 3D semantic instance segmentation of RGB-D scans. In: CVPR (2019)

[36]

Yang, B., Wang, J., Clark, R., Hu, Q., Wang, S., Markham, A., Trigoni, N.: Learning object bounding boxes for 3D instance segmentation on point clouds. In: NeurIPS (2019)

[37]

Lahoud, J., Ghanem, B., Pollefeys, M., Zurich, E., Oswald, M.R.: 3D instance segmentation via multi-task metric learning. In: ICCV (2019)

[38]

Groh, Fabian, Wieschollek, Patrick, Lensch, Hendrik P.A.: Flex-convolution million-scale point-cloud learning beyond grid-worlds. In: ACCV (2018)

[39]

Thomas, H., Deschaud, J.-E., Marcotegui, B., Goulette, F., Le Gall, Y.: Semantic classification of 3D point clouds with multiscale spherical neighborhoods. In: International Conference on 3D Vision (3DV), pp. 390–398 (2018)

[40]

Canny J A computational approach to edge detection IEEE Trans. Pattern Anal. Mach. Intell. 1986 PAMI–8 6 679-698

[41]

De Brabandere, B., Neven, D., Gool, L.V.: Semantic instance segmentation with a discriminative loss function. In: CVPR Workshop (2017)

[42]

Jiang, L., Zhao, H., Liu, S., Shen, X., Fu, C.-W., Jia, J.: Hierarchical point-edge interaction network for point cloud semantic segmentation. In: ICCV (2019)

[43]

Li, Y., Zhao, W., Wang, H., Sung, M., Guibas, L.: GSPN: generative shape proposal network for 3D instance segmentation in point cloud. In: CVPR (2019)

Cited By

Ren DLi JWu ZGuo JWei MGuo Y(2024)MFFNet: multimodal feature fusion network for point cloud semantic segmentationThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-023-02907-w40:8(5155-5167)Online publication date: 1-Aug-2024
https://dl.acm.org/doi/10.1007/s00371-023-02907-w
Cheng XHuang SLiao BWang YLuo X(2024)BG-Net: boundary-guidance network for object consistency maintaining in semantic segmentationThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-023-02787-040:1(373-391)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1007/s00371-023-02787-0
Sharma VTripathi AMittal HParmar ASoni AAmarwal R(2023)WeedGan: a novel generative adversarial network for cotton weed identificationThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-022-02742-539:12(6503-6519)Online publication date: 1-Dec-2023
https://dl.acm.org/doi/10.1007/s00371-022-02742-5
Show More Cited By

Index Terms

BEACon: a boundary embedded attentional convolution network for point cloud instance segmentation
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
  2. Machine learning

Index terms have been assigned to the content through auto-classification.

Recommendations

Instance-Aware Embedding for Point Cloud Instance Segmentation
Computer Vision – ECCV 2020
Abstract
Although recent works have made significant progress in encoding meaningful context information for instance segmentation in 2D images, the works for 3D point cloud counterpart lag far behind. Conventional methods use radius search or other ...
Hybrid supervised instance segmentation by learning label noise suppression
Abstract
To reach top accuracy, current fully supervised instance segmentation methods severely rely on large-scale pixel-wise labeled datasets. They are usually expensive and time-consuming to obtain. Though weakly or semi-supervised methods ...
ChaInNet: Deep Chain Instance Segmentation Network for Panoptic Segmentation
Abstract
We consider the competition between instance and semantic segmentation in panoptic segmentation to develop the deep chain instance segmentation network (ChaInNet) to mitigate this problem. Segmentation competition is caused by the usual ...

Comments

Information & Contributors

Information

Published In

cover image The Visual Computer: International Journal of Computer Graphics

The Visual Computer: International Journal of Computer Graphics Volume 38, Issue 7

Jul 2022

350 pages

ISSN:0178-2789

Issue’s Table of Contents

© The Author(s), under exclusive licence to Springer-Verlag GmbH Germany, part of Springer Nature 2021.

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 01 July 2022

Accepted: 15 March 2021

Author Tags

Qualifiers

Research-article

Funding Sources

National Research Foundation Singapore

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 01 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Ren DLi JWu ZGuo JWei MGuo Y(2024)MFFNet: multimodal feature fusion network for point cloud semantic segmentationThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-023-02907-w40:8(5155-5167)Online publication date: 1-Aug-2024
https://dl.acm.org/doi/10.1007/s00371-023-02907-w
Cheng XHuang SLiao BWang YLuo X(2024)BG-Net: boundary-guidance network for object consistency maintaining in semantic segmentationThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-023-02787-040:1(373-391)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1007/s00371-023-02787-0
Sharma VTripathi AMittal HParmar ASoni AAmarwal R(2023)WeedGan: a novel generative adversarial network for cotton weed identificationThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-022-02742-539:12(6503-6519)Online publication date: 1-Dec-2023
https://dl.acm.org/doi/10.1007/s00371-022-02742-5
He YXia GFeng HWang Z(2023)PCTP: point cloud transformer pooling block for points set abstraction structureThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-022-02688-839:11(5669-5681)Online publication date: 1-Nov-2023
https://dl.acm.org/doi/10.1007/s00371-022-02688-8
Si THe FLi P(2023)Hybrid feature constraint with clustering for unsupervised person re-identificationThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-022-02649-139:10(5121-5133)Online publication date: 1-Oct-2023
https://dl.acm.org/doi/10.1007/s00371-022-02649-1
Song YShen WPeng K(2023)A novel partial point cloud registration method based on graph attention networkThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-021-02391-039:3(1109-1120)Online publication date: 1-Mar-2023
https://dl.acm.org/doi/10.1007/s00371-021-02391-0

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents