Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content

Advertisement

SpringerLink
Log in

A structural-constraint 3D point clouds segmentation adversarial method

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Point cloud segmentation is a key task of shape analysis with various applications. Existing segmentation methods usually apply a single segmentation network to compute point-wise loss for network training. The point-wise loss, which will lose some structure information and resulting in non-consistency labeling. To deal with this problem, we propose a segmentation adversarial framework. Compared with the existing segmentation methods, the proposed method extends the single segmentation network to a complex network by adding a discrimination network, which can compensate adversarial structural loss and enforce spatial label consistency. The discrimination network is added to the single segmentation network, without increasing any computational burden in testing process as it is not required during inference. Moreover, a structural-constraint point cloud segmentation adversarial network is proposed. Especially, in feature-encoding process, we capture the fix-scale neighborhood of each point and concern the relationship between point pairs to provide better local context. Additionally, the proposed method realizes the global constraint through condition setting and the consistency of prediction label by using learnable loss function. Experimental results show the proposed method can produce more reasonable and consistency segmentation and labeling results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Explore related subjects

Find the latest articles, discoveries, and news in related topics.

References

  1. Arjovsky, M., Chintala, S., Bottou, L.: Wasserstein gan (2017). https://arxiv.org/abs/1701.07875

  2. Chen, H., Huang, J., Remil, O., Xie, H., Qin, J., Guo, Y., Wei, M., Wang, J.: Structure-guided shape-preserving mesh texture smoothing via joint low-rank matrix recovery. Comput. Aided Des 115, 122–134 (2019)

    Article  Google Scholar 

  3. Chen, H., Wei, M., Sun, Y., Xie, X., Wang, J.: Multi-Patch Collaborative Point Cloud Denoising Via Low-Rank Recovery with Graph Constraint. IEEE transactions on visualization and computer graphics (2019)

  4. Chen, X., Golovinskiy, A., Funkhouser, T.: A benchmark for 3d mesh segmentation. ACM Trans. Graph. 28(3), 73:1–73:12 (2009)

    Article  Google Scholar 

  5. Denton, E., Chintala, S., Szlam, A., Fergus, R.: Deep generative image models using a laplacian pyramid of adversarial networks. In: Proceedings of the 28th International Conference on Neural Information Processing Systems, pp. 1486–1494 (2015)

  6. Fan, Y., Wang, M., Geng, N., He, D., Chang, J., Zhang, J.J.: A self-adaptive segmentation method for a point cloud. V. Comput. 34(5), 659–673 (2018)

    Google Scholar 

  7. Golovinskiy, A., Funkhouser, T.: Randomized cuts for 3d mesh analysis. ACM Trans. Graph. 27(5), 145:1–145:12 (2008)

    Article  Google Scholar 

  8. Goodfellow, I.J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. In: Proceedings of the 27th International Conference on Neural Information Processing Systems, pp. 2672–2680 (2014)

  9. Guo, K., Zou, D., Chen, X.: 3d mesh labeling via deep convolutional neural networks. ACM Trans. Graph. 35(1), 3:1–3:12 (2015)

    Article  Google Scholar 

  10. Huang, Q., Wang, W., Neumann, U.: Recurrent slice networks for 3d segmentation of point clouds. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2626–2635 (2018)

  11. Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1125–1134 (2017)

  12. Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: B. Leibe, J. Matas, N. Sebe, M. Welling (eds.) European Conference on Computer Vision (ECCV), pp. 694–711 (2016)

  13. Kalogerakis, E., Averkiou, M., Maji, S., Chaudhuri, S.: 3d shape segmentation with projective convolutional networks. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) pp. 6630–6639 (2017)

  14. Kalogerakis, E., Hertzmann, A., Singh, K.: Learning 3d mesh segmentation and labeling. ACM Trans. Graph. 29(4), 102:1–102:12 (2010)

    Article  Google Scholar 

  15. Le, T., Bui, G., Duan, Y.: A multi-view recurrent neural network for 3d mesh segmentation. Comput. Graph. 66, 103–112 (2017). (Shape Modeling International 2017)

    Article  Google Scholar 

  16. Le, T., Duan, Y.: Pointgrid: A deep network for 3d shape understanding. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 9204–9214 (2018)

  17. Ledig, C., Theis, L., Huszar, F., Caballero, J., Aitken, A.P., Tejani, A., Totz, J., Wang, Z., Shi, W.: Photo-realistic single image super-resolution using a generative adversarial network. pp. 4681–4690 (2017)

  18. Li, H., Sun, Z., Li, Q., Shi, J.: 3d shape co-segmentation by combining sparse representation with extreme learning machine. Adv. Multimed. Inf. Process. PCM 2018, 570–581 (2018)

    Google Scholar 

  19. Li, H., Sun, Z., Wu, Y., Li, B.: PPSAN: Perceptual-aware 3d point cloud segmentation via adversarial learning. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 4025–4029 (2019)

  20. Li, J., Chen, B.M., Lee, G.H.: SO-Net: Self-organizing network for point cloud analysis. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 9397–9406 (2018)

  21. Liu, J., Rosin, P.L., Sun, X., Xiao, J., Lian, Z.: Image-driven unsupervised 3d model co-segmentation. V. Comput. 35(6–8), 909–920 (2019)

    Google Scholar 

  22. Luc, P., Couprie, C., Chintala, S., Verbeek, J.: Semantic segmentation using adversarial networks. In: NIPS Workshop on Adversarial Training, pp. 1–9 (2016)

  23. Luo, P., Wu, Z., Xia, C., Feng, L., Ma, T.: Co-segmentation of 3d shapes via multi-view spectral clustering. V. Comput. 29(6–8), 587–597 (2013)

    Google Scholar 

  24. Maas, A.L., Hannun, A.Y., Ng, A.Y.: Rectifier nonlinearities improve neural network acoustic models (2013)

  25. Mao, X., Li, Q., Xie, H., Lau, R.K., Wang, Z., Smolley, S.: Least squares generative adversarial networks. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 2813–2821 (2017)

  26. Maturana, D., Scherer, S.: Voxnet: A 3d convolutional neural network for real-time object recognition. In: 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 922–928 (2015)

  27. Mirza, M., Osindero, S.: Conditional generative adversarial nets. CoRR (2014). http://arxiv.org/abs/1411.1784

  28. Pan, R., Taubin, G.: Automatic segmentation of point clouds from multi-view reconstruction using graph-cut. V. Comput. 32(5), 601–609 (2016)

    Google Scholar 

  29. Qi, C.R., Su, H., Mo, K., Guibas, L.J.: PointNet: Deep learning on point sets for 3d classification and segmentation. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 77–85 (2017)

  30. Qi, C.R., Su, H., Nießner, M., Dai, A., Yan, M., Guibas, L.J.: Volumetric and multi-view cnns for object classification on 3d data. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5648–5656 (2016)

  31. Qi, C.R., Yi, L., Su, H., Guibas, L.J.: PointNet++: Deep hierarchical feature learning on point sets in a metric space. In: Conference on Neural Information Processing Systems, pp. 5099–5108 (2017)

  32. Radford, A., Metz, L., Chintala, S.: Unsupervised representation learning with deep convolutional generative adversarial networks. CoRR (2016). http://arxiv.org/abs/1511.06434

  33. Shapira, L., Shamir, A., Cohen-Or, D.: Consistent mesh partitioning and skeletonisation using the shape diameter function. V. Comput. 24(4), 249–259 (2008)

    Google Scholar 

  34. Shen, Y., Feng, C., Yang, Y., Tian, D.: Mining point cloud local structures by kernel correlation and graph pooling. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4548–4557 (2018)

  35. Shu, Z., Qi, C., Xin, S., Hu, C., Wang, L., Zhang, Y., Liu, L.: Unsupervised 3d shape segmentation and co-segmentation via deep learning. Comput. Aided Geom. Des. 43, 39–52 (2016)

    Article  MathSciNet  Google Scholar 

  36. Souly, N., Spampinato, C., Shah, M.: Semi and weakly supervised semantic segmentation using generative adversarial network. In: IEEE International Conference on Computer Vision (ICCV), pp. 5689–5697 (2017)

  37. Wang, P., Gan, Y., Shui, P., Yu, F., Zhang, Y., Chen, S., Sun, Z.: 3d shape segmentation via shape fully convolutional networks. Comput. Gr. 70, 128–139 (2018). (CAD/Graphics 2017)

    Article  Google Scholar 

  38. Wang, P.S., Liu, Y., Guo, Y.X., Sun, C.Y., Tong, X.: O-cnn: Octree-based convolutional neural networks for 3d shape analysis. ACM Trans. Graph. 36(4), 72:1–72:11 (2017)

    Google Scholar 

  39. Wang, X., Zhou, B., Wang, Z., Zou, D., Chen, X., Zhao, Q.: Efficiently consistent affinity propagation for 3d shapes co-segmentation. V. Comput. 34(6–8), 997–1008 (2018)

    Google Scholar 

  40. Wang, Y., Sun, Y., Liu, Z., Sarma, S.E., Bronstein, M.M., Solomon, J.M.: Dynamic graph cnn for learning on point clouds. CoRR abs/1801.07829 (2018). http://arxiv.org/abs/1801.07829

  41. Wei, M., Huang, J., Xie, X., Liu, L., Wang, J., Qin, J.: Mesh denoising guided by patch normal co-filtering via kernel low-rank recovery. IEEE Trans. V. Comput. Graph. 25(10), 2910–2926 (2019)

    Article  Google Scholar 

  42. Wu, J., Zhang, C., Xue, T., Freeman, W.T., Tenenbaum, J.B.: Learning a probabilistic latent space of object shapes via 3d generative-adversarial modeling. In: Conference on Neural Information Processing Systems (NIPS), pp. 82–90 (2016)

  43. Wu, Z., Song, S., Khosla, A., Yu, F., Zhang, L., Tang, X., Xiao, J.: 3D ShapeNets: A deep representation for volumetric shapes. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1912–1920 (2015)

  44. Xu, H., Dong, M., Zhong, Z.: Directionally convolutional networks for 3d shape segmentation. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 2717–2726 (2017)

  45. Yi, L., Kim, V.G., Ceylan, D., Shen, I.C., Yan, M., Su, H., Lu, C., Huang, Q., Sheffer, A., Guibas, L.: A scalable active framework for region annotation in 3d shape collections. ACM Trans. Graph. 35(6), 210:1–210:12 (2016)

    Article  Google Scholar 

  46. Yi, L., Su, H., Guo, X., Guibas, L.: Syncspeccnn: Synchronized spectral cnn for 3d shape segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6584–6592 (2017)

  47. Yu, F., Liu, K., Zhang, Y., Zhu, C., Xu, K.: PartNet: A recursive part decomposition network for fine-grained and hierarchical shape segmentation. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2019)

  48. Zhang, E., Mischaikow, K., Turk, G.: Feature-based surface parameterization and texture mapping. ACM Trans. Graph. 24(1), 1–27 (2005)

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported by National High Technology Research and Development Program of China (No. 2007AA01Z334); National Natural Science Foundation of China (Nos. 61321491, 61272219); Innovation Fund of State Key Laboratory for Novel Software Technology (Nos. ZZKT2013A12, ZZKT2016A11); Program for New Century Excellent Talents in University of China (No. NCET-04-04605); Nanjing University Innovation and Creative Program for Ph.D. candidate (No. 2016013).

Author information

Authors and Affiliations

  1. State Key Lab for Novel Software Technology, Nanjing University, Nanjing, 210023, China

    Hongyan Li & Zhengxing Sun

Authors
  1. Hongyan Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhengxing Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhengxing Sun.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, H., Sun, Z. A structural-constraint 3D point clouds segmentation adversarial method. Vis Comput 37, 325–340 (2021). https://doi.org/10.1007/s00371-020-01801-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-020-01801-z

Keywords

Search

Navigation