Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content

Advertisement

Springer Nature Link
Log in

Hausdorff point convolution with geometric priors

  • Research Paper
  • Special Focus on Visual Computing with Machine Learning
  • Published:
Science China Information Sciences Aims and scope Submit manuscript

Abstract

Developing point convolution for irregular point clouds to extract deep features remains challenging. Current methods evaluate the response by computing point set distances which account only for the spatial alignment between two point sets, but not quite for their underlying shapes. Without a shape-aware response, it is hard to characterize the 3D geometry of a point cloud efficiently with a compact set of kernels. In this paper, we advocate the use of modified Hausdorff distance as a shape-aware distance measure for calculating point convolutional responses. The technique we present, coined Hausdorff point convolution (HPC), is shape-aware. We show that HPC constitutes a powerful point feature learning with a rather compact set of only four types of geometric priors as kernels. We further develop an HPC-based deep neural network (HPC-DNN). Task-specific learning can be achieved by tuning the network weights for combining the shortest distances between the input and the kernel point sets. We also realize hierarchical feature learning by designing a multi-kernel HPC for multi-scale feature encoding. Extensive experiments demonstrate that HPC-DNN outperforms strong point convolution baselines (e.g., KPConv), achieving 2.8% mIoU performance boost on S3DIS and 1.5% on SemanticKITTI for semantic segmentation task.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Yang Y Q, Feng C, Shen Y R, et al. FoldingNet: point cloud auto-encoder via deep grid deformation. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2018. 206–215

  2. Rao Y M, Lu J W, Zhou J. Spherical fractal convolutional neural networks for point cloud recognition. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2019. 452–460

  3. Wu W X, Qi Z G, Li F X. PointConv: deep convolutional networks on 3D point clouds. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2019. 9621–9630

  4. Lei H, Akhtar N, Mian A. Octree guided CNN with spherical kernels for 3D point clouds. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2019. 9631–9640

  5. Tatarchenko M, Park J, Koltun V, et al. Tangent convolutions for dense prediction in 3D. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2018. 3887–3896

  6. Thomas H, Qi C R, Deschaud J E, et al. KPConv: flexible and deformable convolution for point clouds. In: Proceedings of International Conference on Computer Vision, 2019. 6411–6420

  7. Armeni I, Sax S, Zamir A R, et al. Joint 2D-3D-semantic data for indoor scene understanding. 2017. ArXiv:1702.01105

  8. Behley J, Garbade M, Milioto A, et al. SemanticKITTI: a dataset for semantic scene understanding of LiDAR sequences. In: Proceedings of International Conference on Computer Vision, 2019. 9297–9307

  9. Steder B, Rusu R B, Konolige K, et al. NARF: 3D range image features for object recognition. In: Proceedings of International Conference on Intelligent Robots and Systems, 2010

  10. Rusu R B, Bradski G, Thibaux R, et al. Fast 3D recognition and pose using the viewpoint feature histogram. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, 2010. 2155–2162

  11. Salti S, Tombari F, Di Stefano L. SHOT: unique signatures of histograms for surface and texture description. Comput Vision Image Underst, 2014, 125: 251–264

    Article  Google Scholar 

  12. Qi C R, Su H, Mo K C, et al. PointNet: deep learning on point sets for 3D classification and segmentation. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2017. 652–660

  13. Qi C R, Yi L, Su H, et al. PointNet++: deep hierarchical feature learning on point sets in a metric space. In: Proceedings of Advances in Neural Information Processing Systems, 2017. 5099–5108

  14. Li Y Y, Bu R, Sun M C, et al. PointCNN: convolution on χ-transformed points. In: Proceedings of Advances in Neural Information Processing Systems, 2018. 820–830

  15. Su H, Maji S, Kalogerakis E, et al. Multi-view convolutional neural networks for 3D shape recognition. In: Proceedings of International Conference on Computer Vision, 2015. 945–953

  16. Maturana D, Scherer S. VoxNet: a 3D convolutional neural network for real-time object recognition. In: Proceedings of International Conference on Intelligent Robots and Systems (IROS), 2015. 922–928

  17. Ye Y Y, Chen H J, Zhang C, et al. SARPNET: shape attention regional proposal network for LiDAR-based 3D object detection. Neurocomputing, 2020, 379: 53–63

    Article  Google Scholar 

  18. Zhou H, Zhu X, Song X, et al. Cylinder3D: an effective 3D framework for driving-scene LiDAR semantic segmentation. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2021

  19. Cortinhal T, Tzelepi G, Aksoy E. SalsaNext: fast, uncertainty-aware semantic segmentation of LiDAR point clouds for autonomous driving. In: Proceedings of the 15th International Symposium, San Diego, 2021

  20. Riegler G, Ulusoy A O, Geiger A. OctNet: learning deep 3D representations at high resolutions. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2017. 3577–3586

  21. Klokov R, Lempitsky V. Escape from cells: deep Kd-networks for the recognition of 3D point cloud models. In: Proceedings of International Conference on Computer Vision, 2017. 863–872

  22. Graham B, Engelcke M, van der Maaten L. 3D semantic segmentation with submanifold sparse convolutional networks. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2018. 9224–9232

  23. Su H, Jampani V, Sun D, et al. SPLATNet: sparse lattice networks for point cloud processing. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2018. 2530–2539

  24. Simonovsky M, Komodakis N. Dynamic edge-conditioned filters in convolutional neural networks on graphs. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2017. 3693–3702

  25. Hermosilla P, Ritschel T, Vázquez P P, et al. Monte Carlo convolution for learning on non-uniformly sampled point clouds. ACM Trans Graph, 2019, 37: 1–12

    Article  Google Scholar 

  26. Xu Y F, Fan T Q, Xu M Y, et al. SpiderCNN: deep learning on point sets with parameterized convolutional filters. In: Proceedings of Europe Conference on Computer Vision, 2018. 87–102

  27. Atzmon M, Maron H, Lipman Y. Point convolutional neural networks by extension operators. ACM Trans Graph, 2018, 37: 1–12

    Article  Google Scholar 

  28. Hu Q Y, Yang B, Xie L H, et al. RandLA-Net: efficient semantic segmentation of large-scale point clouds. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2020. 11108–11117

  29. Wang Y, Sun Y B, Liu Z W, et al. Dynamic graph CNN for learning on point clouds. ACM Trans Graph, 2019, 38: 1–12

    Google Scholar 

  30. Liu Y C, Fan B, Xiang S M, et al. Relation-shape convolutional neural network for point cloud analysis. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2019. 8895–8904

  31. Lan S Y, Yu R C, Yu G, et al. Modeling local geometric structure of 3D point clouds using GEO-CNN. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2019. 998–1008

  32. Dubuisson M P, Jain A K. A modified Hausdorff distance for object matching. In: Proceedings of the 12th International Conference on Pattern Recognition, 1994. 566–568

  33. Jesorsky O, Kirchberg K J, Frischholz R W. Robust face detection using the Hausdorff distance. In: Proceedings of International Conference on Audio- and Video-based Biometric Person Authentication. Berlin: Springer, 2001. 90–95

    MATH  Google Scholar 

  34. Dai A, Chang A X, Savva M, et al. ScanNet: richly-annotated 3D reconstructions of indoor scenes. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2017

  35. Hu Z Y, Zhen M M, Bai X Y, et al. JSENet: joint semantic segmentation and edge detection network for 3D point clouds. 2020. ArXiv:2007.06888

  36. Wang S L, Suo S, Ma W C, et al. Deep parametric continuous convolutional neural networks. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2018. 2589–2597

  37. Zhao H S, Jiang L, Fu C W, et al. PointWeb: enhancing local neighborhood features for point cloud processing. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2019. 5565–5573

  38. Jiang L, Zhao H S, Liu S, et al. Hierarchical point-edge interaction network for point cloud semantic segmentation. In: Proceedings of International Conference on Computer Vision, 2019. 10433–10441

  39. Landrieu L, Simonovsky M. Large-scale point cloud semantic segmentation with superpoint graphs. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2018. 4558–4567

  40. Jaritz M, Gu J Y, Su H. Multi-view pointnet for 3D scene understanding. In: Proceedings of 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), 2019. 3995–4003

  41. Han W K, Wen C L, Wang C, et al. Point2Node: correlation learning of dynamic-node for point cloud feature modeling. In: Proceedings of AAAI Conference on Artificial Intelligence, 2020. 10925–10932

  42. Choy C, Gwak J, Savarese S. 4D spatio-temporal convnets: Minkowski convolutional neural networks. 2019. ArXiv:1904.08755

  43. Hackel T, Savinov N, Ladicky L, et al. SEMANTIC3D.NET: a new large-scale point cloud classification benchmark. In: Proceedings of ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2017. 91–98

  44. Geiger A, Lenz P, Urtasun R. Are we ready for autonomous driving? The KITTI vision benchmark suite. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2012. 3354–3361

  45. Zhang Y, Zhou Z X, David P, et al. PolarNet: an improved grid representation for online LiDAR point clouds semantic segmentation. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2020. 9601–9610

  46. Thomas H, Goulette F, Deschaud J E, et al. Semantic classification of 3D point clouds with multiscale spherical neighborhoods. In: Proceedings of International Conference on 3D Vision (3DV), 2018. 390–398

  47. Roynard X, Deschaud J E, Goulette F. Classification of point cloud scenes with multi-scale voxel deep network. 2018. ArXiv:1804.03583

Download references

Acknowledgements

This work was supported by National Natural Science Foundation of China (Grant Nos. U2001206, 61902254), Guangdong Talent Program (Grant No. 2019JC05X328), Guangdong Science and Technology Program (Grant Nos. 2020A0505100064, 2015A030312015), DEGP Key Project (Grant Nos. 2018KZDXM058, 2020SFKC059), Shenzhen Science and Technology Program (Grant No. RCJC20200714114435012), National Engineering Laboratory for Big Data System Computing Technology, and Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ).

Author information

Author notes

  1. Lin L Q and Huang P D have the same contribution to this work.

Authors and Affiliations

  1. College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, 518060, China

    Liqiang Lin, Pengdi Huang, Fuyou Xue, Daniel Cohen-Or & Hui Huang

  2. School of Computer Science, National University of Defense Technology, Changsha, 410073, China

    Kai Xu

Authors
  1. Liqiang Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pengdi Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fuyou Xue
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kai Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Daniel Cohen-Or
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Hui Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hui Huang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lin, L., Huang, P., Xue, F. et al. Hausdorff point convolution with geometric priors. Sci. China Inf. Sci. 64, 210105 (2021). https://doi.org/10.1007/s11432-021-3311-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11432-021-3311-2

Keywords

Search

Navigation