Abstract
Point cloud registration is a challenging problem in the condition of large initial misalignments and noises. A major problem encountered in the registration algorithms is the definition of correspondence between two point clouds. Point clouds contain rich geometric information and the same geometric structure implies the same feature even if they are in different poses, which motivates us to seek a rotation-invariant feature representation for calculating the correspondence. This work proposes a rotation-invariant neural network for point cloud registration. To acquire rotation-invariant features, we firstly propose a rotation-invariant point cloud representation (RIPR) at the input level. Instead of using the original coordinates, we propose to use point pair features (PPF) and the transformed coordinates in the local reference frame (LRF) to represent a point. Then, we design a new convolution operator named Cylinder-Conv which utilizes the symmetry of cylinder-shaped voxels and the hierarchical geometry information of the surface of 3D shapes. By specifying the cylinder-shaped structures and directions, Cylinder-Conv can better capture the local neighborhood geometry of each point and maintain rotation-invariance. Finally, we combine RIPR and Cylinder-Conv to extract normalized rotation-invariant features to generate the correspondence and perform a differentiable singular value decomposition (SVD) step to estimate the rigid transformation. The proposed network presents state-of-the-art performance on point cloud registration. Experiments show that our method is robust to initial misalignments and noises.
Similar content being viewed by others
References
Liu B S, Chen X M, Han Y H, et al. Accelerating DNN-based 3D point cloud processing for mobile computing. Sci China Inf Sci, 2019, 62: 212102
Peng H T, Zhou B, Yin L Y, et al. Semantic part segmentation of single-view point cloud. Sci China Inf Sci, 2020, 63: 224101
Zhao H, Zhu C Y, Xu X, et al. Learning practically feasible policies for online 3D bin packing. Sci China Inf Sci, 2022, 65: 112105
Han L, Gu S, Zhong D, et al. Real-time globally consistent dense 3D reconstruction with online texturing. IEEE Trans Pattern Anal Mach Intell, 2022, 44: 1519–1533
Fang S, Li H, Yang M. LiDAR SLAM based multivehicle cooperative localization using iterated split CIF. IEEE Trans Intell Transp Syst, 2022. doi: https://doi.org/10.1109/TITS.2022.3174479
Holz D, Ichim A E, Tombari F, et al. Registration with the point cloud library: a modular framework for aligning in 3-D. IEEE Robot Automat Mag, 2015, 22: 110–124
Chen Y, Medioni G. Object modelling by registration of multiple range images. Image Vision Computing, 1992, 10: 145–155
Besl P J, McKay N D. Method for registration of 3-D shapes. In: Proceedings of Sensor Fusion IV: Control Paradigms and Data Structures, 1992. 586–606
Segal A, Haehnel D, Thrun S. Generalized-ICP. In: Proceedings of Robotics: Science and Systems, Seattle, 2009. 435
Agamennoni G, Fontana S, Siegwart R Y, et al. Point clouds registration with probabilistic data association. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, 2016. 4092–4098
Magnusson M, Lilienthal A, Duckett T. Scan registration for autonomous mining vehicles using 3D-NDT. J Field Robotics, 2007, 24: 803–827
Zhou Q Y, Park J, Koltun V. Fast global registration. In: Proceedings of European Conference on Computer Vision. Berlin: Springer, 2016. 766–782
Ma J, Jiang X, Fan A, et al. Image matching from handcrafted to deep features: a survey. Int J Comput Vis, 2021, 129: 23–79
Rusu R B, Blodow N, Beetz M. Fast point feature histograms (FPFH) for 3D registration. In: Proceedings of IEEE International Conference on Robotics and Automation, 2009. 3212–3217
Aoki Y, Goforth H, Srivatsan R A, et al. PointNetLK: robust & efficient point cloud registration using pointnet. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019. 7163–7172
Wang Y, Solomon J M. Deep closest point: learning representations for point cloud registration. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019. 3523–3532
Yew Z J, Lee G H. RPM-Net: robust point matching using learned features. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020. 11824–11833
van der Maaten L, Hinton G. Visualizing data using t-SNE. J Mach Learning Res, 2008, 9: 2579–2605
Censi A. An ICP variant using a point-to-line metric. In: Proceedings of IEEE International Conference on Robotics and Automation, 2008. 19–25
Pomerleau F, Colas F, Siegwart R, et al. Comparing ICP variants on real-world data sets. Auton Robot, 2013, 34: 133–148
Yang J, Li H, Campbell D, et al. Go-ICP: a globally optimal solution to 3D ICP point-set registration. IEEE Trans Pattern Anal Mach Intell, 2015, 38: 2241–2254
Rusu R B, Blodow N, Marton Z C, et al. Aligning point cloud views using persistent feature histograms. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, 2008. 3384–3391
Yang J, Quan S, Wang P, et al. Evaluating local geometric feature representations for 3D rigid data matching. IEEE Trans Image Process, 2019, 29: 2522–2535
Ma J, Wu J, Zhao J, et al. Nonrigid point set registration with robust transformation learning under manifold regularization. IEEE Trans Neural Netw Learn Syst, 2018, 30: 3584–3597
Ma J, Zhao J, Jiang J, et al. Locality preserving matching. Int J Comput Vis, 2019, 127: 512–531
Yang H, Shi J, Carlone L. TEASER: fast and certifiable point cloud registration. IEEE Trans Robot, 2020, 37: 314–333
Myronenko A, Song X B. Point set registration: coherent point drift. IEEE Trans Pattern Anal Mach Intell, 2010, 32: 2262–2275
Jian B, Vemuri B C. Robust point set registration using Gaussian mixture models. IEEE Trans Pattern Anal Mach Intell, 2011, 33: 1633–1645
Li L, Yang M, Wang C, et al. Robust point set registration using signature quadratic form distance. IEEE Trans Cybern, 2018, 50: 2097–2109
Su H, Maji S, Kalogerakis E, et al. Multi-view convolutional neural networks for 3D shape recognition. In: Proceedings of the IEEE International Conference on Computer Vision, 2015. 945–953
Wu B, Wan A, Yue X, et al. SqueezeSeg: convolutional neural nets with recurrent CRF for real-time road-object segmentation from 3D lidar point cloud. In: Proceedings of IEEE International Conference on Robotics and Automation, 2018. 1887–1893
Tchapmi L, Choy C, Armeni I, et al. SEGCloud: semantic segmentation of 3D point clouds. In: Proceedings of International Conference on 3D Vision, 2017. 537–547
Maturana D, Scherer S. VoxNet: a 3D convolutional neural network for real-time object recognition. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, 2015. 922–928
Qi C R, Su H, Mo K, et al. PointNet: deep learning on point sets for 3D classification and segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017. 652–660
Qi C R, Yi L, Su H, et al. PointNet++: deep hierarchical feature learning on point sets in a metric space. In: Proceedings of Advances in Neural Information Processing Systems, 2017. 5099–5108
Hua B S, Tran M K, Yeung S K. Pointwise convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018. 984–993
Lin L Q, Huang P D, Xue F Y, et al. Hausdorff point convolution with geometric priors. Sci China Inf Sci, 2021, 64: 210105
Wang Y, Sun Y, Liu Z, et al. Dynamic graph CNN for learning on point clouds. ACM Trans Graph, 2019, 38: 1–12
Liang Z, Yang M, Deng L, et al. Hierarchical depthwise graph convolutional neural network for 3D semantic segmentation of point clouds. In: Proceedings of International Conference on Robotics and Automation, 2019. 8152–8158
Valsesia D, Fracastoro G, Magli E. Learning localized representations of point clouds with graph-convolutional generative adversarial networks. IEEE Trans Multimedia, 2020, 23: 402–414
Cao W M, Zheng C T, Yan Z Y, et al. Geometric deep learning: progress, applications and challenges. Sci China Inf Sci, 2022, 65: 126101
Poulenard A, Rakotosaona M J, Ponty Y, et al. Effective rotation-invariant point CNN with spherical harmonics kernels. In: Proceedings of International Conference on 3D Vision, 2019. 47–56
Li X, Li R, Chen G, et al. A rotation-invariant framework for deep point cloud analysis. IEEE Trans Visual Comput Graph, 2021. doi: https://doi.org/10.1109/TVCG.2021.3092570
Chen C, Li G, Xu R, et al. ClusterNet: deep hierarchical cluster network with rigorously rotation-invariant representation for point cloud analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019. 4994–5002
You Y, Lou Y, Liu Q, et al. Pointwise rotation-invariant network with adaptive sampling and 3D spherical voxel convolution. In: Proceedings of the AAAI Conference on Artificial Intelligence, 2020. 12717–12724
Deng H, Birdal T, Ilic S. PPFNet: global context aware local features for robust 3D point matching. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018. 195–205
Deng H, Birdal T, Ilic S. PPF-FoldNet: unsupervised learning of rotation invariant 3D local descriptors. In: Proceedings of the European Conference on Computer Vision (ECCV), 2018. 602–618
Ao S, Hu Q, Yang B, et al. SpinNet: learning a general surface descriptor for 3D point cloud registration. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021. 11753–11762
Lu W, Wan G, Zhou Y, et al. DeepVCP: an end-to-end deep neural network for point cloud registration. In: Proceedings of the IEEE International Conference on Computer Vision, 2019. 12–21
Khazari A E, Que Y, Sung T L, et al. Deep global features for point cloud alignment. Sensors, 2020, 20: 4032
Gojcic Z, Zhou C, Wegner J D, et al. Learning multiview 3D point cloud registration. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020. 1759–1769
Drost B, Ulrich M, Navab N, et al. Model globally, match locally: efficient and robust 3D object recognition. In: Proceedings of 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2010. 998–1005
Huang H, Li D, Zhang H, et al. Consolidation of unorganized point clouds for surface reconstruction. ACM Trans Graph, 2009, 28: 1–7
Wu Z, Song S, Khosla A, et al. 3D ShapeNets: a deep representation for volumetric shapes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015. 1912–1920
Turk G, Levoy M. The Stanford 3D Scanning Repository. Stanford University Computer Graphics Laboratory, 2005. http://graphics.stanford.edu/data/3Dscanrep
Zeng A, Song S, Nießner M, et al. 3DMatch: learning local geometric descriptors from RGB-D reconstructions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017. 1802–1811
Acknowledgements
This work was supported by National Natural Science Foundation of China (Grant Nos. 62173228, 61873165).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Zhao, H., Liang, Z., He, Y. et al. Cy-CNN: cylinder convolution based rotation-invariant neural network for point cloud registration. Sci. China Inf. Sci. 66, 152102 (2023). https://doi.org/10.1007/s11432-021-3570-5
Received:
Revised:
Accepted:
Published:
DOI: https://doi.org/10.1007/s11432-021-3570-5