research-article

Rank-PointRetrieval: Reranking Point Cloud Retrieval via a Visually Consistent Registration Evaluation

Authors:

Chunxia XiaoAuthors Info & Claims

IEEE Transactions on Visualization and Computer Graphics, Volume 29, Issue 9

Pages 3840 - 3854

https://doi.org/10.1109/TVCG.2022.3170695

Published: 01 September 2023 Publication History

Abstract

Point cloud-based place recognition is a fundamental part of the localization task, and it can be achieved through a retrieval process. Reranking is a critical step in improving the retrieval accuracy, yet little effort has been devoted to reranking in point cloud retrieval. In this paper, we investigate the versatility of rigid registration in reranking the point cloud retrieval results. Specifically, after obtaining the initial retrieval list based on the global point cloud feature distance, we perform registration between the query and point clouds in the retrieval list. We propose an efficient strategy based on visual consistency to evaluate each registration with a registration score in an unsupervised manner. The final reranked list is computed by considering both the original global feature distance and the registration score. In addition, we find that the registration score between two point clouds can also be used as a pseudo label to judge whether they represent the same place. Thus, we can create a self-supervised training dataset when there is no ground truth of positional information. Moreover, we develop a new probability-based loss to obtain more discriminative descriptors. The proposed reranking approach and the probability-based loss can be easily applied to current point cloud retrieval baselines to improve the retrieval accuracy. Experiments on various benchmark datasets show that both the reranking registration method and probability-based loss can significantly improve the current state-of-the-art baselines.

References

[1]

A. Kendall, M. Grimes, and R. Cipolla, “PoseNet: A convolutional network for real-time 6-DOF camera relocalization,” in Proc. IEEE Int. Conf. Comput. Vis., 2015, pp. 2938–2946.

[2]

R. Arandjelovic, P. Gronat, A. Torii, T. Pajdla, and J. Sivic, “NetVLAD: CNN architecture for weakly supervised place recognition,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2016, pp. 5297–5307.

[3]

Y. Ge, H. Wang, F. Zhu, R. Zhao, and H. Li, “Self-supervising fine-grained region similarities for large-scale image localization,” in Proc. Eur. Conf. Comput. Vis., 2020, pp. 369–386.

[4]

H. J. Kim, E. Dunn, and J.-M. Frahm, “Learned contextual feature reweighting for image geo-localization,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2017, pp. 3251–3260.

[5]

M. Angelina Uy and G. Hee Lee, “PointNetVLAD: Deep point cloud based retrieval for large-scale place recognition,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2018, pp. 4470–4479.

[6]

W. Zhang and C. Xiao, “PCAN: 3D attention map learning using contextual information for point cloud based retrieval,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2019, pp. 12428–12437.

[7]

Z. Liu et al., “LPD-Net: 3D point cloud learning for large-scale place recognition and environment analysis,” in Proc. IEEE Int. Conf. Comput. Vis., 2019, pp. 2831–2840.

[8]

Y. Xia et al., “SOE-Net: A self-attention and orientation encoding network for point cloud based place recognition,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2021, pp. 11 348–11 357.

[9]

J. Komorowski, “MinkLoc3D: Point cloud based large-scale place recognition,” in Proc. IEEE/CVF Winter Conf. Appl. Comput. Vis., 2021, pp. 1789–1798.

[10]

D. F. Huber and M. Hebert, “Fully automatic registration of multiple 3D data sets,” Image Vis. Comput., vol. 21, no. 7, pp. 637–650, 2003.

[11]

C. R. Qi, H. Su, K. Mo, and L. J. Guibas, “PointNet: Deep learning on point sets for 3D classification and segmentation,” Proc. Comput. Vis. Pattern Recognit., 2017, pp. 77–85.

[12]

J. Du, R. Wang, and D. Cremers, “DH3D: Deep hierarchical 3D descriptors for robust large-scale 6DoF relocalization,” in Proc. Eur. Conf. Comput. Vis., 2020, pp. 744–762.

[13]

A. E. Johnson and M. Hebert, “Using spin images for efficient object recognition in cluttered 3D scenes,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 21, no. 5, pp. 433–449, 1999.

Digital Library

[14]

A. Frome, D. Huber, R. Kolluri, T. Bülow, and J. Malik, “Recognizing objects in range data using regional point descriptors,” in Proc. Eur. Conf. Comput. Vis., 2004, pp. 224–237.

[15]

R. B. Rusu, N. Blodow, Z. C. Marton, and M. Beetz, “Aligning point cloud views using persistent feature histograms,” in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst., 2008, pp. 3384–3391.

[16]

R. B. Rusu, N. Blodow, and M. Beetz, “Fast point feature histograms (FPFH) for 3D registration,” in Proc. IEEE Int. Conf. Robot. Automat., 2009, pp. 3212–3217.

[17]

S. Salti, F. Tombari, and L. Di Stefano, “Shot: Unique signatures of histograms for surface and texture description,” Comput. Vis. Image Understanding, vol. 125, pp. 251–264, 2014.

[18]

R. B. Rusu and S. Cousins, “3D is here: Point cloud library (PCL),” in Proc. IEEE Int. Conf. Robot. Automat., 2011, pp. 1–4.

[19]

Q.-Y. Zhou, J. Park, and V. Koltun, “Open3D: A modern library for 3D data processing,” 2018,.

[20]

N. Mellado, M. Dellepiane, and R. Scopigno, “Relative scale estimation and 3D registration of multi-modal geometry using growing least squares,” IEEE Trans. Vis. Comput. Graphics, vol. 22, no. 9, pp. 2160–2173, Sep. 2016.

Digital Library

[21]

G. K. Tam et al., “Registration of 3D point clouds and meshes: A survey from rigid to nonrigid,” IEEE Trans. Vis. Comput. Graphics, vol. 19, no. 7, pp. 1199–1217, Jul. 2013.

Digital Library

[22]

A. Zeng, S. Song, M. Nießner, M. Fisher, J. Xiao, and T. Funkhouser, “3DMatch: Learning local geometric descriptors from RGB-D reconstructions,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2017, pp. 199–208.

[23]

G. Elbaz, T. Avraham, and A. Fischer, “3D point cloud registration for localization using a deep neural network auto-encoder,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2017, pp. 2472–2481.

[24]

M. Khoury, Q.-Y. Zhou, and V. Koltun, “Learning compact geometric features,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2017, pp. 153–161.

[25]

H. Deng, T. Birdal, and S. Ilic, “PPFNet: Global context aware local features for robust 3D point matching,” in Proc. IEEE Comput. Vis. Pattern Recognit., 2018, pp. 195–205.

[26]

H. Deng, T. Birdal, and S. Ilic, “PPF-FoldNet: Unsupervised learning of rotation invariant 3D local descriptors,” in Proc. Eur. Conf. Comput. Vis., 2018, pp. 602–618.

[27]

Z. Jian Yew and G. Hee Lee, “3DFeat-Net: Weakly supervised local 3D features for point cloud registration,” in Proc. Eur. Conf. Comput. Vis., 2018, pp. 607–623.

[28]

R. Spezialetti, S. Salti, and L. D. Stefano, “Learning an effective equivariant 3D descriptor without supervision,” in Proc. IEEE/CVF Int. Conf. Comput. Vis., 2019, pp. 6400–6409.

[29]

Z. Gojcic, C. Zhou, J. D. Wegner, and A. Wieser, “The perfect match: 3D point cloud matching with smoothed densities,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2019, pp. 5540–5549.

[30]

C. Choy, J. Park, and V. Koltun, “Fully convolutional geometric features,” in Proc. IEEE Int. Conf. Comput. Vis., 2019, pp. 8957–8965.

[31]

W. Lu, Y. Zhou, G. Wan, S. Hou, and S. Song, “L3-Net: Towards learning based lidar localization for autonomous driving,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2019, pp. 6382–6391.

[32]

W. Lu, G. Wan, Y. Zhou, X. Fu, P. Yuan, and S. Song, “DeepVCP: An end-to-end deep neural network for point cloud registration,” in Proc. IEEE Int. Conf. Comput. Vis., 2019, pp. 12–21.

[33]

Y. Wang and J. M. Solomon, “Deep closest point: Learning representations for point cloud registration,” in Proc. IEEE Int. Conf. Comput. Vis., 2019, pp. 3522–3531.

[34]

Z. Yan, Z. Yi, R. Hu, N. J. Mitra, D. Cohen-Or, and H. Huang, “Consistent two-flow network for tele-registration of point clouds,” IEEE Trans. Vis. Comput. Graphics, to be published.

Digital Library

[35]

S. Ao, Q. Hu, B. Yang, A. Markham, and Y. Guo, “SpinNet: Learning a general surface descriptor for 3D point cloud registration,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2021, pp. 11753–11762.

[36]

L. Zheng, Y. Yang, and Q. Tian, “SIFT meets CNN: A decade survey of instance retrieval,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 40, no. 5, pp. 1224–1244, May 2018.

[37]

O. Chum, J. Philbin, J. Sivic, M. Isard, and A. Zisserman, “Total recall: Automatic query expansion with a generative feature model for object retrieval,” in Proc. IEEE 11th Int. Conf. Comput. Vis., 2007, pp. 1–8.

[38]

R. Arandjelović and A. Zisserman, “Three things everyone should know to improve object retrieval,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2012, pp. 2911–2918.

[39]

X. Shen, Z. Lin, J. Brandt, S. Avidan, and Y. Wu, “Object retrieval and localization with spatially-constrained similarity measure and k-NN re-ranking,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2012, pp. 3013–3020.

[40]

S. Bai and X. Bai, “Sparse contextual activation for efficient visual re-ranking,” IEEE Trans. Image Process., vol. 25, no. 3, pp. 1056–1069, Mar. 2016.

Digital Library

[41]

Z. Zhong, L. Zheng, D. Cao, and S. Li, “Re-ranking person re-identification with k-reciprocal encoding,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2017, pp. 3652–3661.

[42]

J. Garcia, N. Martinel, A. Gardel, I. Bravo, G. L. Foresti, and C. Micheloni, “Discriminant context information analysis for post-ranking person re-identification,” IEEE Trans. Image Process., vol. 26, no. 4, pp. 1650–1665, Apr. 2017.

Digital Library

[43]

M. Ye, C. Liang, Z. Wang, Q. Leng, and J. Chen, “Ranking optimization for person re-identification via similarity and dissimilarity,” in Proc. 23rd ACM Int. Conf. Multimedia, 2015, pp. 1239–1242.

[44]

Y. Aoki, H. Goforth, R. A. Srivatsan, and S. Lucey, “PointNetLK: Robust & efficient point cloud registration using pointnet,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2019, pp. 7156–7165.

[45]

C. Choy, W. Dong, and V. Koltun, “Deep global registration,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2020, pp. 2511–2520.

[46]

Q.-Y. Zhou, J. Park, and V. Koltun, “Fast global registration,” in Proc. Eur. Conf. Comput. Vis., 2016, pp. 766–782.

[47]

R. Raguram, J.-M. Frahm, and M. Pollefeys, “A comparative analysis of ransac techniques leading to adaptive real-time random sample consensus,” in Proc. Eur. Conf. Comput. Vis., 2008, pp. 500–513.

[48]

Y. Ma, Y. Guo, J. Zhao, M. Lu, J. Zhang, and J. Wan, “Fast and accurate registration of structured point clouds with small overlaps,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. Workshops, 2016, pp. 643–651.

[49]

J. D. Foley et al., Computer Graphics: Principles and Practice. Reading, MA, USA: Addison-Wesley, 1996.

Digital Library

[50]

H. Houshiar, D. Borrmann, J. Elseberg, and A. Nüchter, “Panorama based point cloud reduction and registration,” in Proc. 16th Int. Conf. Adv. Robot., 2013, pp. 1–8.

[51]

Y. Eldar, M. Lindenbaum, M. Porat, and Y. Y. Zeevi, “The farthest point strategy for progressive image sampling,” IEEE Trans. Image Process., vol. 6, no. 9, pp. 1305–1315, Sep. 1997.

Digital Library

[52]

Y. Zhong, “Intrinsic shape signatures: A shape descriptor for 3D object recognition,” in Proc. IEEE 12th Int. Conf. Comput. Vis. Workshops, 2009, pp. 689–696.

[53]

G. E. Hinton and S. Roweis, “Stochastic neighbor embedding,” Adv. Neural Inf. Process. Syst., vol. 15, pp. 857–864, 2002.

[54]

L. Liu, H. Li, and Y. Dai, “Stochastic attraction-repulsion embedding for large scale image localization,” in Proc. IEEE Int. Conf. Comput. Vis., 2019, pp. 2570–2579.

[55]

W. Maddern, G. Pascoe, C. Linegar, and P. Newman, “1 year, 1000 km: The oxford robotcar dataset,” Int. J. Robot. Res., vol. 36, no. 1, pp. 3–15, 2017.

Digital Library

Cited By

Zhang RLi GGao WLi T(2024)ComPoint: Can Complex-Valued Representation Benefit Point Cloud Place Recognition?IEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2024.335121525:7(7494-7507)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1109/TITS.2024.3351215
Yin PZhao SLai HGe RZhang JChoset HScherer S(2023)AutoMerge: A Framework for Map Assembling and Smoothing in City-Scale EnvironmentsIEEE Transactions on Robotics10.1109/TRO.2023.329044839:5(3686-3704)Online publication date: 1-Oct-2023
https://dl.acm.org/doi/10.1109/TRO.2023.3290448

Recommendations

Web image retrieval reranking with multi-view clustering
WWW '09: Proceedings of the 18th international conference on World wide web

General image retrieval is often carried out by a text-based search engine, such as Google Image Search. In this case, natural language queries are used as input to the search engine. Usually, the user queries are quite ambiguous and the returned ...
Registration of Structured Light Camera Point Cloud Data with CT Images
Intelligent Robotics and Applications
Abstract
With the advancement of structured-light cameras, surgical robots equipped with such cameras have been utilized for lesion localization during surgeries. Achieving precise registration between CT images and point cloud data remains a challenge. ...
Rank canonical correlation analysis and its application in visual search reranking

Ranking relevance degree information is widely utilized in the ranking models of information retrieval applications, such as text and multimedia retrieval, question answering, and visual search reranking. However, existing feature dimensionality ...

Comments

Information & Contributors

Information

Published In

cover image IEEE Transactions on Visualization and Computer Graphics

IEEE Transactions on Visualization and Computer Graphics Volume 29, Issue 9

Sept. 2023

297 pages

ISSN:1077-2626

Issue’s Table of Contents

1077-2626 © 2022 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See https://www.ieee.org/publications/rights/index.html for more information.

Publisher

IEEE Educational Activities Department

United States

Publication History

Published: 01 September 2023

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 13 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zhang RLi GGao WLi T(2024)ComPoint: Can Complex-Valued Representation Benefit Point Cloud Place Recognition?IEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2024.335121525:7(7494-7507)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1109/TITS.2024.3351215
Yin PZhao SLai HGe RZhang JChoset HScherer S(2023)AutoMerge: A Framework for Map Assembling and Smoothing in City-Scale EnvironmentsIEEE Transactions on Robotics10.1109/TRO.2023.329044839:5(3686-3704)Online publication date: 1-Oct-2023
https://dl.acm.org/doi/10.1109/TRO.2023.3290448

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents