Coarse-to-Fine Visual Place Recognition
Authors: 
Junkun Qi, Rui Wang, Chuan Wang, Xiaochun CaoAuthors Info & Claims
Junkun Qi, Rui Wang, Chuan Wang, Xiaochun CaoAuthors Info & ClaimsPages 28 - 39
Abstract
Visual Place Recognition (VPR) aims to locate one or more images depicting the same place in the geotagged database with a given query and is typically conducted as an image retrieval task. Currently, global-based and local-based descriptors are two mainstream representations to solve VPR. However, they still struggle against viewpoint change, confusion from similar patterns in different places, or high computation complexity. In this paper, we propose a progressive Coarse-To-Fine (CTF-VPR) framework, which has a strong ability on handling irrelevant matches and controlling time consumption. It employs global descriptors to discover visually similar references and local descriptors to filter those with similar but irrelative patterns. Besides, a region-specific representing format called regional descriptor is introduced with region augmentation and increases the possibilities of positive references with partially relevant areas via region refinement. Furthermore, during the spatial verification, we provide the Spatial Deviation Index (SDI) considering coordinate deviation to evaluate the consistency of matches. It discards exhaustive and iterative search and reduces the time consumption hundreds of times. The proposed CTF-VPR outperforms existing approaches by 2%–3% recalls on Pitts250k and Tokyo24/7 benchmarks.
References
[1]
Arandjelovic, R., Gronat, P., Torii, A., Pajdla, T., Sivic, J.: Netvlad: CNN architecture for weakly supervised place recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5297–5307 (2016)
[2]
Arandjelovic, R., Zisserman, A.: All about VLAD. In: Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp. 1578–1585 (2013)
[3]
Bay H, Ess A, Tuytelaars T, and Van Gool L Speeded-up robust features (surf) Comput. Vis. Image Underst. 2008 110 3 346-359
[4]
Cao, B., Araujo, A., Sim, J.: Unifying deep local and global features for image search. In: European Conference on Computer Vision (2020)
[5]
Castle, R., Klein, G., Murray, D.W.: Video-rate localization in multiple maps for wearable augmented reality. In: 2008 12th IEEE International Symposium on Wearable Computers, pp. 15–22. IEEE (2008)
[6]
Csurka, G., Dance, C., Fan, L., Willamowski, J., Bray, C.: Visual categorization with bags of keypoints. In: Workshop on Statistical Learning in Computer Vision, ECCV, pp. 1–22, Prague (2004)
[7]
DeTone, D., Malisiewicz, T., Rabinovich, A.: Superpoint: self-supervised interest point detection and description. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 337–33712 (2018)
[8]
Doan, A.D., Latif, Y., Chin, T.J., Liu, Y., Do, T.T., Reid, I.: Scalable place recognition under appearance change for autonomous driving. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (2019)
[9]
Dusmanu, M., et al.: D2-net: A trainable CNN for joint description and detection of local features. In: 2019 IEEE Conference on Computer Vision and Pattern Recognition (2019)
[10]
Fischler MA and Bolles RC Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography Commun. ACM 1981 24 6 381-395
[11]
Ge, Y., Wang, H., Zhu, F., Zhao, R., Li, H.: Self-supervising fine-grained region similarities for large-scale image localization. In: European Conference on Computer Vision (2020)
[12]
Häne C, Heng L, Lee GH, Fraundorfer F, Furgale P, Sattler T, and Pollefeys M 3d visual perception for self-driving cars using a multi-camera system: calibration, mapping, localization, and obstacle detection Image Vis. Comput. 2017 68 14-27
[13]
Hausler, S., Garg, S., Xu, M., Milford, M., Fischer, T.: Patch-NetVLAD: multi-scale fusion of locally-global descriptors for place recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2021)
[14]
Irschara, A., Zach, C., Frahm, J.M., Bischof, H.: From structure-from-motion point clouds to fast location recognition. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 2599–2606. IEEE (2009)
[15]
Jégou, H., Douze, M., Schmid, C., Pérez, P.: Aggregating local descriptors into a compact image representation. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 3304–3311 (2010)
[16]
Kim, H.J., Dunn, E., Frahm, J.M.: Learned contextual feature reweighting for image geo-localization. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition, pp. 3251–3260 (2017)
[17]
Liu, L., Li, H., Dai, Y.: Stochastic attraction-repulsion embedding for large scale image localization. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2570–2579 (2019)
[18]
Lowe DG Distinctive image features from scale-invariant keypoints Int. J. Comput. Vis. 2004 60 2 91-110
[19]
Lowry S, Sünderhauf N, Newman P, Leonard JJ, Cox D, Corke P, and Milford MJ Visual place recognition: a survey IEEE Trans. Rob. 2016 32 1 1-19
[20]
Middelberg S, Sattler T, Untzelmann O, and Kobbelt L Fleet D, Pajdla T, Schiele B, and Tuytelaars T Scalable 6-DOF localization on mobile devices Computer Vision – ECCV 2014 2014 Cham Springer 268-283
[21]
Mur-Artal R, Montiel JMM, and Tardos JD ORB-SLAM: a versatile and accurate monocular slam system IEEE Trans. Rob. 2015 31 5 1147-1163
[22]
Noh, H., Araujo, A., Sim, J., Weyand, T., Han, B.: Large-scale image retrieval with attentive deep local features. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3456–3465 (2017)
[23]
Perronnin, F., Liu, Y., Sánchez, J., Poirier, H.: Large-scale image retrieval with compressed fisher vectors. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 3384–3391. IEEE (2010)
[24]
Philbin, J., Chum, O., Isard, M., Sivic, J., Zisserman, A.: Object retrieval with large vocabularies and fast spatial matching. In: 2007 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8. IEEE (2007)
[25]
Radenović F, Tolias G, and Chum O Fine-tuning CNN image retrieval with no human annotation IEEE Trans. Pattern Anal. Mach. Intell. 2018 41 7 1655-1668
[26]
Revaud, J., Almazán, J., Rezende, R.S., Souza, C.R.: Learning with average precision: training image retrieval with a listwise loss. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 5107–5116 (2019)
[27]
Sarlin, P.E., DeTone, D., Malisiewicz, T., Rabinovich, A.: SuperGlue: learning feature matching with graph neural networks. In: 2020 IEEE Conference on Computer Vision and Pattern Recognition (2020)
[28]
Sattler, T., Leibe, B., Kobbelt, L.: Fast image-based localization using direct 2D-to-3D matching. In: 2011 International Conference on Computer Vision, pp. 667–674. IEEE (2011)
[29]
Sattler, T., Weyand, T., Leibe, B., Kobbelt, L.: Image retrieval for image-based localization revisited. In: 2012 British Machine Vision Conference, p. 4 (2012)
[30]
Schonberger, J.L., Frahm, J.M.: Structure-from-motion revisited. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4104–4113 (2016)
[31]
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: International Conference on Learning Representations (2015)
[32]
Torii, A., Arandjelović, R., Sivic, J., Okutomi, M., Pajdla, T.: 24/7 place recognition by view synthesis. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1808–1817 (2015)
[33]
Torii A, Sivic J, Okutomi M, and Pajdla T Visual place recognition with repetitive structures IEEE Trans. Pattern Anal. Mach. Intell. 2015 37 11 2346-2359
[34]
Yi KM, Trulls E, Lepetit V, and Fua P Leibe B, Matas J, Sebe N, and Welling M LIFT: learned invariant feature transform Computer Vision – ECCV 2016 2016 Cham Springer 467-483
Index Terms
- Coarse-to-Fine Visual Place Recognition
Index terms have been assigned to the content through auto-classification.
Recommendations
Regional Relation Modeling for Visual Place Recognition
SIGIR '20: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information RetrievalIn the process of visual perception, humans perceive not only the appearance of objects existing in a place but also their relationships (e.g. spatial layout). However, the dominant works on visual place recognition are always based on the assumption ...
An extended-HCT semantic description for visual place recognition
We describe a new semantic descriptor for robots to recognize visual places. The descriptor integrates image features and color information via the hull census transform (HCT) and image histogram indexing. Our approach extracts the semantic description ...
Robust Place Recognition with Combined Image Descriptors
MESAS 2016: Proceedings of the Third International Workshop on Modelling and Simulation for Autonomous Systems - Volume 9991In this paper, a method of place recognition is presented. The method is generally classified under the bag-of-visual-words approach. Information from several global image descriptors is incorporated. The data fusion is performed at the feature level.
...
Comments
Information & Contributors
Information
Published In
Dec 2021
717 pages
ISBN:978-3-030-92272-6
DOI:10.1007/978-3-030-92273-3
© Springer Nature Switzerland AG 2021.
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 08 December 2021
Author Tags
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 28 Jan 2025