Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
SpringerLink
Log in

A Robust Feature Matching Strategy for Fast and Effective Visual Place Recognition in Challenging Environmental Conditions

  • Regular Papers
  • Robot and Applications
  • Published:
International Journal of Control, Automation and Systems Aims and scope Submit manuscript

An Erratum to this article was published on 02 May 2023

This article has been updated

Abstract

A robust visual place recognition (VPR) is one of the significant requirements for visual simultaneous localization and mapping (SLAM). Many researchers have put forth their efforts for developing a robust VPR system that is invariant to changing environmental conditions while ensuring computational and memory efficiency. However, the development of such a system is still a challenge. Vision-based place recognition is highly dependent on the feature extraction and matching algorithms. Therefore, this research primarily focuses the process of robust feature selection and matching for accurate place recognition in challenging environments where appearance of a place drastically changes due to conditional variations, and highlights the limitations of the existing research. In this paper, a fast and affective visual place recognition method is presented that integrates the Bag-of-Words (BoW) vocabulary with robust feature matcher. The BoW reduces the search space for the image matching process by generating the match candidates while the robust feature matcher enhances the place matching performance by identifying and removing the spatially inconsistent feature matches on image plane between current frame and candidate image. The proposed method is evaluated on range of challenging benchmark datasets against state-of-the-art VPR methods. Through experiments the effectiveness of the proposed method is presented. Furthermore, this research elucidates the dilemma: which feature extraction algorithm performs better in a specific type of environment through evaluation of different feature detector-descriptor combinations on publicly available datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Change history

References

  1. C. Cadena, “Past, present, and future of simultaneous localization and mapping: Toward the robust-perception age,” IEEE Transactions on Robotics, vol. 32, no. 6, pp. 1309–1332, December 2016.

    Article  Google Scholar 

  2. N. Sünderhauf and P. Protzel, “Are we there yet? challenging SeqSLAM on a 3000 km journey across all four sea-sons,” Proc. of International Conference on Robotics and Automation (ICRA), p. 3, 2013.

  3. N. Suenderhauf et al., “Place recognition with ConvNet landmarks: Viewpoint-robust, condition-robust, training-free,” Robot. Sci. Syst. XI, pp. 1–10, 2015.

  4. C. McManus, B. Upcroft, and P. Newmann, “Scene signatures: Localised and point-less features for localisation,” Robot. Sci. Syst. X, 2014.

  5. K. Mikolajczyk and C. Schmid, “A performance evaluation of local descriptors,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 27, no. 10, pp. 1615–1630, October 2005.

    Article  Google Scholar 

  6. S. Gauglitz, T. Höllerer, and M. Turk, “Evaluation of interest point detectors and feature descriptors for visual tracking,” International Journal of Computer Vision, vol. 94, pp. 335–360, 2011.

    Article  MATH  Google Scholar 

  7. S. Urban, M. Weinmann, S. Urban, and M. Weinmann, “Finding a good feature detector-descriptor combination for the 2D keypoint-based registration of TLS point clouds,” ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, vol. II-3/W5, pp. 121–128, 2015.

    Article  Google Scholar 

  8. Z. Pusztai, “Quantitative comparison of feature matchers implemented in OpenCV3,” 2016.

  9. H. J. Chien, C. C. Chuang, C. Y. Chen, and R. Klette, “When to use what feature? SIFT, SURF, ORB, or A-KAZE features for monocular visual odometry,” Proc. of International Conference Image and Vision Computing, New Zealand, pp. 1–6, July 2016.

  10. S. A. K. Tareen and Z. Saleem, “A comparative analysis of SIFT, SURF, KAZE, AKAZE, ORB, and BRISK,” Proc. of International Conference on Computing, Mathematics and Engineering Technologies (iCoMET), pp. 1–10, April 2018.

  11. M. Zaffar, A. Khaliq, S. Ehsan, M. Milford, and K. McDonald-Maier, “Levelling the playing field: A comprehensive comparison of visual place recognition approaches under changing conditions,” arXiv Prepr. arXiv1903.09107, March 2019. DOI: https://doi.org/10.48550/arXiv.1903.09107

  12. C. Park, H. W. Chae, and J. B. Song, “Robust place recognition using illumination-compensated image-based deep convolutional autoencoder features,” International Journal of Control, Automation, and Systems, vol. 18, pp. 2699–2707, June 2020.

    Article  Google Scholar 

  13. T. Naseer, L. Spinello, W. Burgard, and C. Stachniss, “Robust visual robot localization across seasons using network flows,” Proc. of the Twenty-Eighth AAAI Conference on Artificial Intelligence, pp. 2564–2570, 2014.

  14. C. Valgren and A. J. Lilienthal, “SIFT, SURF & seasons: Appearance-based long-term localization in outdoor environments,” Rob. Auton. Syst., vol. 58, no. 2, pp. 149–156, February 2010.

    Article  Google Scholar 

  15. P. De Cristóforis, M. Nitsche, T. Krajník, T. Pire, and M. Mejail, “Hybrid vision-based navigation for mobile robots in mixed indoor/outdoor environments,” Pattern Recognit. Lett., vol. 53, pp. 118–128, 2015.

    Article  Google Scholar 

  16. P. Neubert and P. Protzel, “Beyond holistic descriptors, keypoints, and fixed patches: Multiscale superpixel grids for place recognition in changing environments,” IEEE Robotics and Automation Letters, vol. 1, no. 1, pp. 484–491, January 2016.

    Article  Google Scholar 

  17. D. Galvez-Lopez and J. D. Tardos, “Real-time loop detection with bags of binary words,” Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 51–58, September 2011.

  18. D. Gálvez-López and J. D. Tardos, “Bags of binary words for fast place recognition in image sequences,” IEEE Transactions on Robotics, vol. 28, no. 5, pp. 1188–1197, 2012.

    Article  Google Scholar 

  19. C. McManus, P. Furgale, and T. D. Barfoot, “Towards lighting-invariant visual navigation: An appearance-based approach using scanning laser-rangefinders,” Rob. Auton. Syst., vol. 61, no. 8, pp. 836–852, August 2013.

    Article  Google Scholar 

  20. S. Khan and D. Wollherr, “IBuILD: Incremental bag of Binary words for appearance based loop closure detection,” Proc. of IEEE International Conference on Robotics and Automation (ICRA), pp. 5441–5447, June 2015.

  21. R. Mur-Artal and J. D. Tardós, “Fast relocalisation and loop closing in keyframe-based SLAM,” Proc. of IEEE International Conference on Robotics and Automation, pp. 846–853, September 2014.

  22. N. Kejriwal, S. Kumar, and T. Shibata, “High performance loop closure detection using bag of word pairs,” Rob. Auton. Syst., vol. 77, pp. 55–65, March 2016.

    Article  Google Scholar 

  23. S. Lowry and H. Andreasson, “Lightweight, viewpoint-invariant visual place recognition in changing environments,” IEEE Robotics and Automation Letters, vol. 3, no. 2, pp. 957–964, 2018.

    Article  Google Scholar 

  24. T. Krajník, P. Cristóforis, K. Kusumam, P. Neubert, and T. Duckett, “Image features for visual teach-and-repeat navigation in changing environments,” Rob. Auton. Syst., vol. 88, pp. 127–141, February 2017.

    Article  Google Scholar 

  25. W. Maddern and S. Vidas, “Towards robust night and day place recognition using visible and thermal imaging,” Proc. of the RSS 2012 Workshop: Beyond Laser and Vision: Alternative Sensing Techniques for Robotic Perception. pp. 1–6, 2012.

  26. P. Ross, A. English, D. Ball, B. Upcroft, G. Wyeth, and P. Corke, “A novel method for analysing lighting variance,” Proc. of Australian Conference on Robotics and Automation, 2013.

  27. P. Ross, A. English, D. Ball, and P. Corke, “A method to quantify a descriptor’s illumination variance,” Proc. of the 16th Australasian Conference on Robotics and Automation 2014, pp. 1–8, 2014.

  28. D. Schlegel and G. Grisetti, “HBST: A hamming distance embedding binary search tree for feature-based visual place recognition,” IEEE Robotics and Automation Letters, vol. 3, no. 4, pp. 3741–3748, October 2018.

    Article  Google Scholar 

  29. K. A. Tsintotas, L. Bampis, and A. Gasteratos, “Probabilistic appearance-based place recognition through bag of tracked words,” IEEE Robotics and Automation Letters, vol. 4, no. 2, pp. 1737–1744, 2019.

    Article  Google Scholar 

  30. T. Ying, H. Yan, Z. Li, K. Shi, and X. Feng, “Loop closure detection based on image covariance matrix matching for visual SLAM,” International Journal of Control, Automation, and Systems, vol. 19, pp. 3708–3719, September 2021.

    Article  Google Scholar 

  31. S. J. Lee and S. S. Hwang, “Bag of sampled words: A sampling-based strategy for fast and accurate visual place recognition in changing environments,” International Journal of Control, Automation, and Systems, vol. 17, pp. 2597–2609, July 2019.

    Article  Google Scholar 

  32. R. Arandjelovi, P. Gronat Inria, and J. Sivic Inria, “NetVLAD: CNN architecture for weakly supervised place recognition,” Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5297–5307, 2016.

  33. Z. Chen, A. Jacobson, N. Sünderhauf, B. Upcroft, L. Liu, C. Shen, I. Reid, and M. Milford, “Deep learning features at scale for visual place recognition,” Proc. of IEEE International Conference on Robotics and Automation, pp. 3223–3230, July 2017.

  34. Z. Chen, O. Lam, A. Jacobson, and M. Milford, “Convo-lutional neural network-based place recognition,” Proc. of Australas. Conf. Robot. Autom., 2014.

  35. S. Hausler, A. Jacobson, and M. Milford, “Feature map filtering: Improving visual place recognition with convolutional calibration,” arXiv, October 2018. DOI: https://doi.org/10.48550/arXiv.1810.12465

  36. S. Hausler, A. Jacobson, and M. Milford, “Filter early, match late: Improving network-based visual place recognition,” Proc. of IEEE International Workshop on Intelligent Robots and Systems, pp. 3268–3275, June 2019.

  37. J. Zhu, Y. Ai, B. Tian, D. Cao, and S. Scherer, “Visual place recognition in long-term and large-scale environment based on CNN Feature,” Proc. of IEEE Intelligent Vehicles Symposium (IV), pp. 1679–1685, October 2018.

  38. J. M. Facil, D. Olid, L. Montesano, and J. Civera, “Condition-invariant multi-view place recognition,” arXiv, February 2019. DOI: https://doi.org/10.48550/arXiv.1902.09516

  39. S. Garg, A. Jacobson, S. Kumar, and M. Milford, “Improving condition- and environment-invariant place recognition with semantic place categorization,” Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 6863–6870, December 2017.

  40. S. Garg, M. Babu, V. T. Dharmasiri, S. Hausler, N. Suen-derhauf, S. Kumar, T. Drummond, and M. Milford, “Look no deeper: Recognizing places from opposing viewpoints under varying scene appearance using single-view depth estimation,” Proc. of International Conference on Robotics and Automation (ICRA), pp. 4916–4923, May 2019.

  41. S. Garg, N. Suenderhauf, and M. Milford, “Don’t look back: Robustifying place categorization for viewpoint- and condition-invariant place recognition,” Proc. of IEEE International Conference on Robotics and Automation (ICRA), pp. 3645–3652, September 2018.

  42. Z. Xin, Y. Cai, T. Lu, X. Xing, S. Cai, J. Zhang, Y. Yang, and Y. Wang, “Localizing discriminative visual landmarks for place recognition,” Proc. of IEEE International Conference on Robotics and Automation, vol. 2019, pp. 5979–5985, 2019.

    Google Scholar 

  43. S. Garg, N. Sunderhauf, M. Milford, and N. Suenderhauf, “LoST? Appearance-invariant place recognition for opposite viewpoints using visual semantics,” arXiv: 1804.05526 2018. DOI: https://doi.org/10.48550/arXiv.1804.05526

  44. N. Sünderhauf, S. Shirazi, F. Dayoub, B. Upcroft, and M. Milford, “On the performance of ConvNet features for place recognition,” Proc. of IEEE/RSJ international conference on intelligent robots and systems (IROS), pp. 4297–4304, December 2015.

  45. A. Khaliq, S. Ehsan, Z. Chen, M. Milford, and K. McDonald-Maier, “A Holistic visual place recognition approach using lightweight CNNs for significant viewPoint and appearance changes,” IEEE Transactions on Robotics, vol. 36, no. 2, pp. 561–569, April 2020.

    Article  Google Scholar 

  46. Z. Chen, F. Maffra, I. Sa, and M. Chli, “Only look once, mining distinctive landmarks from ConvNet for visual place recognition,” Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 9–16, December 2017.

  47. Z. Chen, L. Liu, I. Sa, Z. Ge, and M. Chli, “Learning context flexible attention model for long-term visual place recognition,” IEEE Robotics and Automation Letters, vol. 3, no. 4, pp. 4015–4022, October 2018.

    Article  Google Scholar 

  48. A. Mousavian and J. Kosecka, “Semantic image based geolocation given a map,” arXiv:1609.00278, September 2016. DOI: https://doi.org/10.48550/arXiv.1609.00278

  49. Y. Hou, H. Zhang, S. Zhou, and H. Zou, “Use of roadway scene semantic information and geometry-preserving landmark pairs to improve visual place recognition in changing environments,” IEEE Access, vol. 5, pp. 7702–7713, 2017.

    Article  Google Scholar 

  50. T. Naseer, G. L. Oliveira, T. Brox, and W. Burgard, “Semantics-aware visual localization under challenging perceptual conditions,” Proc. of IEEE International Conference on Robotics and Automation (ICRA), pp. 2614–2620, July 2017.

  51. “Nordlandsbanen: minute by minute, season by season,” https://nrkbeta.no/2013/01/15/nordlandsbanen-minute-by-minute-season-by-season/

  52. Z. Chen, O. Lam, A. Jacobson, and M. Milford, “Convolutional neural network-based place recognition,” Conf. Robot. Autom. ACRA, vol. 02–04-December-2014, Nov. 2014.

  53. W. Maddern, G. Pascoe, C. Linegar, and P. Newman, “1 year, 1000 km: The Oxford RobotCar dataset” The International Journal of Robotics Research, vol. 36, no. 1, pp. 3–15, January 2017.

    Article  Google Scholar 

  54. M. Cummins and P. Newman, “Appearance-only SLAM at large scale with FAB-MAP 2.0,” The International Journal of Robotics Research, vol. 30, no. 9, pp. 1100–1123, November 2011.

    Article  Google Scholar 

  55. M. J. Milford and G. F. Wyeth, “SeqSLAM: Visual route-based navigation for sunny summer days and stormy winter nights,” Proc. of IEEE International Conference on Robotics and Automation, pp. 1643–1649, 2012.

  56. M. Zaffar, S. Ehsan, M. Milford, and K. McDonald-Maier, “CoHOG: A light-weight, compute-efficient, and trainingfree visual place recognition technique for changing environments,” IEEE Robotics and Automation Letters, vol. 5, no. 2, pp. 1835–1842, April 2020.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Electrical and Computer Engineering, Chungbuk National University, Chungde-ro 1, Seowon-Gu, Cheongju, Chungbuk, 28644, Korea

    Saba Arshad & Gon-Woo Kim

Authors
  1. Saba Arshad
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gon-Woo Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gon-Woo Kim.

Additional information

Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This research was supported in part by the MSIT (Ministry of Science and ICT), Korea, under the Grand Information Technology Research Center support program (IITP-2023-2020-0-01462) supervised by the IITP (Institute for Information & communications Technology Planning & Evaluation), and in part by Korea Institute of Planning and Evaluation for Technology in Food, Agriculture, Forestry (IPET) through Open Field Smart Agriculture Technology Short-term Advancement Program, funded by Ministry of Agriculture, Food and Rural Affairs (MAFRA)(No.322039-3).

Saba Arshad received her B.S. Hons degree in computer science from Pir Mehr Ali Shah Arid Agriculture University, Rawalpindi, Pakistan, in 2015 and an M.S. degree in computer science from the COMSATS University Islamabad, Islamabad, Pakistan, in 2017. She is currently perusing a Ph.D. degree in control and robot engineering at Chungbuk National University, Cheongju, Korea. Her research interests include visual simultaneous localization and mapping, long term visual place recognition, loop closure detection, scene recognition, HD mapping for autonomous vehicle navigation, and deep learning.

Gon-Woo Kim received his M.S. and Ph.D. degrees from Seoul National University, Korea, in 2002 and 2006, respectively. He is currently a Professor in the Department of Intelligent Systems and Robotics at Chungbuk National University, Korea. His research interests include navigation, localization and SLAM for mobile robots and autonomous vehicles.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Arshad, S., Kim, GW. A Robust Feature Matching Strategy for Fast and Effective Visual Place Recognition in Challenging Environmental Conditions. Int. J. Control Autom. Syst. 21, 948–962 (2023). https://doi.org/10.1007/s12555-021-0927-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12555-021-0927-x

Keywords

Search

Navigation