article

Editors Choice Article: Visual SLAM: Why filter?

Authors:

Hauke Strasdat,

J. M. M. Montiel,

Andrew J. DavisonAuthors Info & Claims

Image and Vision Computing, Volume 30, Issue 2

Pages 65 - 77

https://doi.org/10.1016/j.imavis.2012.02.009

Published: 01 February 2012 Publication History

Abstract

While the most accurate solution to off-line structure from motion (SFM) problems is undoubtedly to extract as much correspondence information as possible and perform batch optimisation, sequential methods suitable for live video streams must approximate this to fit within fixed computational bounds. Two quite different approaches to real-time SFM - also called visual SLAM (simultaneous localisation and mapping) - have proven successful, but they sparsify the problem in different ways. Filtering methods marginalise out past poses and summarise the information gained over time with a probability distribution. Keyframe methods retain the optimisation approach of global bundle adjustment, but computationally must select only a small number of past frames to process. In this paper we perform a rigorous analysis of the relative advantages of filtering and sparse bundle adjustment for sequential visual SLAM. In a series of Monte Carlo experiments we investigate the accuracy and cost of visual SLAM. We measure accuracy in terms of entropy reduction as well as root mean square error (RMSE), and analyse the efficiency of bundle adjustment versus filtering using combined cost/accuracy measures. In our analysis, we consider both SLAM using a stereo rig and monocular SLAM as well as various different scenes and motion patterns. For all these scenarios, we conclude that keyframe bundle adjustment outperforms filtering, since it gives the most accuracy per unit of computing time.

References

[1]

Recursive estimation of motion, structure, and focal length. IEEE Trans. Pattern Anal. Mach. Intell. v17 i6. 562-575.

[2]

Bell, B.M. and Cathey, F.W., The iterated Kalman filter update as a Gauss-Newton method. IEEE Trans. Autom. Control. v38 i2. 294-297.

[3]

Chiuso, A., Favaro, P., Jin, H. and Soatto, S., Structure from motion causally integrated over time. IEEE Trans. Pattern Anal. Mach. Intell. v24 i4. 523-535.

[4]

Chli, M. and Davison, A.J., Active matching for visual tracking. Robot. Auton. Syst. v57 i12. 1173-1187.

[5]

Inverse depth parametrization for monocular SLAM. IEEE Trans. Rob. v24 i5. 932-945.

[6]

Civera, J., Grasa, O., Davison, A.J. and Montiel, J.M.M., 1-point RANSAC for EKF filtering. Application to real-time structure from motion and visual odometry. J. Field Rob. v27 i5. 609-631.

[7]

Clemente, L.A., Davison, A.J., Reid, I., Neira, J. and Tardós, J.D., Mapping large loops with a single hand-held camera. In: Proceedings of Robotics: Science and Systems (RSS),

[8]

Cummins, M. and Newman, P., Highly scalable appearance-only SLAM - FAB-MAP 2.0. In: Proceedings of Robotics: Science and Systems (RSS),

[9]

Davison, A.J., Real-time simultaneous localisation and mapping with a single camera. In: Proceedings of the International Conference on Computer Vision (ICCV),

[10]

Davison, A.J., Active search for real-time vision. In: Proceedings of the International Conference on Computer Vision (ICCV),

Digital Library

[11]

Davison, A.J., Molton, N.D., Reid, I. and Stasse, O., MonoSLAM: real-time single camera SLAM. IEEE Trans. Pattern Anal. Mach. Intell. v29 i6. 1052-1067.

[12]

Experimental comparison of techniques for localization and mapping using a bearing-only senser. In: Experimental Robotics VII, pp. 395-404.

[13]

Dellaert, F., Square root SAM. In: Proceedings of Robotics: Science and Systems (RSS),

[14]

Dyer, P. and McReynolds, S., Extension of square-root filtering to include process noise. J. Optim. Theory Appl. v3 i6. 444-458.

[15]

E. Eade. Monocular Simultaneous Localisation and Mapping. PhD thesis, University of Cambridge, 2008.

[16]

Eade, E. and Drummond, T., Scalable monocular SLAM. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR),

Digital Library

[17]

Eade, E. and Drummond, T., Monocular SLAM as a graph of coalesced observations. In: Proceedings of the International Conference on Computer Vision (ICCV),

[18]

Engels, C., Stewénius, H. and Nistér, D., Bundle adjustment rules. In: Proceedings of Photogrammetric Computer Vision,

[19]

Eustice, R.M., Singh, H. and Leonard, J.J., Exactly sparse delayed state filters. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA),

[20]

Gallier, J., Geometric Methods and Applications for Computer Science and Engineering. 2001. Springer-Verlag.

[21]

Harris, C.G. and Pike, J.M., 3D positional integration from image sequences. In: Proceedings of the Alvey Vision Conference, pp. 233-236.

[22]

Hartley, R. and Zisserman, A., Multiple View Geometry in Computer Vision. 2004. second edition. Cambridge University Press.

[23]

Jeong, Y., Nister, D., Steedly, D., Szeliski, R. and Kweon, I.S., Pushing the envelope of modern methods for bundle adjustment. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1474-1481.

[24]

Julier, S.J. and Jeffrey Uhlmann, K., A counter example to the theory of simultaneous localization and map building. In: IEEE International Conference on Robotics and Automation,

[25]

M. Kaess, H. Johannsson, R. Roberts, V. Ila, J. Leonard, and F. Dellaert. iSAM2: Incremental smoothing and mapping using the Bayes tree. Int. J. Rob. Res. (IJRR), 2012. To appear.

[26]

Kaess, M., Ranganathan, A. and Dellaert, F., iSAM: incremental smoothing and mapping. IEEE Trans. Rob. v24 i6. 1365-1378.

[27]

Klein, G. and Murray, D.W., Parallel tracking and mapping for small AR workspaces. In: Proceedings of the International Symposium on Mixed and Augmented Reality (ISMAR),

Digital Library

[28]

Klein, G. and Murray, D.W., Parallel tracking and mapping on a camera phone. In: Proceedings of the International Symposium on Mixed and Augmented Reality (ISMAR),

Digital Library

[29]

FrameSLAM: from bundle adjustment to real-time visual mapping. IEEE Trans. Rob. v24. 1066-1077.

[30]

Kuemmerle, R., Grisetti, G., Strasdat, H., Konolige, K. and Burgard, W., g2o: a general framework for graph optimization. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA),

[31]

Lim, Jongwoo, Pollefeys, Marc and Frahm, Jan-Michael, Online environment mapping. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR),

Digital Library

[32]

Mahony, R. and Manton, J.H., The geometry of the Newton method on non-compact Lie groups. J. Glob. Optim. v23 i3. 309-327.

[33]

McLauchlan, P., Reid, I. and Murray, D., Recursive affine structure and motion from image sequences. In: Proceedings of the European Conference on Computer Vision (ECCV),

[34]

Mei, C., Sibley, G., Cummins, M., Newman, P. and Reid, I., RSLAM: a system for large-scale mapping in constant-time using stereo. Int. J. Comput. Vision. v94. 198-214.

Digital Library

[35]

Montemerlo, M., Thrun, S., Koller, D. and Wegbreit, B., FastSLAM: a factored solution to the simultaneous localization and mapping problem. In: Proceedings of the AAAI National Conference on Artificial Intelligence,

[36]

Montiel, J.M.M., Civera, J. and Davison, A.J., Unified inverse depth parametrization for monocular SLAM. In: Proceedings of Robotics: Science and Systems (RSS),

[37]

Mouragnon, E., Lhuillier, M., Dhome, M., Dekeyser, F. and Sayd, P., Real-time localization and 3D reconstruction. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR),

Digital Library

[38]

Neira, J. and Tardós, J.D., Data association in stochastic mapping using the joint compatibility test. IEEE Trans. Robot. Autom. v17 i6. 890-897.

[39]

Nistér, D., An efficient solution to the five-point relative pose problem. IEEE Trans. Pattern Anal. Mach. Intell. v26 i6. 756-777.

[40]

Nistér, D., Naroditsky, O. and Bergen, J., Visual odometry. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR),

[41]

Nister, D. and Stewenius, H., Scalable recognition with a vocabulary tree. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR),

Digital Library

[42]

Pietzsch, T., Efficient feature parameterisation for visual SLAM using inverse depth bundles. In: Proceedings of the British Machine Vision Conference (BMVC),

[43]

Pinies, P. and Tardós, J.D., Large scale SLAM building conditionally independent local maps: application to monocular vision. IEEE Trans. Rob. v24 i5. 1094-1106.

[44]

Sibley, G., Matthies, L. and Sukhatme, G., Bias reduction filter convergence for long range stereo. In: 12th International Symposium of Robotics Research,

[45]

Sibley, G., Matthies, L. and Sukhatme, G., A sliding window filter for incremental SLAM. In: Unifying Perspectives in Computational and Robot Vision, pp. 103-112.

[46]

Sibley, G., Mei, C., Reid, I. and Newman, P., Adaptive relative bundle adjustment. In: Proceedings of Robotics: Science and Systems (RSS),

[47]

Sim, R., Elinas, P., Griffin, M. and Little, J.J., Vision-based SLAM using the Rao-Blackwellised particle filter. In: Proceedings of the IJCAI Workshop on Reasoning with Uncertainty in Robotics,

[48]

Strasdat, H., Montiel, J.M.M. and Davison, A.J., Real-time monocular SLAM: why filter?. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA),

[49]

Strasdat, H., Montiel, J.M.M. and Davison, A.J., Scale drift-aware large scale monocular SLAM. In: Proceedings of Robotics: Science and Systems (RSS),

[50]

Tikhonov, A.N. and Arsenin, V.I.A., Solutions of Ill-posed Problems. 1977. Winston, Washington, DC.

[51]

Triggs, B., McLauchlan, P., Hartley, R. and Fitzgibbon, A., Bundle adjustment - a modern synthesis. In: Proceedings of the International Workshop on Vision Algorithms, in Association with ICCV,

[52]

Werlberger, M., Pock, T. and Bischof, H., Motion estimation with non-local total variation regularization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR),

Cited By

Chen SHe QLai C(2024)Deep Reinforcement Learning-Based Robot Exploration for Constructing Map of Unknown EnvironmentInformation Systems Frontiers10.1007/s10796-021-10218-526:1(63-74)Online publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1007/s10796-021-10218-5
Zhanabatyrova ALeite CXiao Y(2023)Structure from Motion-Based Mapping for Autonomous Driving: Practice and ExperienceACM Transactions on Internet of Things10.1145/36315335:1(1-25)Online publication date: 6-Nov-2023
https://dl.acm.org/doi/10.1145/3631533
Muzzini FCapodieci NCavicchioli RRouxel BAgrawal KShun J(2023)Brief Announcement: Optimized GPU-accelerated Feature Extraction for ORB-SLAM SystemsProceedings of the 35th ACM Symposium on Parallelism in Algorithms and Architectures10.1145/3558481.3591310(299-302)Online publication date: 17-Jun-2023
https://dl.acm.org/doi/10.1145/3558481.3591310
Show More Cited By

Index Terms

Editors Choice Article: Visual SLAM: Why filter?
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
      2. Computer vision tasks
        Scene understanding

Index terms have been assigned to the content through auto-classification.

Recommendations

Orientation and pose estimation for panoramic imagery
Special issue on Image Databases

In a database of geo-referenced images, determining the exact position of each panorama is an important step in order to ensure the consistency of visual information. This paper addresses the problem of camera pose recovery from spherical (360°) ...
Accurate Camera Calibration from Multi-View Stereo and Bundle Adjustment

The advent of high-resolution digital cameras and sophisticated multi-view stereo algorithms offers the promise of unprecedented geometric fidelity in image-based modeling tasks, but it also puts unprecedented demands on camera calibration to fulfill ...
Editors Choice Article: Face pose estimation with automatic 3D model creation in challenging scenarios

This paper proposes a new method to perform real-time face pose estimation for +/-90^o yaw rotations and under low light conditions. The algorithm works on the basis of a completely automatic and run-time incremental 3D face modelling. The model is ...

Comments

Information & Contributors

Information

Published In

cover image Image and Vision Computing

Image and Vision Computing Volume 30, Issue 2

February, 2012

69 pages

ISSN:0262-8856

Issue’s Table of Contents

Copyright © Elsevier B.V. © 2012.

Publisher

Butterworth-Heinemann

United States

Publication History

Published: 01 February 2012

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

67
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 14 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Chen SHe QLai C(2024)Deep Reinforcement Learning-Based Robot Exploration for Constructing Map of Unknown EnvironmentInformation Systems Frontiers10.1007/s10796-021-10218-526:1(63-74)Online publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1007/s10796-021-10218-5
Zhanabatyrova ALeite CXiao Y(2023)Structure from Motion-Based Mapping for Autonomous Driving: Practice and ExperienceACM Transactions on Internet of Things10.1145/36315335:1(1-25)Online publication date: 6-Nov-2023
https://dl.acm.org/doi/10.1145/3631533
Muzzini FCapodieci NCavicchioli RRouxel BAgrawal KShun J(2023)Brief Announcement: Optimized GPU-accelerated Feature Extraction for ORB-SLAM SystemsProceedings of the 35th ACM Symposium on Parallelism in Algorithms and Architectures10.1145/3558481.3591310(299-302)Online publication date: 17-Jun-2023
https://dl.acm.org/doi/10.1145/3558481.3591310
Nam DGon-Woo K(2023)Learning Type-2 Fuzzy Logic for Factor Graph Based-Robust Pose Estimation With Multi-Sensor FusionIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2023.323459524:4(3809-3821)Online publication date: 1-Apr-2023
https://dl.acm.org/doi/10.1109/TITS.2023.3234595
Lee JBack MHwang SChun I(2023)Improved Real-Time Monocular SLAM Using Semantic Segmentation on Selective FramesIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2022.322852524:3(2800-2813)Online publication date: 1-Mar-2023
https://dl.acm.org/doi/10.1109/TITS.2022.3228525
Liu QHao YLiu WYu BGan YTang JLiu SZhu Y(2023)An Energy Efficient and Runtime Reconfigurable Accelerator for Robotic LocalizationIEEE Transactions on Computers10.1109/TC.2022.323089972:7(1943-1957)Online publication date: 1-Jul-2023
https://dl.acm.org/doi/10.1109/TC.2022.3230899
Müller Cvan Daalen C(2023)Map point selection for visual SLAMRobotics and Autonomous Systems10.1016/j.robot.2023.104485167:COnline publication date: 1-Sep-2023
https://dl.acm.org/doi/10.1016/j.robot.2023.104485
Tang FWu SQian ZWu Y(2023)Efficient 6-DoF camera pose tracking with circular edgesComputer Vision and Image Understanding10.1016/j.cviu.2023.103767235:COnline publication date: 1-Oct-2023
https://dl.acm.org/doi/10.1016/j.cviu.2023.103767
Fan ZHao YZhi TGuo QDu Z(2023)Hardware Acceleration for SLAM in Mobile SystemsJournal of Computer Science and Technology10.1007/s11390-021-1523-538:6(1300-1322)Online publication date: 1-Dec-2023
https://dl.acm.org/doi/10.1007/s11390-021-1523-5
Murai RSaeedi SKelly P(2023)High-frame rate homography and visual odometry by tracking binary features from the focal planeAutonomous Robots10.1007/s10514-023-10122-847:8(1579-1592)Online publication date: 1-Dec-2023
https://dl.acm.org/doi/10.1007/s10514-023-10122-8
Show More Cited By

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents