research-article

SBA: A software package for generic sparse bundle adjustment

Authors:

Manolis I. A. Lourakis,

Antonis A. ArgyrosAuthors Info & Claims

ACM Transactions on Mathematical Software (TOMS), Volume 36, Issue 1

Article No.: 2, Pages 1 - 30

https://doi.org/10.1145/1486525.1486527

Published: 16 March 2009 Publication History

Get Access

Abstract

Bundle adjustment constitutes a large, nonlinear least-squares problem that is often solved as the last step of feature-based structure and motion estimation computer vision algorithms to obtain optimal estimates. Due to the very large number of parameters involved, a general purpose least-squares algorithm incurs high computational and memory storage costs when applied to bundle adjustment. Fortunately, the lack of interaction among certain subgroups of parameters results in the corresponding Jacobian being sparse, a fact that can be exploited to achieve considerable computational savings. This article presents sba, a publicly available C/C++ software package for realizing generic bundle adjustment with high efficiency and flexibility regarding parameterization.

References

[1]

Amestoy, P., Davis, T., and Duff, I. 2004. Algorithm 837: AMD, an Approximate Minimum Degree Ordering Algorithm. ACM Trans. Math. Softw. 30, 3 (Sep.), 381--388.

Digital Library

Google Scholar

[2]

Anderson, E., Bai, Z., Bischof, C., Blackford, S., Demmel, J., Dongarra, J., Croz, J. D., Greenbaum, A., Hammarling, S., McKenney, A., and Sorensen, D. 1999. LAPACK Users' Guide, 3rd ed. SIAM, Philadelphia, PA.

Digital Library

Google Scholar

[3]

Barrett, R., Berry, M., Chan, T. F., Demmel, J., Donato, J., Dongarra, J., Eijkhout, V., Pozo, R., Romine, C., and der Vorst, H. V. 1994. Templates for the Solution of Linear Systems: Building Blocks for Iterative Methods, 2nd ed. SIAM, Philadelphia, PA. http://www.netlib.org/linalg/html_templates/report.html.

Google Scholar

[4]

Beardsley, P., Torr, P., and Zisserman, A. 1996. 3D model acquisition from extended image sequences. In Proceedings of the. European Conference on Computer Vision. Springer-Verlag, Berlin, Germany, 683--695.

Digital Library

Google Scholar

[5]

Besnerais, G. L., Sanfourche, M., and Champagnat, F. 2008. Dense height map estimation from oblique aerial image sequences. Comput. Vis. Image Understand. 109, 2, 204--225.

Digital Library

Google Scholar

[6]

Bianchi, G., Jung, C., Knoerlein, B., Szekely, G., and Harders, M. 2006. High-fidelity visuo-haptic interaction with virtual objects in multi-modal AR systems. In Proceedings of the IEEE/ACM ISMAR'06 (Santa Barbara, CA), 187--196.

Digital Library

Google Scholar

[7]

Brooks, M., Chojnacki, W., Gawley, D., and van den Hengel, A. 2001. What value covariance information in estimating vision parameters&quest; In Proceedings of the International Conference on Computer Vision. Vol. I. IEEE Press, Los Alamitos, CA, 302--308.

Google Scholar

[8]

Brown, D. 1958. A solution to the general problem of multiple station analytical stereo triangulation. Tech. rep. 43. RCA-MTP. Feb.

Google Scholar

[9]

Chen, Y., Davis, T., Hager, W., and Rajamanickam, S. 2006. Algorithm 8xx: CHOLMOD, supernodal sparse cholesky factorization and update/downdate. Tech. rep. 005. Computer and Information Science and Engineering Dept., University of Florida. Gainesville, FL. Submitted to ACM Trans. Math. Software.

Digital Library

Google Scholar

[10]

Conn, A., Gould, N., and Toint, P. 2000. Trust Region Methods. MPS-SIAM Series On Optimization. SIAM, Philadelphia, PA.

Digital Library

Google Scholar

[11]

Davis, T. A. 2005. Algorithm 849: A concise sparse cholesky factorization package. ACM Trans. Math. Softw. 31, 4, 587--591.

Digital Library

Google Scholar

[12]

Demmel, J. 1997. Applied Numerical Linear Algebra. Titles in Applied Mathematics. SIAM Publications, Philadelphia, PA.

Digital Library

Google Scholar

[13]

Demmel, J., Eisenstat, S., Gilbert, J., Li, X., and Liu, J. 1999. A supernodal approach to sparse partial pivoting. SIAM J. Matrix Anal. Appl. 20, 3, 720--755.

Digital Library

Google Scholar

[14]

Dennis, J. 1977. Nonlinear least-squares. In State of the Art in Numerical Analysis, D. Jacobs, Ed. Academic Press, New York, NY, 269--312.

Google Scholar

[15]

Dennis, J., Gay, D., and Welsch, R. 1981. An adaptive nonlinear least-squares algorithm. ACM Trans. Math. Softw. 7, 3 (Sep.), 348--368.

Digital Library

Google Scholar

[16]

Dennis, J. and Schnabel, R. 1996. Numerical Methods for Unconstrained Optimization and Nonlinear Equations. Classics in Applied Mathematics. SIAM Publications, Philadelphia, PA.

Digital Library

Google Scholar

[17]

Fitzgibbon, A. and Zisserman, A. 1998. Automatic camera recovery for closed or open image sequences. In Proceedings of the European Conference on Computer Vision. Springer-Verlag, Berlin, Germany, 311--326.

Digital Library

Google Scholar

[18]

Gay, D. 1990. Usage summary for selected optimization routines. Tech. rep. 153. AT&T Bell Laboratories, Murray Hill, NJ.

Google Scholar

[19]

Golub, G. and van Loan, C. 1996. Matrix Computations, 3rd ed. Johns Hopkins University Press, Baltimore, MD.

Digital Library

Google Scholar

[20]

Griewank, A., Juedes, D., and Utke, J. 1996. ADOL--C, a package for the automatic differentiation of algorithms written in C/C++. ACM Trans. Math. Softw. 22, 2, 131--167.

Digital Library

Google Scholar

[21]

Hartley, R. 1993. Euclidean reconstruction from uncalibrated views. In Applications of Invariance in Computer Vision, J. Mundy and A. Zisserman, Eds. Lecture Notes in Computer Science, Vol. 825. Springer-Verlag, Berlin, Germany, 237--256.

Digital Library

Google Scholar

[22]

Hartley, R. and Zisserman, A. 2000. Multiple View Geometry in Computer Vision, 1st ed. Cambridge University Press, Cambridge, MA.

Digital Library

Google Scholar

[23]

Hiebert, K. 1981. An evaluation of mathematical software that solves nonlinear least squares problems. ACM Trans. Math. Softw. 7, 1 (Mar.), 1--16.

Digital Library

Google Scholar

[24]

Horn, B. 1987. Closed-form solution of absolute orientation using unit quaternions. J. Opt. Soc. Amer. A 4, 4, 629--642.

Crossref

Google Scholar

[25]

Irschara, A., Zach, C., and Bischof, H. 2007. Towards wiki-based dense city modeling. In Proceedings of the International Conference on Computer Vision. IEEE Press, Los Alamitos, CA, 1--8.

Google Scholar

[26]

Kelley, C. 1999. Iterative Methods for Optimization. SIAM Publications, Philadelphia.

Google Scholar

[27]

Lampton, M. 1997. Damping-undamping strategies for the Levenberg-Marquardt nonlinear least-squares method. Comput. Phys. J. 11, 1 (Jan./Feb.), 110--115.

Digital Library

Google Scholar

[28]

Levenberg, K. 1944. A method for the solution of certain non-linear problems in least squares. Quart. Appl. Math. 2, 2 (Jul.), 164--168.

Crossref

Google Scholar

[29]

Lourakis, M. and Argyros, A. 2005a. Efficient, causal camera tracking in unprepared environments. Comput. Vis. and Image Understand. 99, 2 (Aug.), 259--290.

Crossref

Google Scholar

[30]

Lourakis, M. and Argyros, A. 2005b. Is Levenberg-Marquardt the most efficient optimization algorithm for implementing bundle adjustment&quest; In Proceedings of the International Conference on Computer Vision. IEEE Press, Los Alamitos, CA, 1526--1531.

Digital Library

Google Scholar

[31]

Lowe, D. 2004. Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60, 2 (Nov.), 91--110.

Digital Library

Google Scholar

[32]

Lu, C.-P., Hager, G., and Mjolsness, E. 2000. Fast and globally convergent pose estimation from video images. IEEE Trans. Patt. Anal. Mach. Intell. 22, 6 (Jun.), 610--622.

Digital Library

Google Scholar

[33]

Madsen, K., Nielsen, H., and Tingleff, O. 2004. Methods for non-linear least squares problems. Technical University of Denmark. Lecture notes. http://www.imm.dtu.dk/pubdb/views/edoc_download.php/3215/pdf/imm3215.pdf.

Google Scholar

[34]

Marquardt, D. 1963. An algorithm for the least-squares estimation of nonlinear parameters. SIAM J. Appl. Math. 11, 2 (Jun.), 431--441.

Crossref

Google Scholar

[35]

Moré, J. 1977. The Levenberg Marquardt algorithm: implementation and theory. In Numerical Analysis, G. Watson, Ed. Lecture Notes in Mathematics, vol. 630. Springer Verlag, Berlin, Germany, 105--116.

Google Scholar

[36]

Moré, J., Garbow, B., and Hillstrom, K. 1980. User guide for MINPACK-1. Tech. rep. ANL-80-74. Argonne National Laboratory, Argonne, IL.

Google Scholar

[37]

Moré, J. and Sorensen, D. 1983. Computing a trust region step. SIAM J. Sci. Statist. Comput. 4, 553--572.

Digital Library

Google Scholar

[38]

Mundy, J. and Zisserman, A. 1992. Projective geometry for machine vision. In Geometric Invariance in Computer Vision, J. Mundy and A. Zisserman, Eds. MIT Press, Cambridge, MA, 463--519.

Digital Library

Google Scholar

[39]

Nielsen, H. 1999. Damping parameter in Marquardt's method. Tech. rep. IMM-REP-1999-05. Technical University of Denmark, Aarhus, Denmark. http://www.imm.dtu.dk/~hbn.

Google Scholar

[40]

Nocedal, J. and Wright, S. 1999. Numerical Optimization. Springer, New York, NY.

Google Scholar

[41]

Pollefeys, M., Gool, L. V., Vergauwen, M., Verbiest, F., Cornelis, K., Tops, J., and Koch, R. 2004. Visual modeling with a hand-held camera. Int. J. Comput. Vis. 59, 3 (Sep./Oct.), 207--232.

Digital Library

Google Scholar

[42]

Prasolov, V. 1994. Problems and Theorems in Linear Algebra. American Mathematical Society, Providence, RI.

Google Scholar

[43]

Semple, J. and Kneebone, G. 1952. Algebraic Projective Geometry. Oxford University Press, Oxford, U.K.

Google Scholar

[44]

Shi, J. and Tomasi, C. 1994. Good features to track. In Proceedings of the International Conference on Computer Vision and Pattern Recognition. IEEE Press, Los Alamitos, CA, 593--600.

Google Scholar

[45]

Shum, H.-Y., Ke, Q., and Zhang, Z. 1999. Efficient bundle adjustment with virtual key frames: a hierarchical approach to multi-frame structure from motion. In Proceedings of the International Conference on Computer Vision and Pattern Recognition. Vol. 2. IEEE Press, Los Alamitos, CA, 538--543.

Google Scholar

[46]

Slama, C. 1980. Manual of Photogrammetry, ed. American Society of Photogrammetry, Falls Church, VA.

Google Scholar

[47]

Snavely, N., Seitz, S., and Szeliski, R. 2006. Photo tourism: Exploring photo collections in 3D. ACM Trans. Graphics (SIGGRAPH Proceedings) 25, 3, 835--846.

Digital Library

Google Scholar

[48]

Sünderhauf, N., Konolige, K., Lacroix, S., and Protzel, P. 2005. Visual odometry using sparse bundle adjustment on an autonomous outdoor vehicle. In Autonome Mobile Systeme 2005, P. Levi, M. Schanz, R. Lafrenz, and V. Avrutin, Eds. Informatik Aktuell. Springer Verlag, Berlin, Germany, 157--163.

Google Scholar

[49]

Szeliski, R. and Kang, S. 1994. Recovering 3D shape and motion from image streams using nonlinear least squares. J. Vis. Comm. Im. Repr. 5, 1, 10--28.

Crossref

Google Scholar

[50]

Templeton, T., Shim, D., Geyer, C., and Sastry, S. 2007. Autonomous vision-based landing and terrain mapping using an MPC-controlled unmanned rotorcraft. In Proceedings of the IEEE ICRA'07. 1349--1356.

Google Scholar

[51]

Triggs, B., McLauchlan, P., Hartley, R., and Fitzgibbon, A. 1999. Bundle adjustment—a modern synthesis. In Proceedings of the International Workshop on Vision Algorithms: Theory and Practice. 298--372.

Digital Library

Google Scholar

[52]

Vicci, L. 2001. Quaternions and rotations in 3-space: The algebra and its geometric interpretation. Tech. rep. TR01-014. Dept. of Computer Science, University of North Carolina at Chapel Hill, Chapel Hill, NC.

Digital Library

Google Scholar

[53]

Xin, L., Wang, Q., Tao, J., Tang, X., Tan, T., and Shum, H. 2005. Automatic 3D face modeling from video. In Proceedings of the International Conference on Computer Vision. Vol. 2. IEEE Press, Los Alamitos, CA, 1193--1199.

Digital Library

Google Scholar

[54]

Zhang, Z., Deriche, R., Faugeras, O., and Luong, Q.-T. 1995. A robust technique for matching two uncalibrated images through the recovery of the unknown epipolar geometry. AI J. 78, 87--119. (Detailed version in INRIA Tech.rep. RR-2273. INRIA Rocquencourt, France.)

Digital Library

Google Scholar

[55]

Zhang, Z. and Shan, Y. 2003. Incremental motion estimation through modified bundle adjustment. In Proceedings of the International Conference on Image Processing. Vol. 2. 343--346.

Google Scholar

Cited By

View all

Zhang JWolek AWillis A(2024)UAV-Borne Mapping Algorithms for Low-Altitude and High-Speed Drone ApplicationsSensors10.3390/s2407220424:7(2204)Online publication date: 29-Mar-2024
https://doi.org/10.3390/s24072204
Yuhai OCho YChoi AMun J(2024)Enhanced Three-Axis Frame and Wand-Based Multi-Camera Calibration Method Using Adaptive Iteratively Reweighted Least Squares and Comprehensive Error IntegrationPhotonics10.3390/photonics1109086711:9(867)Online publication date: 15-Sep-2024
https://doi.org/10.3390/photonics11090867
Koutlemanis PZabulis XStivaktakis NPartarakis NZidianakis EDemeridou I(2024)A low-cost close-range photogrammetric surface scannerFrontiers in Imaging10.3389/fimag.2024.13413433Online publication date: 31-Jan-2024
https://doi.org/10.3389/fimag.2024.1341343
Show More Cited By

Recommendations

A Regularized Gauss-Newton Trust Region Approach to Imaging in Diffuse Optical Tomography
^† Special Section: 2010 Copper Mountain Conference

We present a new algorithm for the solution of nonlinear least squares problems arising from parameterized imaging problems with diffuse optical tomographic data [D. Boas et al., IEEE Signal Process. Mag., 18 (2001), pp. 57-75]. The parameterization ...
Global Methods for Nonlinear Complementarity Problems

Global methods for nonlinear complementarity problems formulate the problem as a system of nonsmooth nonlinear equations, or use continuation to trace a path defined by a smooth system of nonlinear equations. We formulate the nonlinear complementarity ...
A Curvilinear Search Using Tridiagonal Secant Updates for Unconstrained Optimization

The idea of doing a curvilinear search along the Levenberg–Marquardt path $s(\mu ) = - (H + \mu I)^{ - 1} g$ always has been appealing, but the cost of solving a linear system for each trial value of the parameter $\mu $ has discouraged its ...

Reviews

Reviewer: Charalambos Poullis

Lourakis and Argyros address the common problem of bundle adjustment that is traditionally associated with-and is an imperative component of-three-dimensional (3D) scene reconstruction. In this context, bundle adjustment can be described as a minimization problem, where the 3D structure and viewing (that is, extrinsic and sometimes intrinsic) parameters are optimized, using a set of correspondences between a group of two-dimensional (2D) images and the 3D geometry of the scene. The optimal values are found by "minimizing the reprojection error between the observed and predicted image points," using a nonlinear least-squares algorithm [1]. Bundle adjustment is a well-known problem, in particular to the computer vision community, and is already a well-studied subject. However, this paper presents novel ideas on how to improve the performance and efficiency of the optimization by exploiting the fact that "the normal equations matrix has a sparse block structure owing to the lack of interaction among parameters for different 3D points and cameras" [1]. The paper proposes solutions to the important problem of reducing computational complexity, at a time when 3D reconstructions are becoming increasingly demanding in terms of dimensionality. The paper is well written and its organization makes it very easy to follow. It begins with a thorough description of the problem and a discussion of the existing solutions and their limitations. Next, the authors present their strategy in great detail, using simple examples that can be easily generalized to higher dimensions. The interested reader can then follow through with the implementation details of the software package SBA; this is finally validated experimentally to show that the computational performance is indeed greatly improved. Online Computing Reviews Service

Access critical reviews of Computing literature here

Become a reviewer for Computing Reviews.

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Mathematical Software

ACM Transactions on Mathematical Software Volume 36, Issue 1

March 2009

137 pages

ISSN:0098-3500

EISSN:1557-7295

DOI:10.1145/1486525

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 March 2009

Accepted: 01 May 2008

Revised: 01 February 2006

Received: 01 May 2005

Published in TOMS Volume 36, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Sixth Framework Programme

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

531
Total Citations
View Citations
3,385
Total Downloads

Downloads (Last 12 months)83
Downloads (Last 6 weeks)11

Reflects downloads up to 10 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Zhang JWolek AWillis A(2024)UAV-Borne Mapping Algorithms for Low-Altitude and High-Speed Drone ApplicationsSensors10.3390/s2407220424:7(2204)Online publication date: 29-Mar-2024
https://doi.org/10.3390/s24072204
Yuhai OCho YChoi AMun J(2024)Enhanced Three-Axis Frame and Wand-Based Multi-Camera Calibration Method Using Adaptive Iteratively Reweighted Least Squares and Comprehensive Error IntegrationPhotonics10.3390/photonics1109086711:9(867)Online publication date: 15-Sep-2024
https://doi.org/10.3390/photonics11090867
Koutlemanis PZabulis XStivaktakis NPartarakis NZidianakis EDemeridou I(2024)A low-cost close-range photogrammetric surface scannerFrontiers in Imaging10.3389/fimag.2024.13413433Online publication date: 31-Jan-2024
https://doi.org/10.3389/fimag.2024.1341343
Rader JHedrick T(2024)Turkey vultures tune their airspeed to changing air densityJournal of Experimental Biology10.1242/jeb.246828227:15Online publication date: 1-Aug-2024
https://doi.org/10.1242/jeb.246828
Makovetskii AKober VVoronin SVoronin AKarnaukhov VMozerov M(2024)Global Refinement Algorithm for 3D Scene Reconstruction from a Sequence of Point CloudsJournal of Communications Technology and Electronics10.1134/S106422692312012468:12(1499-1505)Online publication date: 1-Mar-2024
https://doi.org/10.1134/S1064226923120124
Li XZhang W(2024)Position and posture measurement of stereo probes for dental implants with a four-camera vision systemOptical Engineering10.1117/1.OE.63.1.01411063:01Online publication date: 1-Jan-2024
https://doi.org/10.1117/1.OE.63.1.014110
Ge YZhang LWu YHu D(2024)PIPO-SLAM: Lightweight Visual-Inertial SLAM With Preintegration Merging Theory and Pose-Only Descriptions of Multiple View GeometryIEEE Transactions on Robotics10.1109/TRO.2024.336681540(2046-2059)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TRO.2024.3366815
Wang YZhang DSu YLian CDeng K(2024)A Parallel Nonlinear Factor Recovery Method for VIO Based on a Factor GraphIEEE Transactions on Instrumentation and Measurement10.1109/TIM.2023.332075773(1-15)Online publication date: 2024
https://doi.org/10.1109/TIM.2023.3320757
Noh DLim HEoh GChoi DChoi JLim HBaek SMyung H(2024)CLOi-Mapper: Consistent, Lightweight, Robust, and Incremental Mapper With Embedded Systems for Commercial Robot ServicesIEEE Robotics and Automation Letters10.1109/LRA.2024.34275599:9(7541-7548)Online publication date: Sep-2024
https://doi.org/10.1109/LRA.2024.3427559
Makovetskii AVoronin SKober VVoronin A(2024)A Global Refinement Algorithm to 3D Scene Reconstruction2024 X International Conference on Information Technology and Nanotechnology (ITNT)10.1109/ITNT60778.2024.10582384(1-7)Online publication date: 20-May-2024
https://doi.org/10.1109/ITNT60778.2024.10582384
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

A Regularized Gauss-Newton Trust Region Approach to Imaging in Diffuse Optical Tomography

Global Methods for Nonlinear Complementarity Problems

A Curvilinear Search Using Tridiagonal Secant Updates for Unconstrained Optimization

Reviews

Access critical reviews of Computing literature here

Comments

Published In

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Funding Sources

Other Metrics

Article Metrics

Other Metrics

Cited By

Login options

Full Access

PDF

eReader

Abstract

References

Cited By

Index Terms

Recommendations

A Regularized Gauss-Newton Trust Region Approach to Imaging in Diffuse Optical Tomography

Global Methods for Nonlinear Complementarity Problems

A Curvilinear Search Using Tridiagonal Secant Updates for Unconstrained Optimization

Reviews

Access critical reviews of Computing literature here

Comments

Information

Published In

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Funding Sources

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Get Access

Login options

Full Access

View options

PDF

eReader

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations