research-article

Solution of sparse rectangular systems using LSQR and CRAIG

Author:

Michael A. SaundersAuthors Info & Claims

Volume 35, Issue 4

Pages 588 - 604

https://doi.org/10.1007/BF01739829

Published: 01 December 1995 Publication History

Abstract

We examine two iterative methods for solving rectangular systems of linear equations: LSQR for over-determined systemsAx ≈ b, and Craig's method for under-determined systemsAx = b. By including regularization, we extend Craig's method to incompatible systems, and observe that it solves the same damped least-squares problems as LSQR. The methods may therefore be compared on rectangular systems of arbitrary shape.

Various methods for symmetric and unsymmetric systems are reviewed to illustrate the parallels. We see that the extension of Craig's method closes a gap in existing theory. However, LSQR is more economical on regularized problems and appears to be more reliable if the residual is not small.

In passing, we analyze a scaled “augmented system” associated with regularized problems. A bound on the condition number suggests a promising direct method for sparse equations and least-squares problems, based on indefiniteLDL ^T factors of the augmented matrix.

References

[1]

M. Arioli, I. S. Duff and P. P. M. de Rijk,On the augmented system approach to sparse least-squares problems, Numer. Math., 55 (1989), pp. 667–684.

Digital Library

[2]

Å. Björck,Iterative refinement of linear least squares solutions I, BIT, 7 (1967), pp. 257–278.

Digital Library

[3]

Å. Björck,A bidiagonalization algorithm for solving ill-posed systems of linear equations, Report LITH-MAT-R-80-33, Dept. of Mathematics, Linköping University, Linköping, Sweden, 1980.

[4]

Å. Björck,Pivoting and stability in the augmented system method, in D. F. Griffiths and G. A. Watson (eds.),Numerical Analysis 1991: Proceedings of the 14th Dundee Conference, Pitman Research Notes in Mathematics 260, Longman Scientific and Technical, Harlow, Essex, 1992.

[5]

J. E. Craig,The N-step iteration procedures, J. Math. and Phys., 34, 1 (1955), pp. 64–73.

[6]

A. Dax,On row relaxation methods for large constrained least-squares problems, SIAM J. Sci. Comp., 14 (1993), pp. 570–584.

Digital Library

[7]

I. S. Duff and J. K. Reid,Exploiting zeros on the diagonal in the direct solution of indefinite sparse symmetric linear systems, ACM Trans. Math. Softw., to appear.

[8]

D. K. Faddeev and V. N. Faddeeva,Computational Methods of Linear Algebra, Freeman, London, 1963.

[9]

R. W. Freund,Über einige CG-ähnliche Verfahren zur Lösung linearer Gleichungssysteme, Ph.D. Thesis, Universität Würzburg, FRG, 1983.

[10]

P. E. Gill, M. A. Saunders and J. R. Shinnerl,On the stability of Cholesky factorization for quasi-definite systems, SIAM J. Mat. Anal., 17(1) (1996), to appear.

[11]

G. H. Golub and W. Kahan,Calculating the singular values and pseudoinverse of a matrix, SIAM J. Numer. Anal., 2 (1965), pp. 205–224.

[12]

G. H. Golub and C. F. Van Loan,Unsymmetric positive definite linear systems, Linear Alg. and its Appl., 28 (1979), pp. 85–98.

[13]

P. C. Hansen,Test matrices for regularization methods, SIAM J. Sci. Comput., 16(2) (1995), pp. 506–512.

Digital Library

[14]

G. T. Herman, A. Lent and H. Hurwitz,A storage-efficient algorithm for finding the regularized solution of a large, inconsistent system of equations, J. Inst. Math. Appl., 25 (1980), pp. 361–366.

[15]

D. P. O'Leary, Private communication, 1990.

[16]

C. Lanczos,An iteration method for the solution of the eigenvalue problem of linear differential and integral operators, J. Res. Nat. Bur. Standards, 45 (1950), pp. 255–282.

[17]

P. Matstoms,Sparse QR factorization in MATLAB, ACM Trans. Math. Software, 20(1) (1994), pp. 136–159.

Digital Library

[18]

C. C. Paige,Bidiagonalization of matrices and solution of linear equations, SIAM J. Numer. Anal., 11 (1974), pp. 197–209.

[19]

C. C. Paige,Krylov subspace processes, Krylov subspace methods and iteration polynomials, in J. D. Brown, M. T. Chu, D. C. Ellison, and R. J. Plemmons, eds.,Proceedings of the Cornelius Lanczos International Centenary Conference, Raleigh, NC, Dec. 1993, SIAM, Philadelphia, 1994, pp. 83–92.

[20]

C. C. Paige and M. A. Saunders,Solution of sparse indefinite systems of linear equations, SIAM J. Numer. Anal., 12(4) (1975), pp. 617–629.

Digital Library

[21]

C. C. Paige and M. A. Saunders,LSQR: An algorithm for sparse linear equations and sparse least squares, ACM Trans. Math. Software, 8(1) (1982), pp. 43–71.

Digital Library

[22]

C. C. Paige and M. A. Saunders,Algorithm 583. LSQR: Sparse linear equations and least squares problems, ACM Trans. Math. Software, 8(2) (1982), pp. 195–209.

Digital Library

[23]

M. A. Saunders,Cholesky-based methods for sparse least squares: The benefits of regularization, Report SOL 95-1, Dept. of Operations Research, Stanford University, California, USA, 1995.

[24]

R. J. Vanderbei,Symmetric quasi-definite matrices, SIAM J. Optim., 5(1) (1995), pp. 100–113.

Cited By

Zhang ZYezzi AGallego G(2023)Formulating Event-Based Image Reconstruction as a Linear Inverse Problem With Deep Regularization Using Optical FlowIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2022.323072745:7(8372-8389)Online publication date: 1-Jul-2023
https://dl.acm.org/doi/10.1109/TPAMI.2022.3230727
Papež JTichý P(2023)Estimating error norms in CG-like algorithms for least-squares and least-norm problemsNumerical Algorithms10.1007/s11075-023-01691-x97:1(1-28)Online publication date: 7-Nov-2023
https://dl.acm.org/doi/10.1007/s11075-023-01691-x
Scott JTůma M(2022)A Computational Study of Using Black-box QR Solvers for Large-scale Sparse-dense Linear Least Squares ProblemsACM Transactions on Mathematical Software10.1145/349452748:1(1-24)Online publication date: 16-Feb-2022
https://dl.acm.org/doi/10.1145/3494527
Show More Cited By

Index Terms

Solution of sparse rectangular systems using LSQR and CRAIG

Index terms have been assigned to the content through auto-classification.

Recommendations

MINRES-QLP: A Krylov Subspace Method for Indefinite or Singular Symmetric Systems

CG, SYMMLQ, and MINRES are Krylov subspace methods for solving symmetric systems of linear equations. When these methods are applied to an incompatible system (that is, a singular symmetric least-squares problem), CG could break down and SYMMLQ's ...
Iterative Solution of Augmented Systems Arising in Interior Methods

Iterative methods are proposed for certain augmented systems of linear equations that arise in interior methods for general nonlinear optimization. Interior methods define a sequence of KKT equations that represent the symmetrized (but indefinite) ...
Computing projections with LSQR
Abstract
LSQR uses the Golub-Kahan bidiagonalization process to solve sparse least-squares problems with and without regularization. In some cases, projections of the right-hand side vector are required, rather than the least-squares solution itself. We ...

Comments

Information & Contributors

Information

Published In

cover image BIT

BIT Volume 35, Issue 4

Dec 1995

164 pages

ISSN:0006-3835

Issue’s Table of Contents

Copyright © 1995 BIT Foundation.

Publisher

BIT Computer Science and Numerical Mathematics

United States

Publication History

Published: 01 December 1995

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 02 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zhang ZYezzi AGallego G(2023)Formulating Event-Based Image Reconstruction as a Linear Inverse Problem With Deep Regularization Using Optical FlowIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2022.323072745:7(8372-8389)Online publication date: 1-Jul-2023
https://dl.acm.org/doi/10.1109/TPAMI.2022.3230727
Papež JTichý P(2023)Estimating error norms in CG-like algorithms for least-squares and least-norm problemsNumerical Algorithms10.1007/s11075-023-01691-x97:1(1-28)Online publication date: 7-Nov-2023
https://dl.acm.org/doi/10.1007/s11075-023-01691-x
Scott JTůma M(2022)A Computational Study of Using Black-box QR Solvers for Large-scale Sparse-dense Linear Least Squares ProblemsACM Transactions on Mathematical Software10.1145/349452748:1(1-24)Online publication date: 16-Feb-2022
https://dl.acm.org/doi/10.1145/3494527
Scott JTůma M(2022)Solving large linear least squares problems with linear equality constraintsBIT10.1007/s10543-022-00930-262:4(1765-1787)Online publication date: 1-Dec-2022
https://dl.acm.org/doi/10.1007/s10543-022-00930-2
Reichel LSadok HZhang W(2020)Simple stopping criteria for the LSQR method applied to discrete ill-posed problemsNumerical Algorithms10.1007/s11075-019-00852-184:4(1381-1395)Online publication date: 22-Jan-2020
https://dl.acm.org/doi/10.1007/s11075-019-00852-1
Hager W(2000)Iterative Methods for Nearly Singular Linear SystemsSIAM Journal on Scientific Computing10.1137/S106482759834634X22:2(747-766)Online publication date: 1-Jan-2000
https://dl.acm.org/doi/10.1137/S106482759834634X
Fischer BRamage ASilvester DWathen A(1998)Minimum residual methods for augmented systemsBIT10.1007/BF0251025838:3(527-543)Online publication date: 1-Sep-1998
https://dl.acm.org/doi/10.1007/BF02510258
Saunders M(1997)Computing projections with LSQRBIT10.1007/BF0251017537:1(96-104)Online publication date: 1-Mar-1997
https://dl.acm.org/doi/10.1007/BF02510175

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents