Abstract
Many applications in science and engineering require the solution of large linear discrete ill-posed problems. The matrices that define these problems are very ill-conditioned and possibly numerically singular, and the right-hand sides, which represent the measured data, typically are contaminated by measurement error. Straightforward solution of these problems generally is not meaningful due to severe error propagation. Tikhonov regularization seeks to alleviate this difficulty by replacing the given linear discrete ill-posed problem by a penalized least-squares problem, whose solution is less sensitive to the error in the right-hand side. The penalty term is determined by a regularization matrix. A suitable choice of this matrix may result in a computed solution of higher quality than when the regularization matrix is the identity. Two iterative solution methods based on the global Arnoldi decomposition method have been proposed in the literature for the solution of large-scale penalized least-squares problems that stem from Tikhonov regularization. In one of these, the regularization matrix influences the choice of the solution subspace; in the other one, it does not. This paper compares these approaches both with respect to the quality of the computed solution and computing time.
Similar content being viewed by others
References
Bentbib, A.H., El Guide, M., Jbilou, K., Reichel, L.: Global Golub–Kahan bidiagonalization for image restoration. J. Comput. Appl. Math. 300, 233–244 (2016)
Bouhamidi, A., Jbilou, K.: Sylvester Tikhonov-regularization methods in image restoration. J. Comput. Appl. Math. 206, 86–98 (2007)
Bouhamidi, A., Jbilou, K., Reichel, L., Sadok, H.: A generalized global Arnoldi method for ill-posed matrix equations. J. Comput. Appl. Math. 236, 2078–2089 (2012)
Brezinski, C., Redivo–Zaglia, M., Rodriguez, G., Seatzu, S.: Extrapolation techniques for ill-conditioned linear systems. Numer. Math. 81, 1–29 (1998)
Brezinski, C., Rodriguez, G., Seatzu, S.: Error estimates for the regularization of least squares problems. Numer. Algorithms 51, 61–76 (2009)
Calvetti, D., Morigi, S., Reichel, L., Sgallari, F.: Tikhonov regularization and the L-curve for large, discrete ill-posed problems. J. Comput. Appl. Math. 123, 423–446 (2000)
Calvetti, D., Reichel, L., Shuibi, A.: Invertible smoothing preconditioners for linear discrete ill-posed problems. Appl. Numer. Math. 54, 135–149 (2005)
Daniel, J.W., Gragg, W.B., Kaufman, L., Stewart, G.W.: Reorthogonalization and stable algorithms for updating the Gram-Schmidt QR factorization. Math. Comp. 30, 772–795 (1976)
Donatelli, M., Reichel, L.: Square smoothing regularization matrices with accurate boundary conditions. J. Comput. Appl. Math. 272, 334–349 (2014)
Donatelli, M., Neuman, A., Reichel, L.: Square regularization matrices for large linear discrete ill-posed problems. Numer. Linear Algebra Appl. 19, 896–913 (2012)
Dykes, L., Huang, G., Noschese, S., Reichel, L.: Regularization matrices for discrete ill-posed problems in several space-dimensions. Numerical Linear Algebra Appl. (2018, in press)
Elbouyahyaout, L., Messaoudi, A., Sadok, H.: Algebraic properties of the block GMRES and block Arnoldi methods. Electron. Trans. Numer. Anal. 33, 207–220 (2009)
Engl, H.W., Hanke, M., Neubauer, A.: Regularization of Inverse Problems. Kluwer, Dordrecht (1996)
Frommer, A., Lund, K., Szyld, D.B.: Block Krylov subspace methods for functions of matrices. Electron. Trans. Numer. Anal. 47, 100–126 (2017)
Gazzola, S., Novati, P., Russo, M.R.: On Krylov projection methods and Tikhonov regularization. Electron. Trans. Numer. Anal. 44, 83–123 (2015)
Hansen, P.C.: Rank-Deficient and Discrete Ill-Posed Problems. SIAM, Philadelphia (1998)
Hansen, P.C.: Regularization tools version 4.0 for MATLAB 7.3. Numer. Algorithms 46, 189–194 (2007)
Hochstenbach, M.E., Reichel, L.: An iterative method for Tikhonov regularization with a general linear regularization operator. J. Integral Equations Appl. 22, 463–480 (2010)
Horn, R.A., Johnson, C.R.: Topics in Matrix Analysis. Cambridge University Press, Cambridge (1991)
Huang, G., Noschese, S., Reichel, L.: Regularization matrices determined by matrix nearness problems. Linear Algebra Appl. 502, 41–57 (2016)
Jbilou, K., Messaoudi, A., Sadok, H.: Global FOM and GMRES algorithms for matrix equations. Appl. Numer. Math. 31, 49–63 (1999)
Jbilou, K., Sadok, H., Tinzefte, A.: Oblique projection methods for linear systems with multiple right-hand sides. Electron. Trans. Numer. Anal. 20, 119–138 (2005)
Kilmer, M.E., Hansen, P.C., Español, M.I.: A projection-based approach to general-form Tikhonov regularization. SIAM J. Sci. Comput. 29, 315–330 (2007)
Kindermann, S.: Convergence analysis of minimization-based noise level-free parameter choice rules for linear ill-posed problems. Electron. Trans. Numer. Anal. 38, 233–257 (2011)
Lampe, J., Reichel, L., Voss, H.: Large-scale Tikhonov regularization via reduction by orthogonal projection. Linear Algebra Appl. 436, 2845–2865 (2012)
Lewis, B., Reichel, L.: Arnoldi–Tikhonov regularization methods. J. Comput. Appl. Math. 226, 92–102 (2009)
Li, R.C, Ye, Q.: A Krylov subspace method for quadratic matrix polynomials with application to constrained least squares problems. SIAM J. Matrix Anal. Appl. 25, 405–428 (2003)
Morigi, S., Reichel, L., Sgallari, F.: Orthogonal projection regularization operators. Numer. Algorithms 44, 99–114 (2007)
Noschese, S., Reichel, L.: Inverse problems for regularization matrices. Numer. Algorithms 60, 531–544 (2012)
Reichel, L., Rodriguez, G.: Old and new parameter choice rules for discrete ill-posed problems. Numer. Algorithms 63, 65–87 (2013)
Reichel, L., Yu, X.: Matrix decompositions for Tikhonov regularization. Electron. Trans. Numer. Anal. 43, 223–243 (2015)
Reichel, L., Yu, X.: Tikhonov regularization via flexible Arnoldi reduction. BIT 55, 1145–1168 (2015)
Reichel, L., Rodriguez, G., Seatzu, S.: Error estimates for large-scale ill-posed problems. Numer. Algorithms 51, 341–361 (2009)
Reichel, L., Sgallari, F., Ye, Q.: Tikhonov regularization based on generalized Krylov subspace methods. Appl. Numer. Math. 62, 1215–1228 (2012)
Saad, Y.: Iterative Methods for Sparse Linear Systems, 2nd edn. SIAM, Philadelphia (2003)
Funding
Research by GH was supported in part by the Fund of Application Foundation of Science and Technology Department of the Sichuan Province (2016JY0249) and by NNSF (2017YFC0601505, 41672325), research by LR was supported in part by NSF grants DMS-1729509 and DMS-1720259, and research by FY was supported in part by NNSF (11501392).
Author information
Authors and Affiliations
Corresponding author
Additional information
In Memory of Sebastiano Seatzu.
Rights and permissions
About this article
Cite this article
Huang, G., Reichel, L. & Yin, F. On the choice of subspace for large-scale Tikhonov regularization problems in general form. Numer Algor 81, 33–55 (2019). https://doi.org/10.1007/s11075-018-0534-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11075-018-0534-y
Keywords
- Linear discrete ill-posed problems
- Global Arnoldi process
- Matrix Krylov subspace
- Standard Tikhonov problems
- Regularization