article

A comparison of reduced and unreduced KKT systems arising from interior point methods

Authors:

Benedetta Morini,

Valeria Simoncini,

Mattia TaniAuthors Info & Claims

Computational Optimization and Applications, Volume 68, Issue 1

Pages 1 - 27

https://doi.org/10.1007/s10589-017-9907-8

Published: 01 September 2017 Publication History

Abstract

We address the iterative solution of KKT systems arising in the solution of convex quadratic programming problems. Two strictly related and well established formulations for such systems are studied with particular emphasis on the effect of preconditioning strategies on their relation. Constraint and augmented preconditioners are considered, and the choice of the augmentation matrix is discussed. A theoretical and experimental analysis is conducted to assess which of the two formulations should be preferred for solving large-scale problems.

References

[1]

Altman, A., Gondzio, J.: Regularized symmetric indefinite systems in interior point methods for linear and quadratic optimization. Optim. Methods Softw. 11, 275---302 (1999)

[2]

Axelsson, O.: Preconditioners for regularized saddle point matrices. J. Numer. Math. 19(2), 91---112 (2011)

[3]

Baryamureeba, V., Steihaug, T.: On the convergence of an inexact primal-dual interior point method for linear programming. In: Lirkov, I., Margenov, S., Wasniewski, J. (eds.) Proceedings of the 5th International Conference on Large-Scale Scientific Computing. Lecture Notes in Computer Science, vol. 3743, pp. 629---637. Springer, Berlin (2006)

Digital Library

[4]

Bellavia, S.: An inexact interior point method. J. Optim. Theory Appl. 96, 109---121 (1998)

Digital Library

[5]

Benzi, M., Golub, G.H., Liesen, J.: Numerical solution of saddle point problems. Acta Numer. 14, 1---137 (2005)

[6]

Benzi, M., Olshanskii, M.A.: An augmented Lagrangian-based approach to the Oseen problem. SIAM J. Sci. Comput. 28, 2095---2113 (2006)

Digital Library

[7]

Bergamaschi, L., Gondzio, J., Zilli, G.: Preconditioning indefinite systems in interior point methods for optimization. Comput. Optim. Appl. 28, 149---171 (2004)

Digital Library

[8]

Cafieri, S., D'Apuzzo, M., De Simone, V., di Serafino, D.: Stopping criteria for inner iterations in inexact potential reduction methods: a computational study. Comput. Optim. Appl. 36, 165---193 (2007)

Digital Library

[9]

Cao, Zhi-Hao: Augmentation block preconditioners for saddle point-type matrices with singular (1, 1) blocks. Numer. Linear Algebra Appl. 15, 515---533 (2008)

[10]

Castro, J., Cuesta, J.: Quadratic regularizations in an interior-point method for primal block-angular problems. Math. Program. 130, 415---445 (2011)

Digital Library

[11]

D'Apuzzo, M., De Simone, V., di Serafino, D.: On mutual impact of linear algebra and large-scale optimization with focus on interior point methods. Comput. Optim. Appl. 45, 283---310 (2010)

Digital Library

[12]

Dembo, R.S., Eisenstat, S.C., Steihaug, T.: Inexact newton methods. SIAM J. Numer. Anal. 19, 400---408 (1982)

Digital Library

[13]

Dollar, H.S.: Constraint-style preconditioners for regularized saddle-point systems. SIAM J. Matrix Anal. Appl. 29, 672---684 (2007)

Digital Library

[14]

Dollar, H.S., Gould, N.I.M., Schilders, W.H.A., Wathen, A.J.: Implicit-factorization preconditioning and iterative solvers for regularized saddle-point systems. SIAM J. Matrix Anal. Appl. 28, 170---189 (2006)

Digital Library

[15]

Dollar, H.S., Gould, N.I.M., Schilders, W.H.A., Wathen, A.J.: Using constraint preconditioners with regularized saddle-point problems. Comput. Optim. Appl. 36, 249---270 (2007)

Digital Library

[16]

Durazzi, C., Ruggiero, V.: Indefinitely preconditioned conjugate gradient method for large sparse equality and inequality constrained quadratic problems. Numer. Linear Algebra Appl. 10, 673---688 (2003)

[17]

Elman, H.C., Silvester, D.J., Wathen, A.J.: Finite Elements and Fast Iterative Solvers. Oxford University Press, Oxford (2005)

[18]

Forsgren, A.: Inertia-controlling factorizations for optimization algorithms. Appl. Numer. Math. 43, 91---107 (2002)

Digital Library

[19]

Forsgren, A., Gill, P.E., Griffin, J.D.: Iterative solution of augmented systems arising in interior methods. SIAM J. Optim. 18, 666---690 (2007)

Digital Library

[20]

Forsgren, A., Gill, P.E., Shinnerl, J.R.: Stability of symmetric ill-conditioned systems arising in interior methods for constrained optimization. SIAM J. Optim. 17, 187---211 (1996)

Digital Library

[21]

Friedlander, M.P., Orban, D.: A primal-dual regularization interior-point method for convex quadratic programs. Math. Program. Comput. 4, 71---107 (2012)

[22]

Gondzio, J.: Convergence analysis of an inexact feasible interior point method for convex quadratic programming. SIAM J. Optim. 23, 1510---1527 (2013)

[23]

Gould, N.I.M., Orban, D., Toint, PhL: Cuter (and sifdec), a contrained and unconstrained testing environment, revisited. ACM Trans. Math. Softw. 29, 373---394 (2003)

Digital Library

[24]

Gould, N.I.M., Simoncini, V.: Spectral analysis of saddle point matrices with indefinite leading blocks. SIAM J. Matrix Anal. Appl. 31, 1152---1171 (2009)

Digital Library

[25]

Greif, C., Moulding, E., Orban, D.: Bounds on eigenvalues of matrices arising from interior-point methods. SIAM J. Optim. 24, 49---83 (2014)

[26]

Greif, C., Schötzau, D.: Preconditioners for saddle point linear systems with highly singular (1, 1) blocks. Electron. Trans. Numer. Anal. 22, 114---121 (2006)

[27]

Higham, N.J.: Functions of Matrices: Theory and Computation. Society for Industrial and Applied Mathematics, Philadelphia, PA (2008)

Digital Library

[28]

Keller, C., Gould, N.I.M., Wathen, A.J.: Constraint preconditioning for indefinite linear systems. SIAM J. Matrix Anal. Appl. 21, 1300---1317 (2000)

Digital Library

[29]

Lukšan, L., Vlă¿ek, J.: Indefinitely preconditioned inexact Newton method for large sparse equality constrained nonlinear programming problems. Numer. Linear Algebra Appl. 5, 219---247 (1998)

[30]

Maros, I., Meszaros, C.: A repository of convex quadratic programming problems. Optim. Methods Softw. 11&12, 671---681 (1999)

[31]

Morini, B., Simoncini, V., Tani, M.: Spectral estimates for unreduced symmetric KKT systems arising from interior point methods. Numer. Linear Algebra Appl. 23, 776---800 (2016)

[32]

Paige, C.C., Saunders, M.A.: Solution of sparse indefinite systems of linear equations. SIAM J. Numer. Anal. 12, 617---629 (1975)

Digital Library

[33]

PDCO: Primal-Dual interior method for Convex Objectives. http://stanford.edu/group/software/pdco

[34]

Rees, T., Greif, C.: A preconditioner for linear systems arising from interior point optimization methods. SIAM J. Sci. Comput. 29, 1992---2007 (2007)

Digital Library

[35]

Rozlo¿ník, M., Simoncini, V.: Krylov subspace methods for saddle point problem with indefinite preconditioning. SIAM J. Matrix Anal. Appl. 24(2), 368---391 (2002)

Digital Library

[36]

Saad, Y., Schultz, M.H.: GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems. SIAM J. Sci. Stat. Comput. 7, 856---869 (1986)

[37]

Saunders, M.A.: Cholesky-based methods for sparse least-squares: the benefit of regularization. In: Adams, L., Nazareth, J.L. (eds.) Linear and Nonlinear Conjugate Gradient-Related Methods, pp. 92---100. SIAM, Philadelphia (1994)

[38]

Shen, S.Q., Huang, T.Z., Zhang, J.S.: Augmentation block triangular preconditioners for regularized saddle point problems. SIAM J. Matrix Anal. Appl. 33, 721---741 (2012)

[39]

Wright, S.J.: Stability of linear equations solvers in interior-point methods. SIAM J. Matrix Anal. Appl. 16, 1287---1307 (1995)

Digital Library

[40]

Wright, S.J.: Primal-Dual Interior-Point Methods. SIAM, Philadelphia (1997)

Digital Library

Cited By

Cipolla SGondzio J(2023)Proximal Stabilized Interior Point Methods and Low-Frequency-Update Preconditioning TechniquesJournal of Optimization Theory and Applications10.1007/s10957-023-02194-4197:3(1061-1103)Online publication date: 5-Apr-2023
https://dl.acm.org/doi/10.1007/s10957-023-02194-4
Ek DForsgren A(2023)A structured modified Newton approach for solving systems of nonlinear equations arising in interior-point methods for quadratic programmingComputational Optimization and Applications10.1007/s10589-023-00486-z86:1(1-48)Online publication date: 16-Jun-2023
https://dl.acm.org/doi/10.1007/s10589-023-00486-z
Huang N(2020)Variable parameter Uzawa method for solving a class of block three-by-three saddle point problemsNumerical Algorithms10.1007/s11075-019-00863-y85:4(1233-1254)Online publication date: 21-Jan-2020
https://dl.acm.org/doi/10.1007/s11075-019-00863-y
Show More Cited By

A comparison of reduced and unreduced KKT systems arising from interior point methods
1. Computing methodologies
  1. Symbolic and algebraic manipulation
    1. Symbolic and algebraic algorithms

Recommendations

Updating Constraint Preconditioners for KKT Systems in Quadratic Programming Via Low-Rank Corrections

This work focuses on the iterative solution of sequences of KKT linear systems arising in interior point methods applied to large convex quadratic programming problems. This task is the computational core of the interior point procedure, and an efficient ...
Iterative Solution of Augmented Systems Arising in Interior Methods

Iterative methods are proposed for certain augmented systems of linear equations that arise in interior methods for general nonlinear optimization. Interior methods define a sequence of KKT equations that represent the symmetrized (but indefinite) ...
An interior point-proximal method of multipliers for convex quadratic programming
Abstract
In this paper we combine an infeasible Interior Point Method (IPM) with the Proximal Method of Multipliers (PMM). The resulting algorithm (IP-PMM) is interpreted as a primal-dual regularized IPM, suitable for solving linearly constrained convex ...

Comments

Information & Contributors

Information

Published In

cover image Computational Optimization and Applications

Computational Optimization and Applications Volume 68, Issue 1

September 2017

189 pages

ISSN:0926-6003

Issue’s Table of Contents

Copyright © Copyright © 2017 Springer Science+Business Media, LLC.

Publisher

Kluwer Academic Publishers

United States

Publication History

Published: 01 September 2017

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 10 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Cipolla SGondzio J(2023)Proximal Stabilized Interior Point Methods and Low-Frequency-Update Preconditioning TechniquesJournal of Optimization Theory and Applications10.1007/s10957-023-02194-4197:3(1061-1103)Online publication date: 5-Apr-2023
https://dl.acm.org/doi/10.1007/s10957-023-02194-4
Ek DForsgren A(2023)A structured modified Newton approach for solving systems of nonlinear equations arising in interior-point methods for quadratic programmingComputational Optimization and Applications10.1007/s10589-023-00486-z86:1(1-48)Online publication date: 16-Jun-2023
https://dl.acm.org/doi/10.1007/s10589-023-00486-z
Huang N(2020)Variable parameter Uzawa method for solving a class of block three-by-three saddle point problemsNumerical Algorithms10.1007/s11075-019-00863-y85:4(1233-1254)Online publication date: 21-Jan-2020
https://dl.acm.org/doi/10.1007/s11075-019-00863-y
Axelsson OKarátson J(2020)Superior properties of the PRESB preconditioner for operators on two-by-two block form with square blocksNumerische Mathematik10.1007/s00211-020-01143-x146:2(335-368)Online publication date: 1-Oct-2020
https://dl.acm.org/doi/10.1007/s00211-020-01143-x
Gondzio JSobral F(2019)Quasi-Newton approaches to interior point methods for quadratic problemsComputational Optimization and Applications10.1007/s10589-019-00102-z74:1(93-120)Online publication date: 1-Sep-2019
https://dl.acm.org/doi/10.1007/s10589-019-00102-z
Liang ZZhang G(2019)Alternating positive semidefinite splitting preconditioners for double saddle point problemsCalcolo: a quarterly on numerical analysis and theory of computation10.1007/s10092-019-0322-756:3(1-17)Online publication date: 1-Sep-2019
https://dl.acm.org/doi/10.1007/s10092-019-0322-7
Mezzadri FGalligani E(2018)An inexact Newton method for solving complementarity problems in hydrodynamic lubricationCalcolo: a quarterly on numerical analysis and theory of computation10.1007/s10092-018-0244-955:1(1-28)Online publication date: 1-Mar-2018
https://dl.acm.org/doi/10.1007/s10092-018-0244-9
Morini BSimoncini V(2017)Stability and Accuracy of Inexact Interior Point Methods for Convex Quadratic ProgrammingJournal of Optimization Theory and Applications10.1007/s10957-017-1170-8175:2(450-477)Online publication date: 1-Nov-2017
https://dl.acm.org/doi/10.1007/s10957-017-1170-8

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents