Abstract
For the classical quadratic penalty, it is known that the distance from the solution of the penalty subproblem to the solution of the original problem is at worst inversely proportional to the value of the penalty parameter under the linear independence constraint qualification, strict complementarity, and the second-order sufficient optimality conditions. Moreover, using solutions of the penalty subproblem, one can obtain certain useful Lagrange multipliers estimates whose distance to the optimal ones is also at least inversely proportional to the value of the parameter. We show that the same properties hold more generally, namely, under the (weaker) strict Mangasarian–Fromovitz constraint qualification and second-order sufficiency (and without strict complementarity). Moreover, under the linear independence constraint qualification and strong second-order sufficiency (also without strict complementarity), we demonstrate local uniqueness and Lipschitz continuity of stationary points of penalty subproblems. In addition, those results follow from the analysis of general power penalty functions, of which quadratic penalty is a special case.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data Availibility Statement
Data sharing not applicable to this article as no data sets were generated or analyzed during the current study.
References
Avakov, E.R., Arutyunov, A.V., Izmailov, A.F.: On convergence rate estimates for power penalty methods. Comput. Math. Math. Phys. 44, 1684–1695 (2004)
Bertsekas, D.P.: On penalty and multiplier methods for constrained minimization. SIAM J. Control. Optim. 14, 216–235 (1976)
Bertsekas, D.P.: Constrained Optimization and Lagrange Multiplier Methods. Academic Press, New York (1982)
Birgin, E.G., Martínez, J.M.: Practical Augmented Lagrangian Methods for Constrained Optimization. Fundamentals of Algorithms, vol. 10. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA (2014)
Bonnans, J.F., Shapiro, A.: Perturbation Analysis of Optimization Problems. Springer, New York (2000)
Bonnans, J.F., Sulem, A.: Pseudopower expansion of solutions of generalized equations and constrained optimization. Math. Program. 70, 123–148 (1995)
Burke, J.V.: An exact penalization viewpoint of constrained optimization. SIAM J. Control. Optim. 29, 968–998 (1991)
Conn, A.R., Gould, N.I.M., Toint, Ph.L.: Trust-Region Methods. SIAM, Philadelphia (2000)
Courant, R.: Variational methods for the solution of problems of equilibrium and vibration. Bull. Am. Math. Soc. 49, 1–23 (1943)
Fernández, D., Solodov, M.V.: Local convergence of exact and inexact augmented Lagrangian methods under the second-order sufficient optimality condition. SIAM J. Optim. 22, 384–407 (2012)
Fiacco, A.V., McCormick, G.: Nonlinear Programming: Sequential Unconstrained Minimization Techniques. Wiley, New York (1968)
Fletcher, R.: Practical Methods of Optimization, 2nd edn. Wiley, New York (1987)
Gauvin, J.: A necessary and sufficient regularity conditions to have bounded multipliers in nonconvex programming. Math. Program. 12, 136–138 (1977)
Izmailov, A.F., Solodov, M.V.: Error bounds for 2-regular mappings with Lipschitzian derivatives and their applications. Math. Program. 89, 413–435 (2001)
Izmailov, A.F., Solodov, M.V.: Numerical Methods for Optimization. Fizmatlit, Moscow (2003). (In Russain)
Izmailov, A.F., Solodov, M.V.: Newton-type Methods for Optimization and Variational Problems. Springer, Cham (2014)
Luo, Z.-Q., Pang, J.-S., Ralph, D.: Mathematical Programs with Equilibrium Constraints. Cambridge University Press, Cambridge (1996)
Luo, Z.-Q., Pang, J.-S., Ralph, D., Wu, S.-Q.: Exact penalization and stationarity conditions of mathematical programs with equilibrium constraints. Math. Program. 75, 19–76 (1996)
Mangasarian, O.L.: Sufficiency of exact penalty minimization. SIAM J. Control. Optim. 23, 30–37 (1985)
Nocedal, J., Wright, S.J.: Numerical Optimization, 2nd edn. Springer, New York (2006)
Polyak, B.T.: Introduction to Optimization. Optimization Software Inc., Publications Division, New York (1987)
Polyak, B.T., Tret’yakov, N.V.: The method of penalty estimates for conditional extremum problems. USSR Comput. Math. Math. Phys. 13, 42–58 (1973)
Robinson, S.M.: Strongly regular generalized equations. Math. Oper. Res. 5, 43–62 (1980)
Robinson, S.M.: Generalized equations and their solutions, Part II: applications to nonlinear programming. Math. Program. Study 10, 200–221 (1982)
Ruszczynski, A.: Nonlinear Optimization. Princeton University Press, Princeton, NJ (2006)
Acknowledgements
The authors thank the three referees for very helpful comments and suggestions that led us to re-organize and improve the original version of the paper. Research of the second author was supported in part by CNPq Grant 303913/2019-3, by FAPERJ Grant E-26/200.347/2023, and by PRONEX–Optimization.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflicts of interest
The authors declare that they have no conflict of interest of any kind.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Izmailov, A.F., Solodov, M.V. Convergence rate estimates for penalty methods revisited. Comput Optim Appl 85, 973–992 (2023). https://doi.org/10.1007/s10589-023-00476-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10589-023-00476-1
Keywords
- Penalty function
- Quadratic penalty
- Convergence rate
- Strong regularity
- Linear independence constraint qualification
- Strict Mangasarian–Fromovitz constraint qualification
- Second-order sufficient optimality conditions