Mathematics > Optimization and Control
[Submitted on 4 Apr 2024]
Title:A unified Euler--Lagrange system for analyzing continuous-time accelerated gradient methods
View PDF HTML (experimental)Abstract:This paper presents an Euler--Lagrange system for a continuous-time model of the accelerated gradient methods in smooth convex optimization and proposes an associated Lyapunov-function-based convergence analysis framework. Recently, ordinary differential equations (ODEs) with dumping terms have been developed to intuitively interpret the accelerated gradient methods, and the design of unified model describing the various individual ODE models have been examined. In existing reports, the Lagrangian, which results in the Euler-Lagrange equation, and the Lyapunov function for the convergence analysis have been separately proposed for each ODE. This paper proposes a unified Euler--Lagrange system and its Lyapunov function to cover the existing various models. In the convergence analysis using the Lyapunov function, a condition that parameters in the Lagrangian and Lyapunov function must satisfy is derived, and a parameter design for improving the convergence rate naturally results in the mysterious dumping coefficients. Especially, a symmetric Bregman divergence can lead to a relaxed condition of the parameters and a resulting improved convergence rate. As an application of this study, a slight modification in the Lyapunov function establishes the similar convergence proof for ODEs with smooth approximation in nondifferentiable objective function minimization.
References & Citations
Bibliographic and Citation Tools
Bibliographic Explorer (What is the Explorer?)
Litmaps (What is Litmaps?)
scite Smart Citations (What are Smart Citations?)
Code, Data and Media Associated with this Article
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub (What is DagsHub?)
Gotit.pub (What is GotitPub?)
Papers with Code (What is Papers with Code?)
ScienceCast (What is ScienceCast?)
Demos
Recommenders and Search Tools
Influence Flower (What are Influence Flowers?)
Connected Papers (What is Connected Papers?)
CORE Recommender (What is CORE?)
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.