Abstract
In this paper, we propose several calculus rules for the generalized concave Kurdyka–Łojasiewicz (KL) property, which generalize Li and Pong’s results for KL exponents. The optimal concave desingularizing function has various forms and may be nondifferentiable. Our calculus rules do not assume desingularizing functions to have any specific form nor differentiable, while the known results do. Several examples are also given to show that our calculus rules are applicable to a broader class of functions than the known ones.
Similar content being viewed by others
Notes
Unlike many published articles, we use the adjective “concave” because the concavity of desingularizing function is an additional property useful for algorithmic applications and was not assumed in the seminal work on KL property, see, e.g., [6]. We appreciate the anonymous referee of the companion paper [19] for suggesting this terminology.
We thank an anonymous reviewer for pointing out a redundant assumption on continuity in a previous version of this work.
References
Attouch, H., Bolte, J.: On the convergence of the proximal algorithm for nonsmooth functions involving analytic features. Math. Program. 116, 5–16 (2009)
Attouch, H., Bolte, J., Redont, P., Soubeyran, A.: Proximal alternating minimization and projection methods for nonconvex problems: an approach based on the Kurdyka-Łojasiewicz inequality. Math. Oper. Res. 35, 438–457 (2010)
Attouch, H., Bolte, J., Svaiter, B.F.: Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward-backward splitting, and regularized Gauss-Seidel Methods. Math. Program. 137, 91–129 (2013)
Banert, S., Bot, R.I.: A general double-proximal gradient algorithm for D.C. programming. Math. Program. 178, 301–326 (2019)
Bolte, J., Daniilidis, A., Lewis, A.: The Łojasiewicz inequality for nonsmooth subanalytic functions with applications to subgradient dynamical systems. SIAM J. Optim. 17, 1205–1223 (2007)
Bolte, J., Daniilidis, A., Ley, O., Mazet, L.: Characterizations of Łojasiewicz inequalities: subgradient flows, talweg, convexity. Trans. Am. Math. Soc. 362, 3319–3363 (2010)
Bolte, J., Sabach, S., Teboulle, M.: Proximal alternating linearized minimization for nonconvex and nonsmooth problems. Math. Program. 146, 459–494 (2014)
Boţ, R.I., Dao, M.N., Li, G.: Extrapolated proximal subgradient algorithms for nonconvex and nonsmooth fractional programs. Math. Oper. Res. 47, 2415–2443 (2022)
Drusvyatskiy, D., Ioffe, A.D., Lewis, A.S.: Nonsmooth optimization using Taylor-like models: error bounds, convergence, and termination criteria. Math. Program. 185, 357–383 (2021)
Drusvyatskiy, D., Lewis, A.S.: Error bounds, quadratic growth, and linear convergence of proximal methods. Math. Oper. Res. 43, 919–948 (2018)
Kurdyka, K.: On gradients of functions definable in o-minimal structures. Ann. l’inst. Fourier 48, 769–783 (1998)
Li, G., Pong, T.K.: Calculus of the exponent of Kurdyka-Łojasiewicz inequality and its applications to linear convergence of first-order methods. Found. Comput. Math. 18, 1199–1232 (2018)
Liu, H., So, A.M.-C., Wu, W.: Quadratic optimization with orthogonality constraint: explicit Łojasiewicz exponent and linear convergence of retraction-based line-search and stochastic variance-reduced gradient methods. Math. Program. 178, 215–262 (2019)
Mordukhovich, B. S.: Variational Analysis and Applications, Springer (2018)
Ochs, P., Chen, Y., Brox, T., Pock, T.: iPiano: inertial proximal algorithm for nonconvex optimization. SIAM J. Imaging Sci. 7, 1388–1419 (2014)
Rockafellar, R.T., Wets, R.J.-B.: Variational Analysis. Springer-Verlag, Berlin (1998)
Łojasiewicz, S.: Une propriété topologique des sous-ensembles analytiques réels. Les équations aux dérivées Partielles 117, 87–89 (1963)
Stromberg, K. R.: An Introduction to Classical Real Analysis, American Mathematical Society(2015)
Wang, X., Wang, Z.: The exact modulus of the generalized Kurdyka-Łojasiewicz property. Math. Oper. Res. 47, 2765–2783 (2022)
Wen, B., Chen, X.J., Pong, T.K.: A proximal difference-of-convex algorithm with extrapolation. Comput. Optim. Appl. 69, 297–324 (2018)
Wu, Y., Pan, S., Bi, S.: Kurdyka-Łojasiewicz property of zero-norm composite functions. J. Optim. Theory Appl. 188, 94–112 (2021)
Yu, P., Li, G., Pong, T.K.: Kurdyka-Łojasiewicz exponent via inf-projection. Found. Comput. Math. 22, 1171–1217 (2022)
Yu, P., Pong, T.K., Lu, Z.: Convergence rate analysis of a sequential convex programming method with line search for a class of constrained difference-of-convex optimization problems. SIAM J. Optim. 31, 2024–2054 (2021)
Acknowledgements
XW and ZW were partially supported by NSERC Discovery Grants. The authors thank Dr. Heinz H. Bauschke for many helpful discussions. The authors also would like to thank two anonymous referees for their insightful comments, which helped them fixing technical details and improving the quality of this manuscript.
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Hedy Attouch.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendix
Appendix
1.1 A Supplementary Lemmas
Lemma A.1
Let \(m\in {\mathbb {N}}\) obey \(m\ge 2\). For each \(i\in \{1,\ldots ,m\}\), Let \(h_i:(0,\infty )\rightarrow {\mathbb {R}}_+\). Suppose that for each i the function \(\varphi _i:[0,\infty )\rightarrow {\mathbb {R}}_+\) given by \(\varphi _i(t)=\int _0^th_i(s)\textrm{d}s\) for \(t\in (0,\infty )\) and \(\varphi _i(0)=0\), is finite and right-continuous at 0. Then, the function \(\varphi :[0,\infty )\rightarrow {\mathbb {R}}_+\),
and \(\varphi (0)=0\), is finite and right-continuous at 0.
Proof
Let \(m=2\). Then, by using the inequality \(\max \{\alpha ,\beta \}\le \alpha +\beta \) for \(\alpha ,\beta \ge 0\), one has \(\max \{h_1(s),h_2(s)\}\le h_1(s)+h_2(s)\). Note that \(\varphi (t)=\lim _{u\rightarrow 0^+} \int _u^t\max \{ h_1(s),h_2(s)\}\textrm{d}s\) for \(t>0\). Hence, we have for \(t>0\)
where the last equality is implied by the right-continuity of \(\varphi _1\) and \(\varphi _2\) at 0. Taking \(t\rightarrow 0^+\), one gets \(\lim _{t\rightarrow 0^+}\varphi (t)=0\). The desired result then follows from a simple induction. \(\blacksquare \)
Lemma A.2
Let \(\eta \in (0,\infty ]\) and let \(\varphi \in \Phi _\eta \) be concave. Then, the following hold:
-
(i)
Let \(t>0\). Then \(\varphi (t)=\lim _{u\rightarrow 0^+}\int _u^t\varphi _-^\prime (s)\textrm{d}s=\int _0^t\varphi _-^\prime (s)\textrm{d}s\).
-
(ii)
The function \(t\mapsto \varphi _-^\prime (t)\) is decreasing and \(\varphi _-^\prime (t)>0\) for \(t\in (0,\eta )\). If in addition \(\varphi (t)\) is strictly concave, then \(t\mapsto \varphi _-^\prime (t)\) is strictly decreasing.
-
(iii)
For \(0\le s<t<\eta \), \(\varphi ^\prime _-(t)\le \frac{\varphi (t)-\varphi (s)}{t-s}\).
Proof
(i) Invoking Lemma 2.5(ii) yields
where the first equality holds because \(\varphi \) is right-continuous at 0 with \(\varphi (0)=0\). Let \((u_n)_{n\in {\mathbb {N}}}\) be a decreasing sequence with \(u_1<t\) such that \(u_n\rightarrow 0^+\) as \(n\rightarrow \infty \). For each n, define \(h_n:(0,t]\rightarrow {\mathbb {R}}_+\) by \(h_n(s)=\varphi _-^\prime (s)\) if \(s\in (u_n,t]\) and \(h_n(s)=0\) otherwise. Then, the sequence \((h_n)_{n\in {\mathbb {N}}}\) satisfies: (a) \(h_n\le h_{n+1}\) for every \(n\in {\mathbb {N}}\); (b) \(h_n(s)\rightarrow \varphi _-^\prime (s)\) pointwise on (0, t); (c) The integral \(\int _0^th_n(s)\textrm{d}s=\int _{u_n}^t\varphi _-^\prime (s)\textrm{d}s=\varphi (t)-\varphi (u_n)\le \varphi (t)-\varphi (0)<\infty \) for every \(n\in {\mathbb {N}}\). Hence, the monotone convergence theorem implies that
(ii) According to Lemma 2.5 (i), the function \(t\mapsto \varphi _-^\prime (t)\) is decreasing. Suppose that \(\varphi _-^\prime (t_0)=0\) for some \(t_0\in (0,\eta )\). Then, by the monotonicity of \(\varphi _-^\prime \) and (i), we would have \(\varphi (t)-\varphi (t_0)=\int _{t_0}^t\varphi _-^\prime (s)\textrm{d}s\le (t-t_0)\varphi _-^\prime (t_0)=0\) for \(t>t_0\), which contradicts to the assumption that \(\varphi \) is strictly increasing.
(iii) For \(0<s<t<\eta \), applying Lemma 2.5 (iii) to the convex function \(-\varphi \) yields that \(-\varphi (s)+\varphi (t)\ge -\varphi _-^\prime (t)(s-t)\Leftrightarrow \varphi _-^\prime (t)\le \big (\varphi (t)-\varphi (s)\big )/(t-s)\). The desired inequality then follows from the right-continuity of \(\varphi \) at 0. \(\blacksquare \)
Lemma A.3
Let \(\eta \in (0,\infty ]\) and let \(h:(0,\eta )\rightarrow {\mathbb {R}}_+\) be a positive-valued decreasing function. Define \(\varphi (t)=\int _0^th(s)\textrm{d}s\) for \(t\in (0,\eta )\) and set \(\varphi (0)=0\). Suppose that \(\varphi (t)<\infty \) for \(t\in (0,\eta )\). Then, \(\varphi \) is a strictly increasing concave function on \([0,\eta )\) with
for \(t\in (0,\eta )\), and right-continuous at 0. If in addition h is a continuous function, then \(\varphi \) is \(C^1\) on \((0,\eta )\).
Proof Let \(0<t_0<t_1<\eta \). Then \(\varphi (t_1)-\varphi (t_0)=\int _{t_0}^{t_1}h(s)\textrm{d}s\ge (t_1-t_0)\cdot h(t_1)>0\), which means \(\varphi \) is strictly increasing. Applying [18, Theorem 6.79], one concludes that \(\varphi (t)\rightarrow \varphi (0)=0\) as \(t\rightarrow 0^+\). \(\blacksquare \)
1.2 B Properties of the Generalized Concave KL Property and Its Associated Exact Modulus
In this subsection, we recall some pleasant properties of the generalized concave KL property and its associated exact modulus for the sake of completeness.
Proposition B.1
[19, Proposition 2] Let \(f:{\mathbb {R}}^n\rightarrow {\overline{{\mathbb {R}}}}\) be proper lsc and let \({\bar{x}}\in {\text {dom}}\partial f\). Let U be a nonempty neighborhood of \({\bar{x}}\) and \(\eta \in (0,\infty ]\). Let \(\varphi \in \Phi _\eta \) and suppose that f has the generalized concave KL property at \({\bar{x}}\) with respect to U, \(\eta \) and \(\varphi \). Then, the exact modulus of the generalized concave KL property of f at \({\bar{x}}\) with respect to U and \(\eta \), denoted by \({\tilde{\varphi }}\), is well-defined and satisfies
Moreover, the function f has the generalized concave KL property at \({\bar{x}}\) with respect to U, \(\eta \) and \({\tilde{\varphi }}\). Furthermore, the exact modulus \({\tilde{\varphi }}\) satisfies
Example B.2
[19, Example 1] Let \(\rho >0\). Consider the function given by
Then, the function
is the exact modulus of the generalized concave KL property of f at \({\bar{x}}=0\) with respect to \(U={\mathbb {R}}\) and \(\eta =\infty \).
The next proposition restates an example from [5, Section 1] in our extended framework. We include a detailed proof for the sake of completeness.
Proposition B.3
Define \(f:{\mathbb {R}}\rightarrow {\mathbb {R}}\) by \(f(x)=e^{-1/x^2}\) for \(x\ne 0\) and \(f(0)=0\). Then, the following hold:
-
(i)
The function \({\tilde{\varphi }} (t)=\sqrt{-1/\ln (t)}\) for \(t>0\) and \({\tilde{\varphi }} (0)=0\) is the exact modulus of generalized KL property of f at \({\bar{x}}=0\) with respect to \(U=(-\sqrt{2/3},\sqrt{2/3})\) and \(\eta =\exp (-3/2)\).
-
(ii)
For every \(c>0\) and \(\theta \in [0,1)\), the function \(\varphi (t)=c\cdot t^{1-\theta }\) cannot be a desingularizing function of the generalized concave KL property of f at 0 with respect to any neighborhood \(U^\prime \ni 0\) and \(\eta ^\prime \in (0,\infty ]\).
Proof
-
(i)
For \(0<s\le \exp (-3/2)\), \(s\le \exp (-1/x^2)\Leftrightarrow |x|\ge \sqrt{-1/\ln {s}}\). Thus
$$\begin{aligned} h(s)&=\sup \{|f^\prime (x)|^{-1}: x\in U\cap [0<f-f({\bar{x}})<\eta ], s\le f(x)-f({\bar{x}})\}\\&=\sup \{|2x^{-3}\exp (-1/x^2)|^{-1}: \sqrt{-1/\ln {s}}\le |x|<2/3\}=(-\ln (s))^{-3/2}/(2s). \end{aligned}$$Hence, \({\tilde{\varphi }} (t)=\sqrt{-1/\ln (t)}\) for \(t>0\).
-
(ii)
Suppose to the contrary that there were \(c>0\) and \(\theta \in [0,1)\) such that f has the generalized concave KL property at 0 with respect to some \(U^\prime \ni 0\) and \(\eta ^\prime >0\) and \(\varphi (t)\). Taking the intersection if necessary, assume without loss of generality that \(U^\prime \cap [0<f<\eta ^\prime ]\subseteq (-\sqrt{2/3},\sqrt{2/3})\cap [0<f<e^{-3/2}]\). Then, it is easy to see that f is convex and \(C^1\) on \(U^\prime \cap [0<f<\eta ^\prime ]\) with \({\tilde{\varphi }}(t)\) being the associated exact modulus. Hence, Proposition B.1 implies that
$$\begin{aligned} {\tilde{\varphi }}(t)\le \varphi (t)=c\cdot t^{1-\theta },~\forall t\in (0,\min \{\eta ^\prime ,e^{-3/2}\}). \end{aligned}$$(20)Let \(s>0\). Then, \(s={\tilde{\varphi }}(t)\Leftrightarrow t=e^{-1/s^2}\), which further implies that
$$\begin{aligned} \limsup _{t\rightarrow 0^+}\frac{{\tilde{\varphi }}(t)}{t^{1-\theta }}=\limsup _{s\rightarrow 0^+}\frac{s}{e^{-(1-\theta )/s^2}}=\limsup _{s\rightarrow 0^+}\frac{e^{(1-\theta )/s^2}}{s^{-1}}=\infty , \end{aligned}$$which contradicts to (20). \(\blacksquare \)
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Wang, X., Wang, Z. Calculus Rules of the Generalized Concave Kurdyka–Łojasiewicz Property. J Optim Theory Appl 197, 839–854 (2023). https://doi.org/10.1007/s10957-023-02219-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10957-023-02219-y
Keywords
- The generalized concave Kurdyka–Łojasiewicz property
- Optimal concave desingularizing function
- Calculus rules
- The Kurdyka–Łojasiewicz property.