Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content

Calculus Rules of the Generalized Concave Kurdyka–Łojasiewicz Property

  • Published:
Journal of Optimization Theory and Applications Aims and scope Submit manuscript

Abstract

In this paper, we propose several calculus rules for the generalized concave Kurdyka–Łojasiewicz (KL) property, which generalize Li and Pong’s results for KL exponents. The optimal concave desingularizing function has various forms and may be nondifferentiable. Our calculus rules do not assume desingularizing functions to have any specific form nor differentiable, while the known results do. Several examples are also given to show that our calculus rules are applicable to a broader class of functions than the known ones.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

Notes

  1. Unlike many published articles, we use the adjective “concave” because the concavity of desingularizing function is an additional property useful for algorithmic applications and was not assumed in the seminal work on KL property, see, e.g., [6]. We appreciate the anonymous referee of the companion paper [19] for suggesting this terminology.

  2. We thank an anonymous reviewer for pointing out a redundant assumption on continuity in a previous version of this work.

References

  1. Attouch, H., Bolte, J.: On the convergence of the proximal algorithm for nonsmooth functions involving analytic features. Math. Program. 116, 5–16 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  2. Attouch, H., Bolte, J., Redont, P., Soubeyran, A.: Proximal alternating minimization and projection methods for nonconvex problems: an approach based on the Kurdyka-Łojasiewicz inequality. Math. Oper. Res. 35, 438–457 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  3. Attouch, H., Bolte, J., Svaiter, B.F.: Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward-backward splitting, and regularized Gauss-Seidel Methods. Math. Program. 137, 91–129 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  4. Banert, S., Bot, R.I.: A general double-proximal gradient algorithm for D.C. programming. Math. Program. 178, 301–326 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  5. Bolte, J., Daniilidis, A., Lewis, A.: The Łojasiewicz inequality for nonsmooth subanalytic functions with applications to subgradient dynamical systems. SIAM J. Optim. 17, 1205–1223 (2007)

    Article  MATH  Google Scholar 

  6. Bolte, J., Daniilidis, A., Ley, O., Mazet, L.: Characterizations of Łojasiewicz inequalities: subgradient flows, talweg, convexity. Trans. Am. Math. Soc. 362, 3319–3363 (2010)

    Article  MATH  Google Scholar 

  7. Bolte, J., Sabach, S., Teboulle, M.: Proximal alternating linearized minimization for nonconvex and nonsmooth problems. Math. Program. 146, 459–494 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  8. Boţ, R.I., Dao, M.N., Li, G.: Extrapolated proximal subgradient algorithms for nonconvex and nonsmooth fractional programs. Math. Oper. Res. 47, 2415–2443 (2022)

    Article  MathSciNet  MATH  Google Scholar 

  9. Drusvyatskiy, D., Ioffe, A.D., Lewis, A.S.: Nonsmooth optimization using Taylor-like models: error bounds, convergence, and termination criteria. Math. Program. 185, 357–383 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  10. Drusvyatskiy, D., Lewis, A.S.: Error bounds, quadratic growth, and linear convergence of proximal methods. Math. Oper. Res. 43, 919–948 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  11. Kurdyka, K.: On gradients of functions definable in o-minimal structures. Ann. l’inst. Fourier 48, 769–783 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  12. Li, G., Pong, T.K.: Calculus of the exponent of Kurdyka-Łojasiewicz inequality and its applications to linear convergence of first-order methods. Found. Comput. Math. 18, 1199–1232 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  13. Liu, H., So, A.M.-C., Wu, W.: Quadratic optimization with orthogonality constraint: explicit Łojasiewicz exponent and linear convergence of retraction-based line-search and stochastic variance-reduced gradient methods. Math. Program. 178, 215–262 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  14. Mordukhovich, B. S.: Variational Analysis and Applications, Springer (2018)

  15. Ochs, P., Chen, Y., Brox, T., Pock, T.: iPiano: inertial proximal algorithm for nonconvex optimization. SIAM J. Imaging Sci. 7, 1388–1419 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  16. Rockafellar, R.T., Wets, R.J.-B.: Variational Analysis. Springer-Verlag, Berlin (1998)

    Book  MATH  Google Scholar 

  17. Łojasiewicz, S.: Une propriété topologique des sous-ensembles analytiques réels. Les équations aux dérivées Partielles 117, 87–89 (1963)

    MATH  Google Scholar 

  18. Stromberg, K. R.: An Introduction to Classical Real Analysis, American Mathematical Society(2015)

  19. Wang, X., Wang, Z.: The exact modulus of the generalized Kurdyka-Łojasiewicz property. Math. Oper. Res. 47, 2765–2783 (2022)

    Article  MathSciNet  MATH  Google Scholar 

  20. Wen, B., Chen, X.J., Pong, T.K.: A proximal difference-of-convex algorithm with extrapolation. Comput. Optim. Appl. 69, 297–324 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  21. Wu, Y., Pan, S., Bi, S.: Kurdyka-Łojasiewicz property of zero-norm composite functions. J. Optim. Theory Appl. 188, 94–112 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  22. Yu, P., Li, G., Pong, T.K.: Kurdyka-Łojasiewicz exponent via inf-projection. Found. Comput. Math. 22, 1171–1217 (2022)

    Article  MathSciNet  MATH  Google Scholar 

  23. Yu, P., Pong, T.K., Lu, Z.: Convergence rate analysis of a sequential convex programming method with line search for a class of constrained difference-of-convex optimization problems. SIAM J. Optim. 31, 2024–2054 (2021)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

XW and ZW were partially supported by NSERC Discovery Grants. The authors thank Dr. Heinz H. Bauschke for many helpful discussions. The authors also would like to thank two anonymous referees for their insightful comments, which helped them fixing technical details and improving the quality of this manuscript.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Xianfu Wang.

Additional information

Communicated by Hedy Attouch.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

1.1 A Supplementary Lemmas

Lemma A.1

Let \(m\in {\mathbb {N}}\) obey \(m\ge 2\). For each \(i\in \{1,\ldots ,m\}\), Let \(h_i:(0,\infty )\rightarrow {\mathbb {R}}_+\). Suppose that for each i the function \(\varphi _i:[0,\infty )\rightarrow {\mathbb {R}}_+\) given by \(\varphi _i(t)=\int _0^th_i(s)\textrm{d}s\) for \(t\in (0,\infty )\) and \(\varphi _i(0)=0\), is finite and right-continuous at 0. Then, the function \(\varphi :[0,\infty )\rightarrow {\mathbb {R}}_+\),

$$\begin{aligned} (\forall 0<t<\eta )~t\mapsto \int _0^t\max _{1\le i\le m}h_i(s)\textrm{d}s, \end{aligned}$$

and \(\varphi (0)=0\), is finite and right-continuous at 0.

Proof

Let \(m=2\). Then, by using the inequality \(\max \{\alpha ,\beta \}\le \alpha +\beta \) for \(\alpha ,\beta \ge 0\), one has \(\max \{h_1(s),h_2(s)\}\le h_1(s)+h_2(s)\). Note that \(\varphi (t)=\lim _{u\rightarrow 0^+} \int _u^t\max \{ h_1(s),h_2(s)\}\textrm{d}s\) for \(t>0\). Hence, we have for \(t>0\)

$$\begin{aligned} \varphi (t){} & {} \le \lim _{u\rightarrow 0^+}\int _u^th_1(s)+h_2(s)\textrm{d}s=\varphi _1(t)+\varphi _2(t)-\lim _{u\rightarrow 0^+}\left[ \varphi _1(u)+\varphi _2(u)\right] \\{} & {} =\varphi _1(t)+\varphi _2(t)<\infty , \end{aligned}$$

where the last equality is implied by the right-continuity of \(\varphi _1\) and \(\varphi _2\) at 0. Taking \(t\rightarrow 0^+\), one gets \(\lim _{t\rightarrow 0^+}\varphi (t)=0\). The desired result then follows from a simple induction. \(\blacksquare \)

Lemma A.2

Let \(\eta \in (0,\infty ]\) and let \(\varphi \in \Phi _\eta \) be concave. Then, the following hold:

  1. (i)

    Let \(t>0\). Then \(\varphi (t)=\lim _{u\rightarrow 0^+}\int _u^t\varphi _-^\prime (s)\textrm{d}s=\int _0^t\varphi _-^\prime (s)\textrm{d}s\).

  2. (ii)

    The function \(t\mapsto \varphi _-^\prime (t)\) is decreasing and \(\varphi _-^\prime (t)>0\) for \(t\in (0,\eta )\). If in addition \(\varphi (t)\) is strictly concave, then \(t\mapsto \varphi _-^\prime (t)\) is strictly decreasing.

  3. (iii)

    For \(0\le s<t<\eta \), \(\varphi ^\prime _-(t)\le \frac{\varphi (t)-\varphi (s)}{t-s}\).

Proof

(i) Invoking Lemma 2.5(ii) yields

$$\begin{aligned} \varphi (t)=\lim _{u\rightarrow 0^+}\big (\varphi (t)-\varphi (u)\big )=\lim _{u\rightarrow 0^+}\int _{u}^t\varphi _-^\prime (s)\textrm{d}s<\infty , \end{aligned}$$

where the first equality holds because \(\varphi \) is right-continuous at 0 with \(\varphi (0)=0\). Let \((u_n)_{n\in {\mathbb {N}}}\) be a decreasing sequence with \(u_1<t\) such that \(u_n\rightarrow 0^+\) as \(n\rightarrow \infty \). For each n, define \(h_n:(0,t]\rightarrow {\mathbb {R}}_+\) by \(h_n(s)=\varphi _-^\prime (s)\) if \(s\in (u_n,t]\) and \(h_n(s)=0\) otherwise. Then, the sequence \((h_n)_{n\in {\mathbb {N}}}\) satisfies: (a) \(h_n\le h_{n+1}\) for every \(n\in {\mathbb {N}}\); (b) \(h_n(s)\rightarrow \varphi _-^\prime (s)\) pointwise on (0, t); (c) The integral \(\int _0^th_n(s)\textrm{d}s=\int _{u_n}^t\varphi _-^\prime (s)\textrm{d}s=\varphi (t)-\varphi (u_n)\le \varphi (t)-\varphi (0)<\infty \) for every \(n\in {\mathbb {N}}\). Hence, the monotone convergence theorem implies that

$$\begin{aligned} \lim _{u\rightarrow 0^+}\int _{u}^t\varphi _-^\prime (s)\textrm{d}s=\lim _{n\rightarrow \infty }\int _{u_n}^t\varphi _-^\prime (s)\textrm{d}s=\lim _{n\rightarrow \infty }\int _0^th_n(s)\textrm{d}s=\int _0^t\varphi _-^\prime (s)\textrm{d}s. \end{aligned}$$

(ii) According to Lemma 2.5 (i), the function \(t\mapsto \varphi _-^\prime (t)\) is decreasing. Suppose that \(\varphi _-^\prime (t_0)=0\) for some \(t_0\in (0,\eta )\). Then, by the monotonicity of \(\varphi _-^\prime \) and (i), we would have \(\varphi (t)-\varphi (t_0)=\int _{t_0}^t\varphi _-^\prime (s)\textrm{d}s\le (t-t_0)\varphi _-^\prime (t_0)=0\) for \(t>t_0\), which contradicts to the assumption that \(\varphi \) is strictly increasing.

(iii) For \(0<s<t<\eta \), applying Lemma 2.5 (iii) to the convex function \(-\varphi \) yields that \(-\varphi (s)+\varphi (t)\ge -\varphi _-^\prime (t)(s-t)\Leftrightarrow \varphi _-^\prime (t)\le \big (\varphi (t)-\varphi (s)\big )/(t-s)\). The desired inequality then follows from the right-continuity of \(\varphi \) at 0. \(\blacksquare \)

Lemma A.3

Let \(\eta \in (0,\infty ]\) and let \(h:(0,\eta )\rightarrow {\mathbb {R}}_+\) be a positive-valued decreasing function. Define \(\varphi (t)=\int _0^th(s)\textrm{d}s\) for \(t\in (0,\eta )\) and set \(\varphi (0)=0\). Suppose that \(\varphi (t)<\infty \) for \(t\in (0,\eta )\). Then, \(\varphi \) is a strictly increasing concave function on \([0,\eta )\) with

$$\begin{aligned} \varphi _-^\prime (t)\ge h(t) \end{aligned}$$

for \(t\in (0,\eta )\), and right-continuous at 0. If in addition h is a continuous function, then \(\varphi \) is \(C^1\) on \((0,\eta )\).

Proof Let \(0<t_0<t_1<\eta \). Then \(\varphi (t_1)-\varphi (t_0)=\int _{t_0}^{t_1}h(s)\textrm{d}s\ge (t_1-t_0)\cdot h(t_1)>0\), which means \(\varphi \) is strictly increasing. Applying [18, Theorem 6.79], one concludes that \(\varphi (t)\rightarrow \varphi (0)=0\) as \(t\rightarrow 0^+\). \(\blacksquare \)

1.2 B Properties of the Generalized Concave KL Property and Its Associated Exact Modulus

In this subsection, we recall some pleasant properties of the generalized concave KL property and its associated exact modulus for the sake of completeness.

Proposition B.1

[19, Proposition 2] Let \(f:{\mathbb {R}}^n\rightarrow {\overline{{\mathbb {R}}}}\) be proper lsc and let \({\bar{x}}\in {\text {dom}}\partial f\). Let U be a nonempty neighborhood of \({\bar{x}}\) and \(\eta \in (0,\infty ]\). Let \(\varphi \in \Phi _\eta \) and suppose that f has the generalized concave KL property at \({\bar{x}}\) with respect to U, \(\eta \) and \(\varphi \). Then, the exact modulus of the generalized concave KL property of f at \({\bar{x}}\) with respect to U and \(\eta \), denoted by \({\tilde{\varphi }}\), is well-defined and satisfies

$$\begin{aligned} {\tilde{\varphi }}(t)\le \varphi (t),~\forall t\in [0,\eta ). \end{aligned}$$

Moreover, the function f has the generalized concave KL property at \({\bar{x}}\) with respect to U, \(\eta \) and \({\tilde{\varphi }}\). Furthermore, the exact modulus \({\tilde{\varphi }}\) satisfies

$$\begin{aligned}{} & {} {\tilde{\varphi }}=\inf \\{} & {} \quad \big \{\varphi \in \Phi _\eta :\varphi \,\,\text {is a concave desingularizing function of} f \text {at} \,\,{\bar{x}} \text { with respect to}\,\, U \text { and } \eta \big \}. \end{aligned}$$

Example B.2

[19, Example 1] Let \(\rho >0\). Consider the function given by

$$\begin{aligned} f(x)= {\left\{ \begin{array}{ll} 2\rho |x|-3\rho ^2/2,&{}\text {if }|x|>\rho ;\\ |x|^2/2, &{}\text {if }|x|\le \rho . \end{array}\right. } \end{aligned}$$

Then, the function

$$\begin{aligned} {\tilde{\varphi }}(t)={\left\{ \begin{array}{ll}\sqrt{2t},&{}\text {if}0\le t\le \rho ^2/2;\\ t/(2\rho )+3\rho /4, &{}\text {if}t>\rho ^2/2,\end{array}\right. } \end{aligned}$$

is the exact modulus of the generalized concave KL property of f at \({\bar{x}}=0\) with respect to \(U={\mathbb {R}}\) and \(\eta =\infty \).

The next proposition restates an example from [5, Section 1] in our extended framework. We include a detailed proof for the sake of completeness.

Proposition B.3

Define \(f:{\mathbb {R}}\rightarrow {\mathbb {R}}\) by \(f(x)=e^{-1/x^2}\) for \(x\ne 0\) and \(f(0)=0\). Then, the following hold:

  1. (i)

    The function \({\tilde{\varphi }} (t)=\sqrt{-1/\ln (t)}\) for \(t>0\) and \({\tilde{\varphi }} (0)=0\) is the exact modulus of generalized KL property of f at \({\bar{x}}=0\) with respect to \(U=(-\sqrt{2/3},\sqrt{2/3})\) and \(\eta =\exp (-3/2)\).

  2. (ii)

    For every \(c>0\) and \(\theta \in [0,1)\), the function \(\varphi (t)=c\cdot t^{1-\theta }\) cannot be a desingularizing function of the generalized concave KL property of f at 0 with respect to any neighborhood \(U^\prime \ni 0\) and \(\eta ^\prime \in (0,\infty ]\).

Proof

  1. (i)

    For \(0<s\le \exp (-3/2)\), \(s\le \exp (-1/x^2)\Leftrightarrow |x|\ge \sqrt{-1/\ln {s}}\). Thus

    $$\begin{aligned} h(s)&=\sup \{|f^\prime (x)|^{-1}: x\in U\cap [0<f-f({\bar{x}})<\eta ], s\le f(x)-f({\bar{x}})\}\\&=\sup \{|2x^{-3}\exp (-1/x^2)|^{-1}: \sqrt{-1/\ln {s}}\le |x|<2/3\}=(-\ln (s))^{-3/2}/(2s). \end{aligned}$$

    Hence, \({\tilde{\varphi }} (t)=\sqrt{-1/\ln (t)}\) for \(t>0\).

  2. (ii)

    Suppose to the contrary that there were \(c>0\) and \(\theta \in [0,1)\) such that f has the generalized concave KL property at 0 with respect to some \(U^\prime \ni 0\) and \(\eta ^\prime >0\) and \(\varphi (t)\). Taking the intersection if necessary, assume without loss of generality that \(U^\prime \cap [0<f<\eta ^\prime ]\subseteq (-\sqrt{2/3},\sqrt{2/3})\cap [0<f<e^{-3/2}]\). Then, it is easy to see that f is convex and \(C^1\) on \(U^\prime \cap [0<f<\eta ^\prime ]\) with \({\tilde{\varphi }}(t)\) being the associated exact modulus. Hence, Proposition B.1 implies that

    $$\begin{aligned} {\tilde{\varphi }}(t)\le \varphi (t)=c\cdot t^{1-\theta },~\forall t\in (0,\min \{\eta ^\prime ,e^{-3/2}\}). \end{aligned}$$
    (20)

    Let \(s>0\). Then, \(s={\tilde{\varphi }}(t)\Leftrightarrow t=e^{-1/s^2}\), which further implies that

    $$\begin{aligned} \limsup _{t\rightarrow 0^+}\frac{{\tilde{\varphi }}(t)}{t^{1-\theta }}=\limsup _{s\rightarrow 0^+}\frac{s}{e^{-(1-\theta )/s^2}}=\limsup _{s\rightarrow 0^+}\frac{e^{(1-\theta )/s^2}}{s^{-1}}=\infty , \end{aligned}$$

    which contradicts to (20). \(\blacksquare \)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, X., Wang, Z. Calculus Rules of the Generalized Concave Kurdyka–Łojasiewicz Property. J Optim Theory Appl 197, 839–854 (2023). https://doi.org/10.1007/s10957-023-02219-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10957-023-02219-y

Keywords

Mathematics Subject Classification