Abstract
In this paper, we present a forward–backward splitting algorithm with additional inertial term for solving a strongly convex optimization problem of a certain type. The strongly convex objective function is assumed to be a sum of a non-smooth convex and a smooth convex function. This additional knowledge is used for deriving a worst-case convergence rate for the proposed algorithm. It is proved to be an optimal algorithm with linear rate of convergence. For certain problems this linear rate of convergence is better than the provably optimal worst-case rate of convergence for smooth strongly convex functions. We demonstrate the efficiency of the proposed algorithm in numerical experiments and examples from image processing.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
In general, this is analytically very challenging because \(\tilde{m}_\alpha \) also depends on \(\alpha \).
Although FISTA is an accelerated method, a comparison is not completely fair since it does not exploit the strong convexity of the problem.
References
Alvarez, F.: Weak convergence of a relaxed and inertial hybrid projection-proximal point algorithm for maximal monotone operators in Hilbert space. SIAM J. Optim. 14(3), 773–782 (2003)
Alvarez, F., Attouch, H.: An inertial proximal method for maximal monotone operators via discretization of a nonlinear oscillator with damping. Set-Valued Anal. 9(1–2), 3–11 (2001)
Attouch, H., Peypouquet, J., Redont, P.: A dynamical approach to an inertial forward-backward algorithm for convex minimization. SIAM J. Optim. 24(1), 232–256 (2014)
Bauschke, H.H., Combettes, P.L.: Convex Analysis and Monotone Operator Theory in Hilbert Spaces. CMS Books in Mathematics, Springer (2011)
Beck, A., Teboulle, M.: A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Appl. Math. 2(1), 183–202 (2009)
Bioucas-Dias, J.M., Figueiredo, M.: A new twist: two-step iterative shrinkage/thresholding algorithms for image restoration. IEEE Trans. Image Process. 16(12), 2992–3004 (2007)
Chambolle, A., Pock, T.: A first-order primal-dual algorithm for convex problems with applications to imaging. J. Math. Imaging Vision 40(1), 120–145 (2011)
Chambolle, A., Pock, T.: On the ergodic convergence rates of a first-order primal-dual algorithm (2014). to appear
Combettes, P.L., Wajs, V.R.: Signal recovery by proximal forward-backward splitting. Multiscale Modeling Simul. 4(4), 1168–1200 (2005)
Combettes, P.L., Dũng, D., Vũ, B.C.: Dualization of signal recovery problems. Set-Valued Var. Anal. 18(3–4), 373–404 (2010)
Daubechies, I., Defrise, M., De Mol, C.: An iterative thresholding algorithm for linear inverse problems with a sparsity constraint. Commun. Pure Appl. Math. 57(11), 1413–1457 (2004)
Drori, Y., Teboulle, M.: Performance of first-order methods for smooth convex minimization: a novel approach. Math. Program. 145(1–2), 451–482 (2014)
Gelfand, I.: Normierte Ringe. Rec. Math. [Mat. Sbornik] N.S. 9(51), 3–24 (1941)
Goldfarb, D., Ma, S.: Fast multiple-splitting algorithms for convex optimization. SIAM J. Optim. 22(2), 533–556 (2012)
He, B., Yuan, X.: On the \(O(1/n)\) convergence rate of the Douglas-Rachford alternating direction method. SIAM J. Numer. Anal. 50(2), 700–709 (2012)
Hong, M., Luo, Z.-Q.: On the linear convergence of the alternating direction method of multipliers. ArXiv e-prints, August (2012)
Mainberger, M., Weickert, J.: Edge-based image compression with homogeneous diffusion. In: Jiang, Xiaoyi, Petkov, Nicolai (eds.) Computer Analysis of Images and Patterns. Lecture Notes in Computer Science, vol. 5702, pp. 476–483. Springer, Berlin (2009)
Moudafi, A., Oliny, M.: Convergence of a splitting inertial proximal method for monotone operators. J. Comput. Appl. Math. 155, 447–454 (2003)
Nesterov, Y.: A method of solving a convex programming problem with convergence rate \(O(1/k^2)\). Soviet Math. Doklady 27, 372–376 (1983)
Nesterov, Y.: Introductory Lectures on Convex Optimization: A Basic Course. Applied Optimization, vol. 87. Kluwer Academic Publishers, Boston (2004)
Nesterov, Y.: Smooth minimization of non-smooth functions. Math. Program. 103(1), 127–152 (2005)
Nesterov, Y.: Efficiency of coordinate descent methods on huge-scale optimization problems. SIAM J. Optim. 22(2), 341–362 (2012)
Nesterov, Y.: Gradient methods for minimizing composite functions. Math. Program. 140(1), 125–161 (2013)
Ochs, P., Chen, Y., Brox, T., Pock, T.: ipiano: Inertial proximal algorithm for non-convex optimization. SIAM J. Imaging Sci. 7(2), 1388–1419 (2014)
Poljak, B.T.: Introduction to optimization. Optimization Software (1987)
Polyak, B.T.: Some methods of speeding up the convergence of iteration methods. USSR Comput. Math. Math. Phys. 4(5), 1–17 (1964)
Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Phys. D 60, 259–268 (1992)
Shor, N.Z.: Minimization Methods for Non-differentiable Functions. Springer-Verlag New York Inc, New York (1985)
Vegeta from Dragon Ball Z. http://1.bp.blogspot.com/-g3wpzDWE0QI/UbycqlJWjDI/AAAAAAAAAbU/ty0EZl0kqXw/s1600/VEGETA%2B1.jpg
Zavriev, S.K., Kostyuk, F.V.: Heavy-ball method in nonconvex optimization problems. Comput. Math. Model. 4(4), 336–341 (1993)
Acknowledgments
Thomas Pock acknowledges support from the Austrian science fund (FWF) under the START project BIVISION, No. Y729. Peter Ochs and Thomas Brox acknowledge funding by the German Research Foundation (DFG Grant BR 3815/5-1).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Ochs, P., Brox, T. & Pock, T. iPiasco: Inertial Proximal Algorithm for Strongly Convex Optimization. J Math Imaging Vis 53, 171–181 (2015). https://doi.org/10.1007/s10851-015-0565-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10851-015-0565-0