Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
Springer Nature Link
Log in

On the pervasiveness of difference-convexity in optimization and statistics

  • Full Length Paper
  • Series B
  • Published:
Mathematical Programming Submit manuscript

A Correction to this article was published on 01 March 2019

This article has been updated

Abstract

With the increasing interest in applying the methodology of difference-of-convex (dc) optimization to diverse problems in engineering and statistics, this paper establishes the dc property of many functions in various areas of applications not previously known to be of this class. Motivated by a quadratic programming based recourse function in two-stage stochastic programming, we show that the (optimal) value function of a copositive (thus not necessarily convex) quadratic program is dc on the domain of finiteness of the program when the matrix in the objective function’s quadratic term and the constraint matrix are fixed. The proof of this result is based on a dc decomposition of a piecewise \(\hbox {LC}^1\) function (i.e., functions with Lipschitz gradients). Armed with these new results and known properties of dc functions existed in the literature, we show that many composite statistical functions in risk analysis, including the value-at-risk (VaR), conditional value-at-risk (CVaR), optimized certainty equivalent, and the expectation-based, VaR-based, and CVaR-based random deviation functionals are all dc. Adding the known class of dc surrogate sparsity functions that are employed as approximations of the \(\ell _0\) function in statistical learning, our work significantly expands the classes of dc functions and positions them for fruitful applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Change history

  • 07 March 2019

    After the publication of the DOI version of

References

  1. Ahn, M., Pang, J.S., Xin, J.: Difference-of-convex learning: directional stationarity, optimality, and sparsity. SIAM J. Optim. 27, 1637–1665 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  2. Alexandroff, A.D.: Surfaces represented by the difference of convex functions. Doklady Akademii Nauk SSSR (N.S.) 72, 613–616 (1950). [English translation: Siberian Èlektron. Mathetical. Izv. 9, 360–376 (2012)]

  3. Alvarado, A., Scutari, G., Pang, J.S.: A new decomposition method for multiuser DC-programming and its application to physical layer security. IEEE Trans. Signal Process. 62, 2984–2998 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  4. Bačák, M., Borwein, J.M.: On difference convexity of locally Lipschitz functions. Optimization 60, 961–978 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  5. Ben-Tal, A., Teboulle, M.: Expected utility, penalty functions and duality in stochastic nonlinear programmi. Manage. Sci. 32, 14451466 (1986)

    Article  MATH  Google Scholar 

  6. Ben-Tal, A., Teboulle, M.: Penalty functions and duality in stochastic programming via \(\varphi \)-divergence functionals. Math. Oper. Res. 12, 224–240 (1987)

    Article  MathSciNet  MATH  Google Scholar 

  7. Ben-Tal, A., Teboulle, M.: An old-new concept of convex risk measures: the optimized certainty equivalent. Math. Finance 17, 449–476 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  8. Birge, J.R., Louveaux, F.: Introduction to Stochastic Programming. Springer Series in Operations Research. Springer, New York (1997)

    MATH  Google Scholar 

  9. Chang, T.-H., Hong, M., Pang, J.S.: Local minimizers and second-order conditions in composite piecewise programming via directional derivatives. Preprint arXiv:1709.05758 (2017)

  10. Clarke, F.H.: Optimization and Nonsmooth Analysis. Wiley, New York (1983)

    MATH  Google Scholar 

  11. Cui, Y., Pang, J.-S., Sen, B.: Composite difference-max programs for modern statistical estimation problems. Preprint arXiv:1803.00205 (2018)

  12. Eaves, B.C.: On quadratic programming. Manage. Sci. 17, 698–711 (1971)

    Article  MATH  Google Scholar 

  13. Facchinei, F., Pang, J.S.: Finite-Dimensional Variational Inequalities and Complementarity Problems, Volumes I and II. Springer, New York (2003)

    MATH  Google Scholar 

  14. Fan, J., Xue, L., Zou, H.: Strong oracle optimality of folded concave penalized estimation. Ann. Stat. 42(2014), 819–849 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  15. Giannessi, F., Tomasin, E.: Nonconvex quadratic programs, linear complementarity problems, and integer linear programs. In: Conti, R., Ruberti, A. (eds) Fifth Conference on Optimization Techniques (Rome 1973), Part I, Lecture Notes in Computer Science, Vol. 3, pp. 437–449. Springer, Berlin (1973)

  16. Gotoh, J., Takeda, A., Tono, K.: DC formulations and algorithms for sparse optimization problems. Math. Program. Ser. B. (2017). https://doi.org/10.1007/s10107-017-1181-0

  17. Hahn, G., Banergjee, M., Sen, B.: Parameter Estimation and Inference in a Continuous Piecewise Linear Regression Model. Manuscript, Department of Statistics. Columbia University, New York (2016)

    Google Scholar 

  18. Hartman, P.: On functions representable as a difference of convex functions. Pac. J. Math. 9, 707–713 (1959)

    Article  MathSciNet  MATH  Google Scholar 

  19. Hiriart-Urruty, J.B.: Generalized differentiability, duality and optimization for problems dealing with differences of convex functions. In: Ponstein, J. (eds) Convexity and Duality in Optimization. Proceedings of the Symposium on Convexity and Duality in Optimization Held at the University of Groningen, The Netherlands June 22, 1984, pp. 37–70 (1985)

  20. Horst, R., Thoai, N.V.: D.C. programming: overview. J. Optim. Theory Appl. 103, 1–43 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  21. Hu, J., Mitchell, J.E., Pang, J.S.: An LPCC approach to nonconvex quadratic programs. Math. Program. 133, 243–277 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  22. Jara-Moroni, F., Pang, J.-S., Wächter, A.: A study of the difference-of-convex approach for solving linear programs with complementarity constraints. Math. Program. 169(1), 221–254 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  23. Le Thi, H.A., Pham, D.T.: The DC (difference of convex functions) programming and DCA revisited with DC models of real world nonconvex optimization problems. Ann. Oper. Res. 133, 25–46 (2005)

    MathSciNet  MATH  Google Scholar 

  24. Le Thi, H.A., Pham, D.T.: Recent advances in DC programming and DCA. Trans. Comput. Collect. Intell. 8342, 1–37 (2014)

    Google Scholar 

  25. Le Thi, H.A., Pham, D.T.: DC Programming and DCA: Thirty years of Developments Manuscript. University of Lorraine, Lorraine (2016)

    MATH  Google Scholar 

  26. Le Thi, H.A., Pham, D.T., Vo, X.T.: DC approximation approaches for sparse optimization. Eur. J. Oper. Res. 244, 26–46 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  27. Lee, G.M., Tam, N.N., Yen, N.D.: Quadratic Programming and Affine Variational Inequalities A Qualitative Study. Springer, New York (2005)

    MATH  Google Scholar 

  28. Luo, Z.Q., Tseng, P.: Error bound and convergence analysis of matrix splitting algorithms for the affine variational inequality problem. SIAM J. Optim. 2, 43–54 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  29. Luo, Z.Q., Pang, J.S., Ralph, D.: Mathematical Programs With Equilibrium Constraints. Cambridge University Press, Cambridge (1996)

    Book  MATH  Google Scholar 

  30. Nash, J.: Non-cooperative games. Ann. Math. 54(2), 286–295 (1951)

    Article  MathSciNet  MATH  Google Scholar 

  31. Orgeta, J.M., Rheinboldt, W.C.: Iterative Solution of Nonlinear Equations in Several Variables. Classics in Applied Mathematics. SIAM Publications, Philadelphia (2000)

    Google Scholar 

  32. Ovchinnikov, S.: Max-min representation of piecewise linear functions. Contrib. Algebra Geom. 43, 297–302 (2002)

    MathSciNet  MATH  Google Scholar 

  33. Pang, J.S., Tao, M.: Decomposition methods for computing directional stationary solutions of a class of non-smooth nonconvex optimization problems. SIAM J. Optim. Submitted January (2017)

  34. Pang, J.S., Razaviyayn, M., Alvarado, A.: Computing B-stationary points of nonsmooth dc programs. Math. Oper. Res. 42, 95–118 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  35. Pang, J.-S., Sen, S., Shanbhag, U.V.: Two-stage non-cooperative games with risk-averse players. Math. Program. 165(1), 235–290 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  36. Pham, D.T., Le Thi, H.A.: Convex analysis approach to DC programming: theory, algorithm and applications. Acta Math. Vietnam. 22, 289–355 (1997)

    MathSciNet  MATH  Google Scholar 

  37. Razaviyayn, M., Hong, M., Luo, Z.Q., Pang, J.S.: Parallel successive convex approximation for nonsmooth nonconvex optimization. Adv. Neural Inf. Process. Syst. (NIPS), 1440–1448 (2014)

  38. Razaviyayn, M.: Successive Convex Approximations: Analysis and Applications. Ph.D. Dissertation. Department of Electrical and Computer Engineering. University of Minnesota (Minneapolis 2014)

  39. Razaviyayn, M., Hong, M., Luo, Z.Q.: A unified convergence analysis of block successive minimization methods for nonsmooth optimization. SIAM J. Optim. 23, 1126–1153 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  40. Razaviyayn, M., Hong, M., Luo, Z.Q., Pang, J.S.: A unified algorithmic framework for block-structured optimization involving big data. IEEE Signal Process. Mag. 33, 57–77 (2016)

    Article  Google Scholar 

  41. Razaviyayn, M., Sanjabi, M., Luo, Z.Q.: A stochastic successive minimization method for nonsmooth nonconvex optimization with applications to transceiver design in wireless communication networks. Math. Program. Ser. B 157, 515–545 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  42. Rockafellar, R.T., Uryasev, S.: Optimization of conditional value-at-risk. J. Risk 2, 21–41 (2000)

    Article  Google Scholar 

  43. Rockafellar, R.T., Uryasev, S.: Conditional value-at-risk for general loss distributions. J. Bank. Finance 7, 1143–1471 (2002)

    Google Scholar 

  44. Rockafellar, R.T., Uryasev, S.: The fundamental risk quadrangle in risk management, optimization, and statistical estimation. Surv. Oper. Res. Manag. Sci. 18, 33–53 (2013)

    MathSciNet  Google Scholar 

  45. Rockafellar, R.T., Uryasev, S., Zabarankin, M.: Optimality conditions in portfolio analysis with general deviation measures. Math. Program. Ser. B 108, 515–540 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  46. Rockafellar, R.T., Uryasev, S., Zabarankin, M.: Generalized deviations in risk analysis. Finance Stochast. 10, 51–74 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  47. Sarykalin, S., Uryasev, S.: Value-at-risk versus conditional value-at-risk in risk management and optimization. Tutor. Oper. Res. 2008, 269–294 (2008)

    Google Scholar 

  48. Scholtes, S.: Intoduction to Piecewise Differentiable Functions. Springer, Berlin (2012)

    Book  MATH  Google Scholar 

  49. Scutari, G., Facchinei, F., Palomar, D.P., Pang, J.S., Song, P.: Decomposition by partial linearization: parallel optimization of multiuser systems. IEEE Trans. Signal Process. 62, 641–656 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  50. Scutari, G., Alvarado, A., Pang, J.S.: A new decomposition method for multiuser DC-programming and its application to physical layer security. IEEE Trans. Signal Process. 62, 2984–2998 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  51. Shapiro, A., Dentcheva, D., Ruszczynski, A.: Lectures on Stochastic Programming: Modeling and Theory. SIAM Publications, Philadelphia (2009)

    Book  MATH  Google Scholar 

  52. Sun, J.: On the structure of convex piecewise quadratic functions. J. Optim. Theory Appl. 72, 499–510 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  53. Tuy, H.: Convex Analysis and Global Optimization, 2nd edn, vol. 110. Springer Optimization and its Applications (2016). [First edition. Kluwer Publishers (Dordrecht 1998)]

  54. Tuy, H.: Global minimization of a difference of two convex functions. Math. Program. Study 30, 150–187 (1987)

    Article  MathSciNet  MATH  Google Scholar 

  55. Veselý, L., Zajíček, L.: Delta-convex mappings between Banach spaces and applications. Dissertationes Math. (Rozprawy Mat.) 289 (1989)

  56. Veselý, L., Zajíček, L.: On composition of DC functions and mappings. J. Conv. Anal. 16(2), 423–439 (2009)

    MATH  Google Scholar 

  57. Wozabal, D.: Value-at-risk optimization using the difference of convex algorithm. OR Spectr. 34, 861–883 (2012)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

The second author gratefully acknowledges the discussion with Professors Le Thi Hoi An and Pham Dinh Tao in the early stage of this work during his visit to the Université de Lorraine, Metz in June 2016. The three authors acknowledge their fruitful discussion with Professor Defeng Sun at the National University of Singapore during his visit to the University of Southern California. They are also grateful to Professor Marc Teboulle for drawing their attention to the references [5,6,7] that introduce and revisit the OCE. The constructive comments of two referees are also gratefully acknowledged. In particular, the authors are particularly grateful to a referee who has been very patient with our repeated misunderstanding of the work [57] that is now correctly summarized at the end of Sect. 3.1.

Author information

Authors and Affiliations

  1. Daniel J. Epstein Department of Industrial and Systems Engineering, University of Southern California, Los Angeles, CA, 90089-0193, USA

    Maher Nouiehed, Jong-Shi Pang & Meisam Razaviyayn

Authors
  1. Maher Nouiehed
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jong-Shi Pang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Meisam Razaviyayn
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Meisam Razaviyayn.

Additional information

J.-S. Pang: This work of this author was based on research partially supported by the U.S. National Science Foundation Grants CMMI 1538605 and IIS-1632971.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nouiehed, M., Pang, JS. & Razaviyayn, M. On the pervasiveness of difference-convexity in optimization and statistics. Math. Program. 174, 195–222 (2019). https://doi.org/10.1007/s10107-018-1286-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10107-018-1286-0

Mathematics Subject Classification

Search

Navigation