Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
Springer Nature Link
Log in

Fast projection onto the simplex and the \(\pmb {l}_\mathbf {1}\) ball

  • Short Communication
  • Series A
  • Published:
Mathematical Programming Submit manuscript

Abstract

A new algorithm is proposed to project, exactly and in finite time, a vector of arbitrary size onto a simplex or an \(l_1\)-norm ball. It can be viewed as a Gauss–Seidel-like variant of Michelot’s variable fixing algorithm; that is, the threshold used to fix the variables is updated after each element is read, instead of waiting for a full reading pass over the list of non-fixed elements. This algorithm is empirically demonstrated to be faster than existing methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Notes

  1. Actually, Algorithm 4 is an improvement of Michelot’s algorithm, with the test “\(>\)” instead of “\(\ge \)” at step 2.1. This modification has been proposed in [13, Sect. 5.7]. Algorithm 4 is also the same algorithm as in [9].

References

  1. Bentley, J.L., McIlroy, M.D.: Engineering a sort function. Softw. Pract. Exp. 23(11), 1249–1265 (1993)

    Article  Google Scholar 

  2. Bioucas-Dias, J.M., Plaza, A., Dobigeon, N., Parente, M., Du, Q., Gader, P., Chanussot, J.: Hyperspectral unmixing overview: geometrical, statistical, and sparse regression-based approaches. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 5(2), 354–379 (2012)

    Article  Google Scholar 

  3. Blondel, M., Fujino, A., Ueda, N.: Large-scale multiclass support vector machine training via Euclidean projection onto the simplex. In: Proceedings of the 22th International Conference on Pattern Recognition (ICPR), pp. 1289–1294 (2014)

  4. Blum, M., Floyd, R.W., Pratt, V.R., Rivest, R.L., Tarjan, R.E.: Time bounds for selection. J. Comput. Syst. Sci. 7(4), 448–461 (1973)

    Article  MathSciNet  MATH  Google Scholar 

  5. Brodie, J., Daubechies, I., Mol, C.D., Giannone, D., Loris, I.: Sparse and stable Markowitz portfolios. Proc. Natl. Acad. Sci. 106(30), 12267–12272 (2009)

    Article  MATH  Google Scholar 

  6. Cominetti, R., Mascarenhas, W.F., Silva, P.J.S.: A Newton’s method for the continuous quadratic knapsack problem. Math. Program. Comp. 6, 151–169 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  7. Duchi, J., Shalev-Shwartz, S., Singer, Y., Chandra, T.: Efficient projections onto the \(\ell _1\)-ball for learning in high dimensions. In: Proceedings of the 25th International Conference on Machine learning (ICML) (2008)

  8. Fadili, J., Peyré, G.: Total variation projection with first order schemes. IEEE Trans. Image Process. 20(3), 657–669 (2011)

    Article  MathSciNet  Google Scholar 

  9. Gong, P., Gai, K., Zhang, C.: Efficient Euclidean projections via piecewise root finding and its application in gradient projection. Neurocomputing 74, 2754–2766 (2011)

    Article  Google Scholar 

  10. Held, M., Wolfe, P., Crowder, H.: Validation of subgradient optimization. Math. Program. 6, 62–88 (1974)

    Article  MathSciNet  MATH  Google Scholar 

  11. Kiwiel, K.C.: On Floyd and Rivest’s SELECT algorithm. Theor. Comput. Sci. 347, 214–238 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  12. Kiwiel, K.C.: Breakpoint searching algorithms for the continuous quadratic knapsack problem. Math. Program. Ser. A 112, 473–491 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  13. Kiwiel, K.C.: Variable fixing algorithms for the continuous quadratic knapsack problem. J. Optim. Theory Appl. 136, 445–458 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  14. Knuth, D.E.: The Art of Computer Programming, vol. 2, 3rd edn, p. 232. Addison-Wesley, Boston (1998)

    Google Scholar 

  15. Lellmann, J., Kappes, J.H., Yuan, J., Becker, F., Schnörr, C.: Convex multi-class image labeling by simplex-constrained total variation. In: Proceedings of Scale Space and Variational Methods in Computer Vision (SSVM), vol. 5567, pp. 150–162 (2009)

  16. Liu, J., Ye, J.: Efficient Euclidean projections in linear time. In: Proceedings of the 26th International Conference Machine Learning (ICML) (2009)

  17. Michelot, C.: A finite algorithm for finding the projection of a point onto the canonical simplex of \({\mathbb{R}}^n\). J. Optim. Theory Appl. 50(1), 195–200 (1986)

    Article  MathSciNet  MATH  Google Scholar 

  18. Patriksson, M.: A survey on the continuous nonlinear resource allocation problem. Eur. J. Oper. Res. 185, 1–46 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  19. Patriksson, M., Strömberg, C.: Algorithms for the continuous nonlinear resource allocation problem—new implementations and numerical studies. Eur. J. Oper. Res. 243(3), 703–722 (2015)

    Article  MathSciNet  Google Scholar 

  20. van den Berg, E., Friedlander, M.P.: Probing the Pareto frontier for basis pursuit solutions. SIAM J. Sci. Comput. 31, 890–912 (2008)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Grenoble Alpes, GIPSA-Lab, 38000, Grenoble, France

    Laurent Condat

Authors
  1. Laurent Condat
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Laurent Condat.

Additional information

This work has been partially supported by the LabEx PERSYVAL-Lab (ANR-11-LABX-0025).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Condat, L. Fast projection onto the simplex and the \(\pmb {l}_\mathbf {1}\) ball. Math. Program. 158, 575–585 (2016). https://doi.org/10.1007/s10107-015-0946-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10107-015-0946-6

Keywords

Mathematics Subject Classification

Search

Navigation