research-article

Fast algorithms for online stochastic convex programming

Authors:

Shipra Agrawal,

Nikhil R. DevanurAuthors Info & Claims

SODA '15: Proceedings of the twenty-sixth annual ACM-SIAM symposium on Discrete algorithms

Pages 1405 - 1424

Published: 04 January 2015 Publication History

Abstract

We introduce the online stochastic Convex Programming (CP) problem, a very general version of stochastic online problems which allows arbitrary concave objectives and convex feasibility constraints. Many well-studied problems like online stochastic packing and covering, online stochastic matching with concave returns, etc. form a special case of online stochastic CP. We present fast algorithms for these problems, which achieve near-optimal regret guarantees for both the i.i.d. and the random permutation models of stochastic inputs. When applied to the special case online packing, our ideas yield a simpler and faster primal-dual algorithm for this well studied problem, which achieves the optimal competitive ratio. Our techniques make explicit the connection of primal-dual paradigm and online learning to online stochastic CP.

References

[1]

Jacob Abernethy, Peter L. Bartlett, and Elad Hazan. Blackwell approachability and low-regret learning are equivalent. In COLT, 2011.

[2]

Gagan Aggarwal, Gagan Goel, Chinmay Karande, and Aranyak Mehta. Online vertex-weighted bipartite matching and single-bid budgeted allocations. In SODA, 2011.

Digital Library

[3]

S. Agrawal, Z. Wang, and Y. Ye. A dynamic near-optimal algorithm for online linear programming. Operations Research, 62:876--890, 2014.

Digital Library

[4]

Shipra Agrawal and Nikhil R. Devanur. Bandits with concave rewards and convex knapsacks. In Proceedings of the Fifteenth ACM Conference on Economics and Computation, EC '14, 2014.

Digital Library

[5]

Sanjeev Arora, Elad Hazan, and Satyen Kale. The multiplicative weights update method: a meta-algorithm and applications. Theory of Computing, 8(6):121--164, 2012.

[6]

Moshe Babaioff, Shaddin Dughmi, Robert Kleinberg, and Aleksandrs Slivkins. Dynamic pricing with limited supply. In EC, 2012.

Digital Library

[7]

Ashwinkumar Badanidiyuru, Robert Kleinberg, and Aleksandrs Slivkins. Bandits with knapsacks. In FOCS, pages 207--216, 2013.

Digital Library

[8]

Bahman Bahmani and Michael Kapralov. Improved bounds for online stochastic matching. In ESA, pages 170--181, 2010.

Digital Library

[9]

David Blackwell. An analog of the minimax theorem for vector payoffs. Pacific Journal of Mathematics, 6(1):1--8, 1956.

[10]

Niv Buchbinder, Kamal Jain, and Joseph Seffi Naor. Online primal-dual algorithms for maximizing ad-auctions revenue. In Proceedings of the 15th Annual European Conference on Algorithms, ESA'07, 2007.

Digital Library

[11]

Deepayan Chakrabarti and Erik Vee. Traffic shaping to optimize ad delivery. In Proceedings of the 13th ACM Conference on Electronic Commerce, EC '12, 2012.

Digital Library

[12]

Peiji Chen, Wenjing Ma, Srinath Mandalapu, Chandrashekhar Nagarjan, Jayavel Shanmugasundaram, Sergei Vassilvitskii, Erik Vee, Manfai Yu, and Jason Zien. Ad serving using a compact allocation plan. In Proceedings of the 13th ACM Conference on Electronic Commerce, EC '12, 2012.

Digital Library

[13]

Xiao Chen and Zizhuo Wang. A near-optimal dynamic learning algorithm for online matching problems with concave returns. http://arxiv.org/abs/1307.5934, 2013.

[14]

Ye Chen, Pavel Berkhin, Bo Anderson, and Nikhil R. Devanur. Real-time bidding algorithms for performance-based display ad allocation. In Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '11, 2011.

Digital Library

[15]

Nikhil R. Devanur and Thomas P. Hayes. The adwords problem: online keyword matching with budgeted bidders under random permutations. In EC, 2009.

Digital Library

[16]

Nikhil R. Devanur and Kamal Jain. Online matching with concave returns. In Proceedings of the Forty-fourth Annual ACM Symposium on Theory of Computing, STOC '12, 2012.

Digital Library

[17]

Nikhil R. Devanur, Kamal Jain, Balasubramanian Sivan, and Christopher A. Wilkens. Near optimal online algorithms and fast approximation algorithms for resource allocation problems. Full version, accessible from http://research.microsoft.com/en-us/um/people/bsivan/, 2011.

Digital Library

[18]

Nikhil R. Devanur, Kamal Jain, Balasubramanian Sivan, and Christopher A. Wilkens. Near optimal online algorithms and fast approximation algorithms for resource allocation problems. In EC, 2011.

Digital Library

[19]

Nikhil R. Devanur, Zhiyi Huang, Nitish Korula, Vahab S. Mirrokni, and Qiqi Yan. Whole-page optimization and submodular welfare maximization with online bidders. In Proceedings of the Fourteenth ACM Conference on Electronic Commerce, EC '13, 2013.

Digital Library

[20]

J. Feldman, N. Korula, V. Mirrokni, S. Muthukrishnan, and M. Pal. Online ad assignment with free disposal. In WINE, 2009.

Digital Library

[21]

Jon Feldman, Aranyak Mehta, Vahab Mirrokni, and S. Muthukrishnan. Online stochastic matching: Beating 1-1/e. In FOCS '09: Proceedings of the 2009 50th Annual IEEE Symposium on Foundations of Computer Science, 2009.

Digital Library

[22]

Jon Feldman, Monika Henzinger, Nitish Korula, Vahab S. Mirrokni, and Cliff Stein. Online stochastic packing applied to display ad allocation. In Proceedings of the 18th Annual European Conference on Algorithms: Part I, ESA'10, 2010.

Digital Library

[23]

Jon Feldman, Monika Henzinger, Nitish Korula, Vahab S. Mirrokni, and Clifford Stein. Online stochastic ad allocation: Efficiency and fairness. CoRR, abs/1001.5076, 2010.

[24]

Arpita Ghosh, Randolph Preston McAfee, Kishore Papineni, and Sergei Vassilvitskii. Bidding for representative allocations for display advertising. In WINE, 2009.

Digital Library

[25]

Gagan Goel and Aranyak Mehta. Online budgeted matching in random input models with applications to adwords. In SODA '08: Proceedings of the nineteenth annual ACM-SIAM symposium on Discrete algorithms, 2008.

Digital Library

[26]

Anupam Gupta and Marco Molinaro. How the Experts Algorithm Can Help Solve LPs Online. Algorithms - ESA 2014, Lecture Notes in Computer Science, 8737:517--529, 2014.

[27]

Elad Hazan, Amit Agarwal, and Satyen Kale. Logarithmic regret algorithms for online convex optimization. Mach. Learn., 69(2-3), December 2007.

Digital Library

[28]

Wassily Hoeffding. Probability inequalities for sums of bounded random variables. Journal of the American Statistical Association, 58(301):13--30, March 1963.

[29]

Sham M. Kakade, Shai Shalev-Shwartz, and Ambuj Tewari. On the duality of strong convexity and strong smoothness: Learning applications and matrix regularization. Technical report, Toyota Technological Institute - Chicago, USA, 2009. http://ttic.uchicago.edu/~shai/papers/KakadeShalevTewari09.pdf.

[30]

Chinmay Karande, Aranyak Mehta, and Pushkar Tripathi. Online bipartite matching with unknown distributions. In STOC, 2011.

Digital Library

[31]

Chinmay Karande, Aranyak Mehta, and Ramakrishnan Srikant. Optimizing budget constrained spend in search advertising. In Proceedings of the Sixth ACM International Conference on Web Search and Data Mining, WSDM '13, 2013.

Digital Library

[32]

R. M. Karp, U. V. Vazirani, and V. V. Vazirani. An optimal algorithm for on-line bipartite matching. In Proceedings of the Twenty-second Annual ACM Symposium on Theory of Computing, STOC '90, 1990.

Digital Library

[33]

Thomas Kesselheim, Andreas Tönnis, Klaus Radke, and Berthold Vöcking. Primal beats dual on online packing LPs in the random-order model. In STOC, 2014.

Digital Library

[34]

R. Kleinberg. A multiple-choice secretary algorithm with applications to online auctions. In Proceedings of the 16th Annual ACM-SIAM Symposium on Discrete algorithms, pages 630--631, January 2005.

Digital Library

[35]

Robert Kleinberg, Alex Slivkins, and Eli Upfal. Multi-armed bandits in metric spaces. In STOC, 2008.

Digital Library

[36]

Mohammad Mahdian and Qiqi Yan. Online bipartite matching with random arrivals: an approach based on strongly factor-revealing LPs. In STOC, 2011.

Digital Library

[37]

Mohammad Mahdian, Hamid Nazerzadeh, and Amin Saberi. Online optimization with uncertain information. ACM Trans. Algorithms, 8(1), January 2012.

Digital Library

[38]

Vahideh Manshadi, Shayan Gharan, and Amin Saberi. Online stochastic matching: Online actions based on offline statistics. In SODA, 2011.

Digital Library

[39]

Aranyak Mehta, Amin Saberi, Umesh V. Vazirani, and Vijay V. Vazirani. Adwords and generalized online matching. J. ACM, 54(5), 2007.

Digital Library

[40]

Vahab S. Mirrokni, Shayan Oveis Gharan, and Morteza Zadimoghaddam. Simultaneous approximations for adversarial and stochastic online budgeted allocation. In Proceedings of the Twenty-third Annual ACM-SIAM Symposium on Discrete Algorithms, SODA '12, 2012.

Digital Library

[41]

Shai Shalev-Shwartz. Online learning and online convex optimization. Foundations and Trends in Machine Learning, 4(2):107--194, 2012.

Digital Library

[42]

Erik Vee, Sergei Vassilvitskii, and Jayavel Shanmugasundaram. Optimal online assignment with forecasts. In EC '10: Proceedings of the 11th ACM conference on Electronic commerce, 2010.

Digital Library

Cited By

Feng YNiazadeh RSaberi AMarx D(2021)Two-stage stochastic matching with application to ride hailingProceedings of the Thirty-Second Annual ACM-SIAM Symposium on Discrete Algorithms10.5555/3458064.3458234(2862-2877)Online publication date: 10-Jan-2021
https://dl.acm.org/doi/10.5555/3458064.3458234
Molinaro MMarx D(2021)Robust algorithms for online convex problems via primal-dualProceedings of the Thirty-Second Annual ACM-SIAM Symposium on Discrete Algorithms10.5555/3458064.3458188(2078-2092)Online publication date: 10-Jan-2021
https://dl.acm.org/doi/10.5555/3458064.3458188
Dughmi SHartline JKleinberg RNiazadeh R(2021)Bernoulli Factories and Black-box Reductions in Mechanism DesignJournal of the ACM10.1145/344098868:2(1-30)Online publication date: 6-Jan-2021
https://dl.acm.org/doi/10.1145/3440988
Show More Cited By

Fast algorithms for online stochastic convex programming
1. Computing methodologies
2. Theory of computation
  1. Theory and algorithms for application domains
    1. Machine learning theory

Recommendations

Online convex optimization with stochastic constraints
NIPS'17: Proceedings of the 31st International Conference on Neural Information Processing Systems

This paper considers online convex optimization (OCO) with stochastic constraints, which generalizes Zinkevich's OCO over a known simple fixed set by introducing multiple stochastic functional constraints that are i.i.d. generated at each round and are ...
Between stochastic and adversarial online convex optimization: improved regret bounds via smoothness
NIPS '22: Proceedings of the 36th International Conference on Neural Information Processing Systems

Stochastic and adversarial data are two widely studied settings in online learning. But many optimization tasks are neither i.i.d. nor fully adversarial, which makes it of fundamental interest to get a better theoretical understanding of the world ...
Optimistic online mirror descent for bridging stochastic and adversarial online convex optimization
ICML'23: Proceedings of the 40th International Conference on Machine Learning

Stochastically Extended Adversarial (SEA) model is introduced by Sachs et al. (2022) as an interpolation between stochastic and adversarial online convex optimization. Under the smoothness condition, they demonstrate that the expected regret of optimistic ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

SODA '15: Proceedings of the twenty-sixth annual ACM-SIAM symposium on Discrete algorithms

January 2015

2056 pages

Program Chair:
Piotr Indyk
Massachusetts Institute of Technology

Sponsors

SIAM: Society for Industrial and Applied Mathematics

In-Cooperation

SIGACT: ACM Special Interest Group on Algorithms and Computation Theory

Publisher

Society for Industrial and Applied Mathematics

United States

Publication History

Published: 04 January 2015

Check for updates

Qualifiers

Research-article

Conference

SODA '15

Sponsor:

SIAM

SODA '15: ACM SIAM Symposium on Discrete Algorithms

January 4 - 6, 2015

California, San Diego

Acceptance Rates

SODA '15 Paper Acceptance Rate 137 of 495 submissions, 28%;

Overall Acceptance Rate 411 of 1,322 submissions, 31%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

20
Total Citations
View Citations
311
Total Downloads

Downloads (Last 12 months)5
Downloads (Last 6 weeks)0

Reflects downloads up to 27 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Feng YNiazadeh RSaberi AMarx D(2021)Two-stage stochastic matching with application to ride hailingProceedings of the Thirty-Second Annual ACM-SIAM Symposium on Discrete Algorithms10.5555/3458064.3458234(2862-2877)Online publication date: 10-Jan-2021
https://dl.acm.org/doi/10.5555/3458064.3458234
Molinaro MMarx D(2021)Robust algorithms for online convex problems via primal-dualProceedings of the Thirty-Second Annual ACM-SIAM Symposium on Discrete Algorithms10.5555/3458064.3458188(2078-2092)Online publication date: 10-Jan-2021
https://dl.acm.org/doi/10.5555/3458064.3458188
Dughmi SHartline JKleinberg RNiazadeh R(2021)Bernoulli Factories and Black-box Reductions in Mechanism DesignJournal of the ACM10.1145/344098868:2(1-30)Online publication date: 6-Jan-2021
https://dl.acm.org/doi/10.1145/3440988
Balseiro SLu HMirrokni VDaumé HSingh A(2020)Dual mirror descent for online allocation problemsProceedings of the 37th International Conference on Machine Learning10.5555/3524938.3524996(613-628)Online publication date: 13-Jul-2020
https://dl.acm.org/doi/10.5555/3524938.3524996
Sadeghi ORaut PFazel MLarochelle HRanzato MHadsell RBalcan MLin H(2020)A single recipe for online submodular maximization with adversarial or stochastic constraintsProceedings of the 34th International Conference on Neural Information Processing Systems10.5555/3495724.3496957(14712-14723)Online publication date: 6-Dec-2020
https://dl.acm.org/doi/10.5555/3495724.3496957
Dickerson JSankararaman KSarpatwar KSrinivasan AWu KXu PElkind EVeloso MAgmon NTaylor M(2019)Online Resource Allocation with Matching ConstraintsProceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems10.5555/3306127.3331896(1681-1689)Online publication date: 8-May-2019
https://dl.acm.org/doi/10.5555/3306127.3331896
Makhijani RChakrabarti SStruble DLiu YBogers TSaid ABrusilovsky PTikk D(2019)LOREProceedings of the 13th ACM Conference on Recommender Systems10.1145/3298689.3347027(160-168)Online publication date: 10-Sep-2019
https://dl.acm.org/doi/10.1145/3298689.3347027
Devanur NJain KSivan BWilkens C(2019)Near Optimal Online Algorithms and Fast Approximation Algorithms for Resource Allocation ProblemsJournal of the ACM10.1145/328417766:1(1-41)Online publication date: 9-Jan-2019
https://dl.acm.org/doi/10.1145/3284177
Allen-Zhu ZSimchi-Levi DWang X(2018)The lingering of gradientsProceedings of the 32nd International Conference on Neural Information Processing Systems10.5555/3326943.3327058(1252-1261)Online publication date: 3-Dec-2018
https://dl.acm.org/doi/10.5555/3326943.3327058
Huang ZZhu XCzumaj A(2018)Near optimal jointly private packing algorithms via dual multiplicative weight updateProceedings of the Twenty-Ninth Annual ACM-SIAM Symposium on Discrete Algorithms10.5555/3174304.3175291(343-357)Online publication date: 7-Jan-2018
https://dl.acm.org/doi/10.5555/3174304.3175291
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten