research-article

Online resource allocation in Markov Chains

Authors:

Nikolai Gravin,

Zhihao Gavin TangAuthors Info & Claims

WWW '23: Proceedings of the ACM Web Conference 2023

Pages 3498 - 3507

https://doi.org/10.1145/3543507.3583428

Published: 30 April 2023 Publication History

Abstract

A large body of work in Computer Science and Operations Research study online algorithms for stochastic resource allocation problems. The most common assumption is that the online requests have randomly generated i.i.d. types. This assumption is well justified for static markets and/or relatively short time periods. We consider dynamic markets, whose states evolve as a random walk in a market-specific Markov Chain. This is a new model that generalizes previous i.i.d. settings. We identify important parameters of the Markov chain that is crucial for obtaining good approximation guarantees to the expected value of the optimal offline algorithm which knows realizations of all requests in advance. We focus on a stylized single-resource setting and: (i) generalize the well-known Prophet Inequality from the optimal stopping theory (single-unit setting) to Markov Chain setting; (ii) in multi-unit setting, design a simple algorithm that is asymptotically optimal under mild assumptions on the underlying Markov chain.

References

[1]

Gagan Aggarwal, Gagan Goel, Chinmay Karande, and Aranyak Mehta. 2011. Online Vertex-Weighted Bipartite Matching and Single-Bid Budgeted Allocations. In Proceedings of the Twenty-Second Annual ACM-SIAM Symposium on Discrete Algorithms (San Francisco, California) (SODA ’11). Society for Industrial and Applied Mathematics, USA, 1253–1264.

Digital Library

[2]

Shipra Agrawal and Nikhil R. Devanur. 2015. Fast Algorithms for Online Stochastic Convex Programming. SIAM, USA, 1405–1424. https://doi.org/10.1137/1.9781611973730.93 arXiv:https://epubs.siam.org/doi/pdf/10.1137/1.9781611973730.93

[3]

Shipra Agrawal, Zizhuo Wang, and Yinyu Ye. 2014. A dynamic near-optimal algorithm for online linear programming. Operations Research 62, 4 (2014), 876–890.

Digital Library

[4]

Saeed Alaei. 2014. Bayesian Combinatorial Auctions: Expanding Single Buyer Mechanisms to Many Buyers. SIAM J. Comput. 43, 2 (2014), 930–972.

[5]

Saeed Alaei, MohammadTaghi Hajiaghayi, and Vahid Liaghat. 2012. Online Prophet-Inequality Matching with Applications to Ad Allocation. In Proceedings of the 13th ACM Conference on Electronic Commerce (Valencia, Spain) (EC ’12). Association for Computing Machinery, New York, NY, USA, 18–35. https://doi.org/10.1145/2229012.2229018

Digital Library

[6]

Nikhil R. Devanur and Thomas P. Hayes. 2009. The Adwords Problem: Online Keyword Matching with Budgeted Bidders under Random Permutations. In Proceedings of the 10th ACM Conference on Electronic Commerce (Stanford, California, USA) (EC ’09). Association for Computing Machinery, New York, NY, USA, 71–78. https://doi.org/10.1145/1566374.1566384

Digital Library

[7]

Nikhil R. Devanur, Kamal Jain, Balasubramanian Sivan, and Christopher A. Wilkens. 2019. Near Optimal Online Algorithms and Fast Approximation Algorithms for Resource Allocation Problems. J. ACM 66, 1 (2019), 7:1–7:41.

Digital Library

[8]

Nikhil R. Devanur, Balasubramanian Sivan, and Yossi Azar. 2012. Asymptotically Optimal Algorithm for Stochastic Adwords. In Proceedings of the 13th ACM Conference on Electronic Commerce (Valencia, Spain) (EC ’12). Association for Computing Machinery, New York, NY, USA, 388–404. https://doi.org/10.1145/2229012.2229043

Digital Library

[9]

Anupam Gupta and Marco Molinaro. 2016. How the Experts Algorithm Can Help Solve LPs Online. Math. Oper. Res. 41, 4 (2016), 1404–1431.

Digital Library

[10]

Mohammad Taghi Hajiaghayi, Robert Kleinberg, and Tuomas Sandholm. 2007. Automated Online Mechanism Design and Prophet Inequalities. In Proceedings of the 22nd National Conference on Artificial Intelligence - Volume 1 (Vancouver, British Columbia, Canada) (AAAI’07). AAAI Press, USA, 58–65.

[11]

Bala Kalyanasundaram and Kirk Pruhs. 2000. An optimal deterministic algorithm for online b-matching. Theor. Comput. Sci. 233, 1-2 (2000), 319–325.

Digital Library

[12]

R. M. Karp, U. V. Vazirani, and V. V. Vazirani. 1990. An Optimal Algorithm for On-Line Bipartite Matching. In Proceedings of the Twenty-Second Annual ACM Symposium on Theory of Computing (Baltimore, Maryland, USA) (STOC ’90). Association for Computing Machinery, New York, NY, USA, 352–358. https://doi.org/10.1145/100216.100262

Digital Library

[13]

D. P. Kennedy. 1985. Optimal Stopping of Independent Random Variables and Maximizing Prophets. The Annals of Probability 13, 2 (1985), 566 – 571. https://doi.org/10.1214/aop/1176993009

[14]

Douglas P Kennedy. 1987. Prophet-type inequalities for multi-choice optimal stopping. Stochastic Processes and their applications 24, 1 (1987), 77–88.

[15]

Robert P Kertz. 1986. Comparison of optimal value and constrained maxima expectations for independent random variables. Advances in applied probability 18, 2 (1986), 311–340.

[16]

Thomas Kesselheim, Klaus Radke, Andreas Tönnis, and Berthold Vöcking. 2018. Primal Beats Dual on Online Packing LPs in the Random-Order Model. SIAM J. Comput. 47, 5 (2018), 1939–1964.

Digital Library

[17]

Robert Kleinberg and S. Matthew Weinberg. 2019. Matroid prophet inequalities and applications to multi-dimensional mechanism design. Games Econ. Behav. 113 (2019), 97–115.

[18]

Ulrich Krengel and Louis Sucheston. 1977. Semiamarts and finite values. Bull. Amer. Math. Soc. 83, 4 (1977), 745–747.

[19]

Zachary J. Lee, Tongxin Li, and Steven H. Low. 2019. ACN-Data: Analysis and Applications of an Open EV Charging Dataset.

[20]

Xiaocheng Li and Yinyu Ye. 2022. Online Linear Programming: Dual Convergence, New Algorithms, and Regret Bounds. Oper. Res. 70, 5 (2022), 2948–2966. https://doi.org/10.1287/opre.2021.2164

Digital Library

[21]

Aranyak Mehta, Amin Saberi, Umesh V. Vazirani, and Vijay V. Vazirani. 2007. AdWords and generalized online matching. J. ACM 54, 5 (2007), 22.

Digital Library

[22]

Christos Papadimitriou, Tristan Pollner, Amin Saberi, and David Wajc. 2021. Online Stochastic Max-Weight Bipartite Matching: Beyond Prophet Inequalities. In Proceedings of the 22nd ACM Conference on Economics and Computation (Budapest, Hungary) (EC ’21). Association for Computing Machinery, New York, NY, USA, 763–764. https://doi.org/10.1145/3465456.3467613

Digital Library

[23]

Ester Samuel-Cahn. 1984. Comparison of Threshold Stop Rules and Maximum for Independent Nonnegative Random Variables. The Annals of Probability 12, 4 (1984), 1213 – 1216. https://doi.org/10.1214/aop/1176993150

Cited By

Spaeh FTsourakakis CChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Markovletics: Methods and A Novel Application for Learning Continuous-Time Markov Chain MixturesProceedings of the ACM Web Conference 202410.1145/3589334.3645491(4160-4171)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645491

Index Terms

Online resource allocation in Markov Chains
1. Mathematics of computing
2. Theory of computation
  1. Design and analysis of algorithms
    1. Approximation algorithms analysis
    2. Online algorithms
      1. Online learning algorithms
        Scheduling algorithms
  2. Theory and algorithms for application domains
    1. Machine learning theory
      1. Online learning theory

Index terms have been assigned to the content through auto-classification.

Recommendations

An EM algorithm for continuous-time bivariate Markov chains

We study properties and parameter estimation of a finite-state, homogeneous, continuous-time, bivariate Markov chain. Only one of the two processes of the bivariate Markov chain is assumed observable. The general form of the bivariate Markov chain ...
Rapidly Mixing Markov Chains with Applications in Computer Science and Physics

Monte Carlo algorithms often depend on Markov chains to sample from very large data sets. A key ingredient in the design of an efficient Markov chain is determining rigorous bounds on how quickly the chain "mixes," or converges, to its stationary ...
Differential Evolution Markov Chain with snooker updater and fewer chains

Differential Evolution Markov Chain (DE-MC) is an adaptive MCMC algorithm, in which multiple chains are run in parallel. Standard DE-MC requires at least N=2d chains to be run in parallel, where d is the dimensionality of the posterior. This paper ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WWW '23: Proceedings of the ACM Web Conference 2023

April 2023

4293 pages

ISBN:9781450394161

DOI:10.1145/3543507

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 April 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

Innovation Program of Shanghai Municipal Education Commission
NSFC
CCF-AFSG Research Fund
Science and Technology Innovation 2030 ? ?New Generation of Artificial Intelligence? Major Project

Conference

WWW '23

Sponsor:

SIGWEB

WWW '23: The ACM Web Conference 2023

April 30 - May 4, 2023

TX, Austin, USA

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
334
Total Downloads

Downloads (Last 12 months)117
Downloads (Last 6 weeks)11

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Spaeh FTsourakakis CChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Markovletics: Methods and A Novel Application for Learning Continuous-Time Markov Chain MixturesProceedings of the ACM Web Conference 202410.1145/3589334.3645491(4160-4171)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645491

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Table of Conten