article

Discrete-Time Control for Systems of Interacting Objects with Unknown Random Disturbance Distributions: A Mean Field Approach

Authors:

Carmen G. Higuera-Chan,

Héctor Jasso-Fuentes,

J. Adolfo Minjárez-SosaAuthors Info & Claims

Applied Mathematics and Optimization, Volume 74, Issue 1

Pages 197 - 227

https://doi.org/10.1007/s00245-015-9312-6

Published: 01 August 2016 Publication History

Abstract

We are concerned with stochastic control systems composed of a large number of N interacting objects sharing a common environment. The evolution of each object is determined by a stochastic difference equation where the random disturbance density $$\rho $$ź is unknown for the controller. We present the Markov control model (N-model) associated to the proportions of objects in each state, which is analyzed according to the mean field theory. Thus, combining convergence results as $$N\rightarrow \infty $$Nźź (the mean field limit) with a suitable statistical estimation method for $$\rho $$ź, we construct the so-named eventually asymptotically optimal policies for the N-model under a discounted optimality criterion. A consumption-investment problem is analyzed to illustrate our results.

References

[1]

Achdou, I., Capuzzo-Dolcetta, I.: Mean field games: numerical methods. SIAM J. Numer. Anal. 48, 1136---1162 (2010)

Digital Library

[2]

Aoki, M.: New macroeconomic modeling approaches. Hierarchical dynamics and mean field approximation. J. Econ. Dyn. Control 18, 865---877 (1994)

[3]

Bensoussan, A., Frehse, J., Yam, P.: Mean Field Games and Mean Field Control Theory. Springer, New York (2010)

[4]

Bertsekas, D.P.: Dynamic Programming: Deterministic and Stochastic Models. Prentice-Hall, Englewood Cliffs (1987)

Digital Library

[5]

Budhiraja, A., Dupuis, P., Fischer, M.: Large deviation properties of weakly interacting processes via weak convergence methods. Ann. Probab. 40, 74---102 (2012)

[6]

Carmona, R., Fouque, J.P., Vestal, D.: Interacting particle systems for the computation of rare credit portfolio losses. Financ. Stoch. 13, 613---633 (2009)

[7]

Gast, N., Gaujal, B.: A mean field approach for optimization in discrete time. Discret. Event Dyn. Syst. 21, 63---101 (2011)

Digital Library

[8]

Gast, N., Gaujal, B., Le Boudec, J.Y.: Mean field for Markov decision processes: from discrete to continuous optimization. IEEE Trans. Autom. Control 57, 2266---2280 (2012)

[9]

Gomes, D.A., Mohr, J., Souza, R.R.: Discrete time, finite state space mean field games. J. Math. Pures Appl. 93, 308---328 (2010)

[10]

Gordienko, E.I., Minjárez-Sosa, J.A.: Adaptive control for discrete-time Markov processes with unbounded costs: discounted criterion. Kybernetika 34, 217---234 (1998)

[11]

Hernández-Lerma, O.: Adaptive Markov Control Processes. Springer, New York (1989)

[12]

Hernández-Lerma, O., Lasserre, J.B.: Discrete-Time Markov Control Processes: Basic Optimality Criteria. Springer, New York (1996)

[13]

Hilgert, N., Minjárez-Sosa, J.A.: Adaptive control of stochastic systems with unknown disturbance distribution: discounted criterion. Math. Method Oper. Res. 63, 443---460 (2006)

[14]

Huang, M.: Large-population LQG games involving a major player: the Nash certainty equivalence principle. SIAM J. Control Optim. 48, 3318---3353 (2010)

[15]

Kolokoltsov, V.N., Troeva, M., Yang, W.: On the rate of convergence for the mean-field approximation of controlled diffusions with large number of players. Dyn. Games Appl. 4, 208---230 (2014)

[16]

Lachapelle, A., Salomon, J., Turinici, G.: Computation of mean field equilibria in economics. Math. Models Methods Appl. Sci. 20, 567---588 (2010)

[17]

Le Boudec, J.Y., McDonald, D., Mundinger, J.: A generic mean field convergence result for systems of interacting objects. In: 4th International Conference Quantitative Evaluation of Systems (2007). ISBN:0-7695-2883-X/07

Digital Library

[18]

Lasry, J.M., Lions, P.L.: Mean field games. Jap. J. Math. 2, 229---260 (2007)

[19]

Minjárez-Sosa, J.A., Vega-Amaya, O.: Asymptotically optimal strategies for adaptive zero-sum discounted Markov games. SIAM J. Control Optim. 48, 1405---1421 (2009)

Digital Library

[20]

Peyrard, N., Sabbadin, R.: Mean field approximation of the policy iteration algorithm for graph-based Markov decision processes. In: Biewka, G., Coradeschi, S., Perini, A., Traverso, P. (eds.) Frontiers in Artificial Intelligence and Applications, pp. 595---599. IOS Press, Amsterdam (2006)

Digital Library

[21]

Puterman, M.L.: Markov Decision Processes. Discrete Stochastic Dynamic Programming. Wiley, New York (1994)

Digital Library

Cited By

Bäuerle N(2023)Mean Field Markov Decision ProcessesApplied Mathematics and Optimization10.1007/s00245-023-09985-188:1Online publication date: 10-Apr-2023
https://dl.acm.org/doi/10.1007/s00245-023-09985-1
Martínez-Manzanares MMinjárez-Sosa J(2021)A mean field absorbing control model for interacting objects systemsDiscrete Event Dynamic Systems10.1007/s10626-021-00339-z31:3(349-372)Online publication date: 1-Sep-2021
https://dl.acm.org/doi/10.1007/s10626-021-00339-z
Song TLiu B(2021)Discrete‐time mean‐field stochastic linear‐quadratic optimal control problem with finite horizonAsian Journal of Control10.1002/asjc.230623:2(979-989)Online publication date: 1-Mar-2021
https://dl.acm.org/doi/10.1002/asjc.2306

Recommendations

A discrete-time multivariable neuro-adaptive control for nonlinear unknown dynamic systems

First, we assume that the controlled systems contain a nonlinear matrix gain before a linear discrete-time multivariable dynamic system. Then, a forward control based on a nominal system is employed to cancel the system nonlinear matrix gain and track ...
Adaptive fuzzy backstepping output feedback control of nonlinear time-delay systems with unknown high-frequency gain sign

In this paper, an adaptive fuzzy robust feedback control approach is proposed for a class of single-input and single-output (SISO) strict-feedback nonlinear systems with unknown nonlinear functions, time delays, unknown high-frequency gain sign, and ...
Time-optimal control of disturbance-rejection tracking systems for discrete-time time-delayed systems by state feedback

In this paper, we solve the disturbance-rejection and tracking problem for linear multivariable discrete-time systems with time-delayed controlled inputs. A set of necessary and sufficient conditions under which the proposed problem is controllable is ...

Comments

Information & Contributors

Information

Published In

cover image Applied Mathematics and Optimization

Applied Mathematics and Optimization Volume 74, Issue 1

August 2016

222 pages

ISSN:0095-4616

Issue’s Table of Contents

Copyright © Copyright © 2016 Springer Science+Business Media New York.

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 01 August 2016

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 04 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Bäuerle N(2023)Mean Field Markov Decision ProcessesApplied Mathematics and Optimization10.1007/s00245-023-09985-188:1Online publication date: 10-Apr-2023
https://dl.acm.org/doi/10.1007/s00245-023-09985-1
Martínez-Manzanares MMinjárez-Sosa J(2021)A mean field absorbing control model for interacting objects systemsDiscrete Event Dynamic Systems10.1007/s10626-021-00339-z31:3(349-372)Online publication date: 1-Sep-2021
https://dl.acm.org/doi/10.1007/s10626-021-00339-z
Song TLiu B(2021)Discrete‐time mean‐field stochastic linear‐quadratic optimal control problem with finite horizonAsian Journal of Control10.1002/asjc.230623:2(979-989)Online publication date: 1-Mar-2021
https://dl.acm.org/doi/10.1002/asjc.2306

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents