Discrete-Time Hybrid Control Processes with Unbounded Costs
Abstract
This paper extends the results provided in Jasso-Fuentes et al. (Appl Math Optim 81(2):409–441, 2020b) and Jasso-Fuentes et al. (Pure Appl Funct Anal 9(3):675–704, 2024) regarding the study of discrete-time hybrid stochastic models with general spaces and total discounted payoffs. This extension incorporates the handling of negative and/or unbounded costs per stage. In particular, it encompasses interesting applications, such as scenarios where the controller optimizes net costs, social welfare costs, or distances between points. These situations arise when assumptions of both non-negativeness and boundedness on the cost per stage do not apply. Our proposal relies on Lyapunov-like conditions, enabling, among other aspects, the finiteness of the value function and the existence of solutions to the associated dynamic programming equation. This equation is crucial for deriving optimal control policies. To illustrate our theory, we include an example in inventory-manufacturing management, highlighting its evident hybrid nature.
References
[1]
Abate A, Prandini M, Lygeros J, and Sastry S Probabilistic reachability and safety for controlled discrete time stochastic hybrid systems Automatica 2008 44 11 2724-2734
[2]
Amin, S., Abate, A., Prandini, M., Lygeros, J., Sastry, S.: Reachability analysis for controlled discrete time stochastic hybrid systems. In: Hybrid Systems: Computation and Control: 9th International Workshop, HSCC 2006, Santa Barbara, CA, USA, 29–31 March 2006, Proceedings, vol 9, pp. 49–63 (2006)
[3]
Bensoussan A Dynamic Programming and Inventory Control 2011 Amsterdam IOS Press
[4]
Bensoussan A and Lions J-L Impulse Control and Quasi-variational Inequalities 1984 Paris Bordas Editions
[5]
Bensoussan A and Menaldi JL Hybrid control and dynamic programming Dyn. Contin. Discret. Impuls. Syst. 1997 3 4 395-442
[6]
Bensoussan, A., Menaldi, J.L.: Stochastic hybrid control. J. Math. Anal. Appl. 249, 261–288 (2000). (Special issue in honor of Richard Bellman)
[7]
Blankenship G and Menaldi J-L Optimal stochastic scheduling of power generation systems with scheduling delays and large cost differentials SIAM J. Control. Optim. 1984 22 1 121-132
[8]
Branicky MS, Borkar VS, and Mitter SK A unified framework for hybrid control: model and optimal control theory IEEE Trans. Autom. Control 1998 43 1 31-45
[9]
Hernández-Lerma, O., Lasserre, J.B.: Discrete-time Markov control processes, vol. 30. Springer, New York (1996). (Basic optimality criteria)
[10]
Hernández-Lerma, O., Lasserre, J.B.: Further topics on discrete-time Markov control processes, vol. 42. Springer, New York (1999).
[11]
Ilhuicatzi-Roldán R, Cruz-Suárez H, and Chávez-Rodríguez S Markov decision processes with time-varying discount factors and random horizon Kybernetika 2017 53 1 82-98
[12]
Jasso-Fuentes, H., Menaldi, J.-L., Prieto-Rumeau, T.: Discrete-time control with nonconstant discount factor. Math. Methods Oper. Res. 92(2), 377–399 (2020a).
[13]
Jasso-Fuentes, H., Menaldi, J.-L., Prieto-Rumeau, T.: Discrete-time hybrid control in Borel spaces. Appl. Math. Optim. 81(2), 409–441 (2020b).
[14]
Jasso-Fuentes H, Menaldi J-L, and Prieto-Rumeau T Recent results on discrete-time hybrid control models with general state and action spaces Pure Appl. Funct. Anal. 2024 9 3 675-704
[15]
Jasso-Fuentes H, Menaldi J-L, Prieto-Rumeau T, and Robin M Discrete-time hybrid control in Borel spaces: average cost optimality criterion J. Math. Anal. Appl. 2018 462 2 1695-1713
[16]
Menaldi J-L Optimal impulse control problems for degenerate diffusions with jumps Acta Appl. Math. 1987 8 2 165-198
[17]
Minjárez-Sosa JA Markov control models with unknown random state-action-dependent discount factors TOP 2015 23 743-772
[18]
Robin M Contrôle impulsionnel avec retard pour les processus de diffusion CRAS 1976 282 463-466
[19]
Robin, M.: Controle impulsionnel des processus de Markov. Unpublished doctoral dissertation, Université Paris Dauphine-Paris IX (1978)
[20]
Stettner, L.: On Some Stopping and Implusive Control Problems with a General Discount Rate Criteria. Polish Academy of Sciences [PAS], Institute of Mathematics (1988)
[21]
Summers S and Lygeros J Verification of discrete time stochastic hybrid systems: A stochastic reach-avoid decision problem Automatica 2010 46 12 1951-1961
[22]
Wei, Q., Guo, X.: Markov decision processes with state-dependent discount factors and unbounded rewards/costs. Oper. Res. Lett. 39(5), 369–374 (2011)
Index Terms
- Discrete-Time Hybrid Control Processes with Unbounded Costs
Index terms have been assigned to the content through auto-classification.
Recommendations
Discrete-time hybrid control with risk-sensitive discounted costs
AbstractThis paper studies discrete-time hybrid control problems where the controller’s payoff function follows a risk-sensitive discounted model. The associated discount factor can depend on state and action variables, which can even take values of zero ...
Stability Verification for a Class of Stochastic Hybrid Systems by Semidefinite Programming
This paper is concerned with the analysis and verification of stability for a class of stochastic hybrid systems (SHSs), which contain not only continuous flows described by stochastic differential equations but also Markovian switching and discrete jumps ...
Finite-Time Control of Nonlinear Impulsive Switched Positive Systems Based on an Event-Triggered Controller
AbstractIn this paper, the finite-time control of nonlinear impulsive switched positive systems (ISPSs) is studied, where the impulses and bounded disturbance are both fully considered. By designing a novel event-triggered strategy, we present LMI-based ...
Comments
Information & Contributors
Information
Published In
© The Author(s) 2024.
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 25 October 2024
Accepted: 03 October 2024
Author Tags
Author Tags
Qualifiers
- Research-article
Funding Sources
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 04 Feb 2025