Abstract
In a recent paper we presented a numerical technique for solving the optimal stopping problem for a piecewise-deterministic process (P.D.P.) by discretization of the state space. In this paper we apply these results to the impulse control problem. In the first part of the paper we study the impulse control of P.D.P.s. under general conditions. We show that iteration of the single-jump-or-intervention operator generates a sequence of functions converging to the value function of the problem. In the second part of the paper we present a numerical technique for computing optimal impulse controls for P.D.P.s. This technique reduces the problem to a sequence of one-dimensional minimizations. We conclude by presenting some numerical examples.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
A. Bensoussan and J. L. Lions, Nouvelles Methodes en Contrôle Impulsionnel,Appl. Math. Optim.,1 (1975), 289–312.
A. Bensoussan and J. L. Lions,Contrôle Impulsionnel et Inéquations Quasi-Variationnelles, Dunod, Paris, 1982.
C. Berge,Topological Spaces, Macmillan, New York, 1963.
D. Bertsekas and S. E. Shreve,Stochastic Optimal Control: The Discrete-Time case, Academic Press, New York, 1978.
O. L. V. Costa and M. H. A. Davis, Approximations for optimal stopping of a piecewise-deterministic process,Math. Control Signals Systems,1 (1988), 123–146.
M. H. A. Davis, Piecewise-deterministic Markov processes: a general class of non-diffusion stochastic models,J. Roy. Statist. Soc. Ser B,46 (1984), 353–388.
El Karoui,Les Aspects Probabilistes du Contrôle Stochastique, pp. 74–239, Lectures Notes in Mathematics, Vol. 876, Springer-Verlag, Berlin, 1981.
R. K. Getoor,Markov Processes: Ray Processes and Right Processes, Lectures Notes in Mathematics, Vol. 440, Springer-Verlag, Berlin, 1981.
A. Hordijk and F. A. van der Duyn Schouten, Average optimal policies in Markov decision drift processes with applications to a queueing and a replacement model,Adv. in Appl. Probab.,15 (1983), 274–303.
A. Hordijk and F. A. van der Duyn Schouten, Discretization and weak convergence in Markov decision drift processes,Math. Oper. Res.,9 (1984), 112–141.
H. J. Kushner, Approximations and computational methods for optimal stopping and stochastic impulsive control problems,Appl. Math. Optim.,3 (1977), 81–100.
H. J. Kushner,Probability Methods for Approximations in Stochastic Control and for Elliptic Equations, Academic Press, New York, 1977.
J. P. Lepeltier and B. Marchal, Théorie générale du contrôle impulsionnel Markovien,SIAM J. Control Optim.,22 (1984), 645–665.
P. A. Meyer, Renaissance, recollements, mélanges, Ralentissement de processus de Markov,Ann. Inst. Fourier (Grenoble),XXV (1975), 465–495.
A. Schornagel, Optimum Preventive Maintenance for Complex Systems with Independent Equipment Units, preprint, Shell Research Laboratory, Amsterdam, 1987.
F. A. van der Duyn Schouten, Markov Decision Drift Processes, CWI Tract, Centre for Mathematics and Computer Science, Amsterdam, 1983.
D. Williams,Diffusions, Markov Processes & Martingales, Vol. 1, Wiley, New York, 1979.
A. A. Yushkevich, Continuous-time Markov decision processes with intervention,Stochastics,9 (1983), 235–274.
A. A. Yushkevich, Bellman inequalities in Markov decision deterministic drift processes,Stochastics,23 (1987), 25–77.
J. Zabczyk, Stopping problems in stochastic control,Proceedings of the International Congress of Mathematicians, August 16–24, 1983, Warszawa.
J. Zabczyk, Lectures in Stochastic Control, Control Theory Centre Report No. 125, University of Warwick, 1984.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Costa, O.L.V., Davis, M.H.A. Impulse control of piecewise-deterministic processes. Math. Control Signal Systems 2, 187–206 (1989). https://doi.org/10.1007/BF02551384
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF02551384