We consider the class of Markov kernels for which the weak or strong Feller property fails to hol... more We consider the class of Markov kernels for which the weak or strong Feller property fails to hold at some discontinuity set. We provide a sim-ple necessary and sufficient condition for existence of an invariant probabi-lity measure as well as a Foster-Lyapunov sufficient condition. We also characterize a subclass, the quasi (weak or strong) Feller kernels, for which the sequences of expected occupation measures share the same asymptotic properties as for (weak or strong) Feller kernels. In particular, it is shown that the sequences of expected occupation measures of strong and quasi strong-Feller kernels with an invariant probability measure converge set-wise to an invariant measure.
Further Topics on Discrete-Time Markov Control Processes, 1999
In this chapter we study AC-related criteria, some of which have already been studied in previous... more In this chapter we study AC-related criteria, some of which have already been studied in previous chapters from a different viewpoint. We begin by introducing some notation and definitions, and then we outline the contents of this chapter.
As pointed out at the end of the previous chapter, route-based equilibrium models present several... more As pointed out at the end of the previous chapter, route-based equilibrium models present several drawbacks from the modeling and computational viewpoints. In particular, the independence of route travel times is not well suited when dealing with overlapping paths.
Roughly speaking, given a set of parameters Y and an optimization problem whose description depen... more Roughly speaking, given a set of parameters Y and an optimization problem whose description depends on \(y \in Y\) (call it P y), parametric optimization is concerned with the behavior and properties of the optimal value as well as primal (and possibly dual) optimal solutions of P y, when y varies in Y. This is a quite challenging problem for which in general only local information around some nominal value y0 of the parameter can be obtained. Sometimes, in the context of optimization with data uncertainty, some probability distribution on the parameter set Y is available and in this context one is also interested in, e.g., the distributions of the optimal value and optimal solutions, all viewed as random variables.
This paper studies the policy iteration algorithm (PIA) for average cost Markov control processes... more This paper studies the policy iteration algorithm (PIA) for average cost Markov control processes on Borel spaces. Two classes of MCPs are considered. One of them allows some restricted-growth unbounded cost functions and compact control constraint sets; the other one requires strictly unbounded costs and the control constraint sets may be non-compact. For each of these classes, the PIA yields,
Considers the vanishing-discount-factor approach to investigate average optimality in Markov cont... more Considers the vanishing-discount-factor approach to investigate average optimality in Markov control processes with Borel state and action spaces. Under weak assumptions on the discounted differential value function the authors derive existence of average optimal stationary policies for initial states in a possibly proper subset of the state space. This is in contrast to previous works where an implicit “unichain” assumption ensures optimality for all initial states
We consider the class of Markov kernels for which the weak or strong Feller property fails to hol... more We consider the class of Markov kernels for which the weak or strong Feller property fails to hold at some discontinuity set. We provide a sim-ple necessary and sufficient condition for existence of an invariant probabi-lity measure as well as a Foster-Lyapunov sufficient condition. We also characterize a subclass, the quasi (weak or strong) Feller kernels, for which the sequences of expected occupation measures share the same asymptotic properties as for (weak or strong) Feller kernels. In particular, it is shown that the sequences of expected occupation measures of strong and quasi strong-Feller kernels with an invariant probability measure converge set-wise to an invariant measure.
Further Topics on Discrete-Time Markov Control Processes, 1999
In this chapter we study AC-related criteria, some of which have already been studied in previous... more In this chapter we study AC-related criteria, some of which have already been studied in previous chapters from a different viewpoint. We begin by introducing some notation and definitions, and then we outline the contents of this chapter.
As pointed out at the end of the previous chapter, route-based equilibrium models present several... more As pointed out at the end of the previous chapter, route-based equilibrium models present several drawbacks from the modeling and computational viewpoints. In particular, the independence of route travel times is not well suited when dealing with overlapping paths.
Roughly speaking, given a set of parameters Y and an optimization problem whose description depen... more Roughly speaking, given a set of parameters Y and an optimization problem whose description depends on \(y \in Y\) (call it P y), parametric optimization is concerned with the behavior and properties of the optimal value as well as primal (and possibly dual) optimal solutions of P y, when y varies in Y. This is a quite challenging problem for which in general only local information around some nominal value y0 of the parameter can be obtained. Sometimes, in the context of optimization with data uncertainty, some probability distribution on the parameter set Y is available and in this context one is also interested in, e.g., the distributions of the optimal value and optimal solutions, all viewed as random variables.
This paper studies the policy iteration algorithm (PIA) for average cost Markov control processes... more This paper studies the policy iteration algorithm (PIA) for average cost Markov control processes on Borel spaces. Two classes of MCPs are considered. One of them allows some restricted-growth unbounded cost functions and compact control constraint sets; the other one requires strictly unbounded costs and the control constraint sets may be non-compact. For each of these classes, the PIA yields,
Considers the vanishing-discount-factor approach to investigate average optimality in Markov cont... more Considers the vanishing-discount-factor approach to investigate average optimality in Markov control processes with Borel state and action spaces. Under weak assumptions on the discounted differential value function the authors derive existence of average optimal stationary policies for initial states in a possibly proper subset of the state space. This is in contrast to previous works where an implicit “unichain” assumption ensures optimality for all initial states
Uploads