Showing 1–14 of 14 results for author: Jia, G

Inverse reinforcement learning by expert imitation for the stochastic linear-quadratic optimal control problem

Authors: Zhongshi Sun, Guangyan Jia

Abstract: This article studies inverse reinforcement learning (IRL) for the stochastic linear-quadratic optimal control problem, where two agents are considered. A learner agent does not know the expert agent's performance cost function, but it imitates the behavior of the expert agent by constructing an underlying cost function that obtains the same optimal feedback control as the expert's. We first develo… ▽ More This article studies inverse reinforcement learning (IRL) for the stochastic linear-quadratic optimal control problem, where two agents are considered. A learner agent does not know the expert agent's performance cost function, but it imitates the behavior of the expert agent by constructing an underlying cost function that obtains the same optimal feedback control as the expert's. We first develop a model-based IRL algorithm, which consists of a policy correction and a policy update from the policy iteration in reinforcement learning, as well as a cost function weight reconstruction based on the inverse optimal control. Then, under this scheme, we propose a model-free off-policy IRL algorithm, which does not need to know or identify the system and only needs to collect the behavior data of the expert agent and learner agent once during the iteration process. Moreover, the proofs of the algorithm's convergence, stability, and non-unique solutions are given. Finally, a simulation example is provided to verify the effectiveness of the proposed algorithm. △ Less

Submitted 27 May, 2024; originally announced May 2024.

Convergence of Policy Gradient for Stochastic Linear-Quadratic Control Problem in Infinite Horizon

Authors: Xinpei Zhang, Guangyan Jia

Abstract: With the outstanding performance of policy gradient (PG) method in the reinforcement learning field, the convergence theory of it has aroused more and more interest recently. Meanwhile, the significant importance and abundant theoretical researches make the stochastic linear quadratic (SLQ) control problem a starting point for studying PG in model-based learning setting. In this paper, we study th… ▽ More With the outstanding performance of policy gradient (PG) method in the reinforcement learning field, the convergence theory of it has aroused more and more interest recently. Meanwhile, the significant importance and abundant theoretical researches make the stochastic linear quadratic (SLQ) control problem a starting point for studying PG in model-based learning setting. In this paper, we study the PG method for the SLQ problem in infinite horizon and take a step towards providing rigorous guarantees for gradient methods. Although the cost functional of linear-quadratic problem is typically nonconvex, we still overcome the difficulty based on gradient domination condition and L-smoothness property, and prove exponential/linear convergence of gradient flow/descent algorithm. △ Less

Submitted 18 April, 2024; v1 submitted 17 April, 2024; originally announced April 2024.

Robust policy iteration for continuous-time stochastic $H_\infty$ control problem with unknown dynamics

Authors: Zhongshi Sun, Guangyan Jia

Abstract: In this article, we study a continuous-time stochastic $H_\infty$ control problem based on reinforcement learning (RL) techniques that can be viewed as solving a stochastic linear-quadratic two-person zero-sum differential game (LQZSG). First, we propose an RL algorithm that can iteratively solve stochastic game algebraic Riccati equation based on collected state and control data when all dynamic… ▽ More In this article, we study a continuous-time stochastic $H_\infty$ control problem based on reinforcement learning (RL) techniques that can be viewed as solving a stochastic linear-quadratic two-person zero-sum differential game (LQZSG). First, we propose an RL algorithm that can iteratively solve stochastic game algebraic Riccati equation based on collected state and control data when all dynamic system information is unknown. In addition, the algorithm only needs to collect data once during the iteration process. Then, we discuss the robustness and convergence of the inner and outer loops of the policy iteration algorithm, respectively, and show that when the error of each iteration is within a certain range, the algorithm can converge to a small neighborhood of the saddle point of the stochastic LQZSG problem. Finally, we applied the proposed RL algorithm to two simulation examples to verify the effectiveness of the algorithm. △ Less

Submitted 7 February, 2024; originally announced February 2024.

Grid-Aware On-Route Fast-Charging Infrastructure Planning for Battery Electric Bus with Equity Considerations: A Case Study in South King County

Authors: Xinyi Zhao, Chaoyue Zhao, Grace Jia

Abstract: The transition from traditional bus fleets to zero-emission ones necessitates the development of effective planning models for battery electric bus (BEB) charging infrastructure. On-route fast charging stations, distinct from on-base charging stations, present unique challenges related to safe operation and power supply capacity, making it difficult to control grid operational costs. This paper es… ▽ More The transition from traditional bus fleets to zero-emission ones necessitates the development of effective planning models for battery electric bus (BEB) charging infrastructure. On-route fast charging stations, distinct from on-base charging stations, present unique challenges related to safe operation and power supply capacity, making it difficult to control grid operational costs. This paper establishes a novel framework that integrates the bus route network and power network, which leverages the inter-dependency between both networks to optimize the planning outcomes of on-route BEB charging stations in South King County. The problem is formulated as a mixed-integer second-order cone programming model, aiming to minimize the overall planning cost, which includes investments in charging equipment, power facility, and grid operation. Furthermore, fairness measurements are incorporated into the planning process, allowing for the consideration of both horizontal transit equity and vertical transit equity based on different zone merging criteria within the county's existing census tracts. The results of this planning model offer valuable insights into achieving both economic efficiency and social justice in the design of on-route charging facilities for BEBs in South King County. △ Less

Submitted 14 September, 2023; originally announced September 2023.

Comments: 18 pages, 16 figures

Lipschitz constants for a hyperbolic type metric under Möbius transformations

Authors: Yinping Wu, Gendi Wang, Gaili Jia, Xiaohui Zhang

Abstract: Let $D$ be a nonempty open set in a metric space $(X,d)$ with $\partial D\neq \emptyset$. Define \begin{equation*} h_{D,c}(x,y)=\log\left(1+c\frac{d(x,y)}{\sqrt{d_D(x)d_D(y)}}\right), \end{equation*} where $d_D(x)=d(x,\partial D)$ is the distance from $x$ to the boundary of $D$. For every $c\geq 2$, $h_{D,c}$ is a metric. In this paper, we study the sharp Lipschitz constants for the metric… ▽ More Let $D$ be a nonempty open set in a metric space $(X,d)$ with $\partial D\neq \emptyset$. Define \begin{equation*} h_{D,c}(x,y)=\log\left(1+c\frac{d(x,y)}{\sqrt{d_D(x)d_D(y)}}\right), \end{equation*} where $d_D(x)=d(x,\partial D)$ is the distance from $x$ to the boundary of $D$. For every $c\geq 2$, $h_{D,c}$ is a metric. In this paper, we study the sharp Lipschitz constants for the metric $h_{D,c}$ under Möbius transformations of the unit ball, the upper half space, and the punctured unit ball. △ Less

Submitted 7 September, 2023; originally announced September 2023.

Comments: 18 pages

MSC Class: 51M10; 30C65

On the Correspondence and the Risk Contribution for Conditional Coherent and Deviation Risk Measures

Authors: Guangyan Jia, Mengjin Zhao

Abstract: We give an axiomatic framework for conditional generalized deviation measures. Under financially reasonable assumptions, we give the correspondence between conditional coherent risk measures and generalized deviation measures. Moreover, we establish the notion of continuous-time risk contribution for conditional coherent risk measures and generalized deviation measures. With the help of the corres… ▽ More We give an axiomatic framework for conditional generalized deviation measures. Under financially reasonable assumptions, we give the correspondence between conditional coherent risk measures and generalized deviation measures. Moreover, we establish the notion of continuous-time risk contribution for conditional coherent risk measures and generalized deviation measures. With the help of the correspondence between these two different types of risk measures, we give a microscopic interpretation of their risk contributions. Particularly, we show that the risk contributions of time-consistent risk measures are still time-consistent. We also demonstrate that the second element of the BSDE solution $(Y, Z)$ associated with $g$-expectation has the meaning of risk contribution. △ Less

Submitted 18 February, 2023; v1 submitted 28 August, 2022; originally announced August 2022.

On monotonicity and order-preservation for multidimensional G-diffusion processes

Authors: Peng Luo, Guangyan Jia

Abstract: In this paper, we prove a comparison theorem for multidimensional G-SDEs. Moreover we obtain respectively the sufficient conditions and necessary conditions of the monotonicity and order-preservation for two multidimensional G-diffusion processes. Finally, we give some applications. In this paper, we prove a comparison theorem for multidimensional G-SDEs. Moreover we obtain respectively the sufficient conditions and necessary conditions of the monotonicity and order-preservation for two multidimensional G-diffusion processes. Finally, we give some applications. △ Less

Submitted 4 October, 2014; v1 submitted 19 February, 2014; originally announced February 2014.

Comments: arXiv admin note: text overlap with arXiv:1212.5403, arXiv:1306.1929 by other authors. text overlap with arXiv:1212.5403, arXiv:1306.1929, arXiv:0910.3871 by other authors

A note on characterizations of G-normal distribution

Authors: Peng Luo, Guangyan Jia

Abstract: In this paper, we show that the G-normality of X and Y can be characterized according to the form of f such that the distribution of λ+f(λ)Y does not depend on λ, where Y is an independent copy of X and λ is in the domain of f. Without the condition that Y is identically distributed with X, we still have a similar argument. In this paper, we show that the G-normality of X and Y can be characterized according to the form of f such that the distribution of λ+f(λ)Y does not depend on λ, where Y is an independent copy of X and λ is in the domain of f. Without the condition that Y is identically distributed with X, we still have a similar argument. △ Less

Submitted 21 August, 2015; v1 submitted 19 February, 2014; originally announced February 2014.

New Proofs for Several Properties of Capacities

Authors: Guangyan Jia, Na Zhang

Abstract: In this note, we find a new way to prove several properties of 2-alternating capacities. In this note, we find a new way to prove several properties of 2-alternating capacities. △ Less

Submitted 3 July, 2013; originally announced July 2013.

Comments: 8 pages

MSC Class: 28A12

Invariant representation for stochastic differential operator by BSDEs with uniformly continuous coefficients and its applications

Authors: Na Zhang, Guangyan Jia

Abstract: In this paper, we prove that a kind of second order stochastic differential operator can be represented by the limit of solutions of BSDEs with uniformly continuous coefficients. This result is a generalization of the representation for the uniformly continuous generator. With the help of this representation, we obtain the corresponding converse comparison theorem for the BSDEs with uniformly cont… ▽ More In this paper, we prove that a kind of second order stochastic differential operator can be represented by the limit of solutions of BSDEs with uniformly continuous coefficients. This result is a generalization of the representation for the uniformly continuous generator. With the help of this representation, we obtain the corresponding converse comparison theorem for the BSDEs with uniformly continuous coefficients, and get some equivalent relationships between the properties of the generator $g$ and the associated solutions of BSDEs. Moreover, we give a new proof about $g$-convexity. △ Less

Submitted 31 May, 2012; v1 submitted 2 March, 2012; originally announced March 2012.

Comments: This paper has been withdrawn by the author due to suspection to the last inequalities on Page 7. However, it turns out to be right

MSC Class: 60H10; 60H25; 60H30

The Equivalence between Uniqueness and Continuous Dependence of Solution for BSDEs with Continuous Coefficient

Authors: Guangyan Jia, Zhiyong Yu

Abstract: In this paper, we will prove that, if the coefficient $g=g(t,y,z)$ of a BSDE is assumed to be continuous and linear growth in $(y,z)$, then the uniqueness of solution and continuous dependence with respect to $g$ and the terminal value $ξ$ are equivalent. In this paper, we will prove that, if the coefficient $g=g(t,y,z)$ of a BSDE is assumed to be continuous and linear growth in $(y,z)$, then the uniqueness of solution and continuous dependence with respect to $g$ and the terminal value $ξ$ are equivalent. △ Less

Submitted 25 March, 2008; originally announced March 2008.

Comments: 6 pages

MSC Class: 60H10; 60H30

A uniqueness theorem for solution of BSDEs

Authors: Guangyan Jia

Abstract: In this note, we prove that if $g$ is uniformly continuous in $z$, uniformly with respect to $(\oo,t)$ and independent of $y$, the solution to the backward stochastic differential equation (BSDE) with generator $g$ is unique. In this note, we prove that if $g$ is uniformly continuous in $z$, uniformly with respect to $(\oo,t)$ and independent of $y$, the solution to the backward stochastic differential equation (BSDE) with generator $g$ is unique. △ Less

Submitted 5 February, 2008; originally announced February 2008.

MSC Class: 60H10

Jensen's Inequality for g-Convex Function under g-Expectation

Authors: Guangyan Jia, Shige Peng

Abstract: A real valued function defined on}$\mathbb{R}$ {\small is called}$g${\small --convex if it satisfies the following \textquotedblleft generalized Jensen's inequality\textquotedblright under a given}$g${\small -expectation, i.e., }$h(\mathbb{E}^{g}[X])\leq \mathbb{E}% ^{g}[h(X)]${\small, for all random variables}$X$ {\small such that both sides of the inequality are meaningful. In this paper we wi… ▽ More A real valued function defined on}$\mathbb{R}$ {\small is called}$g${\small --convex if it satisfies the following \textquotedblleft generalized Jensen's inequality\textquotedblright under a given}$g${\small -expectation, i.e., }$h(\mathbb{E}^{g}[X])\leq \mathbb{E}% ^{g}[h(X)]${\small, for all random variables}$X$ {\small such that both sides of the inequality are meaningful. In this paper we will give a necessary and sufficient conditions for a }$C^{2}${\small -function being}$% g ${\small -convex. We also studied some more general situations. We also studied}$g${\small -concave and}$g${\small -affine functions. △ Less

Submitted 4 February, 2008; originally announced February 2008.

Comments: 21 pages

MSC Class: 60H10

Construction and Uniqueness for reflected BSDE under linear increasing condition

Authors: G. Jia, Mingyu Xu

Abstract: In this paper, we study the uniqueness of the solution of reflected BSDE with one or two barriers, under continuous and linear increasing condition of generator $g$. Before that we study the construction of solution of of reflected BSDE with one or two barriers. In this paper, we study the uniqueness of the solution of reflected BSDE with one or two barriers, under continuous and linear increasing condition of generator $g$. Before that we study the construction of solution of of reflected BSDE with one or two barriers. △ Less

Submitted 24 January, 2008; originally announced January 2008.

MSC Class: 60H10