This paper studies the combined maneuver of flying and sailing for a robotic system which is refe... more This paper studies the combined maneuver of flying and sailing for a robotic system which is referred to as a flying+sailing drone. Due to the emergence of hybrid systems behavior in tasks which involve both the flying and sailing modes, a hybrid systems formulation of the robotic system is presented. Key characteristics of the system are (i) changes in the dimension of the state space as the system switches from flying to sailing and vice versa and (ii) the presence of autonomous switchings triggered only upon the landing of the drone on the water surface. For the scenario in which the drone’s initial state is given in the flying mode and a fixed terminal state is specified in the sailing mode, the associated optimal control problems are studied within the vertical plane passing through the given points, hence the dynamics of the drone in the flying mode are represented in a five-dimensional state space (associated with three degrees-of-freedom) and in a three-dimensional state spa...
2021 60th IEEE Conference on Decision and Control (CDC), 2021
Novel numerical estimators are proposed for the forward-backward stochastic differential equation... more Novel numerical estimators are proposed for the forward-backward stochastic differential equations (FBSDE) appearing in the Feynman-Kac representation of the value function. In contrast to the current numerical approaches based on discretization of the continuous-time FBSDE results, we propose a converse approach, by first obtaining a discrete-time approximation of the on-policy value function, and then developing a discrete-time result which resembles the continuous-time counterpart. This approach yields improved numerical estimators in the function approximation phase, and demonstrates enhanced error analysis for those value function estimators. Numerical results and error analysis are demonstrated on a scalar nonlinear stochastic optimal control problem, and they show improvements in the performance of the proposed estimators in comparison with the state-of-the-art methodologies.
2021 60th IEEE Conference on Decision and Control (CDC), 2021
The presence of terminal state constraints in terms of expectations is studied for steering the s... more The presence of terminal state constraints in terms of expectations is studied for steering the state of partially observed linear stochastic systems. Three scenarios for the observation process are considered, namely, (i) continuous-time exact observations of the state, (ii) discrete-time exact observations of the state, and (iii) discrete-time exact observations of the state accompanied by continuous-time noisy observations of the state. Closed form expressions are presented for the optimal inputs enforcing the terminal state constraint under these information structures, which are expressed in terms of controllability Gramians and solutions of Riccati and Lyapunov equations. Numerical examples are provided to illustrate the results.
In order to optimally steer the state of a stochastic system to a desired value over a finite tim... more In order to optimally steer the state of a stochastic system to a desired value over a finite time horizon, a novel approach based on the Stochastic Minimum Principle is presented, which enforces a constraint on the expectation of the terminal state at all instances of time. In order to solve the associated optimal control problem, we invoke a version of the Stochastic Minimum Principle which we call the Terminally Constrained Stochastic Minimum Principle (TC-SMP). For linear stochastic systems with quadratic costs, analytical solutions to the adjoint equation of the TC-SMP are derived and are explicitly represented in terms of controllability Gramians and solutions of Riccati equations. Numerical examples are provided to illustrate the results, and the performance of the TC-SMP approach is compared to both penalty-based and covariance-steering alternative approaches.
The paper combines two major contemporary systems and control methodologies to obtain a unique -N... more The paper combines two major contemporary systems and control methodologies to obtain a unique -Nash equilibrium for optimal execution problems within the stock market, namely Mean Field Game (MFG) theory and Hybrid Optimal Control (HOC) theory. Following standard financial models, the stock market is studied in this paper as a large population non-cooperative game where each trader has stochastic linear dynamics with quadratic costs. We consider the case where there exists one major trader with significant influence on market movements together with a large number of minor traders (within two subpopulations), each with individually asymptotically negligible effect on the market. The traders are coupled in their dynamics and cost functions by the market’s average trading rate (a component of the system mean field) and the hybrid feature enters via the indexing of the cessation of trading by one or both subpopulations of minor traders by discrete states. Optimal stopping time strateg...
This paper studies the combined maneuver of flying and sailing for a robotic system which is refe... more This paper studies the combined maneuver of flying and sailing for a robotic system which is referred to as a flying+sailing drone. Due to the emergence of hybrid systems behavior in tasks which involve both the flying and sailing modes, a hybrid systems formulation of the robotic system is presented. Key characteristics of the system are (i) changes in the dimension of the state space as the system switches from flying to sailing and vice versa and (ii) the presence of autonomous switchings triggered only upon the landing of the drone on the water surface. For the scenario in which the drone’s initial state is given in the flying mode and a fixed terminal state is specified in the sailing mode, the associated optimal control problems are studied within the vertical plane passing through the given points, hence the dynamics of the drone in the flying mode are represented in a five-dimensional state space (associated with three degrees-of-freedom) and in a three-dimensional state spa...
2021 60th IEEE Conference on Decision and Control (CDC), 2021
Novel numerical estimators are proposed for the forward-backward stochastic differential equation... more Novel numerical estimators are proposed for the forward-backward stochastic differential equations (FBSDE) appearing in the Feynman-Kac representation of the value function. In contrast to the current numerical approaches based on discretization of the continuous-time FBSDE results, we propose a converse approach, by first obtaining a discrete-time approximation of the on-policy value function, and then developing a discrete-time result which resembles the continuous-time counterpart. This approach yields improved numerical estimators in the function approximation phase, and demonstrates enhanced error analysis for those value function estimators. Numerical results and error analysis are demonstrated on a scalar nonlinear stochastic optimal control problem, and they show improvements in the performance of the proposed estimators in comparison with the state-of-the-art methodologies.
2021 60th IEEE Conference on Decision and Control (CDC), 2021
The presence of terminal state constraints in terms of expectations is studied for steering the s... more The presence of terminal state constraints in terms of expectations is studied for steering the state of partially observed linear stochastic systems. Three scenarios for the observation process are considered, namely, (i) continuous-time exact observations of the state, (ii) discrete-time exact observations of the state, and (iii) discrete-time exact observations of the state accompanied by continuous-time noisy observations of the state. Closed form expressions are presented for the optimal inputs enforcing the terminal state constraint under these information structures, which are expressed in terms of controllability Gramians and solutions of Riccati and Lyapunov equations. Numerical examples are provided to illustrate the results.
In order to optimally steer the state of a stochastic system to a desired value over a finite tim... more In order to optimally steer the state of a stochastic system to a desired value over a finite time horizon, a novel approach based on the Stochastic Minimum Principle is presented, which enforces a constraint on the expectation of the terminal state at all instances of time. In order to solve the associated optimal control problem, we invoke a version of the Stochastic Minimum Principle which we call the Terminally Constrained Stochastic Minimum Principle (TC-SMP). For linear stochastic systems with quadratic costs, analytical solutions to the adjoint equation of the TC-SMP are derived and are explicitly represented in terms of controllability Gramians and solutions of Riccati equations. Numerical examples are provided to illustrate the results, and the performance of the TC-SMP approach is compared to both penalty-based and covariance-steering alternative approaches.
The paper combines two major contemporary systems and control methodologies to obtain a unique -N... more The paper combines two major contemporary systems and control methodologies to obtain a unique -Nash equilibrium for optimal execution problems within the stock market, namely Mean Field Game (MFG) theory and Hybrid Optimal Control (HOC) theory. Following standard financial models, the stock market is studied in this paper as a large population non-cooperative game where each trader has stochastic linear dynamics with quadratic costs. We consider the case where there exists one major trader with significant influence on market movements together with a large number of minor traders (within two subpopulations), each with individually asymptotically negligible effect on the market. The traders are coupled in their dynamics and cost functions by the market’s average trading rate (a component of the system mean field) and the hybrid feature enters via the indexing of the cessation of trading by one or both subpopulations of minor traders by discrete states. Optimal stopping time strateg...
— A class of stochastic hybrid systems with both autonomous and controlled switchings and jumps i... more — A class of stochastic hybrid systems with both autonomous and controlled switchings and jumps is considered where autonomous and controlled state jumps at the switching instants are accompanied by changes in the dimension of the state space. Optimal control problems associated with this class of stochastic hybrid systems are studied where in addition to running and terminal costs, switching between discrete states incurs costs. Necessary optimality conditions are established in the form of the Stochastic Hybrid Minimum Principle. A feature of special importance is the effect of hard constraints imposed by switching manifolds on diffusion-driven state trajectories which influence the boundary conditions for the stochastic Hamiltonian and adjoint processes.
Uploads
Papers by Ali Pakniyat