This special issue is concerned with the optimal control of systems over networks and the use in ... more This special issue is concerned with the optimal control of systems over networks and the use in transport system applications.
In this paper, we consider the optimal coordination of automated vehicles at intersections under ... more In this paper, we consider the optimal coordination of automated vehicles at intersections under fixed crossing orders. We formulate the problem using direct optimal control and exploit the structure to construct a semi-distributed primal-dual interior-point algorithm to solve it by parallelizing most of the computations. Differently from standard distributed optimization algorithms, where the optimization problem is split, in our approach we split the linear algebra steps, such that the algorithm takes the same steps as a fully centralized one, while still performing computations in a distributed fashion. We analyze the communication requirements of the algorithm, and propose an approximation scheme which can significantly reduce the data exchange. We demonstrate the effectiveness of the algorithm in hard but realistic scenarios, which show that the approximation leads to reductions in communicated data of almost 99% of the exact formulation, at the expense of less than 1% suboptim...
Linear models with additive unknown-but-bounded input disturbances are extensively used to model ... more Linear models with additive unknown-but-bounded input disturbances are extensively used to model uncertainty in robust control systems design. Typically, the disturbance set is either assumed to be known a priori or estimated from data through set-membership identification. However, the problem of computing a suitable input disturbance set in case the set of possible output values is assigned a priori has received relatively little attention. This problem arises in many contexts, such as in supervisory control, actuator design, decentralized control, and others. In this paper, we propose a method to compute input disturbance sets (and the corresponding set of states) such that the resulting set of outputs matches as closely as possible a given set of outputs, while additionally satisfying strict (inner or outer) inclusion constraints. We formulate the problem as an optimization problem by relying on the concept of robust invariance. The effectiveness of the approach is demonstrated ...
We present a mathematical model to predict pedestrian motion over a finite horizon, intended for ... more We present a mathematical model to predict pedestrian motion over a finite horizon, intended for use in collision avoidance algorithms for autonomous driving. The model is based on a road map structure, and assumes a rational pedestrian behavior. We compare our model with the state-of-the art and discuss its accuracy, and limitations, both in simulations and in comparison to real data.
Economic Model Predictive Control has recently gained popularity due to its ability to directly o... more Economic Model Predictive Control has recently gained popularity due to its ability to directly optimize a given performance criterion, while enforcing constraint satisfaction for nonlinear systems. Recent research has developed both numerical algorithms and stability analysis for the undiscounted case. The introduction of a discount factor in the cost, however, can be desirable in some cases of interest, e.g., economics, stochastically terminating processes, Markov decision processes, etc. Unfortunately, the stability theory in this case is still not fully developed. In this paper we propose a new dissipativity condition to prove asymptotic stability in the infinite horizon case and we connect our results with existing ones in the literature on discounted economic optimal control. Numerical examples are provided to illustrate the theoretical results.
Re-planning in legged locomotion is crucial to track a given set-point while adapting to the terr... more Re-planning in legged locomotion is crucial to track a given set-point while adapting to the terrain and rejecting external disturbances. In this work, we propose a real-time Nonlinear Model Predictive Control (NMPC) tailored to a legged robot for achieving dynamic locomotion on a wide variety of terrains. We introduce a mobility-based criterion to define an NMPC cost that enhances the locomotion of quadruped robots while maximizing leg mobility and staying far from kinematic limits. Our NMPC is based on the real-time iteration scheme that allows us to re-plan online at 25Hz with a time horizon of 2 seconds. We use the single rigid body dynamic model defined in the center of mass frame that allows to increase the computational efficiency. In simulations, the NMPC is tested to traverse a set of pallets of different sizes, to walk into a V-shaped chimney, and to locomote over rough terrain. We demonstrate the effectiveness of our NMPC with the mobility feature that allowed IIT’s 87.4 ...
Dissipativity theory is central to discussing the stability of policies resulting from minimzing ... more Dissipativity theory is central to discussing the stability of policies resulting from minimzing economic stage costs. In its current form, the dissipativity theory applies to problems based on deterministic dynamics, and does not readily extends to Markov Decision Processes, where the dynamics are stochastic. In this paper, we clarify the core reason for this difficulty, and propose a generalization of the dissipativity theory that circumvents it. This generalization is based on nonlinear stage cost functionals, allowing one to discuss the Lyapunov asymptotic stability of policies for Markov Decision Processes in the set of probability measures. This theory is illustrated in the stochastic Linear Quadratic Regulator case, for which a storage functional can be provided analytically. For the sake of brevity, we limit our discussion to undiscounted MDPs.
We present a framework for urban autonomous driving under dynamic constraints. Nonlinear model pr... more We present a framework for urban autonomous driving under dynamic constraints. Nonlinear model predictive control is used for longitudinal and lateral control, while a recently developed method is generating pedestrian predictions in time. We evaluate the performance using different intersection scenarios with moving pedestrians.
This thesis is concerned with optimal control techniques for optimal trajectory planning and real... more This thesis is concerned with optimal control techniques for optimal trajectory planning and real-time control and estimation. The framework of optimal control is a powerful tool which enjoys increasing popularity due to its applicability to a wide class of problems and its ability to deliver solutions to very complicated problems which cannot be intuitively solved. The downside of optimal control is the computational burden required to compute the optimal solution. Due to recent algorithmic developments and increases in the computational power, this burden has been significantly reduced over the last decades. In order to guarantee effectiveness and reliability of the solver, three main components are necessary: fast and robust algorithms, a good problem formulation, and a mathematical model tailored to optimisation. Indeed, both the model and the optimal control problem can usually be formulated in many different ways, some of which are better suited for optimisation. In this thesi...
Reinforcement Learning (RL) has recently impressed the world with stunning results in various app... more Reinforcement Learning (RL) has recently impressed the world with stunning results in various applications. While the potential of RL is now well-established, many critical aspects still need to be tackled, including safety and stability issues. These issues, while partially neglected by the RL community, are central to the control community which has been widely investigating them. Model Predictive Control (MPC) is one of the most successful control techniques because, among others, of its ability to provide such guarantees even for uncertain constrained systems. Since MPC is an optimization-based technique, optimality has also often been claimed. Unfortunately, the performance of MPC is highly dependent on the accuracy of the model used for predictions. In this paper, we propose to combine RL and MPC in order to exploit the advantages of both and, therefore, obtain a controller which is optimal and safe. We illustrate the results with a numerical example in simulations.
Aim of this exercise is to formulate and solve optimal control problems using YALMIP together wit... more Aim of this exercise is to formulate and solve optimal control problems using YALMIP together with the convex solver SDPT3. We start with a standard linear quadratic optimal control problem as it arises in MPC, and then add an elliptical terminal constraint. Then we add a nonconvex equality constraint and treat the problem with Sequential Convex Programming (SCP). In the appendix is information on YALMIP and on SCP.
In this paper we address the problem of coordinating automated vehicles at intersections, with a ... more In this paper we address the problem of coordinating automated vehicles at intersections, with a special focus on turning maneuvers. The inclusion of rear-end collision avoidance constraints into the problem is decided during turning maneuvers by a smooth function of the vehicle state, rather than integer variables. Moreover, curvature-based acceleration constraints are introduced, which limit the velocity of the vehicle during the turn, and a term in the objective function accounts for passenger comfort. We discuss how the coordination problem is formulated as a nonlinear program and show though simulations that for practical problem instances the proposed approximation is either exact or introduces very little conservativeness.
In this paper, we consider solving discounted Markov Decision Processes (MDPs) under the constrai... more In this paper, we consider solving discounted Markov Decision Processes (MDPs) under the constraint that the resulting policy is stabilizing. In practice MDPs are solved based on some form of policy approximation. We will leverage recent results proposing to use Model Predictive Control (MPC) as a structured policy in the context of Reinforcement Learning to make it possible to introduce stability requirements directly inside the MPC-based policy. This will restrict the solution of the MDP to stabilizing policies by construction. The stability theory for MPC is most mature for the undiscounted MPC case. Hence, we will first show in this paper that stable discounted MDPs can be reformulated as undiscounted ones. This observation will entail that the MPC-based policy with stability requirements will produce the optimal policy for the discounted MDP if it is stable, and the best stabilizing policy otherwise.
This special issue is concerned with the optimal control of systems over networks and the use in ... more This special issue is concerned with the optimal control of systems over networks and the use in transport system applications.
In this paper, we consider the optimal coordination of automated vehicles at intersections under ... more In this paper, we consider the optimal coordination of automated vehicles at intersections under fixed crossing orders. We formulate the problem using direct optimal control and exploit the structure to construct a semi-distributed primal-dual interior-point algorithm to solve it by parallelizing most of the computations. Differently from standard distributed optimization algorithms, where the optimization problem is split, in our approach we split the linear algebra steps, such that the algorithm takes the same steps as a fully centralized one, while still performing computations in a distributed fashion. We analyze the communication requirements of the algorithm, and propose an approximation scheme which can significantly reduce the data exchange. We demonstrate the effectiveness of the algorithm in hard but realistic scenarios, which show that the approximation leads to reductions in communicated data of almost 99% of the exact formulation, at the expense of less than 1% suboptim...
Linear models with additive unknown-but-bounded input disturbances are extensively used to model ... more Linear models with additive unknown-but-bounded input disturbances are extensively used to model uncertainty in robust control systems design. Typically, the disturbance set is either assumed to be known a priori or estimated from data through set-membership identification. However, the problem of computing a suitable input disturbance set in case the set of possible output values is assigned a priori has received relatively little attention. This problem arises in many contexts, such as in supervisory control, actuator design, decentralized control, and others. In this paper, we propose a method to compute input disturbance sets (and the corresponding set of states) such that the resulting set of outputs matches as closely as possible a given set of outputs, while additionally satisfying strict (inner or outer) inclusion constraints. We formulate the problem as an optimization problem by relying on the concept of robust invariance. The effectiveness of the approach is demonstrated ...
We present a mathematical model to predict pedestrian motion over a finite horizon, intended for ... more We present a mathematical model to predict pedestrian motion over a finite horizon, intended for use in collision avoidance algorithms for autonomous driving. The model is based on a road map structure, and assumes a rational pedestrian behavior. We compare our model with the state-of-the art and discuss its accuracy, and limitations, both in simulations and in comparison to real data.
Economic Model Predictive Control has recently gained popularity due to its ability to directly o... more Economic Model Predictive Control has recently gained popularity due to its ability to directly optimize a given performance criterion, while enforcing constraint satisfaction for nonlinear systems. Recent research has developed both numerical algorithms and stability analysis for the undiscounted case. The introduction of a discount factor in the cost, however, can be desirable in some cases of interest, e.g., economics, stochastically terminating processes, Markov decision processes, etc. Unfortunately, the stability theory in this case is still not fully developed. In this paper we propose a new dissipativity condition to prove asymptotic stability in the infinite horizon case and we connect our results with existing ones in the literature on discounted economic optimal control. Numerical examples are provided to illustrate the theoretical results.
Re-planning in legged locomotion is crucial to track a given set-point while adapting to the terr... more Re-planning in legged locomotion is crucial to track a given set-point while adapting to the terrain and rejecting external disturbances. In this work, we propose a real-time Nonlinear Model Predictive Control (NMPC) tailored to a legged robot for achieving dynamic locomotion on a wide variety of terrains. We introduce a mobility-based criterion to define an NMPC cost that enhances the locomotion of quadruped robots while maximizing leg mobility and staying far from kinematic limits. Our NMPC is based on the real-time iteration scheme that allows us to re-plan online at 25Hz with a time horizon of 2 seconds. We use the single rigid body dynamic model defined in the center of mass frame that allows to increase the computational efficiency. In simulations, the NMPC is tested to traverse a set of pallets of different sizes, to walk into a V-shaped chimney, and to locomote over rough terrain. We demonstrate the effectiveness of our NMPC with the mobility feature that allowed IIT’s 87.4 ...
Dissipativity theory is central to discussing the stability of policies resulting from minimzing ... more Dissipativity theory is central to discussing the stability of policies resulting from minimzing economic stage costs. In its current form, the dissipativity theory applies to problems based on deterministic dynamics, and does not readily extends to Markov Decision Processes, where the dynamics are stochastic. In this paper, we clarify the core reason for this difficulty, and propose a generalization of the dissipativity theory that circumvents it. This generalization is based on nonlinear stage cost functionals, allowing one to discuss the Lyapunov asymptotic stability of policies for Markov Decision Processes in the set of probability measures. This theory is illustrated in the stochastic Linear Quadratic Regulator case, for which a storage functional can be provided analytically. For the sake of brevity, we limit our discussion to undiscounted MDPs.
We present a framework for urban autonomous driving under dynamic constraints. Nonlinear model pr... more We present a framework for urban autonomous driving under dynamic constraints. Nonlinear model predictive control is used for longitudinal and lateral control, while a recently developed method is generating pedestrian predictions in time. We evaluate the performance using different intersection scenarios with moving pedestrians.
This thesis is concerned with optimal control techniques for optimal trajectory planning and real... more This thesis is concerned with optimal control techniques for optimal trajectory planning and real-time control and estimation. The framework of optimal control is a powerful tool which enjoys increasing popularity due to its applicability to a wide class of problems and its ability to deliver solutions to very complicated problems which cannot be intuitively solved. The downside of optimal control is the computational burden required to compute the optimal solution. Due to recent algorithmic developments and increases in the computational power, this burden has been significantly reduced over the last decades. In order to guarantee effectiveness and reliability of the solver, three main components are necessary: fast and robust algorithms, a good problem formulation, and a mathematical model tailored to optimisation. Indeed, both the model and the optimal control problem can usually be formulated in many different ways, some of which are better suited for optimisation. In this thesi...
Reinforcement Learning (RL) has recently impressed the world with stunning results in various app... more Reinforcement Learning (RL) has recently impressed the world with stunning results in various applications. While the potential of RL is now well-established, many critical aspects still need to be tackled, including safety and stability issues. These issues, while partially neglected by the RL community, are central to the control community which has been widely investigating them. Model Predictive Control (MPC) is one of the most successful control techniques because, among others, of its ability to provide such guarantees even for uncertain constrained systems. Since MPC is an optimization-based technique, optimality has also often been claimed. Unfortunately, the performance of MPC is highly dependent on the accuracy of the model used for predictions. In this paper, we propose to combine RL and MPC in order to exploit the advantages of both and, therefore, obtain a controller which is optimal and safe. We illustrate the results with a numerical example in simulations.
Aim of this exercise is to formulate and solve optimal control problems using YALMIP together wit... more Aim of this exercise is to formulate and solve optimal control problems using YALMIP together with the convex solver SDPT3. We start with a standard linear quadratic optimal control problem as it arises in MPC, and then add an elliptical terminal constraint. Then we add a nonconvex equality constraint and treat the problem with Sequential Convex Programming (SCP). In the appendix is information on YALMIP and on SCP.
In this paper we address the problem of coordinating automated vehicles at intersections, with a ... more In this paper we address the problem of coordinating automated vehicles at intersections, with a special focus on turning maneuvers. The inclusion of rear-end collision avoidance constraints into the problem is decided during turning maneuvers by a smooth function of the vehicle state, rather than integer variables. Moreover, curvature-based acceleration constraints are introduced, which limit the velocity of the vehicle during the turn, and a term in the objective function accounts for passenger comfort. We discuss how the coordination problem is formulated as a nonlinear program and show though simulations that for practical problem instances the proposed approximation is either exact or introduces very little conservativeness.
In this paper, we consider solving discounted Markov Decision Processes (MDPs) under the constrai... more In this paper, we consider solving discounted Markov Decision Processes (MDPs) under the constraint that the resulting policy is stabilizing. In practice MDPs are solved based on some form of policy approximation. We will leverage recent results proposing to use Model Predictive Control (MPC) as a structured policy in the context of Reinforcement Learning to make it possible to introduce stability requirements directly inside the MPC-based policy. This will restrict the solution of the MDP to stabilizing policies by construction. The stability theory for MPC is most mature for the undiscounted MPC case. Hence, we will first show in this paper that stable discounted MDPs can be reformulated as undiscounted ones. This observation will entail that the MPC-based policy with stability requirements will produce the optimal policy for the discounted MDP if it is stable, and the best stabilizing policy otherwise.
Model Predictive Control (MPC) formulations are typically built on the requirement that a feasibl... more Model Predictive Control (MPC) formulations are typically built on the requirement that a feasible reference trajectory is available. In practical settings, however, references that are infeasible with respect to the system dynamics are used for convenience. In this paper, we prove under which conditions an MPC formulation is Input-to-State Stable (ISS) in closedloop when an infeasible reference is used, and that with proper terminal conditions, asymptotic stability towards an optimal reference may be achieved. We illustrate the theoretical results with a four-dimensional robotic joint example.
Uploads