We are at the very beginning of time for the human race. It is not unreasonable that we grapple w... more We are at the very beginning of time for the human race. It is not unreasonable that we grapple with problems. But there are tens of thousands of years in the future. Our responsibility is to do what we can, learn what we can, improve the solutions, and pass them on.
High-fidelity state preparation represents a fundamental challenge in the application of quantum ... more High-fidelity state preparation represents a fundamental challenge in the application of quantum technology. While the majority of optimal control approaches use feedback to improve the controller, the controller itself often does not incorporate explicit state dependence. Here, we present a general framework for training deep feedback networks for open quantum systems with continuous weak measurement that allows a variety of system and control structures that are prohibitive by many other techniques and can in effect react to unmodeled effects through nonlinear filtering. Our approach benefits from characteristics of both stochastic sampling and gradient-based optimization methods yet does not require differentiability as in backpropagation approaches. We demonstrate that this method is efficient due to inherent parallelizability, robust to open system interactions, and outperforms landmark state-dependent feedback control results in simulation.
Systems involving Partial Differential Equations (PDEs) have recently become more popular among t... more Systems involving Partial Differential Equations (PDEs) have recently become more popular among the machine learning community. However prior methods usually treat infinite dimensional problems in finite dimensions with Reduced Order Models. This leads to committing to specific approximation schemes and subsequent derivation of control laws. Additionally, prior work does not consider spatio-temporal descriptions of noise that realistically represent the stochastic nature of physical systems. In this paper we suggest a new reinforcement learning framework that is mostly model-free for Stochastic PDEs with additive spacetime noise, based on variational optimization in infinite dimensions. In addition, our algorithm incorporates sparse representations that allow for efficient learning of feedback policies in high dimensions. We demonstrate the efficacy of the proposed approach with several simulated experiments on a variety of SPDEs.
A common approach for developing robotic systems leverages separate simulation and control softwa... more A common approach for developing robotic systems leverages separate simulation and control softwares. Although this approach requires minimal coordination and orchestration between softwares, the separation of simulation and control applications presents the designer with unnecessary challenges during development. This paper describes the Autonomous Robot Control and Simulation (ARCS) software, a unified control and simulation environment developed on a common software stack for real world control and simulated experimentation. This unification is attained through the use of a modular data pipeline that allows for data to be generated in simulation, taken from real hardware, or taken from log files. The unification of real world control and simulation software and the adaptability of this pipeline allows for some unique advantages. These include more meaningful virtual experimentation, a streamlined functionality to perform hardware in the loop tests, and the ability to replay collected log files from both simulation and real world testing. This allows more rapid iterative development cycles of autonomous behaviors. This software stack was used to develop three marine robotics platforms, which will serve as an example to discuss the advantages and disadvantages of this unified approach.
Rapidly Exploring Random Trees (RRTs) have gained significant attention due to provable propertie... more Rapidly Exploring Random Trees (RRTs) have gained significant attention due to provable properties such as completeness and asymptotic optimality. However, offline methods are only useful when the entire problem landscape is known a priori. Furthermore, many real world applications have problem scopes that are orders of magnitude larger than typical mazes and bug traps that require large numbers of samples to match typical sample densities, resulting in high computational effort for reasonably low-cost trajectories. In this paper we propose an online trajectory optimization algorithm for uncertain large environments using RRTs, which we call Locally Adaptive Rapidly Exploring Random Tree (LARRT). This is achieved through two main contributions. We use an adaptive local sampling region and adaptive sampling scheme which depend on states of the dynamic system and observations of obstacles. We also propose a localized approach to planning and re-planning through fixing the root node to the current vehicle state and adding tree update functions. LARRT is designed to leverage local problem scope to reduce computational complexity and obtain a total lower-cost solution compared to a classical RRT of a similar number of nodes. Using this technique we can ensure that popular variants of RRT will remain online even for prohibitively large planning problems by transforming a large trajectory optimization approach to one that resembles receding horizon optimization. Finally, we demonstrate our approach in simulation and discuss various algorithmic trade-offs of the proposed approach.
High fidelity state preparation represents a fundamental challenge in the application of quantum ... more High fidelity state preparation represents a fundamental challenge in the application of quantum technology. While the majority of optimal control approaches use feedback to improve the controller, the controller itself often does not incorporate explicit state dependence. Here, we present a general framework for training deep feedback networks for open quantum systems with quantum nondemolition measurement that allows a variety of system and control structures that are prohibitive by many other techniques and can in effect react to unmodeled effects through nonlinear filtering. We demonstrate that this method is efficient due to inherent parallelizability, robust to open system interactions, and outperforms landmark state feedback control results in simulation.
In this paper, we present a Lyapunov function-based optimization approach for designing state and... more In this paper, we present a Lyapunov function-based optimization approach for designing state and output feedback control laws for systems with polynomial nonlinearities. We use local polynomial expansions of a chosen order to approximate a higher-order nonlinear stochastic dynamical system, reformulate stochastic asymptotic stability conditions in the form of a nonlinear constrained optimization problem, and computationally determine the domain of attraction of the synthesized nonlinear controller on the original system. Finally, we illustrate the effectiveness of the proposed algorithm on two illustrative numerical examples.
Stochastic partial differential equations have been used to model diverse applications ranging fr... more Stochastic partial differential equations have been used to model diverse applications ranging from turbulence and plasma physics to partially observable stochastic controlled systems. This paper aims to bridge the gap between abstract theoretical results and implementable algorithms, by developing a variational optimization framework for controlling infinite-dimensional stochastic models. A measuretheoretic approach is employed for systems evolving on Hilbert spaces, which relies on connections between relative entropy and free energy. The derived scheme is applicable to a wide class of problems, and can be used for optimizing parameterized control policies. Our work creates new perspectives and directions for solving stochastic control problems constrained by partial differential equations.
Systems involving Partial Differential Equations (PDEs) have recently become more popular among t... more Systems involving Partial Differential Equations (PDEs) have recently become more popular among the machine learning community. However prior methods usually treat infinite dimensional problems in finite dimensions with Reduced Order Models. This leads to committing to specific approximation schemes and subsequent derivation of control laws. Additionally, prior work does not consider spatio-temporal descriptions of noise that realistically represent the stochastic nature of physical systems. In this paper we suggest a new reinforcement learning framework that is mostly model-free for Stochastic PDEs with additive spacetime noise, based on variational optimization in infinite dimensions. In addition, our algorithm incorporates sparse representations that allow for efficient learning of feedback policies in high dimensions. We demonstrate the efficacy of the proposed approach with several simulated experiments on a variety of SPDEs.
High fidelity state preparation represents a fundamental challenge in the application of quantum ... more High fidelity state preparation represents a fundamental challenge in the application of quantum technology. While the majority of optimal control approaches use feedback to improve the controller, the controller itself often does not incorporate explicit state dependence. Here, we present a general framework for training deep feedback networks for open quantum systems with quantum nondemolition measurement that allows a variety of system and control structures that are prohibitive by many other techniques and can in effect react to unmodeled effects through nonlinear filtering. We demonstrate that this method is efficient due to inherent parallelizability, robust to open system interactions, and outperforms landmark state feedback control results in simulation.
Volume 3: Modeling and Validation; Multi-Agent and Networked Systems; Path Planning and Motion Control; Tracking Control Systems; Unmanned Aerial Vehicles (UAVs) and Application; Unmanned Ground and Aerial Vehicles; Vibration in Mechanical Systems; Vibrations and Control of Systems; Vibrations: M..., 2018
Rapidly Exploring Random Trees (RRTs) have gained significant attention due to provable propertie... more Rapidly Exploring Random Trees (RRTs) have gained significant attention due to provable properties such as completeness and asymptotic optimality. However, offline methods are only useful when the entire problem landscape is known a priori. Furthermore, many real world applications have problem scopes that are orders of magnitude larger than typical mazes and bug traps that require large numbers of samples to match typical sample densities, resulting in high computational effort for reasonably low-cost trajectories. In this paper we propose an online trajectory optimization algorithm for uncertain large environments using RRTs, which we call Locally Adaptive Rapidly Exploring Random Tree (LARRT). This is achieved through two main contributions. We use an adaptive local sampling region and adaptive sampling scheme which depend on states of the dynamic system and observations of obstacles. We also propose a localized approach to planning and re-planning through fixing the root node to...
Systems involving Partial Differential Equations (PDEs) have recently become more popular among t... more Systems involving Partial Differential Equations (PDEs) have recently become more popular among the machine learning community. However prior methods usually treat infinite dimensional problems in finite dimensions with Reduced Order Models. This leads to committing to specific approximation schemes and subsequent derivation of control laws. Additionally, prior work does not consider spatio-temporal descriptions of noise that realistically represent the stochastic nature of physical systems. In this paper we suggest a new reinforcement learning framework that is mostly model-free for Stochastic PDEs with additive spacetime noise, based on variational optimization in infinite dimensions. In addition, our algorithm incorporates sparse representations that allow for efficient learning of feedback policies in high dimensions. We demonstrate the efficacy of the proposed approach with several simulated experiments on a variety of SPDEs.
There is a rising interest in Spatio-temporal systems described by Partial Differential Equations... more There is a rising interest in Spatio-temporal systems described by Partial Differential Equations (PDEs) among the control community. Not only are these systems challenging to control, but the sizing and placement of their actuation is an NP-hard problem on its own. Recent methods either discretize the space before optimziation, or apply tools from linear systems theory under restrictive linearity assumptions. In this work we consider control and actuator placement as a coupled optimization problem, and derive an optimization algorithm on Hilbert spaces for nonlinear PDEs with an additive spatio-temporal description of white noise. We study first and second order systems and in doing so, extend several results to the case of second order PDEs. The described approach is based on variational optimization, and performs joint RL-type optimization of the feedback control law and the actuator design over episodes. We demonstrate the efficacy of the proposed approach with several simulated experiments on a variety of SPDEs.
2018 Annual American Control Conference (ACC), 2018
In this paper, we present a Lyapunov function-based optimization approach for designing state and... more In this paper, we present a Lyapunov function-based optimization approach for designing state and output feedback control laws for systems with polynomial nonlinearities. We use local polynomial expansions of a chosen order to approximate a higher-order nonlinear stochastic dynamical system, reformulate stochastic asymptotic stability conditions in the form of a nonlinear constrained optimization problem, and computationally determine the domain of attraction of the synthesized nonlinear controller on the original system. Finally, we illustrate the effectiveness of the proposed algorithm on two illustrative numerical examples.
Stochastic partial differential equations have been used to model diverse applications ranging fr... more Stochastic partial differential equations have been used to model diverse applications ranging from turbulence and plasma physics to partially observable stochastic controlled systems. This paper aims to bridge the gap between abstract theoretical results and implementable algorithms, by developing a variational optimization framework for controlling infinite-dimensional stochastic models. A measure-theoretic approach is employed for systems evolving on Hilbert spaces, which relies on connections between relative entropy and free energy. The derived scheme is applicable to a wide class of problems, and can be used for optimizing parameterized control policies. Our work creates new perspectives and directions for solving stochastic control problems constrained by partial differential equations.
The design and evaluation of autonomous system behavior is commonly based on an archetypal repres... more The design and evaluation of autonomous system behavior is commonly based on an archetypal representation of a mission within a modeling and simulation environment. In real-world operation however, autonomous systems may encounter a multitude of examples of a given mission, each a variation on the theme. From a design standpoint, it is desirable to simulate the system on many instances of a mission to evaluate robustness, and, at times, the need arises to test an autonomous behavior on a completely different mission for which the behavior is hypothesized to be applicable. The amount of time and effort required to construct new mission models can be a challenge preventing such robust investigations. In response, we describe a flexible problem definition for what we term “event-based missions” to enable evaluation of single- and multi-vehicle autonomous behaviors across a set of missions within the category. Data structures arising from the problem definition enable rapid construction of new models of event-based missions, and the flexibility of the approach is demonstrated on three missions representing extreme cases within the category.
We consider the optimal control problem of a general nonlinear spatio-temporal system described b... more We consider the optimal control problem of a general nonlinear spatio-temporal system described by Partial Differential Equations (PDEs). Theory and algorithms for control of spatio-temporal systems are of rising interest among the automatic control community and exhibit numerous challenging characteristic from a control standpoint. Recent methods focus on finite-dimensional optimization techniques of a discretized finite dimensional ODE approximation of the infinite dimensional PDE system. In this paper, we derive a differential dynamic programming (DDP) framework for distributed and boundary control of spatio-temporal systems in infinite dimensions that is shown to generalize both the spatio-temporal LQR solution, and modern finite dimensional DDP frameworks. We analyze the convergence behavior and provide a proof of global convergence for the resulting system of continuous-time forward-backward equations. We explore and develop numerical approaches to handle sensitivities that ar...
Stochastic spatio-temporal processes are prevalent across domains ranging from the modeling of pl... more Stochastic spatio-temporal processes are prevalent across domains ranging from the modeling of plasma, turbulence in fluids to the wave function of quantum systems. This letter studies a measure-theoretic description of such systems by describing them as evolutionary processes on Hilbert spaces, and in doing so, derives a framework for spatio-temporal manipulation from fundamental thermodynamic principles. This approach yields a variational optimization framework for controlling stochastic fields. The resulting scheme is applicable to a wide class of spatio-temporal processes and can be used for optimizing parameterized control policies. Our simulated experiments explore the application of two forms of this approach on four stochastic spatio-temporal processes, with results that suggest new perspectives and directions for studying stochastic control problems for spatio-temporal systems.
Correlated with the trend of increasing degrees of freedom in robotic systems is a similar trend ... more Correlated with the trend of increasing degrees of freedom in robotic systems is a similar trend of rising interest in Spatio-Temporal systems described by Partial Differential Equations (PDEs) among the robotics and control communities. These systems often exhibit dramatic under-actuation, high dimensionality, bifurcations, and multimodal instabilities. Their control represents many of the current-day challenges facing the robotics and automation communities. Not only are these systems challenging to control, but the design of their actuation is an NP-hard problem on its own. Recent methods either discretize the space before optimization, or apply tools from linear systems theory under restrictive linearity assumptions in order to arrive at a control solution. This manuscript provides a novel sampling-based stochastic optimization framework based entirely in Hilbert spaces suitable for the general class of semi-linear SPDEs which describes many systems in robotics and applied physics. This framework is utilized for simultaneous policy optimization and actuator co-design optimization. The resulting algorithm is based on variational optimization, and performs joint episodic optimization of the feedback control law and the actuation design over episodes. We study first and second order systems, and in doing so, extend several results to the case of second order SPDEs. Finally, we demonstrate the efficacy of the proposed approach with several simulated experiments on a variety of SPDEs in robotics and applied physics including an infinite degree-of-freedom soft robotic manipulator.
We are at the very beginning of time for the human race. It is not unreasonable that we grapple w... more We are at the very beginning of time for the human race. It is not unreasonable that we grapple with problems. But there are tens of thousands of years in the future. Our responsibility is to do what we can, learn what we can, improve the solutions, and pass them on.
High-fidelity state preparation represents a fundamental challenge in the application of quantum ... more High-fidelity state preparation represents a fundamental challenge in the application of quantum technology. While the majority of optimal control approaches use feedback to improve the controller, the controller itself often does not incorporate explicit state dependence. Here, we present a general framework for training deep feedback networks for open quantum systems with continuous weak measurement that allows a variety of system and control structures that are prohibitive by many other techniques and can in effect react to unmodeled effects through nonlinear filtering. Our approach benefits from characteristics of both stochastic sampling and gradient-based optimization methods yet does not require differentiability as in backpropagation approaches. We demonstrate that this method is efficient due to inherent parallelizability, robust to open system interactions, and outperforms landmark state-dependent feedback control results in simulation.
Systems involving Partial Differential Equations (PDEs) have recently become more popular among t... more Systems involving Partial Differential Equations (PDEs) have recently become more popular among the machine learning community. However prior methods usually treat infinite dimensional problems in finite dimensions with Reduced Order Models. This leads to committing to specific approximation schemes and subsequent derivation of control laws. Additionally, prior work does not consider spatio-temporal descriptions of noise that realistically represent the stochastic nature of physical systems. In this paper we suggest a new reinforcement learning framework that is mostly model-free for Stochastic PDEs with additive spacetime noise, based on variational optimization in infinite dimensions. In addition, our algorithm incorporates sparse representations that allow for efficient learning of feedback policies in high dimensions. We demonstrate the efficacy of the proposed approach with several simulated experiments on a variety of SPDEs.
A common approach for developing robotic systems leverages separate simulation and control softwa... more A common approach for developing robotic systems leverages separate simulation and control softwares. Although this approach requires minimal coordination and orchestration between softwares, the separation of simulation and control applications presents the designer with unnecessary challenges during development. This paper describes the Autonomous Robot Control and Simulation (ARCS) software, a unified control and simulation environment developed on a common software stack for real world control and simulated experimentation. This unification is attained through the use of a modular data pipeline that allows for data to be generated in simulation, taken from real hardware, or taken from log files. The unification of real world control and simulation software and the adaptability of this pipeline allows for some unique advantages. These include more meaningful virtual experimentation, a streamlined functionality to perform hardware in the loop tests, and the ability to replay collected log files from both simulation and real world testing. This allows more rapid iterative development cycles of autonomous behaviors. This software stack was used to develop three marine robotics platforms, which will serve as an example to discuss the advantages and disadvantages of this unified approach.
Rapidly Exploring Random Trees (RRTs) have gained significant attention due to provable propertie... more Rapidly Exploring Random Trees (RRTs) have gained significant attention due to provable properties such as completeness and asymptotic optimality. However, offline methods are only useful when the entire problem landscape is known a priori. Furthermore, many real world applications have problem scopes that are orders of magnitude larger than typical mazes and bug traps that require large numbers of samples to match typical sample densities, resulting in high computational effort for reasonably low-cost trajectories. In this paper we propose an online trajectory optimization algorithm for uncertain large environments using RRTs, which we call Locally Adaptive Rapidly Exploring Random Tree (LARRT). This is achieved through two main contributions. We use an adaptive local sampling region and adaptive sampling scheme which depend on states of the dynamic system and observations of obstacles. We also propose a localized approach to planning and re-planning through fixing the root node to the current vehicle state and adding tree update functions. LARRT is designed to leverage local problem scope to reduce computational complexity and obtain a total lower-cost solution compared to a classical RRT of a similar number of nodes. Using this technique we can ensure that popular variants of RRT will remain online even for prohibitively large planning problems by transforming a large trajectory optimization approach to one that resembles receding horizon optimization. Finally, we demonstrate our approach in simulation and discuss various algorithmic trade-offs of the proposed approach.
High fidelity state preparation represents a fundamental challenge in the application of quantum ... more High fidelity state preparation represents a fundamental challenge in the application of quantum technology. While the majority of optimal control approaches use feedback to improve the controller, the controller itself often does not incorporate explicit state dependence. Here, we present a general framework for training deep feedback networks for open quantum systems with quantum nondemolition measurement that allows a variety of system and control structures that are prohibitive by many other techniques and can in effect react to unmodeled effects through nonlinear filtering. We demonstrate that this method is efficient due to inherent parallelizability, robust to open system interactions, and outperforms landmark state feedback control results in simulation.
In this paper, we present a Lyapunov function-based optimization approach for designing state and... more In this paper, we present a Lyapunov function-based optimization approach for designing state and output feedback control laws for systems with polynomial nonlinearities. We use local polynomial expansions of a chosen order to approximate a higher-order nonlinear stochastic dynamical system, reformulate stochastic asymptotic stability conditions in the form of a nonlinear constrained optimization problem, and computationally determine the domain of attraction of the synthesized nonlinear controller on the original system. Finally, we illustrate the effectiveness of the proposed algorithm on two illustrative numerical examples.
Stochastic partial differential equations have been used to model diverse applications ranging fr... more Stochastic partial differential equations have been used to model diverse applications ranging from turbulence and plasma physics to partially observable stochastic controlled systems. This paper aims to bridge the gap between abstract theoretical results and implementable algorithms, by developing a variational optimization framework for controlling infinite-dimensional stochastic models. A measuretheoretic approach is employed for systems evolving on Hilbert spaces, which relies on connections between relative entropy and free energy. The derived scheme is applicable to a wide class of problems, and can be used for optimizing parameterized control policies. Our work creates new perspectives and directions for solving stochastic control problems constrained by partial differential equations.
Systems involving Partial Differential Equations (PDEs) have recently become more popular among t... more Systems involving Partial Differential Equations (PDEs) have recently become more popular among the machine learning community. However prior methods usually treat infinite dimensional problems in finite dimensions with Reduced Order Models. This leads to committing to specific approximation schemes and subsequent derivation of control laws. Additionally, prior work does not consider spatio-temporal descriptions of noise that realistically represent the stochastic nature of physical systems. In this paper we suggest a new reinforcement learning framework that is mostly model-free for Stochastic PDEs with additive spacetime noise, based on variational optimization in infinite dimensions. In addition, our algorithm incorporates sparse representations that allow for efficient learning of feedback policies in high dimensions. We demonstrate the efficacy of the proposed approach with several simulated experiments on a variety of SPDEs.
High fidelity state preparation represents a fundamental challenge in the application of quantum ... more High fidelity state preparation represents a fundamental challenge in the application of quantum technology. While the majority of optimal control approaches use feedback to improve the controller, the controller itself often does not incorporate explicit state dependence. Here, we present a general framework for training deep feedback networks for open quantum systems with quantum nondemolition measurement that allows a variety of system and control structures that are prohibitive by many other techniques and can in effect react to unmodeled effects through nonlinear filtering. We demonstrate that this method is efficient due to inherent parallelizability, robust to open system interactions, and outperforms landmark state feedback control results in simulation.
Volume 3: Modeling and Validation; Multi-Agent and Networked Systems; Path Planning and Motion Control; Tracking Control Systems; Unmanned Aerial Vehicles (UAVs) and Application; Unmanned Ground and Aerial Vehicles; Vibration in Mechanical Systems; Vibrations and Control of Systems; Vibrations: M..., 2018
Rapidly Exploring Random Trees (RRTs) have gained significant attention due to provable propertie... more Rapidly Exploring Random Trees (RRTs) have gained significant attention due to provable properties such as completeness and asymptotic optimality. However, offline methods are only useful when the entire problem landscape is known a priori. Furthermore, many real world applications have problem scopes that are orders of magnitude larger than typical mazes and bug traps that require large numbers of samples to match typical sample densities, resulting in high computational effort for reasonably low-cost trajectories. In this paper we propose an online trajectory optimization algorithm for uncertain large environments using RRTs, which we call Locally Adaptive Rapidly Exploring Random Tree (LARRT). This is achieved through two main contributions. We use an adaptive local sampling region and adaptive sampling scheme which depend on states of the dynamic system and observations of obstacles. We also propose a localized approach to planning and re-planning through fixing the root node to...
Systems involving Partial Differential Equations (PDEs) have recently become more popular among t... more Systems involving Partial Differential Equations (PDEs) have recently become more popular among the machine learning community. However prior methods usually treat infinite dimensional problems in finite dimensions with Reduced Order Models. This leads to committing to specific approximation schemes and subsequent derivation of control laws. Additionally, prior work does not consider spatio-temporal descriptions of noise that realistically represent the stochastic nature of physical systems. In this paper we suggest a new reinforcement learning framework that is mostly model-free for Stochastic PDEs with additive spacetime noise, based on variational optimization in infinite dimensions. In addition, our algorithm incorporates sparse representations that allow for efficient learning of feedback policies in high dimensions. We demonstrate the efficacy of the proposed approach with several simulated experiments on a variety of SPDEs.
There is a rising interest in Spatio-temporal systems described by Partial Differential Equations... more There is a rising interest in Spatio-temporal systems described by Partial Differential Equations (PDEs) among the control community. Not only are these systems challenging to control, but the sizing and placement of their actuation is an NP-hard problem on its own. Recent methods either discretize the space before optimziation, or apply tools from linear systems theory under restrictive linearity assumptions. In this work we consider control and actuator placement as a coupled optimization problem, and derive an optimization algorithm on Hilbert spaces for nonlinear PDEs with an additive spatio-temporal description of white noise. We study first and second order systems and in doing so, extend several results to the case of second order PDEs. The described approach is based on variational optimization, and performs joint RL-type optimization of the feedback control law and the actuator design over episodes. We demonstrate the efficacy of the proposed approach with several simulated experiments on a variety of SPDEs.
2018 Annual American Control Conference (ACC), 2018
In this paper, we present a Lyapunov function-based optimization approach for designing state and... more In this paper, we present a Lyapunov function-based optimization approach for designing state and output feedback control laws for systems with polynomial nonlinearities. We use local polynomial expansions of a chosen order to approximate a higher-order nonlinear stochastic dynamical system, reformulate stochastic asymptotic stability conditions in the form of a nonlinear constrained optimization problem, and computationally determine the domain of attraction of the synthesized nonlinear controller on the original system. Finally, we illustrate the effectiveness of the proposed algorithm on two illustrative numerical examples.
Stochastic partial differential equations have been used to model diverse applications ranging fr... more Stochastic partial differential equations have been used to model diverse applications ranging from turbulence and plasma physics to partially observable stochastic controlled systems. This paper aims to bridge the gap between abstract theoretical results and implementable algorithms, by developing a variational optimization framework for controlling infinite-dimensional stochastic models. A measure-theoretic approach is employed for systems evolving on Hilbert spaces, which relies on connections between relative entropy and free energy. The derived scheme is applicable to a wide class of problems, and can be used for optimizing parameterized control policies. Our work creates new perspectives and directions for solving stochastic control problems constrained by partial differential equations.
The design and evaluation of autonomous system behavior is commonly based on an archetypal repres... more The design and evaluation of autonomous system behavior is commonly based on an archetypal representation of a mission within a modeling and simulation environment. In real-world operation however, autonomous systems may encounter a multitude of examples of a given mission, each a variation on the theme. From a design standpoint, it is desirable to simulate the system on many instances of a mission to evaluate robustness, and, at times, the need arises to test an autonomous behavior on a completely different mission for which the behavior is hypothesized to be applicable. The amount of time and effort required to construct new mission models can be a challenge preventing such robust investigations. In response, we describe a flexible problem definition for what we term “event-based missions” to enable evaluation of single- and multi-vehicle autonomous behaviors across a set of missions within the category. Data structures arising from the problem definition enable rapid construction of new models of event-based missions, and the flexibility of the approach is demonstrated on three missions representing extreme cases within the category.
We consider the optimal control problem of a general nonlinear spatio-temporal system described b... more We consider the optimal control problem of a general nonlinear spatio-temporal system described by Partial Differential Equations (PDEs). Theory and algorithms for control of spatio-temporal systems are of rising interest among the automatic control community and exhibit numerous challenging characteristic from a control standpoint. Recent methods focus on finite-dimensional optimization techniques of a discretized finite dimensional ODE approximation of the infinite dimensional PDE system. In this paper, we derive a differential dynamic programming (DDP) framework for distributed and boundary control of spatio-temporal systems in infinite dimensions that is shown to generalize both the spatio-temporal LQR solution, and modern finite dimensional DDP frameworks. We analyze the convergence behavior and provide a proof of global convergence for the resulting system of continuous-time forward-backward equations. We explore and develop numerical approaches to handle sensitivities that ar...
Stochastic spatio-temporal processes are prevalent across domains ranging from the modeling of pl... more Stochastic spatio-temporal processes are prevalent across domains ranging from the modeling of plasma, turbulence in fluids to the wave function of quantum systems. This letter studies a measure-theoretic description of such systems by describing them as evolutionary processes on Hilbert spaces, and in doing so, derives a framework for spatio-temporal manipulation from fundamental thermodynamic principles. This approach yields a variational optimization framework for controlling stochastic fields. The resulting scheme is applicable to a wide class of spatio-temporal processes and can be used for optimizing parameterized control policies. Our simulated experiments explore the application of two forms of this approach on four stochastic spatio-temporal processes, with results that suggest new perspectives and directions for studying stochastic control problems for spatio-temporal systems.
Correlated with the trend of increasing degrees of freedom in robotic systems is a similar trend ... more Correlated with the trend of increasing degrees of freedom in robotic systems is a similar trend of rising interest in Spatio-Temporal systems described by Partial Differential Equations (PDEs) among the robotics and control communities. These systems often exhibit dramatic under-actuation, high dimensionality, bifurcations, and multimodal instabilities. Their control represents many of the current-day challenges facing the robotics and automation communities. Not only are these systems challenging to control, but the design of their actuation is an NP-hard problem on its own. Recent methods either discretize the space before optimization, or apply tools from linear systems theory under restrictive linearity assumptions in order to arrive at a control solution. This manuscript provides a novel sampling-based stochastic optimization framework based entirely in Hilbert spaces suitable for the general class of semi-linear SPDEs which describes many systems in robotics and applied physics. This framework is utilized for simultaneous policy optimization and actuator co-design optimization. The resulting algorithm is based on variational optimization, and performs joint episodic optimization of the feedback control law and the actuation design over episodes. We study first and second order systems, and in doing so, extend several results to the case of second order SPDEs. Finally, we demonstrate the efficacy of the proposed approach with several simulated experiments on a variety of SPDEs in robotics and applied physics including an infinite degree-of-freedom soft robotic manipulator.
Uploads
Papers by Ethan Evans