research-article

Hamilton-Jacobi reachability: A brief overview and recent advances

Authors:

Sylvia Herbert,

Claire J. TomlinAuthors Info & Claims

2017 IEEE 56th Annual Conference on Decision and Control (CDC)

Pages 2242 - 2253

https://doi.org/10.1109/CDC.2017.8263977

Published: 12 December 2017 Publication History

Abstract

Hamilton-Jacobi (HJ) reachability analysis is an important formal verification method for guaranteeing performance and safety properties of dynamical systems; it has been applied to many small-scale systems in the past decade. Its advantages include compatibility with general nonlinear system dynamics, formal treatment of bounded disturbances, and the availability of well-developed numerical tools. The main challenge is addressing its exponential computational complexity with respect to the number of state variables. In this tutorial, we present an overview of basic HJ reachability theory and provide instructions for using the most recent numerical tools, including an efficient GPU-parallelized implementation of a Level Set Toolbox for computing reachable sets. In addition, we review some of the current work in high-dimensional HJ reachability to show how the dimensionality challenge can be alleviated via various general theoretical and application-specific insights.

References

[1]

J. Sethian, “A fast marching level set method for monotonically advancing fronts,” National Academy of Sciences, vol. 93, no. 4, pp. 1591–1595, 1996.

[2]

S. Osher and R. Fedkiw, Level Set Methods and Dynamic Implicit Surfaces. Springer-Verlag, 2006.

[3]

I. Mitchell, “Application of level set methods to control and reachability problems in continuous and hybrid systems,” Ph. D. dissertation, Stanford University, 2002.

[4]

I. Mitchell, “A toolbox of level set methods,” Department of Computer Science, University of British Columbia, Vancouver, BC, Canada, http://www.cs.ubc.ca/-mitchell/ToolboxLS/toolboxLS.pdf, Tech. Rep. TR-2004-09, 2004.

[5]

A. Bayen, I. Mitchell, M. Osihi, and C. Tomlin, “Aircraft autolander safety analysis through optimal control-based reach set computation,” AIAA J. Guidance, Control, and Dynamics, vol. 30, no. 1, pp. 68–77, 2007.

[6]

J. Ding, J. Sprinkle, S. Sastry, and C. Tomlin, “Reachability calculations for automated aerial refueling,” in Proc. IEEE Conf. Decision and Control, 2008.

[7]

P. Bouffard, “On-board model predictive control of a quadrotor helicopter: Design, implementation, and experiments,” Master's thesis, University of California, Berkeley, 2012.

[8]

A. Aswani, H. Gonzalez, S. Sastry, and C. Tomlin, “Provably safe and robust learning-based model predictive control,” Automatica, vol. 49, no. 5, pp. 1216–1226, 2013.

Digital Library

[9]

H. Huang, J. Ding, W. Zhang, and C. Tomlin, “A differential game approach to planning in adversarial scenarios: A case study on capture-the-flag,” in Proc. IEEE Int. Conf. Robotics and Automation, 2011.

[10]

M. Chen, S. Bansal, J. Fisac, and C. Tomlin, “Robust Sequential Path Planning Under Disturbances and Adversarial Intruder,” IEEE Trans. Control Syst. Technol., to appear.

[11]

M. Chen, Q. Hu, C. Mackin, J. Fisac, and C. Tomlin, “Safe platooning of unmanned aerial vehicles via reachability,” in Proc. IEEE Conf. Decision and Control, 2015.

[12]

S. Herbert, M. Chen, S. Han, S. Bansal, J. Fisac, and C. Tomlin, “FaSTrack: a modular framework for fast and guaranteed safe motion planning,” Proc. IEEE Conf. Decision and Control, 2017.

[13]

M. Chen, S. Herbert, M. Vashishtha, S. Bansal, and C. Tomlin, “Decomposition of reachable sets and tubes for a class of nonlinear systems,” arXiv preprint arXiv:, 2016.

[14]

M. Chen, S. Herbert, and C. Tomlin, “Fast reachable set approximations via state decoupling disturbances,” in Proc. IEEE Conf. Decision and Control, 2016.

[15]

J. Darbon and S. Osher, “Algorithms for overcoming the curse of dimensionality for certain Hamilton-Jacobi equations arising in control theory and elsewhere,” Research in the Math. Sciences, vol. 3, no. 1, p. 19, 2016.

[16]

Y. Chow, J. Darbon, S. Osher, and W. Yin, “Algorithm for overcoming the curse of dimensionality for time-dependent non-convex Hamilton-Jacobi equations arising from optimal control and differential games problems,” J. Scientific Computing, pp. 1–27, 2016.

[17]

C. Baier, J. Katoen, and K. Larsen, Principles of Model Checking. Cambridge, MA: MIT Press, 2008.

Digital Library

[18]

C. Belta, B. Yordanov, and E. Gol, Formal Methods for Discrete-Time Dynamical Systems, ser. Studies in Systems, Decision and Control. Springer International Publishing, 2017, vol. 89.

[19]

M. Reynolds, “Continuous temporal models,” in Proc. Australian loint Conf. Artificial Intelligence, 2001.

[20]

G. Fainekos, A. Girard, H. Kress-Gazit, and G. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica, vol. 45, no. 2, pp. 343–352, 2009.

Digital Library

[21]

S. Jha, V. Raman, D. Sadigh, and S. Seshia, “Safe autonomy under perception uncertainty using chance-constrained temporal logic,” J. Automated Reasoning, 2017.

[22]

S. Coogan, M. Arcak, and C. Belta, “Formal methods for control of traffic flow: Automated control synthesis from finite-state transition models,” IEEE Control Systems, vol. 37, no. 2, pp. 109–128, 2017.

[23]

M. Bolton, E. Bass, and R. Siminiceanu, “Using formal verification to evaluate human-automation interaction: A review,” IEEE Trans. Syst. Man Cybern. A., Syst. Humans, vol. 43, no. 3, pp. 488–503, 2013.

[24]

G. Frehse, C. Le Guernic, A. Donzé, S. Cotton, R. Ray, O. Lebeltel, R. Ripado, A. Girard, T. Dang, and O. Maler, “SpaceEx: Scalable verification of hybrid systems,” in Proc. Int. Conf. Computer Aided Verification, 2011.

[25]

X. Chen, E. Ábrahám, and S. Sankaranarayanan, “Flow*: An analyzer for non-linear hybrid systems,” in Proc. Int. Conf. Computer Aided Verification, 2013.

[26]

M. Althoff, “An introduction to CORA 2015,” in Proc. ARCH@ CPSWeek, 2015.

[27]

P. Duggirala, S. Mitra, M. Viswanathan, and M. Potok, “C2E2: A verification tool for stateflow models,” in Proc. Int. Conf. Tools and Algorithms for the Construction and Analysis of Systems, 2015.

[28]

C. Fan, B. Qi, S. Mitra, M. Viswanathan, and P. Duggirala, “Automatic reachability analysis for nonlinear hybrid models with C2E2,” in Proc. Int. Conf. Computer Aided Verification, 2016.

[29]

S. Kong, S. Gao, W. Chen, and E. Clarke, “dReach: b-Reachability analysis for hybrid systems,” in Proc. Int. Conf. Tools and Algorithms for the Construction and Analysis of Systems, 2015. [Online]. Available: http://link.springer.comf10.1007/978-3-662-46681-0_15

[30]

P. Duggirala, C. Fan, M. Potok, B. Qi, S. Mitra, M. Viswanathan, S. Bak, S. Bogomolov, T. Johnson, L. Nguyen et al., “Tutorial: Software tools for hybrid systems verification, transformation, and synthesis: C2E2, HyST, and TuLiP,” in Proc. Conf. Control Applications, 2016.

[31]

P. Parrilo, “Structured semidefinite programs and semialgebraic geometry methods in robustness and optimization,” Ph. D. Dissertation, California Institute of Technology, 2000. [Online]. Available: http://resolver.caltech.edu/CaltechETD:etd-05062004-055516

[32]

R. Tedrake, I. Manchester, M. Tobenkin, and J. Roberts, “LQR-trees: Feedback motion planning via sums-of-squares verification,” Int. J. Robotics Research, vol. 29, no. 8, pp. 1038–1052, 2010.

Digital Library

[33]

A. Barry, A. Majumdar, and R. Tedrake, “Safety verification of reactive controllers for UAV flight in cluttered environments using barrier certificates,” in Proc. IEEE Int. Conf. Robotics and Automation, 2012.

[34]

A. Majumdar, A. Ahmadi, and R. Tedrake, “Control design along trajectories with sums of squares programming,” in Proc. IEEE Int. Conf. Robotics and Automation, 2013.

[35]

A. Majumdar and R. Tedrake, “Funnel libraries for real-time robust feedback motion planning,” Int. J. Robotics Research, vol. 36, no. 8, pp. 947–982, 2017.

Digital Library

[36]

E. Barron, “Differential games with maximum cost,” Nonlinear analysis: Theory, methods & applications, vol. 14, no. 11, pp. 971–989, 1990.

Digital Library

[37]

I. Mitchell, A. Bayen, and C. Tomlin, “A time-dependent Hamilton-Jacobi formulation of reachable sets for continuous dynamic games,” IEEE Trans. Autom. Control, vol. 50, no. 7, pp. 947–957, 2005.

[38]

O. Bokanowski and H. Zidani, “Minimal time problems with moving targets and obstacles,” IFAC Proceedings Volumes, vol. 44, no. 1, pp. 2589–2593, 2011.

[39]

J. Fisac, M. Chen, C. Tomlin, and S. Sastry, “Reach-avoid problems with time-varying dynamics, targets and constraints,” in Proc. ACM Int. Conf. Hybrid Systems: Computation and Control, 2015.

[40]

A. Kurzhanski and P. Varaiya, “Ellipsoidal techniques for reachability analysis: internal approximation,” Systems & control letters, vol. 41, no. 3, pp. 201–211, 2000.

[41]

A. Kurzhanski and P. Varaiya, “On ellipsoidal techniques for reachability analysis. Part II: Internal approximations box-valued constraints,” Optimization Methods and Software, vol. 17, no. 2, pp. 207–237, 2002.

[42]

J. Maidens, S. Kaynama, I. Mitchell, M. Oishi, and G. Dumont, “Lagrangian methods for approximating the viability kernel in high-dimensional systems,” Automatica, vol. 49, no. 7, pp. 2017–2029, 2013.

[43]

A. Majumdar, R. Vasudevan, M. Tobenkin, and R. Tedrake, Convex optimization of nonlinear feedback controllers via occupation measures, 2014, vol. 33, no. 9, pp. 1209–1230.

[44]

T. Dreossi, T. Dang, and C. Piazza, “Parallelotope bundles for polynomial reachability,” in Proc. ACM Int. Conf. Hybrid Systems: Computation and Control, 2016.

[45]

D. Henrion and M. Korda, “Convex computation of the region of attraction of polynomial control systems,” IEEE Trans. Autom. Control, vol. 59, no. 2, pp. 297–312, 2014.

[46]

E. Coddington and N. Levinson, Theory of ordinary differential equations. Tata McGraw-Hill Education, 1955.

[47]

L. Evans and P. Souganidis, “Differential games and representation formulas for solutions of Hamilton-Jacobi-Isaacs equations,” WIS-CONSIN UNIV-MADISON MATHEMATICS RESEARCH CEN-TER, Tech. Rep., 1983.

[48]

L. Evans, Partial differential equations. Amer. Math. Soc., 2010.

[49]

I. Mitchell, “Comparing forward and backward reachability as tools for safety analysis,” in Proc. Int. Workshop on Hybrid Systems: Computation and Control, 2007.

[50]

J. Lygeros, “On reachability and minimum cost optimal control,” Automatica, vol. 40, no. 6, pp. 917–927, 2004.

Digital Library

[51]

K. Margellos and J. Lygeros, “HamiltonJacobi formulation for reachavoid differential games,” IEEE Trans. Autom. Control, vol. 56, no. 8, pp. 1849–1861, 2011.

[52]

M. Chen, S. Herbert, and C. Tomlin, “Exact and efficient Hamilton-Jacobi-based guaranteed safety analysis via system decomposition,” Proc. IEEE Int. Conf. Robotics and Automation, 2017.

[53]

I. Mitchell and C. Tomlin, “Overapproximating reachable sets by Hamilton-Jacobi projections,” J. Scientific Computing, vol. 19, no. 1-3, pp. 323–346, 2003.

Digital Library

[54]

S. Kaynama and M. Oishi, “Complexity reduction through a Schur-based decomposition for reachability analysis of linear time-invariant systems,” Int. J. Control, vol. 84, no. 1, pp. 165–179, 2011.

[55]

S. Kaynama and M. Oishi, “A modified Riccati transformation for decentralized computation of the viability kernel under LTI dynamics,” IEEE Trans. Autom. Control, vol. 58, no. 11, pp. 2878–2892, 2013.

[56]

J. Fisac, A. Akametalu, M. Zeilinger, S. Kaynama, J. Gillula, and C. Tomlin, “A general safety framework for learning-based control in uncertain robotic systems,” arXiv preprint arXiv:, 2017.

[57]

A. Akametalu, J. Fisac, J. Gillula, S. Kaynama, M. Zeilinger, and C. Tomlin, “Reachability-based safe learning with Gaussian processes,” in Proc. IEEE Conf. Decision and Control, 2014.

[58]

K. N. Niarchos and J. Lygeros, “A neural approximation to continuous time reachability computations,” in Proc. IEEE Conf. Decision and Control, 2006.

[59]

B. Djeridane and J. Lygeros, “Neural approximation of PDE solutions: An application to reachability computations,” in Proc. IEEE Conf. Decision and Control, 2006.

[60]

V. R. Royo and C. Tomlin, “Recursive regression with neural networks: Approximating the HJI PDE solution,” 2016. [Online]. Available: http://arxiv.org/abs/1611.02739

[61]

F. Jiang, G. Chou, M. Chen, and C. Tomlin, “Using neural networks to compute approximate and guaranteed feasible Hamilton-Jacobi-Bellman PDE solutions,” 2016. [Online]. Available: http://arxiv.org/abs/1611.03158

[62]

T. Prevot, J. Rios, P. Kopardekar, J. Robinson III, M. Johnson, and J. Jung, “UAS Traffic Management (UTM) concept of operations to safely enable low altitude flight operations,” in Proc. AIAA Aviation Technol., Integration, and Operations Conf., 2016.

[63]

M. Chen, Q. Hu, J. Fisac, K. Akametalu, C. Mackin, and C. Tomlin, “Reachability-based safety and goal satisfaction of unmanned aerial platoons on air highways,” AIAA J. Guidance, Control, and Dynamics, pp. 1–14, 2017.

[64]

M. Chen, S. Bansal, K. Tanabe, and C. Tomlin, “Provably safe and robust drone routing via sequential path planning: A case study in San Francisco and the Bay Area,” 2017. [Online]. Available: http://arxiv.org/abs/1705.04585

[65]

S. Bansal, M. Chen, J. Fisac, and C. Tomlin, “Safe sequential path planning of multi-vehicle systems under presence of disturbances and imperfect information,” in Proc. Amer. Control Conf., 2017.

[66]

M. Chen, J. Shih, and C. Tomlin, “Multi-vehicle collision avoidance via Hamilton-Jacobi reachability and mixed integer programming,” in Proc. IEEE Conf. Decision and Control, 2016.

[67]

M. Chen, Z. Zhou, and C. Tomlin, “Multiplayer reach-avoid games via pairwise outcomes,” IEEE Trans. Autom. Control, vol. 62, no. 3, pp. 1451–1457, 2017.

Cited By

Goktas DPrakash AGreenwald AOh ANaumann TGloberson ASaenko KHardt MLevine S(2023)Convex-concave 0-sum Markov stackelberg gamesProceedings of the 37th International Conference on Neural Information Processing Systems10.5555/3666122.3669039(66818-66832)Online publication date: 10-Dec-2023
https://dl.acm.org/doi/10.5555/3666122.3669039
Rahman MLiu TAlqahtani SElkind E(2023)Adversarial behavior exclusion for safe reinforcement learningProceedings of the Thirty-Second International Joint Conference on Artificial Intelligence10.24963/ijcai.2023/54(483-491)Online publication date: 19-Aug-2023
https://dl.acm.org/doi/10.24963/ijcai.2023/54
Rahman MAlqahtani SPintor MChen XTramèr F(2023)Task-Agnostic Safety for Reinforcement LearningProceedings of the 16th ACM Workshop on Artificial Intelligence and Security10.1145/3605764.3623913(139-148)Online publication date: 30-Nov-2023
https://dl.acm.org/doi/10.1145/3605764.3623913
Show More Cited By

Index Terms

Hamilton-Jacobi reachability: A brief overview and recent advances

Index terms have been assigned to the content through auto-classification.

Recommendations

Reachability and reverse reachability analysis of CFSMs

Reachability analysis and the recently proposed reverse reachability analysis are two important verification techniques for communicating finite state machines (CFSM). The issue of the relative efficiency of reachability analysis and reverse ...
Recent Advances on Reachability Problems for Valence Systems (Invited Talk)
Reachability Problems
Abstract
Valence systems are an abstract model of computation that consists of a finite-state control and some storage mechanism. In contrast to traditional models, the storage mechanism is not fixed, but given as a parameter. This allows us to precisely ...
Functional Broadside Tests with Different Levels of Reachability
VLSID '07: Proceedings of the 20th International Conference on VLSI Design held jointly with 6th International Conference: Embedded Systems

Functional broadside tests for transition faults are broadside tests that detect transition faults using only states that the circuit can visit during functional operation, or reachable states. Reachability was used before as a binary property, i.e., a ...

Comments

Information & Contributors

Information

Published In

cover image Guide Proceedings

2017 IEEE 56th Annual Conference on Decision and Control (CDC)

Dec 2017

6709 pages

Copyright © 2017.

Publisher

IEEE Press

Publication History

Published: 12 December 2017

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 23 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Goktas DPrakash AGreenwald AOh ANaumann TGloberson ASaenko KHardt MLevine S(2023)Convex-concave 0-sum Markov stackelberg gamesProceedings of the 37th International Conference on Neural Information Processing Systems10.5555/3666122.3669039(66818-66832)Online publication date: 10-Dec-2023
https://dl.acm.org/doi/10.5555/3666122.3669039
Rahman MLiu TAlqahtani SElkind E(2023)Adversarial behavior exclusion for safe reinforcement learningProceedings of the Thirty-Second International Joint Conference on Artificial Intelligence10.24963/ijcai.2023/54(483-491)Online publication date: 19-Aug-2023
https://dl.acm.org/doi/10.24963/ijcai.2023/54
Rahman MAlqahtani SPintor MChen XTramèr F(2023)Task-Agnostic Safety for Reinforcement LearningProceedings of the 16th ACM Workshop on Artificial Intelligence and Security10.1145/3605764.3623913(139-148)Online publication date: 30-Nov-2023
https://dl.acm.org/doi/10.1145/3605764.3623913
Hildebrandt Cvon Stein MElbaum SJust RFraser G(2023)PhysCov: Physical Test Coverage for Autonomous VehiclesProceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3597926.3598069(449-461)Online publication date: 12-Jul-2023
https://dl.acm.org/doi/10.1145/3597926.3598069
Raz DYang LUmberger BOzay N(2023)Poster Abstract: Reachability and Controlled Invariance for Human Stability during Sit-to-StandProceedings of the 26th ACM International Conference on Hybrid Systems: Computation and Control10.1145/3575870.3589548(1-2)Online publication date: 9-May-2023
https://dl.acm.org/doi/10.1145/3575870.3589548
Hsu KRen ANguyen DMajumdar AFisac J(2023)Sim-to-Lab-to-RealArtificial Intelligence10.1016/j.artint.2022.103811314:COnline publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1016/j.artint.2022.103811
Liao WLiang TWang CYang D(2023)A dimensionality reduction method for computing reachable tubes based on piecewise pseudo-time dependent Hamilton–Jacobi equationApplied Mathematics and Computation10.1016/j.amc.2022.127696441:COnline publication date: 15-Mar-2023
https://dl.acm.org/doi/10.1016/j.amc.2022.127696
Parwana HPanagou D(2022)Recursive Feasibility Guided Optimal Parameter Adaptation of Differential Convex Optimization Policies for Safety-Critical Systems2022 International Conference on Robotics and Automation (ICRA)10.1109/ICRA46639.2022.9812398(6807-6813)Online publication date: 23-May-2022
https://dl.acm.org/doi/10.1109/ICRA46639.2022.9812398
Xue BWang QZhan NFränzle MOzay NPrabhakar P(2019)Robust invariant sets generation for state-constrained perturbed polynomial systemsProceedings of the 22nd ACM International Conference on Hybrid Systems: Computation and Control10.1145/3302504.3311810(128-137)Online publication date: 16-Apr-2019
https://dl.acm.org/doi/10.1145/3302504.3311810

View Options

View options

Media

Figures

Other

Tables

View Table of Contents