research-article

Open access

Strategy synthesis for partially-known switched stochastic systems

Authors:

Morteza LahijanianAuthors Info & Claims

HSCC '21: Proceedings of the 24th International Conference on Hybrid Systems: Computation and Control

Article No.: 6, Pages 1 - 11

https://doi.org/10.1145/3447928.3456649

Published: 19 May 2021 Publication History

Abstract

We present a data-driven framework for strategy synthesis for partially-known switched stochastic systems. The properties of the system are specified using linear temporal logic (LTL) over finite traces (LTL_f), which is as expressive as LTL and enables interpretations over finite behaviors. The framework first learns the unknown dynamics via Gaussian process regression. Then, it builds a formal abstraction of the switched system in terms of an uncertain Markov model, namely an Interval Markov Decision Process (IMDP), by accounting for both the stochastic behavior of the system and the uncertainty in the learning step. Then, we synthesize a strategy on the resulting IMDP that maximizes the satisfaction probability of the LTL_f specification and is robust against all the uncertainties in the abstraction. This strategy is then refined into a switching strategy for the original stochastic system. We show that this strategy is near-optimal and provide a bound on its distance (error) to the optimal strategy. We experimentally validate our framework on various case studies, including both linear and non-linear switched stochastic systems.

References

[1]

Alessandro Abate, Frank Redig, and Ilya Tkachev. 2014. On the effect of perturbation of conditional probabilities in total variation. Statistics & Probability Letters 88 (2014), 1--8.

[2]

Mohamadreza Ahmadi, Arie Israel, and Ufuk Topcu. 2017. Safety assessemt based on physically-viable data-driven models. In 2017 IEEE 56th Annual Conference on Decision and Control (CDC). IEEE, 6409--6414.

Digital Library

[3]

Anayo K Akametalu, Shahab Kaynama, Jaime F Fisac, Melanie N Zeilinger, Jeremy H Gillula, and Claire J Tomlin. 2014. Reachability-based safe learning with Gaussian processes. In IEEE 53rd Annual Conference on Decision and Control (CDC), 2014: 15--17 Dec. 2014, Los Angeles, California, USA. IEEE, 1424--1431.

[4]

Christel Baier, Joost-Pieter Katoen, et al. 2008. Principles of model checking. MIT press Cambridge.

[5]

Felix Berkenkamp, Matteo Turchetta, Angela Schoellig, and Andreas Krause. 2017. Safe model-based reinforcement learning with stability guarantees. In Advances in neural information processing systems. 908--918.

[6]

Arno Blaas, Andrea Patane, Luca Laurenti, Luca Cardelli, Marta Kwiatkowska, and Stephen Roberts. 2020. Adversarial robustness guarantees for classification with gaussian processes. In International Conference on Artificial Intelligence and Statistics. 3372--3382.

[7]

Luca Cardelli, Marta Kwiatkowska, Luca Laurenti, and Andrea Patane. 2019. Robustness guarantees for bayesian inference with gaussian processes. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 7759--7768.

Digital Library

[8]

Nathalie Cauchi, Luca Laurenti, Morteza Lahijanian, Alessandro Abate, Marta Kwiatkowska, and Luca Cardelli. 2019. Efficiency through Uncertainty: Scalable Formal Synthesis for Stochastic Hybrid Systems. In Proceedings of the 2019 22nd ACM International Conference on Hybrid Systems: Computation and Control. ACM, Montreal, QC, Canada.

Digital Library

[9]

Sayak Ray Chowdhury and Aditya Gopalan. 2017. On kernelized multi-armed bandits. In Proceedings of the 34th International Conference on Machine Learning-Volume 70. JMLR. org, 844--853.

Digital Library

[10]

Giuseppe De Giacomo and Moshe Y Vardi. 2013. Linear temporal logic and linear dynamic logic on finite traces. In IJCAI'13 Proceedings of the Twenty-Third international joint conference on Artificial Intelligence. Association for Computing Machinery, 854--860.

[11]

Laurent Doyen, Goran Frehse, George J Pappas, and André Platzer. 2018. Verification of hybrid systems. In Handbook of Model Checking. Springer, 1047--1110.

[12]

Souradeep Dutta, Susmit Jha, Sriram Sankaranarayanan, and Ashish Tiwari. 2018. Learning and verification of feedback control systems using feedforward neural networks. IFAC-PapersOnLine 51, 16 (2018), 151--156.

[13]

Pascal Germain, Francis Bach, Alexandre Lacoste, and Simon Lacoste-Julien. 2016. PAC-Bayesian theory meets Bayesian inference. In Advances in Neural Information Processing Systems. 1884--1892.

[14]

Robert Givan, Sonia Leach, and Thomas Dean. 2000. Bounded-parameter Markov decision processes. Artificial Intelligence 122, 1--2 (2000), 71--109.

Digital Library

[15]

Sofie Haesaert, Nathalie Cauchi, and Alessandro Abate. 2017. Certified policy synthesis for general Markov decision processes: An application in building automation systems. Performance Evaluation 117 (2017), 75--103.

[16]

Sofie Haesaert, Paul MJ Van den Hof, and Alessandro Abate. 2017. Data-driven and model-based verification via Bayesian identification and reachability analysis. Automatica 79 (2017), 115--126.

[17]

Ernst Moritz Hahn, Vahid Hashemi, Holger Hermanns, Morteza Lahijanian, and Andrea Turrini. 2019. Interval Markov decision processes with multiple objectives: From Robust strategies to Pareto curves. ACM Transactions on Modeling and Computer Simulation 29, 4 (2019), 1--31.

Digital Library

[18]

John Jackson, Luca Laurenti, Eric Frew, and Morteza Lahijanian. 2020. Safety verification of unknown dynamical systems via gaussian process regression. IEEE CDC (2020).

Digital Library

[19]

A Agung Julius, Ádám Halász, M Selman Sakar, Harvey Rubin, Vijay Kumar, and George J Pappas. 2008. Stochastic modeling and control of biological systems: the lactose regulation system of Escherichia coli. IEEE Trans. Automat. Control 53, Special Issue (2008), 51--65.

[20]

Joris Kenanian, Ayca Balkan, Raphael M Jungers, and Paulo Tabuada. 2019. Data driven stability analysis of black-box switched linear systems. Automatica 109 (2019), 108533.

[21]

Harold Kushner and Paul G Dupuis. 2013. Numerical methods for stochastic control problems in continuous time. Vol. 24. Springer Science & Business Media.

[22]

Morteza Lahijanian, Sean B Andersson, and Calin Belta. 2015. Formal verification and synthesis for discrete-time stochastic systems. IEEE Trans. Automat. Control 60, 8 (2015), 2031--2045.

[23]

Luca Laurenti, Morteza Lahijanian, Alessandro Abate, Luca Cardelli, and Marta Kwiatkowska. 2020. Formal and efficient synthesis for continuous-time linear stochastic hybrid processes. IEEE Trans. Automat. Control 66, 1 (2020), 17--32.

[24]

Armin Lederer, Jonas Umlauft, and Sandra Hirche. 2019. Uniform Error Bounds for Gaussian Process Regression with Application to Safe Control. In Advances in Neural Information Processing Systems. 657--667.

[25]

Ryan Luna, Morteza Lahijanian, Mark Moll, and Lydia E. Kavraki. 2014. Asymptotically Optimal Stochastic Motion Planning with Temporal Goals. In Int'l Workshop on the Algorithmic Foundations of Robotics (WAFR). Istanbul, Turkey, 335--352.

[26]

Pascal Massart. 2007. Concentration inequalities and model selection. Vol. 6. Springer.

[27]

Vern I Paulsen and Mrinal Raghupathi. 2016. An introduction to the theory of reproducing kernel Hilbert spaces. Vol. 152. Cambridge University Press.

[28]

Kyriakos Polymenakos, Luca Laurenti, Andrea Patane, Jan-Peter Calliess, Luca Cardelli, Marta Kwiatkowska, Alessandro Abate, and Stephen Roberts. 2019. Safety Guarantees for Planning Based on Iterative Gaussian Processes. arXiv preprint arXiv:1912.00071 (2019).

[29]

Carl Edward Rasmussen. 2003. Gaussian processes in machine learning. In Summer School on Machine Learning. Springer, 63--71.

[30]

Sadegh Esmaeil Zadeh Soudjani, Caspar Gevaerts, and Alessandro Abate. 2015. FAUST2: Formal Abstractions of Uncountable-STate STochastic Processes. In International Conference on Tools and Algorithms for the Construction and Analysis of Systems. Springer, 272--286.

Digital Library

[31]

Niranjan Srinivas, Andreas Krause, Sham M Kakade, and Matthias W Seeger. 2012. Information-theoretic regret bounds for gaussian process optimization in the bandit setting. IEEE Transactions on Information Theory 58, 5 (2012), 3250--3265.

Digital Library

[32]

Ingo Steinwart. 2001. On the influence of the kernel on the consistency of support vector machines. Journal of machine learning research 2, Nov (2001), 67--93.

[33]

Yanan Sui, Alkis Gotovos, Joel W Burdick, and Andreas Krause. 2015. Safe exploration for optimization with Gaussian processes. Proceedings of Machine Learning Research 37 (2015), 997--1005.

[34]

Andrew M. Wells, Morteza Lahijanian, Lydia E. Kavraki, and Moshe Y. Vardi. 2020. LTLf Synthesis on Probabilistic Systems. In 11th International Symposium on Games, Automata, Logics, and Formal Verification (Electronic Proceedings in Theoretical Computer Science, Vol. 326), Jean-Francois Raskin and Davide Bresolin (Eds.). Open Publishing Association, 166--181.

[35]

Di Wu and Xenofon Koutsoukos. 2008. Reachability analysis of uncertain systems using bounded-parameter Markov decision processes. Artificial Intelligence 172, 8--9 (2008), 945--954.

Digital Library

Cited By

Delimpaltadakis GLaurenti LMazo M(2024)Formal Analysis of the Sampling Behavior of Stochastic Event-Triggered ControlIEEE Transactions on Automatic Control10.1109/TAC.2023.333374869:7(4491-4505)Online publication date: Jul-2024
https://doi.org/10.1109/TAC.2023.3333748
Calbert JJungers R(2023)Data-driven heuristic symbolic models and application to limit-cycle detection2023 American Control Conference (ACC)10.23919/ACC55779.2023.10156175(4351-4356)Online publication date: 31-May-2023
https://doi.org/10.23919/ACC55779.2023.10156175
Jiang JCoogan SZhao Y(2023)Abstraction-Based Planning for Uncertainty-Aware Legged NavigationIEEE Open Journal of Control Systems10.1109/OJCSYS.2023.32960002(221-234)Online publication date: 2023
https://doi.org/10.1109/OJCSYS.2023.3296000
Show More Cited By

Index Terms

Strategy synthesis for partially-known switched stochastic systems

Recommendations

Distributionally Robust Strategy Synthesis for Switched Stochastic Systems
HSCC '23: Proceedings of the 26th ACM International Conference on Hybrid Systems: Computation and Control

We present a novel framework for formal control of uncertain discrete-time switched stochastic systems against probabilistic reach-avoid specifications. In particular, we consider stochastic systems with additive noise, whose distribution lies in an ...
Symbolic models for stochastic switched systems

Stochastic switched systems are a relevant class of stochastic hybrid systems with probabilistic evolution over a continuous domain and control-dependent discrete dynamics over a finite set of modes. In the past few years several different techniques ...
Feedback control of switched stochastic systems using randomly available active mode information

Almost sure asymptotic stabilization of a discrete-time switched stochastic system is investigated. Information on the active operation mode of the switched system is assumed to be available for control purposes only at random time instants. We propose ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

HSCC '21: Proceedings of the 24th International Conference on Hybrid Systems: Computation and Control

May 2021

300 pages

ISBN:9781450383394

DOI:10.1145/3447928

Program Chairs:
Sergiy Bogomolov
Newcastle University, UK
,
Raphaël Jungers
UCLouvain, Belgium

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGBED: ACM Special Interest Group on Embedded Systems

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 May 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Results Reproduced / v1.1

Author Tags

Qualifiers

Research-article

Funding Sources

NSF (National Science Foundation)

Conference

HSCC '21

Sponsor:

SIGBED

HSCC '21: 24th ACM International Conference on Hybrid Systems: Computation and Control

May 19 - 21, 2021

Tennessee, Nashville

Acceptance Rates

HSCC '21 Paper Acceptance Rate 27 of 77 submissions, 35%;

Overall Acceptance Rate 153 of 373 submissions, 41%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
437
Total Downloads

Downloads (Last 12 months)89
Downloads (Last 6 weeks)8

Reflects downloads up to 09 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Delimpaltadakis GLaurenti LMazo M(2024)Formal Analysis of the Sampling Behavior of Stochastic Event-Triggered ControlIEEE Transactions on Automatic Control10.1109/TAC.2023.333374869:7(4491-4505)Online publication date: Jul-2024
https://doi.org/10.1109/TAC.2023.3333748
Calbert JJungers R(2023)Data-driven heuristic symbolic models and application to limit-cycle detection2023 American Control Conference (ACC)10.23919/ACC55779.2023.10156175(4351-4356)Online publication date: 31-May-2023
https://doi.org/10.23919/ACC55779.2023.10156175
Jiang JCoogan SZhao Y(2023)Abstraction-Based Planning for Uncertainty-Aware Legged NavigationIEEE Open Journal of Control Systems10.1109/OJCSYS.2023.32960002(221-234)Online publication date: 2023
https://doi.org/10.1109/OJCSYS.2023.3296000
Reed RLaurenti LLahijanian M(2023)Promises of Deep Kernel Learning for Control SynthesisIEEE Control Systems Letters10.1109/LCSYS.2023.33409957(3986-3991)Online publication date: 2023
https://doi.org/10.1109/LCSYS.2023.3340995
Skovbekk JLaurenti LFrew ELahijanian M(2023)Formal Abstraction of General Stochastic Systems via Noise PartitioningIEEE Control Systems Letters10.1109/LCSYS.2023.33406217(3711-3716)Online publication date: 2023
https://doi.org/10.1109/LCSYS.2023.3340621
Mathiesen FRomao LCalvert SAbate ALaurenti L(2023)Inner Approximations of Stochastic Programs for Data-Driven Stochastic Barrier Function Design2023 62nd IEEE Conference on Decision and Control (CDC)10.1109/CDC49753.2023.10383306(3073-3080)Online publication date: 13-Dec-2023
https://doi.org/10.1109/CDC49753.2023.10383306
Adams SLahijanian MLaurenti L(2022)Formal Control Synthesis for Stochastic Neural Network Dynamic ModelsIEEE Control Systems Letters10.1109/LCSYS.2022.31781436(2858-2863)Online publication date: 2022
https://doi.org/10.1109/LCSYS.2022.3178143
Jiang JZhao YCoogan S(2022)Safe Learning for Uncertainty-Aware Planning via Interval MDP AbstractionIEEE Control Systems Letters10.1109/LCSYS.2022.31739936(2641-2646)Online publication date: 2022
https://doi.org/10.1109/LCSYS.2022.3173993

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents