research-article

Open access

A Neurosymbolic Approach to the Verification of Temporal Logic Properties of Learning-enabled Control Systems

Authors:

Tomoya Yamaguchi,

Danil Prokhorov,

Georgios Fainekos,

Jyotirmoy DeshmukhAuthors Info & Claims

ICCPS '23: Proceedings of the ACM/IEEE 14th International Conference on Cyber-Physical Systems (with CPS-IoT Week 2023)

Pages 98 - 109

https://doi.org/10.1145/3576841.3585928

Published: 09 May 2023 Publication History

Abstract

Signal Temporal Logic (STL) has become a popular tool for expressing formal requirements of Cyber-Physical Systems (CPS). The problem of verifying STL properties of neural network-controlled CPS remains a largely unexplored problem. In this paper, we present a model for the verification of Neural Network (NN) controllers for general STL specifications using a custom neural architecture where we map an STL formula into a feed-forward neural network with ReLU activation. In the case where both our plant model and the controller are ReLU-activated neural networks, we reduce the STL verification problem to reachability in ReLU neural networks. We also propose a new approach for neural network controllers with general activation functions; this approach is a sound and complete verification approach based on computing the Lipschitz constant of the closed-loop control system. We demonstrate the practical efficacy of our techniques on a number of examples of learning-enabled control systems.

References

[1]

Takumi Akazaki and Ichiro Hasuo. 2015. Time robustness in MTL and expressivity in hybrid system falsification. In International Conference on Computer Aided Verification. Springer, 356--374.

[2]

Erling D Andersen and Knud D Andersen. 2000. The MOSEK interior point optimizer for linear programming: an implementation of the homogeneous algorithm. In High performance optimization. Springer, 197--232.

[3]

Kendall Atkinson, Weimin Han, and David E Stewart. 2011. Numerical solution of ordinary differential equations. John Wiley & Sons.

[4]

Trevor Avant and Kristi A Morgansen. 2020. Analytical bounds on the local lipschitz constants of affine-relu functions. arXiv preprint arXiv:2008.06141 (2020).

[5]

Kurtland Chua, Roberto Calandra, Rowan McAllister, and Sergey Levine. 2018. Deep reinforcement learning in a handful of trials using probabilistic dynamics models. Advances in neural information processing systems 31 (2018).

[6]

Marc Peter Deisenroth, Dieter Fox, and Carl Edward Rasmussen. 2013. Gaussian processes for data-efficient learning in robotics and control. IEEE transactions on pattern analysis and machine intelligence 37, 2 (2013), 408--423.

[7]

Alexandre Donzé and Oded Maler. 2010. Robust satisfaction of temporal logic over real-valued signals. In International Conference on Formal Modeling and Analysis of Timed Systems. Springer, 92--106.

Digital Library

[8]

Souradeep Dutta, Xin Chen, and Sriram Sankaranarayanan. 2019. Reachability analysis for neural feedback systems using regressive polynomial rule inference. In Proceedings of the 22nd ACM International Conference on Hybrid Systems: Computation and Control. 157--168.

Digital Library

[9]

Souradeep Dutta, Susmit Jha, Sriram Sanakaranarayanan, and Ashish Tiwari. 2017. Output range analysis for deep neural networks. arXiv preprint arXiv:1709.09130 (2017).

[10]

Ruediger Ehlers. 2017. Formal verification of piece-wise linear feed-forward neural networks. In International Symposium on Automated Technology for Verification and Analysis. Springer, 269--286.

[11]

Georgios E Fainekos and George J Pappas. 2006. Robustness of temporal logic specifications. In Formal approaches to software testing and runtime verification. Springer, 178--192.

[12]

Mahyar Fazlyab, Alexander Robey, Hamed Hassani, Manfred Morari, and George Pappas. 2019. Efficient and accurate estimation of lipschitz constants for deep neural networks. Advances in Neural Information Processing Systems 32 (2019).

[13]

Ian Goodfellow, Yoshua Bengio, and Aaron Courville. 2016. Deep learning. MIT press.

Digital Library

[14]

Navid Hashemi, Justin Ruths, and Mahyar Fazlyab. 2021. Certifying incremental quadratic constraints for neural networks via convex optimization. In Learning for Dynamics and Control. PMLR, 842--853.

[15]

Hsi-Ming Ho, Joël Ouaknine, and James Worrell. 2014. Online monitoring of metric temporal logic. In International Conference on Runtime Verification. Springer, 178--192.

[16]

Chao Huang, Jiameng Fan, Xin Chen, Wenchao Li, and Qi Zhu. 2021. POLAR: A Polynomial Arithmetic Framework for Verifying Neural-Network Controlled Systems. arXiv preprint arXiv:2106.13867 (2021).

[17]

Chao Huang, Jiameng Fan, Wenchao Li, Xin Chen, and Qi Zhu. 2019. Reachnn: Reachability analysis of neural-network controlled systems. ACM Transactions on Embedded Computing Systems (TECS) 18, 5s (2019), 1--22.

Digital Library

[18]

Xiaowei Huang, Marta Kwiatkowska, Sen Wang, and Min Wu. 2017. Safety verification of deep neural networks. In International conference on computer aided verification. Springer, 3--29.

[19]

Radoslav Ivanov, James Weimer, Rajeev Alur, George J Pappas, and Insup Lee. 2019. Verisig: verifying safety properties of hybrid systems with neural network controllers. In Proceedings of the 22nd ACM International Conference on Hybrid Systems: Computation and Control. 169--178.

Digital Library

[20]

Matt Jordan and Alexandros G Dimakis. 2020. Exactly computing the local lipschitz constant of relu networks. Advances in Neural Information Processing Systems 33 (2020), 7344--7353.

[21]

Guy Katz, Clark Barrett, David L Dill, Kyle Julian, and Mykel J Kochenderfer. 2017. Reluplex: An efficient SMT solver for verifying deep neural networks. In International conference on computer aided verification. Springer, 97--117.

[22]

Guy Katz, Derek A Huang, Duligur Ibeling, Kyle Julian, Christopher Lazarus, Rachel Lim, Parth Shah, Shantanu Thakoor, Haoze Wu, Aleksandar Zeljić, et al. 2019. The marabou framework for verification and analysis of deep neural networks. In International Conference on Computer Aided Verification. Springer, 443--452.

[23]

Panagiotis Kouvaros and Alessio Lomuscio. 2018. Formal verification of cnn-based perception systems. arXiv preprint arXiv:1811.11373 (2018).

[24]

Fabian Latorre, Paul Rolland, and Volkan Cevher. 2020. Lipschitz constant estimation of neural networks via sparse polynomial optimization. arXiv preprint arXiv:2004.08688 (2020).

[25]

Jianlin Li, Jiangchao Liu, Pengfei Yang, Liqian Chen, Xiaowei Huang, and Lijun Zhang. 2019. Analyzing deep neural networks with symbolic propagation: Towards higher precision and faster verification. In International static analysis symposium. Springer, 296--319.

[26]

Changliu Liu, Tomer Arnon, Christopher Lazarus, Christopher Strong, Clark Barrett, Mykel J Kochenderfer, et al. 2021. Algorithms for verifying deep neural networks. Foundations and Trends® in Optimization 4, 3--4 (2021), 244--404.

[27]

Johan Lofberg. 2004. YALMIP: A toolbox for modeling and optimization in MATLAB. In 2004 IEEE international conference on robotics and automation (IEEE Cat. No. 04CH37508). IEEE, 284--289.

[28]

Alessio Lomuscio and Lalit Maganti. 2017. An approach to reachability analysis for feed-forward relu neural networks. arXiv preprint arXiv:1706.07351 (2017).

[29]

Oded Maler and Dejan Nickovic. 2004. Monitoring temporal properties of continuous signals. In Formal Techniques, Modelling and Analysis of Timed and Fault-Tolerant Systems. Springer, 152--166.

[30]

Luca Pulina and Armando Tacchella. 2010. An abstraction-refinement approach to verification of artificial neural networks. In International Conference on Computer Aided Verification. Springer, 243--257.

Digital Library

[31]

Aditi Raghunathan, Jacob Steinhardt, and Percy S Liang. 2018. Semidefinite relaxations for certifying robustness to adversarial examples. Advances in Neural Information Processing Systems 31 (2018).

[32]

Carl Edward Rasmussen. 2003. Gaussian processes in machine learning. In Summer school on machine learning. Springer, 63--71.

[33]

Alëna Rodionova, Lars Lindemann, Manfred Morari, and George J Pappas. 2022. Combined left and right temporal robustness for control under stl specifications. IEEE Control Systems Letters (2022).

[34]

Xiaowu Sun, Haitham Khedr, and Yasser Shoukry. 2019. Formal verification of neural network controlled autonomous systems. In Proceedings of the 22nd ACM International Conference on Hybrid Systems: Computation and Control. 147--156.

Digital Library

[35]

Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. 2013. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199 (2013).

[36]

Hoang-Dung Tran, Diago Manzanas Lopez, Patrick Musau, Xiaodong Yang, Luan Viet Nguyen, Weiming Xiang, and Taylor T Johnson. 2019. Star-based reachability analysis of deep neural networks. In International symposium on formal methods. Springer, 670--686.

Digital Library

[37]

Hoang-Dung Tran, Xiaodong Yang, Diego Manzanas Lopez, Patrick Musau, Luan Viet Nguyen, Weiming Xiang, Stanley Bak, and Taylor T Johnson. 2020. NNV: the neural network verification tool for deep neural networks and learning-enabled cyber-physical systems. In International Conference on Computer Aided Verification. Springer, 3--17.

Digital Library

[38]

Shiqi Wang, Kexin Pei, Justin Whitehouse, Junfeng Yang, and Suman Jana. 2018. Efficient formal safety analysis of neural networks. Advances in Neural Information Processing Systems 31 (2018).

[39]

Huan Zhang, Tsui-Wei Weng, Pin-Yu Chen, Cho-Jui Hsieh, and Luca Daniel. 2018. Efficient neural network robustness certification with general activation functions. Advances in neural information processing systems 31 (2018).

Cited By

Hashemi NHoxha BProkhorov DFainekos GDeshmukh J(2024)Scaling Learning based Policy Optimization for Temporal Logic Tasks by Controller Network DropoutACM Transactions on Cyber-Physical Systems10.1145/3696112Online publication date: 16-Sep-2024
https://doi.org/10.1145/3696112
Pal NJohnson T(2023)Formal Verification of Long Short-Term Memory based Audio Classifiers: A Star based ApproachElectronic Proceedings in Theoretical Computer Science10.4204/EPTCS.395.12395(162-179)Online publication date: 15-Nov-2023
https://doi.org/10.4204/EPTCS.395.12
An ZWang XT. Johnson TSprinkle JMa M(2023)Runtime Monitoring of Accidents in Driving Recordings with Multi-type Logic in Empirical ModelsRuntime Verification10.1007/978-3-031-44267-4_21(376-388)Online publication date: 3-Oct-2023
https://dl.acm.org/doi/10.1007/978-3-031-44267-4_21

Recommendations

NNV: The Neural Network Verification Tool for Deep Neural Networks and Learning-Enabled Cyber-Physical Systems
Computer Aided Verification
Abstract
This paper presents the Neural Network Verification (NNV) software tool, a set-based verification framework for deep neural networks (DNNs) and learning-enabled cyber-physical systems (CPS). The crux of NNV is a collection of reachability ...
Model-Agnostic Reachability Analysis on Deep Neural Networks
Advances in Knowledge Discovery and Data Mining
Abstract
Verification plays an essential role in the formal analysis of safety-critical systems. Most current verification methods have specific requirements when working on Deep Neural Networks (DNNs). They either target one particular network category, ...
Transformation of neural network weight trajectories on a 2D plane for a learning-type neural network direct controller

Through a simulation of the tracking method for a neural network weight change on a 2D plane, we noticed that in some cases it was hard for untrained users to observe the neural network weight performance. To overcome this problem, we applied a ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ICCPS '23: Proceedings of the ACM/IEEE 14th International Conference on Cyber-Physical Systems (with CPS-IoT Week 2023)

May 2023

291 pages

ISBN:9798400700361

DOI:10.1145/3576841

General Chairs:
Sayan Mitra
University of Illinois at Urbana Champaign
,
Nalini Venkatasubramanian
University of California at Irvine
,
Program Chairs:
Abhishek Dubey
Vanderbilt University
,
Lu Feng
University of Virginia
,
Publications Chairs:
Mahsa Ghasemi
Purdue University
,
Jonathan Sprinkle
Vanderbilt University

Copyright © 2023 Owner/Author(s).

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

SIGBED: ACM Special Interest Group on Embedded Systems
IEEE TCRTS

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 May 2023

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Conference

ICCPS '23

Sponsor:

SIGBED

ICCPS '23: ACM/IEEE 14th International Conference on Cyber-Physical Systems (with CPS-IoT Week 2023)

May 9 - 12, 2023

TX, San Antonio, USA

Acceptance Rates

Overall Acceptance Rate 25 of 91 submissions, 27%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
671
Total Downloads

Downloads (Last 12 months)419
Downloads (Last 6 weeks)52

Reflects downloads up to 23 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Hashemi NHoxha BProkhorov DFainekos GDeshmukh J(2024)Scaling Learning based Policy Optimization for Temporal Logic Tasks by Controller Network DropoutACM Transactions on Cyber-Physical Systems10.1145/3696112Online publication date: 16-Sep-2024
https://doi.org/10.1145/3696112
Pal NJohnson T(2023)Formal Verification of Long Short-Term Memory based Audio Classifiers: A Star based ApproachElectronic Proceedings in Theoretical Computer Science10.4204/EPTCS.395.12395(162-179)Online publication date: 15-Nov-2023
https://doi.org/10.4204/EPTCS.395.12
An ZWang XT. Johnson TSprinkle JMa M(2023)Runtime Monitoring of Accidents in Driving Recordings with Multi-type Logic in Empirical ModelsRuntime Verification10.1007/978-3-031-44267-4_21(376-388)Online publication date: 3-Oct-2023
https://dl.acm.org/doi/10.1007/978-3-031-44267-4_21

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents