research-article

Formal Requirement Debugging for Testing and Verification of Cyber-Physical Systems

Authors:

Adel Dokhanchi,

Georgios FainekosAuthors Info & Claims

ACM Transactions on Embedded Computing Systems (TECS), Volume 17, Issue 2

Article No.: 34, Pages 1 - 26

https://doi.org/10.1145/3147451

Published: 12 December 2017 Publication History

Abstract

A framework for the elicitation and debugging of formal specifications for Cyber-Physical Systems is presented. The elicitation of specifications is handled through a graphical interface. Two debugging algorithms are presented. The first checks for erroneous or incomplete temporal logic specifications without considering the system. The second can be utilized for the analysis of reactive requirements with respect to system test traces. The specification debugging framework is applied on a number of formal specifications collected through a user study. The user study establishes that requirement errors are common and that the debugging framework can resolve many insidious specification errors.

References

[1]

Houssam Abbas, Georgios Fainekos, Sriram Sankaranarayanan, Franjo Ivančić, and Aarti Gupta. 2013. Probabilistic temporal logic falsification of cyber-physical systems. ACM Transactions on Embedded Computing Systems 12, 2s, Article 95 (May 2013), 30 pages.

Digital Library

[2]

Rajeev Alur, Costas Courcoubetis, Nicolas Halbwachs, Thomas A. Henzinger, Pei-Hsin Ho, Xavier Nicollin, Alfredo Olivero, Joseph Sifakis, and Sergio Yovine. 1995. The algorithmic analysis of hybrid systems. Theoretical Computer Science 138, 1 (1995), 3--34.

Digital Library

[3]

Rajeev Alur, Tomás Feder, and Thomas A. Henzinger. 1996. The benefits of relaxing punctuality. Journal of the ACM 43, 1 (1996), 116--146.

Digital Library

[4]

Glenn Ammons, David Mandelin, Rastislav Bodík, and James R. Larus. 2003. Debugging temporal specifications with concept analysis. In Proceedings of the ACM SIGPLAN 2003 Conference on Programming Language Design and Implementation. 182--195.

Digital Library

[5]

Yashwanth Singh Rahul Annapureddy, Che Liu, Georgios E. Fainekos, and Sriram Sankaranarayanan. 2011. S-TaLiRo: A tool for temporal logic falsification for hybrid systems. In Tools and Algorithms for the Construction and Analysis of Systems (LNCS), Vol. 6605. Springer, 254--257.

Digital Library

[6]

Marco Autili, Paola Inverardi, and Patrizio Pelliccione. 2007. Graphical scenarios for specifying temporal properties: An automated approach. Automated Software Engineering 14, 3 (2007), 293--340.

Digital Library

[7]

Thomas Ball and Orna Kupferman. 2008. Vacuity in testing. In Proceedings of Tests and Proofs, Second International Conference (TAP’08). 4--17.

Digital Library

[8]

Derek L. Beatty and Randal E. Bryant. 1994. Formally verifying a microprocessor using a simulation methodology. In Proceedings of the 31th Design Automation Conference (DAC’94). 596--602.

Digital Library

[9]

Ilan Beer, Shoham Ben-David, Cindy Eisner, and Yoav Rodeh. 2001. Efficient detection of vacuity in temporal model checking. Formal Methods in System Design 18, 2 (2001), 141--163.

Digital Library

[10]

Shoham Ben-David, Fady Copty, Dana Fisman, and Sitvanit Ruah. 2015. Vacuity in practice: Temporal antecedent failure. Formal Methods in System Design 46, 1 (2015), 81--104.

Digital Library

[11]

Marcello M. Bersani, Matteo Rossi, and Pierluigi San Pietro. 2014. A logical characterization of timed (non-)regular languages. In Proceedings of the 39th International Symposium on Mathematical Foundations of Computer Science 2014, Part I (MFCS’14). 75--86.

[12]

Marcello M. Bersani, Matteo Rossi, and Pierluigi San Pietro. 2016. A tool for deciding the satisfiability of continuous-time metric temporal logic. Acta Informatica 53, 2 (2016), 171--206.

Digital Library

[13]

Marcello Maria Bersani, Matteo Rossi, and Pierluigi San Pietro. 2013. Deciding the satisfiability of MITL specifications. In Proceedings of the 4th International Symposium on Games, Automata, Logics and Formal Verification (EPTCS’13), Vol. 119. Open Publishing Association, 64--78.

[14]

Hana Chockler and Ofer Strichman. 2009. Before and after vacuity. Formal Methods in System Design 34, 1 (Feb. 2009), 37--58.

Digital Library

[15]

Alessandro Cimatti, Marco Roveri, Viktor Schuppan, and Andrei Tchaltsev. 2008. Diagnostic information for realizability. In Proceedings of the 9th International Conference on Verification, Model Checking, and Abstract Interpretation (VMCAI’08). 52--67.

Digital Library

[16]

Edmund M. Clarke, Orna Grumberg, and Doron A. Peled. 1999. Model Checking. MIT Press, Cambridge, MA.

Digital Library

[17]

Stéphane Demri and Deepak D’Souza. 2007. An automata-theoretic approach to constraint LTL. Information and Computation 205, 3 (2007), 380--415.

Digital Library

[18]

Adel Dokhanchi, Bardh Hoxha, and Georgios E. Fainekos. 2015. Metric interval temporal logic specification elicitation and debugging. In ACM/IEEE International Conference on Formal Methods and Models for Codesign (MEMOCODE’15). 70--79.

Digital Library

[19]

Adel Dokhanchi, Shakiba Yaghoubi, Bardh Hoxha, and Georgios Fainekos. 2017. Vacuity aware falsification for MTL request-response specifications. In Proceedings of the 13th IEEE Conference on Automation Science and Engineering (CASE’17).

[20]

Alexandre Donze. 2010. Breach, A toolbox for verification and parameter synthesis of hybrid systems. In Computer Aided Verification. LNCS, Vol. 6174. Springer, 167--170.

Digital Library

[21]

Matthew B. Dwyer, George S. Avrunin, and James C. Corbett. 1998. Property specification patterns for finite-state verification. In Proceedings of the 2nd Workshop on Formal Methods in Software Practice (FMSP’98). ACM, 7--15.

Digital Library

[22]

Rüdiger Ehlers and Vasumathi Raman. 2014. Low-effort specification debugging and analysis. In Proceedings 3rd Workshop on Synthesis (SYNT’14).117--133.

[23]

Georgios Fainekos, Sriram Sankaranarayanan, Koichi Ueda, and Hakan Yazarel. 2012. Verification of automotive control applications using S-TaLiRo. In Proceedings of the American Control Conference.

[24]

Georgios E. Fainekos. 2011. Revising temporal logic specifications for motion planning. In IEEE International Conference on Robotics and Automation (ICRA’11). 40--45.

[25]

Dana Fisman, Orna Kupferman, Sarai Sheinvald-Faragy, and Moshe Y. Vardi. 2008. A framework for inherent vacuity. In Proceedings of the 4th International Haifa Verification Conference Hardware and Software: Verification and Testing (HVC’08). 7--22.

Digital Library

[26]

Klaus Havelund and Grigore Rosu. 2004. Efficient monitoring of safety properties. Journal on Software Tools for Technology Transfer 6, 2 (2004), 158--173.

Digital Library

[27]

Florian Horn, Wolfgang Thomas, Nico Wallmeier, and Martin Zimmermann. 2015. Optimal strategy synthesis for request-response games. RAIRO - Theoretical Informatics and Applications 49, 3 (2015), 179--203.

[28]

Bardh Hoxha, Houssam Abbas, and Georgios Fainekos. 2014a. Benchmarks for temporal logic requirements for automotive systems. In Proceedings of Applied Verification for Continuous and Hybrid Systems.

[29]

Bardh Hoxha, Hoang Bach, Houssam Abbas, Adel Dokhanchi, Yoshihiro Kobayashi, and Georgios Fainekos. 2014b. Towards formal specification visualization for testing and monitoring of cyber-physical systems. In Proceedings of the International Workshop on Design and Implementation of Formal Tools and Systems.

[30]

Bardh Hoxha, Nikolaos Mavridis, and Georgios Fainekos. 2015. ViSpec: A graphical tool for easy elicitation of MTL requirements. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems.

[31]

Xiaoqing Jin, Alexandre Donze, Jyotirmoy Deshmukh, and Sanjit Seshia. 2013. Mining requirements from closed-loop control models. In Hybrid Systems: Computation and Control. ACM Press.

Digital Library

[32]

James Kapinski, Jyotirmoy V. Deshmukh, Xiaoqing Jin, Hisahiro Ito, and Kenneth R. Butts. 2015. Simulation-guided approaches for verification of automotive powertrain control systems. In Proceedings of the American Control Conference (ACC’15). 4086--4095.

[33]

Kangjin Kim, Georgios E. Fainekos, and Sriram Sankaranarayanan. 2012. On the revision problem of specification automata. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA’12). 5171--5176.

[34]

Robert Könighofer, Georg Hofferek, and Roderick Bloem. 2013. Debugging formal specifications: A practical approach using model-based diagnosis and counterstrategies. Journal on Software Tools for Technology Transfer 15, 5--6 (2013), 563--583.

[35]

Sascha Konrad and Betty H. C. Cheng. 2005. Real-time specification patterns. In Proceedings of the 27th International Conference on Software Engineering (ICSE’05). ACM, 372--381.

Digital Library

[36]

Ron Koymans. 1990. Specifying real-time properties with metric temporal logic.Real-Time Systems 2, 4 (1990), 255--299.

Digital Library

[37]

Orna Kupferman and Moshe Y. Vardi. 2003. Vacuity detection in temporal model checking. Journal on Software Tools for Technology Transfer 4, 2 (2003), 224--233.

[38]

Jianwen Li, Lijun Zhang, Geguang Pu, Moshe Y. Vardi, and Jifeng He. 2013. LTL satisfiability checking revisited. In 2013 20th International Symposium on Temporal Representation and Reasoning. 91--98.

Digital Library

[39]

Oded Maler and Dejan Nickovic. 2004. Monitoring temporal properties of continuous signals. In Proceedings of FORMATS-FTRTFT (LNCS), Vol. 3253. 152--166.

[40]

Roland Meyer, Johannes Faber, Jochen Hoenicke, and Andrey Rybalchenko. 2008. Model checking duration calculus: A practical approach. Formal Aspects of Computing 20, 4 (2008), 481--505.

[41]

Amalinda Post, Jochen Hoenicke, and Andreas Podelski. 2011. Vacuous real-time requirements. In 19th IEEE International Requirements Engineering Conference (RE’11). 153--162.

Digital Library

[42]

Vasumathi Raman and Hadas Kress-Gazit. 2011. Analyzing unsynthesizable specifications for high-level robot behavior using LTLMoP. In Proceedings of the 23rd International Conference on Computer Aided Verification (CAV’11). 663--668.

Digital Library

[43]

Kristin Y. Rozier and Moshe Y. Vardi. 2010. LTL satisfiability checking. Journal on Software Tools for Technology Transfer 12, 2 (2010), 123--137.

Digital Library

[44]

Hengyi Yang, Bardh Hoxha, and Georgios Fainekos. 2012. Querying parametric temporal logic properties on embedded systems. In Testing Software and Systems. Springer, 136--151.

[45]

Pengcheng Zhang, Bixin Li, and Lars Grunske. 2010. Timed property sequence chart. Journal of Systems and Software 83, 3 (2010), 371--390.

Digital Library

Cited By

Hekmatnejad MHoxha BDeshmukh JYang YFainekos G(2024)Formalizing and evaluating requirements of perception systems for automated vehicles using spatio-temporal perception logicInternational Journal of Robotics Research10.1177/0278364923122354643:2(203-238)Online publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1177/02783649231223546
Bartocci EBloem RMaderbacher BManjunath NNičković D(2022)Adaptive testing for specification coverage and refinement in CPS modelsNonlinear Analysis: Hybrid Systems10.1016/j.nahs.2022.10125446(101254)Online publication date: Nov-2022
https://doi.org/10.1016/j.nahs.2022.101254
Masmoudi CMarange PBonjour ELevrat EKerbrat A(2022)Adopting formal methods on requirements verification and validation for cyber-physical systems: A systematic literature reviewIFAC-PapersOnLine10.1016/j.ifacol.2022.10.13155:10(3274-3279)Online publication date: 2022
https://doi.org/10.1016/j.ifacol.2022.10.131
Show More Cited By

Index Terms

Formal Requirement Debugging for Testing and Verification of Cyber-Physical Systems
1. Computing methodologies
  1. Modeling and simulation
    1. Model development and analysis
      1. Model verification and validation
2. Theory of computation
  1. Logic
    1. Logic and verification

Recommendations

Requirement Specification for Transportation Cyber Physical Systems
GREENCOM-ITHINGS-CPSCOM '13: Proceedings of the 2013 IEEE International Conference on Green Computing and Communications and IEEE Internet of Things and IEEE Cyber, Physical and Social Computing

Software requirements errors are considered to account for a majority of software failures, and a number of accidents happen because of software requirements errors. The requirements specification of transportation cyber physical system is particularly ...
Modeling and formal analysis of virtually synchronous cyber-physical systems in AADL
Abstract
This paper presents the HybridSynchAADL modeling language and formal analysis tool for virtually synchronous cyber-physical systems with complex control programs, continuous behaviors, and bounded clock skews, network delays, and execution times. ...
Debugging formal specifications: a practical approach using model-based diagnosis and counterstrategies

Creating a formal specification for a design is an error-prone process. At the same time, debugging incorrect specifications is difficult and time consuming. In this work, we propose a debugging method for formal specifications that does not require an ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Embedded Computing Systems

ACM Transactions on Embedded Computing Systems Volume 17, Issue 2

Special Issue on MEMCODE 2015 and Regular Papers (Diamonds)

March 2018

640 pages

ISSN:1539-9087

EISSN:1558-3465

DOI:10.1145/3160927

Editor:
Sandeep K. Shukla
Indian Institute of Technology, India

Issue’s Table of Contents

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Journal Family

ACM Journals for the Design of Smart and Connected Systems

Publication History

Published: 12 December 2017

Accepted: 01 September 2017

Revised: 01 April 2017

Received: 01 January 2016

Published in TECS Volume 17, Issue 2

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

NSF CNS
NSF I/UCRC Center for Embedded Systems
NSF IIP

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

15
Total Citations
View Citations
374
Total Downloads

Downloads (Last 12 months)15
Downloads (Last 6 weeks)3

Reflects downloads up to 10 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Hekmatnejad MHoxha BDeshmukh JYang YFainekos G(2024)Formalizing and evaluating requirements of perception systems for automated vehicles using spatio-temporal perception logicInternational Journal of Robotics Research10.1177/0278364923122354643:2(203-238)Online publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1177/02783649231223546
Bartocci EBloem RMaderbacher BManjunath NNičković D(2022)Adaptive testing for specification coverage and refinement in CPS modelsNonlinear Analysis: Hybrid Systems10.1016/j.nahs.2022.10125446(101254)Online publication date: Nov-2022
https://doi.org/10.1016/j.nahs.2022.101254
Masmoudi CMarange PBonjour ELevrat EKerbrat A(2022)Adopting formal methods on requirements verification and validation for cyber-physical systems: A systematic literature reviewIFAC-PapersOnLine10.1016/j.ifacol.2022.10.13155:10(3274-3279)Online publication date: 2022
https://doi.org/10.1016/j.ifacol.2022.10.131
Wang RSato YLiu S(2021)Mutated Specification-Based Test Data Generation with a Genetic AlgorithmMathematics10.3390/math90403319:4(331)Online publication date: 7-Feb-2021
https://doi.org/10.3390/math9040331
Zhong BJordan CProvost J(2021)Extending Signal Temporal Logic with Quantitative Semantics by Intervals for Robust Monitoring of Cyber-physical SystemsACM Transactions on Cyber-Physical Systems10.1145/33778685:2(1-25)Online publication date: 4-Jan-2021
https://dl.acm.org/doi/10.1145/3377868
Pecker-Marcosig EZudaire SGarrett MUchitel SCastro R(2020)Unified DEVS-based platform for modeling and simulation of hybrid control systemsProceedings of the Winter Simulation Conference10.5555/3466184.3466303(1051-1062)Online publication date: 14-Dec-2020
https://dl.acm.org/doi/10.5555/3466184.3466303
Pecker-Marcosig EZudaire SGarrett MUchitel SCastro R(2020)Unified Devs-Based Platform for Modeling and Simulation of Hybrid Control Systems2020 Winter Simulation Conference (WSC)10.1109/WSC48552.2020.9384025(1051-1062)Online publication date: 14-Dec-2020
https://doi.org/10.1109/WSC48552.2020.9384025
Eddeland JClaessen KSmallbone NRamezani ZMiremadi SAkesson K(2020)Enhancing Temporal Logic Falsification with Specification Transformation and Valued BooleansIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems10.1109/TCAD.2020.2966480(1-1)Online publication date: 2020
https://doi.org/10.1109/TCAD.2020.2966480
Yamaguchi THoxha BProkhorov DDeshmukh J(2020)Specification-guided Software Fault Localization for Autonomous Mobile Systems2020 18th ACM-IEEE International Conference on Formal Methods and Models for System Design (MEMOCODE)10.1109/MEMOCODE51338.2020.9315067(1-12)Online publication date: 2-Dec-2020
https://doi.org/10.1109/MEMOCODE51338.2020.9315067
Afzal AGoues CHilton MTimperley C(2020)A Study on Challenges of Testing Robotic Systems2020 IEEE 13th International Conference on Software Testing, Validation and Verification (ICST)10.1109/ICST46399.2020.00020(96-107)Online publication date: Oct-2020
https://doi.org/10.1109/ICST46399.2020.00020
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents