Checking Complex Source Code-Level Constraints using Runtime Verification
Pages 255 - 265
Abstract
Runtime Verification (RV) is the process of taking a trace, representing an execution of some computational system, and checking it for satisfaction of some specification, written in a specification language. RV approaches are often aimed at being used as part of software development processes. In this case, engineers might maintain a set of specifications that capture properties concerning their source code’s behaviour at runtime. To be used in such a setting, an RV approach must provide a specification language that is practical for engineers to use regularly, along with an efficient monitoring algorithm that enables program executions to be checked quickly. This work develops an RV approach that has been adopted by two industry partners. In particular, we take a source code fragment of an existing specification language, which enables properties of interest to our partners to be captured easily, and develop 1) a new semantics for the fragment, 2) an instrumentation approach, and 3) a monitoring procedure for it. We show that our monitoring procedure scales to program execution traces containing up to one million events, and describe initial applications of our prototype framework (that implements our instrumentation and monitoring procedures) by the partners themselves.
References
[1]
Stanley Bak. 2023. The AeroBenchVVPython GitHub Repository. https://github.com/stanleybak/AeroBenchVVPython Accessed: 2023-08-11
[2]
David Basin, Bhargav Nagaraja Bhatt, Srđ an Krstić, and Dmitriy Traytel. 2019. Almost event-rate independent monitoring. Formal Methods in System Design, 54, 3 (2019), 449–478. https://doi.org/10.1007/s10703-018-00328-3
[3]
David Basin, Germano Caronni, Sarah Ereth, Matúš Harvan, Felix Klaedtke, and Heiko Mantel. 2016. Scalable Offline Monitoring of Temporal Specifications. Form. Methods Syst. Des., 49, 1–2 (2016), oct, 75–108. issn:0925-9856 https://doi.org/10.1007/s10703-016-0242-y
[4]
Andreas Bauer, Martin Leucker, and Christian Schallhart. 2011. Runtime Verification for LTL and TLTL. ACM Trans. Softw. Eng. Methodol., 20, 4 (2011), Article 14, sep, 64 pages. issn:1049-331X https://doi.org/10.1145/2000799.2000800
[5]
Eric Bodden, Patrick Lam, and Laurie Hendren. 2010. Clara: A Framework for Partially Evaluating Finite-State Runtime Monitors Ahead of Time. In Runtime Verification, Howard Barringer, Ylies Falcone, Bernd Finkbeiner, Klaus Havelund, Insup Lee, Gordon Pace, Grigore Roşu, Oleg Sokolsky, and Nikolai Tillmann (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg. 183–197. isbn:978-3-642-16612-9
[6]
Borzoo Bonakdarpour, Samaneh Navabpour, and Sebastian Fischmeister. 2011. Sampling-Based Runtime Verification. In FM 2011: Formal Methods, Michael Butler and Wolfram Schulte (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg. 88–102. isbn:978-3-642-21437-0
[7]
Chaima Boufaied, Claudio Menghi, Domenico Bianculli, and Lionel C. Briand. 2023. Trace Diagnostics for Signal-Based Temporal Properties. IEEE Transactions on Software Engineering, 49, 5 (2023), 3131–3154. https://doi.org/10.1109/TSE.2023.3242588
[8]
Agnishom Chattopadhyay and Konstantinos Mamouras. 2020. A Verified Online Monitor for Metric Temporal Logic with Quantitative Semantics. In Runtime Verification. Springer International Publishing, Cham. 383–403. isbn:978-3-030-60508-7
[9]
Christoph Czepa, Amirali Amiri, Evangelos Ntentos, and Uwe Zdun. 2019. Modeling compliance specifications in linear temporal logic, event processing language and property specification patterns: a controlled experiment on understandability. Software and Systems Modeling, 18, 6 (2019), 3331–3371. https://doi.org/10.1007/s10270-019-00721-4
[10]
C. Czepa and U. Zdun. 2018. On the Understandability of Temporal Properties Formalized in Linear Temporal Logic, Property Specification Patterns and Event Processing Language. IEEE Trans. Softw. Eng., 46 (2018), 1–13. issn:0098-5589
[11]
Joshua Dawes and Domenico Bianculli. 2024. Checking Complex Source Code-level Constraints using Runtime Verification - FSE 2024 (Industry papers track) - Artefact. https://doi.org/10.6084/m9.figshare.25783266.v1
[12]
Joshua Dawes and Alexander Vatov. 2024. SCSL-Trace-Checker - Source Code. https://doi.org/10.6084/m9.figshare.25784355.v1
[13]
Joshua Heneage Dawes and Domenico Bianculli. 2021. Specifying Properties over Inter-procedural, Source Code Level Behaviour of Programs. In Runtime Verification, Lu Feng and Dana Fisman (Eds.). Springer International Publishing, Cham. 23–41. isbn:978-3-030-88494-9
[14]
Joshua Heneage Dawes and Domenico Bianculli. 2021. Specifying Properties over Inter-procedural, Source Code Level Behaviour of Programs. In Runtime Verification. Springer International Publishing, Cham. 23–41. isbn:978-3-030-88494-9
[15]
Joshua Heneage Dawes and Domenico Bianculli. 2022. Specifying Source Code and Signal-based Behaviour of Cyber-Physical System Components. In Formal Aspects of Component Software. Springer International Publishing, Cham. 20–38. isbn:978-3-031-20872-0
[16]
Joshua Heneage Dawes, Marta Han, Omar Javed, Giles Reger, Giovanni Franzoni, and Andreas Pfeiffer. 2020. Analysing the Performance of Python-Based Web Services with the VyPR Framework. In Runtime Verification: 20th International Conference, RV 2020, Los Angeles, CA, USA, October 6–9, 2020, Proceedings. Springer-Verlag, Berlin, Heidelberg. 67–86. isbn:978-3-030-60507-0 https://doi.org/10.1007/978-3-030-60508-7_4
[17]
Joshua Heneage Dawes and Giles Reger. 2019. Specification of Temporal Properties of Functions for Runtime Verification. In Proceedings of the 34th ACM/SIGAPP Symposium on Applied Computing (SAC ’19). Association for Computing Machinery, New York, NY, USA. 2206–2214. isbn:9781450359337 https://doi.org/10.1145/3297280.3297497
[18]
Adel Dokhanchi, Bardh Hoxha, and Georgios Fainekos. 2014. On-Line Monitoring for Temporal Logic Robustness. CoRR, abs/1408.0045 (2014), arXiv:1408.0045. arxiv:1408.0045
[19]
Cindy Eisner, Dana Fisman, John Havlicek, Yoad Lustig, Anthony McIsaac, and David Van Campenhout. 2003. Reasoning with Temporal Logic on Truncated Paths. In Computer Aided Verification, Warren A. Hunt and Fabio Somenzi (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg. 27–39. isbn:978-3-540-45069-6
[20]
Thomas Ferrère, Oded Maler, and Dejan Ničković. 2015. Trace diagnostics using temporal implicants. In International Symposium on Automated Technology for Verification and Analysis. Springer, Cham. 241–258.
[21]
M.C.W. Geilen. 2001. On the Construction of Monitors for Temporal Logic Properties. Electronic Notes in Theoretical Computer Science, 55, 2 (2001), 181–199. issn:1571-0661 https://doi.org/10.1016/S1571-0661(04)00252-X RV’2001, Runtime Verification (in connection with CAV ’01)
[22]
Klaus Havelund and Grigore Roşu. 2002. Synthesizing Monitors for Safety Properties. In Tools and Algorithms for the Construction and Analysis of Systems. Springer Berlin Heidelberg, Berlin, Heidelberg. 342–356. isbn:978-3-540-46002-2
[23]
Hsi-Ming Ho, Joël Ouaknine, and James Worrell. 2014. Online Monitoring of Metric Temporal Logic. In Runtime Verification. Springer International Publishing, Cham. 178–192. isbn:978-3-319-11164-3
[24]
Moonzoo Kim, Mahesh Viswanathan, Sampath Kannan, Insup Lee, and Oleg Sokolsky. 2004. Java-MaC: A Run-Time Assurance Approach for Java Programs. Form. Methods Syst. Des., 24, 2 (2004), mar, 129–155. issn:0925-9856 https://doi.org/10.1023/B:FORM.0000017719.43755.7c
[25]
Konstantinos Mamouras, Agnishom Chattopadhyay, and Zhifu Wang. 2021. A Compositional Framework for Quantitative Online Monitoring over Continuous-Time Signals. In Runtime Verification. Springer International Publishing, Cham. 142–163. isbn:978-3-030-88494-9
[26]
Amjad Nusayr and Jonathan Cook. 2009. Using AOP for Detailed Runtime Monitoring Instrumentation. In Proceedings of the Seventh International Workshop on Dynamic Analysis (WODA ’09). Association for Computing Machinery, New York, NY, USA. 8–14. isbn:9781605586564 https://doi.org/10.1145/2134243.2134246
[27]
Giles Reger, Helena Cuenca Cruz, and David Rydeheard. 2015. MarQ: Monitoring at Runtime with QEA. In Tools and Algorithms for the Construction and Analysis of Systems. Springer Berlin Heidelberg, Berlin, Heidelberg. 596–610. isbn:978-3-662-46681-0
[28]
Joshua Schneider, David Basin, Frederik Brix, Srdjan Krstic, and Dmitriy Traytel. 2019. Adaptive Online First-Order Monitoring. In Automated Technology for Verification and Analysis - 17th International Symposium, ATVA 2019, Taipei, Taiwan, October 28-31, 2019, Proceedings (Lecture Notes in Computer Science, Vol. 11781). Springer, 133–150. https://doi.org/10.1007/978-3-030-31784-3_8
[29]
Joshua Schneider, David Basin, Frederik Brix, Srundefinedan Krstić, and Dmitriy Traytel. 2021. Scalable Online First-Order Monitoring. Int. J. Softw. Tools Technol. Transf., 23, 2 (2021), apr, 185–208. issn:1433-2779 https://doi.org/10.1007/s10009-021-00607-1
[30]
Cristina Stratan, Joshua Dawes, and Domenico Bianculli. 2024. Diagnosing Violations of Time-based Properties Captured in iCFTL. In Proceedings of the 2024 International Conference on Formal Methods in Software Engineering (FormaliSE 2024), co-located with ICSE 2024, Lisbon, Portugal. ACM.
[31]
Moshe Y. Vardi. 1996. An Automata-Theoretic Approach to Linear Temporal Logic. In Proceedings of the VIII Banff Higher Order Workshop Conference on Logics for Concurrency : Structure versus Automata : Structure versus Automata. Springer-Verlag, Berlin, Heidelberg. 238–266. isbn:3540609156
Index Terms
- Checking Complex Source Code-Level Constraints using Runtime Verification
Recommendations
Introduction to the special issue on runtime verification
This article introduces the extended versions of selected papers from the refereed proceedings of the 16th International Conference on Runtime Verification (RV 2016) held in Madrid, Spain, in September 2016. Runtime verification encompasses all aspects ...
Efficient and expressive bytecode-level instrumentation for Java programs
AbstractWe present an efficient and expressive tool for the instrumentation of Java programs at the bytecode level. BISM (Bytecode-Level Instrumentation for Software Monitoring) is a lightweight Java bytecode instrumentation tool that features an ...
BISM: Bytecode-Level Instrumentation for Software Monitoring
Runtime VerificationAbstractBISM (Bytecode-level Instrumentation for Software Monitoring) is a lightweight Java bytecode instrumentation tool which features an expressive high-level control-flow-aware instrumentation language. The language follows the aspect-oriented ...
Comments
Information & Contributors
Information
Published In
Copyright © 2024 Owner/Author.
This work is licensed under a Creative Commons Attribution International 4.0 License.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 10 July 2024
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- Horizon 2020 Framework Programme
Conference
FSE '24
Sponsor:
FSE '24: 32nd ACM International Conference on the Foundations of Software Engineering
July 15 - 19, 2024
Porto de Galinhas, Brazil
Acceptance Rates
Overall Acceptance Rate 112 of 543 submissions, 21%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 11Total Downloads
- Downloads (Last 12 months)11
- Downloads (Last 6 weeks)11
Reflects downloads up to 10 Aug 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in