abstract

Evolutionary generation of metamorphic relations for cyber-physical systems

Authors:

Maite ArratibelAuthors Info & Claims

GECCO '22: Proceedings of the Genetic and Evolutionary Computation Conference Companion

Pages 15 - 16

https://doi.org/10.1145/3520304.3534077

Published: 19 July 2022 Publication History

Get Access

Abstract

A problem when testing Cyber-Physical Systems (CPS) is the difficulty of determining whether a particular system output or behaviour is correct or not. Metamorphic testing alleviates such a problem by reasoning on the relations expected to hold among multiple executions of the system under test, which are known as Metamorphic Relations (MRs). However, the development of effective MRs is often challenging and requires the involvement of domain experts. This paper summarizes our recent publication: "Generating Metamorphic Relations for Cyber-Physical Systems with Genetic Programming: An Industrial Case Study", presented at ESEC/FSE 2021. In that publication we presented GAssertMRs, the first technique to automatically generate MRs for CPS, leveraging GP to explore the space of candidate solutions. We evaluated GAssertMRs in an industrial case study, outperforming other baselines.

Supplementary Material

PDF File (p15-ayerdi-suppl.pdf)

Supplemental material.

Download
457.59 KB

References

[1]

John Ahlgren, Maria Eugenia Berezin, Kinga Bojarczuk, Elena Dulskyte, Inna Dvortsova, Johann George, Natalija Gucevska, Mark Harman, Maria Lomeli, Erik Meijer, et al. 2021. Testing Web Enabled Simulation at Scale Using Metamorphic Testing. In ICSE-SEIP.

Google Scholar

[2]

Jon Ayerdi, Sergio Segura, Aitor Arrieta, Goiuria Sagardui, and Maite Arratibel. 2020. QoS-aware Metamorphic Testing: An Elevation Case Study. In ISSRE. 104--114.

Google Scholar

[3]

Jon Ayerdi, Valerio Terragni, Aitor Arrieta, Paolo Tonella, Goiuria Sagardui, and Maite Arratibel. 2021. Generating metamorphic relations for cyber-physical systems with genetic programming: an industrial case study. In ESEC/FSE. 1264--1274.

Google Scholar

[4]

Earl T Barr, Mark Harman, Phil McMinn, Muzammil Shahbaz, and Shin Yoo. 2014. The oracle problem in software testing: A survey. IEEE TSE 41, 5 (2014), 507--525.

Digital Library

Google Scholar

[5]

T Y. Chen, S. C. Cheung, and S. M. Yiu. 1998. Metamorphic Testing: A New Approach for Generating Next Test Cases. Technical Report. HKUST-CS98-01.

Google Scholar

[6]

Gunel Jahangirova, David Clark, Mark Harman, and Paolo Tonella. 2016. Test oracle assessment and improvement. In ISSTA. 247--258.

Google Scholar

[7]

Orona. 2021. Orona Group. https://www.orona-group.com/.

Google Scholar

[8]

S. Segura, G. Fraser, A. Sanchez, and A. Ruiz-Cortés. 2016. A Survey on Metamorphic Testing. IEEE TSE 42 (2016), 805--824.

Crossref

Google Scholar

[9]

Valerio Terragni, Gunel Jahangirova, Paolo Tonella, and Mauro Pezzè. 2020. Evolutionary Improvement of Assertion Oracles. In ESEC/FSE. 1178--1189.

Google Scholar

Cited By

View all

Xu CTerragni VZhu HWu JCheung S(2024)MR-Scout: Automated Synthesis of Metamorphic Relations from Existing Test CasesACM Transactions on Software Engineering and Methodology10.1145/365634033:6(1-28)Online publication date: 29-Jun-2024
https://dl.acm.org/doi/10.1145/3656340
Clark AFoster MPrifling BWalkinshaw NHierons RSchmidt VTurner R(2023)Testing Causality in Scientific Modelling SoftwareACM Transactions on Software Engineering and Methodology10.1145/360718433:1(1-42)Online publication date: 24-Nov-2023
https://dl.acm.org/doi/10.1145/3607184

Index Terms

Evolutionary generation of metamorphic relations for cyber-physical systems

Recommendations

Feedback-Directed Metamorphic Testing
Over the past decade, metamorphic testing has gained rapidly increasing attention from both academia and industry, particularly thanks to its high efficacy on revealing real-life software faults in a wide variety of application domains. On the basis of a ...
Generating metamorphic relations for cyber-physical systems with genetic programming: an industrial case study
ESEC/FSE 2021: Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering

One of the major challenges in the verification of complex industrial Cyber-Physical Systems is the difficulty of determining whether a particular system output or behaviour is correct or not, the so-called test oracle problem. Metamorphic testing ...
A New Method for Constructing Metamorphic Relations
QSIC '12: Proceedings of the 2012 12th International Conference on Quality Software

A fundamental problem for software testing is the oracle problem, which means that in many practical situations, it is extremely expensive, if not impossible, to verify the test result given any possible program input. Metamorphic testing is an approach ...

Comments

Information & Contributors

Information

Published In

GECCO '22: Proceedings of the Genetic and Evolutionary Computation Conference Companion

July 2022

2395 pages

ISBN:9781450392686

DOI:10.1145/3520304

Editor:
Jonathan E. Fieldsend
University of Exeter
,
General Chair:
Markus Wagner
The University of Adelaide

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 July 2022

Check for updates

Author Tags

Qualifiers

Abstract

Funding Sources

H2020 European Research Council

Conference

GECCO '22

Sponsor:

SIGEVO

GECCO '22: Genetic and Evolutionary Computation Conference

July 9 - 13, 2022

Massachusetts, Boston

Acceptance Rates

Overall Acceptance Rate 1,669 of 4,410 submissions, 38%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
84
Total Downloads

Downloads (Last 12 months)17
Downloads (Last 6 weeks)0

Reflects downloads up to 03 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Xu CTerragni VZhu HWu JCheung S(2024)MR-Scout: Automated Synthesis of Metamorphic Relations from Existing Test CasesACM Transactions on Software Engineering and Methodology10.1145/365634033:6(1-28)Online publication date: 29-Jun-2024
https://dl.acm.org/doi/10.1145/3656340
Clark AFoster MPrifling BWalkinshaw NHierons RSchmidt VTurner R(2023)Testing Causality in Scientific Modelling SoftwareACM Transactions on Software Engineering and Methodology10.1145/360718433:1(1-42)Online publication date: 24-Nov-2023
https://dl.acm.org/doi/10.1145/3607184

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Feedback-Directed Metamorphic Testing

Generating metamorphic relations for cyber-physical systems with genetic programming: an industrial case study

A New Method for Constructing Metamorphic Relations