Dependent input sampling strategies: using metaheuristics for generating parameterised random sampling regimes
Pages 1451 - 1458
Abstract
Understanding extreme execution times is of great importance in gaining assurance in real-time embedded systems. The standard benchmark for dynamic testing---uniform randomised testing---is inadequate for reaching extreme execution times in these systems. Metaheuristics have been shown to be an effective means of directly searching for inputs with such behaviours but the increasing complexity of modern systems is now posing challenges to the effectiveness of this approach. The research reported in this paper investigates the use of metaheuristic search to discover biased random sampling regimes. Rather than search for test inputs, we search for distributions of test inputs that are then sampled. The search proceeds to discover and exploit relationships between test input variables, leading to sampling regimes where the distribution of a sampled parameter depends on the values of previously sampled input parameters. Our results show that test vectors indirectly generated from our dependent approach produce significantly more extreme (longer) execution times than those generated by direct metaheuristic searches.
References
[1]
Wasif Afzal, Richard Torkar, and Robert Feldt. 2009. A systematic review of search-based testing for non-functional system properties. Information and Software Technology 51, 6 (2009), 957 -- 976.
[2]
Ilhem Boussad, Julien Lepagnot, and Patrick Siarry. 2013. A survey on optimization metaheuristics. Information Sciences 237 (2013), 82 -- 117. Prediction, Control and Diagnosis using Advanced Neural Computations.
[3]
A. Burns and J. A. McDermid. 1994. Real-time safety-critical systems: analysis and synthesis. Software Engineering Journal 9, 6 (Nov 1994), 267--281.
[4]
Gunter Dueck and Tobias Scheuer. 1990. Threshold accepting: A general purpose optimization algorithm appearing superior to simulated annealing. J. Comput. Phys. 90, 1 (1990), 161 -- 175.
[5]
A. Engel. 2010. Verification, Validation and Testing of Engineered Systems. John Wiley & Sons. https://books.google.co.uk/books?id=H6N2CgAAQBAJ
[6]
M. Galassi, J. Davies, J. Theiler, B. Gough, G. Jungman, P. Alken, M. Booth, F. Rossi, and R. Ulerich. 2015. GNU Scientific Library Reference Manual (third ed.). The GSL Team.
[7]
Patrick Graydon and Iain Bate. 2014. Realistic safety cases for the timing of systems. Comput. J. 57, 5 (2014), 759--774.
[8]
M. Harman, Y. Jia, and Y. Zhang. 2015. Achievements, Open Problems and Challenges for Search Based Software Testing. In 2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST). 1--12.
[9]
Courtney E. Howard. 2012. Modern microprocessors: Robust, high-performance aerospace and defense systems harness the power of innovative microprocessors. (March 2012). http://www.militaryaerospace.com/articles/print/volume-23/issue-3/technology-focus/modern-microprocessors.html/ {Online; posted 1-March-2012}.
[10]
S. Kirkpatrick, C. D. Gelatt, and M. P. Vecchi. 1983. Optimization by Simulated Annealing. Science 220, 4598 (1983), 671--680. arXiv:http://science.sciencemag.org/content/220/4598/671.full.pdf
[11]
M. Levy and T. M. Conte. 2009. Embedded Multicore Processors and Systems. IEEE Micro 29, 3 (May 2009), 7--9.
[12]
Sean Luke. 2013. Essentials of Metaheuristics (second ed.). Lulu. Available for free at http://cs.gmu.edu/~sean/book/metaheuristics/.
[13]
Sean Luke. 2017. ECJ 24 and 25: A Java-based Evolutionary Computation Research System. (2017). http://cs.gmu.edu/-eclab/projects/ecj/ {Online; accessed 21-October-2017}.
[14]
P. McMinn. 2011. Search-Based Software Testing: Past, Present and Future. In 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops. 153--163.
[15]
Matthew Patrick, Rob Alexander, Manuel Oriol, and John A. Clark. 2015. Subdomain-based test data generation. Journal of Systems and Software 103 (2015), 328 -- 342.
[16]
Hartmut Pohlheim and Joachim Wegener. 1999. Testing the Temporal Behavior of Real-time Software Modules Using Extended Evolutionary Algorithms. In Proceedings of the 1st Annual Conference on Genetic and Evolutionary Computation - Volume 2 (GECCO'99). Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 1795--1795. http://dl.acm.org/citation.cfm?id=2934046.2934210
[17]
Freescale semiconductor. 2011. P4080 Development System User's Guide. Technical Report.
[18]
Freescale semiconductor. 2014. Running AMP, SMP, or BMP Mode for Multicore Embedded Systems. Technical Report.
[19]
David R. White. 2012. Software review: the ECJ toolkit. Genetic Programming and Evolvable Machines 13,1 (2012), 65--67.
[20]
Reinhard Wilhelm, Jakob Engblom, Andreas Ermedahl, Niklas Holsti, Stephan Thesing, David Whalley, Guillem Bernat, Christian Ferdinand, Reinhold Heckmann, Tulika Mitra, Frank Mueller, Isabelle Puaut, Peter Puschner, Jan Staschulat, and Per Stenström. 2008. The Worst-case Execution-time Problem --- Overview of Methods and Survey of Tools. ACM Trans. Embed. Comput. Syst. 7, 3, Article 36 (May 2008), 53 pages
Index Terms
- Dependent input sampling strategies: using metaheuristics for generating parameterised random sampling regimes
Recommendations
One-test-at-a-time heuristic search for interaction test suites
GECCO '07: Proceedings of the 9th annual conference on Genetic and evolutionary computationAlgorithms for the construction of software interaction test suites have focussed on the special case of pairwise coverage; less is known about efficiently constructing test suites for higher strength coverage. The combinatorial growth of t-tuples ...
A New Swarm-Based Simulated Annealing Hyper-Heuristic Algorithm for Clustering Problem
AbstractHyper-heuristic is a new generation of heuristic algorithms which include a set of heuristics that could generate a set of heuristics or automate the process of selecting a batch of heuristics to address different optimization problems. Most of ...
Investigation on Cooling Schemes and Parameters of Simulated Annealing for Timetabling University Courses
ICACTE '08: Proceedings of the 2008 International Conference on Advanced Computer Theory and EngineeringEvery academic year, educational institutions usually have a problem related to course timetabling solved by academic staff with/without scheduling tool. The course timetabling is known to be NP-hard problem, which means that it is often impossible to ...
Comments
Information & Contributors
Information
Published In
July 2018
1578 pages
Copyright © 2018 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 02 July 2018
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
Conference
GECCO '18
Sponsor:
Acceptance Rates
Overall Acceptance Rate 1,669 of 4,410 submissions, 38%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 105Total Downloads
- Downloads (Last 12 months)7
- Downloads (Last 6 weeks)0
Reflects downloads up to 25 Dec 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in