research-article

Using automated search to generate test data for matlab

Authors:

Simon Poulding,

John A. ClarkAuthors Info & Claims

GECCO '09: Proceedings of the 11th Annual conference on Genetic and evolutionary computation

Pages 1697 - 1704

https://doi.org/10.1145/1569901.1570128

Published: 08 July 2009 Publication History

Abstract

The critical functionality of many software applications relies on code that performs mathematically complex computations. However, such code is often difficult to test owing to the compound datatypes used and complicated mathematical operations performed. This paper proposes the use of automated search as an efficient means of generating test data for this type of software. Taking Matlab as an example of widely-used mathematical software, a technical framework is described that extends previous work on search-based test data generation in order to handle matrix datatypes and associated relational operators. An empirical evaluation demonstrates the feasibility of this approach.

References

[1]

Matlab, version 2007b. The Mathworks, Inc., Natick, MA, USA.

[2]

Matlab Central File Exchange. www.mathworks.com/matlabcentral/~leexchange.

[3]

Simulink. The Mathworks, Inc., Natick, MA, USA.

[4]

A. Arcuri and X. Yao. Search based software testing of object-oriented containers. Information Sciences, 178, 2008.

Digital Library

[5]

M. Harman. The current state and future of search based software engineering. Proc. 29th Int'l Conf. Software Eng. (ICSE 2007), Future of Software Engineering (FoSE), pages 342--357, 2007.

Digital Library

[6]

B. Korel. Automatic software test data generation. IEEE Trans. Software Eng., 16(8):870--879, August 1990.

Digital Library

[7]

K. Lakhotia, M. Harman, and P. McMinn. A multi-objective approach to search-based test data generation. In Proc. 9th Annual Conf. on Genetic and Evolutionary Computation (GECCO'07), pages 1098--1105, ACM, 2007.

Digital Library

[8]

N.L. Leech and A.J. Onwuegbuzie. A call for greater use of nonparametric statistics. Technical report, US Dept. Education, Educational Resources Information Center, 2002.

[9]

P. McMinn. Search-based software test data generation: a survey. Software Testing, Verification and Reliability, 14(2):105--156, 2004.

Digital Library

[10]

B.W. Murrill. Automated test data generation and reliability assessment for software in high assurance systems. In Proc. 10th IEEE High Assurance Systems Eng. Symp. (HASE'07), pages 409--410, IEEE Computer Society, 2007.

Digital Library

[11]

T.J. Parr. The Definitive ANTLR Reference: Building Domain0Specific Languages. The Pragmatic Programmers, 2007.

Digital Library

[12]

P. Tonella. Evolutionary testing of classes. In Proc. 2004 ACM SIGSOFT Int'l Symp. Software Testing and Analysis (ISSTA'04), pages 119--128. ACM, 2004.

Digital Library

[13]

N. Tracey, J. Clark, and K. Mander. Automated program flaw finding using simulated annealing. Software Eng. Notes, 23(2):73--81, 1998.

Digital Library

[14]

N. Tracey, J. Clark, and K. Mander. The way forward for unifying dynamic test-case generation: The optimisation-based approach. In Proc. IFIP Int'l Workshop Dependable Computing and Its Applications (DCIA), pages 169--180, 1998.

[15]

N.J. Tracey. A Search-Based Automated Test-Data Generation Framework for Safety-Critical Software. PhD thesis, Dept. Computer Science, University of York, 2000.

[16]

A. Vargha and H. Delaney. A critique and improvement of the CL common language effect size statistics of McGraw and Wong. J. Educational and Behavioral Statistics, 25(2):101--132, 2000.

[17]

J. Wegener, A. Baresel, and H. Sthamer. Evolutionary test environment for automatic structural testing. Information and Software Technology, 43:841--854, 2001.

[18]

F. Wilcoxon. Individual comparisons by ranking methods. Biometrics Bulletin, 1(6):80--83, 1945.

Cited By

Xu XZhu ZJiao LBosman P(2017)An adaptive fitness function based on branch hardness for search based testingProceedings of the Genetic and Evolutionary Computation Conference10.1145/3071178.3071184(1335-1342)Online publication date: 1-Jul-2017
https://dl.acm.org/doi/10.1145/3071178.3071184
Huuhtanen TItkonen JLassenius C(2015)Combining Algebraic and Domain Testing to Design Adequate Test Cases for Signal Processing Algorithms2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST)10.1109/ICST.2015.7102583(1-10)Online publication date: Apr-2015
https://doi.org/10.1109/ICST.2015.7102583
Arcuri A(2011)It really does matter how you normalize the branch distance in search‐based software testingSoftware Testing, Verification and Reliability10.1002/stvr.45723:2(119-147)Online publication date: 23-Mar-2011
https://doi.org/10.1002/stvr.457
Show More Cited By

Index Terms

Recommendations

Automated Software Testing for Matlab

Software testing can improve software quality. To test effectively, scientists and engineers should know how to write and run tests, define appropriate test cases, determine expected outputs, and correctly handle floating-point arithmetic. Using Matlab ...
Whole Test Suite Generation

Not all bugs lead to program crashes, and not always is there a formal specification to check the correctness of a software test's outcome. A common scenario in software testing is therefore that test data are generated, and a tester manually adds test ...
A Search-Based Approach for Cost-Effective Software Test Automation Decision Support and an Industrial Case Study
ICSTW '14: Proceedings of the 2014 IEEE International Conference on Software Testing, Verification, and Validation Workshops

Test automation is a widely-used approach to reduce the cost of manual software testing. However, if it is not planned or conducted properly, automated testing would not necessarily be more cost effective than manual testing. Deciding what parts of a ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

GECCO '09: Proceedings of the 11th Annual conference on Genetic and evolutionary computation

July 2009

2036 pages

ISBN:9781605583259

DOI:10.1145/1569901

General Chair:
Franz Rothlauf
University of Mainz, Germany

Copyright © 2009 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 July 2009

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

GECCO09

Sponsor:

GECCO09: Genetic and Evolutionary Computation Conference

July 8 - 12, 2009

Québec, Montreal, Canada

Acceptance Rates

Overall Acceptance Rate 1,669 of 4,410 submissions, 38%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
318
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 08 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Xu XZhu ZJiao LBosman P(2017)An adaptive fitness function based on branch hardness for search based testingProceedings of the Genetic and Evolutionary Computation Conference10.1145/3071178.3071184(1335-1342)Online publication date: 1-Jul-2017
https://dl.acm.org/doi/10.1145/3071178.3071184
Huuhtanen TItkonen JLassenius C(2015)Combining Algebraic and Domain Testing to Design Adequate Test Cases for Signal Processing Algorithms2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST)10.1109/ICST.2015.7102583(1-10)Online publication date: Apr-2015
https://doi.org/10.1109/ICST.2015.7102583
Arcuri A(2011)It really does matter how you normalize the branch distance in search‐based software testingSoftware Testing, Verification and Reliability10.1002/stvr.45723:2(119-147)Online publication date: 23-Mar-2011
https://doi.org/10.1002/stvr.457
Gay GMenzies TKoru G(2010)A baseline method for search-based software engineeringProceedings of the 6th International Conference on Predictive Models in Software Engineering10.1145/1868328.1868332(1-11)Online publication date: 12-Sep-2010
https://dl.acm.org/doi/10.1145/1868328.1868332

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten