short-paper

Automatic test image generation using procedural noise

Authors:

Matthew Patrick,

Matthew D. Castle,

Richard O. J. H. Stutt,

Christopher A. GilliganAuthors Info & Claims

ASE '16: Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering

Pages 654 - 659

https://doi.org/10.1145/2970276.2970333

Published: 25 August 2016 Publication History

Abstract

It is difficult to test programs that input images, due to the large number of (pixel) values that must be chosen and the complex ways these values interact. Typically, such programs are tested manually, using images that have known results. However, this is a laborious process and limited in the range of tests that can be applied. We introduce a new approach for testing programs that input images automatically, using procedural noise and spatial statistics to create inputs that are both realistic and can easily be tuned to have specific properties. The effectiveness of our approach is illustrated on an epidemiological simulation of a recently introduced tree pest in Great Britain: Oriental Chestnut Gall Wasp. Our approach produces images that match the real landscapes more closely than other techniques and can be used (alongside metamorphic relations) to detect smaller (artificially introduced) errors with greater accuracy.

References

[1]

S. Anand, E. Burke, T. Chen, J. Clark, M. Cohen, W. Grieskamp, M. Harman, M. J. Harrold, and P. McMinn. An Orchestrated Survey of Methodologies for Automated Software Test Case Generation. J. Systems Software, 86(8):1978–2001, 2013.

Digital Library

[2]

Andrew Leonard. Why facial recognition failed. http://www.salon.com/2013/04/22/why facial recognition failed/, 2013. Accessed: 2016-04-26.

[3]

M. Dustler, P. Bakic, H. Petersson, P. Timberg, A. Tingberg, and S. Zackrisson. Application of the Fractal Perlin Noise Algorithm for the Generation of Simulated Breast Tissue. SPIE Medical Imaging, 9412, 2015.

[4]

A. Eiben and J. Smith. Introduction to Evolutionary Computing. Springer, Heidelberg, Germany, 2nd edition, 2015.

[5]

Forestry Commission. Oriental Chestnut Gall Wasp. http://www.forestry.gov.uk/forestry/beeh-9xjbhf, 2016. Accessed: 2016-04-24.

[6]

G. Griffin, A. Holub, and P. Perona. Caltech-256 Object Category Data Set. Technical Report 7694, California Inst. of Technology, 2007.

[7]

R. Guderlei and J. Mayer. Towards Automatic Testing of Imaging Software by Means of Random and Metamorphic Testing. Int. J. Software Engineering Knowledge Engineering, 17(6):757–781, 2007.

[8]

R. Just and F. Schweiggert. Evaluating Testing Strategies for Imaging Software by Means of Mutation Analysis. In Proc. 2nd IEEE Int. Conf. Software Testing, Verification and Validation Works., pages 205–209, 2009.

Digital Library

[9]

J. Koljonen and J. T. Alander. Deformation Image Generation for Testing a Strain Measurement Algorithm. SPIE Machine Vision Pattern Recognition, 47(10), 2008.

[10]

T. J. Mantere and J. T. Alander. Automatic Image Generation by Genetic Algorithms for Testing Halftoning Methods. Intelligent Robots and Computer Vision, 4197, 2000.

[11]

D. Martin, C. Fowlkes, D. Tal, and J. Malik. A Database of Human Segmented Natural Images and its Application to Evaluating Segmentation Algorithms and Measuring Ecological Statistics. In Proc. 8th IEEE Int. Conf. Computer Vision, pages 416–423, 2001.

[12]

P. Moran. Notes on Continuous Stochastic Phenomena. Biometrika, 37(1):17–23, 1950.

[13]

K. Perlin. An Image Synthesizer. Computer Graphics, 19(3):287–296, 1985.

Digital Library

[14]

M. Petrou and C. Petrou. Image Processing: The Fundamentals. Wiley, Chichester, UK, 2010.

[15]

S. Segura, G. Fraser, A. B. Sánchez, and A. Ruiz-Cortés. A Survey on Metamorphic Testing. IEEE Tran. Software Engineering, 2016 {in press}.

[16]

J. Stilwell and G. Clarke. Applied GIS and Spatial Analysis. Wiley, Chichester, UK, 2004.

[17]

R. L. Streit. Poisson Point Processes: Imaging, Tracking, and Sensing. Springer, New York, NY, 2010.

[18]

R. Szeliski. Computer Vision: Algorithms and Applications. Springer, London, UK, 2010.

Cited By

Bajammal MStocco AMazinanian DMesbah A(2022)A Survey on the Use of Computer Vision to Improve Software Engineering TasksIEEE Transactions on Software Engineering10.1109/TSE.2020.303298648:5(1722-1742)Online publication date: 1-May-2022
https://doi.org/10.1109/TSE.2020.3032986
Temple PAcher MJezequel J(2021)Empirical Assessment of Multimorphic TestingIEEE Transactions on Software Engineering10.1109/TSE.2019.292697147:7(1511-1527)Online publication date: 1-Jul-2021
https://doi.org/10.1109/TSE.2019.2926971

Index Terms

Automatic test image generation using procedural noise
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis
        Software testing and debugging

Recommendations

Strong mutation-based test data generation using hill climbing
SBST '16: Proceedings of the 9th International Workshop on Search-Based Software Testing

Mutation Testing is an effective test criterion for finding faults and assessing the quality of a test suite. Every test criterion requires the generation of test cases, which turns to be a manual and difficult task. In literature, search-based ...
System testing for object-oriented systems with test case prioritization

This paper presents an approach to generate test cases from UML 2.0 sequence diagrams and subsequently prioritize those test cases using model information encapsulated in the sequence diagrams. The test cases generated according to the proposed approach ...
An Output-Oriented Approach of Test Data Generation Based on Genetic Algorithm
Proceedings of the ICA3PP International Workshops and Symposiums on Algorithms and Architectures for Parallel Processing - Volume 9532

Using genetic algorithm to transform test data generation problem into numerical optimization problem, evolution test is one of the hot topics in test data automatic generation. This paper proposed a software test data generation method based on ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ASE '16: Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering

August 2016

899 pages

ISBN:9781450338455

DOI:10.1145/2970276

General Chair:
David Lo
Singapore Management University, Singapore
,
Program Chairs:
Sven Apel
University of Passau, Germany
,
Sarfraz Khurshid
University of Texas at Austin, USA

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGAI: ACM Special Interest Group on Artificial Intelligence
SIGSOFT: ACM Special Interest Group on Software Engineering
IEEE-CS: Computer Society

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 August 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Funding Sources

Wellcome Trust

Conference

ASE'16

Sponsor:

SIGAI
SIGSOFT
IEEE-CS

ASE'16: ACM/IEEE International Conference on Automated Software Engineering

September 3 - 7, 2016

Singapore, Singapore

Acceptance Rates

Overall Acceptance Rate 82 of 337 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
158
Total Downloads

Downloads (Last 12 months)1
Downloads (Last 6 weeks)0

Reflects downloads up to 07 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Bajammal MStocco AMazinanian DMesbah A(2022)A Survey on the Use of Computer Vision to Improve Software Engineering TasksIEEE Transactions on Software Engineering10.1109/TSE.2020.303298648:5(1722-1742)Online publication date: 1-May-2022
https://doi.org/10.1109/TSE.2020.3032986
Temple PAcher MJezequel J(2021)Empirical Assessment of Multimorphic TestingIEEE Transactions on Software Engineering10.1109/TSE.2019.292697147:7(1511-1527)Online publication date: 1-Jul-2021
https://doi.org/10.1109/TSE.2019.2926971

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents