research-article

Using dynamic symbolic execution to generate inputs in search-based GUI testing

Authors:

Kevin Salvesen,

Juan P. Galeotti,

Andreas ZellerAuthors Info & Claims

SBST '15: Proceedings of the Eighth International Workshop on Search-Based Software Testing

Pages 32 - 35

Published: 16 May 2015 Publication History

Abstract

Search-based testing has been successfully applied to generate complex sequences of events for graphical user interfaces (GUIs), but typically relies on simple heuristics or random values for data widgets like text boxes. This may greatly reduce the effectiveness of test generation for applications which expect specific input values to be entered in their GUI by users. Generating such specific input values is one of the virtues of dynamic symbolic execution (DSE), but DSE is less suitable to generate sequences of events. Therefore, this paper describes a hybrid approach that uses search-based testing to generate sequences of events, and DSE to build input data for text boxes. This is achieved by replacing standard widgets in a system under test with symbolic ones, allowing us to execute GUIs symbolically. In this paper, we demonstrate an extension of the search-based GUI testing tool EXSYST, which uses DSE to successfully increase the obtained code coverage on two case study applications.

References

[1]

A. M. Memon, "An event-flow model of gui-based applications for testing: Research articles," Softw. Test. Verif. Reliab., vol. 17, no. 3, pp. 137--157, Sep. 2007. {Online}. Available: http://dx.doi.org/10.1002/stvr.v17:3

Digital Library

[2]

B. A. Myers, "User interface software tools," ACM Trans. Comput.-Hum. Interact., vol. 2, no. 1, pp. 64--103, Mar. 1995. {Online}. Available: http://doi.acm.org/10.1145/200968.200971

Digital Library

[3]

S. Anderson, "Gui testing," http://www.inf.ed.ac.uk/teaching/courses/st/2013-14/12_guitesting.pdf, 2013, {Online; accessed 28-November-2014}.

[4]

M. Grechanik, Q. Xie, and C. Fu, "Maintaining and evolving gui-directed test scripts," in Proceedings of the 31st International Conference on Software Engineering, ser. ICSE '09. Washington, DC, USA: IEEE Computer Society, 2009, pp. 408--418. {Online}. Available: http://dx.doi.org/10.1109/ICSE.2009.5070540

Digital Library

[5]

E. Dustin, J. Rashka, and J. Paul, Automated Software Testing: Introduction, Management, and Performance. Boston, MA, USA: Addison-Wesley Longman Publishing Co., Inc., 1999.

Digital Library

[6]

A. Tools, "Ui/application exerciser monkey," http://developer.android.com/tools/help/monkey.html, 2014, {Online; accessed 28-November-2014}.

[7]

A. Holmes and M. Kellogg, "Automating functional tests using selenium," in Agile Conference, 2006, July 2006, pp. 6 pp.--275.

Digital Library

[8]

B. Nguyen, B. Robbins, I. Banerjee, and A. Memon, "Guitar: an innovative tool for automated testing of gui-driven software," Automated Software Engineering, pp. 1--41, 2013. {Online}. Available: http://dx.doi.org/10.1007/s10515-013-0128-9

Digital Library

[9]

F. Gross, G. Fraser, and A. Zeller, "Search-based system testing: High coverage, no false alarms," in Proceedings of the 2012 International Symposium on Software Testing and Analysis, ser. ISSTA '12. New York, NY, USA: ACM, 2012, pp. 67--77.

Digital Library

[10]

G. Fraser and A. Arcuri, "Whole test suite generation," IEEE Transactions on Software Engineering, vol. 39, no. 2, pp. 276--291, 2013.

Digital Library

[11]

K. Sen, D. Marinov, and G. Agha, "Cute: a concolic unit testing engine for c," in Proceedings of the 10th European software engineering conference held jointly with 13th ACM SIGSOFT international symposium on Foundations of software engineering, ser. ESEC/FSE-13. New York, NY, USA: ACM, 2005, pp. 263--272. {Online}. Available: http://doi.acm.org/10.1145/1081706.1081750

Digital Library

[12]

P. Godefroid, N. Klarlund, and K. Sen, "Dart: directed automated random testing," in Proceedings of the 2005 ACM SIGPLAN conference on Programming language design and implementation, ser. PLDI '05. New York, NY, USA: ACM, 2005, pp. 213--223. {Online}. Available: http://doi.acm.org/10.1145/1065010.1065036

Digital Library

[13]

J. P. Galeotti, G. Fraser, and A. Arcuri, "Improving search-based test suite generation with dynamic symbolic execution," in 2013 IEEE 24th International Symposium on Software Reliability Engineering (ISSRE), 2013, pp. 360--369.

[14]

J. Galeotti, G. Fraser, and A. Arcuri, "Extending a search-based test generator with adaptive dynamic symbolic execution," in Proceedings of the 2014 International Symposium on Software Testing and Analysis, ser. ISSTA 2014. New York, NY, USA: ACM, 2014, pp. 421--424. {Online}. Available: http://doi.acm.org/10.1145/2610384.2628049

Digital Library

[15]

S. Ganov, C. Killmar, S. Khurshid, and D. E. Perry, "Event listener analysis and symbolic execution for testing gui applications," in Proceedings of the 11th International Conference on Formal Engineering Methods: Formal Methods and Software Engineering, ser. ICFEM '09. Berlin, Heidelberg: Springer-Verlag, 2009, pp. 69--87. {Online}. Available: http://dx.doi.org/10.1007/978-3-642-10373-5_4

Digital Library

[16]

S. Arlt, P. Borromeo, M. Schäf, and A. Podelski, "Parameterized gui tests," in ICTSS, 2012, pp. 247--262.

[17]

Oracle, "The java tutorials: The event dispatch thread," https://docs.oracle.com/javase/tutorial/uiswing/concurrency/dispatch.html, 2014, {Online; accessed 12-December-2014}.

[18]

C. S. Jensen, M. R. Prasad, and A. Møller, "Automated testing with targeted event sequence generation," in Proceedings of the 2013 International Symposium on Software Testing and Analysis, ser. ISSTA 2013. New York, NY, USA: ACM, 2013, pp. 67--77. {Online}. Available: http://doi.acm.org/10.1145/2483760.2483777

Digital Library

[19]

K. Inkumsah and T. Xie, "Improving structural testing of object-oriented programs via integrating evolutionary testing and symbolic execution," in ASE'08: Proc. of the 23rd IEEE/ACM Int. Conference on Automated Software Engineering, 2008, pp. 297--306.

Digital Library

[20]

A. Baars, M. Harman, Y. Hassoun, K. Lakhotia, P. McMinn, P. Tonella, and T. Vos, "Symbolic search-based testing," 2011.

[21]

J. Malburg and G. Fraser, "Combining search-based and constraint-based testing," in Proceedings of the 2011 26th IEEE/ACM International Conference on Automated Software Engineering, ser. ASE '11. Washington, DC, USA: IEEE Computer Society, 2011, pp. 436--439. {Online}. Available: http://dx.doi.org/10.1109/ASE.2011.6100092

Digital Library

[22]

M. Harman, Y. Jia, and W. B. Langdon, "Strong higher order mutation-based test data generation," in Symposium on the Foundations of Software Engineering (FSE), 2011, pp. 212--222.

Digital Library

Cited By

Cohen MGaleotti JPetke J(2017)The evolutionary landscape of SBSTProceedings of the 10th International Workshop on Search-Based Software Testing10.5555/3105427.3105437(47-48)Online publication date: 20-May-2017
https://dl.acm.org/doi/10.5555/3105427.3105437
Haraldsson SWoodward JBrownlee AGaleotti JPetke J(2017)The use of automatic test data generation for genetic improvement in a live systemProceedings of the 10th International Workshop on Search-Based Software Testing10.5555/3105427.3105433(28-31)Online publication date: 20-May-2017
https://dl.acm.org/doi/10.5555/3105427.3105433
Haraldsson SWoodward JBrownlee ASiggeirsdottir K(2017)Fixing bugs in your sleepProceedings of the Genetic and Evolutionary Computation Conference Companion10.1145/3067695.3082517(1513-1520)Online publication date: 15-Jul-2017
https://dl.acm.org/doi/10.1145/3067695.3082517

Index Terms

Using dynamic symbolic execution to generate inputs in search-based GUI testing

Recommendations

Using Dynamic Symbolic Execution to Generate Inputs in Search-Based GUI Testing
SBST '15: Proceedings of the 2015 IEEE/ACM 8th International Workshop on Search-Based Software Testing

Search-based testing has been successfully applied to generate complex sequences of events for graphical user interfaces (GUIs), but typically relies on simple heuristics or random values for data widgets like text boxes. This may greatly reduce the ...
Generating source inputs for metamorphic testing using dynamic symbolic execution
MET '16: Proceedings of the 1st International Workshop on Metamorphic Testing

Metamorphic testing uses domain-specific properties about a program's intended behaviour to alleviate the oracle problem. From a given set of source test inputs, a set of follow-up test inputs are generated which have some relation to the source inputs, ...
Event Listener Analysis and Symbolic Execution for Testing GUI Applications
ICFEM '09: Proceedings of the 11th International Conference on Formal Engineering Methods: Formal Methods and Software Engineering

Graphical User Interfaces (GUIs) are composed of virtual objects, widgets, which respond to events triggered by user actions. Therefore, test inputs for GUIs are event sequences that mimic user interaction. The nature of these sequences and the values ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SBST '15: Proceedings of the Eighth International Workshop on Search-Based Software Testing

May 2015

68 pages

Program Chairs:
Gregory Gay
University of Minnesota
,
Giuliano Antoniol
École Polytechnique de Montréal, Canada

Sponsors

ACM: Association for Computing Machinery
SIGSOFT: ACM Special Interest Group on Software Engineering
IEEE-CS\DATC: IEEE Computer Society
TCSE: IEEE Computer Society's Tech. Council on Software Engin.

Publisher

IEEE Press

Publication History

Published: 16 May 2015

Check for updates

Qualifiers

Research-article

Conference

ICSE '15

Sponsor:

ACM
SIGSOFT
IEEE-CS\DATC
TCSE

ICSE '15: 37th International Conference on Software Engineering

May 16 - 24, 2015

Florence, Italy

Upcoming Conference

ICSE 2025

2025 IEEE/ACM 46th International Conference on Software Engineering

April 26 - May 3, 2025

Ottawa , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
121
Total Downloads

Downloads (Last 12 months)1
Downloads (Last 6 weeks)0

Reflects downloads up to 31 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Cohen MGaleotti JPetke J(2017)The evolutionary landscape of SBSTProceedings of the 10th International Workshop on Search-Based Software Testing10.5555/3105427.3105437(47-48)Online publication date: 20-May-2017
https://dl.acm.org/doi/10.5555/3105427.3105437
Haraldsson SWoodward JBrownlee AGaleotti JPetke J(2017)The use of automatic test data generation for genetic improvement in a live systemProceedings of the 10th International Workshop on Search-Based Software Testing10.5555/3105427.3105433(28-31)Online publication date: 20-May-2017
https://dl.acm.org/doi/10.5555/3105427.3105433
Haraldsson SWoodward JBrownlee ASiggeirsdottir K(2017)Fixing bugs in your sleepProceedings of the Genetic and Evolutionary Computation Conference Companion10.1145/3067695.3082517(1513-1520)Online publication date: 15-Jul-2017
https://dl.acm.org/doi/10.1145/3067695.3082517

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten