research-article

Leveraging existing tests in automated test generation for web applications

Authors:

Amin Milani Fard,

Mehdi Mirzaaghaei,

Ali MesbahAuthors Info & Claims

ASE '14: Proceedings of the 29th ACM/IEEE International Conference on Automated Software Engineering

Pages 67 - 78

https://doi.org/10.1145/2642937.2642991

Published: 15 September 2014 Publication History

Abstract

To test web applications, developers currently write test cases in frameworks such as Selenium. On the other hand, most web test generation techniques rely on a crawler to explore the dynamic states of the application. The first approach requires much manual effort, but benefits from the domain knowledge of the developer writing the test cases. The second one is automated and systematic, but lacks the domain knowledge required to be as effective. We believe combining the two can be advantageous. In this paper, we propose to (1) mine the human knowledge present in the form of input values, event sequences, and assertions, in the human-written test suites, (2) combine that inferred knowledge with the power of automated crawling, and (3) extend the test suite for uncovered/unchecked portions of the web application under test. Our approach is implemented in a tool called Testilizer. An evaluation of our approach indicates that Testilizer (1) outperforms a random test generator, and (2) on average, can generate test suites with improvements of up to 150% in fault detection rate and up to 30% in code coverage, compared to the original test suite.

References

[1]

Claroline. http://www.claroline.net/.

[2]

JavaParser. https://code.google.com/p/javaparser/.

[3]

Jscover. http://tntim96.github.io/JSCover/.

[4]

Organizer. http://www.apress.com/9781590596951.

[5]

Phormer Photogallery. http://sourceforge.net/projects/rephormer/.

[6]

Selenium HQ. http://seleniumhq.org/.

[7]

Testilizer. http://salt.ece.ubc.ca/software/testilizer.

[8]

WolfCMS. https://github.com/wolfcms/wolfcms.

[9]

WSO2 EnterpriseStore. https://github.com/wso2/enterprise-store.

[10]

N. Alshahwan and M. Harman. State aware test case regeneration for improving web application test suite coverage and fault detection. In Proceedings of the International Symposium on Software Testing and Analysis (ISSTA), pages 45--55, 2012.

Digital Library

[11]

S. Artzi, J. Dolby, S. Jensen, A. Müller, and F. Tip. A framework for automated testing of JavaScript web applications. In Proceedings of the International Conference on Software Engineering (ICSE), pages 571--580. ACM, 2011.

Digital Library

[12]

C.-C. Chang and C.-J. Lin. LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2:27:1--27:27, 2011. Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm.

Digital Library

[13]

S. R. Choudhary, M. Prasad, and A. Orso. Crosscheck: Combining crawling and differencing to better detect cross-browser incompatibilities in web applications. In Proc. International Conference on Software Testing, Verification and Validation (ICST), pages 171--180. IEEE Computer Society, 2012.

Digital Library

[14]

V. Dallmeier, N. Knopp, C. Mallon, S. Hack, and A. Zeller. Generating test cases for specification mining. In Proceedings of the International Symposium on Software Testing and Analysis (ISSTA), pages 85--96, 2010.

Digital Library

[15]

S. Elbaum, G. Rothermel, S. Karre, and M. Fisher. Leveraging user-session data to support web application testing. IEEE Transactions on Software Engineering, 31(3):187--202, 2005.

Digital Library

[16]

L. Inozemtseva and R. Holmes. Coverage is not strongly correlated with test suite effectiveness. In Proceedings of the International Conference on Software Engineering (ICSE), 2014.

Digital Library

[17]

S. McAllister, E. Kirda, and C. Kruegel. Leveraging user interactions for in-depth testing of web applications. In Recent Advances in Intrusion Detection, volume 5230 of LNCS, pages 191--210. Springer, 2008.

Digital Library

[18]

A. Mesbah, A. van Deursen, and S. Lenselink. Crawling Ajax-based web applications through dynamic analysis of user interface state changes. ACM Transactions on the Web (TWEB), 6(1):3:1--3:30, 2012.

Digital Library

[19]

A. Mesbah, A. van Deursen, and D. Roest. Invariant-based automatic testing of modern web applications. IEEE Transactions on Softw. Eng., 38(1):35--53, 2012.

Digital Library

[20]

A. Milani Fard and A. Mesbah. Feedback-directed exploration of web applications to derive test models. In Proceedings of the International Symposium on Software Reliability Engineering (ISSRE), pages 278--287. IEEE Computer Society, 2013.

[21]

S. Mirshokraie, A. Mesbah, and K. Pattabiraman. Efficient JavaScript mutation testing. In Proc. of the International Conference on Software Testing, Verification and Validation (ICST). IEEE Computer Society, 2013.

Digital Library

[22]

M. Mirzaaghaei and A. Mesbah. DOM-based test adequacy criteria for web applications. In Proceedings of the International Symposium on Software Testing and Analysis (ISSTA), pages 71--81. ACM, 2014.

Digital Library

[23]

M. Mirzaaghaei, F. Pastore, and M. Pezze. Supporting test suite evolution through test case adaptation. In Proceedings of the International Conference on Software Testing, Verification and Validation (ICST), pages 231--240. IEEE Computer Society, 2012.

Digital Library

[24]

C. Pacheco, S. K. Lahiri, M. D. Ernst, and T. Ball. Feedback-directed random test generation. In Proc. International Conference on Software Engineering (ICSE), pages 75--84. IEEE Computer Society, 2007.

Digital Library

[25]

K. Pattabiraman and B. Zorn. DoDOM: Leveraging DOM invariants for web 2.0 application robustness testing. In Proceedings of the International Symposium on Sw. Reliability Eng. (ISSRE), pages 191--200. IEEE Computer Society, 2010.

Digital Library

[26]

M. Pezze, K. Rubinov, and J. Wuttke. Generating effective integration test cases from unit ones. In Proc. International Conference on Software Testing, Verification and Validation (ICST), pages 11--20. IEEE, 2013.

Digital Library

[27]

K. Rubinov and J. Wuttke. Augmenting test suites automatically. In Proceedings of the 2012 International Conference on Software Engineering, ICSE 2012, pages 1433--1434, Piscataway, NJ, USA, 2012. IEEE Press.

Digital Library

[28]

M. Schur, A. Roth, and A. Zeller. Mining behavior models from enterprise web applications. In Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering, Proceedings of the Foundations of Software Engineering (ESEC/FSE), pages 422--432. ACM, 2013.

Digital Library

[29]

R. Song, H. Liu, J.-R. Wen, and W.-Y. Ma. Learning important models for web page blocks based on layout and content analysis. ACM SIGKDD Explorations Newsletter, 6(2):14--23, 2004.

Digital Library

[30]

S. Sprenkle, E. Gibson, S. Sampath, and L. Pollock. Automated replay and failure detection for web applications. In Proceedings of the ACM/IEEE International Conference on Automated Software Engineering (ASE), pages 253--262. ACM, 2005.

Digital Library

[31]

S. Thummalapenta, K. V. Lakshmi, S. Sinha, N. Sinha, and S. Chandra. Guided test generation for web applications. In Proceedings of the International Conference on Software Engineering (ICSE), pages 162--171. IEEE Computer Society, 2013.

Digital Library

[32]

V. Vapnik. The nature of statistical learning theory. springer, 2000.

Digital Library

[33]

Y. Wang, S. Person, S. Elbaum, and M. B. Dwyer. A framework to advise tests using tests. In Proc. of ICSE NIER. ACM, 2014.

Digital Library

[34]

T. Xie and D. Notkin. Mutually enhancing test generation and specification inference. In Formal Approaches to Software Testing, pages 60--69. Springer, 2004.

[35]

D. Xu, W. Xu, B. K. Bavikati, and W. E. Wong. Mining executable specifications of web applications from selenium ide tests. In Software Security and Reliability (SERE), 2012 IEEE Sixth International Conference on, pages 263--272. IEEE, 2012.

Digital Library

[36]

Z. Xu, Y. Kim, M. Kim, G. Rothermel, and M. B. Cohen. Directed test suite augmentation: techniques and tradeoffs. In Proceedings of the International Symposium on Foundations of Software Engineering (FSE), pages 257--266. ACM, 2010.

Digital Library

[37]

S. Yoo and M. Harman. Test data regeneration: generating new test data from existing test data. Software Testing, Verification and Reliability, 22(3):171--201, 2012.

Digital Library

[38]

X. Yuan and A. M. Memon. Using GUI run-time state as feedback to generate test cases. In Proceedings of the 29th International Conference on Software Engineering, ICSE '07, pages 396--405, Washington, DC, USA, 2007. IEEE Computer Society.

Digital Library

[39]

X. Yuan and A. M. Memon. Iterative execution-feedback model-directed GUI testing. Information and Software Technology, 52(5):559--575, 2010.

Digital Library

Cited By

Stafeev APellegrino GBalzarotti DXu W(2024)SoKProceedings of the 33rd USENIX Conference on Security Symposium10.5555/3698900.3698941(719-737)Online publication date: 14-Aug-2024
https://dl.acm.org/doi/10.5555/3698900.3698941
Alnuhait HAlzyadat WAlthunibat AKahtan HZaqaibeh BAl-Khawaja H(2024)Web application performance assessment: A study of responsiveness, throughput, and scalabilityInternational Journal of ADVANCED AND APPLIED SCIENCES10.21833/ijaas.2024.09.02311:9(214-226)Online publication date: Sep-2024
https://doi.org/10.21833/ijaas.2024.09.023
Schäfer MNadi SEghbali ATip F(2024)An Empirical Evaluation of Using Large Language Models for Automated Unit Test GenerationIEEE Transactions on Software Engineering10.1109/TSE.2023.333495550:1(85-105)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TSE.2023.3334955
Show More Cited By

Index Terms

Leveraging existing tests in automated test generation for web applications
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis
        Software testing and debugging

Recommendations

Code Coverage Aware Test Generation Using Constraint Solver
Software Engineering and Formal Methods. SEFM 2020 Collocated Workshops
Abstract
Code coverage has been used in the software testing context mostly as a metric to assess a generated test suite’s quality. Recently, code coverage analysis is used as a white-box testing technique for test optimization. Most of the research ...
The Effectiveness of T-Way Test Data Generation
SAFECOMP '08: Proceedings of the 27th international conference on Computer Safety, Reliability, and Security

This paper reports the results of a study comparing the effectiveness of automatically generated tests constructed using random and <em>t</em>-way combinatorial techniques on safety related industrial code using mutation adequacy criteria. A reference ...
A detailed investigation of the effectiveness of whole test suite generation

A common application of search-based software testing is to generate test cases for all goals defined by a coverage criterion (e.g., lines, branches, mutants). Rather than generating one test case at a time for each of these goals individually, whole ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ASE '14: Proceedings of the 29th ACM/IEEE International Conference on Automated Software Engineering

September 2014

934 pages

ISBN:9781450330138

DOI:10.1145/2642937

General Chair:
Ivica Crnkovic
Mälardalen University, Sweden
,
Program Chairs:
Marsha Chechik
University of Toronto, Canada
,
Paul Grünbacher
Johannes Kepler Universität Linz, Austria

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGAI: ACM Special Interest Group on Artificial Intelligence
SIGSOFT: ACM Special Interest Group on Software Engineering
Mälardalen University: Mälardalen University
IEEE CS

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 September 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Conference

ASE '14

Sponsor:

SIGAI
SIGSOFT
Mälardalen University

ASE '14: ACM/IEEE International Conference on Automated Software Engineering

September 15 - 19, 2014

Vasteras, Sweden

Acceptance Rates

ASE '14 Paper Acceptance Rate 82 of 337 submissions, 24%;

Overall Acceptance Rate 82 of 337 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

51
Total Citations
View Citations
831
Total Downloads

Downloads (Last 12 months)56
Downloads (Last 6 weeks)4

Reflects downloads up to 09 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Stafeev APellegrino GBalzarotti DXu W(2024)SoKProceedings of the 33rd USENIX Conference on Security Symposium10.5555/3698900.3698941(719-737)Online publication date: 14-Aug-2024
https://dl.acm.org/doi/10.5555/3698900.3698941
Alnuhait HAlzyadat WAlthunibat AKahtan HZaqaibeh BAl-Khawaja H(2024)Web application performance assessment: A study of responsiveness, throughput, and scalabilityInternational Journal of ADVANCED AND APPLIED SCIENCES10.21833/ijaas.2024.09.02311:9(214-226)Online publication date: Sep-2024
https://doi.org/10.21833/ijaas.2024.09.023
Schäfer MNadi SEghbali ATip F(2024)An Empirical Evaluation of Using Large Language Models for Automated Unit Test GenerationIEEE Transactions on Software Engineering10.1109/TSE.2023.333495550:1(85-105)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TSE.2023.3334955
Jiang PWang SLiu Y(2024)Tree-Based Synthesis of Web Test Sequences from Manual ActionsTheoretical Aspects of Software Engineering10.1007/978-3-031-64626-3_14(242-260)Online publication date: 14-Jul-2024
https://doi.org/10.1007/978-3-031-64626-3_14
Nour BPourzandi MKamran Qureshi RDebbabi M(2024)Accurify: Automated New Testflows Generation for Attack Variants in Threat HuntingFoundations and Practice of Security10.1007/978-3-031-57540-2_5(52-69)Online publication date: 25-Apr-2024
https://doi.org/10.1007/978-3-031-57540-2_5
Mridha NMahbubul Alam Joarder (2023)Automated web testing over the last decade: A systematic literature reviewSystematic Literature Review and Meta-Analysis Journal10.54480/slr-m.v4i1.504:1(32-44)Online publication date: 16-Oct-2023
https://doi.org/10.54480/slr-m.v4i1.50
Ganji MAlimadadi STip FChandra SBlincoe KTonella P(2023)Code Coverage Criteria for Asynchronous ProgramsProceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3611643.3616292(1307-1319)Online publication date: 30-Nov-2023
https://dl.acm.org/doi/10.1145/3611643.3616292
Lin JSalehnamadi NMalek S(2023)Route: Roads Not Taken in UI TestingACM Transactions on Software Engineering and Methodology10.1145/357185132:3(1-25)Online publication date: 26-Apr-2023
https://dl.acm.org/doi/10.1145/3571851
Ghorbani NJabbarvand RSalehnamadi NGarcia JMalek S(2023)DeltaDroid: Dynamic Delivery Testing in AndroidACM Transactions on Software Engineering and Methodology10.1145/356321332:4(1-26)Online publication date: 26-May-2023
https://dl.acm.org/doi/10.1145/3563213
Zhou JJiang TWang HWu MChen T(2023)DAppHunter: Identifying Inconsistent Behaviors of Blockchain-based Decentralized Applications2023 IEEE/ACM 45th International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP)10.1109/ICSE-SEIP58684.2023.00008(24-35)Online publication date: May-2023
https://doi.org/10.1109/ICSE-SEIP58684.2023.00008
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents