A Gaming Quest to Improve Web Locators Robustness
Pages 10 - 17
Abstract
Web applications have become fundamental to our lives, thus testing methods ensuring their quality are essential. Usually, Web test automation frameworks and tools use locators to interact with the GUI, serving as hooks to the widgets within Web pages. However, locators are known to be one of the most significant points of fragility in Web testing, due to their susceptibility to rapid software evolution impacting the Web pages structure. While gamification strategies have been recently incorporated into the Web testing process to increase testers engagement, by means of tasks and rewards tailored around test activities, they have not yet been applied to locators robustness. In this paper, we introduce TestQuest, a tool under development that adapts gamification to improve locators robustness, by employing evaluation metrics and best practices designed to make testing via locators more engaging for testers, potentially enhancing the overall test quality.
References
[1]
Kartik Bajaj, Karthik Pattabiraman, and Ali Mesbah. 2015. LED: Tool for synthesizing Web element locators. In 30th IEEE/ACM International Conference on Automated Software Engineering (ASE). 848–851.
[2]
Jonathan Bell, Swapneel Sheth, and Gail Kaiser. 2011. Secret ninja testing with HALO software engineering. In Proceedings of the 4th International Workshop on Social Software Engineering (SSE). 43–47.
[3]
Matteo Biagiola, Filippo Ricca, and Paolo Tonella. 2017. Search based path and input data generation for Web application testing. In 9th International Symposium on Search-Based Software Engineering (SSBSE). 18–32.
[4]
Raquel Blanco, Manuel Trinidad, Maria Jose Suarez-Cabal, Alejandro Calderón, Mercedes Ruiz, and Javier Tuya. 2023. Can gamification help in software testing education? Findings from an empirical study. Journal of Systems and Software (JSS), 200 (2023), 111647.
[5]
Shaoheng Cao, Minxue Pan, Yu Pei, Wenhua Yang, Tian Zhang, Linzhang Wang, and Xuandong Li. 2024. Comprehensive Semantic Repair of Obsolete GUI Test Scripts for Mobile Applications. In Proceedings of the IEEE/ACM 46th International Conference on Software Engineering (ICSE). 1–13.
[6]
Y.K. Chou. 2015. Actionable Gamification: Beyond Points, Badges, and Leaderboards. Createspace Independent Publishing Platform. isbn:9781511744041
[7]
Shauvik Roy Choudhary, Dan Zhao, Husayn Versee, and Alessandro Orso. 2011. WATER: Web application test repair. In Proceedings of the First International Workshop on End-to-End Test Script Engineering (ETSE). 24–29.
[8]
Diego Clerissi, Giovanni Denaro, Marco Mobilio, and Leonardo Mariani. 2024. Guess the State: Exploiting Determinism to Improve GUI Exploration Efficiency. IEEE Transactions on Software Engineering (TSE), 1–18.
[9]
Riccardo Coppola, Tommaso Fulcini, Luca Ardito, Marco Torchiano, and Emil Alègroth. 2023. On Effectiveness and Efficiency of Gamified Exploratory GUI Testing. IEEE Transactions on Software Engineering (TSE).
[10]
Gabriela Martins de Jesus, Fabiano Cutigi Ferrari, Daniel de Paula Porto, and Sandra Camargo Pinto Ferraz Fabbri. 2018. Gamification in software testing: A characterization study. In Proceedings of the III Brazilian Symposium on Systematic and Automated Software Testing (SAST). 39–48.
[11]
Marco De Luca, Anna Rita Fasolino, and Porfirio Tramontana. 2024. Investigating the robustness of locators in template-based Web application testing using a GUI change classification model. Journal of Systems and Software (JSS), 210 (2024), 111932.
[12]
Sebastian Deterding, Miguel Sicart, Lennart Nacke, Kenton O’Hara, and Dan Dixon. 2011. Gamification: Using game-design elements in non-gaming contexts. In CHI’11 Extended Abstracts on Human Factors in Computing Systems. 2425–2428.
[13]
Sergio Di Meglio and Luigi Libero Lucio Starace. 2024. Towards Predicting Fragility in End-to-End Web Tests. In Proceedings of the 28th International Conference on Evaluation and Assessment in Software Engineering (EASE). 387–392.
[14]
Anna Rita Fasolino and Porfirio Tramontana. 2022. Towards the generation of robust E2E test cases in template-based Web applications. In 48th Euromicro Conf. on Software Engineering and Advanced Applications (SEAA). 104–111.
[15]
Martin Fowler. 2013. Page Object. https://martinfowler.com/bliki/PageObject.html
[16]
Gordon Fraser, Alessio Gambi, Marvin Kreis, and José Miguel Rojas. 2019. Gamifying a software testing course with code defenders. In Proceedings of the 50th ACM Technical Symposium on Computer Science Education (SIGCSE TS). 571–577.
[17]
Tommaso Fulcini, Riccardo Coppola, Luca Ardito, and Marco Torchiano. 2023. A review on tools, mechanics, benefits, and challenges of gamified software testing. Comput. Surveys, 55, 14s (2023), 1–37.
[18]
Tommaso Fulcini, Giacomo Garaccione, Riccardo Coppola, Luca Ardito, and Marco Torchiano. 2022. Guidelines for GUI testing maintenance: A linter for test smell detection. In Proceedings of the 13th International Workshop on Automating Test Case Design, Selection and Evaluation (A-TEST). 17–24.
[19]
Giacomo Garaccione, Tommaso Fulcini, Paolo Stefanut Bodnarescul, Riccardo Coppola, and Luca Ardito. 2024. Gamified GUI testing with Selenium in the IntelliJ IDE: A Prototype Plugin. ArXiv, abs/2403.09842 (2024), https://api.semanticscholar.org/CorpusID:268510502
[20]
Giacomo Garaccione, Tommaso Fulcini, and Marco Torchiano. 2022. GERRY: A gamified browser tool for GUI testing. In Proceedings of the 1st International Workshop on Gamification of Software Development, Verification, and Validation (GAMIFY). 2–9.
[21]
Mouna Hammoudi, Gregg Rothermel, and Andrea Stocco. 2016. WATERFALL: An incremental approach for repairing record-replay tests of Web applications. In Proceedings of the 24th International Symposium on Foundations of Software Engineering (FSE). IEEE, 751–762.
[22]
Mouna Hammoudi, Gregg Rothermel, and Paolo Tonella. 2016. Why do record/replay tests of Web applications break? In 2016 IEEE International Conference on Software Testing, Verification and Validation (ICST). 180–190.
[23]
Hiroyuki Kirinuki, Haruto Tanno, and Katsuyuki Natsukawa. 2019. COLOR: Correct Locator Recommender for Broken Test Scripts using Various Clues in Web Application. In Proceedings of the 26th International Conference on Software Analysis, Evolution and Reengineering (SANER). 310–320.
[24]
Maurizio Leotta, Matteo Biagiola, Filippo Ricca, Mariano Ceccato, and Paolo Tonella. 2020. A Family of Experiments to Assess the Impact of Page Object Pattern in Web Test Suite Development. In Proceedings of 13th IEEE International Conference on Software Testing, Verification and Validation (ICST). IEEE, 263–273. https://doi.org/10.1109/ICST46399.2020.00035
[25]
Maurizio Leotta, Diego Clerissi, Filippo Ricca, and Cristiano Spadaro. 2013. Comparing the Maintainability of Selenium WebDriver Test Suites Employing Different Locators: A Case Study. In Proceedings of 1st International Workshop on Joining AcadeMiA and Industry Contributions to testing Automation (JAMAICA). ACM, 53–58. isbn:978-1-4503-2161-7 https://doi.org/10.1145/2489280.2489284
[26]
Maurizio Leotta, Diego Clerissi, Filippo Ricca, and Paolo Tonella. 2013. Capture-Replay vs. Programmable Web Testing: An Empirical Assessment during Test Case Evolution. In Proceedings of 20th Working Conference on Reverse Engineering (WCRE). IEEE, 272–281. isbn:978-1-4799-2931-3 https://doi.org/10.1109/WCRE.2013.6671302
[27]
Maurizio Leotta, Antonio Molinari, and Filippo Ricca. 2022. ASSESSOR: a PO-Based WebDriver Test Suites Generator from Selenium IDE Recordings. In Proceedings of 15th International Conference on Software Testing, Verification and Validation (ICST). IEEE, 389–399. https://doi.org/10.1109/ICST53961.2022.00045
[28]
Maurizio Leotta, Filippo Ricca, and Paolo Tonella. 2021. SIDEREAL: Statistical Adaptive Generation of Robust Locators for Web Testing. Journal of Software: Testing, Verification and Reliability (STVR), 31 (2021), issn:1099-1689 https://doi.org/10.1002/stvr.1767
[29]
Maurizio Leotta, Andrea Stocco, Filippo Ricca, and Paolo Tonella. 2015. Using Multi-Locators to Increase the Robustness of Web Test Cases. In Proceedings of 8th IEEE International Conference on Software Testing, Verification and Validation (ICST). IEEE, 1–10. isbn:978-1-4799-7125-1 https://doi.org/10.1109/ICST.2015.7102611
[30]
Maurizio Leotta, Andrea Stocco, Filippo Ricca, and Paolo Tonella. 2016. ROBULA+: An Algorithm for Generating Robust XPath Locators for Web Testing. Journal of Software: Evolution and Process (JSEP), 28, 3 (2016), 177–204. issn:2047-7481 https://doi.org/10.1002/smr.1771
[31]
Olivier Liechti, Jacques Pasquier, and Rodney Reis. 2017. Supporting agile teams with a test analytics platform: A case study. In 2017 IEEE/ACM 12th International Workshop on Automation of Software Testing (AST). 9–15.
[32]
Michel Nass, Emil Alégroth, Robert Feldt, Maurizio Leotta, and Filippo Ricca. 2023. Similarity-based Web Element Localization for Robust Test Automation. ACM Transactions on Software Engineering and Methodology (TOSEM), 32, 3 (2023), Article 75, 30 pages. issn:1049-331X https://doi.org/10.1145/3571855
[33]
Vu Nguyen, Thanh To, and Gia-Han Diep. 2021. Generating and selecting resilient and maintainable locators for Web automated testing. Software Testing, Verification and Reliability (STVR), 31, 3 (2021), e1760.
[34]
Page Object. 2023. Page Object Models. https://www.selenium.dev/documentation/test_practices/encouraged/page_object_models/
[35]
Oscar Pedreira, Félix García, Nieves Brisaboa, and Mario Piattini. 2015. Gamification in software engineering - A systematic mapping. Information and Software Technology, 57 (2015), 157–168.
[36]
Wei Ren. 2023. Gamification in test-driven development practice. In Proceedings of the 2nd International Workshop on Gamification in Software Development, Verification, and Validation (GAMIFY). 38–46.
[37]
Andrea Stocco, Maurizio Leotta, Filippo Ricca, and Paolo Tonella. 2017. APOGEN: Automatic Page Object Generator for Web Testing. Software Quality Journal (SQJ), 25, 3 (2017), 1007–1039. issn:0963-9314 https://doi.org/10.1007/s11219-016-9331-9
[38]
Andrea Stocco, Rahulkrishna Yandrapally, and Ali Mesbah. 2018. Visual Web test repair. In Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE). 503–514.
[39]
Philipp Straubinger and Gordon Fraser. 2024. Improving Testing Behavior by Gamifying IntelliJ. In Proceedings of the 46th IEEE/ACM International Conference on Software Engineering (ICSE). 1–13.
[40]
Arie Van Deursen. 2015. Testing Web applications with state objects. Commun. ACM, 58, 8 (2015), 36–43.
[41]
Selenium WebDriver. 2024. Selenium WebDriver Documentation. https://www.selenium.dev/documentation/webdriver
[42]
Bing Yu, Lei Ma, and Cheng Zhang. 2015. Incremental Web application testing using Page Object. In 2015 Third IEEE Workshop on Hot Topics in Web Systems and Technologies (HotWeb). 1–6.
Index Terms
- A Gaming Quest to Improve Web Locators Robustness
Recommendations
Towards Predicting Fragility in End-to-End Web Tests
EASE '24: Proceedings of the 28th International Conference on Evaluation and Assessment in Software EngineeringAutomated end-to-end web tests are typically implemented as scripts that leverage dedicated libraries to simulate user interactions with web pages in a remotely controlled web browser. These tests are crucial for ensuring the functionality and ...
Diversity-based web test generation
ESEC/FSE 2019: Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software EngineeringExisting web test generators derive test paths from a navigational model of the web application, completed with either manually or randomly generated input values. However, manual test data selection is costly, while random generation often results in ...
Reducing Web Test Cases Aging by Means of Robust XPath Locators
ISSREW '14: Proceedings of the 2014 IEEE International Symposium on Software Reliability Engineering WorkshopsIn the context of web regression testing, the main aging factor for a test suite is related to the continuous evolution of the underlying web application that makes the test cases broken. This rapid decay forces the quality experts to evolve the test ...
Comments
Information & Contributors
Information
Published In
September 2024
37 pages
ISBN:9798400711138
DOI:10.1145/3678869
- General Chairs:
- Riccardo Coppola,
- Luca Ardito,
- Gordon Fraser,
- Maurizio Leotta
Copyright © 2024 Copyright held by the owner/author(s).
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s).
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 13 September 2024
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- European Union - Next Generation EU within the PRIN 2022 program
Conference
Gamify '24
Sponsor:
Gamify '24: 3rd ACM International Workshop on Gamification in Software Development, Verification, and Validation
September 17, 2024
Vienna, Austria
Upcoming Conference
ISSTA '25
- Sponsor:
- sigsoft
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 141Total Downloads
- Downloads (Last 12 months)141
- Downloads (Last 6 weeks)36
Reflects downloads up to 08 Feb 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in