research-article

An effective approach for regression test case selection using pareto based multi-objective harmony search

Authors:

Ankur Choudhary,

Arun Prakash Agrawal,

Arvinder KaurAuthors Info & Claims

SBST '18: Proceedings of the 11th International Workshop on Search-Based Software Testing

Pages 13 - 20

https://doi.org/10.1145/3194718.3194722

Published: 28 May 2018 Publication History

Abstract

Regression testing is a way of catching bugs in new builds and releases to avoid the product risks. Corrective, progressive, retest all and selective regression testing are strategies to perform regression testing. Retesting all existing test cases is one of the most reliable approaches but it is costly in terms of time and effort. This limitation opened a scope to optimize regression testing cost by selecting only a subset of test cases that can detect faults in optimal time and effort. This paper proposes Pareto based Multi-Objective Harmony Search approach for regression test case selection from an existing test suite to achieve some test adequacy criteria. Fault coverage, unique faults covered and algorithm execution time are utilised as performance measures to achieve optimization criteria. The performance evaluation of proposed approach is performed against Bat Search and Cuckoo Search optimization. The results of statistical tests indicate significant improvement over existing approaches.

References

[1]

David Binkley and Ieee Computer Society. 1997. Test Cost Reduction. 23, 8 (1997), 498--516.

[2]

Hyunsook Do, Sebastian Elbaum, and Gregg Rothermel. 2005. Supporting controlled experimentation with testing techniques: An infrastructure and its potential impact. Empir. Softw. Eng. 10, 4 (2005), 405--435.

Digital Library

[3]

Emelie Engström, Mats Skoglund, and Per Runeson. 2008. Empirical evaluations of regression test selection techniques: a systematic review. Int. Symp. Empir. Softw. Eng. Meas. (2008), 22--31.

Digital Library

[4]

Juan Pablo Galeotti, Gordon Fraser, and Andrea Arcuri. 2014. Extending a search-based test generator with adaptive dynamic symbolic execution. Proc. 2014 Int. Symp. Softw. Test. Anal. - ISSTA 2014 (2014), 421--424.

Digital Library

[5]

Todd L Graves, Jung-Min Kim, and Adam Porter. 2001. An Empirical Study of Regression Test Selection Techniques. ACM Trans. Softw. Eng. Methodol. 10, 2 (2001), 184--208.

Digital Library

[6]

Florian Gross, Gordon Fraser, and Andreas Zeller. 2012. Search-based system testing: high coverage, no false alarms. Proc. 2012 Int. Symp. Softw. Test. Anal. - ISSTA 2012 (2012), 67.

Digital Library

[7]

Mark Harman, Yue Jia, and Yuanyuan Zhang. 2015. Achievements, Open Problems and Challenges for Search Based Software Testing. Proc. 8th IEEE Int. Conf. Softw. Testing, Verif. Valid. Icst (2015), 1--12.

[8]

M.J. Harrold and M.L. Souffa. 1988. An incremental approach to unit testing during maintenance. Proceedings. Conf. Softw. Maintenance, 1988. (1988), 362--367.

[9]

Mary Jean Harrold, Ieee Computer Society, David Rosenblum, Senior Member, Gregg Rothermel, Ieee Computer Society, Elaine Weyuker, and Senior Member. 2001. Empirical Studies of a Prediction Model for Regression Test Selection. 27, 3 (2001), 248--263.

Digital Library

[10]

Rafaqut Kazmi, Dayang N. A. Jawawi, Radziah Mohamad, and Imran Ghani. 2017. Effective Regression Test Case Selection. ACM Comput. Surv. 50, 2 (2017), 1--32.

Digital Library

[11]

Andrea De Lucia, Massimiliano Di Penta, Rocco Oliveto, and Annibale Panichella. 2012. On the role of diversity measures for multi-objective test case selection. 2012 7th Int. Work. Autom. Softw. Test, AST 2012 - Proc. (2012), 145--151.

Digital Library

[12]

Camila Loiola Brito Maia, Rafael Augusto Ferreira Do Carmo, Fabrício Gomes De Freitas, Gustavo Augusto Lima De Campos, and Jerffeson Teixeira De Souza. 2009. A Multi-Objective Approach For The Regression Test Case Selection Problem. XLI Brazilian Symp. Oper. Res. XLI SBPO 2009. (2009), 1824--1835. Retrieved from http://sobrapo.org.br/simposios/XLI-2009/XLI_SBPO_2009_artigos/artigos/56096.pdf

[13]

Alessandro Marchetto and Mahfuzul Islam. 2013. A Multi-Objective Technique for Test Suite Reduction. ICSEA 2013, Eighth ... c (2013), 18--24. Retrieved from http://www.thinkmind.org/index.php?view=article&articleid=icsea_2013_1_30_10119

[14]

Phil McMinn. 2011. Search-Based Software Testing: Past, Present and Future. In 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops, 153--163.

Digital Library

[15]

Siavash Mirarab, Soroush Akhlaghi, and Ladan Tahvildari. 2012. Size-constrained regression test case selection using multicriteria optimization. IEEE Trans. Softw. Eng. 38, 4 (2012), 936--956.

Digital Library

[16]

Debajyoti Mondal, Hadi Hemmati, and Stephane Durocher. 2015. Exploring test suite diversification and code coverage in multi-objective test case selection. 2015 IEEE 8th Int. Conf. Softw. Testing, Verif. Validation, ICST 2015 - Proc. (2015).

[17]

Samaila Musa, Abu Bakar MD Sultan, Abd Azim Bin Abdul Ghani, and Salmi Baharom. 2014. Regression Test Case Selection & Prioritization Using Dependence Graph and Genetic Algorithm. IOSR J. Comput. Eng. IV, 3 (2014), 2278--661.

[18]

S Nachiyappan, A Vimaladevi, C B Selvalakshmi, and SelvaLakshmi CB. 2010. An evolutionary algorithm for regression test suite reduction. Proc. 2010 Int. Conf. Commun. Comput. Intell. (INCOCCI). (2010), 503--508.

[19]

Annibale Panichella, Rocco Oliveto, Massimiliano Di Penta, and Andrea De Lucia. 2015. Improving multi-objective test case selection by injecting diversity in genetic algorithms. IEEE Trans. Softw. Eng. 41, 4 (2015), 358--383.

Digital Library

[20]

J.S. Ramberg, S.M. Sanchez, P.J. Sanchez, and L.J. Hollick. Designing simulation experiments: Taguchi methods and response surface metamodels. In 1991 Winter Simulation Conference Proceedings., 167--176.

Digital Library

[21]

G. Rothermel and M.J. Harrold. 2012. A framework for evaluating regression test selection techniques. Proc. 16th Int. Conf. Softw. Eng. April (2012), 201--210.

Digital Library

[22]

Ruhul A Sarker, Saber M Elsayed, and Tapabrata Ray. 2014. Selection for Optimization Problems. Ieee Trans. Evol. Comput. 18, 5 (2014), 689--707.

[23]

Luciano S De Souza and Ricardo B C Prud. 2014. Multi-Objective Test Case Selection : A study of the influence of the Catfish effect on PSO based strategies. (2014), 3--58.

[24]

G Sun and A Zhang. 2013. A hybrid genetic algorithm and gravitational search algorithm for image segmentation using multilevel thresholding. Iber. Conf. Pattern Recognit. Image (2013). Retrieved April 29, 2017 from http://link.springer.com/chapter/10.1007/978-3-642-38628-2_84

[25]

Mattia Vivanti, Andre Mis, Alessandra Gorla, and Gordon Fraser. 2013. Search-based data-flow test generation. 2013 IEEE 24th Int. Symp. Softw. Reliab. Eng. ISSRE 2013 (2013), 370--379.

[26]

Shuai Wang, Shaukat Ali, and Arnaud Gotlieb. 2015. Cost-effective test suite minimization in product lines using search techniques. J. Syst. Softw. 103, C (2015), 370--391.

Digital Library

[27]

S Yoo and M Harman. 2007. Regression Testing Minimisation, Selection and Prioritisation: A Survey. Test. Verif. Reliab 0, (2007), 1--7.

Digital Library

[28]

Shin Yoo and Mark Harman. 2007. Pareto efficient multi-objective test case selection. Proc. 2007 Int. Symp. Softw. Test. Anal. - ISSTA '07 (2007), 140.

Digital Library

Cited By

Guan HLi DLi HZhao MWong W(2024)A Review of the Applications of Heuristic Algorithms in Test Case Generation Problem2024 IEEE 24th International Conference on Software Quality, Reliability, and Security Companion (QRS-C)10.1109/QRS-C63300.2024.00114(856-865)Online publication date: 1-Jul-2024
https://doi.org/10.1109/QRS-C63300.2024.00114
Habib AKhan SHussain SIbrahim NNisa HYousafzai A(2024)A similarity-based multi-objective test optimization technique using search algorithmSystems and Soft Computing10.1016/j.sasc.2024.2001646(200164)Online publication date: Dec-2024
https://doi.org/10.1016/j.sasc.2024.200164
Zarrad ABahsoon RManimaran P(2024)Optimizing regression testing with AHP-TOPSIS metric system for effective technical debt evaluationAutomated Software Engineering10.1007/s10515-024-00458-531:2Online publication date: 8-Jul-2024
https://doi.org/10.1007/s10515-024-00458-5
Show More Cited By

Recommendations

Multi-objective black-box test case selection for system testing
GECCO '17: Proceedings of the Genetic and Evolutionary Computation Conference

Testing is a fundamental task to ensure software quality. Regression testing aims to ensure that changes to software do not introduce new failures. As resources are often limited and testing comprises a vast amount of test cases, different regression ...
Pareto efficient multi-objective test case selection
ISSTA '07: Proceedings of the 2007 international symposium on Software testing and analysis

Previous work has treated test case selection as a single objective optimisation problem. This paper introduces the concept of Pareto efficiency to test case selection. The Pareto efficient approach takes multiple objectives such as code coverage, past ...
A Static Approach to Prioritizing JUnit Test Cases

Test case prioritization is used in regression testing to schedule the execution order of test cases so as to expose faults earlier in testing. Over the past few years, many test case prioritization techniques have been proposed in the literature. Most ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SBST '18: Proceedings of the 11th International Workshop on Search-Based Software Testing

May 2018

84 pages

ISBN:9781450357418

DOI:10.1145/3194718

Program Chairs:
Juan Pablo Galeotti
Universidad de Buenos Aires, Argentina
,
Alessandra Gorla
IMDEA Software, Spain

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGSOFT: ACM Special Interest Group on Software Engineering
IEEE-CS: Computer Society

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 May 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ICSE '18

Sponsor:

SIGSOFT
IEEE-CS

ICSE '18: 40th International Conference on Software Engineering

May 28 - 29, 2018

Gothenburg, Sweden

Upcoming Conference

ICSE 2025

2025 IEEE/ACM 46th International Conference on Software Engineering

April 26 - May 3, 2025

Ottawa , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

17
Total Citations
View Citations
191
Total Downloads

Downloads (Last 12 months)7
Downloads (Last 6 weeks)0

Reflects downloads up to 21 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Guan HLi DLi HZhao MWong W(2024)A Review of the Applications of Heuristic Algorithms in Test Case Generation Problem2024 IEEE 24th International Conference on Software Quality, Reliability, and Security Companion (QRS-C)10.1109/QRS-C63300.2024.00114(856-865)Online publication date: 1-Jul-2024
https://doi.org/10.1109/QRS-C63300.2024.00114
Habib AKhan SHussain SIbrahim NNisa HYousafzai A(2024)A similarity-based multi-objective test optimization technique using search algorithmSystems and Soft Computing10.1016/j.sasc.2024.2001646(200164)Online publication date: Dec-2024
https://doi.org/10.1016/j.sasc.2024.200164
Zarrad ABahsoon RManimaran P(2024)Optimizing regression testing with AHP-TOPSIS metric system for effective technical debt evaluationAutomated Software Engineering10.1007/s10515-024-00458-531:2Online publication date: 8-Jul-2024
https://doi.org/10.1007/s10515-024-00458-5
Arrieta AValle PAgirre JSagardui G(2023)Some Seeds Are Strong: Seeding Strategies for Search-based Test Case SelectionACM Transactions on Software Engineering and Methodology10.1145/353218232:1(1-47)Online publication date: 13-Feb-2023
https://dl.acm.org/doi/10.1145/3532182
Chaudhary SChoudhary ASeth J(2023)Nature Inspired Approaches for Test Case Selection in Regression Testing: A Review2023 13th International Conference on Cloud Computing, Data Science & Engineering (Confluence)10.1109/Confluence56041.2023.10048797(644-649)Online publication date: 19-Jan-2023
https://doi.org/10.1109/Confluence56041.2023.10048797
Habib AKhan SFelix E(2023)A systematic review on search‐based test suite reductionIET Software10.1049/sfw2.1210417:2(93-136)Online publication date: 20-Feb-2023
https://dl.acm.org/doi/10.1049/sfw2.12104
Pandey ABanerjee S(2022)Test Suite Optimization Using Firefly and Genetic AlgorithmResearch Anthology on Agile Software, Software Development, and Testing10.4018/978-1-6684-3702-5.ch077(1635-1651)Online publication date: 2022
https://doi.org/10.4018/978-1-6684-3702-5.ch077
Lawanna TWongwuttiwat JOunchaoen R(2022)Selection and Deletion Model under Discovering Valid and Invalid Test Case2022 6th International Conference on Information Technology, Information Systems and Electrical Engineering (ICITISEE)10.1109/ICITISEE57756.2022.10057897(1-6)Online publication date: 13-Dec-2022
https://doi.org/10.1109/ICITISEE57756.2022.10057897
Verma AChoudhary ATiwari S(2022)A novel chaotic archimedes optimization algorithm and its application for efficient selection of regression test casesInternational Journal of Information Technology10.1007/s41870-022-01031-715:2(1055-1068)Online publication date: 25-Jul-2022
https://doi.org/10.1007/s41870-022-01031-7
Verma AChoudhary ATiwari S(2021)Regression Test Case Selection: A Comparative Analysis of Metaheuristic AlgorithmsProceedings of Second Doctoral Symposium on Computational Intelligence10.1007/978-981-16-3346-1_49(605-615)Online publication date: 20-Sep-2021
https://doi.org/10.1007/978-981-16-3346-1_49
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents