survey

Effective Regression Test Case Selection: A Systematic Literature Review

Authors:

Dayang N. A. Jawawi,

Radziah Mohamad,

Imran GhaniAuthors Info & Claims

ACM Computing Surveys (CSUR), Volume 50, Issue 2

Article No.: 29, Pages 1 - 32

https://doi.org/10.1145/3057269

Published: 25 May 2017 Publication History

Abstract

Regression test case selection techniques attempt to increase the testing effectiveness based on the measurement capabilities, such as cost, coverage, and fault detection. This systematic literature review presents state-of-the-art research in effective regression test case selection techniques. We examined 47 empirical studies published between 2007 and 2015. The selected studies are categorized according to the selection procedure, empirical study design, and adequacy criteria with respect to their effectiveness measurement capability and methods used to measure the validity of these results.

The results showed that mining and learning-based regression test case selection was reported in 39% of the studies, unit level testing was reported in 18% of the studies, and object-oriented environment (Java) was used in 26% of the studies. Structural faults, the most common target, was used in 55% of the studies. Overall, only 39% of the studies conducted followed experimental guidelines and are reproducible.

There are 7 different cost measures, 13 different coverage types, and 5 fault-detection metrics reported in these studies. It is also observed that 70% of the studies being analyzed used cost as the effectiveness measure compared to 31% that used fault-detection capability and 16% that used coverage.

References

[1]

J. Anderson, S. Salem, and H. Do. 2014. Improving the effectiveness of test suite through mining historical data. In Proceedings of the 11th Working Conference on Mining Software Repositories. ACM, 142--151.

Digital Library

[2]

J. H. Andrews, L. C. Briand, Y. Labiche, and A. S. Namin. 2006. Using mutation analysis for assessing and comparing testing coverage criteria. IEEE Trans. Software Eng. 32, 608--624.

Digital Library

[3]

M. A. Askarunisa, M. L. Shanmugapriya, and D. N. Ramaraj. 2010. Cost and coverage metrics for measuring the effectiveness of test case prioritization techniques. INFOCOMP J. Comput. Sci. 9, 43--52.

[4]

A. Assis Lobo De Oliveira, C. Gonyalves Camilo-Junior, and A. M. Vincenzi. 2013. A coevolutionary algorithm to automatic test case selection and mutant in mutation testing. In Proceedings of the IEEE Congress on Evolutionary Computation (CEC’13). IEEE, 829--836.

[5]

B. Beizer. 1995. Black-Box Testing: Techniques for Functional Testing of Software and Systems, John Wiley 8 Sons, Inc.

Digital Library

[6]

D. Binkley. 1995. Reducing the cost of regression testing by semantics guided test case selection. In Proceedings of the International Conference on Software Maintenance. IEEE, 251--260.

Digital Library

[7]

S. Biswas, R. Mall, M. Satpathy, and S. Sukumaran. 2011a. Regression test selection techniques: A survey. Informat.: Int. J. Comput. Informat. 35, 289--321.

[8]

S. Biswas, R. Mall, M. Satpathy, and S. Sukumaran. 2011b. Regression test selection techniques: A survey. Informatica 35.

[9]

P. Brereton, B. A. Kitchenham, D. Budgen, M. Turner, and M. Khalil. 2007. Lessons from applying the systematic literature review process within the software engineering domain. J. Syst. Software 80, 571--583.

Digital Library

[10]

X. Cai and M. R. Lyu. 2005. The effect of code coverage on fault detection under different testing profiles. ACM SIGSOFT Software Eng. Notes 30, 1--7.

Digital Library

[11]

E. G. Cartaxo, P. D. Machado, and F. G. O. Neto. 2011. On the use of a similarity function for test case selection in the context of model-based testing. Software Test. Verificat. Reliabil. 21, 75--100.

Digital Library

[12]

S. Chen, Z. Chen, Z. Zhao, B. Xu, and Y. Feng. 2011a. Using semi-supervised clustering to improve regression test selection techniques. In Proceedings of the IEEE 4th International Conference on Software Testing, Verification and Validation (ICST’11). IEEE, 1--10.

Digital Library

[13]

Z. Chen, Y. Duan, Z. Zhao, B. Xu, and J. Qian. 2011b. Using program slicing to improve the efficiency and effectiveness of cluster test selection. Int. J. Software Eng. Knowl. Eng. 21, 759--777.

[14]

P. K. Chittimalli and M. J. Harrold. 2009. Recomputing coverage information to assist regression testing. IEEE Trans. Software Eng. 35, 452--469.

Digital Library

[15]

H. Cibulski and A. Yehudai. 2011. Regression test selection techniques for test-driven development. In Proceedings of the 4th International Conference on Software Testing, Verification and Validation Workshops (ICSTW’11). IEEE, 115--124.

Digital Library

[16]

C. A. C. Coello, D. A. Van Veldhuizen, and G. B. Lamont. 2002. Evolutionary Algorithms for Solving Multi-Objective Problems. Springer.

Digital Library

[17]

L. S. De Souza, P. B. De Miranda, R. B. Prudencio, and F. De Barros. 2011. A multi-objective particle swarm optimization for test case selection based on functional requirements coverage and execution effort. In Proceedings of the23rd IEEE International Conference on Tools with Artificial Intelligence (ICTAI’11). IEEE, 245--252.

Digital Library

[18]

L. S. De Souza, R. B. Prudencio, and D. A. Barros. 2014a. A hybrid binary multi-objective particle swarm optimization with local search for test case selection. In Proceedings of the 2014 Brazilian Conference on Intelligent Systems (BRACIS’14), 2014a. IEEE, 414--419.

Digital Library

[19]

L. S. De Souza, R. B. Prudencio, and F. D. Barros. 2014b. A comparison study of binary multi-objective particle swarm optimization approaches for test case selection. In Proceedings of the 2014 IEEE Congress on Evolutionary Computation (CEC’14). IEEE, 2164--2171.

[20]

L. S. De Souza, R. B. Prudêncio, F. D. A. Barros, and E. H. D. S. Aranha. 2013. Search based constrained test case selection using execution effort. Expert Syst. Appl. 40, 4887--4896.

Digital Library

[21]

K. Deb. 2001. Multi-Objective Optimization Using Evolutionary Algorithms, John Wiley 8 Sons.

Digital Library

[22]

M. E. Delamaro and J. Offutt. 2014. Assessing the influence of multiple test case selection on mutation experiments. In Proceedings of the 2014 IEEE 7th International Conference on Software Testing, Verification and Validation Workshops (ICSTW’14). IEEE, 171--175.

Digital Library

[23]

H. Do and G. Rothermel. 2006. An empirical study of regression testing techniques incorporating context and lifetime factors and improved cost-benefit models. In Proceedings of the 14th ACM SIGSOFT International Symposium on Foundations of Software Engineering. ACM, 141--151.

Digital Library

[24]

T. Dyba, B. A. Kitchenham, and M. Jorgensen. 2005. Evidence-based software engineering for practitioners. IEEE Software, 22, 58--65.

Digital Library

[25]

I. Eee. 1990. Standard G lossary of softwareengineering terminology. IEEE Software Eng. Stand. Cll ect. IEEE, 610.12--190.

[26]

W. S. A. El-Hamid, S. S. El-Etriby, and M. M. Hadhoud. 2010. Regression test selection technique for multi-programming language. In Proceedings of the 7th International Conference on Informatics and Systems (INFOS’10). IEEE, 1--5.

[27]

S. Elbaum, P. Kallakuri, A. Malishevsky, G. Rothermel, and S. Kanduri. 2003. Understanding the effects of changes on the cost-effectiveness of regression testing techniques. Software Test. Verificat. Reliabil. 13, 65--83.

[28]

S. Elbaum, A. Malishevsky, and G. Rothermel. 2001. Incorporating varying test costs and fault severities into test case prioritization. In Proceedings of the 23rd International Conference on Software Engineering. IEEE Computer Society, 329--338.

Digital Library

[29]

S. Elbaum, A. G. Malishevsky, and G. Rothermel. 2002. Test case prioritization: A family of empirical studies. IEEE Trans. Software Eng. 28, 159--182.

Digital Library

[30]

E. Engström, P. Runeson, and M. Skoglund. 2010. A systematic review on regression test selection techniques. Informat. Software Technol. 52, 14--30.

Digital Library

[31]

E. Engström, M. Skoglund, and P. Runeson. 2008. Empirical evaluations of regression test selection techniques: A systematic review. In Proceedings of the 2nd ACM-IEEE International Symposium on Empirical Software Engineering and Measurement. ACM, 22--31.

Digital Library

[32]

K. F. Fischer. 1977. A test case selection method for the validation of software maintenance modifications. Proceedings of COMPSAC, 1977. 421--426.

[33]

E. Fourneret, J. Cantenot, F. Bouquet, B. Legeard, and J. Botella. 2014. Setgam: Generalized technique for regression testing based on UML/OCL models. In Proceedings of the 8th International Conference on Software Security and Reliability (SERE’14), 2014. IEEE, 147--156.

Digital Library

[34]

M. Gligoric, L. Eloussi, and D. Marinov. 2015. Practical regression test selection with dynamic file dependencies. In Proceedings of the 2015 International Symposium on Software Testing and Analysis. ACM, 211--222.

Digital Library

[35]

M. Gligoric, A. Groce, C. Zhang, R. Sharma, M. A. Alipour, and D. Marinov. 2014. Guidelines for coverage-based comparisons of non-adequate test suites. Space 6, 1, 142.

[36]

J. E. González, N. Juristo, and S. Vegas. 2014. A systematic mapping study on testing technique experiments: has the situation changed since 2000? In Proceedings of the 8th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, 2014. ACM, 3.

Digital Library

[37]

T. L. Graves, M. J. Harrold, J. M. Kim, A. Porter, and G. Rothermel. 2001. An empirical study of regression test selection techniques. ACM Trans. Software Eng. Methodol (TOSEM) 10, 184--208.

Digital Library

[38]

F. Haftmann, D. Kossmann, and E. Lo. 2007. A framework for efficient regression tests on database applications. VLDB J.: Int. J. Very Large Data Bases 16, 145--164.

Digital Library

[39]

M. Harman and N. Alshahwan. 2008. Automated session data repair for web application regression testing. In Proceedings of the 1st International Conference on Software Testing, Verification, and Validation. IEEE, 298--307.

Digital Library

[40]

H. Hemmati, A. Arcuri, and L. Briand. 2010a. Reducing the cost of model-based testing through test case diversity. In Testing Software and Systems. Springer.

Digital Library

[41]

H. Hemmati, A. Arcuri, and L. Briand. 2011. Empirical investigation of the effects of test suite properties on similarity-based test case selection. In Proceedings of the IEEE 4th International Conference on Software Testing, Verification and Validation (ICST’11). IEEE, 327--336.

Digital Library

[42]

H. Hemmati and L. Briand. 2010. An industrial investigation of similarity measures for model-based test case selection. In Proceedings of the IEEE 21st International Symposium on Software Reliability Engineering (ISSRE’10) IEEE, 141--150.

Digital Library

[43]

H. Hemmati, L. Briand, A. Arcuri, and S. Ali. 2010b. An enhanced test case selection approach for model-based testing: An industrial case study. In Proceedings of the 18th ACM SIGSOFT International Symposium on Foundations of Software Engineering. ACM, 267--276.

Digital Library

[44]

K. Hla, Y. Choi, and J. S. Park. 2008. Applying particle swarm optimization to prioritizing test cases for embedded real time software retesting. In Proceedings of the IEEE 8th International Conference on Computer and Information Technology Workshops. IEEE, 527--532.

Digital Library

[45]

S. Huang, Y. Chen, J. Zhu, Z. J. Li, and H. F. Tan. 2009. An optimized change-driven regression testing selection strategy for binary Java applications. In Proceedings of the 2009 ACM Symposium on Applied Computing. ACM, 558--565.

Digital Library

[46]

S. Huang, Z. J. Li, J. Zhu, Y. Xiao, and W. Wang. 2011. A novel approach to regression test selection for J2EE applications. In Proceedings of the 27th IEEE International Conference on Software Maintenance (ICSM’11). IEEE, 13--22.

Digital Library

[47]

IEEE-STD-610. 12-1990. 1990. IEEE standard glossary of software engineering terminology (IEEE Std 610.12-1990). IEEE Computer Society, Los Alamitos. CA.

[48]

S. A. Infrastructure. 2016. SIR. Retrieved from http://sir.unl.edu/portal/index.php.

[49]

L. Inozemtseva and R. Holmes. 2014. Coverage is not strongly correlated with test suite effectiveness. In Proceedings of the 36th International Conference on Software Engineering. ACM, 435--445.

Digital Library

[50]

M. Z. Z. Iqbal, Z. Malik, and M. Riebisch. 2010. A model-based regression testing approach for evolving software systems with flexible tool support. In Proceedings of the 17th IEEE International Conference and Workshops on Engineering of Computer Based Systems (ECBS’10). IEEE, 41--49.

Digital Library

[51]

L. Jiang and Z. Su. 2007. Context-aware statistical debugging: from bug predictors to faulty control flow paths. In Proceedings of the 22nd IEEE/ACM International Conference on Automated Software Engineering. ACM, 184--193.

Digital Library

[52]

D. S. Johnson. 2002. A theoretician's guide to the experimental analysis of algorithms. Data Structures, Near Neighbor Searches, and Methodology: Fifth and Sixth DIMACS Implementation Challenges, 59, 215--250.

[53]

B. Kitchenham. 2004. Procedures for performing systematic reviews. Keele University, Keele, UK. 33, 1--26.

[54]

B. A. Kitchenham, T. Dyba, and M. Jorgensen. 2004. Evidence-based software engineering. In Proceedings of the 26th International Conference on Software Engineering. IEEE Computer Society, 273--281.

Digital Library

[55]

B. A. Kitchenham, S. L. Pfleeger, L. M. Pickard, P. W. Jones, D. C. Hoaglin, K. El Emam, and J. Rosenberg. 2002. Preliminary guidelines for empirical research in software engineering. IEEE Trans. Software Eng. 28, 721--734.

Digital Library

[56]

M. Kumar, A. Sharma, and R. Kumar. 2013. Fuzzy entropy-based framework for multi-faceted test case classification and selection: An empirical study. IET Software 8, 103--112.

[57]

M. Kumar, A. Sharma, and R. Kumar. 2015. An empirical evaluation of a three-tier conduit framework for multifaceted test case classification and selection using fuzzy-ant colony optimisation approach. Software: Pract. Exp. 45, 949--971.

Digital Library

[58]

D. C. Kung, C. H. Liu, and P. Hsia. 2000. An object-oriented web test model for testing web applications. In Proceedings of the 1st Asia-Pacific Conference on Quality Software. IEEE, 111--120.

Digital Library

[59]

H. K. Leung and L. White. 1989. Insights into regression testing [software testing]. In Proceedings of the 1989 Conference on Software Maintenance. IEEE, 60--69.

[60]

H. K. Leung and L. White. 1991. A cost model to compare regression test strategies. In Proceedings of the 1991 Conference on Software Maintenance. IEEE, 201--208.

[61]

W. E. Lewis. 2008. Software Testing and Continuous Quality Improvement, CRC Press, Boca Raton, FL.

Digital Library

[62]

W. E. Lewis. 2016. Software Testing and Continuous Quality Improvement, CRC Press, Boca Raton, FL.

Digital Library

[63]

B. Li, D. Qiu, H. Leung, and D. Wang. 2012. Automatic test case selection for regression testing of composite service based on extensible BPEL flow graph. J. Syst. Software 85, 1300--1324.

Digital Library

[64]

Y. D. Lin, C. H. Chou, Y. C. Lai, T. Y. Huang, S. Chung, J. T. Hung, and F. C. Lin. 2012. Test coverage optimization for large code problems. J. Syst. Software 85, 16--27.

Digital Library

[65]

N. Mansour, H. Takkoush, and A. Nehme. 2011. UML-based regression testing for OO software. J. Software Maint. Evol.: Res. Pract. 23, 51--68.

Digital Library

[66]

A. Memon, A. Nagarajan, and Q. Xie. 2005. Automating regression testing for evolving GUI software. J. Software Maint. Evol.: Res. Pract. 17, 27--64.

Digital Library

[67]

A. M. Memon. 2008. Automatically repairing event sequence-based GUI test suites for regression testing. ACM Transactions on Software Engineering and Methodology (TOSEM) 18, 4.

Digital Library

[68]

A. M. Memon and M. L. Soffa. 2003. Regression testing of GUIs. ACM SIGSOFT Software Engineering Notes 28, 118--127.

Digital Library

[69]

S. Mirarab, S. Akhlaghi, and L. Tahvildari. 2012a. Size-constrained regression test case selection using multicriteria optimization. IEEE Trans. Software Eng. 38, 936--956.

Digital Library

[70]

S. Mirarab, S. Akhlaghi, and L. Tahvildari. 2012b. Size-constrained regression test case selection using multicriteria optimization. IEEE Trans. Software Eng. 38, 936--956.

Digital Library

[71]

J. D. Musa. 1993. Operational profiles in software-reliability engineering. IEEE Software 10, 14--32.

Digital Library

[72]

R. Nagar, A. Kumar, S. Kumar, and A. S. Baghel. 2014. Implementing test case selection and reduction techniques using meta-heuristics. In Proceedings of the 5th International Conference on Confluence The Next Generation Information Technology Summit (Confluence’14). IEEE, 837--842.

[73]

A. S. Namin and J. H. Andrews. 2009. The influence of size and coverage on test suite effectiveness. Proceedings of the 18th International Symposium on Software Testing and Analysis. ACM, 57--68.

Digital Library

[74]

A. Nanda, S. Mani, S. Sinha, M. J. Harrold, and A. Orso. 2011. Regression testing in the presence of non-code changes. In Proceedings of the IEEE 4th International Conference on Software Testing, Verification and Validation (ICST’11). IEEE, 21--30.

Digital Library

[75]

D. D. Nardo, N. Alshahwan, L. Briand, and Y. Labiche. 2015. Coverage-based regression test case selection, minimization and prioritization: A case study on an industrial system. Software Test. Verificat. Reliabil. 25, 371--396.

Digital Library

[76]

A. Orso and G. Rothermel. 2014. Software testing: A research travelogue (2000--2014). In Proceedings on the Future of Software Engineering. ACM, 117--132.

Digital Library

[77]

A. Orso, N. Shi, and M. J. Harrold. 2004. Scaling regression testing to large software systems. ACM SIGSOFT Software Engineering Notes, 2004. ACM, 241--251.

Digital Library

[78]

T. J. Ostrand, E. J. Weyuker, and R. M. Bell. 2005. Predicting the location and number of faults in large software systems. IEEE Trans. Software Eng. 31, 340--355.

Digital Library

[79]

Y. Pang, X. Xue, and A. S. Namin. 2013. Identifying effective test cases through k-means clustering for enhancing regression testing. In Proceedings of the 12th International Conference on Machine Learning and Applications (ICMLA’13). IEEE, 78--83.

Digital Library

[80]

A. Panichella, R. Oliveto, M. D. Penta, and A. De Lucia. 2015. Improving multi-objective test case selection by injecting diversity in genetic algorithms. IEEE Trans. Software Eng. 41, 358--383.

Digital Library

[81]

A. Pasala, Y. Lew Yaw Fung, F. Akladios, G. Appala Raju, and R. P. Gorthi. 2008. Selection of regression test suite to validate software applications upon deployment of upgrades. In Proceedings of the 19th Australian Conference on Software Engineering (ASWEC’08). IEEE, 130--138.

Digital Library

[82]

S. Poulding, P. Emberson, I. Bate, and J. Clark. 2007. An efficient experimental methodology for configuring search-based design algorithms. In Proceedings of the 10th IEEE High Assurance Systems Engineering Symposium (HASE’07). IEEE, 53--62.

Digital Library

[83]

X. Qu, M. B. Cohen, and G. Rothermel. 2008. Configuration-aware regression testing: an empirical study of sampling and prioritization. In Proceedings of the 2008 International Symposium on Software Testing and Analysis. ACM, 75--86.

Digital Library

[84]

N. Rachatasumrit and M. Kim. 2012. An empirical investigation into the impact of refactoring on regression testing. In Proceedings of the 28th IEEE International Conference on Software Maintenance (ICSM’12). IEEE, 357--366.

Digital Library

[85]

D. Roest, A. Mesbah, and A. Van Deursen. 2010. Regression testing ajax applications: Coping with dynamism. In Proceedings of the 3rd International Conference on Software Testing, Verification and Validation (ICST’10). IEEE, 127--136.

Digital Library

[86]

E. Rogstad, L. Briand, and R. Torkar. 2013. Test case selection for black-box regression testing of database applications. Informat. Software Technol. 55, 1781--1795.

Digital Library

[87]

D. S. Rosenblum and E. J. Weyuker. 1997. Using coverage information to predict the cost-effectiveness of regression testing strategies. IEEE Trans. Software Engineering 23, 146--156.

Digital Library

[88]

G. Rothermel. 1996. Efficient, Effective Regression Testing Using Safe Test Selection Techniques. Clemson University.

[89]

G. Rothermel and M. J. Harrold. 1994. A framework for evaluating regression test selection techniques. In Proceedings of the 16th International Conference on Software Engineering (ICSE-16). IEEE, 201--210.

Digital Library

[90]

G. Rothermel and M. J. Harrold. 1996. Analyzing regression test selection techniques. IEEE Trans. Software Eng. 22, 529--551.

Digital Library

[91]

G. Rothermel, M. J. Harrold, J. Ostrin, and C. Hong. 1998. An empirical study of the effects of minimization on the fault detection capabilities of test suites. In Proceedings of the 1998 Proceedings of the International Conference on Software Maintenance. IEEE, 34--43.

Digital Library

[92]

G. Rothermel, R. H. Untch, C. Chu, and M. J. Harrold. 2001. Prioritizing test cases for regression testing. IEEE Trans. Software Engineering, 27, 929--948.

Digital Library

[93]

P. Sapna and H. Mohanty. 2010. Clustering test cases to achieve effective test selection. Proceedings of the 1st Amrita ACM-W Celebration on Women in Computing in India, 2010. ACM, 15.

Digital Library

[94]

W. Schütz. 1994. Fundamental issues in testing distributed real-time systems. Real-Time Syst. 7, 129--157.

Digital Library

[95]

W. R. Shadish, T. D. Cook, and D. T. Campbell. 2002. Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Wadsworth Cengage Learning.

[96]

A. Shi, T. Yung, A. Gyori, and D. Marinov. 2015. Comparing and combining test-suite reduction and regression test selection. In Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering. ACM, 237--247.

Digital Library

[97]

Y. Singh, A. Kaur, and B. Suri. 2010. A hybrid approach for regression testing in interprocedural program. JIPS, 6, 21--32.

[98]

C. Tao, B. Li, X. Sun, and C. Zhang. 2010a. An approach to regression test selection based on hierarchical slicing technique. In Proceedings of the IEEE 34th Annual Computer Software and Applications Conference Workshops (COMPSACW’10). IEEE, 347--352.

Digital Library

[99]

C. Tao, B. Li, X. Sun, and Y. Zhou. 2010b. A hierarchical model for regression test selection and cost analysis of Java programs. In Proceedings of the 17th Asia Pacific Software Engineering Conference (APSEC’10). IEEE, 290--299.

Digital Library

[100]

W. T. Tsai, X. Zhou, R. A. Paul, Y. Chen, and X. Bai. 2009. A coverage relationship model for test case selection and ranking for multi-version software. In High Assurance Services Computing. Springer.

[101]

R. Victor. 2003. Iterative and incremental development: A brief history. IEEE Computer Society, 47--56.

Digital Library

[102]

L. White. 1989. Insights IntoRegressionTesting. In Proceedings of the Conference on Software Maintenance. IEEE Computer Society Press, 60--69.

[103]

L. White and B. Robinson. 2004. Industrial real-time regression testing and analysis using firewalls. Proceedings on the 20th IEEE International Conference on Software Maintenance, 2004. IEEE, 18--27.

Digital Library

[104]

C. Wohlin, P. Runeson, M. Höst, M. C. Ohlsson, B. Regnell, and A. Wesslén. 2012. Experimentation in Software Engineering, Springer Science 8 Business Media.

Digital Library

[105]

W. E. Wong, J. R. Horgan, A. P. Mathur, and A. Pasquini. 1997. Test set size minimization and fault detection effectiveness: A case study in a space application. In Proceedings on the 21st Annual International Computer Software and Applications Conference (COMPSAC’97). IEEE, 522--528.

Digital Library

[106]

G. Xu and A. Rountev. 2007. Regression test selection for AspectJ software. In Proceedings of the 29th International Conference on Software Engineering (ICSE’07). IEEE, 65--74.

Digital Library

[107]

L. Xu, B. Xu, Z. Chen, J. Jiang, and H. Chen. 2003. Regression testing for web applications based on slicing. In Proceedings on the 27th Annual International Computer Software and Applications Conference (COMPSAC’03). IEEE, 652--656.

Digital Library

[108]

Z. Xu, K. Gao, T. M. Khoshgoftaar, and N. Seliya. 2014. System regression test planning with a fuzzy expert system. Informat. Sci. 259, 532--543.

Digital Library

[109]

Z. Xu, Y. Liu, and K. Gao. 2013. A novel fuzzy classification to enhance software regression testing. In Proceedings of the IEEE Symposium on Computational Intelligence and Data Mining (CIDM’13). IEEE, 53--58.

[110]

R. K. Yin. 2003. Case study research design and methods third edition. Applied Social Research Methods Series, 5.

[111]

S. Yoo and M. Harman. 2007. Pareto efficient multi-objective test case selection. In Proceedings of the 2007 International Symposium on Software Testing and Analysis. ACM, 140--150.

Digital Library

[112]

S. Yoo and M. Harman. 2012. Regression testing minimization, selection and prioritization: A survey. Software Test. Verificat. Reliabil. 22, 67--120.

Digital Library

[113]

L. Yu, L. Xu, and W. T. Tsai. 2010. Time-constrained test selection for regression testing. Advanced Data Mining and Applications. Springer.

Digital Library

[114]

T. Yu, X. Qu, M. Acharya, and G. Rothermel. 2013. Oracle-based regression test selection. In Proceedings of the IEEE 6th International Conference on Software Testing, Verification and Validation (ICST’13). IEEE, 292--301.

Digital Library

[115]

L. Zhang, S. S. Hou, C. Guo, T. Xie, and H. Mei. 2009. Time-aware test-case prioritization using integer linear programming. In Proceedings of the 18th International Symposium on Software Testing and Analysis. ACM, 213--224.

Digital Library

[116]

J. Zheng, L. Williams, B. Robinson, and K. Smiley. 2007. Regression test selection for black-box dynamic link library components. In Proceedings of the 2nd International Workshop on Incorporating COTS Software Into Software Systems: Tools and Techniques. IEEE Computer Society, 9.

Digital Library

Cited By

Haas RNömmer RJuergens EApel S(2024)Optimization of Automated and Manual Software Tests in Industrial Practice: A Survey and Historical AnalysisIEEE Transactions on Software Engineering10.1109/TSE.2024.341819150:8(2005-2020)Online publication date: 1-Aug-2024
https://dl.acm.org/doi/10.1109/TSE.2024.3418191
Li MFazzini M(2024)Automatically Removing Unnecessary Stubbings from Test Suites2024 IEEE Conference on Software Testing, Verification and Validation (ICST)10.1109/ICST60714.2024.00029(233-244)Online publication date: 27-May-2024
https://doi.org/10.1109/ICST60714.2024.00029
Altiero FCorazza ADi Martino SPeron ALibero Lucio Starace L(2024)Regression test prioritization leveraging source code similarity with tree kernelsJournal of Software: Evolution and Process10.1002/smr.265336:8Online publication date: 5-Aug-2024
https://dl.acm.org/doi/10.1002/smr.2653
Show More Cited By

Index Terms

Effective Regression Test Case Selection: A Systematic Literature Review
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Empirical software validation
      2. Software defect analysis
        Software testing and debugging

Recommendations

Coverage Specification for Test Case Intent Preservation in Regression Suites
ICSTW '13: Proceedings of the 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation Workshops

Regression testing ensures that previous faults do not recur. When a fault is reported and fixed, the testing team augments the test suite with a new test case that exercises the fault in the original program. Typically the new test case covers patterns ...
Test Case Prioritization for Continuous Regression Testing: An Industrial Case Study
ICSM '13: Proceedings of the 2013 IEEE International Conference on Software Maintenance

Regression testing in continuous integration environment is bounded by tight time constraints. To satisfy time constraints and achieve testing goals, test cases must be efficiently ordered in execution. Prioritization techniques are commonly used to ...
Evaluating non-adequate test-case reduction
ASE '16: Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering

Given two test cases, one larger and one smaller, the smaller test case is preferred for many purposes. A smaller test case usually runs faster, is easier to understand, and is more convenient for debugging. However, smaller test cases also tend to ...

Comments

Information & Contributors

Information

Published In

cover image ACM Computing Surveys

ACM Computing Surveys Volume 50, Issue 2

March 2018

567 pages

ISSN:0360-0300

EISSN:1557-7341

DOI:10.1145/3071073

Editor:
Sartaj Sahni
Department of Computer and Information Science and Engineering / University of Florida / Gainesville, FL 32611

Issue’s Table of Contents

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 May 2017

Accepted: 01 February 2017

Revised: 01 November 2016

Received: 01 February 2016

Published in CSUR Volume 50, Issue 2

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Survey
Research
Refereed

Funding Sources

Kementerian Sains, Teknologi dan Inovasi

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

82
Total Citations
View Citations
1,658
Total Downloads

Downloads (Last 12 months)176
Downloads (Last 6 weeks)13

Reflects downloads up to 23 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Haas RNömmer RJuergens EApel S(2024)Optimization of Automated and Manual Software Tests in Industrial Practice: A Survey and Historical AnalysisIEEE Transactions on Software Engineering10.1109/TSE.2024.341819150:8(2005-2020)Online publication date: 1-Aug-2024
https://dl.acm.org/doi/10.1109/TSE.2024.3418191
Li MFazzini M(2024)Automatically Removing Unnecessary Stubbings from Test Suites2024 IEEE Conference on Software Testing, Verification and Validation (ICST)10.1109/ICST60714.2024.00029(233-244)Online publication date: 27-May-2024
https://doi.org/10.1109/ICST60714.2024.00029
Altiero FCorazza ADi Martino SPeron ALibero Lucio Starace L(2024)Regression test prioritization leveraging source code similarity with tree kernelsJournal of Software: Evolution and Process10.1002/smr.265336:8Online publication date: 5-Aug-2024
https://dl.acm.org/doi/10.1002/smr.2653
García-Romero CCarrillo Bilbao GNavarro JMartin-Solano SSaegerman C(2023)Arboviruses in Mammals in the Neotropics: A Systematic Review to Strengthen Epidemiological Monitoring Strategies and Conservation MedicineViruses10.3390/v1502041715:2(417)Online publication date: 1-Feb-2023
https://doi.org/10.3390/v15020417
Raju SGopalan STowfek SSukumar AKhafaga DAlkahtani HAlahmadi T(2023)Test Case Selection through Novel Methodologies for Software Application DevelopmentsSymmetry10.3390/sym1510195915:10(1959)Online publication date: 23-Oct-2023
https://doi.org/10.3390/sym15101959
Greca RMiranda BBertolino A(2023)State of Practical Applicability of Regression Testing Research: A Live Systematic Literature ReviewACM Computing Surveys10.1145/357985155:13s(1-36)Online publication date: 13-Jul-2023
https://dl.acm.org/doi/10.1145/3579851
Kotti ZGalanopoulou RSpinellis D(2023)Machine Learning for Software Engineering: A Tertiary StudyACM Computing Surveys10.1145/357290555:12(1-39)Online publication date: 2-Mar-2023
https://dl.acm.org/doi/10.1145/3572905
Obaid Barraood SMohd HBaharom FAlmogahed A(2023)Systematic Literature Review on Test Case Quality Characteristics and Metrics2023 3rd International Conference on Emerging Smart Technologies and Applications (eSmarTA)10.1109/eSmarTA59349.2023.10293544(01-08)Online publication date: 10-Oct-2023
https://doi.org/10.1109/eSmarTA59349.2023.10293544
Querejeta MJee ELiu LValle PArrieta ARezabal M(2023)Search-based Test Case Selection for PLC Systems using Functional Block Diagram Programs2023 IEEE 34th International Symposium on Software Reliability Engineering (ISSRE)10.1109/ISSRE59848.2023.00040(228-239)Online publication date: 9-Oct-2023
https://doi.org/10.1109/ISSRE59848.2023.00040
Cao JChen BHu LGao JHuang KSong XPeng X(2023)Characterizing the Complexity and Its Impact on Testing in ML-Enabled Systems : A Case Sutdy on Rasa2023 IEEE International Conference on Software Maintenance and Evolution (ICSME)10.1109/ICSME58846.2023.00034(258-270)Online publication date: 1-Oct-2023
https://doi.org/10.1109/ICSME58846.2023.00034
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents