Object coverage criteria for supporting object-oriented testing
Authors: 
M. Ghoreshi, H. HaghighiAuthors Info & Claims
M. Ghoreshi, H. HaghighiAuthors Info & ClaimsPages 1369 - 1414
Abstract
Code coverage criteria are widely used in object-oriented (OO) domains as test quality indicators. However, these criteria are based on the procedural point of view, and therefore do not address the specific features of OO programs. In this article, we extend the code coverage criteria and introduce a new set of criterion, called “object coverage criteria,” which cope with OO features like object instantiation, inheritance, polymorphism, and dynamic binding. Unlike previous criteria, the new criteria regard the actual type of the object under test and some inherited codes from the parent/ancestor classes that represent the object’s states and behaviors. The new criteria have been implemented in a prototype tool called OCov4J for the Java language. Using this tool and conducting an empirical study on 270 classes (with about 50 k lines of code without blank lines and comments) from several large and widely used open source projects, we have found a considerable positive correlation between the object coverage level (defined via the new proposed criteria) and the number of detected specific OO failures. Not only do the proposed criteria provide ease of use, high automation, and low execution cost, but also they can effectively be applied to real-world OO programs.
References
[1]
Alexander, R. T., & Offutt, A. J. (2000, October). Criteria for testing polymorphic relationships. In Proceedings 11th International Symposium on Software Reliability Engineering. ISSRE 2000 (pp. 15–23). IEEE.
[2]
Alexander, R. T., Offutt, J., & Bieman, J. M. (2002, December). Syntactic fault patterns in oo programs. In Eighth IEEE International Conference on Engineering of Complex Computer Systems (pp. 193–202). IEEE.
[3]
Alexander RT, Offutt J, and Stefik A Testing coupling relationships in object-oriented programs Software Testing, Verification and Reliability 2010 20 4 291-327
[4]
Ammann P, Offutt J. (2016, December 13). Introduction to software testing. Cambridge University Press.
[5]
Aziz SR, Khan T, and Nadeem A Experimental validation of inheritance metrics’ impact on software fault prediction IEEE Access 2019 7 85262-85275
[6]
Binder, R. (2000). Testing object-oriented systems: Models, patterns, and tools. Addison-Wesley Professional.
[7]
Bloch, J. (2008). Effective java (the java series). Prentice Hall PTR.
[8]
Braione, P., & Denaro, G. (2019, May). SUSHI and TARDIS at the SBST2019 tool competition. In 2019 IEEE/ACM 12th International Workshop on Search-Based Software Testing (SBST) (pp. 25–28). IEEE.
[9]
Coles, H., Laurent, T., Henard, C., Papadakis, M., & Ventresque, A. (2016, July). Pit: A practical mutation testing tool for java. In Proceedings of the 25th international symposium on software testing and analysis (pp. 449–452).
[10]
Denaro, G., Margara, A., Pezze, M., & Vivanti, M. (2015, May). Dynamic data flow testing of object oriented systems. In 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering (Vol. 1, pp. 947–958). IEEE.
[11]
Devroey, X., Panichella, S., & Gambi, A. (2020, June). Java unit testing tool competition: Eighth round. In Proceedings of the IEEE/ACM 42nd International Conference on Software Engineering Workshops (pp. 545–548).
[12]
Fraser, G., & Arcuri, A. (2011, September). Evosuite: Automatic test suite generation for object-oriented software. In Proceedings of the 19th ACM SIGSOFT symposium and the 13th European conference on Foundations of software engineering (pp. 416–419).
[13]
Fraser G and Arcuri A 1600 faults in 100 projects: Automatically finding faults while achieving high coverage with evosuite Empirical Software Engineering 2015 20 3 611-639
[14]
Gamma, E., Johnson, R., Helm, R., Johnson, R. E., & Vlissides, J. (1995). Design patterns: Elements of reusable object-oriented software. Pearson Deutschland GmbH.
[15]
Gay G, Staats M, Whalen M, and Heimdahl MP The risks of coverage-directed test case generation IEEE Transactions on Software Engineering 2015 41 8 803-819
[16]
Ghoreshi M and Haghighi H An incremental method for extracting tests from object-oriented specification Information and Software Technology 2016 78 1-26
[17]
Gopinath, R., Jensen, C., & Groce, A. (2014, May). Code coverage for suite evaluation by developers. In Proceedings of the 36th International Conference on Software Engineering (pp. 72–82).
[18]
Harrold, M. J., McGregor, J. D., & Fitzpatrick, K. J. (1992, June). Incremental testing of object-oriented class structures. In Proceedings of the 14th international conference on Software engineering (pp. 68–80).
[19]
Hemmati, H. (2015, August). How effective are code coverage criteria? In 2015 IEEE International Conference on Software Quality, Reliability and Security (pp. 151–156). IEEE.
[20]
Just, R., Jalali, D., & Ernst, M. D. (2014, July). Defects4J: A database of existing faults to enable controlled testing studies for Java programs. In Proceedings of the 2014 International Symposium on Software Testing and Analysis (pp. 437–440).
[21]
Kifetew, F., Devroey, X., & Rueda, U. (2019, May). Java unit testing tool competition-seventh round. In 2019 IEEE/ACM 12th International Workshop on Search-Based Software Testing (SBST) (pp. 15–20). IEEE.
[22]
Kim SW, Clark JA, and McDermid JA Investigating the effectiveness of object-oriented testing strategies using the mutation method Software Testing, Verification and Reliability 2001 11 4 207-225
[23]
Liu, B., Ge, H., Chen, J., & Bao, Q. (2019, November). An automatic testing platform for object-oriented software based on code coverage. In Proceedings of the 2019 the 9th International Conference on Communication and Network Security (pp. 20–24).
[24]
Ma, Y. S., Offutt, J., & Kwon, Y. R. (2006, May). MuJava: A mutation system for Java. In Proceedings of the 28th international conference on Software engineering (pp. 827–830).
[25]
Madeyski L and Radyk N Judy–a mutation testing tool for Java IET Software 2010 4 1 32-42
[26]
Mcheick, H., Dhiab, H., Dbouk, M., & Mcheik, R. (2010, May). Detecting type errors and secure coding in C/C++ applications. In ACS/IEEE International Conference on Computer Systems and Applications-AICCSA 2010 (pp. 1–9). IEEE.
[27]
Molina, U. R., Kifetew, F., & Panichella, A. (2018, May). Java unit testing tool competition-sixth round. In 2018 IEEE/ACM 11th International Workshop on Search-Based Software Testing (SBST) (pp. 22–29). IEEE.
[28]
Najumudheen, E. S. F., Mall, R., & Samanta, D. (2019, February). Modeling and coverage analysis of programs with exception handling. In Proceedings of the 12th Innovations on Software Engineering Conference (formerly known as India Software Engineering Conference) (pp. 1–11).
[29]
Offutt, J., Alexander, R., Wu, Y., Xiao, Q., & Hutchinson, C. (2001, November). A fault model for subtype inheritance and polymorphism. In Proceedings 12th International Symposium on Software Reliability Engineering (pp. 84–93). IEEE.
[30]
Offutt, J., Ma, Y. S., & Kwon, Y. R. (2006, May). The class-level mutants of MuJava. In Proceedings of the 2006 international workshop on Automation of software test (pp. 78–84).
[31]
Orso, A., & Pezze, M. (1999, June). Integration testing of procedural object-oriented languages with polymorphism. In Proceedings of the 16th International Conference on Testing Computer Software: Future Trends in Testing (TCS’99).
[32]
Pacheco, C., & Ernst, M. D. (2007, October). Randoop: Feedback-directed random testing for Java. In Companion to the 22nd ACM SIGPLAN conference on Object-oriented programming systems and applications companion (pp. 815–816).
[33]
Papadakis, M., Kintis, M., Zhang, J., Jia, Y., Le Traon, Y., & Harman, M. (2019). Mutation testing advances: An analysis and survey. In Advances in Computers (Vol. 112, pp. 275–378). Elsevier.
[34]
Perry DE and Kaiser GE Adequate testing and object-oriented programming Journal of Object-Oriented Programming 1990 2 5 13-19
[35]
Prasetya, I. W. B. (2015, August). T3i: A tool for generating and querying test suites for java. In Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering (pp. 950–953).
[36]
Razali NM and Wah YB Power comparisons of Shapiro-Wilk, Kolmogorov-Smirnov, Lilliefors and Anderson-Darling tests Journal of Statistical Modeling and Analytics 2011 2 1 21-33
[37]
Saha, R. K., Lyu, Y., Lam, W., Yoshida, H., & Prasad, M. R. (2018, May). Bugs. jar: A large-scale, diverse dataset of real-world java bugs. In Proceedings of the 15th international Conference on Mining Software Repositories (pp. 10–13).
[38]
Schwartz A, Puckett D, Meng Y, and Gay G Investigating faults missed by test suites achieving high code coverage Journal of Systems and Software 2018 144 106-120
[39]
Segura S, Hierons RM, Benavides D, and Ruiz-Cortés A Mutation testing on an object-oriented framework: An experience report Information and Software Technology 2011 53 10 1124-1136
[40]
Smith, G. (2012). The Object-Z specification language (Vol. 1). Springer Science & Business Media.
[41]
Smith, M. D., & Robson, D. J. (1990, November). Object-oriented programming-the problems of validation. In Proceedings Conference on Software Maintenance 1990 (pp. 272–281). IEEE.
[42]
Su T, Wu K, Miao W, Pu G, He J, Chen Y, and Su Z A survey on data-flow testing ACM Computing Surveys (CSUR) 2017 50 1 1-35
[43]
Zou, Y., Chen, Z., Zheng, Y., Zhang, X., & Gao, Z. (2014, July). Virtual DOM coverage for effective testing of dynamic web applications. In Proceedings of the 2014 International Symposium on Software Testing and Analysis (pp. 60–70).
Recommendations
An automated object-oriented testing for C++ inheritance hierarchy
ASE '97: Proceedings of the 12th international conference on Automated software engineering (formerly: KBSE)This paper proposes a concept named unit repeated inheritance (URI) in Z notation to realize object-oriented testing of an inheritance hierarchy. Based on this unit, an inheritance level technique (ILT) method as a guide to test object-oriented software ...
Object State Testing for Object-Oriented Programs
COMPSAC '95: Proceedings of the 19th International Computer Software and Applications ConferenceAbstract: Since class objects are the major components in object oriented (OO) programs during run time, testing the behavior of these objects become an important task in testing of object oriented programs to complement other types of testing, such as, ...
A dependence graph-based representation for test coverage analysis of object-oriented programs
Test coverage analysis is used to determine the extent to which different features of interest are exercised by a given test suite. For object-oriented programs, it involves determining the extent to which features such as inheritance, polymorphism, ...
Comments
Information & Contributors
Information
Published In
© The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2023. Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
Publisher
Kluwer Academic Publishers
United States
Publication History
Published: 30 June 2023
Accepted: 02 June 2023
Author Tags
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 09 Nov 2024
Other Metrics
Citations
View Options
View options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in