Cited By
View all- Barth MJakobs M(2024)Refining CEGAR-Based Test-Case Generation with Feasibility AnnotationsTests and Proofs10.1007/978-3-031-72044-4_3(45-64)Online publication date: 9-Sep-2024
BlueCov: Integrating Test Coverage and Model Checking with JBMC
Pages 1695 - 1697
Abstract
Automated test case generation tools help businesses to write tests and increase the safety net provided by high regression test coverage when making code changes. Test generation needs to cover as much as possible of the uncovered code while avoiding generating redundant tests for code that is already covered by an existing test-suite.
In this paper we present our work on a tool for the real world application of integrating formal analysis with automatic test case generation. The test case generation is based on coverage analysis using the Java bounded model checker (JBMC). Counterexamples of the model checker can be translated into Java method calls with specific parameters.
In order to avoid the generation of redundant tests, it is necessary to measure the coverage in the exact same way as JBMC generates its coverage goals. Each existing coverage measurement tool uses a slightly different instrumentation and thus a different coverage criterion. This makes integration with a test case generator based on formal analysis difficult. Therefore, we developed BlueCov as a specific runtime coverage measurement tool which uses the exact same coverage criteria as JBMC does. This approach also allows for incremental test-case generation, only generating test coverage for previously untested code, e.g., to complete existing test suites.
References
[1]
Beyer, D.: Automatic verification of C and Java programs: SV-COMP 2019. In: TACAS. LNCS, vol. 11429, pp. 133--155. Springer (2019).
[2]
Beyer, D., Henzinger, T.A., Keremoglu, M.E., Wendler, P.: Conditional model checking: a technique to pass information between verifiers. In: FSE. p. 57. ACM (2012).
[3]
Bloem, R., Könighofer, R., Röck, F., Tautschnig, M.: Automating test-suite augmentation. In: QSIC. pp. 67--72. IEEE (2014).
[4]
Cordeiro, L., Kesseli, P., Kroening, D., Schrammel, P., Trtik, M.: JBMC: A bounded model checking tool for verifying Java bytecode. In: CAV. Springer (2018)
[5]
Cordeiro, L.C., Kroening, D., Schrammel, P.: Benchmarking of Java verification tools at the Software Verification Competition (SV-COMP). ACM SIGSOFT Softw. Eng. Notes abs/1809.03739(4), 56 (2018), http://arxiv.org/abs/1809.03739
[6]
Cordeiro, L.C., Kroening, D., Schrammel, P.: JBMC: bounded model checking for Java bytecode - (competition contribution). In: TACAS. LNCS, vol. 11429. Springer (2019).
[7]
Güdemann, M., Schrammel, P.: Bluecov: Integrating test coverage and model checking with jbmc (2022). https://arxiv.org/abs/2212.14779
[8]
Holzer, A., Schallhart, C., Tautschnig, M., Veith, H.: Query-driven program testing. In: VMCAI. Lecture Notes in Computer Science, vol. 5403, pp. 151--166. Springer (2009).
[9]
JaCoCo: Java code coverage, https://www.jacoco.org/jacoco/
[10]
Kroening, D., Tautschnig, M.: CBMC - C bounded model checker. In: TACAS. pp. 389--391. Springer (2014)
[11]
Nellis, A., Kesseli, P., Conmy, P.R., Kroening, D., Schrammel, P., Tautschnig, M.: Assisted coverage closure. In: NFM. LNCS, vol. 9690. Springer (2016).
[12]
Schrammel, P.: How testable is business software? In: 2020 Formal Methods in Computer Aided Design, FMCAD 2020, Haifa, Israel, September 21-24, 2020. p. 1. IEEE (2020), https://arxiv.org/abs/2011.00630
[13]
Schrammel, P., Melham, T., Kroening, D.: Chaining test cases for reactive system testing. In: ICTSS. LNCS, vol. 8254, pp. 133--148. Springer (2013).
[14]
Schrammel, P., Melham, T., Kroening, D.: Generating test case chains for reactive systems. Int. J. Softw. Tools Technol. Transf. 18(3), 319-334 (2016).
Index Terms
- BlueCov: Integrating Test Coverage and Model Checking with JBMC
Recommendations
Increasing Diversity in Coverage Test Suites Using Model Checking
QSIC '09: Proceedings of the 2009 Ninth International Conference on Quality SoftwareAutomated test case generation often results in test suites containing significant redundancy such as test cases that are duplicates, prefixes of other test cases, or cover the same test requirements. In this paper we consider the fact that items ...
Checked Coverage and Object Branch Coverage: New Alternatives for Assessing Student-Written Tests
SIGCSE '15: Proceedings of the 46th ACM Technical Symposium on Computer Science EducationMany educators currently use code coverage metrics to assess student-written software tests. While test adequacy criteria such as statement or branch coverage can also be used to measure the thoroughness of a test suite, they have limitations. Coverage ...
Towards a deeper understanding of test coverage
Test coverage is traditionally considered as how much of the code is covered by the test suite in whole. However, test suites typically contain different types of tests with different roles, such as unit tests, integration tests and functional tests. As ...
Comments
Information & Contributors
Information
Published In
March 2023
1932 pages
ISBN:9781450395175
DOI:10.1145/3555776
- Conference Chairs:
- Jiman Hong,
- Maart Lanperne,
- Program Chairs:
- Juw Won Park,
- Tomas Cerny,
- Publication Chair:
- Hossain Shahriar
Copyright © 2023 Owner/Author(s).
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s).
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 07 June 2023
Check for updates
Author Tags
Qualifiers
- Poster
Conference
SAC '23
Sponsor:
Acceptance Rates
Overall Acceptance Rate 1,650 of 6,669 submissions, 25%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 49Total Downloads
- Downloads (Last 12 months)34
- Downloads (Last 6 weeks)4
Reflects downloads up to 09 Nov 2024
Other Metrics
Citations
Cited By
View all- Barth MJakobs M(2024)Refining CEGAR-Based Test-Case Generation with Feasibility AnnotationsTests and Proofs10.1007/978-3-031-72044-4_3(45-64)Online publication date: 9-Sep-2024
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in