research-article

Free access

CHOC'LATE: a framework for specification-based testing

Authors:

Pak-Lok Poon,

Sau-Fun Tang,

T. H. Tse,

T. Y. ChenAuthors Info & Claims

Communications of the ACM, Volume 53, Issue 4

Pages 113 - 118

https://doi.org/10.1145/1721654.1721687

Published: 01 April 2010 Publication History

All formats PDF

Abstract

In spite of its importance in software reliability, testing is labor intensive and expensive. It has been found that software testing without a good strategy may not be more effective than testing the system with random data. Obviously, the effectiveness of testing relies heavily on how well the test suite --- the set of test cases actually used --- is generated. This is because the comprehensiveness of the test suite will affect the scope of testing and, hence, the chance of revealing software faults.

There are two main approaches to generating test suites: specification-based and code-based. The former generates a test suite from information derived from the specification, without requiring the knowledge of the internal structure of the program. The latter approach, on the other hand, generates a test suite based on the source code of the program. Neither of these approaches is sufficient; they are complementary to one another.

In software development, the requirements have to be established before implementation, and the specification should exist prior to coding. In this respect, the specification-based approach to test suite generation is particularly useful because test cases can be generated before coding has been completed. This facilitates software development phases to be performed in parallel, thus allowing time for preparing more thorough test plans and yet shortening the length of the whole process.

References

[1]

Beydeda, S., Gruhn, V., and Stachorski, M. A graphical class representation for integrated black- and white-box testing. In Proceedings of the IEEE International Conference on Software Maintenance. IEEE Computer Society Press, Los Alamitos, CA, 2001, 706--715.

Digital Library

Google Scholar

[2]

Binder, R.V. Category-partition test design pattern. Excerpt from Testing Object-Oriented Systems: Models, Patterns, and Tools. Addison Wesley, Reading, MA, 2000: http://www.rbsc.com/docs/TestPatternCategoryPartition.pdf.

Digital Library

Google Scholar

[3]

Chen, T.Y., Poon, P.-L., and Tse, T.H., A choice relation framework for supporting category-partition test case generation. IEEE Transactions on Software Engineering 29, 7, (2003) 577--593.

Digital Library

Google Scholar

[4]

Gupta, N., Mathur, A.P., and Soffa, M.L. Generating test data for branch coverage. In Proceedings of the 15th IEEE International Conference on Automated Software Engineering. IEEE Computer Society Press, Los Alamitos, CA, 2000.

Digital Library

Google Scholar

[5]

Hierons, R.M., Harman, M., and Singh, H. Automatically generating information from a Z specification to support the classification tree method. In Proceedings of the 3rd International Conference of B and Z Users 2651 of Lecture Notes in Computer Science, 388--407. Springer, Berlin, Germany, 2003.

Digital Library

Google Scholar

[6]

Kobayashi, N., Tsuchiya, T., and Kikuno, T. A new method for constructing pair-wise covering designs for software testing. Information Processing Letters 81, 2, (2002) 85--91.

Digital Library

Google Scholar

[7]

Lau, M.F. and Yu, Y.T. An extended fault class hierarchy for specification-based testing. ACM Transactions on Software Engineering and Methodology 14, 3, (2005), 247--276.

Digital Library

Google Scholar

[8]

Li, J.J., Weiss, D., and Yee, H. Code-coverage guided prioritized test generation. Information and Software Technology 48, 12, (2006) 1187--1198.

Digital Library

Google Scholar

[9]

Stocks, P. and Carrington, D.A. A framework for specification-based testing. IEEE Transactions on Software Engineering 22, 11, (1996), 777--793.

Digital Library

Google Scholar

[10]

Tai, K.-C. and Lei, Y. A test generation strategy for pairwise testing. IEEE Transactions on Software Engineering 28, 1, (2002), 109--111.

Digital Library

Google Scholar

[11]

Tsuchiya, T. and Kikuno, T. On fault classes and error detection capability of specification-based testing. ACM Transactions on Software Engineering and Methodology 11, 1,(2002), 58--62.

Digital Library

Google Scholar

[12]

Vilkomir, S.A., and Bowen, J.P. From MC/DC to RC/DC: formalization and analysis of control-flow testing criteria. Formal Aspects of Computing 18, 1, (2006), 42--62.

Crossref

Google Scholar

Cited By

View all

Sun CFu APoon PXie XLiu HChen T(2019)METRIC+: A Metamorphic Relation Identification Technique Based on Input plus Output DomainsIEEE Transactions on Software Engineering10.1109/TSE.2019.2934848(1-1)Online publication date: 2019
https://doi.org/10.1109/TSE.2019.2934848
Huang RChen JLu Y(2014)Adaptive Random Testing with Combinatorial Input DomainThe Scientific World Journal10.1155/2014/8432482014(1-16)Online publication date: 2014
https://doi.org/10.1155/2014/843248
Poon PChen TTse T(2013)Incremental Identification of Categories and Choices for Test Case GenerationProceedings of the 2013 13th International Conference on Quality Software10.1109/QSIC.2013.14(230-237)Online publication date: 29-Jul-2013
https://dl.acm.org/doi/10.1109/QSIC.2013.14
Show More Cited By

Index Terms

CHOC'LATE: a framework for specification-based testing
1. Computing methodologies
  1. Machine learning
2. Software and its engineering
  1. Software creation and management

Recommendations

A Static Approach to Prioritizing JUnit Test Cases

Test case prioritization is used in regression testing to schedule the execution order of test cases so as to expose faults earlier in testing. Over the past few years, many test case prioritization techniques have been proposed in the literature. Most ...
Spartan: a spectral and entropy-based partial-scan and test point insertion algorithm
Faster mutation testing inspired by test prioritization and reduction
ISSTA 2013: Proceedings of the 2013 International Symposium on Software Testing and Analysis

Mutation testing is a well-known but costly approach for determining test adequacy. The central idea behind the approach is to generate mutants, which are small syntactic transformations of the program under test, and then to measure for a given test ...

Reviews

Reviewer: James Dennis Kiper

A formal specification can be used to generate test cases for a software system. A choice relation framework system provides a mechanism and a supporting tool for systematically converting an informal specification into a relatively more formal specification in the form of a choice table. The graphical user interface (GUI) provided by this tool clearly allows a tester to avoid the learning curve associated with most formal specification languages. The tester identifies the system inputs and associated categories (equivalent classes) for these inputs. The tester also specifies constraints on combinations of the inputs. The system can then automatically generate many of the choice relations from which test cases can be generated. The authors have also recognized that testing resources are limited. Thus, testers can specify which of the test frames have the highest priority. Poon et al. make some claims about both the effectiveness and efficiency of this system, compared to other test generators. They report the results of some experiments and the fact that they used this tool in four case studies, but no details of these experiments or case studies are presented. If these claims are verified, this framework is a reasonably effective testing method. Online Computing Reviews Service

Access critical reviews of Computing literature here

Become a reviewer for Computing Reviews.

Comments

Information & Contributors

Information

Published In

Communications of the ACM Volume 53, Issue 4

April 2010

131 pages

ISSN:0001-0782

EISSN:1557-7317

DOI:10.1145/1721654

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 April 2010

Published in CACM Volume 53, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article
Popular
Refereed

Funding Sources

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
913
Total Downloads

Downloads (Last 12 months)80
Downloads (Last 6 weeks)16

Reflects downloads up to 03 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Sun CFu APoon PXie XLiu HChen T(2019)METRIC+: A Metamorphic Relation Identification Technique Based on Input plus Output DomainsIEEE Transactions on Software Engineering10.1109/TSE.2019.2934848(1-1)Online publication date: 2019
https://doi.org/10.1109/TSE.2019.2934848
Huang RChen JLu Y(2014)Adaptive Random Testing with Combinatorial Input DomainThe Scientific World Journal10.1155/2014/8432482014(1-16)Online publication date: 2014
https://doi.org/10.1155/2014/843248
Poon PChen TTse T(2013)Incremental Identification of Categories and Choices for Test Case GenerationProceedings of the 2013 13th International Conference on Quality Software10.1109/QSIC.2013.14(230-237)Online publication date: 29-Jul-2013
https://dl.acm.org/doi/10.1109/QSIC.2013.14
Labiche YSadeghi F(2013)Experimenting with Category Partition's 1-Way and 2-Way Test Selection CriteriaProceedings of the 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation Workshops10.1109/ICSTW.2013.41(301-310)Online publication date: 18-Mar-2013
https://dl.acm.org/doi/10.1109/ICSTW.2013.41
Chen TPoon PTang STse T(2012)DESSERTIEEE Transactions on Software Engineering10.1109/TSE.2011.6938:4(794-809)Online publication date: 1-Jul-2012
https://dl.acm.org/doi/10.1109/TSE.2011.69
Poon PChen TTse T(2012)Choices, choicesProceedings of the 17th Ada-Europe international conference on Reliable Software Technologies10.1007/978-3-642-30598-6_12(162-176)Online publication date: 11-Jun-2012
https://dl.acm.org/doi/10.1007/978-3-642-30598-6_12
Chen Y(2011)Generation of Test Cases for Concurrent Software Systems Based on Data-Flow-Oriented SpecificationsProceedings of the 2011 First ACIS/JNU International Conference on Computers, Networks, Systems and Industrial Engineering10.1109/CNSI.2011.49(170-177)Online publication date: 23-May-2011
https://dl.acm.org/doi/10.1109/CNSI.2011.49
Poon PTse TTang SKuo F(2011)Contributions of tester experience and a checklist guideline to the identification of categories and choices for software testingSoftware Quality Journal10.1007/s11219-010-9109-419:1(141-163)Online publication date: 1-Mar-2011
https://dl.acm.org/doi/10.1007/s11219-010-9109-4
Yu YChan EPoon P(2009)On the Coverage of Program Code by Specification-Based TestsProceedings of the 2009 Ninth International Conference on Quality Software10.1109/QSIC.2009.14(41-50)Online publication date: 24-Aug-2009
https://dl.acm.org/doi/10.1109/QSIC.2009.14

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Magazine Site

View this article on the magazine site (external)

Magazine Site

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Abstract

References

Cited By

Index Terms

Recommendations

A Static Approach to Prioritizing JUnit Test Cases

Spartan: a spectral and entropy-based partial-scan and test point insertion algorithm

Faster mutation testing inspired by test prioritization and reduction

Reviews

Access critical reviews of Computing literature here

Comments

Information

Published In

Publisher

Publication History

Permissions

Check for updates

Qualifiers

Funding Sources

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

View options

PDF

eReader

Magazine Site

Get Access

Login options

Full Access

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations