article

Free access

The evaluation of program-based software test data adequacy criteria

Author:

E. J. WeyukerAuthors Info & Claims

Communications of the ACM, Volume 31, Issue 6

Pages 668 - 675

https://doi.org/10.1145/62959.62963

Published: 01 June 1988 Publication History

Abstract

In earlier work, a preliminary set of axioms for software test data adequacy was introduced in order to formalize properties which should be satisfied by any good program-based adequacy criterion. Here, we extend this work by augmenting the set with additional axioms which substantially strengthen the set. In doing so, we rule out several types of unsuitable notions of adequacy.

References

[1]

Budd, T.A. "Mutation Analysis: Ideas, Examples. Problems and Prospects.'' In Computer Program Testing, ChandrasekararL and Radicchi, 129-148. North-Holland, New York, 1981.

[2]

Davis, M.D. and Weyuker, E.J. Computability, Complexity, and Languages. Academic Press, New York, 1983.

Digital Library

[3]

Davis, M.D. and Weyuker, E.J. A formal notion of program-based test data adequacy. Inf. and Control, 56, 1-2 (Jan.-Feb. lc,83), 52-71.

[4]

Davis, M.D. and Weyuker, E.}. Metric space based test data adequacy criteria. The Computer Journal, 30, 4 (19871.17-24.

Digital Library

[5]

DeMillo, R.A., Lipton, R.}. and Sayward, F.G. Hints on test data selection: Help for the practicing programmer. Computer, 11, 4 (Apr. I978}, 34-41.

Digital Library

[6]

Frankl, P.G. "The Use of Data Flow Information for lhe Selection and Evaluation of Software Test Data." Ph.D. diss., New York University, 1987.

Digital Library

[7]

FrankI, P.G. and Weyuker, E.J. Data flow testing in the presence of unexecutable paths. In Proc. Workshop on Software Testing (Banff, Alberta, Canada, July 15-17, 1986), 4-13.

[8]

Hamlet, R.G. Testing programs with the aid of a computer. IEEE Trans. Software Eng., SE-3, 4 {July 1977), 279-290.

[9]

Hamlet. R.G. Reliability theory of program testing. Acta Informatica, /6, (19811, 31-43.

[10]

Hoare, C.A.R. Hayes, I.J., Jifeng, He, Morgan, C.C., Roscoe, A.W. Sanders, J.W., Sorensen, I.H., Spivey, J.M. and Sufrin, B.A. Laws of programming. Comm. ACM, 30, 8 (Aug. 1987), 672-686.

Digital Library

[11]

Iannino, A., Musa, J.D., Okumoto, K. and Littlewood, B. Criteria for software reliability model comparisons. IEEE Trans. Software Eng., SE-IO, 6 (Nov. 1984), 687-691.

Digital Library

[12]

Myers, G.{. Software Reliability, Principles and Practices. John Wiley & Sons, New York, 1976.

Digital Library

[13]

Rapps, S. and Weyuker, E.J. Selecting software test data using data flow information. IEEE Trans. Software Eng., SE-/1, 4 (April 1985), 367-375.

Digital Library

[14]

Weyuker, E.J. The applicability of program schema results to programs. Int. J. Computer and Information Sci., 8, 5 (Nov. 1979), 387-403.

[15]

Weyuker, E.J. and Ostrand, T.J. Theories of program testing and the application of revealing subdomains. IEEE Trans. Software Eng., SE-6, 3 (May 1980), 236-246.

Digital Library

[16]

Weyuker, E.J. Axiomatizing software test data adequacy. IEEE Trans. Software Eng., SE-12, 12 (Dec. 1986), 1128-1138.

Digital Library

[17]

Weyuker, E.l. Evaluating software complexity measures. IEEE Trans. Software Eng., to be published.

Digital Library

[18]

Weyuker, E.J. An empirical study of the complexity of data flow testing, In Proc. Second Workshop Testing, Verification and Analysis (Banff. Alberta, Canada, July 19-21, 1988). to appear.

Cited By

Hossain SRoychoudhury APaiva AAbreu RStorey M(2024)Ensuring Critical Properties of Test Oracles for Effective Bug DetectionProceedings of the 2024 IEEE/ACM 46th International Conference on Software Engineering: Companion Proceedings10.1145/3639478.3639791(176-180)Online publication date: 14-Apr-2024
https://dl.acm.org/doi/10.1145/3639478.3639791
Hossain SFilieri ADwyer MElbaum SVisser WChandra SBlincoe KTonella P(2023)Neural-Based Test Oracle Generation: A Large-Scale Evaluation and Lessons LearnedProceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3611643.3616265(120-132)Online publication date: 30-Nov-2023
https://dl.acm.org/doi/10.1145/3611643.3616265
Hildebrandt Cvon Stein MElbaum SJust RFraser G(2023)PhysCov: Physical Test Coverage for Autonomous VehiclesProceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3597926.3598069(449-461)Online publication date: 12-Jul-2023
https://dl.acm.org/doi/10.1145/3597926.3598069
Show More Cited By

Index Terms

The evaluation of program-based software test data adequacy criteria
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis
        Software testing and debugging
2. Theory of computation
  1. Logic
  2. Semantics and reasoning
    1. Program reasoning

Recommendations

Revisiting the relationship between fault detection, test adequacy criteria, and test set size
ASE '20: Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering

The research community has long recognized a complex interrelationship between fault detection, test adequacy criteria, and test set size. However, there is substantial confusion about whether and how to experimentally control for test set size when ...
Software unit test coverage and adequacy

Objective measurement of test quality is one of the key issues in software testing. It has been a major research focus for the last two decades. Many test criteria have been proposed and studied for this purpose. Various kinds of rationales have been ...
A Formal Analysis of the Subsume Relation Between Software Test Adequacy Criteria

Software test adequacy criteria are rules to determine whether a software system has been adequately tested. A central question in the study of test adequacy criteria is how they relate to fault detecting ability. In this paper, we identify two ...

Comments

Information & Contributors

Information

Published In

cover image Communications of the ACM

Communications of the ACM Volume 31, Issue 6

June 1988

149 pages

ISSN:0001-0782

EISSN:1557-7317

DOI:10.1145/62959

Editor:
P. J. Denning
NASA Ames Research Center, Moffett Field, CA

Issue’s Table of Contents

Copyright © 1988 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 June 1988

Published in CACM Volume 31, Issue 6

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

95
Total Citations
View Citations
2,094
Total Downloads

Downloads (Last 12 months)92
Downloads (Last 6 weeks)11

Reflects downloads up to 04 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Hossain SRoychoudhury APaiva AAbreu RStorey M(2024)Ensuring Critical Properties of Test Oracles for Effective Bug DetectionProceedings of the 2024 IEEE/ACM 46th International Conference on Software Engineering: Companion Proceedings10.1145/3639478.3639791(176-180)Online publication date: 14-Apr-2024
https://dl.acm.org/doi/10.1145/3639478.3639791
Hossain SFilieri ADwyer MElbaum SVisser WChandra SBlincoe KTonella P(2023)Neural-Based Test Oracle Generation: A Large-Scale Evaluation and Lessons LearnedProceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3611643.3616265(120-132)Online publication date: 30-Nov-2023
https://dl.acm.org/doi/10.1145/3611643.3616265
Hildebrandt Cvon Stein MElbaum SJust RFraser G(2023)PhysCov: Physical Test Coverage for Autonomous VehiclesProceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3597926.3598069(449-461)Online publication date: 12-Jul-2023
https://dl.acm.org/doi/10.1145/3597926.3598069
Dola SDwyer MSoffa M(2023)Input Distribution Coverage: Measuring Feature Interaction Adequacy in Neural Network TestingACM Transactions on Software Engineering and Methodology10.1145/357604032:3(1-48)Online publication date: 26-Apr-2023
https://dl.acm.org/doi/10.1145/3576040
Leng JZhong YLin ZXu KMourtzis DZhou XZheng PLiu QZhao JShen W(2023)Towards resilience in Industry 5.0: A decentralized autonomous manufacturing paradigmJournal of Manufacturing Systems10.1016/j.jmsy.2023.08.02371(95-114)Online publication date: Dec-2023
https://doi.org/10.1016/j.jmsy.2023.08.023
Dola SDwyer MSoffa M(2021)Distribution-Aware Testing of Neural Networks Using Generative ModelsProceedings of the 43rd International Conference on Software Engineering10.1109/ICSE43902.2021.00032(226-237)Online publication date: 22-May-2021
https://dl.acm.org/doi/10.1109/ICSE43902.2021.00032
Antonio Pierro GTonelli R(2020)PASO: A Web-Based Parser for Solidity Language Analysis2020 IEEE International Workshop on Blockchain Oriented Software Engineering (IWBOSE)10.1109/IWBOSE50093.2020.9050263(16-21)Online publication date: Feb-2020
https://doi.org/10.1109/IWBOSE50093.2020.9050263
Lee JKang SJung P(2020)Test coverage criteria for software product line testing: Systematic literature reviewInformation and Software Technology10.1016/j.infsof.2020.106272122(106272)Online publication date: Jun-2020
https://doi.org/10.1016/j.infsof.2020.106272
Wang XXie ZHe JZhao GNie R(2019)Basis Path Coverage Criteria for Smart Contract Application Testing2019 International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery (CyberC)10.1109/CyberC.2019.00016(34-41)Online publication date: Oct-2019
https://doi.org/10.1109/CyberC.2019.00016
Paredes-Velastegui DLluma-Noboa AOlmedo-Vizueta DAvila-Pesantez DHernandez-Ambato J(2018)Augmented reality implementation as reinforcement tool for public textbooks education in Ecuador2018 IEEE Global Engineering Education Conference (EDUCON)10.1109/EDUCON.2018.8363372(1243-1250)Online publication date: Apr-2018
https://doi.org/10.1109/EDUCON.2018.8363372
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Media

Figures

Other

Tables

View Issue’s Table of Contents