article

Two notions of correctness and their relation to testing

Authors:

Timothy A. Budd,

Dana AngluinAuthors Info & Claims

Acta Informatica, Volume 18, Issue 1

Pages 31 - 45

https://doi.org/10.1007/BF00625279

Published: 01 March 1982 Publication History

Abstract

We consider two interpretations for what it means for test data to demonstrate correctness. For each interpretation, we examine under what conditions data sufficient to demonstrate correctness exists, and whether it can be automatically detected and/or generated. We establish the relation between these questions and the problem of deciding equivalence of two programs.

References

[1]

Angluin, D.: On the complexity of minimum inference of regular sets. Information and Control39(3): 337---350 (1978)

[2]

Baldwin, D., Sayward, F.: Heuristics for determining equivalence of program mutations. Technical Report Number 161, Yale University 1979

[3]

Barzdin, J.M., Bicevskis, J.J., Kalninsh, A.A.: Construction of complete sample system for correctness testing. In: Lecture Notes in Computer Science, Vol. 32, pp. 1---12. Berlin: Springer-Verlag 1975

[4]

Biermann, A.W., Krishnaswamy, R.: Constructing programs from example computations. IEEE Transactions on Software Engineering SE-2(3): 141---153 (1976)

Digital Library

[5]

Blum, M.: On effective procedures for speeding up algorithms. Journal of the ACM18(2): 290---305 (1971)

[6]

Blum, L., Blum, M.: Toward a mathematical theory of inductive inference. Information and Control28(2): 125---155 (1975)

[7]

Brooks, M.: Automatic generation of test data for recursive programs having simple errors. PhD Thesis, Stanford University 1980

[8]

Budd, T.A., Lipton, R.J., DeMillo, R.A., Sayward, F.G.: Mutation analysis. Technical Report Number 155, Yale University 1979

[9]

Budd, T.A.: Mutation analysis of program test data. PhD Thesis, Yale University 1980

[10]

Chow, T.S.: Testing software design modeled by finite-state machines. IEEE Transactions on Software Engineering SE-4(3): 178---187 (1978)

Digital Library

[11]

Clarke, L.A.: A system to generate test data and symbolically execute programs. IEEE Transactions on Software Engineering SE-2(3): 215---222 (1976)

Digital Library

[12]

DeMillo, R.A., Lipton, R.J., Sayward, F.G.: Hints on test data selection: Help for the practicing programmer. Computer11(4): 34---43 (1978)

[13]

DeMillo, R.A., Lipton, R.J.: A probabilistic remark on algebraic program testing. Information Processing Letters7(4): 193---195 (1978)

[14]

Foster, K.A.: Error sensitive test cases. IEEE Transactions on Software Engineering SE-6(3): 258---264 (1980)

Digital Library

[15]

Geller, M.: Test data as an aid in proving program correctness. Communications of the ACM21(5): 368---375 (1978)

[16]

Goodenough, J.B., Gerhart, S.L.: Towards a theory of test data selection. IEEE Transactions on Software Engineering SE-1(2):156---173 (1975)

Digital Library

[17]

Goodenough, J.B.: A survey of program testing issues. pp. 316---340. In: Research Directions in Software Technology (P. Wegner, ed.), pp. 316---340. Cambridge, MA: MIT Press, 1979

[18]

Hamlet, R.: Testing programs with the aid of a compiler. IEEE Transactions on Software Engineering SE-3(4): 279---290 (1977)

Digital Library

[19]

Hamlet, R.: Testing programs with finite sets of data. The Computer Journal20(3): 232---237 (1977)

[20]

Hamlet, R.: Test reliability and software maintenance. In: Proceeding of the Compsac conference, pp. 315---320. IEEE 1978

[21]

Hamlet, R.: Critique of reliability theory. In: Digest of the Workshop on Software Testing and Test Documentation, pp. 57---69. Fort Lauderdale 1978

[22]

Hantler, S.L., King, J.C.: An introduction to proving the correctness of programs. ACM Computer Surveys8(3): 331---353 (1976)

[23]

Hopcroft, J.E., Ullman, J.D.: Formal languages and their relation to automata. New York: Addison-Wesley 1969

[24]

Howden, W.E.: Reliability of the path analysis testing strategy. IEEE Transactions on Software Engineering SE-2(3): 208---214 (1976)

Digital Library

[25]

Howden, W.E.: Lindenmayer grammers and symbolic testing. Information Processing L etters7(1): 36---39 (1978)

[26]

Howden, W.E.: Algebraic program testing. Acta Informat.10(1): 53---66 (1978)

Digital Library

[27]

Howden, W.E.: Functional program testing. IEEE Transactions on Software Engineering SE-6(2): 162---169 (1980)

Digital Library

[28]

Kernighan, B.W., Plauger, P.J.: Software tools. New York: Addison-Wesley 1976

[29]

Miller, E.F., Melton, R.A.: Automated generation of testcase datasets. In: Proceeding 1975 International Conference on Reliable Software, pp. 51---58. Los Angeles 1975

[30]

Ostrand, T.J., Weyuker, E.J.: Remarks on the theory of test data selection. In: Digest of the Workshop on Software Testing and Test Documentation, pp. 1---18. Fort Lauderdale 1978

[31]

Rogers, H. Jr.: Theory of recursive functions and effective computability. New York: McGraw-Hill 1967

[32]

Rowland, J.H., Davis, P.J.: On the selection of test data for recursive mathematical sub-routines. Siam Journal on Computing10(1): 59---72 (1981)

[33]

White, L.J., Cohen, E.I.: A domain strategy for computer program testing. IEEE Transactions on Software Engineering SE-6(3): 247---257 (1980)

Digital Library

Cited By

Li ZShin DBosch JLewis GCleland-Huang JMuccini H(2024)Mutation-based Consistency Testing for Evaluating the Code Understanding Capability of LLMsProceedings of the IEEE/ACM 3rd International Conference on AI Engineering - Software Engineering for AI10.1145/3644815.3644946(150-159)Online publication date: 14-Apr-2024
https://dl.acm.org/doi/10.1145/3644815.3644946
Moradi Moghadam MBagherzadeh MKhatchadourian RBagheri HChandra SBlincoe KTonella P(2023)𝜇Akka: Mutation Testing for Actor Concurrency in Akka using Real-World BugsProceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3611643.3616362(262-274)Online publication date: 30-Nov-2023
https://dl.acm.org/doi/10.1145/3611643.3616362
Deiner AFeldmeier PFraser GSchweikl SWang W(2023)Automated test generation for Scratch programsEmpirical Software Engineering10.1007/s10664-022-10255-x28:3Online publication date: 13-May-2023
https://dl.acm.org/doi/10.1007/s10664-022-10255-x
Show More Cited By

Two notions of correctness and their relation to testing
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis

Recommendations

A relation algebraic model of robust correctness

We propose a new and uniform abstract relational approach to demonic nondeterminism and robust correctness similar to Hoares chaos semantics. It is based on a specific set of relations on flat lattices. This set forms a complete lattice. Furthermore, we ...
Verifying functional correctness of C programs with VCC
NFM'11: Proceedings of the Third international conference on NASA Formal methods

VCC [2] is an industrial-strength verification environment for low-level concurrent systems code written in C. VCC takes a program (annotated with function contracts, state assertions, and type invariants) and attempts to prove the correctness of these ...
Correctness witnesses: exchanging verification results between verifiers
FSE 2016: Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering

Standard verification tools provide a counterexample to witness a specification violation, and, since a few years, such a witness can be validated by an independent validator using an exchangeable witness format. This way, information about the ...

Comments

Information & Contributors

Information

Published In

cover image Acta Informatica

Acta Informatica Volume 18, Issue 1

March 1982

114 pages

ISSN:0001-5903

Issue’s Table of Contents

Copyright © Copyright © 1982 Springer-Verlag.

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 01 March 1982

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

67
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 09 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Li ZShin DBosch JLewis GCleland-Huang JMuccini H(2024)Mutation-based Consistency Testing for Evaluating the Code Understanding Capability of LLMsProceedings of the IEEE/ACM 3rd International Conference on AI Engineering - Software Engineering for AI10.1145/3644815.3644946(150-159)Online publication date: 14-Apr-2024
https://dl.acm.org/doi/10.1145/3644815.3644946
Moradi Moghadam MBagherzadeh MKhatchadourian RBagheri HChandra SBlincoe KTonella P(2023)𝜇Akka: Mutation Testing for Actor Concurrency in Akka using Real-World BugsProceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3611643.3616362(262-274)Online publication date: 30-Nov-2023
https://dl.acm.org/doi/10.1145/3611643.3616362
Deiner AFeldmeier PFraser GSchweikl SWang W(2023)Automated test generation for Scratch programsEmpirical Software Engineering10.1007/s10664-022-10255-x28:3Online publication date: 13-May-2023
https://dl.acm.org/doi/10.1007/s10664-022-10255-x
Cañizares PNúñez AFilgueira Rde Lara J(2023)Parallel mutation testing for large scale systemsCluster Computing10.1007/s10586-023-04074-y27:2(2071-2097)Online publication date: 20-Jun-2023
https://dl.acm.org/doi/10.1007/s10586-023-04074-y
Fernandes LRibeiro MGheyi RDelamaro MGuimarães MSantos A(2022)Put Your Hands In The Air! Reducing Manual Effort in Mutation TestingProceedings of the XXXVI Brazilian Symposium on Software Engineering10.1145/3555228.3555233(198-207)Online publication date: 5-Oct-2022
https://dl.acm.org/doi/10.1145/3555228.3555233
Laurent TArcaini PTrubiani CVentresque A(2022)Mutation-based analysis of queueing network performance modelsJournal of Systems and Software10.1016/j.jss.2022.111385191:COnline publication date: 22-Jun-2022
https://dl.acm.org/doi/10.1016/j.jss.2022.111385
Ojdanić MMa WLaurent TChekam TVentresque APapadakis M(2022)On the use of commit-relevant mutantsEmpirical Software Engineering10.1007/s10664-022-10138-127:5Online publication date: 1-Sep-2022
https://dl.acm.org/doi/10.1007/s10664-022-10138-1
Jeangoudoux CDarulova ELauter CCadar CZhang X(2021)Interval constraint-based mutation testing of numerical specificationsProceedings of the 30th ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3460319.3464808(388-399)Online publication date: 11-Jul-2021
https://dl.acm.org/doi/10.1145/3460319.3464808
Cornejo OPastore FBriand L(2021)Mutation Analysis for Cyber-Physical Systems: Scalable Solutions and Results in the Space DomainIEEE Transactions on Software Engineering10.1109/TSE.2021.310768048:10(3913-3939)Online publication date: 27-Aug-2021
https://dl.acm.org/doi/10.1109/TSE.2021.3107680
Ma WChekam TPapadakis MHarman M(2021)MuDeltaProceedings of the 43rd International Conference on Software Engineering10.1109/ICSE43902.2021.00086(897-909)Online publication date: 22-May-2021
https://dl.acm.org/doi/10.1109/ICSE43902.2021.00086
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents