research-article

Use of Sequencing Constraints for Specification-Based Testing of Concurrent Programs

Authors:

Richard H. Carver,

Kuo-Chung TaiAuthors Info & Claims

IEEE Transactions on Software Engineering, Volume 24, Issue 6

Pages 471 - 490

https://doi.org/10.1109/32.689403

Published: 01 June 1998 Publication History

Abstract

This paper presents and evaluates a specification-based methodology for testing concurrent programs. This methodology requires sequencing constraints, which specify restrictions on the allowed sequences of synchronization events. Sequencing constraints for a concurrent program can be derived from the program's formal or informal specification. Details of the proposed testing methodology based on the use of Constraints on Succeeding and Preceding Events (CSPE) are given. How to achieve coverage and detect violations of CSPE constraints for a concurrent program, according to deterministic and nondeterministic testing of this program, are described. A coverage criterion for CSPE-based testing is defined and analyzed. The results of empirical studies of CSPE-based testing for four concurrent problems are reported. These results indicate that the use of sequencing constraints for specification-based testing of concurrent programs is a promising approach.

References

[1]

M.A. Vouk D. McAllister and K.C. Tai, "An Experimental Evaluation of the Effectiveness of Random Testing of Fault-Tolerant Software," Proc. ACM Software Testing Workshop, pp. 74-81, July 1986.

[2]

B. Beizer, Software Testing Techniques, second edition. Van Nostrand, 1990.

Digital Library

[3]

R.N. Taylor D.L. Levine and C.D. Kelly, "Structural Testing of Concurrent Programs," IEEE Trans. Software Eng., vol. 18, no. 3, pp. 206-215, Mar. 1992.

Digital Library

[4]

R.D. Yang and C.G. Chung, "Path Analysis Testing of Concurrent Programs," Information and Software Technology, vol. 34. no. 1, pp. 43-56, Jan. 1992.

Digital Library

[5]

S.K. Damodaran-Kamal and J.M. Francioni, "Nondeterminancy: Testing and Debugging in Message Passing Parallel Programs," Proc. ACM/ONR Workshop Parallel and Distributed Debugging, ACM SIGPLAN Notices, vol. 28, no. 12, pp. 118-128, Dec. 1993.

Digital Library

[6]

G.H. Hwang K.C. Tai and T.L. Huang, "Reachability Testing: An Approach to Testing Concurrent Software," Proc. Int'l J. Software Eng. and Knowledge Eng., vol. 5, no. 4, pp. 493-510, Dec. 1995.

[7]

K.C. Tai, "Reachability Testing of Asynchronous Message-Passing Programs," Proc. IEEE Int'l Workshop Software Eng. for Parallel and Distributed Systems, pp. 50-61, May 1997.

Digital Library

[8]

G.J. Holzman, Design and Validation of Computer Protocols. Englewood Cliffs, N.J.: Prentice Hall, 1991.

Digital Library

[9]

G. v. Bochmann and A. Petrenko, "Protocol Testing: Review of Methods and Relevance for Software Testing," Proc. ACM Symp. Software Testing and Analysis, pp. 109-124 1994.

Digital Library

[10]

L.K. Dillon G. Kutty L.E. Moser P.M. Melliar-Smith and Y.S. Ramakrishna, "Graphical Specifications for Concurrent Software Systems," Proc. 14th Int'l Conf. Software Eng., pp. 214-224, 1992.

Digital Library

[11]

D. Helmbold and D. Luckham, "TSL: Task Sequencing Language," Proc. Ada Int'l Conf., pp. 255-274, 1985.

Digital Library

[12]

L.K. Dillon and Q. Yu, "Oracles for Checking Temporal Properties of Concurrent Systems," Proc. Second ACM SIGSOFT Symp. Foundations of Software Eng., pp. 140-153, 1994.

Digital Library

[13]

D. Helmbold and D. Luckham, "Debugging Ada Tasking Programs," IEEE Software, vol. 2, no. 2, pp. 47-57, Mar. 1985.

Digital Library

[14]

K.C. Tai and R.H. Carver, "Testing of Distributed Programs," ch. 33 of Handbook of Parallel and Distributed Computing, A. Zomaya, ed., pp. 955-978. McGraw-Hill, 1996.

[15]

R.H. Carver and K.C. Tai, "Replay and Testing for Concurrent Programs," IEEE Software, pp. 66-74, Mar. 1991.

Digital Library

[16]

K.C. Tai R.H. Carver and E.E. Obaid, "Debugging Concurrent Ada Programs by Deterministic Execution," IEEE Trans. Soft. Eng., vol. 17, no. 1, pp. 45-63, Jan. 1991.

Digital Library

[17]

T.J. LeBlanc and J.M. Mellor-Crummey, "Debugging Parallel Programs with Instant Replay," IEEE Trans. Computers, vol. 36, no. 4, pp. 471-482, Apr. 1987.

Digital Library

[18]

R.H. Carver and K.C. Tai, "Static Analysis of Concurrent Software for Deriving Synchronization Constraints," Proc. IEEE Int'l Conf. Distributed Computing Systems, pp. 544-551, May 1991.

[19]

R.H. Carver, "Testing Abstract Distributed Programs and Their Implementations," J. Systems and Software, special issue on Software Eng. for Distributed Computing, pp. 223-237, June 1996.

Digital Library

[20]

G. Bruns, Distributed Systems Analysis with CCS. Prentice Hall, 1997.

Digital Library

[21]

K.C. Tai and R.H. Carver, "A Specification-Based Methodology for Testing Concurrent Programs," Proc. Europe Software Engineering Conf., Lecture Notes in Computer Science 989, W. Schafer and P. Botella, eds., pp. 154-172. Springer-Verlag, 1995.

Digital Library

[22]

R.H. Carver and J. Chen, "Incremental Conformance Testing Using Lotos Specifications," Proc. Fifth Int'l Conf. Computer Comm. and Networks, pp. 42-47, Oct. 1996.

[23]

P.V. Koppol and K.C. Tai, "An Incremental Approach to Structural Testing of Concurrent Software" ACM Int'l Symp. Software Testing and Analysis, pp. 14-23, Jan. 1996.

Digital Library

[24]

A.J. Offutt C.Z. Rothermel and C. Zapf, "An Experimental Evaluation of Selective Mutation," Proc. Int'l Conf. Software Eng., pp. 100-107, 1993.

Digital Library

[25]

K.C. Tai, "A Graphical Representation of Rendezvous Sequences of Concurrent Ada Programs," ACM Ada Letters, vol. VI, no. 1, pp. 94-103, Jan./Feb. 1986.

Digital Library

[26]

Using Formal Description Techniques: An Introduction to Estelle, Lotos, and SDL, K.J. Turner, ed., John Wiley & Sons, 1993.

Digital Library

[27]

R.H. Carver and R. Durham, "Integrating Formal Methods and Testing for Concurrent Programs," Proc. 10th IEEE Conf. Computer Assurance, pp. 25-33, June 1995.

[28]

K.C. Tai and Y.C. Young, "Synchronizable Test Sequences of Finite State Machines," 1998. to appear in Computer Network and ISDN Systems

Digital Library

[29]

P. Tripathy and K. Naik, "Generation of Adaptive Tests from Nondeterministic Finite State Models," Proc. Protocol Test Systems, V, pp. 309-320, 1992.

Digital Library

[30]

J. Arjjo, "On the Existence and Production of State Identification Machines for Labeled Transition Systems," Proc. Formal Description Techniques, VI, pp. 351-366, 1993.

Digital Library

[31]

P.V. Koppol and K.C. Tai, "Conformance Testing of Protocols Specified as Labeled Transitions Systems," Proc. IFIP Eighth Int'l Workshop on Protocol Test Systems, A. Cavalli and S. Budkowski, eds., pp. 135-150, Chapman and Hall, 1996.

[32]

G. v. Bochmann R. Dssouli and J. Zhao, "Trace Analysis for Conformance and Arbitration Testing," IEEE Trans. Software. Eng., vol. 15, no. 11, pp. 1,347-1,356, Nov. 1989.

Digital Library

[33]

Y.C. Young and K.C. Tai, "Observation Inaccuracy in Conformance Testing with Multiple Testers," Proc. IEEE First Workshop on Application-Specific Software Eng. and Technology, pp. 77-82, Mar. 1998.

Digital Library

[34]

D. Rosenblum, "Specifying Concurrent Systems with TSL," IEEE Software, pp. 52-61, May 1991.

Digital Library

[35]

E.J. Weyuker T. Goradia and A. Singh, "Automatically Generating Test Data from Boolean Specification," IEEE Trans. Software Eng., vol. 20, no. 5, pp. 353-363, May 1994.

Digital Library

[36]

A.M. Pardkar and K.C. Tai, "Test Generation for Boolean Expressions," Proc. Int'l Symp. Software Reliability Eng., pp. 106-115, 1995.

[37]

A.M. Paradkar K.C. Tai and M.A. Vouk, "Automatic Test Generation for Predicates," IEEE Trans. Reliability, pp. 515-530, Dec. 1996.

[38]

K.C. Tai, "Theory of Fault-Based Predicate Testing for Computer Programs," IEEE Trans. Software Eng., vol. 22, no. 8, pp. 552-562, Aug. 1996.

Digital Library

[39]

P.C. Bates and J.C. Wileden, "High Level Debugging of Distributed Systems: The Behavior Abstraction Approach," J. Software & Systems, vol. 3, pp. 255-264, 1984.

Digital Library

[40]

W.S. Cheng and V.E. Wallentine, "DEBL: A Knowledge-Based Language for Specifying and Debugging Distributed Programs," Comm. ACM, vol. 32, no. 9, pp. 1,079-1,084, Sept. 1989.

Digital Library

[41]

C. Stirling, "An Introduction to Modal and Temporal Logics for CCS," Lecture Notes in Computer Science 491, pp. 1-20. Springer-Verlag, 1991.

Digital Library

[42]

R. Cleaveland J. Parrow and B. Steffen, "The Concurrency WorkBench: A Semantics Tool for the Verification of Concurrent Systems," ACM Trans. Programming Languages and Systems, vol. 15, no. 1, pp. 36-72, Jan. 1993,.

Digital Library

Cited By

Bianchi FMargara APezze M(2018)A Survey of Recent Trends in Testing Concurrent Software SystemsIEEE Transactions on Software Engineering10.1109/TSE.2017.270708944:8(747-783)Online publication date: 1-Aug-2018
https://dl.acm.org/doi/10.1109/TSE.2017.2707089
Li SZhang YLuo HChen YLu CGuo D(2018)Race-Condition-Aware and Hardware-Oriented Task Partitioning and Scheduling Using Entropy MaximizationIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2017.278482929:7(1589-1604)Online publication date: 1-Jul-2018
https://dl.acm.org/doi/10.1109/TPDS.2017.2784829
Choi YByun T(2017)Constraint-based test generation for automotive operating systemsSoftware and Systems Modeling (SoSyM)10.1007/s10270-014-0449-616:1(7-24)Online publication date: 1-Feb-2017
https://dl.acm.org/doi/10.1007/s10270-014-0449-6
Show More Cited By

Index Terms

Recommendations

Fault-based test suite prioritization for specification-based testing

Context: Existing test suite prioritization techniques usually rely on code coverage information or historical execution data that serve as indicators for estimating the fault-detecting ability of test cases. Such indicators are primarily empirical in ...
A Method for Structural Testing of Ada Concurrent Programs Using the Event Interactions Graph
APSEC '96: Proceedings of the Third Asia-Pacific Software Engineering Conference

Software testing generally proceeds as follows: generating test-cases, selecting test-data, executing a test target program, inspecting execution result and evaluating whether testing has already been sufficient or not yet. As for methods for structural ...
Saturation-based testing of concurrent programs
ESEC/FSE '09: Proceedings of the 7th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on The foundations of software engineering

Coverage measures help to determine whether a test suite exercises a program adequately according to a testing criterion. Many existing measures, however, are defined over coverage domains that cannot be precisely calculated, rendering them of limited ...

Comments

Information & Contributors

Information

Published In

cover image IEEE Transactions on Software Engineering

IEEE Transactions on Software Engineering Volume 24, Issue 6

June 1998

96 pages

ISSN:0098-5589

Editor:
Richard A. Kemmerer
Univ. of California, Santa Barbara

Issue’s Table of Contents

Copyright © Copyright © 1998 IEEE. All Rights Reserved.

Publisher

IEEE Press

Publication History

Published: 01 June 1998

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

26
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 27 Jul 2024

Other Metrics

View Author Metrics

Citations

Cited By

Bianchi FMargara APezze M(2018)A Survey of Recent Trends in Testing Concurrent Software SystemsIEEE Transactions on Software Engineering10.1109/TSE.2017.270708944:8(747-783)Online publication date: 1-Aug-2018
https://dl.acm.org/doi/10.1109/TSE.2017.2707089
Li SZhang YLuo HChen YLu CGuo D(2018)Race-Condition-Aware and Hardware-Oriented Task Partitioning and Scheduling Using Entropy MaximizationIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2017.278482929:7(1589-1604)Online publication date: 1-Jul-2018
https://dl.acm.org/doi/10.1109/TPDS.2017.2784829
Choi YByun T(2017)Constraint-based test generation for automotive operating systemsSoftware and Systems Modeling (SoSyM)10.1007/s10270-014-0449-616:1(7-24)Online publication date: 1-Feb-2017
https://dl.acm.org/doi/10.1007/s10270-014-0449-6
Yu TSung ASrisa-an WRothermel G(2014)An approach to testing commercial embedded systemsJournal of Systems and Software10.5555/2747015.274718588:C(207-230)Online publication date: 1-Feb-2014
https://dl.acm.org/doi/10.5555/2747015.2747185
Carver RLei Y(2010)A class library for implementing, testing, and debugging concurrent programsInternational Journal on Software Tools for Technology Transfer (STTT)10.5555/3220893.322107012:1(69-88)Online publication date: 1-Feb-2010
https://dl.acm.org/doi/10.5555/3220893.3221070
Wojcicki MStrooper P(2007)Maximising the information gained from a study of static analysis technologies for concurrent softwareEmpirical Software Engineering10.1007/s10664-007-9044-612:6(617-645)Online publication date: 1-Dec-2007
https://dl.acm.org/doi/10.1007/s10664-007-9044-6
Rutherford MCarzaniga AWolf AYoung MDevanbu P(2006)Simulation-based test adequacy criteria for distributed systemsProceedings of the 14th ACM SIGSOFT international symposium on Foundations of software engineering10.1145/1181775.1181804(231-241)Online publication date: 5-Nov-2006
https://dl.acm.org/doi/10.1145/1181775.1181804
Wojcicki MStrooper PTravassos GMaldonado JWohlin C(2006)Maximising the information gained from an experimental analysis of code inspection and static analysis for concurrent java componentsProceedings of the 2006 ACM/IEEE international symposium on Empirical software engineering10.1145/1159733.1159761(174-183)Online publication date: 21-Sep-2006
https://dl.acm.org/doi/10.1145/1159733.1159761
Liu HTan HZhu HHorgan JCheung SLi J(2006)Automated verification and test case generation for input validationProceedings of the 2006 international workshop on Automation of software test10.1145/1138929.1138936(29-35)Online publication date: 23-May-2006
https://dl.acm.org/doi/10.1145/1138929.1138936
Briand LLabiche YSówka MOsterweil LRombach DSoffa M(2006)Automated, contract-based user testing of commercial-off-the-shelf componentsProceedings of the 28th international conference on Software engineering10.1145/1134285.1134300(92-101)Online publication date: 28-May-2006
https://dl.acm.org/doi/10.1145/1134285.1134300
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents