research-article

Trace Analysis for Conformance and Arbitration Testing

Authors:

G. V. Bochmann,

J. R. ZhaoAuthors Info & Claims

IEEE Transactions on Software Engineering, Volume 15, Issue 11

Pages 1347 - 1356

https://doi.org/10.1109/32.41328

Published: 01 November 1989 Publication History

Abstract

The authors explore a testing approach where the concern for selecting the appropriate test input provided to the implementation under test (IUT) is separated as much as possible from the analysis of the observed output. Particular emphasis is placed on the analysis of the observed interactions of the IUT in order to determine whether the observed input/output trace conforms to the IUT's specification. The authors consider this aspect of testing with particular attention to testing of communication protocol implementations. Various distributed test architectures are used for this purpose, where partial input/output traces are observable by local observers at different interfaces. The error-detection power of different test configurations is determined on the basis of the partial trace visible to each local observer and their global knowledge about the applied test case. The automated construction of trace analysis modules from the formal specification of the protocol is also discussed. Different transformations of the protocol specification may be necessary to obtain the reference specification, which can be used by a local or global observer for checking the observed trace. Experience with the construction of an arbiter for the OSI (open systems interconnection) transport protocol is described.

References

[1]

{1} A. V. Aho and J. D. Ullman, Principles of Compiler Design . Reading, MA: Addison-Wesley, 1977.

[2]

{2} J. M. Ayache, P. Azema, and M. Diaz, "Observer: A concept for on-line detection for control errors in concurrent systems," in Proc. 9th Int. Symp. FTC, Madison, WI, June 1979.

[3]

{3} G.v. Bochmann, E. Cerny, M. Maksud, and B. Sarikaya, "Testing of transport protocol implementations," in Proc. CIPS Conf., Ottawa, 1983, pp. 123-129.

[4]

{4} G.v. Bochmann, "Usage of protocol development tools: the results of a survey" (invited paper), presented at the 7th IFIP Symp. Protocol Specification, Testing and Verification, Zurich, May 1987.

[5]

{5} G.v. Bochmann, G. Gerber, and J. M. Serre, "Semiautomatic implementation of communication protocols," IEEE Trans. Software Eng., vol. SE-13, no. 9, pp. 989-1000, Sept. 1987; reprinted in Automatic Implementation and Conformance Testing of OSI Protocols. D. P. Sidhu, Ed. New York: IEEE, 1989.

Digital Library

[6]

{6} G.v. Bochmann, C. He, D. Ouimet, and R. Zhao, "Protocol testing using automatic trace analysis," in Proc. IEEE Canadian Conf. Electrical and Computer Engineering, Sept. 1989.

[7]

{7} G.v. Bochmann, "Protocol specification for OSI," Comput. Networks and ISDN Syst., to be published.

[8]

{8} S. S. Lam and A. U. Shankar, "Protocol verification via projections," IEEE Trans. Software Eng., vol. SE-10, no. 4, pp. 325-342, July 1984.

Digital Library

[9]

{9} G.v. Bochmann and O. Bellal, "Test result analysis in respect to formal specifications," submitted for publication.

[10]

{10} I. C. Davidson, "The NCC protocol testing service," in Proc. Work shop Introduction of High Level Protocol Standards for OSI (BNI/AFNOR), Paris, June 1983, pp. 273-279.

[11]

{11} R. Dssouli and G.v. Bochmann, "Error detection with multiple observers," in Proc. IFIP Workshop Protocol Specification, Testing, and Validation, Toulouse, France, June 1985.

[12]

{12} R. Dssouli and G.v. Bochmann, "Conformance testing with multiple observers," in Proc. IFIP Workshop Protocol Specification, Testing, and Validation, 1986, pp. 217-229.

[13]

{13} R. Dssouli, "Etude des methodes de test pour les implantations de protocoles de communication basees sur les specifications formelles," Ph.D. dissertation, Univ. Montreal, 1986.

[14]

{14} ISO IS9074, "Estelle: A formal description technique based on an extended state transition model," 1989.

[15]

{15} C. Jard and G.v. Bochmann, "An approach to testing specifications," J. Syst. Software, vol. 3, no. 4, pp. 315-323, Dec. 1983.

Digital Library

[16]

{16} L. Lamport, "Time, clocks and the ordering of events in a distributed system," Commun. ACM, vol. 21, no. 7, pp. 558-565, July 1978.

Digital Library

[17]

{17} L. Logrippo et al., "An interpreter for LOTOS: A specification language for distributed systems," Software Practice and Experience, vol. 18, no. 4, pp. 365-385, Apr. 1988.

Digital Library

[18]

{18} ISO IS8807, "LOTOS: A formal description technique," 1989.

[19]

{19} P. Merlin and G.v. Bochmann, "On the construction of submodule specifications and communication protocols," ACM Trans. Program, Lang. Syst., vol. 5, no. 1, pp. 1-25, Jan. 1983.

Digital Library

[20]

{20} R. Molva, M. Diaz, and J. M. Ayache, "Observer: A run time checking tool for local area networks," 5th IFIP Workshop Protocol Specification, Testing and Verification, Toulouse, 1985.

[21]

{21} ISO TC97/SC21, DIS 9646/1, and DIS 9646/2, "OSI Conformance testing methodology and framework, Part 1: General Concepts, Part 2: Abstract Test Suite Specification," 1989.

[22]

{22} ISO TC97/SC6, IS 8073, "OSI--Connection oriented transport protocol specification."

[23]

{23} D. Rafiq, R. Castanet, C. Chraibi, J. P. Goursaud, J. Haddad, and X. Perdu, "Towards and environment for testing OSI protocols," in Proc. IFIP Workshop Protocol Specification, Verification and Testing , Toulouse, 1985.

[24]

{24} D. Rayner, "A system for testing protocol implementations," Comput. Networks, vol. 6, no. 6, Dec. 1982.

[25]

{25} CCITT SG X1, Recommendation Z.100, 1987.

[26]

{26} B. Sarikaya and G.v. Bochmann, "Synchronization and specifcation issues in protocol testing," IEEE Trans. Commun., vol. COM-32, no. 4, pp. 389-395, Apr. 1984.

[27]

{27} B. Sarikaya, G.v. Bochmann, and E. Cerny, "A test design methodology for protocol testing," IEEE Trans. Software Eng., vol. SE- 13, pp. 518-531, Apr. 1987.

Digital Library

[28]

{28} H. Ural and R. L. Probert, "Automated testing of protocol specifications and their implementations," in Proc. ACM SIGCOMM Symp., 1984.

[29]

{29} C. Vissers, "Formal description techniques for OSI," in Proc. IFIP Congress, Information Processing '86. Amsterdam, The Netherlands: North-Holland, 1986.

[30]

{30} H. X. Zeng and D. Rayner, "The impact of the ferry concept on protocol testing," in Protocol Specification, Testing and Verification (IFIP Workshop), M. Diaz, Ed. Amsterdam, The Netherlands: North-Holland, 1986, pp. 533-544.

Cited By

Schumann JMbaya TMengshoel OPipatsrisawat KSrivastava AChoi ADarwiche A(2013)Software health management with Bayesian networksInnovations in Systems and Software Engineering10.1007/s11334-013-0214-y9:4(271-292)Online publication date: 1-Dec-2013
https://dl.acm.org/doi/10.1007/s11334-013-0214-y
Desmoulin AViho C(2009)Formalizing interoperability for test case generation purposeInternational Journal on Software Tools for Technology Transfer (STTT)10.5555/3220918.322122911:3(261-267)Online publication date: 1-Jul-2009
https://dl.acm.org/doi/10.5555/3220918.3221229
Desmoulin AViho C(2007)A new method for interoperability test generationProceedings of the 19th IFIP TC6/WG6.1 international conference, and 7th international conference on Testing of Software and Communicating Systems10.5555/2391293.2391298(58-73)Online publication date: 26-Jun-2007
https://dl.acm.org/doi/10.5555/2391293.2391298
Show More Cited By

Index Terms

Trace Analysis for Conformance and Arbitration Testing
1. Networks
  1. Network architectures
  2. Network protocols
2. Theory of computation
  1. Logic
  2. Semantics and reasoning
    1. Program reasoning

Recommendations

Update The Derivation of Conformance Tests From LOTOS Specifications

The derivation of conformance tests for communication protocols is discussed. Protocol specifications are considered in the formal description technique LOTOS, which has been developed by the International Standards Organization. Test processes which ...
Formal test specifications in open systems
ISESS '95: Proceedings of the 2nd IEEE Software Engineering Standards Symposium

The development of formal test specifications for an open system standard is described. The effort is being conducted within the environment provided by the Clemson Automated Testing System (CATS). CATS features the ability to automatically translate ...
An evaluation of MIA conformance tests
ISESS '95: Proceedings of the 2nd IEEE Software Engineering Standards Symposium

NTT has developed a set of specifications called Multivendor Integration Architecture (MIA) for open systems codeveloped with vendors, etc. NTT then implemented conformance testing for MIA products, and is now using it's results for procurement. We ...

Comments

Information & Contributors

Information

Published In

cover image IEEE Transactions on Software Engineering

IEEE Transactions on Software Engineering Volume 15, Issue 11

November 1989

178 pages

ISSN:0098-5589

Editor:
Victor R. Basili
Univ. of Maryland, College Park

Issue’s Table of Contents

Copyright © Copyright © 1989 IEEE. All Rights Reserved.

Publisher

IEEE Press

Publication History

Published: 01 November 1989

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

20
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 09 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Schumann JMbaya TMengshoel OPipatsrisawat KSrivastava AChoi ADarwiche A(2013)Software health management with Bayesian networksInnovations in Systems and Software Engineering10.1007/s11334-013-0214-y9:4(271-292)Online publication date: 1-Dec-2013
https://dl.acm.org/doi/10.1007/s11334-013-0214-y
Desmoulin AViho C(2009)Formalizing interoperability for test case generation purposeInternational Journal on Software Tools for Technology Transfer (STTT)10.5555/3220918.322122911:3(261-267)Online publication date: 1-Jul-2009
https://dl.acm.org/doi/10.5555/3220918.3221229
Desmoulin AViho C(2007)A new method for interoperability test generationProceedings of the 19th IFIP TC6/WG6.1 international conference, and 7th international conference on Testing of Software and Communicating Systems10.5555/2391293.2391298(58-73)Online publication date: 26-Jun-2007
https://dl.acm.org/doi/10.5555/2391293.2391298
Desmoulin AViho C(2007)Automatic interoperability test case generation based on formal definitionsProceedings of the 12th international conference on Formal methods for industrial critical systems10.5555/1793603.1793622(234-250)Online publication date: 1-Jul-2007
https://dl.acm.org/doi/10.5555/1793603.1793622
Brzezinski KMalinski N(2007)Self-tuned passive testers for grey-box distributed systems with indefinite communication delaysProceedings of the 25th conference on Proceedings of the 25th IASTED International Multi-Conference: parallel and distributed computing and networks10.5555/1295581.1295602(127-132)Online publication date: 13-Feb-2007
https://dl.acm.org/doi/10.5555/1295581.1295602
Andrews JZhang Y(2003)General Test Result Checking with Log File AnalysisIEEE Transactions on Software Engineering10.1109/TSE.2003.121432729:7(634-648)Online publication date: 1-Jul-2003
https://dl.acm.org/doi/10.1109/TSE.2003.1214327
Bhargavan KGunter CLee ISokolsky OKim MObradovic DViswanathan M(2002)VerisimIEEE Transactions on Software Engineering10.1109/32.98849528:2(129-145)Online publication date: 1-Feb-2002
https://dl.acm.org/doi/10.1109/32.988495
Hunter CStrooper P(2001)Systematically deriving partial oracles for testing concurrent programsAustralian Computer Science Communications10.5555/545595.54557523:1(83-91)Online publication date: 1-Jan-2001
https://dl.acm.org/doi/10.5555/545595.545575
Hunter CStrooper POudshoorn M(2001)Systematically deriving partial oracles for testing concurrent programsProceedings of the 24th Australasian conference on Computer science10.5555/545564.545575(83-91)Online publication date: 29-Jan-2001
https://dl.acm.org/doi/10.5555/545564.545575
Bhargavan KGunter CKim MLee IObradovic DSokolsky OViswanathan M(2000)VerisimACM SIGSOFT Software Engineering Notes10.1145/347636.34783325:5(2-13)Online publication date: 1-Aug-2000
https://dl.acm.org/doi/10.1145/347636.347833
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents