Specification-based Testing

Formal methods have traditionally been used for specification and development of software. However there are potential benefits for the testing stage as well. The panel session associated with this paper explores the usefulness or otherwise of formal methods in various contexts for improving software testing. A number of different possibilities for the use of formal methods are explored and questions raised. The contributors are all members of the UK FORTEST Network on formal methods and testing. Although the authors generally believe that formal methods are useful in aiding the testing process, this paper is intended to provoke discussion. Dissenters are encouraged to put their views to the panel or individually to the authors.

Software testing is one of the most widely used methods for quality assurance and fault detection purposes. However, it is one of the most expensive, tedious and time consuming activities in software development life cycle. Code-based and specification-based testing has been going on for almost four decades. Model-based testing (MBT) is a relatively new approach to software testing where the

The formal methods community is in general very good at undertaking research into the mathematical aspects of formal methods, but not so good at promulgating the use of formal methods in an engineering environment and at an industrial scale. Technology transfer is an extremely important part of the overall effort necessary in the acceptance of formal techniques. This paper explores some of the more informal aspects of applying formal methods and presents some maxims with associated discussion that may help in the application of formal methods in an industrial setting.

Developers of distributed systems routinely construct discrete-event simulations to help them un- derstand and evaluate the behavior of inter-component protocols. Typically written using an imperative programming language, these simulations capture basic algorithmic functionality at the same time as they focus attention on properties critical to distribution, including topology, timing, bandwidth, and overall scalability. We ask the following question: Can simulations

For testing software, test case generation is the most important part. The automation of specification based test case generation needs formal or semi formal specification. As a semiformal modeling, UML is widely used to describe analysis and design specification by both academia and industry. Thus UML models become the sources of test case generation naturally. This paper proposes a method to generate test cases from UML activity diagrams. We have proposed conditioned slicing as a general slicing framework for test case generation from activity diagrams. Our method first builds a flow dependence graph from an ordinary UML activity diagram and then applies conditioned slicing on a predicate node of the graph, to generate test cases. It minimizes the number of test cases generated while deriving all practically useful test cases. The effectiveness of a test case is based on how well the test covers and exercises the modeled behaviors. Our proposed method satisfies high path coverage ...

A technique for testing analog and mixed-signal linear cir-cuit components based on their impulse response (IR) sig-natures is presented in this paper. A simple DFT structure is proposed to enable the on-chip generation of the impulse response signatures from the ...

It is presented a promising approach to test interactive components, supporting the automatic generation of test cases from a specification. The relevance and difficulties (issues and challenges) associated with the testing of interactive components are first presented. It is shown that a formal specification with certain characteristics allows the automatic generation of test cases while solving some of the issues presented. The approach is illustrated with an example of automatic testing of the conformity between the ...

Even with increasing use of formal methods for software development, software testing continues to be a dominant technique for verification and validation. With specification-based testing, the precision of formal specifications can make the testing more systematic. This paper describes a tool to assist the derivation of test cases from Object-Z specifications. The tool supports the Test Template Framework and manages the testing information for the user. Management of the generated testing information is regarded as a high priority and is one of the tool's features. The tool is demonstrated on a simple example to indicate its core functionality

We show that the quantile regression estimator is consistent and asymptotically normal when the error terms are correlated within clusters but independent across clusters. A consistent estimator of the covariance matrix of the asymptotic distribution is provided and we propose a specification test capable of detecting the presence of intra-cluster correlation. A small simulation study illustrates the finite sample performance of the test and of the covariance matrix estimator. JEL classification code: C12, C21, C23.

Economics, University of Essex, for private circulation to interested readers. They often represent preliminary reports on work in progress and should therefore be neither quoted nor referred to in published work without the written consent of the author. Quantile regression with clustered data ∗

It is presented a promising approach to test interactive components, supporting the automatic generation of test cases from a specification. The relevance and difficulties (issues and challenges) associated with the testing of interactive components are first presented. It is shown that a formal specification with certain characteristics allows the automatic generation of test cases while solving some of the issues presented. The approach is illustrated with an example of automatic testing of the conformity between the ...

Regression testing is important activity during the software maintenance to deal with adverse effects of changes. Our approach is important for safety critical system as usually formal methods are preferred and highly recommended for the safety critical systems but they are also applied for the systems development of other than critical system. Our approach is based on Regression testing using

High-capability software services, like transaction processing systems, need to satisfy a range of non-functional characteristics such as performance, availability, and scalability. To fulfill these needs, the core business logic is usually extended with a large amount of non-domain logic in the form of frameworks, libraries, and custom code, which sometimes cannot be cleanly separated from the domain logic. So, it is nearly impossible to generate test cases for the whole system systematically guided by structural metrics on the source code. In this paper, we propose a specification-based approach to generate test cases. In this approach, the domain logic is specified in a functional notation (based on Gallina). Test cases are generated using a search-based approach where the fitness function is defined in terms of the structural coverage of the specification (measured over an equivalent Haskell implementation). An experiment on an industrial stock exchange trading engine indicates promising results in the effectiveness of our proposed approach.

... 6. REFERENCES [1] JESS (Java Expert System Shell). http://herzberg.ca.sandia.gov/jess/. [2] T. Alspaugh, D. Richardson, T. Standish, and H. Ziv. Scenario-driven specification-based testing against goals and requirements. In REFSQ'05, pages 187–202, 2005. ...

This paper describes the specification-based testing and analysis tools, and associated processes, that were used to develop and certify safety-critical avionics systems in an industrial organization. These tools comprise an integrated development environment supporting specification acquisition and analysis, requirement-based automatic test vector generation, test coverage analysis, test driver generation, and test results analysis. The paper describes the specification model, method, development environment, and tool qualification approach. The capabilities of the automatic test generator are compared with foundational concepts and related testing strategies and mechanisms