ISEB/ISTQB Foundation Certification Syllabus Covers three TEST DESIGN

TECHNIQUES
The 3 categories are:
1) Specification based or Black box testing.
2) Structured-based or White-Box Techniques.
3) Experienced Based Techniques.

TEST DESIGN TECHNIQUES for Structured-based (or) White-Box Techniques are:

-> Statement Testing Coverage
-> Decision Testing Coverage

{Statement Coverage & Decision Coverage : These 2 topics are covered in

ISEB/ISTQB foundation Syllabus in 4th chapter "TEST DESIGN TECHNIQUES".}

Structured-based or White-Box Techniques:

White Box Testing:

• Testing based on Knowledge of internal structure and logic.
• Logic errors and incorrect assumptions are inversely proportional to a path’s
execution probability.
• We often believe that a path is not likely to be executed, but reality is often
counter intuitive.
• Measure Coverage.

Structured-based or White-Box Techniques are based on an identified structure of

the software or system, as seen in the following examples:

Component level: The structure is that of the code itself, i.e., statements, decisions
or branches.
Integration level: The structure may be a call three (a diagram in which modules call
other modules).
System level: the structure may be a menu structure, business process or webpage
structure.

Structure-based or white-box testing can be applied at different levels of testing.

Here we will be focusing on white-box testing at the code level, but it can be applied
wherever we want to test the structure of something - for example ensuring that all
modules in a particular system have been executed.

Now coming to the actual topic:

1. Statement testing & Coverage:

A Statement is:
'An entity in a programming language, which is typically the smallest indivisible unit
of execution' (ISTQB Def).
A Statement coverage is:
'The percentage of executable statements that has been exercised by a test suite'
(ISTQB Def)

Statement coverage:
-> Does not ensure coverage of all functionality
->
The objective if the statement testing is to show that the executable statements
within a program have been executed at least once. An executable statement can be
described as a line of program source code that will carry out some type of action.
For example:

If all statements in a program have been executed by a set of tests then 100%
statement coverage has been achieved. However, if only half of the statement have
been executed by a set of tests then 50% statement coverage has been achieved.

The aim is to achieve the maximum amount of statement coverage with the
minimum number of test cases.

>>Statement testing test cases to execute specific statements, normally to increase

statement coverage.
>>100% statement coverage for a component is achieved by executing all of the
executable statements in that component.

If we require carrying out statement testing, the amount of statement coverage

required for the component should be stated in the test coverage requirements in
the test plan. We should aim to achieve at least the minimum coverage requirements
with our test cases. If 100% statement coverage is not required, then we need to
determine which areas of the component are more important to test by this method.

>>Consider the following lines of code:

>> 1 test would be required to execute all three executable statements.

If our component consists of three lines of code we will execute all with one test
case, thus achieving 100% statement coverage. There is only one way we can
execute the code - starting at line number 1 and finishing at line number 3.

>>statement testing is more complicated when there is logic in the code

>>For example:
>>Here there is one executable statement i.e., "Display error message"
>> hence 1 test is required to execute all executable statements.

Program code becomes tough when logic is introduced. It is likely what a component
will have to carry out different actions depending upon circumstances at the time of
execution. In the code example shown, the component will do different things
depending on whether the age input is less than 17 or if it is 17 and above. With the
statement testing we have to determine the routes through the code we need to take
in order to execute the statements and the input required to get us there!

In this example, the statement will be executed if the age is less than 17, so we
would create a test case accordingly.

>> For more complex logic we could use control flow graphing
>> Control flow graphs consists of nodes, edges and regions

Control flow graphs describes the logic structure if the software programs - it is a
method by which flows through the program logic are charted, using the code itself
rather than the program specification.
Each flow graph has nodes and edges - The nodes represent computational
statements or expressions, and the edges represent transfer of control between the
nodes. Together the nodes and edges encompass an area known as a region.

In the diagram, the structure represents an 'If Then Else Endif' costurct. Nodes are
shown for the 'If' and the 'Endif'. Edges are shown for the 'Then' (the true path) and
the 'Else (the false path). The region is the area enclosed by the nodes and the
edges.

>>All programs consist of these basic structures:

This is hazel notation that only shows logic flow.

There are 4 basic structures that are used withn control-flow graphong.

The 'DoWhile' structure will execute a section of code whilst a field or indicator is set
to a certain value. For example,

The 'Do until' structure will execute a section of code until a field or indicator is set to
a certain value. Foe example,

The evaluation of the condition occurs after the code is executed.

The 'Go To' structure will divert the program execution to the program section in
question. For example

>> SO the logic flow code could now be shown as follows:

If we applied control-flow graphing to our sample code, then 'if Then Else' structure
is applicable.
However, while it shows us the structure of the code, it doesn’t show us where the
executable statements are, and so it doesn’t help us at the moment with determining
the tests we required for statement coverage.

>> we can introduce extra nodes to indicate where the executable statements are

>> And we can see the path we need to travel to execute the statement in the code.

What we can do is introduce extra nodes to indicate where the statements occur in
the program code.
Now in our example we can see that we need to answer 'yes' to the question being
posed to traverse the code and execute the statement on line 2.

>> Now consider this code and control flow graph:

>> We will need 2 tests to achieve 100% statement coverage.

Program logic can be a lot more complicated than the examples I have given so far!
In the source code shown here, we have executable statements associated with each
outcome of the question being asked. We have to display an error message if the
age is less than 17(answering 'yes' to the question), and we have display 'customer
OK' if we answer 'No'.
We can only traverse the code only once with a given test; therefore we require two
tests to achieve 100% statement coverage.

>> And this example...

>> we will need 3 tests to achieve 100% statement coverage.

Now it gets even more complicated!

In this example, we have a supplementary question, or what is known as a 'nested
if'. If we answer 'yes' to 'If fuel tank empty?' we then have a further question asked,
and each outcome of this question has an associated statement.

Therefore we will need two tests that answer 'yes' to 'if fuel tank empty'
* Fuel tank empty AND petrol engine ( to execute line 3)
* Fuel tanl empty AND NOT petrol engine( to execute line 5)
one further test will be required where we answer 'no' to 'if fuel tank empty' to
enable us to execute the statement at line 8.

>>And this will be the last example for statement coverage.. we will then go for
decision coverage.
>> We will need 2 tests to achieve 100% statement coverage.

In this example, we have two separate questions that are being asked.

The tests have shown are

* A coffee drinker who wants cream
* A non coffee drinker who doesn’t want cream

Our 2 tests achieve 100% statement coverage, but equally we could have had 2
tests with:
* A coffee drinker who doesn’t want cream
* A no-coffee drinker who wants cream

If we were being asked to achieve 100% statement coverage, and if all statements
were of equal importance, it wouldn’t matter which set if tests we choose.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Checking your calculation values:

Minimum tests required to achieve 100%

Decision coverage >= Statement coverage

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Decision testing & Coverage

A Decision is :
>> ' A program point at which the control flow has two or more alternative routes. A
node with two or more links to separate branches.'(ISTQB Def)
A Decision Coverage is :
>> ' the percentage if the decision outcomes that have been exercised by a test
suite. 100% decision coverage implies both 100% branch coverage and 100%
statement coverage.' (ISTQB Def)

Decision Coverage :

>>

The objective of decision coverage testing is to show all the decisions within a
component have been executed at least once.
A decision can be described as a line of source code that asks a question.
For example:

If all decisions within a component have exercised by a given set of tests then 100%
decision coverage has been achieved. However if only half of the decisions have been
taken with a given set of tests then you have only achieved 50% decision coverage.
Again, as with statement testing, the aim is to achieve the maximum amount of
coverage with the minimum number of tests.

>> Decision testing derives test cases to execute specific decision outcomes,
normally to increase decision coverage.
>> Decision testing is a form of control flow testing as it generates a specific flow of
control through the decision points.

If we are required to carry out decision testing, the amount of decision coverage
required for a component should be stated in the test requirements in the test plan.
We should aim to achieve atleast the minimum coverage requirements with our test
cases. If 100% decision coverage is not required, then we need to determine which
areas of the component are more important to test by this method.

>> Decision coverage is stronger than statement coverage.

>> 100% decision coverage for a component is achieved by exercising all decision
outcomes in the component.
>> 100% decision coverage guarantees 100% statement coverage, but not vice
versa.

Decision testing can be considered as the next logical progression from statement
testing in that we are not much concerned with testing every statement but the true
and false outcomes from every decision.
As we saw in our earlier examples of the statement testing, not every decision
outcome has a statement (or statements) to execute.
If we achieve 100% decision coverage, we would have executed every outcome of
every decision, regardless of whether there were associated statements or not..

>>Lets take earlier example we had for statement testing:

>>This would require 2 tests to achieve 100% decision coverage, but only 1 test to
achieve 100% statement coverage.

In this example there is one decision, and therefore 2 outcomes.

To achieve 100% decision coverage we could have two tests:
* Age less than 17(answer 'yes')
* Age equal to or greater than 17 (answer 'no')
This is a greater number of tests than would be required for statement testing as
statements are only associated with one decision outcome(line 2).

>> Again, consider this earlier example :

>> we will need 2 tests to achieve 100% decision coverage & also 2 tests to achieve
100% statement coverage.

This example would still result in two tests, as there is one decision therefore 2
outcomes to tests.
However, we would need two tests to achieve 100% statement coverage, as there
are statements with each outcome of the decision.
So, in this instance, statement and decision testing would give us the same number
of tests. Note that if 100% coverage is required, statement testing can give us the
same number of tests as decision testing, BUT NEVER MORE!

>>Lets look at some more examples now..

>> We will need 3 tests to achieve 100% decision coverage, but only 1 test to
achieve 100% statement coverage.

Here we have an example of a supplementary question, or a 'nested if'.

We have 2 decisions, so you may think that 4 tests may be required to achieve
100% decision coverage (two for each decision).
This is NOT the case! We can achieve 100% decision coverage with three tests - we
need to exercise the 'Yes' outcome from the first decision ( line 1) twice, in order to
subsequently exercise the 'Yes' and then the 'No' outcome from the supplementary
question(line 2).
We need a further third test to ensure we exercise the 'No' outcome of the first
decision (line 1).
There is only one decision outcome that has an associated statement - this means
that 100% statement coverage can be achieved with one test.

>> As more statements are added, the tests for decision coverage are the same:

>> 3 tests to achieve 100% decision coverage, and 2 tests to achieve 100%
statement coverage.

We have now introduced a statement that is associated with 'No' outcome of the
decision on line 2.
This change affects the number of tests required to achieve 100% statement
coverage, but does NOT alter the number of tests required to achieved 100%
decision coverage - it is still three!

>And again an example..

>> 3 tests to achieve both decision and statement coverage.

Finally, we have statements associated with each outcome of each decision - the
number of tests to achieve 100% statement coverage and 100% decision coverage
are now the same.

>> And Last Example:

>> We will need 2 tests to achieve 100% decision coverage and 100% statement
coverage.

We looked at this example of the "if Then Else' structure when considering statement
testing.
As the decisions are separate questions we only need two tests to achieve 100%
decision coverage( the same as the number required for statement coverage).
You may have thought that four tests were required - exercising the four different
routes through the code, but remember, with decision testing our concern is to
exercise each outcome of each decision at least once - as long as we have answered
'Yes' and 'No' to each decision we have satisfied the requirements of the technique.
The tests we have illustrated would need the following input conditions:
* Coffee drinker wanting cream.
* Non Coffee drinker not wanting cream (but milk).

Equally, we could have chosen the following input conditions:

* Coffee drinker not wanting cream ( but milk).
* Non coffee drinker wanting cream.

>> Then what about loops?

>> If we choose an initial value of p=4, we only need 1 test to achieve 100%
statement and 100% decision coverage.

The control-flow graphs we showed earlier depicted a 'Do While' construct.

To reiterate, thw 'Do While' structure will execute a section of code whist a field or
indicator is set to a certain value. For example,

The evaluation of the condition occurs before the code is executed.

Unlike the 'If Then Else', we can loop around the 'Do While' structure, which means
that we exercise different routes through the code with one test.
As in the above diagram, if we set 'p' with an initial value '4', the first time through
the code will :
* Go from line 1 to line 2
* Answer 'Yes' to the 'If' on line 2 ( if p<5)
* Execute the statement in line 3 (p=p*2, so p now equals 8)
* Go from line 3, through line 4 to line 5
* Execute the statement on line 5 ( which adds 1 to 'p', making it`s value '9')
* Execute the statement on line 6, which takes it back up to line 1.

Again we execute the code, with the value of 'P' now '9'
* GO from line1 to line2
* Answer 'NO' to the 'if' on line 2 (If p>5)
* Go from line 4 to line 5
* Execute the statement on line 5(which adds 1 to 'p', making it’s value '10')
* Execute the statement on line 6, which takes it back up to line 1.

Once more we execute the code

* Line 1 - 'P' is not less than '10' (it is equal to 10), therefore, we exit this structure.

1 test - it achieves 100% statement coverage and 100% decision coverage.

>> And it’s same for 'Do until' structure

>> IF we choose an initial value of A =15, we only need 1 test to achieve 100%
Decision coverage and 100% statement coverage.

The control flow structures we showed earlier also depicted a 'Do Until' structure.
To reiterate, the 'Do Until' structure will execute a section of code until a field or
indicator is set to a certain value. For example,

The evaluation of the condition occurs after the code is executed.

Unlike the 'If Then Else', we can loop around the 'Do Until' structure, which means
that we exercise different routes through the code with one test.
In the example above, If we set 'A' with an initial value of '15', the first time through
the code will:
* Go from line 1 to line 2
* Answer 'Yes' to the 'If' on line 2 (If A<20)
* Execute the statement on line 3 ( A=A*2,which makes A=30)
* GO from line 3, through the line 4 to line 5
* Execute the statement on line 5( which adds 1 to 'A', making its value '31'.
* Execute the statement on line 6, Which takes back to line 1

Again we execute the code, with the value of 'A' now '31'
* Go from line 1 to line 2
* Answer 'No' to the 'If' on line 2 (If A < 20)
* Go from line 2, through line 4 to line 5
* Execute the statement on line 5( which adds 1 to 'A', making it`s value '32')
* Execute the statement on line 6, which exits the structure('A' is greater than 31)

1 test - it achieves 100% statement coverage and 100% decision coverage.

END...