ISTQB Certified Tester Foundation Level: - Module 4 of 6: Test Design Techniques
ISTQB Certified Tester Foundation Level: - Module 4 of 6: Test Design Techniques
ISTQB Certified Tester Foundation Level: - Module 4 of 6: Test Design Techniques
February, 2017
Study Sessions
• These study packs are intended as sessions directly from the chapters in the
syllabus
2
Covered in this session...
3
Test Development Process
Test Development Process
The process can be Remember
Remember Principle
Principle 6
6 –– Testing
Testing is
is context
context
• Very informal with little or no dependent
dependent
documentation E.g. May test differently on web
• Very formal with lots of info page vs. Safety critical
breaking system
documentation
5
How would you test software without requirements?
User
Manual Heuristics
What we
Experience
d User know of
software
Software
6
How would you test software with 1,000 documents?
Business
Business Technical
Technical Architecture
Architecture
Use
Use Cases
Cases Process
Process Flows
Flows Transactions
Transactions
Requirements
Requirements Requirements
Requirements Diagrams
Diagrams
7
Test Design and the Fundamental Test Process
Planning
Planning and
and
Control
Control
Evaluate
Evaluate Exit
Exit
Criteria and
Criteria and
Reporting
Reporting
Test
Test Closure
Closure
8
Test Development Process
9
Step 1: Analyse the Test Basis
• Prioritise
• Which test basis to use for testing first?
10
Step 2: Identify Test Conditions
• Prioritise
• Ensure the most important test conditions are identified
• Traceability
• From test conditions back to specifications and requirements
• Enables impact analysis when things change
• Helps to determine requirements coverage in both planning and execution
11
What is a Test Condition?
• Testers usually brainstorm all the possible test conditions for the requirement/feature that they
are testing
• Then we pick the most appropriate test conditions and build into test cases
12
Examples of Test Conditions
Employee Record
13
Step 3: Identify Test Cases
14
What is a Test Case?
15
Step 4: Develop the Test Procedure Specification
Test
• Specifies sequence of action Procedure
1
• Manual test script
• Automated test procedure when script
1.
executed by a tool 2.
3.
4.
5.
• Prioritise Test Procedure Specifications
• Want to execute the most important ones first!
16
Step 5: Develop the Test Execution Schedule
Other
Other structure-based
structure-based
Techniques
White-box
techniques
techniques
Structure-based
Decision
Decision testing
testing and
and
coverage
coverage
Statement
Statement testing
testing and
and
coverage
coverage
Exploratory
Exploratory testing
testing approach
approach
Experienced-
Fault
Fault attack
attack
based
Black-box Techniques
Categories of Test Design Techniques
Error
Error guessing
guessing
Use
Use case
case testing
testing
Specification-based
State
State transition
transition testing
testing
Decision
Decision table
table testing
testing
Boundary
Boundary value
value analysis
analysis
Equivalence
Equivalence partitioning
partitioning
Why use test design techniques?
• Consistency
• “Best” practice
• Systematic
• Testers may have different levels of experience
• Save time
• Impossible to test all permutations, therefore we have to choose what we will test
Remember
Remember Principle
Principle 2
2 –– Exhaustive
Exhaustive testing
testing is
is
impossible
impossible
20
Specification-based or black-box techniques
• Systematically derive tests from formal or informal models (descriptions) used for the specification
of the software
• Analysis of the test basis
• Based on documentation or what we know
• “what” the system should do
• Without reference to its internal structure
21
Structure-based or white-box techniques
• The extent of test coverage can be measured (by a tool) for existing test cases and further test
cases can be derived to increase coverage
00
22
Experienced-based techniques
23
Specification-based (Black-box) Techniques
Equivalence Partitioning
Equivalence Partitioning
• Inputs are divided into groups that are expected to behave similar
25
Equivalence Partitioning
Activity
• Let’s derive the equivalence classes for an application that lets you apply for your driver’s license
• In Victoria, Australia you must be at least 18 yrs. old to apply for your probationary license
• Let’s say theoretically you must stop driving when you are 90 yrs. old
17 18 89 90
Others?
Others?
Non-
Non-
integer
integer
26
Specification-based (Black-box) Techniques
28
Boundary Value Analysis
Back to our activity…
• In Victoria you must be at least 18 yrs. old to apply for your probationary license
• Let’s say theoretically you must stop driving when you are 90 yrs Old
17 18 89 90
29
Why do both EP and BVA?
• If you test only BVA and you find an error, you have to test more options anyway – is just the
boundary wrong or the whole partition?
• But could be useful if there are time pressures and you are looking to find defects first, so test
boundaries only.
• If you do boundaries only, you have covered all the partitions as well.
• Boundary testing requires a lot more tests and may be expensive to set up.
30
Question: EP
• An airline offers pricing as follows; infants < 2 yrs. travel free, children aged between 2-11 yrs. at
75% of adult fare and adults at full price.
• Which of the following are age values from different VALID partitions for airfare pricing?
a) 0, 2, 10
b) -1, 1, 3
c) 1, 7, 45
d) 2, 9, 12
31
Question: 2-BVA
• An electricity company charges usage <1000 kW at $0.13 per kW, between 1000 and 5000 kW
inclusive at $0.22 per kW and >5001 kW at $0.28 per kW
32
Question: 3-BVA
• A bank’s saving account earns an additional 2% interest when the balance of the account is
greater than $10,000
33
Specification-based (Black-box) Techniques
Decision Tables
Decision Table Testing
• The strength of decision tables is that it creates combinations of conditions, inputs situations and
events that might be missed
• The top part of the table contains combinations of input conditions
• The bottom part contains the resultant actions as True or False (Boolean)
• Each column can then be considered as a test case of a business rule
• Coverage standard could be at least one test for each column to trigger all combinations of
conditions.
Exam Tip: Decision Table Testing is good for
testing business rules or combinations
35
Example: Shipping an online order
Conditions
• Cart has something in it
• Shipping address is a PO box
• Account balance is in credit
Actions
• Item is shipped
• Account is debited
36
Construct the decision table
Spec
Spec doesn’t
doesn’t
say
say
Not
Not feasible,
feasible, so
so Same
Same resulting
resulting
actions… Each
Each column
column is
is aa
remove?
remove? actions… do
do
conditions matter? test
test case
case
conditions matter?
37
Rationalise the decision table
38
Question: Decision table
39
Specification-based (Black-box) Techniques
money inserted
ask for selection invalid selection
beep
Model shows:
• states the software can occupy (the circles)
• transitions between the states (the lines)
• events that cause the transitions
• actions that result from the transition
42
State Transition Testing
• Different events may trigger a specific action and transition to another state of the system.
• Tests can be designed to cover every state, a typical sequence, every transition or to test invalid
transitions
43
State Transition Testing – Vending Machine Example
Cancel S6
S1 Suspend
start Password Acct
invalid
S2 S3 S4 Password
Enter Password
1st 2nd 3rd invalid
Username invalid
attempt attempt attempt
45
State Transition Table
Cancel Password S6
S1
S2 S3 invalid S4 Suspend
Start
1st 2nd 3rd Acct
Password
Enter attempt attempt attempt
invalid Password
Username Password
Password invalid
Password valid
valid
valid S5
Access
Acct
47
Specification-based (Black-box) Techniques
49
Use Case – Table Format
50
Use Case – Table Format Example
51
Use Case – Diagram Format
Source: http://argouml-stats.tigris.org/nonav/documentation/manual-0.22/ch04s03.html
Actors: Bank Engineer, Customer, Loan Bank Official and Central Computer
Further Use Cases e.g. for Withdraw Cash for more detail on this transaction
52
Structure-based (White-box) Techniques
54
Statement Testing and Coverage
Test Coverage
• Here we devise test suites with information on how many statements they
exercise to achieve a coverage target
• Usually involves a tool that measures statement coverage during test execution
55
How to draw a control flow diagram
IF
• Decisions (e.g. IF) are represented by diamonds
• Be consistent
• Eg. Always draw to the right for “true” decisions
56
Statement Testing and Coverage: Example 1
Qty
Question: How many statements are
exercised if we set the input value of
1. Read Qty “Qty” to 10?
IF
True Answer = 4 out of 5
2. IF Qty > 50 >50 statements
Statement Coverage = 80%
Else
X=Qty
*0.8
4. End If
Print X
5. Print Qty
57
Statement Testing and Coverage: Example 2
1. Read Qty Qty Question: How many statements
are exercised if we set the input
value of “Qty” to 100?
2. IF Qty > 50
True
IF Answer = 5 out of 5 statements
>50 Statement Coverage = 100%!!!
Else
X=Qty
3. Qty = Qty * 0.8 *0.8
End If
4. End If
Print X
5. Print Qty
But wait... Are we missing something?
58
Structure-based (White-box) Techniques
• Decision coverage is the assessment of the percentage of decision outcomes exercised (e.g. the
True and False options of the IF statement)
• 100% decision coverage guarantees 100% statement coverage, but not vice versa
60
Decision Testing and Coverage
1. Read Qty Qty Question 1: How many tests
are required to achieve 100%
statement coverage?
True
Answer = 1 Test
2. IF Qty > 50 IF e.g. Value Qty = 100
>50
Tip:
Tip: 100%
100% Decision
Decision Coverage
Coverage means
means cover
cover all
all of
of
5. Print Qty the lines!
the lines!
Print X
Tip:
Tip: 100%
100% Decision
Decision Coverage
Coverage is
is always
always == or
or >> nbr
nbr of
of tests
tests needed
needed for
for 100%
100% Statement
Statement coverage.
coverage. Never
Never less
less
than!
than!
61
Decision Testing: IF with Else Question 1: How many tests are
Qty
required to achieve 100%
statement coverage?
1. Read Qty IF
>50
True Answer = 2 Tests,
2. IF Qty > 50 e.g. values 1 and 100
Else X=Qty
3. Qty = Qty * 0.8 *0.8
62
Decision Testing: Nested IF Statements
1. Read Qty
2. IF Qty > 50 Have a go at drawing the
3. X = Qty * 0.8 control flow diagram in
4. Else your notes
5. IF Qty < 0
6. Print “Error”
7. Else
8. X = Qty * 2
9. End If
10. End If
11. Print X
63
Decision Testing: Nested IF Statements
Qty Question 1: How many tests
are required to achieve 100%
IF
True statement coverage?
1. Read Qty >50
2. IF Qty > 50 Answer = 3 Tests,
Else
e.g. Values 100, -1
3. X = Qty * 0.8 and 25
True
IF Print X=Qty
4. Else “Error” *0.8
<0
5. IF Qty < 0 Else
6. Print “Error” X=Qty Question 2: How many
7. Else *2
tests are required to
8. X = Qty * 2 End If achieve 100% decision
9. End If coverage?
10. End If End If Answer = 3
11. Print X
Print X
64
Decision Testing: WHILE LOOP
Read
Question 1: How many tests
are required to achieve 100%
1. Read Nbr_Exams statement coverage?
WHILE
65
WHILE LOOP with nested IF Statement
Read Question 1: Minimum tests
1. Read Number of Exams
to achieve 100% statement
2. WHILE more exams DO
coverage?
WHILE
False
Answer = 1 Test
3. Mark answers True
7. End IF
End If
Question 1: Minimum tests
8. END WHILE to achieve 100%
End
9. Class Result = Fail/Exams WHILE statement coverage?
10. IF Fail < 30% Class Answer = 2
Result
11. Print “Smart Class”
Question 2: Minimum tests
12. ELSE IF < 30 Smart to achieve 100% decision
13. Print “Poor Trainer” coverage?
14. END IF Poor
Trainer
Answer = 2
END IF
67
Other Structure-based Techniques
• We have been looking at detailed low-level coverage, but coverage can also be
applied to higher-levels like…
• System Testing: % of menu items covered by test cases
• Integration Testing: % components exercised
68
Experienced-based techniques
• Utilising tester’s skill, intuition and experience with similar applications or technologies
• Useful when complementing systematic techniques and when special tests not easily captured by
formal techniques
• Error Guessing
• Anticipate defects based on experience
• Common knowledge of system, data and why system fails
• Or develop a list of possible errors and design tests to systemically attack these errors: this
approach is called Fault Attack
70
Exploratory Testing
• Concurrent test design, execution, logging and learning how the system behaves
71
Exploratory Testing
72
Choosing Techniques
Choosing Test Techniques
74
Question 1: Which techniques would you use?
Scenario Techniques
• Banking application
• Previous problems with Equivalence Partitioning
rounding errors Boundary Value Analysis
• Lots of combinations
State transition testing
of possible factors
• Good documentation Decision tables
• Experienced testers Use case testing
• Time constraints
Statement testing
Decision testing
Error guessing
Exploratory testing
75
Question 2: Which techniques would you use?
Scenario Techniques
• Web site
• Little documentation Equivalence Partitioning
• Previous problems with navigation Boundary Value Analysis
• Experienced testers
State transition testing
• Time constraints
Decision tables
Use case testing
Statement testing
Decision testing
Error guessing
Exploratory testing
76
Question 3: Which techniques would you use?
Scenario Techniques
• Train automatic braking system
• Lots of documentation Equivalence Partitioning
• Previous problems with Boundary Value Analysis
untested combinations
State transition testing
• Safety standards require
decision coverage Decision tables
• High risk Use case testing
• Experienced testers
Statement testing
Decision testing
Error guessing
Exploratory testing
77
End of Module 4
QUESTIONS
End of Module Learning Check
79
ISTQB Practice Exam Question 1
One of the coverage goals for the project is 100% decision coverage. The following 3 tests have been
executed for the control flow graph below. A
80
ISTQB Practice Exam Question 2
A bank application determines the creditworthiness of customers.
The application uses a set of rules to determine the upper limit of the
credit amount. Which of the following black-box test design
techniques is best for testing the application?
81