ĐẠI HỌC FPT CẦN THƠ

SOFTWARE TESTING
 ĐẠI HỌC FPT CẦN THƠ
Software Testing Chapter 4
Foundation Of Software Testing

1 Principles 2 Lifecycle 3 Static testing

4 Test design
5 Management 6 Tools
techniques

Test Design Techniques

 Content

1
• The test development process

2
• Categories of test design techniques

3
• Black and White box testing

4
• Black box test techniques

5
• White box test techniques

6
• Experience-based techniques

7
• Choosing a test technique

<tên topic><Tên GV> 3

 White Box test design and measurement
techniques
 Techniques defined in BS 7925-2
Statement testing Also a measurement
technique?
Branch / Decision testing = Yes
Data flow testing = No

Branch condition testing

Branch condition combination testing
Modified condition decision testing
LCSAJ testing
 Also defines how to specify other techniques
 Introduction
White-Box Testing

Testing based on analysis of internal logic (design, code,

etc.). (But expected results still come from requirements.)
Also know as structural testing.
White-box testing concerns techniques for designing
tests; it is not a level of testing.
White-box testing techniques apply primarily to lower levels
of testing (e.g., unit and component).
 Introduction
White-Box Testing

Benefits:
Testers can identify defects that cannot be detected through other
testing techniques.
Testers can create more comprehensive and effective test cases that
cover all code paths.
Testers can ensure that the code meets coding standards and is
optimized for performance.

Disadvantages:
Complexity.
Not Suitable for Large Applications.
Limited Comprehensive Testing.
 Introduction
Common approaches
Statement coverage :
Each statement executed at least once
Branch/Decision coverage :
Each branch traversed (and every entry point taken) at least
once
Condition coverage:
Each condition True at least once and False at least once
 Control-flow-based Testing

A traditional form of white-box testing

Step 1: From the source, create a graph describing
the flow of control
Called the control flow graph
The graph is created (extracted from the source
code) manually or automatically
Step 2: Design test cases to cover certain elements
of this graph
Nodes, edges, paths (types of logic coverage)

8
 Control Flow Graphs (CFGs)

Nodes in the control flow graph are basic blocks

A basic block is a sequence of statements always
entered at the beginning of the block and exited at
the end
Edges in the
if (x < y)control
{ flow graph representB the
(x control
< y) 0 flow
x = 5 * y; Y N
x = x + 3;
} x = 5 * y B1 B2 y = 5
else x = x + 3
y = 5;
x = x+y;

x = x+y B3
• Each block has a sequence of statements
• No jump from or to the middle of the block
• Once a block starts executing, it will execute till the end
 Example of a Control Flow Graph (CFG)

s:=0; 1
d:=0;
while (x<y) { 2
x:=x+3;
y:=y+2; 3

if (x+y < 100) 4

s:=s+x+y; 5 6
else 7
d:=d+x-y;
8
End if
}
11
CFG representations for Program
Constructs
 What is coverage?

Example:
Program has 100 statements
Tests exercise 87 statements
Statement coverage = 87%
A coverage item is whatever we have been able to count
and see whether a test has exercised or used this item.
NB! 100% coverage does not mean that 100% tested!
 Using structural coverage

Spe Enough
c Software tests?
Tests
Results OK?
What's
covered
More tests ? Coverage OK?

Stronger structural
techniques (different
structural elements)

Increasing coverage
 The test coverage trap

Function exercised, better testing

insufficient structure
Functional
testedness

Structure exercised,
insufficient function

% Statement % Decision % Condition

Combination
Structural testedness

100% coverage does Coverage is not

not mean 100% tested! Thoroughness
Statement Coverage
 Statement Coverage

Statement Coverage requires that each statement will

have been executed at least once.
It does not require any detailed analysis of the condition
points or exercising each decision.
Example 1: public int test01(int a,
a
int x) {
if (a > 0)
x = x + 1; a>0 c
return x;
} b x=x+1

ID Input path Expected Measure

output
1 (1, 2) acb 3 100%

Coverage Measure
= number of executed statements/ total number of statements
 Statement Coverage
Example
Consider this example: a
public int test02( int a,
a>1
Y
int b, int x) { AND
b==0
c
if (a>1 && b==0) {
x = x/a; b N

} x = x/a

if (a==2 || x>1) {
a==2
x = x+1; OR
x>1
Y
e
}
return x; d N

} x=x+1
ID Input path Expected
output
1 (2, 0, 6) acef 4
f

All statements covered for path: aced

This is considered the weakest form of coverage
What is untested in this example?
Paths: acdf, abef, abdf
Not acceptable to release code based on statement coverage alone.
Statement Coverage
Example
 Statement coverage

 percentage of executable Statement coverage

is normally measured
statements exercised by a test suiteby a software tool.
= number of statements exercised
total number of statements
 example: ?
program has 100 statements
tests exercise 87 statements
statement coverage = 87%

Typical ad hoc testing achieves 60 - 75%

 Example of statement coverage

1 read(a) Test Input Expected

2 IF a > 6 THEN case output
3 b=a
1 7 7
4 ENDIF
5 print b

As all 5 statements are ‘covered’ by

this test case, we have achieved
100% statement coverage
Statement
numbers
Branch/Decision Coverage
 Branch/Decision Coverage

Decision coverage (also known as branch coverage) reports

whether boolean expressions in control structures are
evaluated to both true and false values by the test cases.
Note in decision coverage we are not interested in all combinations
of True/False values of all program predicates, just that we have
exercised all the different values in our test cases.
The entire boolean expression is considered one true-or-false
predicate.
A decision is an IF statement, a loop control statement (e.g.
DO-WHILE or REPEAT-UNTIL), or a CASE statement, where
there are two or more outcomes from the statement.
With an IF statement, the exit can either be TRUE or FALSE,
depending on the value of the logical condition that comes after IF.
 Branch/Decision Coverage
Example 2:
public int test02( int a, int a
b, int x) { a>1 Y
if (a>1 && b==0) { AND
b==0
c
x = x/a;
N
}
if (a==2 || x>1) { b x = x/a

x = x+1;
a==2
} OR
x>1
Y
e
return x;
N
}
ID Input path Expected
output d x=x+1

1 (2, 0, 6) acef 4
2 (4, 3, 1) abdf 1
f
Branch or decision coverage is to test each branch at least once.
All branches covered for paths
ACEF & ABDF, OR
ABEF & ACDF
Note: without detailed analysis of the condition causing the branch.
 Branch/Decision Coverage

public int test02( int ID Input path Expected

a, int b, int x) { output
if (a>1 && b==0) { 1 (2, 0, 6) acef 4
x = x/a; 2 (4, 3, 1) abdf 1
}
if (a==2 || x>1) {
x = x+1;
} a a
return x; a>1
Y
}
a>1
AND
Y
c AND
b==0
c
b==0
N
b N

x = x/a
b x = x/a

a==2
a==2
OR Y
e OR
x>1
Y
e
x>1

d N
N

d x=x+1
x=x+1

f f
 Branch/Decision Coverage
a
Example 3: ID: 3
Y
ID: 1

public String test03(int a,

value > 0 c
int b) { N

int value= a + b; b ID: 2 red

String result=null;
e
if (value > 0) value < Y
0
result="red";
N
else if (value < 0)
result="blue"; d blue

else
result="green"; green

return result;
}
f
ID Input path Expected output

1 (4, 3) acf red

2 -4, -3 abef blue
3 3, -3 abdf green
Branch/Decision Coverage
 Paths through code
1234

12 12 123 ?

? ? ? ?

?
 Paths through code with loops

1 2 3 4 5 6 7 8 ….

for as many times as it

is possible to go round
? the loop (this can be
unlimited, i.e. infinite)
 Example 1
Wait

Wait for card to be inserted Valid

Yes
Display
IF card is a valid card THEN card? “Enter..
display “Enter PIN number” No
IF PIN is valid THEN
Reject Valid Yes Select
select transaction
card PIN? trans...
ELSE (otherwise) No
display “PIN invalid”
ELSE (otherwise) Display
reject card “PIN in..
End

End
 Example 2
Read

Read A Yes Yes

A>0 A=21
IF A > 0 THEN
IF A = 21 THEN No No
Print
Print “Key”
ENDIF
ENDIF

End
3
Cyclomatic complexity: _____
Minimum tests to achieve:
1
Statement coverage: ______
3
Branch coverage: _____
 Example 3
Read

Read A Yes No
Read B A>0 B=0 Print
IF A > 0 THEN No Yes
Yes
IF B = 0 THEN Print A>21 Print
Print “No values”
No
ELSE
Print B
IF A > 21 THEN End
Print A 4
Cyclomatic complexity: _____
ENDIF
ENDIF Minimum tests to achieve:
ENDIF Statement coverage: ______
2
4
Branch coverage: _____
 Yes
Example 4
Read A<0 Print
No

Read A Note: there Print

Read B are 4 paths
IF A < 0 THEN Yes
Print “A negative” B<0 Print
ELSE No
Print “A positive” Print
ENDIF
IF B < 0 THEN
Print “B negative” End
ELSE 3
Cyclomatic complexity: _____
Print “B positive”
ENDIF Minimum tests to achieve:
2
Statement coverage: ______
2
Branch coverage: _____
 Example 5

Yes
Read A<0 Print
Read A No
Read B
IF A < 0 THEN
Yes
Print “A negative” B<0 Print
ENDIF No
IF B < 0 THEN
Print “B negative”
ENDIF End

3
Cyclomatic complexity: _____
Minimum tests to achieve:
1
Statement coverage: ______
2
Branch coverage: _____
 Example 6
Yes
Read A<0 Print
No

Read A
Yes
IF A < 0 THEN A>0 Print
Print “A negative”
No
ENDIF
IF A > 0 THEN
Print “A positive” End
ENDIF

3
Cyclomatic complexity: _____
Minimum tests to achieve:
2
Statement coverage: ______
2
Branch coverage: _____
 Decision coverage
(Branch coverage)
Decision coverage
 percentage of decision outcomesis normally measured
exercised by a test suite by a software tool.

= number of decisions outcomes exercised

total number of decision outcomes
 example: False
?
program has 120 decision outcomes True
tests exercise 60 decision outcomes
decision coverage = 50%

Typical ad hoc testing achieves 40 - 60%

 Condition Coverage

A branch predicate may have

more than one condition.
input(X,Y)
if (Y<=0) or (X=0) then
Y := -Y
end_if
while (Y>0) and (not EOF) do
input(X)
Y := Y-1
end_while
 Condition Coverage (cont’d)

Condition Coverage requires that

each condition will have been True
at least once and False at least
once.
What is the relationship between
Branch and Condition Coverage?
 Condition Coverage (cont’d)

if A or B then
A B Branch
s1
test 1 T F true
else
s2 test 2 F F false
end_if_then_el
se
 Condition Coverage (cont’d)

if A or B then
A B Branch
s1
else test 3 T F true

s2 test 4 F T true
end_if_then_el
se
 Condition coverage
This is closely related to decision coverage but has better sensitivity to
the control flow.
Condition coverage reports the true or false outcome of each condition.
Condition coverage ensures that every possible condition is executed at
least once
public int test04(int a, int b,
int x) {
a
a < 0
if (a < 0 && b > 20) Y
x = x + 1;
&& b >
20
c
else N

x = x * 2; b red

return x; blue
}
ID a<0 b>20 x Expected output
1 True -1 True 21 2 3 N

2 True -1 False 19 2 4 d
3 False 1 True 21 2 4
4 False 1 False 19 4 8
 Condition coverage
Example 2
Each condition in a decision outcome covered by a test case
This is often considered impossible to achieve this form of
coverage a
Test cases cover 8 combinations: a>1
AND
Y
c
a>1, b=0 T
b==0

N
a>1, b!=0 T F
b
F T
a<=1, b=0 F
x = x/a

a<=1, b!=0 a, b, x Path a==2

OR
Y
e
True a>1 , True b==0 2, 0 AC
a=2, x>1 x>1
True a>1 , False b!=0 2, 1 AB N
a=2, x<=1 False a<=1, True b==0 0, 0 AB d x=x+1
a!=2, x>1 False a<=1, False b!=0 0, 1 AB
True a==2, True x>1 2, , 2 EF
a!=2, x<=1 True a==2, False x<=1 2, ,0 EF
f
False a!=2, True x>1 3, ,2 EF
42
False a!=2, False x<=1 3, ,0 DF
Condition Coverage
Example
Example
Example
Example
Example
Example
Example
Cyclomatic Complexity (CC)
 Cyclomatic Complexity (CC)
Evaluates the complexity of an algorithm in a method.
It measures the number of linearly independent paths
through a program's source code
Three equivalent definitions
V(G) = E-N+2
Or given a flow graph: = edges – nodes + 2
V(G) = P + 1 (Where P = number of predicate nodes)
Defined to be one larger than the number of decision points
(if/case-statements, while-statements, etc.) in a module
(function, procedure, chart node, etc.).
– Note that in an if or while statement, a complex boolean counts
each part, (e.g. if( a<b and c>0) counts as two decision
points.
V(G) = R (Where R = number of regions)
 Cyclomatic Complexity (CC)

1-10 11-20

Simple procedure More complex

Little risk Moderate risk

21-50 >50

Complex Untestable
High risk Very high risk
 Cyclomatic Complexity (CC)

V(G) = P+1= 2+1=3 or V(G) = E-N+2 = 7-6+2=3

 Cyclomatic Complexity (CC)

V(G) = E-N+2 = 9-8+2=3

 Cyclomatic Complexity Examples

statement1
statement2
if (x > 5) {
… V(G) = 1 V(G) = 2
statement1
statementn }

if (x > 5) {
statement1
} else {
V(G) = 2
statement2
}
 Cyclomatic Complexity Examples

switch (exp) {
case v1: statement1; break;
case v2: statement2; break;
if (x > 5 || x <= 10) {
…
statement1
case vn: statementn; break; V(G) = 3
}
default: statementn+1;
} if (x >= 0 && x <= 100) {
statement1
V(G) = n + 1 } else if (x <= 30) {
V(G) = 4 statement1
} else {
statement2
}
 Cyclomatic Complexity Examples

while (i < 100) {

statement1;
}
for (i = 0; i < 10; i++) {
V(G) = 2 statement1
V(G) = 2
}

for (i = 0; i <= 10; i++) {

while (i < 100 && a != null) {
statement1;
statement1;
if (i > 5 || i <= 10) {
}
V(G) = 2 statement2
V(G) = 4 }
}
 Content

1
• The test development process

2
• Categories of test design techniques

3
• Black and White box testing

4
• Black box test techniques

5
• White box test techniques

6
• Experience-based techniques

7
• Choosing a test technique

<tên topic><Tên GV> 59

Experience-based techniques

60
 Experience-based techniques

Error guessing and fault attacks: Error guessing is

a technique that should always be used as a
complement to other more formal techniques.
Exploratory testing: is a hands-on approach in
which testers are involved in minimum planning
and maximum test execution.
 Experience-based techniques

Test cases are derived from the tester’s skill and intuition,
and their experience with similar applications and
technologies.
Can be helpful in identifying tests that were not easily
identified by other more systematic techniques.
Depending on tester’s approach and experience, these
techniques may achieve widely varying degrees of coverage
and effectiveness.
Coverage can be difficult to assess
and may be unmeasurable with these
techniques.
 Error Guessing

A methodical approach to the error guessing technique

is to create a list of possible mistakes, defects & failures
 design tests that will expose those failures & the
defects that caused them.
These mistake, defect, failure lists can be built based on
experience, defect and failure data, or from common
knowledge about why software fails.
 Exploratory Testing

A hands-on approach in which testers are involved in

minimum planning and maximum test execution.
Test design & execution activities are performed in
parallel, often without formal documentation.
Informal tests are designed, executed, logged, and
evaluated dynamically during test execution.
The test results are used to learn more about the
component or system, and to create tests for the areas
that may need more testing.
 Exploratory Testing

Exploratory testing is sometimes conducted using

session-based testing to structure the activity.
In session-based testing, exploratory testing is
conducted within a defined time-box, and the
tester uses a test charter containing test
objectives to guide the testing.
Exploratory testing is most useful when there are
few or inadequate specifications or significant
time pressure on testing.
 Checklist-based Testing

Experienced tester uses a checklist to design,

implement and execute tests based on the items or
test conditions in the list.
As part of analysis, testers create a new checklist or
expand an existing checklist, but testers may also use
an existing checklist without modification.
Such checklists can be built based on:
experience of the tester
knowledge about what is important for the user
understanding of why and how software fails
 Content

1
• The test development process

2
• Categories of test design techniques

3
• Black and White box testing

4
• Black box test techniques

5
• White box test techniques

6
• Experience-based techniques

7
• Choosing a test technique

<tên topic><Tên GV> 67

 Choosing test techniques

The choice of which test techniques to use depends

on a number of factors, including:
 Choosing test techniques

Formality. The use of techniques can vary in formality,

depending on several factors:
Safety or regulatory industries
The maturity of the organisation
The life cycle model
Knowledge and skills of testers
 Choosing test techniques

Factors influencing the decision about which

technique to use:
Type of component / system
Component / system complexity
Regulatory standards
Customer or contractual requirements
Risk levels and Risk types
Test objectives
Available documentation
 Choosing test techniques

Factors influencing the decision about which technique to

use (continued):
Tester knowledge & skills
Available tools
Time & budget
SDLC model
Expected use of the software
Previous experience with using the test techniques
Expected types of defects
 Choosing a test technique

 The internal factors that influence the decision about

which technique to use are:
Models used
Tester knowledge/experience
Likely defects
Test objective
Documentation
Life cycle model
 Choosing a test technique

The external factors that influence the decision about

which technique to use are:
Risk
Customer/contractual requirements
Type of system
Regulatory requirements
Time and budget
ĐẠI HỌC FPT CẦN THƠ