White Box Testing, Experience Based Techniques

White Box Testing ,
Experience Based
Techniques
--PROF RUCHI S, NMIMS
FACULTY:PROF RUCHI S,NMIMS,MUMBAI

Outlines
1. Structure-based or White-box Techniques
2. Experience-based Techniques
3. Choosing Test Techniques

STRUCTURE-BASED OR WHITE-BOX
TECHNIQUES
 Structure-based or white box techniques /1
The following techniques will be explained in detail:

Statement Testing and Coverage
Decision Testing and Coverage (=branch)
Condition Testing and Coverage
Path Testing Coverage
Remark
These techniques represent the most important and the most widely
used dynamic testing techniques. They relate to the static analysis
techniques which were described earlier.
Other white box testing techniques include but are not limited to:
LCSAJ (Linear Code Sequence and Jump)
Data flow based techniques
TECHNIQUES
 Structure-based or white box techniques /2
It is based on an identified structure of the software or the system

Component-level: the structure of a software component, i.e.
statements, decisions, branches, distinct paths
Integration-level: the structure may be a call tree (a diagram in
which modules call other modules)
System-level: the structure may be a menu structure, business
process or web page structure
Three code-related structural test design techniques for code coverage based
on statements, branches and decisions.

TECHNIQUES
 Structure-based or white box techniques -Tools /1
During white-box testing, the program under test is executed, same as in black
box testing. Both make up dynamic testing.
Theory states that all parts of a program should be executed at least
once during testing.
The rate of coverage of the program is measured using tools (e.g. coverage
analysers):
Code instrumentation is performed in order to count execution
paths, i.e. counters are inserted into the program code of the test object
These counters are initialized holding the value of zero, each
execution path increments the respective counter.
Counters that remain zero after testing indicate program parts that
have not been executed.

TECHNIQUES
 Structure-based or white box techniques -Tools /2
White-box techniques need the support of tools in many areas, there are:
Test case specification
Automatically generating a control flow graph from program source
code
Test execution
Tools to monitor and control the program flow inside the test objects
Tool support ensures the quality of the tests and increases efficiency
Because of the complexity of the necessary measures for white box
testing, manual test execution is
Time consuming, resource consuming
Difficult to implement and prone to errors

TECHNIQUES
 The main types of coverage
Statement Testing and Coverage

The percentage of executable statements that have been exercised by the
test cases
Can also be applied to modules, classes, menu points etc.
Decision Testing and Coverage (=branch coverage)
The percentage of decision outcomes, that have been exercised by the test
cases
Path Testing and Coverage
The percentage of execution paths, that have been exercised by the test
cases
Condition Testing and Coverage
The percentage of all single condition outcomes independently affecting a
decision outcome, that have been exercised by the test cases
Condition coverage comes in various degrees, e.g. single, multiple and
minimal multiple condition coverage.

TECHNIQUES
 Statement Testing and Coverage
The statement of the program code is the focus of attention

What test cases are necessary in order to execute all (or a certain
percentage) of the existing code statements?
Basis of this investigation in the control flow graph
All instructions are represented as nodes and the control flow
between the instructions is represented as an edge (arrow)
Multiple instructions are combined in a single node if they can only
be executed in one particular sequence
Aim of the test (test exit criteria) is to achieve the coverage of selected
percentage of all statements, called the statement coverage (C0 code coverage).
Statement coverage (C0) = Number of executed statements / Total

number of statements * 100%
TECHNIQUES
 Statement Testing and Coverage – Example 1/1
We are assessing the following segment of program code, which is

represented by the control flow graph (see right side):
if(i>0) {
j=f(i);
if(j>10){
while(k>10){
…
}
}
}

TECHNIQUES
 Statement Testing and Coverage – Example 1 /2
Consider the program represented by the control flow

graph on the right
Contains two if-statements and a loop (do-while) inside the
second if-statement
There are three different “routes” through the program
segment
The first if-statements allows two directions
The right-hand direction of the first if-statement is divided
again using the second if-statement
All the statements of this program can be reached using
the route to the right
A single test case will be enough to reach 100% statement
coverage

TECHNIQUES
 Statement Testing and Coverage – Example 2
In this example the graph is slightly more complex:

The program contains the if-statements and a loop
(inside one if statement)
Four different “routes” lead through this program segment
The first if-statement allows two directions
In both branches of the if-statement another if-statement
allows for again two different directions
For a 100% statement coverage four test cases are
needed

TECHNIQUES
 Statement Testing and Coverage – General Conclusions
The measurement of coverage is done using specifically designed tools

These tools are called Coverage Analysis Tools or Coverage Analysers
Benefits/drawbacks of this method
Dead code, that is, code made up of statements that are never executed, will
be discovered
If there is dead code within a program, a 100% coverage cannot be
achieved
Missing instructions, that is, code which is necessary in order to fulfil the
specification, cannot be detected
Testing is only done with respect to the executed statements: can all
code be reached/executed?
Missing code cannot be detected using white box test techniques
(coverage analysis)
TECHNIQUES
 Decision Testing and Coverage*
Instead of statements, decision coverage focuses on the control flow with a
program segment (not the nodes, but the edges of a control flow graph)
All edges of the control flow graph have to be covered at least once
Which test cases are necessary to cover each edge of the control flow
graph at least once?
Aim of this test (test exit criteria) is to achieve the coverage of a selected
percentage of all decisions , called the decision coverage (C1 code coverage)
Decision coverage (C1 ) = Number of executed decisions/ Total number of all
decisions * 100%
Which is synonymous to:
Branch coverage (C1 ) = Number of covered decisions/ Total number of all

branches * 100%
* Also referred to as “branch coverage” FACULTY:PROF RUCHI S,NMIMS,MUMBAI
TECHNIQUES
 Decision Testing and Coverage – Example 1
The control flow graph on the right represents the
program segment to be inspected
Three different “routes” lead through the graph of this
program segment
The first if-statements leads onto two different
directions
One path of the first if-statement is divided
again in two different paths, one of which holds
a loop.
All edges can only be reached via a
combination of three possible paths
Three test cases are needed to achieve a
decision coverage of 100 %
Only using the two directions on the right,
nine out of ten edges can be covered (C1 -
value=90%)
TECHNIQUES
 Decision Testing and Coverage – Example 2
In this example the graph is slightly more complex
Four different “routes” lead through the graph of this
program segment
The first if-statements allows two directions.
In both branches of the if-statement another
if-statement allows again for two different
directions
In this example, the loop is not counted as an
additional decision
For a 100% decision coverage four test cases
are needed
In this example, the same set of test cases is
also required for 100% statement coverage!

TECHNIQUES
 Decision Testing and Coverage – General Conclusions
Achieving 100% decision coverage requires at least as many test cases as
100% statement coverage – in most cases more
A 100% decision coverage always includes a 100% statement
coverage
In most cases edges are covered multiple times
Drawbacks:
Missing statements cannot be detected
Not sufficient to test complex conditions
Not sufficient to test loops extensively
No consideration of dependencies between loops

TECHNIQUES
 Statement Testing and Coverage and Decision Testing and Coverage
Both methods relate to the paths through the control flow graph
They differ in the amount of test cases necessary to achieve 100%
coverage
Only the final result of a condition is considered, although the resulting

condition can be made up of several atomic conditions
The condition If((a>2) OR (b<6)) may only be true or false
Which path of the program is executed depends only on the final
outcome of the combined condition
Failures due to a wrong implementation of parts of a combined
condition may not be detected

TECHNIQUES
 Condition Testing and Coverage
The complexity of a condition that is made up of several atomic conditions is taken
into account
An atomic condition cannot be divided further into smaller condition
statements
This method aims at finding defects resulting from the implementation of multiple
conditions(combined conditions)
Multiple conditions are made up of atomic conditions which are combined
using logical operators like OR, AND, XOR, etc. Example: ((a>2)OR(b<6))
Atomic conditions do not contain logical operators but only relational
operators and the NOT-operator (=, >, <, etc.)
There are three types of condition coverage

Simple condition coverage
Multiple condition coverage
Minimal multiple condition coverage
TECHNIQUES
 Simple Condition Testing and Coverage
Every atomic sub-condition of a combined condition statement has to take at

least once the logical values true as well as false
• This example is used to explain
condition coverage, using a multiple
condition expression.
Example 2 • With only two test cases, a simple
Consider the following condition: condition coverage can be achieved
• Each sub-condition has taken on
a >0 OR b<6
the value true and the value
Test cases for simple condition coverage could be
false.
for example: • However, the combined result is true
-a=3 (true) b=7(false) a>2 OR b<6 (true) in both cases
a=1 (false) b=5 (true) a>2 OR b<6 (true) • True OR false = true
• False OR true = true

TECHNIQUES
 Multiple Condition Testing and Coverage
All combinations that can be created using permutation of the atomic sub
conditions must be part of the tests.
• This example is used to explain
condition coverage, using a multiple
condition expression.
Example 2
• With four test cases, the multiple
condition coverage can be achieved
Consider the following condition:
• All possible combinations of true
a >0 OR b<6 and false were created.
Test cases for multiple condition coverage could be for • All possible results of the multiple
example: conditions were achieved.
a=3 (true) b=7(false) a>2 OR b<6 (true)
• The number of test cases increases
exponentially:
a=3 (true) b=5 (true) a>2 OR b<6 (true)
• n =number of atomic conditions
a=1 (false) b=5 (true) a>2 OR b<6 (true) • 2n =number of test cases
a=1 (false) b=7 (false) a>2 OR b<6 (false)

TECHNIQUES
 Minimal Multiple Condition Coverage (defined c.c.)
All combinations that can be created using the logical results of the sub-
conditions must be part of the test, only if the change of the outcome of one
sub-condition changes the result of the combined condition
• This example is used to explain condition
coverage, using a multiple condition
Example 2 expression.
Consider the following condition:
• For three out of the four test cases the
change of a sub-condition changes the
a >0 OR b<6
overall result
Test cases for multiple condition coverage could be for
• Only for case no. 2 (true OR true
example: =true) the change of a sub-condition
a=3 (true) b=7(false) a>2 OR b<6 (true) will not result in a change of the
a=3 (true) b=5 (true) a>2 OR b<6 (true)
overall condition. This test case can
be omitted!
a=1 (false) b=5 (true) a>2 OR b<6 (true)
• The number of test cases can be reduced
a=1 (false) b=7 (false) a>2 OR b<6 (false) to a value between n+1 and 2n
TECHNIQUES

All complex decisions must be tested – the minimal multiple condition

coverage is a suitable method to achieve this goal.

TECHNIQUES
 Path Testing and Coverage /1
Path coverage focuses on the execution of all possible paths through a

program
A path is a combination of program segments (in a control flow
graph: an alternating sequence of nodes and edges)
For decision coverage, a single path through a loop is sufficient. For
path coverage they are additional test cases:
One test case not entering the loop
One additional test case of loop executions
This may easily lead to a very high number of test cases

TECHNIQUES
 Path Testing and Coverage /2
Focus of the coverage analysis is the control flow graph

Statements are nodes
Control flow is represented by the edges
Every path is a unique way from the beginning to the end of the
control flow graph
The aim of this test(test exit criteria) is to reach a defined path coverage
percentage:
Path coverage = Number of covered paths / Total number of paths * 100%

TECHNIQUES
Path Testing and Coverage – Example
1

program segment to be inspected. It contains three
if-statements
Three different paths leading through the graph of
this program segment achieve full decision
coverage.
However, five different possible paths may be
executed
Five test cases are required to achieve 100%
path coverage
Only two are needed for 100% C0 -three are
needed for 100% C1 -coverage

TECHNIQUES
Path Testing and Coverage – Example 2

program segment to be inspected. It contains two if-
statements and a loop inside the second if-statement
Three different paths leading through the graph of
this program segment achieve full decision coverage.
Four different paths are possible, if the loop is
executed twice
Every increment of the loop counter adds a new
test case.

TECHNIQUES
Path Testing and Coverage – General conclusions
100% path coverage can only be achieved for very simple programs
A single loop can lead to a test case explosion every possible
number of loop execution constitutes a new test case
Theoretically an indefinite of paths is possible
Path coverage is much more comprehensive that statement or
decision coverage
Every possible path through the program is executed
100% path coverage includes 100% decisions coverage, which again
contains 100% statement coverage

TECHNIQUES
Exercise: White-box Techniques
Using a control flow graph (part 1) and pseudo code (part 2) the minimum
number of test cases has to be determined to achieve 100% coverage of:
Statements
Branches/decisions
Paths

Summary TECHNIQUES
White box and black box methods are dynamic methods, the test object is
executed during test
White box comprise:
Statement coverage
Decision coverage
Path coverage
Condition coverage(single, multiple, minimum multiple)
Only existing code can be tested. If functions are missing this fact cannot be
discovered. Dead and superfluous code, however, can be discovered using white
box testing.
White box methods are used mainly in lower test levels like component testing or
integration testing
The methods differ in their intensity of test (test depth)
Depending on the method, the number of test cases differ

Agenda

EXPERIENCE-BASED
•
TECHNIQUES
DEFINITION OF EXPERIENCE-BASED TECHNIQUES
✔
Equivalence partitioning
black box
Boundary value analysis
• Practice of creating test cases State transition testing
without a clear methodical Decision tables
approach based on the
dynamic
Use case based testing
✔
intuition and experience of
Analytical QA
experience-based techniques
the tester. Statement coverage
• Test cases are based on
white box
Branch coverage
intuition and experience Condition coverage
• Where have errors
✔
Path coverage
accumulated in the past?
• Where does software Reviews / walkthroughs
static
often fail? Control flow analysis
Data flow analysis

Compiler metrics/analyzer

EXPERIENCE-BASED TECHNIQUES
Fundamentals
Experience based testing is also called intuitive testing and includes: error guessing
(weak point oriented testing) and exploratory testing (iterative testing based on gained
knowledge about the system)
Mostly applied in order to complement other, more formally created test cases
Does not meet the criteria for systematical testing
Often produce additional test cases that might not be created with other practices,
for example:
Testing a leap year after 2060 (known problems of the past)
Empty sets within input values (a similar application has had errors on this)

Test Case Design
The tester must dispose of applicable experience or knowledge
Intuition – Where can defects be hiding?
Intuition characterizes a good tester
Experience – What defects were encountered where in the past?
Knowledge based on experience
An alternative is to set up a list of recurring defects
Knowledge / Awareness – Where are specific defects expected?
Specific details of the project are incorporated
Where will defects be made due to time pressure and complexity ?
Are inexperienced programmers involved?

Intuitive Test Case Design –possible sources
Test results and practical experience with similar systems
Possibly a predecessor of the software or another system with similar functionality
User experience
Exchange of experience with the system as a user
Focus on deployment
What parts of the system will be used the most?
Development problems
Are there any weak points out of difficulties in the development process?

Error Guessing in practice
Check defect lists

List possible defects
Weight factors depending on risk and probability of occurrence
Test case design
Creating test cases aimed at producing the defects on the list
Prioritizing test cases by their risk value
Update the defect list along testing
Iterative procedure
A structured collection of experience is useful when repeating the procedure in future
projects

Exploratory Testing
Test case design procedure especially suitable when the information basis is weakly structured
Also useful when time for testing is scarce
Procedure:
Examine the single parts of the test object
Execute few test cases, exclusively on the parts to be tested, applying error guessing
Analyze results, develop a rough model of how the test object functions
Iteration: Design new test objects applying the knowledge recently acquired
Thereby focusing on conspicuous areas and on exploring further characteristics of the test
object
Capture tools may be useful for logging test activities

Exploratory Testing – Principles
Choose small objects and/or concentrate on particular aspects of a test object

A single iteration should not take more than 2 hours
The results of the iteration form, the information basis of the following iteration
Additional test cases are derived from the particular test situation
Modelling takes place during testing
Test design, test execution, test logging and learning, based on a test
charter containing test objectives are concurrent and carried out within
time-boxes
Preparing further tests
Herewith, knowledge can be gained to support the appropriate choice of
test case design methods

Experience-based versus Specification-based Techniques
Intuitive test case design is a good complement to systematical approaches

It should still be treated as a complementary activity
It cannot give proof of completeness – the number of test cases can vary considerably
Tests are executed in the same way as with systematically defined test cases
The difference is the way in which the test cases were designed/identified
Through intuitive testing defects can be detected that may not be found through
systematical testing methods

Summary
Experience based techniques complement systematical techniques to determine test case

They depend strongly individual ability of the tester
Error guessing and Explorative testing are two of the more widely used techniques of
experience based testing

Outlines

CHOOSING TEST TECHNIQUES
Criteria for choosing the appropriate Test Techniques / 1
State of information about the test object
Can white-box tests be made at all (source code available)?
Is there sufficient specification material to define black-box tests, or are explorative tests needed
to start with?
Predominant test goals
Are functional tests explicitly requested?
Which non-functional test are needed?
Are structural tests needed to attain the test goals?
Risk aspects
Is serious damage expected from hidden defects?
How high is the sequence of usage for the test object?
Are there contractual or legal standards on test execution and test coverage that have to be met?

Project preconditions
How much time and who is planed for testing?
How high is the risk, that testing might not be completed as planed?
Which software development method are used?
 What are the weak points of the project process?
Characteristics of the test object
What possibilities for testing does the test object offer?
What is the availability of the test object?
Contractual and client requirements
Were there any specific agreements made with the client / originator of the project about the test
procedures?
What documents are to be handed over at the time of deployment of the system?

Best practice
Which approaches have proven to be appropriate on similar structures?
What experience was gained with which approaches in the past?
Test levels
At which test levels should tests be done?
Further criteria should be applied depending on the specific situation!

Different interests cause different test design approaches
Interest of the project manager:
To create software of ordered quality
meeting time and budget restrictions
Interests of the client / initiator of the project
To receive software of best quality (functionality, reliability, usability, efficiency, portability and
maintainability)
Meeting time and budget restrictions
Interests of the test manager:
Sufficient and intensive testing / adequate deployment of the required techniques, from the testing point
of view
To assess the quality level that the project has reached
To allocate and use the resources planed for tests in an optimal way

Summary
Testers generally use a combination of test techniques including process, rule and
data-driven techniques to ensure adequate coverage of the object under test
Criteria for choosing the appropriate test case design approach:
Test basis (Information basis about the test objects)
Testing goals (What conclusions are to be reached through testing)
Risk aspects
Project framework / preconditions
Contractual / client requirements

White Box Testing, Experience Based Techniques

Uploaded by

Copyright:

Available Formats

You might also like

White Box Testing, Experience Based Techniques

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

White Box Testing, Experience Based Techniques

Uploaded by

Copyright:

Available Formats

White Box Testing ,

FACULTY:PROF RUCHI S,NMIMS,MUMBAI

1. Structure-based or White-box Techniques

3. Choosing Test Techniques

FACULTY:PROF RUCHI S,NMIMS,MUMBAI

The following techniques will be explained in detail:

It is based on an identified structure of the software or the system

FACULTY:PROF RUCHI S,NMIMS,MUMBAI

FACULTY:PROF RUCHI S,NMIMS,MUMBAI

FACULTY:PROF RUCHI S,NMIMS,MUMBAI

Statement Testing and Coverage

FACULTY:PROF RUCHI S,NMIMS,MUMBAI

The statement of the program code is the focus of attention

Statement coverage (C0) = Number of executed statements / Total

We are assessing the following segment of program code, which is

FACULTY:PROF RUCHI S,NMIMS,MUMBAI

Consider the program represented by the control flow

FACULTY:PROF RUCHI S,NMIMS,MUMBAI

In this example the graph is slightly more complex:

FACULTY:PROF RUCHI S,NMIMS,MUMBAI

The measurement of coverage is done using specifically designed tools

Which is synonymous to:

Branch coverage (C1 ) = Number of covered decisions/ Total number of all

FACULTY:PROF RUCHI S,NMIMS,MUMBAI

FACULTY:PROF RUCHI S,NMIMS,MUMBAI

Only the final result of a condition is considered, although the resulting

FACULTY:PROF RUCHI S,NMIMS,MUMBAI

There are three types of condition coverage

Every atomic sub-condition of a combined condition statement has to take at

FACULTY:PROF RUCHI S,NMIMS,MUMBAI

FACULTY:PROF RUCHI S,NMIMS,MUMBAI

All complex decisions must be tested – the minimal multiple condition

FACULTY:PROF RUCHI S,NMIMS,MUMBAI

Path coverage focuses on the execution of all possible paths through a

FACULTY:PROF RUCHI S,NMIMS,MUMBAI

Focus of the coverage analysis is the control flow graph

Path coverage = Number of covered paths / Total number of paths * 100%

FACULTY:PROF RUCHI S,NMIMS,MUMBAI

The control flow graph on the right represents the

FACULTY:PROF RUCHI S,NMIMS,MUMBAI

The control flow graph on the right represents the

FACULTY:PROF RUCHI S,NMIMS,MUMBAI

FACULTY:PROF RUCHI S,NMIMS,MUMBAI

FACULTY:PROF RUCHI S,NMIMS,MUMBAI

FACULTY:PROF RUCHI S,NMIMS,MUMBAI

1. Structure-based or White-box Techniques

3. Choosing Test Techniques

FACULTY:PROF RUCHI S,NMIMS,MUMBAI

Data flow analysis

FACULTY:PROF RUCHI S,NMIMS,MUMBAI

FACULTY:PROF RUCHI S,NMIMS,MUMBAI

FACULTY:PROF RUCHI S,NMIMS,MUMBAI

FACULTY:PROF RUCHI S,NMIMS,MUMBAI

Check defect lists

FACULTY:PROF RUCHI S,NMIMS,MUMBAI

FACULTY:PROF RUCHI S,NMIMS,MUMBAI

Choose small objects and/or concentrate on particular aspects of a test object