Professional Documents
Culture Documents
CC6003 Week 10-11: Test Design Techniques
CC6003 Week 10-11: Test Design Techniques
CC6003 Week 10-11: Test Design Techniques
Dr
Dr Tingkai
Tingkai Wang
Wang ASE
ASE slides
slides
Test Design Techniques
The objectives identify what you will be able to do following
the completion of each module
• The Test Development Process (K3)
• Categories of Test Design Techniques (K2)
• Specification-based or Black-box Techniques (K3)
• Structure-based or White-box Techniques (K4)
• Experience-based Techniques (K2)
• Choosing Test Techniques (K2)
Dr
Dr Tingkai
Tingkai Wang
Wang ASE
ASE slides
slides
Objectives of this section
• To differentiate between(k1)
– A test design specification
– A test case specification
– A test procedure specification
• To compare the terms (K2)
– Test condition
– Test case
– Test procedure
Dr
Dr Tingkai
Tingkai Wang
Wang ASE
ASE slides
slides
The Test Development Process
• The test development process described in this
section can be done in different ways,
– very informal with little or no documentation,
– very formal (as it is described following slides).
• The level of formality depends on
– the context of the testing,
– The organization
– the maturity of testing and development processes,
– time constraints,
– safety or regulatory requirements,
– the people involved.
Dr
Dr Tingkai
Tingkai Wang
Wang ASE
ASE slides
slides
The Test Development Process
• During test analysis, the test basis
documentation is analysed in order to
determine what to test, i.e., to identify the
test conditions.
• A test condition is
– defined as an item or event that could be
verified by one or more test cases (e.g., a
function, transaction, quality characteristic or
structural element).
Dr
Dr Tingkai
Tingkai Wang
Wang ASE
ASE slides
slides
Example of test condition
• Component test condition:
– To demonstrate that the date of birth is stored as CCYYMMDD
Dr
Dr Tingkai
Tingkai Wang
Wang ASE
ASE slides
slides
Example of test condition
• Acceptance test condition
– To demonstrate that everyone over 65 years of age will receive a
£100 gift voucher on their birthday
Dr
Dr Tingkai
Tingkai Wang
Wang ASE
ASE slides
slides
Example of system test condition
Dr
Dr Tingkai
Tingkai Wang
Wang ASE
ASE slides
slides
Traceability
• Traceability
– The ability to trace test conditions back to the
specifications and requirements
• Traceability allows effective impact analysis when
requirements change,
• Traceability ensures requirements test coverage
can be determined for a set of tests.
• During test analysis the detailed test approach is
implemented to select the test design techniques to
use based on, among other considerations, the
risks identified
Dr
Dr Tingkai
Tingkai Wang
Wang ASE
ASE slides
slides
Test design
• During test design the test cases and test data are
created and specified.
• A test case consists of
– a set of input values,
– execution preconditions,
– expected results and
– execution post-conditions,
• developed to cover a certain test objective(s) or test
condition(s).
• The 'Standard for Software Test Documentation' (IEEE
STD 829-1998) describes the content of test
design specifications (containing test conditions) and test
case specifications.
Dr
Dr Tingkai
Tingkai Wang
Wang ASE
ASE slides
slides
Expected results
• Expected results should be produced as part of the
specification of a test case and include
– System outputs,
– changes to data and states,
– any other consequences of the test.
• If expected results have not been defined, then a
plausible, but erroneous, result may be interpreted
as the correct one.
• Expected results should ideally be defined prior to
test execution.
Dr
Dr Tingkai
Tingkai Wang
Wang ASE
ASE slides
slides
Test implementation
• During test implementation the test cases are
developed, implemented, prioritised and organized
in the test procedure specification (IEEE STD
829-1998).
• The test procedure (or manual test script) specifies
the sequence of actions for the execution of a test.
• If tests are run using a test execution tool, the
sequence of actions is specified in a test
script (which is an automated test procedure).
Dr
Dr Tingkai
Tingkai Wang
Wang ASE
ASE slides
slides
test execution schedule
• The various test procedures and automated test
scripts are subsequently formed into a test
execution schedule
• test execution schedule defines the following:
– the order in which the various test procedures, (and
possibly automated test scripts) are to be executed.
– When they are scheduled to be executed
– And by whom
• A test execution schedule should take into account
such as:
– factors as regression tests,
– prioritisation, and technical and logical dependencies.
Dr
Dr Tingkai
Tingkai Wang
Wang ASE
ASE slides
slides
Categories of Test Design Techniques
We will cover:
• Test Design Techniques
• Black-box (also called specification-based)
technique
• white-box (also called structure-based)
technique
• experience-based technique
• Categories test design technique
Dr
Dr Tingkai
Tingkai Wang
Wang ASE
ASE slides
slides
Test Design Techniques
• The purpose of a test design technique is to
identify test conditions, test cases, and test
data.
• It is a classic distinction to denote test
techniques as black-box or white-box.
Dr
Dr Tingkai
Tingkai Wang
Wang ASE
ASE slides
slides
Black-box (specification-based) technique
• Black-box test design techniques are a way to derive and
select test conditions, test cases, or test data based on an
analysis of the test basis documentation.
• This includes both functional and non-functional testing of
a component or system to be tested.
• Black-box testing, by definition, does not use any
information regarding the internal structure of the
component or system to be tested.
Black box
Data Input Actual Results
If the actual results = the expected results OK
Dr
Dr Tingkai
Tingkai Wang
Wang ASE
ASE slides
slides
Black-box (specification-based)
technique
• Common characteristics of specification-
based test design techniques include:
– Models, either formal or informal, are used for
the specification of the problem to be solved,
the software or its components
– Test cases can be derived systematically from
these models
Dr
Dr Tingkai
Tingkai Wang
Wang ASE
ASE slides
slides
White-box (structure-based) techniques
• White-box test design techniques (also called
structural or structure-based techniques) are based on
an analysis of the structure of the component or
system.
• White-box techniques test the logical structure of the
code
Decision point
Dr
Dr Tingkai
Tingkai Wang
Wang ASE
ASE slides
slides
White-box (structure-based) techniques
Dr
Dr Tingkai
Tingkai Wang
Wang ASE
ASE slides
slides
experience-based techniques
• experience-based techniques, such as error
guessing, rely on the knowledge and experience of
the person designing the test.
• It can be used for functional and non-functional
testing of component or system.
• Common characteristics of experience-based test
design techniques include:
– The knowledge and experience of people are used to derive the test
cases
– The knowledge of testers, developers, users and other
stakeholders about the software, its usage and its environment is one
source of information
– Knowledge about likely defects and their distribution is another
source of information
Dr
Dr Tingkai
Tingkai Wang
Wang ASE
ASE slides
slides
Specification-based or Black-box
Techniques
We will cover:
– Equivalence Partitioning
– Boundary Value Analysis
– Decision Table Testing
– State Transition Testing
– Use Case Testing
Dr
Dr Tingkai
Tingkai Wang
Wang ASE
ASE slides
slides
Equivalence Partitioning
• Inputs to the software or system are divided into
groups that are expected to exhibit similar
behaviour, so they are likely to be processed in the
same way.
• Equivalence partitions (or classes) can be found
for both valid data, i.e., values that should be
accepted and invalid data, i.e., values that should
be rejected.
• We can select one input value that is considered to
be representative of each partition or class
Dr
Dr Tingkai
Tingkai Wang
Wang ASE
ASE slides
slides
Equivalence Partitioning
• Partitions can also be identified for
– outputs,
– internal values,
– time-related values (e.g., before or after an event)
– interface parameters (e.g., integrated components being tested
during integration testing).
• Tests can be designed to cover all valid and invalid
partitions.
• Equivalence partitioning is applicable at all levels of
testing.
• can be used to achieve input and output coverage goals.
• It can be applied to human input, input via interfaces to a
system, or interface parameters in integration testing.
Dr
Dr Tingkai
Tingkai Wang
Wang ASE
ASE slides
slides
Equivalence Partitioning
Requirement: “The customer is eligible for car
insurance if they are at least 17 but to older than 75
years of age, age can be entered in the range 0-150”
Dr
Dr Tingkai
Tingkai Wang
Wang ASE
ASE slides
slides
Boundary Value Analysis
• Behaviour at the edge of each equivalence partition
is more likely to be incorrect than behaviour within
the partition, so boundaries are an area where
testing is likely to yield defects.
• The maximum and minimum values of a partition
are its boundary values.
• A boundary value for a valid partition is a valid
boundary value; the boundary of an invalid
partition is an invalid boundary value.
• Tests can be designed to cover both valid and
invalid boundary values.
Dr
Dr Tingkai
Tingkai Wang
Wang ASE
ASE slides
slides
Boundary Value Analysis
• When designing test cases, a test for each
boundary value is chosen.
• Boundary value analysis can be applied at
all test levels.
• It is relatively easy to apply and its defect-
finding capability is high.
• Detailed specifications are helpful in
determining the interesting boundaries.
Dr
Dr Tingkai
Tingkai Wang
Wang ASE
ASE slides
slides
Boundary Value Analysis
• This technique is often considered as an
extension of equivalence partitioning or
other black-box test design techniques.
• It can be used on equivalence classes for
user input on screen as well as, for example,
– time ranges (e.g., time out, transactional speed
requirements)
– table ranges (e.g., table size is 256*256).
Dr
Dr Tingkai
Tingkai Wang
Wang ASE
ASE slides
slides
Boundary Value Analysis
Requirement: “The customer is eligible for car
insurance if they are at least 17 but to older than 75
years of age, age can be entered in the range 0-150”
Dr
Dr Tingkai
Tingkai Wang
Wang ASE
ASE slides
slides
Example EqP and BVA
• Users input item price and order quantity to the
system, which calculates total cost including any
discount.
• Items can be ordered for any number up to 999,
but the minimum order is for 5 items
• Order for quantities of 100 and above will receive
a 20% discount
• Apply equivalence partitioning and boundary
value analysis to identity valid partitions and valid
boundaries.
Dr
Dr Tingkai
Tingkai Wang
Wang ASE
ASE slides
slides
Example EqP
• Number of item (valid partitions):
0 to 4 5 to 99 100 to 999
Dr
Dr Tingkai
Tingkai Wang
Wang ASE
ASE slides
slides
Equivalence Partitioning test cases
Test case are then produced for the valid partitions
Test case ID 1 2 3
Dr
Dr Tingkai
Tingkai Wang
Wang ASE
ASE slides
slides
Example BVA
• Number of item (valid boundaries):
Not Sell Sell
allowed Full price 20% discount
0 to 4 5 to 99 100 to 999
Dr
Dr Tingkai
Tingkai Wang
Wang ASE
ASE slides
slides
Boundary Value Analysis test cases
Test cases are then produced for the valid boundaries
Test case ID 1 2 3 4 5 6
actions Order for 0 Order for 4 Order for 5 Order for 99 Order for 100 Order for 999
items cast £10 items cast £5 items cast £20 items cast £10 items cast items cast £10
£150
Total cost £0 £20 £100 £990 £15000 £9990
Expected Sale rejected Sale rejected Sale £100 Sale £990 Sale £12000 Sale £7992
outcome
Dr
Dr Tingkai
Tingkai Wang
Wang ASE
ASE slides
slides
Decision Table Testing
• Decision tables are a good way to capture system
requirements that contain logical conditions, and to
document internal system design.
• They may be used to record complex business rules
that a system is to implement.
• When creating decision tables, the specification is
analysed, and conditions and actions of the system
are identified.
• The input conditions and actions are most often
stated in such a way that they must be true or false
(Boolean).
Dr
Dr Tingkai
Tingkai Wang
Wang ASE
ASE slides
slides
Decision Table Testing
• The decision table contains
– the triggering conditions, often combinations of true and false for all
input conditions,
– the resulting actions for each combination of conditions.
• Each column of the table corresponds to a business
rule that defines a unique combination of conditions
and which result in the execution of the actions
associated with that rule.
• The coverage standard commonly used with
decision table testing is to have at least one test per
column in the table, which typically involves
covering all combinations of triggering conditions.
Dr
Dr Tingkai
Tingkai Wang
Wang ASE
ASE slides
slides
Decision Table Testing
• The strength of decision table testing is that
it creates combinations of conditions that
otherwise might not have been exercised
during testing.
• It may be applied to all situations when the
action of the software depends on several
logical decisions.
Dr
Dr Tingkai
Tingkai Wang
Wang ASE
ASE slides
slides
State Transition Testing
• A system may exhibit a different response depending on
current conditions or previous history (its state).
• In this case, that aspect of the system can be shown with a
state transition diagram.
• It allows the tester to view the software in terms of :
– its states,
– transitions between states,
– the inputs or events that trigger state changes (transitions)
– the actions which may result from those transitions.
• The states of the system or object under test are separate,
identifiable and finite in number.
Dr
Dr Tingkai
Tingkai Wang
Wang ASE
ASE slides
slides
State Transition diagram
The state transition diagram shows the states, events
and possible transitions
A B C D
SS S1 S2 S3 SE
E
F
State:
SS, S1, S2, S3, SE
Events: Possible transitions:
A, B, C, D, E, F SS-S1, S1-S2, S2-S1, S2-S3, S3-S3, S3-SE
Dr
Dr Tingkai
Tingkai Wang
Wang ASE
ASE slides
slides
State table
A stable can be produced to show the relationship between the
states and inputs, and can highlight possible transitions that are
invalid.
Events
States A B C D E F
SS S1 Null Null Null Null Null
S1 Null S2 Null Null Null Null
S2 Null Null S3 Null Null Null
S3 Null Null Null SE Null S3
SE Null Null Null Null Null Null
We will cover:
• Background
• Statement Testing and Coverage
• Decision Testing and Coverage
• Other Structure-based Techniques
analysis
Static
Non-functional testing