Professional Documents
Culture Documents
Verification PDF
Verification PDF
Verification PDF
Software Testing
These techniques are useful during functional testing where the objective is to test
whether or not an application, unit, system, or subsystem, correctly implements the
functionality as per the given requirements.
Black Box testing takes an external perspective of the test object to derive test cases.
These tests can be functional or non-functional, though usually functional. The test
designer selects valid and invalid inputs and determines the correct output. There is no
knowledge of the test object's internal structure.
This method of test design is applicable to all levels of software testing: unit, integration,
functional testing, system and acceptance. The higher the level, and hence the bigger and
more complex the box, the more one is forced to use black box testing to simplify. While
this method can uncover unimplemented parts of the specification, one cannot be sure
that all existent paths are tested.
Black box testing (also called functional testing) is testing that ignores the internal
mechanism of a system or component and focuses solely on the outputs generated in
response to selected inputs and execution conditions.
Black box testing, also called functional testing and behavioral testing, focuses on
determining whether or not a program does what it is supposed to do based on its
functional requirements. Black box testing attempts to find errors in the external behavior
of the code in the following categories: (1) incorrect or missing functionality; (2)
interface errors; (3) errors in data structures used by interfaces; (4) behavior or
performance errors; and (5) initialization and termination errors.
It is best if the person who plans and executes black box tests is not the programmer of
the code and does not know anything about the structure of the code. The programmers
Testing Overview and Black-Box Testing Techniques of the code are innately biased and
are likely to test that the program does what they programmed it to do. What are needed
are tests to make sure that the program does what the customer wants it to do. As a result,
most organizations have independent testing groups to perform black box testing. These
testers are not the developers and are often referred to as third-party testers. Testers
should just be able to understand and specify what the desired output should be for a
given input into the program, as shown in Figure.
Ideally, we’d like to test every possible thing that can be done with our program. But, as
we said, writing and executing test cases is expensive. We want to make sure that we
definitely write test cases for the kinds of things that the customer will do most often or
even fairly often. Our objective is to find as many defects as possible in as few test cases
as possible. We want to avoid writing redundant test cases that won’t tell us anything new
(because they have similar conditions to other test cases we already wrote). Each test case
should probe a different mode of failure. We also want to design the simplest test cases
that could possibly reveal this mode of failure – test cases themselves can be error-prone
if we don’t keep this in mind.
Black box test cases are based on customer requirements. We begin by looking at each
customer requirement. To start, we want to make sure that every single customer
requirement has been tested at least once. As a result, we can trace every requirement to
its test case(s) and every test case back to its stated customer requirement. The first test
case we’d write for any given requirement is the most-used success path for that
requirement. By success path, we mean that we want to execute some desirable
functionality (something the customer wants to work) without any error conditions. We
proceed by planning more success path test cases, based on other ways the customer
wants to use the functionality and some test cases that execute failure paths. Intuitively,
failure paths intentionally have some kind of errors in them, such as errors that users can
accidentally input. We must make sure that the program behaves predictably and
gracefully in the face of these errors. Finally, we should plan the execution of our tests
out so that the most troublesome, risky requirements are tested first. This would allow
more time for fixing problems before delivering the product to the customer. It would be
devastating to find a critical flaw right before the product is due to be delivered.
We’ll start with one basic requirement. We can write many test cases based on this one
requirement, which follows below. As we’ve said before, it is impossible to test every
single possible combination of input.
INPUT
Pass / Fail
“Correct” Output
Strategy:
• Based on the analysis of Requirements / Responsibilities, Specify Inputs.
• For the specified inputs, determine the expected, correct outputs.
• Run the Test as per the Specified Test Procedure.
• Compare the actual output with the expected output
• Pass the verdict: PASS or FAIL.
Black Box and White Box techniques are not mutually exclusive, rather they are
mutually complementary.
It is a primary basis for test design and consumes nearly 70 % of testing effort. It is
Expensive, needs careful analysis, design and planning. The well – designed test
cases achieve good code coverage also at lesser cost. The specifications of inputs can
be from the analysis of output domain also. The implementation Details are not
visible.
A Simple Example
You are to test the Fine Computation Module of a Library Software System.
Borrower can be student or staff.
Requirement 1: Fine is collected only from the students. If the borrower is staff no
fine is collected.
Requirement 4 :
Administrator can change the slabs for fine as well as the amount of fine.
But this requirement is not in the scope of current testing
( Must be noted in the Requirements Tracing Matrix )
Requirement 5 :
Information regarding Students and Staff And Fine Policy : Stored in a Relational
Database ;
Access Control , Database Recovery policies are specified.
But this requirement is not in the scope of current testing
( Must be noted in the Requirements Tracing Matrix )
Testing Requirements
Testing Requirement 1:
Set due date < current date ( controllability ! )
Set the borrower as Staff and Test that fines are not levied.
Set the borrower as Student and Test that fines are levied.
Testing Requirement 2 :
Testing Requirement 3:
Set the due date such that the number of overdue days is : 0 , 14 , 15 , 16 ,and
maximum possible value.
For each of these values verify that the computed fine value is correct.
Requirement 4 is not being tested now. Ensure that the fact is recorded clearly in your
test report.
Requirement 5 is not being tested now. Ensure that the fact is recorded clearly in your
test report.
Requirements Model
Test Plan: Describe scope, approach, resources, test schedule, items to be tested,
deliverables, responsibilities, approvals needed. It could be used at the system test level
or at lower levels.
Test design spec: Identifies a subset of features to be tested and identifies the test cases to
test the features in this subset.
Test case spec: Lists inputs, expected outputs, features to be tested by this test case, and
any other special requirements e.g. setting of environment variables and test procedures.
Dependencies with other test cases are specified here. Each test case has a unique ID for
reference in other documents.
Test procedure spec: Describe the procedure for executing a test case.
Test transmittal report: Identifies the test items being provided for testing, e.g. a database.
Test incident report: Document any special event that is recommended for further
investigation.
Test summary: Summarize the results of testing activities and provide an evaluation.
Four techniques are considered: equivalence partitioning, boundary value analysis, cause
effect graphing, and predicate based test generation.
Each of these test generation techniques is a black-box technique and useful for
generating test cases during functional testing.
Requirements serve as the starting point for the generation of tests. During the initial
phases of development, requirements may exist only in the minds of one or more people.
These requirements, more aptly ideas, are then specified rigorously using modeling
elements such as use cases, sequence diagrams, and state charts in UML.
Rigorously specified requirements are often transformed into formal requirements using
requirements specification languages such as Z, S, and RSML.
Equivalence Partitioning:
Equivalence partitioning is a software testing technique that divides the input data of a
software unit into partition of data from which test cases can be derived. In principle, test
cases are designed to cover each partition at least once. This technique tries to define test
case that uncovers classes of errors, thereby reducing the total number of test cases that
must be developed.
there are two further partitions of invalid ranges. The first invalid partition would be <= 0
and the second invalid partition would be >= 13.
... -2 -1 0 1 .............. 12 13 14 15 .....
--------------|-------------------|---------------------
invalid partition 1 valid partition invalid partition 2
The testing theory related to equivalence partitioning says that only one test case of each
partition is needed to evaluate the behavior of the program for the related partition. In
other words it is sufficient to select one test case out of each partition to check the
behavior of the program. To use more or even all test cases of a partition will not find
new faults in the program. The values within one partition are considered to be
"equivalent". Thus the number of test cases can be reduced considerably.
Equivalence Partitioning
Divide the input domain into classes of data for which test cases can be
generated.
Here test cases are written to uncover classes of errors for every input condition.
Partition refers to collection of mutually disjoint subsets whose union is the entire set.
Chose one data element from each partitioned set. The KEY is the choice of equivalence
relation. EP based testing allows - to have a sense of complete testing and helps avoid
redundancy. A set of data elements from the input domain.The same output results for all
the elements of the EC.First Step is to partition the input domain into ECs. The ECs must
include both valid and invalid input values.Then generate test cases based on the Ecs .
A Simple Example :
• If the input specifies a range of valid values , define one valid EC ( within range )
and two invalid ECs ( one outside each end of the range )
If the input specifies a set of valid values, define one valid EC (members of the set)
one invalid EC ( non-members of the set )
2 ECs
• Input requires a specific value: one valid EC , and two invalid ECs ( one < , one
>)
Example : Password required : six character string
• Input is boolean or specifies a “must” condition , specify one valid EC and one
invalid.
•
Example : The first character must be numeric
If during test case design , you find that all values in the equivalence class are not really
to be treated identically , divide the equivalence class further.
( Rule 2 ) more than or equal to 15 days but less than 45 days : fine is Rs. 2 per day ;
ECs Identified :
Let D denote the input domain of a program P. The test selection problem is to select a
subset T of tests such that execution of P against each element of T will reveal all errors
in P.
In general there does not exist any algorithm to construct such a test set. However, there
are heuristics and model based methods that can be used to generate tests that will reveal
certain type of faults
The challenge is to construct a test set T⊆D that will reveal as many errors in P as
possible. The problem of test selection is difficult due primarily to the size and
complexity of the input domain of P.
Exhaustive testing
The large size of the input domain prevents a tester from exhaustively testing the
program under test against all possible inputs. By ``exhaustive" testing we mean testing
the given program against every element in its input domain.
Consider program P that is required to sort a sequence of integers into ascending order.
Assuming that P will be executed on a machine in which integers range from -32768 to
32767, the input domain of program consists of all possible sequences of integers in the
range [-32768, 32767].
N
S = ∑ vi , S is the size of the input domain, where v is the
i=0
number of possible values for each element of the input sequence
If there is no limit on the size of the sequence that can be input, then the input domain of
P is infinitely large and P can never be tested exhaustively. If the size of the input
sequence is limited to, say Nmax>1, then the size of the input domain depends on the
value of N.
999 possible values of ID, 2720 possible character string representing names
21 hourly pay rates, and 61 possible work hours, the number of possible
records are 999 X 2720 X 21 X 61 = 5.42 X 10 34
Test selection using equivalence partitioning allows a tester to subdivide the input
domain into a relatively small number of sub-domains, say N>1.
In strict mathematical terms, the sub-domains by definition are disjoint. The four subsets
shown in (a) constitute a partition of the input domain while the subsets in (b) are not.
Each subset is known as an equivalence class.
Sub domains
The equivalence classes are created assuming that the program under test exhibits the
same behavior on all elements, i.e. tests, within a class.
These assumptions allow the tester to select exactly one test from each equivalence class
resulting in a test suite of exactly N tests.
Faults targeted
The entire set of inputs to any application can be divided into at least two subsets: one
containing all the expected, or legal, inputs (E) and the other containing all unexpected,
or illegal, inputs (U).
Each of the two subsets, can be further subdivided into subsets on which the application
is required to behave differently (e.g. E1, E2, E3, and U1, U2).
Equivalence class partitioning selects tests that target any faults in the application that
cause it to behave incorrectly when the input is in either of the two classes or their
subsets.
Example 1
Consider an application A that takes an integer denoted by age as input. Let us suppose
that the only legal values of age are in the range [1..120]. The set of input values is now
divided into a set E containing all integers in the range [1..120] and a set U containing the
remaining integers.
Further, assume that the application is required to process all values in the range [1..61]
in accordance with requirement R1 and those in the range [62..120] according to
requirement R2. Thus E is further subdivided into two regions depending on the expected
behavior.
Similarly, it is expected that all invalid inputs less than or equal to 1 are to be treated in
one way while all greater than 120 are to be treated differently. This leads to a
subdivision of U into two categories.
All integers
Other integers
[1..120]
<1
[62-120] >12
[1..61]
Tests selected using the equivalence partitioning technique aim at targeting faults in the
application under test with respect to inputs in any of the four regions, i.e. two regions
containing expected inputs and two regions containing the unexpected inputs.
It is expected that any single test selected from the range [1..61] will reveal any fault with
respect to R1. Similarly, any test selected from the region [62..120] will reveal any fault
with respect to R2. A similar expectation applies to the two regions containing the
unexpected inputs.
Effectiveness
The effectiveness of tests generated using equivalence partitioning for testing application
A, is judged by the ratio of the number of faults these tests are able to expose to the total
faults lurking in A.
As is the case with any test selection technique in software testing, the effectiveness of
tests selected using equivalence partitioning is less than 1 for most practical applications.
Example 2
This example shows a few ways to define equivalence classes based on the knowledge of
requirements and the program text.
Consider that wordCount method takes a word w and a filename f as input and returns
the number of occurrences of w in the text contained in the file named f. An exception is
raised if there is no file with name f.
begin
String w, f
Input w, f
if (not exists(f) {raise exception; return(0);}
if(length(w)==0)return(0);
if(empty(f))return(0);
return(getCount(w,f));
end
Using the partitioning method described in the examples above, we obtain the following
equivalence classes.
Equivalence class w f
Note that the number of equivalence classes without any knowledge of the program code
is 2, whereas the number of equivalence classes derived with the knowledge of partial
code is 6.
Of course, an experienced tester will likely derive the six equivalence classes given
above, and perhaps more, even before the code is available.
In some cases the equivalence classes are based on the output generated by the program.
For example, suppose that a program outputs an integer.
It is worth asking: ``Does the program ever generate a 0? What are the maximum and
minimum possible values of the output?"
These two questions lead to two the following equivalence classes based on outputs:
Based on the output equivalence classes one may now derive equivalence classes for the
inputs. Thus each of the four classes given above might lead to one equivalence class
consisting of inputs.
One class with values inside speed ∈[60..90] { {50} , {75} , {92} }
the range and two with
values outside the range.
Arrays in Java and records, or structures, in C++, are compound types. Such input types
may arise while testing components of an application such as a function or an object.
While generating equivalence classes for such inputs, one must consider legal and illegal
values for each component of the structure. The next example illustrates the derivation of
equivalence classes for an input variable that has a compound type.
struct transcript
{
string fName; // First name.
string lName; // Last name.
string cTitle [200]; // Course titles.
char grades [200]; // Letter grades corresponding
to course titles.
}
Exercise: Derive equivalence classes for each component of R and combine them!
One-dimensional partitioning
One way to partition the input domain is to consider one input variable at a time.
Multidimensional partitioning
Another way is to consider the input domain I as the set product of the input variables
and define a relation on I.
Partitioning Example
Consider an application that requires two integer inputs x and y. Each of these inputs is
expected to lie in the following ranges: 3≤ x≤7 and 5≤y≤9.
For multidimensional partitioning we consider the input domain to be the set product X x
Y. This leads to 9 equivalence classes.
1. Identify the input domain: Read the requirements carefully and identify all input
and output variables, their types, and any conditions associated with their use.
Environment variables, such as class variables used in the method under test and
environment variables in UNIX, Windows, and other operating systems, also serve as
input variables. Given the set of values each variable can assume, an approximation
to the input domain is the product of these sets.
2. Equivalence classing: Partition the set of values of each variable into disjoint
subsets. Each subset is an equivalence class. Together, the equivalence classes based
on an input variable partition the input domain. Partitioning the input domain using
values of one variable, is done based on the expected behavior of the program.
Values for which the program is expected to behave in the ``same way" are grouped
together. Note that ``same way" needs to be defined by the tester.
3. Combine equivalence classes: This step is usually omitted and the equivalence
classes defined for each variable are directly used to select test cases. However, by
not combining the equivalence classes, one misses the opportunity to generate useful
tests.
For example, suppose that an application is tested via its GUI, i.e. data is input using
commands available in the GUI. The GUI might disallow invalid inputs by offering a
palette of valid inputs only. There might also be constraints in the requirements that
render certain equivalence infeasible.
The control software of BCS, abbreviated as CS, is required to offer several options.
One of the options, C (for control), is used by a human operator to give one of four
commands (cmd): change the boiler temperature (temp), shut down the boiler
(shut), and cancel the request (cancel).
Command temp causes CS to ask the operator to enter the amount by which the
temperature is to be changed (tempch). Values of tempch are in the range -10..10 in
increments of 5 degrees Fahrenheit. An temperature change of 0 is not an option.
Selection of option C forces the BCS to examine variable V. If V is set to GUI, the
operator is asked to enter one of the three commands via a GUI. However, if V is set
to file, BCS obtains the command from a command file.
The command file may contain any one of the three commands, together with the
value of the temperature to be changed if the command is temp. The file name is
obtained from variable F.
V, F: Environment variables
cmd: command V F
(temp, shut, cancel)
cmd
Control Software
tempch G (CS)
UI
tempch: desired
temperature change
datafile
(-10..10)
V ∈{GUI, file}
F: file name if V is set to “file.”
In response to temp and shut commands, the control software is required to generate
appropriate signals to be sent to the boiler heating system.
The GUI forces the tester to select from a limited set of values as specified in the
requirements. For example, the only options available for the value of tempch are -
10, -5, 5, and 10. We refer to these four values of tempch as t-valid while all other
values as t-invalid.
First we examine the requirements, identify input variables, their types, and values.
These are listed in the following table.
Input domain⊆S=V×F×cmd×tempch
(GUI,--,temp,-5), (GUI,--,cancel,--)
Variable Partition
F {{f-valid}, {f-invalid}}
Note that tinvalid, tvalid, finvalid, and fvalid denote sets of values. “undefined”
denotes one value.
The variable V has 3 EC; F has 2 EC; cmd has 4 EC; and tempch has 2 EC.
Note that each of the classes listed above represents an infinite number of input
values for the control software. For example, {(GUI}}, f-valid, temp, -10)} denotes
an infinite set of values obtained by replacing f-valid by a string that corresponds to
the name of an existing file. Each value is a potential input to the BCS.
Note that the GUI requests for the amount by which the boiler temperature is to be
changed only when the operator selects temp for cmd. Thus all equivalence classes that
match the following template are infeasible.
This parent-child relationship between cmd and tempch renders infeasible a total of
3×2×3×5=90 equivalence classes.
GUI does not allow invalid values of temperature change to be input. This leads to two
more infeasible ECs.
Application might not ask for the value of cmd and tempch. When v=file and F contains a
file name that does not exist.
In this case five additional Equivalence classes are discarded. Each of these Equivalence
classes are described by the following template:
Application will not allow values of cmd and tempch to be input when V is undefined.
Thus, yet another five equivalence classes that match the following template are renders
infeasible.
After having discarded all infeasible equivalence classes (90+2+5+5 = 102), we are left
with a total of 18 testable (or feasible) equivalence classes.
Given a set of equivalence classes that form a partition of the input domain, it is
relatively straightforward to select tests. However, complications could arise in the
presence of infeasible data and don't care values.
In the most general case, a tester simply selects one test that serves as a representative of
each equivalence class.
While designing equivalence classes for programs that obtain input exclusively from a
keyboard, one must account for the possibility of errors in data entry. For example, the
requirement for an application.
The application places a constraint on an input variable X such that it can assume integral
values in the range 0..4. However, testing must account for the possibility that a user may
inadvertently enter a value for X that is out of range.
Suppose that all data entry to the application is via a GUI front end. Suppose also that the
GUI offers exactly five correct choices to the user for X.
In such a situation it is impossible to test the application with a value of X that is out of
range. Hence only the correct values of X will be input. See figure on the next slide.
A Black Box Testing Method, complements to Equivalence partition and BVA leads to a
selection of test cases that exercise bounding values.
The expected input and output values should be extracted from the component
specification. The input and output values to the software component are then grouped
into sets with identifiable boundaries. Each set, or partition, contains values that are
expected to be processed by the component in the same way. Partitioning of test data
ranges is explained in the equivalence partitioning test case design technique. It is
important to consider both valid and invalid partitions when designing test cases.
For an example, if the input values were months of the year expressed as integers, the
input parameter 'month' might have the following partitions:
... -2 -1 0 1 ...................12 13 14 15 .....
---------------|-----------------|---------------------
invalid partition 1 valid partition invalid partition 2
The boundaries are the values on and around the beginning and end of a partition. If
possible test cases should be created to generate inputs or outputs that will fall on and to
either side of each boundary. This would result in three cases per boundary. The test
cases on each side of a boundary should be in the smallest increment possible for the
component under test. In the example above there are boundary values at 0,1,2 and
11,12,13. If the input values were defined as decimal data type with 2 decimal places then
the smallest increment would be the 0.01.
Where a boundary value falls within the invalid partition the test case is designed to
ensure the software component handles the value in a controlled manner. Boundary value
analysis can be used throughout the testing cycle and is equally applicable at all testing
phases.
After determining the necessary test cases with equivalence partitioning and subsequent
boundary value analysis, it is necessary to define the combination of the test cases when
there are multiple inputs to a software component.
In the general application of Boundary Value Analysis can be done in a uniform manner. The
basic form of implementation is to maintain all but one of the variables at their nominal
(normal or average) values and allowing the remaining variable to take on its extreme values.
• Min ---------------------------------------Minimal
• Min+ -------------------------------------Just above Minimal
• Nom --------------------------------------Average
• Max- -------------------------------------Just below Maximum
• Max --------------------------------------Maximum
In summary:
•Empirical evidence : Errors are more common at the boundaries of domains than at the
“center” .
• Complements and is a refinement of EC Analysis
• Select an element from the “edge” of an EC rather than picking up one arbitrarily
• Analyze the output domain also to derive test cases.
• Guidelines for BVA are similar to those of ECA.
Input is a range ( m to n ): Select inputs as: m , n , “just less” than m , “just more” than n.
Input is a number of values: select minimum and maximum numbers and values; just
below minimum, and just above maximum.
Apply the above two rules to the output domain also and Select required input.
BVA can be complex, but has good value.
For the example Fine Module described earlier, BVA leads to Overdue period for staff:
0 , 44 , 45 , 46 , maximum possible value for the defined type for overdue period ( or
max , max + 1 if designers give you max ! )
Exercise : BVA
• For the feature of the RA module of the Student Information System described
earlier.
- Range
- Value
- Rather than select any element in an equivalence class, select those at the
''edge' of the class.
Examples:
- For a range of values bounded by a and b, test (a- precision value), a, (a+ precision
value), (b- precision value), b, (b+ precision value).
- If input conditions specify a number of values n, test with (n-1), n and (n+1) input
values.
- If internal program data structures have boundaries (e.g., buffer size, table limits),
use input data to exercise structures on boundaries.
Experience indicates that programmers make mistakes in processing values at and near
the boundaries of equivalence classes.
Obviously, this fault is revealed, though not necessarily, when M is tested against x=0
but not if the input test set is, for example, {-4, 7} derived using equivalence partitioning.
In this example, the value x=0, lies at the boundary of the equivalence classes x≤0 and
x>0.
Boundary value analysis is a test selection technique that targets faults in applications at
the boundaries of equivalence classes.
While equivalence partitioning selects tests from within equivalence classes, boundary
value analysis focuses on tests at and near the boundaries of equivalence classes.
Certainly, tests derived using either of the two techniques may overlap.
BVA: Procedure
2 Identify the boundaries for each partition. Boundaries may also be identified using
special relationships amongst the inputs.
3 Select test data such that each boundary value occurs in at least one test input.
BVA: Example:
Assuming that an item code must be in the range 99..999 and quantity in the range
1..100.
2. Identify boundaries
* x * * x *
99 999
E1 E3
E2
0 2 99 101
* x * * x *
E4 1 100 E6
E5
Equivalence classes and boundaries for findPrice. Boundaries are indicated with an x.
Points near the boundary are marked *.
Test selection based on the boundary value analysis technique requires that tests must
include, for each variable, values at and around the boundary. Consider the following test
set:
To be able to demonstrate or explain the need for certain methods and their relative merits I
will introduce two testing examples proposed by P.C. Jorgensen [1]. These examples will
provide more extensive ranges to show where certain testing techniques are required and
provide a better overview of the methods usability.
The NextDate problem is a function of three variables: day, month and year. Upon the input
of a certain date it returns the date of the day after that of the input.
1 ≤ Day ≤ 31.
1 ≤ month ≤ 12.
1812 ≤ Year ≤ 2012.
(Here the year has been restricted so that test cases are not too large).
There are more complicated issues to consider due to the dependencies between variables.
st
For example there is never a 31 of April no matter what year we are in. The nature of these
dependencies is the reason this example is so useful to us. All errors in the NextDate problem
are denoted by “Invalid Input Date.”
The triangle problem accepts three integers (a, b and c)as its input, each of which are taken to
be sides of a triangle. The values of these inputs are used to determine the type of the triangle
(Equilateral, Isosceles, Scalene or not a triangle).
For the inputs to be declared as being a triangle they must satisfy the six conditions:
C1. 1 ≤ a ≤ 200.
C2. 1 ≤ b ≤ 200.
C3. 1 ≤ c ≤ 200.
C4. a < b + c.
C5. b < a + c.
C6. c < a + b.
The type of the triangle, provided the conditions are met, is determined as follows:
Limitations of BVA
Boundary Value Analysis works well when the Program Under Test (PUT) is a “function of
several independent variables that represent bounded physical quantities”. When these
conditions are met BVA works well but when they are not we can find deficiencies in the
results.
For example the NextDate problem, where Boundary Value Analysis would place an even
testing regime equally over the range, tester’s intuition and common sense shows that we
require more emphasis towards the end of February or on leap years.
The reason for this poor performance is that BVA cannot compensate or take into
consideration the nature of a function or the dependencies between its variables. This lack of
intuition or understanding for the variable nature means that BVA can be seen as quite
rudimentary
BVA can provide a relatively simple and formal testing technique that can be very powerful
when used correctly. When issues arise such as dependencies between variables or a need for
foresight into the system’s functionality, we can find Boundary Value Analysis restrictive (as
shown by the NextDate problem).
The underlying fact is that generally Boundary Value Testing techniques are computationally
and theoretically inexpensive in the creation of test cases. For this reason in many cases it can
be desirable in its results to effort ratio. This means that Boundary Value Analysis still has a
part to play in modern day testing practices.
BVA: Exercise
Is T the best possible test set for findPrice? Answer this question based on T’s ability to
detect missing code for checking the validity of age.
Is there an advantage of separating the invalid values of code and quantity different test
cases?
BVA: Recommendations
Relationships amongst the input variables must be examined carefully while identifying
boundaries along the input domain. This examination may lead to boundaries that are not
evident from equivalence classes obtained from the input and output variables
Additional tests may be obtained when using a partition of the input domain obtained by
taking the product of equivalence classes created using individual variables.
A method for creating functional test suites has been developed in which a test engineer
analyzes the system specification, writes a series of formal test specifications, and then
uses a generator tool to produce test descriptions from which test scripts are written.
The advantages of this method are that the tester can easily modify the test specification
when necessary, and can control the complexity and number of the tests by annotating the
tests specification with constraints.
The method includes the use of formal test specifications and is supported by a generator
tool that produces test case descriptions from test specifications.
The method goes through a series of decompositions, starting with the original functional
specification, and continuing through the individual details of each subprogram being
tested.
The test designer’s key job is to develop categories, which are defined to be the major
characteristics of the input domain of the function under test
The choices in each category must be disjoint, and together the choices in each category
must cover the input domain
A. The test specification is a concise and uniform representation of the test information
for a function.
B. The test specification can be easily modified, if this is necessitated by changes in the
functional specification of a command; mistakes in an original test specification; and a
desire for more or fewer test cases.
C. The test specification gives the tester a logical way to control the volume of tests.
D. The generator tool provides an automated way to produce thorough tests for each
function, and to avoid impossible or undesirable combinations of parameters and
environments.
E. The method emphasizes both the specification coverage and the error detection aspects
of testing.
It helps software testers create test cases by refining the functional specification of a
program into test specifications. It identifies the elements that influence the functions of
the program and generates test cases by methodically varying these elements over all
values of interest.
1. Decompose the functional specification into functional units that can be tested
independently.
2. Identify the parameters (the explicit inputs to a functional unit) and environment
conditions (the state of the system at the time of execution) that affect the execution
behavior of the function.
4. Partition each category into choices, which include all the different kinds of values that
are possible for that category.
5. Determine the constraints among the choices of different categories. For example, one
choice may require that another is absent, or has a particular value.
6. Write the test specification (which is a list of categories, choices, and constraints in a
predefined format) using the test specification language TSL.
7. Use a generator to produce test frames from the test specification. Each generated test
frame is a set of choices such that each category contributes no more than one choice.
8. For each generated test frame, create a test case by selecting a single element from
each choice in that test frame.
– Automatic generator
It helps software testers create test cases by refining the functional specification of a
program into test specifications. It identifies the elements that influence the functions of
the program and generates test cases by methodically varying these elements over all
values of interest.
• Examples
– Or a function
– Identify parameters
- Environmental conditions
Test Cases
– Find categories
• Examples
– Find choices
» Examples: (IE 5.0, IE 4.5, Netscape 7.0), (WindowsNT, Linux), (100, 0, -1)
– List of categories
• Constraints
Each category is partitioned into choices that correspond to one or more values for the
input or the state of an environment object.
Test specifications also contain constraints on the choices so that only reasonable and
valid sets of tests are generated.
Test specifications are input to a test-frame generator that produces a number of test
frames from which test scripts are generated.
A test frame serves as a template for one or more test cases that are combined into one or
more test scripts.
Analyze the specification. The tester identifies individual functional units that can be
separately tested. For each unit, the tester identifies:
3. objects in the environment whose state could affect the functional unit’s operation;
The tester then classifies these items into categories that have an effect on the behavior of
the functional unit.
Partition the categories into choices. The tester determines the different significant
cases that can occur within each parameter/environment category.
Determine constraints among the choices. The tester decides how the choices interact,
how the occurrence of one choice can affect the existence of another, and what special
restrictions might affect any choice.
Write and process test specification. The category, choice, and constraint information is
written in a formal Test Specification. The written specification is then processed by a
generator that produces a set of test frames for the functional unit.
Evaluate generator output. The tester examines the test frames produced by the
generator, and determines if any changes to the test specification are necessary. Reasons
for changing the test specification include the absence of some obviously necessary test
situation, the appearance of impossible test combinations, or a judgment that too many
test cases have been produced.
Transform into test scripts. When the test specification is stable, the tester converts the
test frames produced by the tool into test cases, and organizes the test cases into test
scripts.
Example : findPrice
Function: findPrice takes three inputs: code, qty, and weight Item code is represented by
string of eight digits contained in variable code. The quantity purchased is contained in
qty. The weight of the item purchased is contained in weight.
Function fP accesses a database to find and display the unit price, the description, and the
total price of the item corresponding to code.
fP is required to display error message, and return, if either of the three inputs is
incorrect.
The leftmost digit of the code decides hoe the values of qty and weight are to be used.
code is an eight digit string that denotes product type.
The use of parameters qty and weight depends on the leftmost digit in code.
qty indicates the quantity purchased, an integer, when the left most digit is 0 or 3; weight
is ignored.
weight is the weight of the item purchased when the leftmost digit is 2; quantity is
ignored.
qty is the value of the discount when the leftmost digit is 5; again weight is ignored.
when the leftmost digit is 5, the second digit from the left specifies dollar amount and the
third and fourth digits are cents.
In this step, the tester identifies each functional unit that can be tested separately.
For large systems, a functional unit may correspond to a subsystem that can be tested
Independently.
For each testable unit, the given specification is analyzed and the inputs isolated.
In addition, the objects in the environment, for example, file also need to be identified.
Some characteristics are stated explicitly, others might need to be derived by a careful
examination of the Specification.
fP has three input parameters: code, qty, weight.The specification mention various
characteristics of these Parameters such as their type and interpretation. qty and weight
are related to code.
qty: integer
weight: float
database: contents
For each category, the tester determines different cases against which the functional unit
must be tested.
It is useful to partition each category into at least two subsets, a set containing correct
values and another consisting of erroneous values.
code:
length
Valid (eight digits)
Invalid (less than or greater than eight digits)
leftmost digit
0
2
3
5
others
remaining digits
valid string
invalid string (eg. 0X5987Y)
qty:
integer
valid quantity
invalid quantity (eg. 0)
weight:
float
valid weight
invalid weight (eg. 0)
Environments:
database:
contents
item exists
item does not exist
A test for a functional unit consists of a combination of choices for each parameter and
environment object.
Certain combinations might not be possible while others must satisfy specific
relationships.
A special property written [error] can be assigned to choices that represent error
conditions.
Another special property as [single] allows the tester to specify that the associated choice
is not to be combined with choices of other parameters, or environment objects,
while generating test frames in step 6.
0 [property ordinary-grocery]
2 [property variable-weight]
code:
length
Valid
Invalid [error]
leftmost digit
0 [property ordinary-grocery]
2 [property variable-weight]
3 [property Health-related]
5 [property Coupon]
remaining digits
Valid string
Invalid string [error]
qty:
Valid quantity
Invalid quantity [error]
weight:
Environments:
database:
contents
item exists
item does not exist [error]
Length: valid
Leftmost digit: 2
Remaining digits: valid
qty: ignored
weight: 3.19
database: item exists
Limitations