Black Box Testing

Amaar Hassan
Black Box Testing
 Block-box testing (or functional testing, or behavior testing) focuses on the
functional requirements of the software.
 In black-box testing, the system is like a black-box whose behavior can
only be determined by studying its inputs and the related outputs
 A set of test cases is derived to exercise all functional requirements for the
program.
 These tests are generally derived from the program specification
 Black-box testing attempts to find the error in the following categories
 Incorrect or missing functions
 Interface errors
 Error in data structures or external database access
 Performance errors
 Initialization and termination error
Black Box Testing
Black Box Testing
 Equivalence classes
 Boundary value analysis
Why testing is hard
 Let’s take a trivial example
 Test the addition operation on a 32-bit machine

c=a+b
 How many tests are needed?

 How many values for a? 232

 How many values for b? 232
 Total possible input combinations? 232 x 232 = 264
Why testing is hard
 Total possible input combinations? 232 x 232 = 264
 Assume:
 1 addition test requires 1 CPU cycle to execute
 3GHz machine
 1GH = 1,000,000,000 Hz = 108
 1GHz means 1 billion cycles per second
 One addition test/~1 cycle = 3x108 test/sec
 Time = 264 / 3x108 = 6 x 1010 sec
= 694,445 days

 Clearly need a smarter technique!!

An Efficient way
 Many tests are redundant
 In the adder example, most tests are equivalent.
 They do not exercise any new part of the underlying circuit
 For example, you might argue that
all additions of +ve numbers without overflow are equivalent
Addition of 0 to a +ve number is the same for all +ve numbers
Similarly for 0 + -ve number … etc.

 This divides the tests into Equivalence Classes

 Only one representative of each class need be tested!
Equivalence Classes
 A set of equivalence classes is a partition of a set such that
Each element of the set is a member of exactly one
equivalence class

 Only one representative of each class needs to be tested!

 All other tests of inputs in the same equivalence class just repeat the
first one!
Dramatic reduction in total number of tests
No loss of ‘coverage’ or satisfaction that tests are complete
 Equivalence classes

Adder Example

Clearly, we’ve achieved a dramatic reduction in

number of required tests!
Equivalence classes (Formal Def.)
 The elements of an equivalence class, C, are classified by an
equivalence relation, ~
 If a  C and b  C , then a ~ b
 The equivalence relation is
 Reflexive a ~ a
 Transitive if a ~ b and b ~ c, then a ~ c
 Symmetric if a ~ b then b ~ a

 A Representative of each class is an arbitrary member of the class

 They’re all ‘equivalent’ – so choose any one!
 Equivalence classes

Age Example

 Equivalence classes are derived from rules in the specification

eg admission price: if age < 6, then free
if age < 16, then 50%
else full price
would lead to 3 equivalence classes:
age < 6; age  6  age < 16; age  16;

 Choose representatives 3, 9 and 29 (or many other sets)

Equivalence classes
Other special cases
 Nulls
 Identity under addition: x + 0 = x
 Unity
 Identity under multiplication: x  1 = x
 Range Maxima and Minima
 illegal values
 Should raise exceptions or return errors
 Read the specification to determine behavior!
Equivalence classes
 Consider this function
int max( int a, int b ){
if( a > b ) return a;
else return b;
}
 There are two paths through this code, so the inputs falls into two
classes
Those for which a > b and
the rest

 This implies that we have only two tests to make:

 (a=5, b=3) and
 (a=4, b=6)
Boundary Value Analysis (BVA)
 If you can safely and confidently walk along the edge of a cliff without
falling off, you can almost certainly walk in the middle of a field.
 If software can operate on the edge of its capabilities, it will almost
certainly operate well under normal conditions.
Boundary Value Analysis (BVA)
 First–1/Last+1
 Start–1/Finish+1
 Less than Empty/More than Full
 Even Slower/Even Faster
 Largest+1/Smallest–1
 Min–1/Max+1
 Just Over/Just Under
 Even Shorter/Longer
 Even Sooner/Later
 Highest+1/Lowest–1
Example of EC
 A Program which edits the credit limits within a given range
($10,000 - $15,000) would have 3 equivalence classes
< 10,000 (invalid)
Between 10,000 and 15,000 (valid)
> 15,000 (invalid)
Example of BVA
 A Program which edits the credit limits within a given range
($10,000 - $15,000) would have 3 equivalence classes
Lower boundary: - / + 1 (9,999 and 10,001)
On the boundary: (10,000 and 15,000)
Upper boundary: - / + 1 (14,999 and 15,001)
Types of testing under B.B.T.
The types of testing performed under Black box
testing
Integration testing (partial)
System testing
Acceptance testing
Regression testing (partial)
Testing after new changes are made
System testing
 System testing is testing conducted on complete, integrated
system to evaluate system compliance with its requirements.
 It comes after Integration testing

 Objectives
 To check whether the software system is made according to the
customer needs
 To check for both functional and non-functional testing
 To execute real life scenarios on the software
Acceptance testing
 Acceptance testing is testing performed to check the acceptability of
the system.
 Acceptance Testing is performed after System Testing and before
making the system available for actual use.

 There are two types of Acceptance testing

1. Internal Acceptance testing (Alpha Testing)
 It is performed by members of the organization that developed the
software but are not directly involved in the project (Development or
Testing).
 Usually, it is the members of Product Management, Sales and/or
Customer Support and it is performed inside the software
organization.
Acceptance testing
2. External Acceptance testing
It is performed by customer and users.
It has two types

Customer Acceptance Testing

This is done by customers who asked the organization to build the
software for them.
User Acceptance Testing (Beta Testing)
 This is done by the final users of the software at their place
The Testing path
In a bigger picture
 During the process of manufacturing a ballpoint pen, the cap, the
body, the tail and clip, the ink cartridge and the ballpoint are
produced separately and unit tested separately.
 When two or more units are ready, they are assembled and
Integration Testing is performed.
 When the complete pen is integrated, System Testing is performed
to see if meets the specification.
 Once System Testing is complete, Acceptance Testing is performed
so as to confirm that the ballpoint pen is ready to be made available
to the end-users.
All in All (WBT and BBT)
Differences
between
WBT and
BBT
Gray Box Testing
 Gray box testing is a combination of white and black box testing

 The tester knows a little about internal working and functional

requirements as well.
 In that way tester is in better position to judge the implementation
as well as check conformance with the requirements.
 Regarded as unbiased testing
Gray Box Testing…Example
 Validation of a website is tested in a grey-box way
 Grey box tester knows that validation is done using a JavaScript at
client-side but does not know implementation details
 Now that we know the underlying mechanism, we can challenge it,
for its strengths and weaknesses.
 He can turn off library and see how system would handle validation
without JavaScript library

 One can perform better testing of mobile if he has a little idea about
its working (OS and Hardware).
Gray Box Testing…Example
 Which scenario presents higher probability for a good testing?

 1. Testing a car without knowing the technologies It uses.

 1. Testing a car, while knowing the technologies (hw sw) it uses
That’s all folks