Black Box Testing

04/20/20 Black Box Testing Slide : 2

What is Black Box Testing?
 Black box testing is done without the knowledge of the
internals of the system under test.

04/20/20 Black Box Testing Slide : 3

Characteristics of Black Box
Testing
 Done based on requirements
 Addresses (should address) stated as well as implied
requirements
 Encompasses the end-user perspective
 Checks for valid and invalid conditions / inputs
 May or may not know the technology aspects of the
product

04/20/20 Black Box Testing Slide : 4

Differences Between White Box and
Black Box Testing

Black Box Testing White Box Testing

Has no access to program Has access to program

code code

Requires external Requires knowledge of

perspective program code

Set of techniques Typically applies only to

applicable to all other unit testing, where code is
phases of testing involved

04/20/20 Black Box Testing Slide : 5

Principles of Black Box Testing
 It is not possible to exhaustively test a product
 Choose tests so that we can
– Test “as much of the external functionality as
possible”
– Uncover “as many defects as possible”
– Use “as short a time as possible”

Have methodologies that choose

tests that have
a higher likelihood of uncovering
new defects.
04/20/20 Black Box Testing Slide : 6
Techniques / Methodologies of
Black Box Testing
 Requirements-based testing
 Positive and negative testing
 Boundary value analysis
 Decision tables
 Equivalence partitioning
 State-based testing
 Compatibility testing
 User documentation testing
 Domain testing (leads to ad hoc testing)

04/20/20 Black Box Testing Slide : 7

General Format of Discussion of
Techniques
 Present some reasoning where applicable.
 List out one or two examples.
 Walk through the examples.
 Summarize the process for using the
technique.

04/20/20 Black Box Testing Slide : 8

Requirements-Based Testing
 Done to ensure that all requirements in SRS are tested
 Difference between implicit and explicit requirements
 Review requirements first to ensure they are
consistent, correct, complete and testable
 Review enables translation of (some of) the implied
requirements to stated requirements
 A reviewed SRS tabulates requirements, along with a
requirements id and a priority
 This is the genesis of a Requirements Traceability
Matrix

04/20/20 Black Box Testing Slide : 9

Positive and Negative Testing
 Positive testing to check that the product does what
it is supposed to
– Behaves correctly when given right inputs
– Maps to a specific requirement
– “Coverage” is defined better
 Negative testing to show that the product does not
fail when given unexpected inputs
– Tries to break the system
– No direct mapping to a specific requirement
– “Coverage” more challenging
 Table 4.4 and 4.5, Page 83

04/20/20 Black Box Testing Slide : 10

Boundary Value Analysis
 Most defects come up near “boundaries”
 Reasons from a white box perspective:
– Programmers tentativeness in using the right relational
operator (< or < = ?)
– Multiple ways of implementing loops
– Confusing array subscripts
 Reasons from a black box perspective:
– Unclear requirements
– Ambiguous or “it depends” mindset!

04/20/20 Black Box Testing Slide : 11

Boundary Value Analysis (BVA):
Summary of Process
 Look for any kind of gradation or discontinuity in data values that
affects computation – the discontinuities are the boundary values,
requiring thorough testing.
 Look for any internal limits like limits on resources (like the
example of buffers given above). The behaviour of the product at
these limits should also be the subject of boundary value testing.
 Also include in the list of boundary values, documented limits on
hardware resources. For example, if it is documented that a
product will run with minimum 4MB of RAM, make sure you
include test cases for the minimum RAM (i.e., 4MB in this case).
 The examples given above discuss boundary conditions for input
data – the same analysis needs to be done for output variables
also.

04/20/20 Black Box Testing Slide : 12

Decision Tables
 A program’s behaviour is characterized by several
decision variables.
 Each decision variable specifies a boolean condition.
 The distinct combinations of these decision variables
lead to different scenarios.
 Each scenario occupies a row in the decision table,
and the row also has expected results.
 One representative data point from each scenario
needs to be tested.

04/20/20 Black Box Testing Slide : 13

Summary of Decision Table
 Identify the decision variables.
 Identify the possible values of each of the decision variables.
 Enumerate the combinations of the allowed values of each of the
variables.
 Identify the cases in which values assumed by a variable (or by
sets of variables) are immaterial for a given combination of other
input variables. Represent such variables by the Don’t Care
symbol.
 For each combination of values of decision variables
(appropriately minimized with the Don’t Care scenarios), list out
the action or expected result.
 Form a table, listing on each but the last column one decision
variable. On the last column, list the action item for the
combination of variables in that row (including Don’t Cares, as
appropriate)

04/20/20 Black Box Testing Slide : 14

Equivalence Partitioning (EVP)
 Generalization of BVA / decision table
 Divide the (potentially infinite) set of values into a
set of equivalence classes or partitions
 One element of the class can act as a
representative for the entire class
 Results of the test for that one element is
extrapolated to all the other elements

04/20/20 Black Box Testing Slide : 15

Summary of Process for EVP
 Choose criteria for doing the equivalence partitioning (range, list
of values, etc.).
 Identify the valid equivalence classes based on the above criteria
(number of ranges, allowed values, etc.) .
 Select a sample data from that partition.
 Write the expected result based on the requirements given.
 Identify special values, if any, and include them in the table.
 Check to have expected results for all the cases prepared
 If the expected result is not clear for any particular test case, mark
appropriately and escalate for corrective actions. If you cannot
answer a question, or find an inappropriate answer, consider
whether you want to record this issue on your log.

04/20/20 Black Box Testing Slide : 16

State- or Graph-Based Testing
 Useful for
– Language processors or compilers
– Workflow modelling
– Dataflow modelling

04/20/20 Black Box Testing Slide : 17

Process for Language processors
Testing Using Graph Method
 Identify the grammar for the scenario. In the above
example, we have represented the diagram as a
state machine. In some cases, the scenario can be a
context-free grammar, which may require a more
sophisticated representation of a “state diagram”.
 Design test cases corresponding to each valid state-
input combination.
 Design test cases corresponding to the most
common invalid combinations of state-input.

04/20/20 Black Box Testing Slide : 18

 Requisition Approval
Approving
Authority Finance
Requestor

Budget Update &

y

Purchase Order
er
eli v

Delivery
D
m
Available Ite

Payment
Buy &

Note
Off the shelf Expense
(Transaction:
Expense Report)

Received
Item
Vendor

04/20/20 Black Box Testing Slide : 19

Workflow Modelling

Manager Feasible
Employee Verify
Ensure
Desires Leave Eligibility Approve
Feasibility
Eligible

Leave
Not Feasible
Application
Form Ineligible

Reject

04/20/20 Black Box Testing Slide : 20

General Characteristics of Graph-
Based Testing
 The application can be characterized by a set of
states.
 The data values (screens, mouse clicks, etc.) that
cause the transition from one state to another are
well understood.
 The methods of processing within each state to
process the input received are also well understood.

04/20/20 Black Box Testing Slide : 21

Compatibility Testing
 To ensure that the product works consistently with
infrastructure components
 Could be parameters of hardware, OS, network…
 Concept of a compatibility matrix (Table 4.11, Page 98)
 Backward compatibility and forward compatibility

04/20/20 Black Box Testing Slide : 22

User Documentation Testing
 To check if what is stated in the document is available
in the product
 To check if what is there in the product is available in
the document
 Documents include user guides, installation guides,
set-up guides, read-me files, software release notes
and on-line help

04/20/20 Black Box Testing Slide : 23

Benefits of Documentation Testing
 User documentation testing aids in highlighting
problems that have been overlooked during reviews.
 High-quality user documentation ensures the
consistency of documentation and product, thus
minimizing possible defects reported by customers.
 Results in less difficult support calls.
 Contributes to better customer satisfaction and better
morale of support staff.
 New programmers and testers who join a project
group can use the documentation to learn the external
functionality of the product.
 Customers need less training and can proceed more
quickly to advanced training and product usage.
 Reduced training costs for customers.

04/20/20 Black Box Testing Slide : 24

Domain Testing

04/20/20 Black Box Testing Slide : 25

Domain Testing
 Have “domain people” perform tests by using the
software
 Business flow determines the test, not “logic” or
“steps”
 Tests what the users do on a typical business day
 Captures the typical problems faced by users (not
necessarily captured in SRS)

04/20/20 Black Box Testing Slide : 26

When to Use What…
When you want to test scenarios that have… The most effective black box testing technique
is likely to be…
Output values are dictated by certain Decision Tables
conditions depending upon values of
input variables
Input values being in ranges, with each range Boundary Value Analysis
exhibiting a particular functionality
Input values being divided into classes (like Equivalence Partitioning
ranges, list of values, etc), with each class
exhibiting a particular functionality
Positive and negative testing
Workflows, process flows or language Graph based testing
processors
To ensure that requirements are tested and Requirements Based Testing
met properly
To test using the domain expertise rather than Domain Testing
the product specification
To ensure that the documentation is Documentation Testing
consistent with the product

04/20/20 Black Box Testing Slide : 27