Software Testing 5 Units

Phases of Software Project
❑ Requirement gathering and Analysis

❑ Planning
❑ Design
❑ Development or Coding
❑ Testing
❑ Deployment
❑ Maintenance
Requirement gathering and Analysis
❑ The specific requirements of the software to be built are
gathered and documented.
❑ It is important to ensure that the right requirements are
captured.
❑ The requirements get documented in the form of a System
Requirement Specification (SRS) document.
❑ SRS act as a bridge between the customer and designers.
Planning
❑ The planning phase comes up with a schedule, the scope, and
resource requirements for a release.
❑ A plan defines how the requirements will be met and by which
time.
❑ Planning take into account the requirements - what will be met
and what will not be met- for the current release, look out
resource availability, and to come out with set of milestones and
release date for the project.
❑ The phase is applicable for both development and testing
activities.
❑ Both project plan and test plan documents are delivered.
Design
❑ The phase figures out how to satisfy the requirements stated in
the SRS documents.
❑ Design in split into two levels: high-level design and low-level or
detailed design.
❑ The phase delivers a representation and System Design
Description (SDD) – data design, architecture design, interface
design, procedural design, that will be used by the development
phase.
Development or Coding
❑ coding the programs in the chosen programming language.
❑ Also involves in the creation of product documentation.
❑ The phase delivers the software that meets the requirements.
Testing
❑ The product is subjected to testing after the coding is
completed.
❑ As the programs are coded, they are also tested.
❑ Testing is the process of exercising the software product in the
pre-defined ways to check if the behavior is the same as
expected behavior.
❑ The phase delivers the Quality ensured Defect-free product.
Deployment
❑ A tested product is deployed in the customer environment.
❑ As the users start using the product in their environments, they
may observe discrepancies (which end up as product defects)
between the actual behavior of the product and the expected.
Maintenance
❑ The product is maintained to satisfy the changes that arise from
customer expectations, environmental changes, etc.
❑ Maintenance is made up of
✓corrective maintenance (ex: fixing customer-reported
problems)
✓adaptive maintenance (ex: making the software run on a
new version of OS or d/b)
✓preventive maintenance (ex: changing the application
program to avoid potential security hole on OS code)
Quality, Quality Assurance, and Quality Control
❑ Requirements get translated into software features
❑ Each feature being designed to meet one or more of the
requirements.
❑ For each feature, the expected behavior is characterized by a set
of test cases.
❑ Each test case is further characterized by:
➢the environment under which the test cases is to be
executed
➢ inputs that should be provided for that test case
➢ how these inputs should be processed
➢ what changes should be produced in the internal state on
the environment
➢what outputs should be produced.
❑ If the actual behavior and the expected behavior are identical in
all their characteristics, then the test case is said to be passed. If
not, the given software is said to have a defect on that test case.
❑ Quality control and Quality assurance are the methods used
to increase the chances of a product meeting the requirements
expected.
❑ Quality control attempts to build a product, test it for
expected behavior after it is built, and if the expected behavior is
not the same as the actual behavior of the product it fixes the
product as necessary and rebuilds the product.
❑ The process is repeated till the expected behavior of the
product matches the actual behavior for the scenarios tested.
❑ Quality control is defect-detection and defect-correction
oriented, and works on the product rather than on the process.
❑ Quality Assurance attempts defect prevention by
concentrating on the process of producing the product.
❑ The quality Assurance approach reviews the design before
the product is built and corrects the design errors.
❑ To ensure the production of a better code, a quality
assurance process may mandate coding standards to be
followed by all programmers.
Testing, Verification, and Validation
❑ The purpose of testing is to uncover defects in the system.
❑ Verification and validation catch defects within each phase,
without letting them reach the testing phase.
❑ Verification takes care of activities to focus on the question "Are
we building the product right?” The activities imposed to build
the product right are considered as "proactive". Verification
prevents the defects before they take shape.
❑ Validation takes care of a set of activities to address the
question "Are we building the right product?". The activities
carried out to validate whether the product is built as per
specifications are considered as "reactive". Validation finds
defects that affect the product and fix them as soon as they are
introduced.
Process Model to Represent Different Phases
❑ A process model (Entry Task Verification eXit - ETVX model) is a
way to represent any given phase of software development to
prevent and minimize the delay between defect injection and
defect detection.
❑ In this model, each phase of a software project is characterized
by
✓ Entry criteria - which specify when that phase can be

started, and the input for that phase.
✓ Tasks need to be carried out in that phase, along with the
measurements that characterize the tasks.
✓ Verification, which specifies methods of checking that the
tasks have been carried out correctly.
✓ Exit criteria - which can stipulate the conditions under
which one can consider the phase as done. Also the outputs
for that phase are also included.
Process Model to Represent Different Phases
❑ Advantages of ETVX model

✓Clear entry criteria
✓ Verification of each phase
✓ Documentation of the detailed task that comprise each
phase
✓ Clear exit criteria
Life Cycle Models
Characteristics of Life Cycle Models:
❑ The activities performed - technical activities (porting) and
non-technical activities (hiring); apart from common activities
❑ The deliverables form each activity: - Ex: SRS, SDD documents
❑ Methods of validation of the deliverables - It is necessary to have
proper validation criteria for each output.
❑ The sequence of activities - For example, the process of requirement
gathering may involve steps such as interview with customers,
documentation of requirements, validation of documented
requirements with customers, and freezing of requirements.
Life Cycle Models
Characteristics of Life Cycle Models….
❑ Methods of verification of each activity, including the mechanism of
communication amongst the activities - For example, when a defect
is found in one activity and is traced back to the causes in an earlier
activity, proper verification methods are needed to retrace steps from
the point of defect to the cause of the defect.
Life Cycle Models
Requirement
Water fall Model s Gathering
& Analysis
❑ A project is divided into a set of
phases (or activities). Plan
❑ A project starts with an initial phase,

Design
and upon completion of the phase,
moves on to the next phase.
Development
❑ The phases are strictly time
sequenced. Test
Deployment
Maintenance
Life Cycle Models
Requirement
& Analysis
❑ A waterfall model is characterized by
three attributes: Plan
✓ The project is divided into

Design
separate distinct phases.
✓ Each phase communicates to the

Development
next through pre-specified
outputs Test
✓ When an error is detected, it is

Deployment
traced back to one previous
phases at a time, until it gets Maintenance
resolved at some earlier phase.

Life Cycle Models
Requirement
& Analysis
❑ A problem in one phase could
potentially be traced back to any of Plan
the previous phase.
❑ Before a phase starts, the completion

Design
criteria for the previous phase can be

Development
checked and this can act as a
verification mechanism for the phase. Test
Deployment
Maintenance
Life Cycle Models
Requirement
& Analysis
Advantages
❑ The main strength is its simplicity. Plan
❑ A schedule can be set with deadlines
for each stage of development. Design
❑ Phases are processed and completed

Development
one at a time.
❑ Works well for smaller projects where
Test
requirements are very well
understood. Deployment
Maintenance
Life Cycle Models
Requirement
& Analysis
Disadvantages
❑ Not a good model for complex and Plan
object-oriented projects
❑ Not suitable for projects where Design
requirements are subject to high risk

Development
of changing.
Test
Deployment
Maintenance
Life Cycle Models
Spiral or Iterative Model
❑ In this model, the requirements
gathering, design, coding, and testing are
performed iteratively till the
requirements are met.
❑ If a defect is produced in any phase of a
given requirement, it may cause that
requirement to revisit an earlier phase.
Life Cycle Models
Spiral or Iterative Model
❑ The model enables incremental
development whereby the product
evolves with requirements getting added
to it dynamically.
❑ Even though it will be very difficult to
plan a release date, the model allows the
progress to be tracked.
❑ The customer approvals to be obtained
at regular intervals, thereby reducing the
risk of finding major defects at a later
point of time.
Life Cycle Models
The V-Model
❑ The waterfall model viewed testing as a
post-development activity.
❑ The Spiral model breaks up the product
into increments each of which can be
tested separately.
❑ In V model different types of testing
apply at different levels.
Life Cycle Models
The V-Model
❑ The kinds of tests apply at each level are:
✓ Overall business requirements -
acceptance testing is done with the final
product.
✓ Before the product gets deployed in
customer’s environment – system
testing on the entire unit of the software
system.
✓ High-level design corresponds to
integration testing, where individual
subsystems should be integrated and
tested.
Life Cycle Models
The V-Model
✓ Low-level design phase corresponds
with component testing, where the
components are tested independently
before being integrated.
✓ The smaller program units have to be
tested independently before forming as
components, and this is called as unit
testing.
Life Cycle Models
The V-Model
❑ The V-model splits testing into two parts- design and execution.
❑ Test design is done early, while test execution is done in the end.
Life Cycle Models
The Modified V-Model
❑ An assumption made with V model was that even though the
activity of test execution was split into execution of tests of different
types, the execution cannot happen until the entire product is built.
❑In V model, for a given product, the different units and components
can be in different stages of evolution.
❑ For example, one unit could be still under development and thus be
in unit testing phase whereas another unit could be ready for
integration testing.
Life Cycle Models
❑ In the modified V Model, each unit or component or module is
given explicit exit criteria to pass on to the subsequent stage.
❑ Such units or components or modules that satisfy a given phase of
testing move to the next phase of testing, without waiting for all
the units or components or modules to move from one phase of
testing to another.
Life Cycle Models
❑ Rather than view the product as going through different types of
tests (V model), the modified V model views each part of the
product to go through different phases of testing.
❑ A phase of testing has a one-to-one mapping to the types of testing.
❑ For example, once a unit has completed the unit testing phase, it
becomes part of a component and enters the component testing
phase. It then moves to integration testing phase and so on.
❑ While starting a phase of testing, the readiness of the product for
the testing is determined by a set of entry criteria. The exit criteria
determine the completeness of the test activities for each phase.
White Box Testing
❖ What is White Box Testing?
❖ Static Testing
✓ Static Testing by Humans
✓ Static Analysis Tools
❖ Structural Testing
✓ Unit/Code Functional Testing
✓ Code Coverage Testing
✓ Code Complexity Testing
❖ Challenges in White Box Testing
What is White Box Testing?
❑ Testing the external functionality of the code
❑ White box testing takes into account the program code, code
structure, and internal design flow.
❑ Known also as clear box, or glass box or open box testing.
❑ It reduces the delay between the injection of a defect in the
program code and its detection.
Static Testing
❑ Stating testing requires only the source code of the product, not the
binaries or executables.
❑ It does not execute the programs on computers
❑ It involves selected people going through the code to find out
whether
✓ the code works according to the functional requirement,
✓ the code is written according to the design developed,
✓ the code for any functionality has been missed out,
✓ the code handles error properly.
Static Testing
Static Testing by Humans
❑ A proactive testing method.
❑ Humans read the program code to detect errors rather than
executing the code to find errors.
Advantages
❑ Sometimes humans can find errors that computer cannot. For
example: declaration of similar variables and wrong variable used by
mistake in the expression.
❑ Multiple perspectives can be received on evaluation of multiple
humans.
❑ Can compare the code against the specifications
❑ Can also try to identify the root cause of the detected errors.
Static Testing
Static Testing by Humans
Advantages (Cont…..)
❑ By making humans test the code before execution, computer
resources can be saved at the expense of human resources.
❑ Minimizes the delay in identification of the problems.
Static Testing Methods

❑ Desk Checking of the code
❑ Code Walkthrough
❑ Formal/Code Inspection
Static Testing
Static Testing Methods: Desk Checking
❑ A method to verify the portions of the code for correctness by
comparing the code with the design or specifications to make sure
that the code does what it is suppose to do.
❑ Done by the author before compiling and executing the code.
❑ Whenever errors are found, the author applies the corrections.
❑ The method is characterized by
✓no structured method or formalism to ensure completeness
and effectiveness
✓no maintaining of a log of checklist
❑Effective for coding errors and will not be effective in detecting
errors due to incorrect understanding or incomplete requirements.
Static Testing
Static Testing Methods: Desk Checking (Contd……)
Advantages
❑ A skilled programmer can read and understand his own code
❑ The defects are detected and corrected with minimum time delay.
Disadvantages
❑ The developer may be tunnel visioned and have blind spots to
certain types of problems.
❑ Developers prefer to write code rather than do any form of testing.
❑ The method is person dependent and informal
Static Testing
Static Testing Methods: Code Walkthrough
❑ a group oriented method
❑ Code walkthrough brings multiple perspectives
❑ A set of people look at the program code and raise questions for
the author.
❑ The author finds the answers for unanswered questions.
Static Testing
Static Testing Methods: Formal Inspection
❑ Code inspection (Fagan Inspection) detects all faults, violations,
and other side-effects
❑ Increases the number of defects detected by,
✓ demanding through preparation before an inspection/review
✓ Enlisting multiple diverse views
✓ assigning specific roles to the multiple participants
✓ going sequentially through the code in a structured manner.
❑ When the code is ready for inspection, an inspection meeting
(defect logging meeting) is arranged.
Static Testing
Static Testing Methods: Formal Inspection (Cont…..)
❑ Roles in the inspection
✓ author of the code
✓ moderator – who is expected to formally run the inspection
according to the process.
✓ inspectors – people who actually provides, review comments
for the code
✓ scribe – who takes detailed notes during the inspection meeting
and circulates them to the inspection team after the meeting.
❑ The author or the moderator selects the review team.
Static Testing
❑ In an introductory meeting, the inspectors get copies of the code to
be inspected along with the supported documents such as design
document, requirements document, etc. The inspectors get
adequate time to go through the document and the program.
❑ The author also presents his perspective of what the program is
intended to do along with any specific issue that the inspection team
to put extra focus on.
❑ The moderator informs the team about the date, time, and venue
of the inspection meeting.
❑ In an inspection meeting, the moderator asks each inspector if
there are any defects in the part of the code.
Static Testing
❑ If any of the inspectors raises a defect, then the inspection team
deliberate on the defect and, when agreed that there is a defect,
classifies it in two dimensions – minor/major and
systemic/misexecution.
❑ A mis-execution defect happens because of an error on the part of
the author. Example, a wrong variable in the statement.
❑ A systemic defect requires correction at a different level.
❑ Minor defects may not significantly affect a program, and major
defects need immediate attention.
Static Testing
❑ A scribe documents the defect found in the inspection meeting and
the author takes care of fixing these defects.
❑ Formal inspections have been found very effective in catching
defects early.
Challenges in formal inspections:
❑ Time consuming
❑ The organizing and scheduling can become an issue since multiple
people are involved
❑ Not always possible to go through every line of code, with several
parameters and their combinations in mind.
Static Testing
Static Analysis Tools
❑ A Static Analysis Tools reduces the manual work involved in formal
inspection
❑Performs analysis of the code to find out errors such as:
➢ Unreachable codes
➢ Variables declared but not used
➢ Mismatch in definition and assignment of values to variables
➢ Illegal or error prone typecasting of variables
➢ Use of non-portable or architecture-dependent programming
constructs
➢ Memory allocated but not having corresponding statements
for freeing up them.
Static Testing
Static Analysis Tools (Cont…..)
❑These tools are also considered as an extension of compilers as they
use the same concepts to locate errors.
❑ Some tools can also check for:
➢ fulfillment of coding standards as prescribed by standards such
as POSIX (Portable Operating System Interface for UNIX).
➢ consistency in coding guidelines, such as naming conventions,
allowed data types etc.
Static Testing
❑ It is useful to have a checklist while using any of the human checking
methods.
❑ A checklist may be at two levels:
➢ Organization-wide checklist – includes issues such as
organization coding standards, documentation standards, etc.
➢ Product or project-specific checklist – addresses issues specific
to the product or project.
Static Testing
Fig: Code Review Checklist
Static Testing
Static Testing
Static Testing
Structural Testing
❑ Structural testing takes into account the code, code structure,
internal design, and how they are coded.
❑ Tests are done by executing the built product, whereas in static
testing, the product is tested by humans using just the source code
and not the executables or binaries.
❑ The actual product is run against some pre-designed test cases to
exercise as much of the code as possible or necessary.
❑ Structural testing can be further classified into
➢ unit/code functional testing
➢ code coverage testing
➢ code complexity testing
Structural Testing
Unit/Code Functional Testing
❑ A developer performs unit/code functional testing before subjecting
the code to more extensive code coverage testing or code complexity
testing.
❑ The several methods of unit testing are:
(1)
✓ The developer can test, knowing the input variables and the
corresponding expected output variables.
✓ This method checks out any noticeable mistakes.
✓ The tests are repeated for multiple values of input variables.
✓ This can even be done prior to formal reviews of static testing.
Structural Testing
(2)
✓ For modules with complex logic or conditions, the developer can
build a "debug version" of the product by putting intermediate
print statements and making sure the program is passing
through the right loops and iterations the right number of
times.
✓ It is important to remove the intermediate print statements
after the defects are fixed.
Structural Testing
(3)
➢Run the product under a debugger or an Integrated
Development Environment (IDE).
➢These tools allow
✓single stepping of instructions (allowing the developer to
stop at the end of each instruction, view or modify the
contents of variables, and so on),
✓ setting break points at any function or instruction, and
✓viewing the various system parameters or program
variable values.
Structural Testing
Code Coverage Testing
❑ Code coverage testing involves designing and executing test cases
and finding out the percentage of code that is covered by testing.
❑ The percentage of code covered by a test is found by a technique
called instrumentation of code, and specialized tools are available
to achieve instrumentation.
❑ Instrumentation rebuilds the product, linking the product with a set
of libraries provided by the tool vendors. This instrumented code can
monitor and keep an audit of what portions of code are covered.
❑ The tools also allow reporting on the portions of the code that are
covered frequently, so that the critical or most-often portions of
code can be identified.
Structural Testing
Code Coverage Testing (Cont….)
❑ Code coverage testing is made up of the following types of coverage.
1. Statement coverage
2. Path coverage
3. Condition coverage
4. Function coverage
1. Statement Coverage
❑ Program constructs in most conventional programming languages

can be classified as
1. Sequential control flow
2. Two-way decision statements like if then else
3. Multi-way decision statements like Switch
4. Loops like while do, repeat until and for
Structural Testing
1. Code Coverage Testing: Statement Coverage
❑ Statement coverage refers to writing test cases that execute each of
the program statements.
❑ For a section of code that consists of statements that are
sequentially executed, test cases can be designed to run through
from top to bottom.
❑ For two-way decision construct like if statement, we should also
have test cases to cover the then and else parts.
❑ To cover all possible switch cases, there would be multiple test
eases.
Structural Testing
1. Code Coverage Testing: Statement Coverage (Cont….)
❑ Loops fail in what are called 'boundary conditions'. One of the
common looping errors is that the termination condition of the loop
is not properly stated.
❑ In order to make sure that there is better statement coverage for
statements within a loop, there should be test cases that
1. Skip the loop completely, so that the situation of the
termination condition being true before starting the loop is tested.
2. Exercise the loop between once and the maximum number
of times, to check all possible "normal" operations of the loop.
3. Try covering the loop, around the "boundary" of n—that is,
just below n, n, and just above n.
Structural Testing
1. Code Coverage Testing: Statement Coverage (Cont….)
Statement coverage =
(total stmts. exercised / total no. of executable stmts. in program) * 100
❑ Even a very high level of statement coverage does not mean that
the program is defect-free.
❑ If the program implements wrong requirements and this wrongly
implemented code is "fully tested," with 100 percent code coverage,
it still is a wrong program and hence the 100 percent code coverage
does not mean anything.
Structural Testing
2. Code Coverage Testing: Path Coverage
❑ In path coverage, a program is split into a number of distinct paths.
❑ A program (or a part of a program) can start from the beginning and
take any of the paths to its completion.
Path coverage =
❑ Regardless
(totalof the exercised
paths number of/ total
statements in each
no. of paths of these*paths,
in program) 100 if we
can execute these paths, then we would have covered most of the
typical scenarios.
❑ Path coverage provides a stronger condition of coverage than
statement coverage.
Structural Testing
2. Code Coverage Testing: Path Coverage (Cont…)
✓A
✓ B-D-G
✓ B-D-H
✓ B-C-E-G
✓ B-C-E-H
✓ B-C-F-G
✓ B-C-F-H
Structural Testing
3. Code Coverage Testing: Condition Coverage
❑ Even if we have covered all the paths possible, it would not mean that the
program is fully tested.
❑ Example 1: we can make the program take the path A by giving a value
less than 1 (for example, 0) to mm and find that we have covered the path
A and the program has detected that the month is invalid. But, the
program may still not be correctly testing for the other condition namely
mm > 12.
❑Example 2: When there is an OR condition (as in the first IF statement
above), once the first part of the IF (for example, ro < 1) is found to be
true, the second part will not be evaluated at all. Similarly, when there is
an AND condition in a Boolean expression, when the first condition
evaluates to FALSE, the rest of the expression need not be evaluated at all.
Structural Testing
3. Code Coverage Testing: Condition Coverage (Cont…)
❑ For all these reasons, it is necessary to have test cases that exercise
each Boolean expression and have test cases test produce the TRUE
as well as FALSE paths.
Condition coverage =
(total decisions exercised / total no. of decisions in program) * 100
Structural Testing
4. Code Coverage Testing: Function Coverage
❑ The requirements of a product are mapped into functions during the
design phase and each of the functions form a logical unit.
❑ For example, in a database software, 'inserting a now into the
database' could be a function.
❑ Function coverage technique identifies how many program functions
are covered by test cases.
❑ While providing function coverage, test cases can be written to
exercise each of the different functions in the code.
Structural Testing
4. Code Coverage Testing: Function Coverage (Cont…)
❑ The advantages of function coverage than the other types or
coverage are:
1. Functions are easier to identify and hence it is easier to write
test cases to provide function coverage.
2. It is easier to achieve 100 percent function coverage than 100
percent coverage in any of the earlier methods.
3. Functions have a more logical mapping to requirements and
hence can provide a more direct correlation to the test coverage of
the product.
4. It provides a way of testing requirement traceability.
5. It is easier to prioritize the functions for testing.
Structural Testing
4. Code Coverage Testing: Function Coverage (Cont…)
❑ Most often called functions can be identified and hence these
functions become the target of any performance testing and
optimization.
Function coverage =
(total functions exercised / total no. of functions in program) * 100
Structural Testing
4. Code Coverage Testing
❑ Other uses of code coverage methods:
❑Performance analysis & optimization on most frequently
executed code.
❑ Checking of critical sections or concurrency related parts of
code: Critical sections are those parts of a code that cannot have
multiple processes executing at the same time. Instrumented
code identifies such violations in the critical sections.
❑Identifying memory leaks: Most debuggers or instrumented
code can tally allocated and freed memory.
❑ White box testing can help identify security holes effectively,
especially in a dynamically generated code.
Structural Testing
Code Complexity Testing
❑ Two questions may arise while using coverage:
1. Which of the paths are independent? If two paths are not
independent, then we may be able to minimize the number of tests.
2. Is there an upper bound on the number of tests that must be
run to ensure that all the statements have been executed at least
once?
❑ Cyclomatic complexity is a metric that quantifies the complexity of a
program and provides answers to the above questions.
Cyclomatic Complexity = Number of predicate nodes + 1
Cyclomatic Complexity = E - N + 2
Structural Testing
❑ A program is represented in the form of a flow graph. A flow graph
consists of nodes and edges.
❑ In order to convert a standard flow chart into a flow graph to
compute cyclomatic complexity, the following steps can be taken.
1. Identify the predicates or decision points in the program.
2. Ensure that the predicates are simple (i.e., no and/or).
If there are loop constructs,
break the loop
termination checks
into simple predicates.
Structural Testing
3. Combine all sequential statements into a single node.
4. When a set of sequential statements are followed by a simple
predicate, combine all the sequential statements and the predicate
check into one node and have two edges starting from this one node.
Such nodes with two edges starting from them are called predicate
nodes.
5. Make sure that all the edges terminate at some node.
Structural Testing
Structural Testing
❖ no decision points.
❖ has two nodes, one for the code and
one for the termination node.
❖ only one edge, and is the only
independent path. Hence, for this flow
graph, cyclomatic complexity is equal
to one.
✓ no predicate nodes because there are no decision points.
Hence, the cyclomatic complexity = 0 (the number of predicate
nodes) + 1.
✓ the edges (E) = 1; nodes (N) = 2. The cyclomatic complexity = E - N + 2
=> l + 2 - 2 =>1
Structural Testing
❖ two independent paths, and thus, the
cyclomatic complexity of the graph
equals 2 [ the number of predicate
nodes (1) + 1].
❖ E = 4, N= 4, So, E – N + 2 => 4-4+2 = 2.
❑ An independent path can be defined
as a path in the flow graph that has at
least one edge that has not been
traversed before in other paths.
Structural Testing
❑ A set of independent paths that cover all the edges is a basis set.
❑ Once the basis set is formed, test cases should be written to execute
all the paths in the basis set.
Complexity What it means?
Code Complexity Testability Cost/effort to maintain
1 – 10 Well-written High Low
10 – 20 Moderate Medium Medium
20 – 40 Very complex Low High
>40 Not testable Cost/effort may not be

enough
Structural Testing
Calculating and using cyclomatic complexity
❖ For small programs cyclomatic complexity can be calculated
manually.
❖ Tools are used to calculate cyclomatic complexity for thousands
of lines of code.
Challenges in White-box Testing

❖ Human tendency of a developer being unable to find the
defects in his code
❖ Fully tested code may not correspond to realistic scenarios
Black Box Testing
What is black box testing?
❑ Black box testing involves looking at the specifications and does not
require examining the code of a program.
❑ Done from the customers view point.
❑ The test engineer only knows the set of inputs and expected outputs and
is unaware of how those inputs are transformed into outputs by the
software.
❑ Requires functional knowledge of the product to be tested..
Black Box Testing
Why Black box testing?
❑ Helps in verifying overall functionality of the system under test.
❑ Done based on requirements – identifies any incomplete, conflicting
requirements.
❑ Addresses the stated requirements as well as implied requirements
(inclusion of dates, page header, and footer may not be explicitly stated).
❑ It covers the end user perspectives.
❑ It handles valid and invalid inputs.
Black Box Testing
When to do black box testing?
❑ Black box testing activities require involvement of the testing team from
the beginning of the software project life cycle.
❑ Testers can get involved right from the requirements gathering.
❑ Test scenarios and test data are prepared during the test construction
phase of the test life cycle, when the software is in design phase.
❑ Once the code is ready for testing, the code is exercised with all the
constructed test scenarios.
Black Box Testing
How to do black box testing?
❑A set of test cases is needed to test the external functionality, uncovering
as many defect as possible, in short time as possible.
❑ The various techniques are:
1. Requirement based testing 2. Positive and negative testing
3. Boundary value analysis 4. Decision tables
5. Equivalence partitioning 6. State based testing
7. Compatibility testing 8. User documentation testing
9. Domain testing
Black Box Testing
Requirement based Testing
❑ Requirement testing deals with validating the requirements (explicit &
implicit) given in the Software Requirements Specification (SRS)
❑ Explicit - those stated in the SRS
❑ Implicit – those that are not documented but are assumed to be
incorporated in the system
❑ Precondition for requirement testing -> detailed review of the SRS
❑ Test Requirement Specification (TRS) -> document of implicit and explicit
requirements.
❑ Requirements are tracked by Requirement Traceability Matrix (RTM). The
matrix evolves through the life cycle of the product.
Black Box Testing
Black Box Testing
Black Box Testing
❑ Each requirement is given a unique ID along with a brief description.
❑ Each requirement is assigned a requirement priority (H/L/M).
❑ Tests for higher priority requirements will get precedence over tests for
lower priority requirements.
❑ Test conditions – lists the different ways of testing the requirement. These
test conditions are grouped together to form a single test case,
alternatively, each test condition can be mapped to one test case.
❑ Test case ID – maps the test cases with the requirements.
❑ A requirement is subjected to multiple phases of testing – unit,
component, integration, and system testing.
❑ Phase of testing – indicates when a requirement will be tested and at
what phase of testing.
Black Box Testing
❑ RTM helps in identifying the relationship between the requirements and
the test cases.
❑ One to one
❑ One to many
❑ Many to one
❑ Many to many
❑ One to none
Black Box Testing
❑ Role of RTM in Requirement based testing:
❑ RTM track the testing status of each requirement.
❑ RTM enables the testers to prioritize the test cases based on the
prioritized requirements.
❑ Test conditions can be grouped to create test cases or can be
represented as unique test cases.
❑ Test conditions/cases can be used as inputs
Black Box Testing
❑ Metrics collected from RTM:
➢ Requirements addressed priority wise
➢ Number of test cases requirement wise
➢ Total number of test cases prepared.
❑ After test execution, the test results can be used to collect metrics such
as:
➢ total number of test cases passed.
➢ total number of test cases failed
➢ total number of defects in the requirement
➢ number of requirements completed
➢ number of requirements pending
Black Box Testing
Black Box Testing
Black Box Testing
Equivalence Partitioning
❑ In this method the input domain data is divided into different equivalence
data classes.
❑ This method is typically used to reduce the total number of test cases to a
finite set of testable test cases, still covering maximum requirements.
❑ In short it is the process of taking all possible test cases and placing them
into classes.
❑ One test value is picked from each class while testing.
❑E.g.: If you are testing for an input box accepting numbers from 1 to 1000
then there is no use in writing thousand test cases for all 1000 valid input
numbers plus other test cases for invalid data.
Black Box Testing
❑ Using equivalence partitioning method above test cases can be divided
into three sets of input data called as classes.
❑ Each test case is a representative of respective class.
❑ So in above example we can divide our test cases into three equivalence
classes of some valid and invalid inputs.
❑ Test cases for input box accepting numbers between 1 and 1000 using
Equivalence Partitioning:
1) One input data class with all valid inputs. Pick a single value from
range 1 to 1000 as a valid test case. If you select other values between 1
and 1000 then result is going to be same. So one test case for valid input
data should be sufficient.
Black Box Testing
2) Input data class with all values below lower limit. i.e. any value
below 1, as a invalid input data test case.
3) Input data with any value greater than 1000 to represent third
invalid input class.
❑ So using equivalence partitioning you have categorized all possible test
cases into three classes.
❑ We have selected one representative from every input class to design our
test cases.
❑ Equivalence partitioning uses fewest test cases to cover maximum
requirements.
Black Box Testing
Boundary Value Partitioning
❑ More application errors occur at the boundaries of input domain.
❑ ‘Boundary value analysis’ testing technique is used to identify errors at
boundaries rather than finding those exist in center of input domain.
❑ Boundary value analysis is a next part of Equivalence partitioning for
designing test cases where test cases are selected at the edges of the
equivalence classes.
❑ Test cases for input box accepting numbers between 1 and 1000 using
Boundary value analysis:
1) Test cases with test data exactly as the input boundaries of input
domain i.e. values 1 and 1000 in our case.
Black Box Testing
Boundary Value Partitioning
2) Test data with values just below the extreme edges of input
domains i.e. values 0 and 999.
3) Test data with values just above the extreme edges of input domain
i.e. values 2 and 1001.
❑ Boundary value analysis is often called as a part of stress and negative
testing.
Black Box Testing
Positive and Negative Testing
Positive Testing Negative Testing
Performed on the system by Performed on the system by

providing the valid data as input. providing invalid data as input.
It checks whether an application It checks whether an application

behaves as expected with the behaves as expected (does not fail)
positive input. with the negative (unexpected)
input.
This is to test to check the This is to test the application that

application that does what it is does not do anything that it is not
supposed to do so. supposed to do so.
Black Box Testing
Example
There is a text box in an application which can accept only
numbers of range 0 to 99999. Any other values apart from this
should not be acceptable.
To do positive testing, set the valid input values from 0 to 99999

and check whether the system is accepting the values.
Black Box Testing
Example
There is a text box in an application which can accept only
numbers of range 0 to 99999. Any other values apart from this
should not be acceptable.
Negative testing can be performed by testing by entering

alphabets characters from A to Z or from a to z.
Either system text box should not accept the values or else it
should throw an error message for these invalid data inputs.
Black Box Testing
Decision Table
❑ A decision table is a good way to deal with different combination
inputs with their associated outputs and also called cause-effect
table.
❑ Decision table testing is black box test design technique to
determine the test scenarios for complex business logic.
❑ A decision tables lists the various decision variables, the conditions
(or values) assumed by each of the decision variables, and the
outcome/action to take in each combination of conditions.
❑ Don’t cares – the values of the appropriate decision variables which
do not affect the outcome of the decision; usually denoted by ‘-’.
Black Box Testing
Decision Table
Considering the behavior of Flight Button for different combinations of Fly
From & Fly To.
Rule 1: When destination for both Fly From & Fly To are not set, the Flight
Icon is disabled.
In the decision table,

register False for Fly From
& Fly To and the outcome
Would be False, which is
Flights Button will be
disabled.
Black Box Testing
Decision Table
Rule 2: When Fly From destination is set but not Fly to, Flight button
is disabled.
Rule 3: When Fly from destination is not set but Fly to destination is
set, Flight button is disabled.
Rule 4: Only when Fly to and
Fly from destinations are set,
Flights button is enabled.
Make the corresponding entries

in the decision table.
Black Box Testing
Decision Table
Steps in forming a decision table
1. Identify the decision variables
2. Identify the possible values of each of the decision variables
3. Enumerate the conditions of the allowed values of each of
the variables.
4. Identify don’t care cases
5. For each combination of values of decision variables, list out
the action or expected result.
Black Box Testing
Decision Table
Advantage of decision table technique
✓Any complex business flow can be easily converted into the test
scenarios & test cases using this technique.
✓Simple to understand
✓Provides complete coverage of test cases which help to reduce
the rework on writing test scenarios & test cases.
Black Box Testing
State based or Graph based Testing
❑ State based testing is useful in situations where
✓ The product under test is a language processor
✓ Workflow modeling where, depending upon the current state
and appropriate combinations of input variables, specific
workflows are carried out, resulting in new output and new
state
✓ Dataflow modeling, where the system is modeled as a set of
dataflow, leading from one state to another
Black Box Testing
Consider an application that is required to validate a number
according to the following simple rules.
1. A number can start with an optional sign.
2. The optional sign can be followed by any number of digits.
3. The digits can be optionally followed by a decimal point,
represented by a period.
4. If there is a decimal point, then there should be two digits
after the decimal.
5. Any number—whether or not it has a decimal point, should
be terminated by a blank.
Black Box Testing
The state transition diagram can be

converted into a state transition
table
Current state Input Next State
1 Digit 2
1 + 2
1 - 2
2 Digit 2
2 Blank 6
2 Decimal point 3
3 Digit 4
4 Digit 5
5 Blank 6
Black Box Testing
❑ The above state transition table can be used to derive test cases to
test valid and invalid numbers.
❑ Valid test cases can be generated by:
1. Start from the ‘Start’ State (State #1 in the example).
2. Choose a path that leads to the next state (for example.
+/-/digit to go from State I to State 2)
3. if you encounter an invalid input in a given state (for example,
encountering an alphabetic character in State 2), generate an error
condition test case.
4. Repeat the process till you reach the final state (State 6 in this
example).
Black Box Testing
Compatibility Testing
❑ Compatibility Testing checks whether the software is capable of
running on different hardware, operating systems, applications,
network environments or Mobile devices.
❑ Compatibility Testing is a type of the Non-functional testing
❑ The parameters that generally affect the compatibility are:
❑Processor
❑ Machine Architecture
❑ Resource availability
❑ Operating System
❑ Mid-tier infrastructure components
❑ Back end components, etc
Black Box Testing
❑ A compatibility Matrix is created with compatibility parameters as
columns. Each row represents a unique combination of a specific set
of values of the parameters.
Server Application Server Web Server Client Browser MS Office Mail Server
Black Box Testing
❑ Some common techniques used for performing compatibility
testing, using the compatibility table are:
✓ Horizontal Combination
❖ Each feature of the product has to be tested with each row
in the compatibility matrix.
❖ Involves huge effort and time.
✓ Intelligent Sampling
❖ The selection of intelligent samples is based on information
collected on the set of dependencies of the product with
the parameters.
Black Box Testing
❑ Compatibility testing can be further classified into two types:
1. Backward Compatibility testing - to verify the behavior of
the developed hardware/software with the older versions of
the hardware/software.
2. Forward Compatibility testing - is to verify the behavior of
the developed hardware/software with the newer versions
of the hardware/software.
Black Box Testing
User Documentation Testing
❑ User documentation covers all the manuals, user guides, installation
guides, setup guides, read me files, software release notes, and
online help for the software.
❑ The objective of documentation testing:
❑ To check if what is stated in the document is available in the
product.
❑ To check if what is there in the product is explained correctly in
the document.
❑ When a product is upgraded, the corresponding product
documentation should also get updated as necessary to reflect any
changes that may affect the user.
Black Box Testing
User Documentation Testing
❑ Benefits of Documentation testing:
✓ Minimizes the possible defects reported by customers.
✓ Reduces the time taken for each support call
✓ Reduces in less difficult support calls.
✓ New programmers or testers can use the documentation to
learn the external functionality of the product.
Integration Testing
Bottum-Up Integration
❑ In this approach testing is conducted from sub module to main
module.
❑ If the main module is not developed, a temporary program called
DRIVERS is used to simulate the main module.
❑ A DRIVER is a piece of software that creates necessary ‘Inputs’
required for the Unit and then invokes the Unit.
❑ Advantageous if major flaws occur toward the bottom of the
program.
❑ Producing Driver Modules is a remarkable challenge in this model.
Integration Testing
Integration Testing
Bi-Directional Integrtation
❑ A combination of top-down and bottom-up
integration approaches.
❑ Also known as Sandwich integration
❑ In the figure, assume the components become
available in the order mentioned by the
component numbers.
❑ The individual components are tested
separately.
❑ Bi-directional integration is done initially with
the use of stubs and drivers.
Integration Testing
Bi-Directional Integrtation
❑ Drivers -> provides upstream connectivity, and redirects the requests
to some other component.
❑ Stubs -> provides downstream connectivity, and stimulate the
behavior of the missing components.
❑ Stubs and drivers are discarded once the missing components 6,7,8
are made available.
Integration Testing
System Integrtation
❑ All the components of the system are integrated and tested as a
single unit.
❑ The approach waits till all the components are available and one
round integration testing is done, and the approach is called as
big-bang integration.
❑ Big-bang approach reduces testing effort and removes duplication in
testing.
❑ Drawbacks:
❑ Difficult to locate the defects.
❑ Difficult to pinpoint the ownership for correcting the defect.
❑ As the release date may approach closer, the quality might be
compromised.
Integration Testing
Choosing Integration Method
Integration Testing
Integration as a Phase of Testing
❑ Starts-> when 2 components are ready to test; Ends-> when all
components are ready to work together as a complete system.
❑ Focus on finding defects raised because of combining various
components for testing.
❑ Focus on testing interfaces as well as usage flow.
❑ Ensures completeness and coverage testing for functionality.
❑ it can be achieved through planned test execution and also on
unplanned testing known as ‘ad hoc testing’.
(exploratory/monkey/out of box testing).
❑ ad hoc testing -> to uncover the defects not covered through
planned test cases.
Integration Testing
Integration as a Phase of Testing
❑ When the functionality of the different components are combined
and tested together for a sequence of related operations, they are
called scenarios.
❑ Scenario testing is a planned activity to explore different usage
patterns and combine them into test cases called scenario test cases.
Scenario Testing
Scenario Testing
❑ Set of realistic user activities that are used for evaluating the
product.
❑ Methods of scenario testing
➢ System Scenarios
➢ Use-case scenarios/role-based scenarios
System Scenario
❑ The approaches to develop System Scenario:
➢ Story line
➢ Life cycle / State Transition
➢ Business Verticals
➢ Battle Ground
Scenario Testing
System Scenario
➢ Story line
USERS:
enters office – logs in – check mail – responds to mails –
compiles program – so on….
➢ Life cycle / State Transition
Saving Account:
new account with initial amount – deposit – with drawl –
calculate interest – so on..
the account is applied to many changes and the different
transformations applied to many becomes different scenarios.
Scenario Testing
System Scenario
➢ Business Verticals
Purchase software
pharmaceuticals
software house
government organizations
➢ Battle Ground
➢ scenarios to justify that the product works
➢ some scenarios to justify the product does not works
Scenario Testing
Use Case Scenarios
➢ is a stepwise procedure on how a user determined to use a system,
with different user roles and associated parameters.
➢ A Use Case can involve several roles or class of users who typically
perform different activities based on the role.
➢ Actors – users with different roles
➢ System behavior – what the product should do for a particular
activity
➢ Agents – Users with a specific role to interact between the actors
and the system. In a completely automated system involving the
customer and the system, use cases can be written without
considering the agent portion.
Integration Testing
Use Case Scenarios
Integration Testing
Use Case Scenarios
Defect Bash
➢ Defect bash is an adhoc testing where people performing different
roles in an organization test the product together at the same time.
➢ This adhoc testing is not based on written test cases. What is to be
tested is left to an individual’s decision and creativity. They also try
with some operations which are beyond the product specifications.
➢ Defect bash bring together plenty of good practices such as:
➢ Enabling people to cross boundaries and test beyond assigned
areas
➢Bringing different people performing different roles and with
varying level of product understanding together in the
organization for testing
Defect Bash
➢ Letting everyone in the organization use the product before
delivery
➢ Bringing fresh pairs of eyes to uncover new defects
➢ Enabling people to say "system works" as well as enabling them
to "break the system“
❑ All the activities in defect bash is planned, except for what to be
tested.
❑ The steps involving in defect bash:
➢ Step 1 Choosing the frequency and duration of defect bash
▪ Optimizing the frequency and duration is a big saving as a
large number of people are involved
Defect Bash
➢ Step 2 Selecting the right product build
▪Where a large number of people are involved, a good
quality product build gives confidence on the product and
progress.
➢ Step 3 Communicating the objective of each defect bash to
everyone
▪Defects that a test engineer would find easily should not be
the objective of a defect bash.
▪ Once they are told in advance, the members of defect bash
team will be in a better position to contribute towards
stated objectives.
Defect Bash
➢ Step 4 Setting up and monitoring the lab for defect bash
▪Finding out the right configuration, resources (hardware,
software, and set of people to perform defect bash) have to
be planned carefully before a bash actually starts.
▪ Defect bash brings out both functional and non-functional
defects.
▪ functional defects - The defects that are in the product
▪ non-functional defects - defects that are uncovered while
monitoring the system resources, such as memory leak, long
turnaround time, missed requests, high impact and
utilization of system resources, and so on.
Defect Bash
➢ Step 5 Taking actions and fixing issues
▪All the issues reported from a defect bash need to be taken
through a complete code and design inspections, analyzed,
and fixed together in places where a defect could evolve
from.
➢ Step 6 Optimizing the effort involved in defect bash
▪One way to optimize the effort and to meet the purpose is
by having a tested build, keeping the right setup, sharing the
objectives, and so on.
Defect Bash
▪ Another approach is to conduct "micro level' defect bashes
before conducting one on a large scale.
▪Since a defect bash is an integration testing phase activity, it
can be experimented by integration test team before they
open it up for others.
System Testing Overview
❑ System testing is a testing phase conducted on the complete
integrated system, to evaluate the system completeness with its
specified requirements.
❑ It brings out issues that are fundamental to design, architecture, and
code of the whole product.
❑ It is the only phase of testing which tests the both functional and
non-functional aspects of the product.
❑ On the functional side, it focuses on real-life customer usage of the
product.
❑ On the non-functional side, it brings in different testing types such
as:
➢Performance/Load testing - To evaluate the response time of
the system to perform its required functions in comparison with
different versions of same product(s) or a different competitive
product
➢ Scalability testing - ensures that an application can handle the
projected increase in user traffic, data volume, transaction
counts frequency, etc.
➢ Reliability testing - To evaluate the ability of the system to
perform its required functions repeatedly for a specified period
of time is called reliability testing.
➢ Stress testing Evaluating a system beyond the limits of the
specified requirements or system resources to ensure the
system does not break down unexpectedly
➢ Interoperability testing - This testing is done to ensure that two
or more products can exchange information, use the
information, and work closely.
➢ Localization testing - testing is the software testing process for
checking the localized version of a product for that particular
culture or locale settings.
Why System Testing is done?
System testing is done for the following reasons.
1. Provide independent perspective in testing
2. Bring in customer perspective in testing
3. Provide a "fresh pair of eyes" to discover defects not found
earlier by testing
4. Test both functional and nonfunctional aspects of the product
5. Build confidence in the product
6. Analyze and reduce the risk of releasing the product
7. Ensure the product for acceptance testing.
Functional vs Non-functional Testing
Testing aspects Functional Testing Non-functional Testing
Product features and
Involves Quality factors
functionality
Tests Product behavior Behavior and experience
Simple steps written to check Huge data collected and
Result Conclusion
expected results analyzed
Product implementation,
Results varies due to Product implementation
resources, and configurations
Testing focus Defect detection Qualification of product
Product, domain, design,
Knowledge required Product and domain
architecture, statistical skills.
Failures normally due to Code Architecture, design, and code
Unit, component, integration,
Testing Phase System
system
Repeated only in case of
Test case repeatability Repeated many times failures and for different
configurations
One-time setup for a set of Configuration changes for
Configuration
test cases each test case
Functional System Testing
Some common techniques of Functional System Testing are:
➢ Design/Architecture verification
➢ Business Vertical Testing
➢ Deployment Testing
➢ Beta Testing
➢ Certification, Standards, and testing for compliance.
Design/Architecture verification
➢ Test cases are developed and checked against the design and
architecture to see whether they are product-level test cases.
➢ A functional system test focus on the behavior of the complete
product.
➢ This technique validates the product features that are written
based on customer scenarios and verifying them using product
implementation.
➢ If there is a test case that is a customer scenario but failed
validation using this technique, then it is moved appropriately to
component or integration testing phases.
Design/Architecture verification
Guidelines to reject test cases for system functional testing
➢ if focused on code logic, data structures, and unit of the
product, then it belongs to unit testing.
➢ If specified in the functional specification of any component,
then it belongs to component testing.
➢ If specified in design and architecture specification for
integration testing, then it belongs to integration testing.
➢ If focused on product implementation but not visible to
customers, then it is to be v=covered in
unit/component/integration testing.
Deployment Testing
➢ offsite deployment - the deployment (simulated) testing that
happens in a product development company to ensure that
customer deployment requirements are met
➢ onsite deployment – conducted after the release of the product
by utilizing the resources and setup available in customer’s
location.
➢ The system testing team involves in completing the onsite
deployment test.
➢ onsite deployment is a part of acceptance testing.
Deployment Testing
➢Onsite deployment is done at two stages.
✓ Stage 1: actual data from the live system and configurations
are mirrored, the operations from the users are rerun on
the mirrored deployment machine.
✓ Stage 2: the mirrored system is made a live system that
runs the new product.
➢ An intelligent recorders are used to record the transactions that
happen on a live system and commit these operations on a
mirrored system and then compare the results against the live.
Beta Testing
➢ Reasons for a product not meeting the customer requirements
✓ Implicit and explicit requirements
✓ Requirements may become obsolete or changed by the
time the product is delivered
✓ Unresolved ambiguous areas of the requirements.
✓ Wrong implementation of the requirements
✓ Lack of documentation
➢ The mechanism of sending the product that is under test to the
customers and receiving the feedback is called beta testing.
Beta Testing
➢ The activities involved in beta program are:
1. Collecting the list of customers
2. Working out a beta program schedule and informing
the customers.
3. Sending the document for reading in advance and
training the customer on product usage.
4. Testing the product to ensure it meets “beta testing
entry criteria”
5. sending the beta product to the customer and enable
them to carry out testing
Beta Testing
➢ The activities involved in beta program are:
6. Collecting the feedback periodically
7. Responding the customers feedback with product fixes
8. Analyzing and concluding whether the beta program
met the exit criteria
9. Formally closing the beta program
10. Incorporating the appropriate changes in the product.
Certification. Standards and Testing tor connonance
➢ Testing a product for certified hardwares, operating system,
database, and other infrastructure pieces is called certification
testing.
➢ The certification agencies produce automated test suites, and
the product development organization runs those test suites and
corrects the problems in the product to ensure that tests are
successful.
➢ Once the tests are successfully run, the results are sent to the
certification agencies and they give the certification for the
product.
➢ The test suite may be rerun by the certification agencies to
verify the results.
➢ There are many standards (for example, IPv6 in networking) for
each technology area and the product may need to conform to
those standards.
➢ The product development companies select the standards to be
implemented at the beginning of the product cycle.
➢ Testing the product to ensure that these standards are properly
implemented is called testing for standards.
➢ Testing the product for contractual, legal, and statutory
compliance is one of the critical activities of the system testing
team.
➢ The following are some examples of compliance testing.
✓ 508 accessibility guidelines - This accessibility set of guidelines
requires the product to meet some requirements for its
physically challenged users.
✓ OFAC and Patriot Act - This act requires the transactions of the
banking applications be audited for misuse of funds for
terrorism.
Non Functional System Testing
Setting Up the Configuration
➢ There are two ways the setup is done:
✓ simulated environment
✓ real-life customer environment
➢ Simulated setup is used for non-functional testing where actual
configuration is difficult to get.
➢ Setting up a configuration is a challenge for the following reasons.
1. It is very difficult to predict the type of environment that will
be used commonly by the customers.
2. Testing a product with different permutations and
combinations of configurations may not prove effective since
the same combination of environment may not used by the
customer
Setting Up the Configuration
➢Setting up a configuration is a challenge for the following reasons.
3. The cost involved in setting up such environments is quite
high.
4. Some of the components of the environment could be from
competing companies products and it may not be easy to get
these.
5. The people may not have the skills to set up the
environment.
6. Confidentiality is involved in the data used by the customer,
such information is not passed on to the testing team.
Coming Up with Entry / Exit Criteria
➢ Meeting the entry criteria is the responsibility of the previous
testing phase.
Balancing Key Resources
➢ The key resources CPU, disk, memory, and network requires equal
attention to enhance the quality factors of the product.
➢ When customers are asked to upgrade the resources, the aspect of
return on investment needs to be justified.
➢ The customers have to be known about the resources the product
intensively uses and the resources that are critical for product usage.
➢It is very difficult for the customer to increase all the resources in the
server, as all the products are expected to run in the same server, at
the same time.
➢Certain basic assumptions have to be made by the testing team and
by the customers before starting the testing.
1) The CPU can be fully utilized as long as it can be freed when
a high priority job comes in.
2) The available memory can be completely used by the
product as long as the memory is relinquished when
another job requires memory.
3) 3. The resources can be added easily to get better
performance as long as we can quantify and justify the
benefits for each added resource.
4) The product can generate many network packets as long as
the network bandwidth and latency is available and does
not cost much.
5) More disk space or the complete I/O bandwidth can be used
for the product as long as they are available.
6) The customer gets the maximum return on investment only
if the resources are optimally used.
7) Graceful degradation in non-functional aspects can be
expected when resources in the machine are also utilized
for different activities in the server.
8) Predictable variations in performance or scalability are
acceptable for different configurations of the same product.
Scalability Testing
➢ Scalability testing finds the maximum capability of the product
parameters.
➢ Ex: Finding out how many client machines can simultaneously log
in to the server.
➢ When the scalability requirements from the customer are more than
what design/architecture can provide, the scalability testing is
suspended, the design is reworked.
➢ Scalability testing does not end when the requirements are met. The
testing continues till the maximum capability of a scalable
parameter is found out for a particular configuration.
Scalability Testing
➢ Salability tests help in identifying the major bottlenecks in a
product. When resources are found to be the bottlenecks, they are
increased after validating the assumptions.
➢ If the bottlenecks are in the product, they are fixed.
➢ Scalability tests are performed on different configurations to check
the product's behavior. For each configuration, data are collected and
analyzed.
➢ When there are no product bottleneck, resources need to be
upgraded to complete scalability testing.
➢ Scalability testing requires significant resources and is expensive.
Reliability Testing
➢ Reliability testing is done to evaluate the product's ability to
perform its required functions under stated conditions for a
specified period of time or for a large number of iterations.
➢ Ex: Querying a database continuously for 48 hours and performing
login operations 10,000 times.
➢ Product reliability is achieved by focusing on the following activities.
➢ Defined engineering processes
➢ Review of work products at each stage
➢ Change management procedures
➢ Review of Testing Coverage
➢ Ongoing monitoring of the product
Reliability Testing
➢ The objective behind performing reliability testing are,
✓ To find the structure of repeating failures and its main cause
✓ To find the number of failures occurring is the specified amount
of time.
➢ Software reliability testing includes Feature Testing, Load Testing and
Regression Testing
✓ Feature Testing - check the feature provided by the software
✓ Load Testing - to check the performance of the software under
maximum work load.
✓ Regression Test - to check whether any new bugs have been
introduced because of the fixing of previous bugs/after every
change or updation of the software features.

Software Testing 5 Units

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Software Testing 5 Units

Uploaded by

Copyright:

Available Formats

Phases of Software Project

❑ Requirement gathering and Analysis

✓ Entry criteria - which specify when that phase can be

❑ Advantages of ETVX model

❑ A project starts with an initial phase,

✓ The project is divided into

✓ Each phase communicates to the

✓ When an error is detected, it is

resolved at some earlier phase.

❑ Before a phase starts, the completion

criteria for the previous phase can be

❑ Phases are processed and completed

requirements are subject to high risk

Static Testing Methods

❑ Program constructs in most conventional programming languages

Complexity What it means?

Code Complexity Testability Cost/effort to maintain

1 – 10 Well-written High Low

10 – 20 Moderate Medium Medium

20 – 40 Very complex Low High

>40 Not testable Cost/effort may not be

Challenges in White-box Testing

Positive Testing Negative Testing

Performed on the system by Performed on the system by

It checks whether an application It checks whether an application

This is to test to check the This is to test the application that

To do positive testing, set the valid input values from 0 to 99999

Negative testing can be performed by testing by entering

In the decision table,

Make the corresponding entries

The state transition diagram can be

You might also like