Department of Computer Science and Engineering: Subject Name: Subject Code: CS E71

Sri ManakulaVinayagar Engineering College
Department of Computer Science and Engineering
Subject Name: SOFTWARE QUALITY ASSURANCE AND TESTING Subject Code: CS E71
Prepared by:
Mr. B. THIYAGARAJAN, AP/CSE
Ms. R. NIJANTHY, AP/CSE
Ms. R. GAYATHRI, AP/CSE
Verified by: Approved by:
Unit –II
Testing Strategies: White box testing techniques - Statement coverage – Branch
Coverage - Condition coverage - Decision/Condition coverage - Multiple condition coverage
- Dataflow coverage - Mutation testing - Automated code coverage analysis - Black box
testing techniques - Boundary value analysis – Robustness testing - Equivalence
partitioning - Syntax testing - Finite state testing
Software quality assurance and testing Page 1

1.What is meant by testing strategy?
The choice of software testing strategy is highly dependent on the nature of the
developed software. For example, if the software is highly data intensive then a strategy
that checks structures and values properly to ensure that all inputs given to the software
are correct and complete should be developed. The design and architecture of the software
are also useful in choosing testing strategy. A number of software testing strategies are
developed in the testing process. All these strategies provide the tester a template, which is
used for testing.
2. Mention the characteristics of testing strategies.
All testing strategies have following characteristics.
Testing proceeds in an outward manner. It starts from testing the individual

units, progresses to integrating these units, and finally, moves to system
testing.
Testing techniques used during different phases of software development are
different.
Testing is conducted by the software developer and by an ITG.
Testing and debugging should not be used synonymously. However, any
testing strategy must accommodate debugging with itself.
3. What are the types of software testing strategies?
There are different types of software testing strategies, which are selected by the
testers depending upon the nature and size of the software. The commonly used software
testing strategies are
Analytic testing strategy: It takes a complete overview of requirements, design, and

implementation of objects to determine the motive of testing. In addition, it gathers
complete information about the software, targets to be achieved, and the data required for
testing the software.
Model-based testing strategy: This strategy tests the functionality of the software
according to the real world scenario (like software functioning in an organization). It
recognizes the domain of data and selects suitable test cases according to the probability of
errors in that domain.

Methodical testing strategy: It tests the functions and status of software according to the
checklist, which is based on user requirements. This strategy is also used to test the
functionality, reliability, usability, and performance of the software.
Process-oriented testing strategy: It tests the software according to already existing
standards such as the IEEE standards. In addition, it checks the functionality of the
software by using automated testing tools.
Dynamic testing strategy: This tests the software after having a collective decision of the
testing team. Along with testing, this strategy provides information about the software such
as test cases used for testing the errors present in it.
Philosophical testing strategy: It tests the software assuming that any component of the
software can stop functioning anytime. It takes help from software developers, users and
systems analysts to test the software.
4. What is white box testing?
White Box Testing is also known as Code-Based Testing or Structural Testing. White
box testing is the software testing method in which internal structure is being known to
tester who is going to test the software. In this method of testing the test cases are
calculated based on analysis internal structure of the system based on Code coverage,
branches coverage, paths coverage, condition Coverage etc. It is one of the best method to
find out the errors in the software application in early stage of software development life
cycle
5. What are white box testing techniques?
The white box techniques are
Statement Coverage
Condition Coverage
Decision/Condition Coverage
Multiple Condition Coverage
Branch coverage
Dataflow coverage
Mutation testing
6.What are the advantages and disadvantages of white box testing?
Advantages
Accuracy of implemented code can be determined.

Uncovers programmer errors at initial level of testing.

Knowledge of internal structure or working of application helpful in

designing test cases.
As the tester is aware of internal coding structure, then it is helpful to derive
which type of input data is needed to testing software application effectively.
White box testing allows you to help in the code optimization.
Disadvantages
To test the software application a highly skilled resource is required to carry

out testing who know the deep knowledge of internal structure of the code
which increase the cost.
Update test script is required if changing the implementation too frequently.
If the application under test large is size then exhaustive testing is
impossible.
White box testing very expensive type of testing.
To test each paths or conditions may require different input conditions, so to
test full application tester need to create fill range of inputs which may be a
time consuming.
7.What is statement coverage?[APRIL 2012]
Statement coverage is one of the code coverage analysis techniques.This technique

requires every possible statement in the code to be tested at least once during the testing
process. Statement coverage does not report whether loops reach their termination
condition - only whether the loop body was executed. With C, C++, and Java, this limitation
affects loops that contain break statements. Cantata++ can be used as white box testing tool
to test the Statement Coverage .
8.what is branch coverage or decision coverage?
Branch coverage is closely related to decision coverage . Decision coverage

measures the coverage of conditional branches; branch coverage measures the coverage of
both conditional and unconditional branches.This technique simply checks every possible
path (if-else and other conditional loops) of a software application. . Examples of branch or
decision statements are switch, do-while, and if-else statements. Multi way GOTO
statements qualify in some programming languages such as FORTRAN. A tool that handles
branch coverage testing for C, C++ and Java applications is TCAT-PATH.
9.What is multiple condition coverage?
Multiple-condition coverage is a more detailed kind of coverage than condition

coverage, because it checks if all possible combinations of conditions of sub expressions
within an expression have been tested. There are several places within a Java program

where the multiple-condition coverage can be measured, such as for if-, for-, do-, while and
switch-statements. Also, a regular Boolean expression can contain multiple sub-
expressions
10. What is mutation testing?
In Mutation Testing, certain statements are changed in the source code and check if
the test cases are able to find the errors. It is a type of white box testing which is mainly
used for unit testing. The changes in mutant program are kept extremely small, so it does
not affect the overall objective of the program. The goal of Mutation Testing is to assess the
quality of the test cases which should be robust enough to fail mutant code. This method is
also called as Fault based testing strategy as it involves creating fault in the program.
11.How to create mutant programs?
A mutation is nothing but a single syntactic change that is made to the program
statement. Each mutant program should differ from the original program by one mutation.
This method uncovers ambiguities in the source code, and has the capacity to detect all the
faults in the program.
Original Program Mutant Program
If (x>y) If(x<Y< strong="">)</Y<>
Print "Hello" Print "Hello"
Else Else
Print "Hi" Print "Hi"
12. What is Dataflow coverage?
Data flow coverage involves tracking a piece of data completely through the
software. At the unit test level this would just be through an individual module or function.
The same tracking could be done through several integrated modules or even through the
entire software product although it would be more time-consuming to do so.

13. What is code coverage analysis?
Code coverage is an important type of test effectiveness measurement. Code

coverage is a way of determining which code statements or paths have been exercised
during testing. With respect to testing, coverage analysis helps in identifying areas of code
not exercised by a set of test cases. Alternatively, coverage analysis can also help in
identifying redundant test cases that do not increase coverage
14. What is Black box technique?
Black Box Testing is a software testing method in which the internal structure or
design or implementation of the item being tested is not known to the tester. These tests
can be functional or non-functional, though usually functional. In this approach, test data
are derived solely from the specifications (i.e., without taking advantage of knowledge of
the internal structure of the program).
15 .Mention the techniques of black box testing?
The Black box testing techniques are
Boundary-Value Analysis
Robustness Testing
Equivalence partitioning
Syntax testing
Finite state testing
16. What is boundary value analysis? [NOVEMBER 2013]
Boundary Value Analysis is a test Functional Testing technique where the extreme
boundary values are chosen. Boundary values include maximum, minimum, just
inside/outside boundaries, typical values, and error values. Many systems have tendency to
fail on boundary. So testing boundary values of application is important.
17. Define Robustness testing.
Robustness testing is a testing methodology to detect the vulnerabilities of a

component under unexpected inputs or in a stressful environment. As components may fail
differently in different states, we use a state machine based approach to robustness
testing.The most interesting part of robustness testing is not with the inputs, but with the
expected outputs.

18. Write about Equivalence portioning.
It is a software test design technique that involves dividing input values into valid
and invalid partitions and selecting representative values from each partition as test data.
Equivalence partitioning relates to the commonality and variances among the different
situations in which a system is expected to work. If situations are equivalent, or essentially
similar, to one another, it is adequate to test only one of them, not all.
19. What is Syntax testing?
Syntax Testing, a black box testing technique, involves testing the System inputs and
it is usually automated because syntax testing produces a large number of tests.
20. What are the steps involved in syntax testing?
Syntax testing consists of the following steps:
1. Identify the target language or format (hidden or explicit).
2. Define the syntax (format) of the language, formally, in a convenient notation such as
Backus-Naur form (BNF).
3. Test and debug the syntax to assure that it is complete and consistent and that it
satisfies the intended semantics.
4. Normal condition testing consists of a covering set of input strings including critical loop
values. The difficult part about normal case testing is predicting the outcome and verifying
that the processing was correct.
5. Syntax testing methods pay off best for dirty testing. Test design is a top-down process
that consists of methodically identifying which component is to be cruddied-up and how.
6. Much of syntax test design can and should be automated by relatively simple means.
7. Test execution automation is essential for syntax testing because this method produces
so many tests.
21. What is finite state testing or state transition testing? [APRIL 2012]
State transition testing is used where some aspect of the system can be described in
what is called a ‘finite state machine’. This simply means that the system can be in a (finite)
number of different states, and the transitions from one state to another are determined by
the rules of the machine. the finite-state machine model is an excellent model for testing
menu-driven applications. It is also important because of its widespread use in object-
oriented design.

22.What are the advantages and disadvantages of Black box testing ?
Advantages
Tests are done from a user’s point of view and will help in exposing discrepancies in
the specifications.
Tester need not know programming languages or how the software has been
implemented.
Tests can be conducted by a body independent from the developers, allowing for an
objective perspective and the avoidance of developer-bias.
Test cases can be designed as soon as the specifications are complete.
Disadvantages
Only a small number of possible inputs can be tested and many program paths will
be left untested.
Without clear specifications, which are the situation in many projects, test cases will
be difficult to design.
Tests can be redundant if the software designer/ developer has already run a test
case.
Ever wondered why a soothsayer closes the eyes when foretelling events? So is
almost the case in Black Box Testing.
23. Differentiate between Black box testing and White box testing. [APRIL 13]
Black box testing White box testing
Black box testing is the Software testing White box testing is the software testing
method which is used to test the software method in which internal structure is being
without knowing the internal structure of known to tester who is going to test the
code or program testing. software testing
Implementation Knowledge is not required Implementation Knowledge is required to

to carry out Black Box Testing. carry out White Box Testing.
Testing is applicable on higher levels of Testing is applicable on lower level of

testing like System Testing, Acceptance testing like Unit Testing, Integration testing.
testing.
Black box testing means functional test or White box testing means structural test or
external testing. interior testing.

Black Box testing can be started based on White Box testing can be started based on
Requirement Specifications documents. Detail Design documents.
This type of testing is carried out by testers. Generally, this type of testing is carried out
by software developers.
In Black Box Testing no knowledge In White Box Testing knowledge regarding

regarding internal logic of code is needed internal logic of code is needed means need
means no need of programming is of programming is mandatory.
necessary.
24. What is Basic path testing? [November 2013]
In software engineering, basis path testing, or structured testing is a white for

designing test cases. The method analyzes the control flow graph of a program to find a set
of linearly independent paths of execution. The method normally uses McCabe' cyclamate
complexity to determine the number of linearly independent paths and then generates test
cases for each path thus obtained.[2] Basis path testing guarantees complete branch
coverage (all CFG edges), but achieves that without covering all possible CFG paths—the
latter is usually too costly
11 MARKS
1.Write in detail about Software testing strategies [APRIL 2012]
A strategy for software testing integrates software test case design methods into a
well-planned series of steps that result in the successful construction of software.
The strategy provides a road map that describes the steps to be conducted as part of
testing, when these steps are planned and then undertaken, and how much effort,
time, and resources will be required.
Therefore, any testing strategy must incorporate test planning, test case design, test
execution, and resultant data collection and evaluation.
A software testing strategy should be flexible enough to promote a customized
testing approach. At the same time, it must be rigid enough to promote reasonable
planning and management tracking as the project progresses.

In many ways, testing is an individualistic process, and the number of different

types of tests varies as much as the different development approaches.
For many years, our only defense against programming errors was careful design
and the native intelligence of the programmer.
We are now in an era in which modern design techniques are helping us to reduce
the number of initial errors that are inherent in the code. Similarly, different test
methods are beginning to cluster themselves into several distinct approaches and
philosophies.
These “approaches and philosophies” are what we shall call strategy.
A STRATEGIC APPROACH TO SOFTWARE TESTING
Testing is a set of activities that can be planned in advance and conducted systematically.
For this reason a template for software testing—a set of steps into which we can place
specific test case design techniques and testing methods—should be defined for the
software process.
A number of software testing strategies have been proposed in the literature. All provide
the software developer with a template for testing and all have the following generic
characteristics:
 Testing begins at the component level2 and works "outward" toward the integration
of the entire computer-based system.
 Different testing techniques are appropriate at different points in time.
 Testing is conducted by the developer of the software and (for large projects) an
independent test group.
 Testing and debugging are different activities, but debugging must be
accommodated in any testing strategy.
A strategy for software testing must accommodate low-level tests that are necessary
to verify that a small source code segment has been correctly implemented as well

as high-level tests that validate major system functions against customer

requirements.
A strategy must provide guidance for the practitioner and a set of milestones for the
manager.
Because the steps of the test strategy occur at a time when deadline pressure begins
to rise, progress must be measurable and problems must surfaces early as possible.
Verification and Validation
Software testing is one element of a broader topic that is often referred to as verification
and validation (V&V).
Verification refers to the set of activities that ensure that software correctly implements a
specific function.
Validation refers to a different set of activities that ensure that the software that has been
built is traceable to customer requirements. Boehm states this another way:
Verification: "Are we building the product right?"

Validation: "Are we building the right product?"
The definition of V&V encompasses many of the activities that we have referred to as
software quality assurance (SQA).
Verification and validation encompasses a wide array of SQA activities that include formal
technical reviews, quality and configuration audits, performance monitoring, simulation,
feasibility study, documentation review, database review, algorithm analysis, development
testing, qualification testing, and installation testing Although testing plays an extremely
important role in V&V, many other activities are also necessary.
Testing does provide the last bastion from which quality can be assessed and, more
pragmatically, errors can be uncovered.
But testing should not be viewed as a safety net. As they say, "You can't test in
quality. If it's not there before you begin testing, it won’t be there when you're
finished testing."

Quality is incorporated into software throughout the process of software

engineering. Proper application of methods and tools, effective formal technical
reviews, and solid management and measurement all lead to quality that is
confirmed during testing.
ORGANIZING FOR SOFTWARE TESTING
For every software project, there is an inherent conflict of interest that occurs as testing
begins. The people who have built the software are now asked to test the software.
This seems harmless in itself; after all, who knows the program better than its developers?
Unfortunately, these same developers have a vested interest in demonstrating that

the program is error free, that it works according to customer requirements, and that it will
be completed on schedule and within budget.
Each of these interests mitigate against thorough testing.
From a psychological point of view, software analysis and design (along with
coding) are constructive tasks.
The software engineer creates a computer program, its documentation, and related
data structures. Like any builder, the software engineer is proud of the edifice that
has been built and looks askance at anyone who attempts to tear it down.
When testing commences, there is a subtle, yet definite, attempt to "break" the thing
that the software engineer has built. From the point of view of the builder, testing
can be considered to be (psychologically) destructive. So the builder treads lightly,
designing and executing tests that will demonstrate that the program works, rather
than uncovering errors. Unfortunately, errors will be present. And, if the software
engineer doesn't find them, the customer will!
The software developer is always responsible for testing the individual units (components)
of the program, ensuring that each performs the function for which it was designed. In
many cases, the developer also conducts integration testing—a testing step that leads to
the construction (and test) of the complete program structure.

Only after the software architecture is complete does an independent test group
become involved.
The role of an independent test group (ITG) is to remove the inherent problems
associated with letting the builder test the thing that has been built. Independent
testing removes the conflict of interest that may otherwise be present.
After all, personnel in the independent group team are paid to find errors.
However, the software engineer doesn't turn the program over to ITG and
walkaway.
The developer and the ITG work closely throughout a software project to ensure
that thorough tests will be conducted.
While testing is conducted, the developer must be available to correct errors that
are uncovered.
The ITG is part of the software development project team in the sense that it
becomes involved during the specification activity and stays involved (planning and
specifying test procedures) throughout a large project.
However, in many cases the ITG reports to the software quality assurance
organization, thereby achieving a degree of independence that might not be possible
if it were a part of the software engineering organization.
A Software Testing Strategy
Initially, system engineering defines the role of software and leads to software
requirements analysis, where the information domain, function, behavior,
performance, constraints, and validation criteria for software are established.
Moving inward along the spiral, we come to design and finally to coding. To develop
computer software, we spiral inward along streamlines that decrease the level of
abstraction on each turn.

Figure1-Testing strategy
A strategy for software testing may also be viewed in the context of the spiral(Figure
1) .
Unit testing begins at the vortex of the spiral and concentrates on each unit (i.e.,
component) of the software as implemented in source code.
Testing progresses by moving outward along the spiral to integration testing, where
the focus is on design and the construction of the software architecture.
Taking another turn outward on the spiral, we encounter validation testing, where
requirements established as part of software requirements analysis are validated
against the software that has been constructed.
Finally, we arrive at system testing, where the software and other system elements
are tested as a whole. To test computer software, we spiral out along streamlines
that broaden the scope of testing with each turn.
Considering the process from a procedural point of view, testing within the context
of software engineering is actually a series of four steps that are implemented
sequentially.
The steps are shown in Figure 2. Initially, tests focus on each component
individually, ensuring that it functions properly as a unit.

Hence, the name unit testing. Unit testing makes heavy use of white-box testing
techniques, exercising specific paths in a module's control structure to ensure
complete coverage and maximum error detection.
Next, components must be assembled or integrated to form the complete software
package. Integration testing addresses the issues associated with the dual problems
of verification and program construction.
Black-box test case design techniques are the most prevalent during integration,
although a limited amount of white-box testing may be used to ensure coverage of
major control paths. After the software has been integrated (constructed), a set of
high-order tests are conducted.
Validation criteria (established during requirements analysis) must be tested.
Validation testing provides final assurance that software meets all functional,
behavioral, and performance requirements. Black-box testing techniques are used
exclusively during validation.
Figure 2-Software testing steps
The last high-order testing step falls outside the boundary of software engineering
and into the broader context of computer system engineering.

Software, once validated, must be combined with other system elements (e.g.,
hardware, people, databases).
System testing verifies that all elements mesh properly and that overall system
function/performance is achieved.
CRITERIA FOR COMPLETION OF TESTING
A classic question arises every time software testing is discussed: "When are we done
testing how do we know that we've tested enough?"
Although few practitioners would argue with these responses, a software engineer needs
more rigorous criteria for determining when sufficient testing has been conducted. Musa
and Ackerman [MUS89] suggest a response that is based on statistical criteria:
"No, we cannot be absolutely certain that the software will never fail, but relative to a
theoretically sound and experimentally validated statistical model, we have done sufficient
testing to say with 95 percent confidence that the probability of 1000 CPU hours of failure
free operation in a probabilistically defined environment is at least 0.995."
Figure 3 -Failure intensity as a function of execution time

Using statistical modeling and software reliability theory, models of software failures
(uncovered during testing) as a function of execution time can be developed . A version of
the failure model, called a logarithmic Poisson execution-time model, takes the form
f(t) = (1/p) ln [l0 pt+ 1] (1)
where
f(t) = cumulative number of failures that are expected to occur once the software
has been tested for a certain amount of execution time, t,
l0 = the initial software failure intensity (failures per time unit) at the beginning of
testing,
p = the exponential reduction in failure intensity as errors are uncovered and

repairs are made.
The instantaneous failure intensity, l(t) can be derived by taking the derivative of f(t)
l(t) = l0 / (l0 pt+ 1) (2)
Using the relationship noted in Equation (2), testers can predict the drop-off of errors as
testing progresses. The actual error intensity can be plotted against the predicted curve
(Figure 3). If the actual data gathered during testing and the logarithmic
Poisson execution time model are reasonably close to one another over a number of data
points, the model can be used to predict total testing time required to achieve acceptably
low failure intensity. By collecting metrics during software testing and making use of
existing software reliability models, it is possible to develop meaningful guidelines for
answering the question: "When are we done testing?" There is little debate that further
work remains to be done before quantitative rules for testing can be established, but the
empirical approaches that currently exist are considerably better than raw intuition.

2. Discuss the techniques involved in white box testing. [NOVEMBER 2011]
INTRODUCTION
White-box testing, sometimes called glass-box testing, is a test case design method that
uses the control structure of the procedural design to derive test cases. Using white-box
testing methods, the software engineer can derive test cases that
1. Guarantee that all independent paths within a module have been exercised at
least once,
2. Exercise all logical decisions on their true and false sides,
3. Execute all loops at their boundaries and within their operational bounds, and
4. Exercise internal data structures to ensure their validity.
The objective of white box testing is to verify the correctness of the software’s
statements, code paths, conditions, loops, and data flow. This objective is often
referred to as logic coverage.
The prerequisites for white box testing include the software requirements, use
cases, the executable program, its data, and its source code.
The software developer normally does white box testing as an extension of code
debugging activity early in the development cycle. Software developers usually
focus on “making the code work” according to use case activities, which gives them
the tendency to debug only the code they know works (selective logic test
coverage).
Testers add value to developer debugging activity by helping the developer plan and
debug more of the code than usual (more thorough logic test coverage).
The more the logic test coverage you attain while debugging, the fewer the defects
will be discovered later by other kinds of testing. The business motivation behind
white box testing is expected economies of testing.
Much of the research you will find in white box testing will relate to hypotheses,
algorithms, and procedures that attempt to achieve 100% logic test coverage under
certain, very controlled circumstances.
Research has not yet produced a white box approach that guarantees 100% logic
test coverage for all situations.

Statement Coverage Technique
Statement coverage techniques focus on determining what percentage of the

source code lines in a program has been executed. If there are 5,000 lines of
source code in a program and you can determine manually or with a tool that
you have executed 4,537 lines of source code, then you have achieved a 90.7%
statement coverage (exceptionally high for complex programs).
Branch (Simple Condition) Coverage Technique
Branch coverage techniques focus on determining what percentage of the source

code branch (true/false) logic in a program has been executed. If there are
1,500source code branch points in a program and you can determine manually
or with a tool that you have executed 1,145 branches (count true and false
branch executions separately), then you have achieved a 76.3% branch point
coverage (exceptionally high for complex programs).
Compound Condition Coverage Technique
The compound condition coverage technique extends the branch coverage

technique to branches with compound conditions, ones that contain
combinations of Boolean operators AND, OR, and NOT along with pairs of
parentheses, possibly nested. Here is a compound condition containing two
simple conditions.
(AGE _ 18 AND SEX _ M)
Decision Coverage Technique
A stronger logic-coverage criterion is known as decision coverage or branch

coverage. This criterion states that you must write enough test cases that each decision has
a true and a false outcome at least once. In other words, each branch direction must be
traversed at least once. Examples of branch or decision statements are switch, do-while,
and if-else statements.

However, there are at least three exceptions:
Programs with no decisions.

Programs or subroutines/methods with multiple entry points. A given statement
might be executed only if the program is entered at a particular entry point.
Statements within ON-units. Traversing every branch direction will not
necessarily cause all ON-units to be executed. Since we have deemed statement
coverage to be a necessary condition, decision coverage, a seemingly better
criterion, should be defined to include statement coverage.
Figure :2.a- A small program

Multiple condition coverage
A criterion that covers the above problem, and then some, is multiple-condition
coverage. This criterion requires that you write sufficient test cases that all possible
combinations of condition outcomes in each decision, and all points of entry, are invoked at
least once. For instance, consider the following sequence of pseudo code.
NOTFOUND=TRUE;
DO I=1 to TABSIZE WHILE (NOTFOUND); /*SEARCH TABLE*/
...searching logic...;
END
The four situations to be tested are:
1. I<=TABSIZE and NOTFOUND is true.
2. I<=TABSIZE and NOTFOUND is false (finding the entry before hitting the end of the
table).
3. I>TABSIZE and NOTFOUND is true (hitting the end of the table without finding the
entry).
4. I>TABSIZE and NOTFOUND is false (the entry is the last one in the table).
Data Flow coverage
Data flow coverage involves tracking a piece of data completely through the
software. At the unit test level this would just be through an individual module or function.
The same tracking could be done through several integrated modules or even through the
entire software product although it would be more time-consuming to do so.
Mutation test
One major problem when all tests cases have been designed and executed for all
tests is to be reasonably confident that the cases will find all bugs. Mutation analysis helps

by identifying omissions. Mutation analysis validates tests and their data by running them
against many copies of the program each containing a different, single, deliberately-
inserted change. If the tests discover the change then they are assumed to be good. Typical
mutations are exchanging all + and . There are two theories underlyingthe approach:
Automated code coverage analysis
Code coverage is used in software testing, as it measures the quality of your test
procedures, by describing the degree to which the source code of a program has been
tested. All embedded developers are used to debugging; the process used to understand the
behaviour of a program such that an error can be corrected. Debugging however, requires
that an error has been detected in the first place. Programming errors that you do not
know exist cannot be debugged or corrected.
The analysis is done such that,
 Input data points are detected

 Insecure methods are identified
 API’s are verified.
 Memory allocation is checked.
 Critical parameters are traced.
 Information flow is known
3. Discuss the techniques involved in Black box testing. [APRIL 2013]
Black-box testing, also called behavioural testing, focuses on the functional

requirements of the software. That is, black-box testing enables the software engineer to
derive sets of input conditions that will fully exercise all functional requirements for a
program. Black-box testing is not an alternative to white-box techniques. Rather, it is a
complementary approach that is likely to uncover a different class of errors than white-box
methods.

BLACK BOX TESTING TECHNIQUES
Boundary-Value Analysis
Experience shows that test cases that explore boundary conditions have a higher
payoff than test cases that do not. Boundary conditions are those situations directly on,
above, and beneath the edges of input equivalence classes and output equivalence classes.
Boundary-value analysis differs from equivalence partitioning in two respects:
1. Rather than selecting any element in an equivalence class as being representative,

boundary-value analysis requires that one or more elements be selected such that
each edge of the equivalence class is the subject of a test.
2. Rather than just focusing attention on the input conditions (input space), test cases
are also derived by considering the result space (output equivalence classes).
A few general guidelines are as follows:
If an input condition specifies a range of values, write test cases for the ends of the
range, and invalid-input test cases for situations just beyond the ends. For instance,
if the valid domain of an input value is −1.0–+1.0, write test cases for
The situations −1.0, 1.0, −1.001, and 1.001.
If an input condition specifies a number of values, write test cases for the minimum
and maximum number of values and one beneath and beyond these values. For
instance, if an input file can contain 1–255 records, write test cases for 0, 1, 255, and
256 records.
Use guideline 1 for each output condition. For instance, if a program computes the
monthly FICA deduction and if the minimum is $0.00 and the maximum is
$1,165.25, write test cases that cause $0.00 and $1,165.25 to be deducted. Also, see
if it is possible to invent test cases that might cause a negative deduction or a
deduction of more than $1,165.25. Note that it is important to examine the
boundaries of the result space because it is not always the case that the boundaries
of the input domains represent the same set of circumstances as the boundaries of
the output ranges (e.g., consider a sine subroutine). Also, it is not always possible to

generate a result outside of the output range, but it is worth considering the
possibility, nonetheless.
Use guideline 2 for each output condition. If an information- retrieval system
displays the most relevant abstracts based on an input request, but never more than
four abstracts, write test cases such that the program displays zero, one, and four
abstracts, and write a test case that might cause the program to erroneously display
five abstracts.
If the input or output of a program is an ordered set (a sequential file, for example,
or a linear list or a table), focus attention on the first and last elements of the set.
In addition, use your ingenuity to search for other boundary conditions. The triangle
analysis program can illustrate the need for boundary-value analysis.
Robustness Testing
Robustness testing is a simple extension of boundary value analysis: in addition to the

five boundary value analysis values of a variable, we see what happens when the
extreme are exceeded with a value slightly greater than the maximum (max+) and a
value slightly less than the minimum (min-).
The most interesting part of robustness testing is not with the inputs, but with the
expected outputs.
What happens when a physical quantity exceeds its maximum? If it is the angle of attack
of an airplane wing, the aircraft might stall. If it’s the load capacity of a public elevator,
we hope nothing special would happen.
Equivalence partitioning
Equivalence partitioning identifies ranges of inputs and initial conditions that are
expected to produce the same result. Equivalence partitioning relates to the commonality
and variances among the different situations in which a system is expected to work. If
situations are equivalent, or essentially similar, to one another, it is adequate to test only
one of them, not all. Although equivalence is usually intuitively obvious, it is necessary to
be careful about what is assumed to be equivalent.

Equivalence classes may be defined according to the following guidelines:
If an input condition specifies a range, one valid and two invalid equivalence classes
are defined.
If an input condition requires a specific value, one valid and two invalid equivalence
classes are defined.
If an input condition specifies a member of a set, one valid and one invalid
equivalence class are defined.
If an input condition is Boolean, one valid and one invalid class are defined.
SYNTAX TESTING
Identify the target language or format (hidden or explicit).
Define the syntax (format) of the language, formally, in a convenient notation such as
Backus-Naur form (BNF).
2. Normal condition testing consists of a covering set of input strings including critical
loop values. The difficult part about normal case testing is predicting the outcome
and verifying that the processing was correct. That’s ordinary functional testing—
i.e., semantics. Covering the syntax graph assures that all options have been tested.
This is a minimum mandatory requirement with the analogous strengths and
weaknesses of branch testing for control flow graphs. It isn’t “complete” syntax
testing by any measure.
3. Syntax testing methods pay off best for dirty testing. Test design is a top-down
process that consists of methodically identifying which component is to be crud
died-up and how.

5. Test execution automation is essential for syntax testing because this method
produces so many tests.
Originally motivated by hardware logic testing, the finite-state machine model is an

excellent model for testing menu-driven applications. It is also important because of its
widespread use in object-oriented design.
External Prerequisite Terms: application, behaviour, boot, bug, character, close file,
control, data, design, hardware, file, function key, integer, load, logic, LSI, menu, menu bar,
mapping, memory, message, method, mouse, mouse click, MS-DOS, object-oriented
software, open file, operating system, serial port, software, subroutine, symbolic debugger,
Windows.
Internal Prerequisite Terms: behavioural testing, coincidental correctness, domain,

domain testing, graph, input, link, link cover, link list, link weight, model, node, node cover,
node weight, null, output, path, state, strongly connected, syntax testing, system, value.
State: States are depicted by nodes.
Input event: A distinct, repeatable event or fixed sequence of events of interest

characterized by inputs (or input sequence) to the system. What constitutes an interesting
event is specific to the application. In the context of finite-state models, I'll use the term
"input" where not ambiguous to denote "input event."
Examples: an external stimulus, an input message, a character, a menu selection, a fixed

sequence of user commands. The inputs of finite-state models concern control rather than
data: inputs that change the behaviour of the system rather than just produce different
outputs. Examples: the value of an integer control parameter, inputs whose values cause
software to follow different processing paths, a representative input for each input domain.
Each of these is considered a distinct input event.

Input encoding: Each event can be given a name or number: that is, the input events can
be mapped onto the integers, say, or onto a set of characters. This is called an input
encoding. The behaviour of a finite-state machine is not changed by changing the input
encoding.
Input symbols: The set of distinct names or values of the input encoding. Number of input
symbols: The input encoding assigns an integer from 1 to n, say, to every distinct input
event. We assume that there are no gaps in these numbers. In most models, the number of
input symbols is small typically under 20. Bigger numbers can be handled, but usually not
without tools.
State code: The states can be numbered. This is called a state encoding. States have
operational meaning. For example, the states of a floppy disk drive might be:
(1) start-up,
(2) motor-on,
(3) seeking track,
(4) seeking sector,
(5) reading,
(6) writing,
(7) erasing,
(8) motor off.
Current state: At any instant, the system is in one state, the current state.
Initial state: A special state, called the initial state, usually is the state of the system prior
to any input.
in booting up. But a good program does provide a way to return to the initial state:
consider, the CONTROL-ALT-DELETE combination in Windows

Working states: Eventually, the system leaves the initial state set and reaches a strongly
connected set of working state; this is where most of testing is done.
4. Write in detail about Syntax testing.
Syntax Testing, a black box testing technique, involves testing the System inputs and
it is usually automated because syntax testing produces a large number of tests.
The syntax may be,

 Formally specified
 Undocumented
 Just understood
 Does exist
There are 3 possible kinds of incorrect actions,
 Recognizer does not recognize a good setting.
 Recognizer accepts a bad string
 Recognizer crashes during attempts to recognize a string.
The sources of syntax are,
 Designer test cooperation
 Manuals
 Help screens
 Design documents
 Prototypes
 Programmer interviews
 Experiment
1. Identify the target language or format (hidden or explicit).

2. Define the syntax (format) of the language, formally, in a convenient notation such
as Backus-Naur form (BNF).

4. Normal condition testing consists of a covering set of input strings including critical
loop values. The difficult part about normal case testing is predicting the outcome
and verifying that the processing was correct. That’s ordinary functional testing—
i.e., semantics. Covering the syntax graph assures that all options have been tested.
This is a minimum mandatory requirement with the analogous strengths and
weaknesses of branch testing for control flowgraphs. It isn’t “complete” syntax
testing by any measure.
5. Syntax testing methods pay off best for dirty testing. Test design is a top-down
process that consists of methodically identifying which component is to be
cruddied-up and how.
7. Test execution automation is essential for syntax testing because this method
produces so many tests.
Grammar for formats
Objectives
Every input has syntax. That syntax may be formally specified or undocumented and
“just understood,” but it does exist. Data validation consists (in part) of checking the
input for correct syntax. It’s best when the syntax is defined in a formal language—best
for the designer and the tester.
Whether the designer creates the data-validation software from a formal specification
or not is not important to the tester, but the tester needs a formal specification to create
useful garbage.
That specification is conveniently expressed in Backus-Naur form, which is very similar
to regular expressions.
Regular expressions are an algebraic representation of all the paths in a graph. It’s
usually more convenient to deal with the algebraic version of graphs than with the
pictorial version.

Get comfortable with going back and forth between algebraic forms and pictorial forms
for graphs and with talk about “covering a graph” even if there’s no pictorial graph
around. This isn’t new to you because you worked with paths through an algebraic
representation of a graph long before you heard about flow graphs
BNF Notation
The Elements
Every input can be considered as if it were a string of characters. The software

accepts valid strings and rejects invalid ones. If the software fails on a string, we’ve really
got it. If it accepts an invalid string, then it’s guilty of GIGO. There’s nothing we can do about
syntactically valid strings whose values are valid but wrong—that kind of garbage we have
to accept. The syntax definition must be formal, starting with the most elementary parts,
the characters themselves. Here’s a sample definition:
Alpha characters ::= A/B/C/D/E/F/G/H/l/J/K/L/M/N/O/P/Q/
R/S/T/U/V/W/X/Y/Z
Numerals ::= 1/2/3/4/5/6/7/8/9
Zero ::= 0
Signs ::= !/#/$/%/&/*/(/)/-/+/=/;/:/“/’/,/./?
Space ::= sp
The left-hand side of the definitions is the name given to the collection of objects on
the right-hand side. The string “::=” is interpreted as a single symbol that means “is defined
as.” The slash “/” means “or.” We could have used the plus sign for that purpose as with
regular expressions but that wouldn’t be in keeping with the established conventions for
BNF. We are using BNF to define a miniature language.

The “:=” is part of the language in which we talk about the mini language, called the
meta language. Spaces are always confusing because we can’t display them on paper. We
use sp to mean a space. The actual spaces on this page have no meaning. Similarly, an
italicized (or underlined) symbol is used for any other single character that can’t
conveniently be printed, such as null (nl), end-of-text (eot), clear-screen, carriage-return
(cr), line-feed (lf), tab, shift-up (su), shift-down (sd), index, backspace (bs), and so on. The
underlined space, as in alpha_characters, is used as usual in programming languages to
connect words that comprise a single object.
BNF Operators
The operators are the same as those used in path expressions and regular
expressions: “or,” concatenate, (which doesn’t need a special symbol), “×”, and “+”.
Exponents, such as An, have the same meaning as before—n repetitions of the strings
denoted by the letter A. Here’s an example:
word ::= alpha_characteralpha_character / numeral sp numeral
Defined an input string called word as a pair of alpha characters or a pair of numerals
separated by a space. Here are examples of words and no words, by this definition:
Words: AB, DE, XY, 3 sp 4, 6 sp 7, 9 sp 9, 1 sp 2
nonwords : AAA, A sp A1, A), 11, 111, WORD, NOT sp WORD, +
There are 722 possible words in this format and an infinite number of nonwords. If
the strings are restricted to four characters, there are more than a million nonwords. The
designer wants to detect and accept words and reject no words; the tester wants to
generate no words and force the program to deal with them.

Repetitions
As before, object1-3 means one to three objects, object* means zero or more repetitions
of object without limit, and object+ means one or more repetitions of object. Neither the
star (*) nor the plus (+) can legitimately appear in any syntax because both symbols
mean a possibly infinite number of repetitions.
That can’t be done in finite memory or in the real world. The software must have some
means to limit repetitions. It can be done by an explicit test associated with every + or *
operator, in which case you should replace the operator with a number. Another way to
limit the repetitions is by placing a global limit on the length of any string.
The limit then applies to all commands and it may be difficult to predict what the actual
limit is for any specific command. You test this kind of limit by maximum-length strings.
Yet another way to implement limits is to limit a common resource such as stack or
array size. Again, the limits for a specific command may be unpredictable because it is a
global limit rather than a format-specific limit.
The way to test this situation is with many repetitions of short strings.
One of the signs of weak software is the ease with which you can destroy it by
overloading its repetition-limiting mechanisms. If the mechanism doesn’t exist, you can
probably scribble all over the stack or code—crash-crash, tinkle-tinkle, goody-good.
5. Write in detail about finite state testing technique.
Originally motivated by hardware logic testing, the finite-state machine model is an

excellent model for testing menu-driven applications.
It is also important because of its widespread use in object-oriented design.
External Prerequisite Terms: application, behaviour, boot, bug, character, close file,
control, data, design, hardware, file, function key, integer, load, logic, LSI, menu, menu bar,
mapping, memory, message, method, mouse, mouse click, MS-DOS, object-oriented

software, open file, operating system, serial port, software, subroutine, symbolic debugger,
Windows.
Internal Prerequisite Terms: behavioural testing, coincidental correctness, domain,

domain testing, graph, input, link, link cover, link list, link weight, model, node, node cover,
node weight, null, output, path, state, strongly connected, syntax testing, system, value.
State: States are depicted by nodes.
Input event: A distinct, repeatable event or fixed sequence of events of interest

characterized by inputs (or input sequence) to the system. What constitutes an interesting
event is specific to the application. In the context of finite-state models, I'll use the term
"input" where not ambiguous to denote "input event."
Input encoding: Each event can be given a name or number: that is, the input events can
be mapped onto the integers, say, or onto a set of characters. This is called an input
encoding. The behaviour of a finite-state machine is not changed by changing the input
encoding.
Input symbols: The set of distinct names or values of the input encoding. Number of input
symbols: The input encoding assigns an integer from 1 to n, say, to every distinct input
event. We assume that there are no gaps in these numbers. In most models, the number of
input symbols is small typically under 20. Bigger numbers can be handled, but usually not
without tools. State code: The states can be numbered. This is called a state encoding.
States have operational meaning. For example, the states of a floppy disk drive might be:
(1) start-up,
(2) motor-on,
(3) seeking track,
(4) seeking sector,

(5) reading,
(6) writing,
(7) erasing,
(8) motor off.
Current state: At any instant, the system is in one state, the current state.
Initial state: A special state, called the initial state, usually is the state of the system prior
to any input.
State counter: A hypothetical or actual memory location that holds the state code of the
current state. The state counter often is implicit. The program counter of a computer is an
explicit state counter.
Number of states: The state counter has a maximum value. If there are no gaps in the state
code, this maximum value is the number of states in the model. As with input symbols, the
number of states in a model usually is small (under 30). Bigger models can be handled with
appropriate tools.
Finite-state machine: An abstract machine (e.g., program, logic circuit, car's transmission)
for which the number of states and input symbols are both finite and fixed. A finite-state
machine consists of states (nodes), transitions (links), inputs (link weights), and outputs
(link weights). Finite-state machines in this book are depicted by state graphs.
Transitions: A system responds to input events; the state may change as a result. This is
called a state transition, or transition for short. A system transmits a message to another.
While it was transmitting, it was in the ''transmitting" state.
Self-transition: Denotes no change of state; it is a link from a state back to that state. There
may be an output associated with that transition.

Output encoding: There may be an output associated with a transition: that is, an output
action is initiated as a result of a state change. Output actions can also be mapped onto
integers. This mapping is an output encoding. The behaviour of a finite-state model is not
affected by the specifics of the output encoding.
Output event: The system may produce an output as a result of a state change. This is
equivalent to outputting an integer, such as the output code.
Null output: A hypothetical null output event: for example, do nothing.
Reachable state: A state B is reachable from a state A if there is a sequence of inputs such
that, when starting from state A, the model will end up in state B. That is, the state graph
has a path from A to B.
Unreachable state: A state is unreachable from other states, especially from the initial
state, if it is not reachable. Unreachable states usually mean a bug.
Strongly connected: Most states of practical finite-state machines used in testing are
strongly connected if there are no bugs.
Isolated states: A set of states not reachable from the initial state. Within a set of isolated
states, the states may or may not be strongly connected. It is their unreachability from the
initial state that isolates them. Isolated states in software testing models are suspicious if
not buggy.
Reset: A special input that forces a transition to the initial state from any state. If every
state is reachable from the initial state and if there is a reset, the state graph is strongly
connected. A reset is not essential, but it makes test design and testing much easier. Reset
also may be implemented as a reset to a specified state that is not the true initial state of
the model.
Initial state set: A set of states with the following properties: It includes the initial state(s).
The initial state set may or may not be strongly connected. Once a transition is made out of
that set to a state not in that set, it is impossible to return to the initial state set. Consider

booting up a system. The software goes through a series of steps, each of which can be
modelled by a state.
connected set of working state; this is where most of testing is done
Null output: A hypothetical null output event: for example, do nothing.
Reachable state: A state B is reachable from a state A if there is a sequence of inputs such
that, when starting from state A, the model will end up in state B. T Unreachable state: A
state is unreachable from other states, especially from the initial state, if it is not reachable.
Unreachable states usually mean a bug. Hat is, the state graph has a path from A to B.
Strongly connected: Most states of practical finite-state machines used in testing are
strongly connected if there are no bugs.
Isolated states: A set of states not reachable from the initial state. Within a set of isolated
states, the states may or may not be strongly connected. It is their unreachability from the
initial state that isolates them. Isolated states in software testing models are suspicious if
not buggy.
Reset: A special input that forces a transition to the initial state from any state. If every
state is reachable from the initial state and if there is a reset, the state graph is strongly
connected. A reset is not essential, but it makes test design and testing much easier. Reset
also may be implemented as a reset to a specified state that is not the true initial state of
the model.
Initial state set: A set of states with the following properties: It includes the initial state(s).
The initial state set may or may not be strongly connected. Once a transition is made out of
that set to a state not in that set, it is impossible to return to the initial state set. Consider
booting up a system. The software goes through a series of steps, each of which can be
modelled by a state. Once the software is loaded, it is impossible to return to the initial
state set because you can't "deboot" a program. The system might progress through several
initial state sets: for example, the initial sets consist of one state each and denote the stages

in booting up. But a good program does provide a way to return to the initial state: consider
the CONTROL-ALT-DELETE combination in Windows.
connected set of working state; this is where most of testing is done.
What to Verify:
1. Verify Input Encoding. Does your model's notion of input encoding correspond to what's
implemented? If they differ, which is wrong, the model or the software? How you verify
that depends on what you're encoding. If the inputs are numerical, then you might use
domain testing to verify input encoding. If character strings, then syntax testing, or for that
matter, another state model. One way or the other you must confirm the correspondence
between your model's notion of input events and the software's and explain all
discrepancies.
2. Verify Output Encoding. There are (should be) relatively few distinct output events or
action sequences. You've given each of them a name. If the coded output ''A" means: "open
file, update file, close file, report back," then you must confirm that all that happens and
that it happens in that order. You may have to examine intermediate computation steps, file
activity logs, use a symbolic debugger . . . But however you do it, you must confirm that the

actions of your output codes match the sequence of actions in the real software and explain
any discrepancy if they don't.
3. Are You in the Initial State? The techniques discussed in this book assume that there is at
least one initial state and that it is possible to get there. There could be several initial states
in a complicated system. For example, the initial state for single-user operation is unlikely
to be the same as for network operation. You can't begin to test unless you are in the initial
state and know it. The software telling you that it is in the initial state isn't enoughafterall,
that could be the very bug for which we're looking. The initial state has properties such as
which files are open, resource usages, programs active, and so on. If there's an initial state
set, such as a boot or load sequence, then you must confirm every state and transition in
that set of states. This can be difficult if you don't have design support because such
sequences often are automatic. A step mode is a very useful test aid.
4. Can You Reach Every Exit State? Typically there's not one but several exit states or a set of
exit states: states that, once entered, will effectively take the processing out of your model.
The obvious one is the state in which you close the program you're testing. But it isn't
always that simple. For example, if I choose to close my word processor, I'm asked to
confirm it. If I confirm that I want to exit and I have any open files that weren't previously
stored, the program then asks me if I want to save the change. That's a nontrivial set of exit
states.
5. Verify State. Are all the states that should be there present? The purpose of doing a tour to
a target state is to show that the state exists and is reachable. You must, of course, have
some positive means of knowing the state at all times. Usually this is possible in software
testing, especially if there is an explicit state counter and if the designers have thought
about testability. If you don't have a means to identify the state, testing is still.
6. Are There Extra States? That's difficult to determine both theoretically and operationally.
The heuristic prescription however, usually finds extra states if they exist.
7. Confirm Every Transition. There is a potential output associated with every transition.
More precisely, there is an outcome of every transition. The transition consists of two

things: the new state and outputs if any. If there is a null "output" associated with a
transition, that too must be confirmed. You defined a state encoding. That means you have
detailed characteristics that will uniquely identify states. Confirm every component of the
state encoding. But you could be fooled because of the possibility of extra states, discussed
in the next section. Similarly, you must confirm every output action defined in your model's
output encoding especially null outputs. Make sure that nothing really happens for null
outputs. If you end up in the correct state and had the right outputs (including null) for that
transition, then it's a good bet that the transition was correct.
6 .Explain in detail on various levels of testing. [NOVEMBER 2013]
There are four levels of software testing:
UNIT TESTING
The process taking of a module and run it in isolation from the rest of the software product
by using prepared test cases and comparing actual result with expected output.
The purpose of this test is to find (and remove)as many errors in the software as practical
WHY UNIT TESTING?
1. Unit testing is a way of managing the combined elements of testing, since attention
is focused on smaller units of the program.
2. Unit testing eases the task of debugging, when an error is found, it is known to exist
in a particular module.

3. Introduces parallelism into the program testing process by presenting with the
opportunity to test multiple modules simultaneously.
• Unit Testing checks that an individual program unit (subprogram, object
class, package and module) behaves correctly.
– Static Testing
• testing a unit without executing the unit code
– Dynamic Testing
• testing a unit by executing a program unit using test data
Example:
• To test a unit designed to compute the “greatest common divisor” (GCD) of a pair of
integers (not both zero).
– GCD(a,b) = c where
• c is a positive integer
• c is a common divisor of a and b (e.g., c divides a and c divides b)
• c is greater than all other common divisors of a and b.
– For example
• GCD(45, 27) = 9
• GCD (7,13) = 1
• GCD(-12, 15) = 3
• GCD(13, 0) = undefined
• GCD(0, 0) undefined
GCD Test Planning
• How do we proceed to determine the tests cases?

1. Design an algorithm for the GCD function.
2. Analyze the algorithm using basic path analysis.
3. Determine appropriate equivalence classes for the input data.
4. Determine the boundaries of the equivalence classes.
5. Then, choose tests cases that include the basic path set, data frm each
equivalence class, and data at and near the boundaries.

GCD Algorithm
1. function gcd (int a, int b) {

2. int temp, value;
3. a := abs(a);
4. b := abs(b);
5. if (a = 0) then
6. value := b; // b is the GCD
7. else if (b = 0) then
8. raise exception;
9. else
10. loop
11. temp := b;
12. b := a mod b;
13. a := temp;
14. until (b = 0)
15. value := a;
16. end if;
17. return value;
18. end gcd

7
6
9
1
0
17
18
• Basic Path Set

– V(G) = 4
– (1,5,6,17,18), (1,5,7,18), (1,5,7,9,10,17,18), (1,5,7,9,10,9,10,17,18)
• Equivalence Classes
– Although the the GCD algorithm should accept any integers as input, one
could consider 0, positive integers and negative integers as “special” values.
This yields the following classes:
• a < 0 and b < 0, a < 0 and b > 0, a > 0 and b < 0
• a > 0 and b > 0, a = 0 and b < 0, a = 0 and b > 0
• a > 0 and b = 0, a > 0 and b = 0, a = 0 and b = 0
• Boundary Values
– a = -231, -1, 0, 1, 231-1 and b = -231, -1, 0, 1, 231-1

Test Description / Data Expected Results Test Experience / Actual Results

Basic Path Set
path (1,5,6,17,18)  (0, 15) 15
path (1,5,7,18)  (15, 0) 15
path (1,5,7,9,10,17,18)  (30, 15) 15
path (1,5,7,9,10,9,10,17,18)  (15, 30) 15
Equivalence Classes
a < 0 and b < 0  (-27, -45) 9
a < 0 and b > 0  (-72, 100) 4
a > 0 and b < 0  (121, -45) 1
a > 0 and b > 0  (420, 252) 28
a = 0 and b < 0  (0, -45) 45
a = 0 and b > 0  (0 , 45) 45
a > 0 and b = 0  (-27, 0) 27
a > 0 and b = 0  (27, 0) 27
a = 0 and b = 0  (0 , 0) exception raised
Boundary Points
(1 , 0) 1
(-1 , 0) 1
(0 , 1) 1
(0 , -1) 1
(0 , 0) (redundant) exception raised
(1, 1) 1
(1, -1) 1
(-1, 1) 1
(-1, -1) 1
UNIT TESTING IN TEST CASE DESIGN
• Motto: “Clean code that works.” (Ron Jeffries)

• Unit testing has “broader goals” that just insuring quality
– Improve developers lives (coping, confidence)
– Support design flexibility and change
– Allow iterative development with working code early
UNIT TESTING BENEFITS
• Developers can work in a predictable way of developing code

• Programmers write their own unit tests
• Get rapid response for testing small changes
• Build many highly-cohesive loosely-coupled modules to make unit testing easier

INTEGRATION TESTING
• Integration Testing is testing interfaces between components

• It is the first step after Unit Testing
• Components may work alone but fail when put together
• Defect may exist in one module but manifest in another
• Black-box tests
TYPES OF INTEGRATION TESTING
TOP-DOWN
BOTTOM-UP
MODULE STRUCTURE
A B
leve C H
level 0 E F G I
• A uses C and D; B uses D; C uses E and F; D uses F, G, H and I; H uses I

• Modules A and B are at level 3; Module D is at level 2 Modules C and H are at level
1; Modules E, F, G, I are at level 0
• level 0 components do not use any other components

• level i components use at least one component on level i-1 and no component at a
level higher than i-1
TOP-DOWN INTEGRATION TESTING
The Top-down method begins by testing the main program unit (the root of the
tree) with one lower level node. Any other lower level units/nodes that may be
connected should create as a stub.
Only modules tested in isolation are the modules which are at the highest level
After a module is tested, the modules directly called by that module are merged with
the already tested module and the combination is tested
Requires stub modules to simulate the functions of the missing modules that may be
called.
However, drivers are not needed since we are starting with the modules which is
not used by any other module and use already tested modules when testing modules
in the higher levels
A B
C
H
E F G I

Advantage of using top-down approach
The major advantage of this top-down method is the fact that a system prototype
can be developed early on in the project process.
This is a very attractive property of this integration testing technique as usually the
client of a software company will have little or no software engineering knowledge
and therefore if the client cannot see something tangible then it would be easy for
the client to assume that little or no work is being completed.
In addition to this, having an early prototype will give the client a better feel for how
the final product will look which is useful as any changes that the client might think
of will appear early in the development process (where it would be easier to
implement).
Disadvantage:
One disadvantage of this method is the fact that the programmers will haveto
produce a large number of stubs, this means extra work is needed on topof
producing the actual system by producing units that will later be thrownaway.
Assuming the developer is using the decomposition-tree model the formula below
can be used to calculate the number of stubs that must be produced.
1. BOTTOM – UP APPROACH
The bottom-up method begins by testing one of the leaves6 of the program with its
parent node first.
Any higher nodes that are directly connected to the nodes being tested are constructed
as drivers.
Only terminal modules (i.e., the modules that do not call other modules) are
tested in isolation
Modules at lower levels are tested using the previously tested higher level
modules
Non-terminal modules are not tested in isolation

Requires a module driver for each module to feed the test case input to the
interface of the module being tested
However, stubs are not needed since we are starting with the terminal modules
and use already tested modules when testing modules in the lower levels
A B
C H
E G I
F
Advantage:
The major advantage of this method of integration testing is that the program
itself is fully functional at every stage.
This is in contrast from the top-down method, where as a prototype can be
created at an early stage, but it will have little or no functionality.
Disadvantage:
Inability to create an early prototype for the client. This could easily lead to
problems down the line, such as the client may question the progress of the

project or when the prototype is finally produced the client might decide on
changes that are difficult to implement at that late stage of the project.
The formula can be used to calculate the number of stubs that must be produced.
number of drivers = (number of nodes - number of leaves).
OTHER APPROACHES TO INTEGRATION TESTING
Sandwich Integration
– Compromise between bottom-up and top-down testing

– Simultaneously begin bottom-up and top-down testing and meet at a
predetermined point in the middle
Big Bang Integration
– Every module is unit tested in isolation

– After all of the modules are tested they are all integrated together at once
and tested
– No driver or stub is needed
– However, in this approach, it may be hard to isolate the bugs!
SYSTEM TESTING
• System test is concerned with testing the entire system which may be composed of
many very large “sub-systems,” which may themselves be large enough to be
considered a system.
• The problems we encounter in system test may have originated from any of the
previous activities:
– Requirements Processing
– Designing
– Programming
– Fixing Defects
– System integration and build

• Prior to this stage, we have focused on defect discovery and fixing; now, at system
test, we are more interested in seeing if the whole system works - - - a slight shift in
mentality.
TYPES OF SYSTEM TESTING
1. Functionality test:
– This part is looking at the interactions among the sub-systems and delivering
joint functionality more than just individual, simple function.
• e.g. payroll in HR sub-system should affect the accounts payable and
general ledger functions in the Financial subsystem - - - giving us an
updated ledger.
2. Performance test:
– This part is looking at several areas of the over-all system:
• Throughput
• Response time
• Security
• Reliability & Recovery
3. Acceptance Test:
– This part ensures that, while the developers may be happy with the system,
the actual users are in fact “happy” with the system
• Functionality completeness
• Usability in terms of human interaction with the system (a la prototype)
• Reliability (in terms of error tolerance and “undo” or “re-do”)
4. Installation Test:
– This is ensuring that the software can be installed in the users’ environment
• System match
• DB match
• Network match
• Initialization of data and defaults are set properly

1. FUNCTIONALITY TEST
During system test we are not so much testing individual simple functions, but focused
more on complex sets of functions that may be inter-related by combination of
conditions.
Black box testing techniques are mostly used here to check out high level.
A popular technique is to use the “decision table” or “truth table” approach from logic.
Components of a Decision Table
Rules
R1 R2 R3 R4 R5 R6 R7 R8
C1 T TTT F FFF
T T F F T T F F Values of conditions
C2
C3 T F T F T F T F
a1 x xxx
a2 x x
Actions taken
Actions x x
a3
x xxx
a4
x x
a5
Read a Decision Table by columns of rules : R1 says when all conditions
are T, then actions a1, a2, and a5 occurs
Decision Table and Test Cases
• Each rule may be viewed as a test case.

• If we want to exercise all the combinations of conditions, then we would have to
have 2n test cases for n conditions that take on binary values.

• If we want to cover all the actions, we may or may not need to execute all the
combinations of conditions
2. Performance Tests
Performance tests include a long list of different tests:
a. Stress and Volume test to evaluate the system’s capability to handle full or
more than full level of processing (e.g. users, devices, transactions, storage,
etc.)
b. Configuration, environment, and compatibility test to ensure that the
application does run on different platform configurations, as specified in
requirements and sales brochure.
c. Timing test in terms of response time and transaction time must be met,
especially for real-time processing
d. Security test ensures that the integrity, confidentiality, and protection
functions perform properly
e. Recovery test ensures that the system responds properly to errors and loss
of information, or loss of device.
f. Documentation test ensures that the specified documents exist, are accurate,
and matches the system.
g. Human Factors and Usability test ensures that the system meets the users
“ease of use” requirements
h. Quality test evaluates the systems reliability, availability and maintainability.
Quality Test
• Quality may be looked from many different perspectives. One is to consider the
following 3 areas:
– Reliability of the system: proper behavior of the system over a period of
time

– Availability of the system: the accessibility of the system at any instant of

time
– Maintainability of the system: the fixability or supportability of the system
System may have defect and still run for a long period of time
--- is it reliable?
- System may have defects and still be available for processing in other
Functional areas - - - is it available?
- System may have defects that can be easily fixed and be back running again –
- - - is it maintainable?
ACCEPTANCE TEST
• This is the testing of the system mainly by users and customers (sometimes with the
help of testers and developers) to ensure that the system is what the users need - - -
may not be exactly what the users said in the requirements document.
• It is NOT obvious that users will automatically perform this - - - it needs to be
“planned” because users and customers all have fulltime jobs.
TYPES OF ACCEPTANCE TEST
1. Benchmarking:
– Users have some very special needs (performance or functional) and want to
make sure that those are met.
– Users will be preparing those special test cases
– Sometimes used for comparing and purchasing existing applications.
2. Pilot test:
– Users want to make sure that the typical, everyday usage is covered.

– Test all functions and all major business flows

– May be in alpha (in house users) and beta (external users) forms
• Parallel:
– Testing the new system next to the existing system for a period a time
– Allows comparison and contrast to the existing system
– Users gradually transition to the new system
INSTALLATION TEST
• If user acceptance test is performed and a broad set of user environments (all the
combination of platforms and interfaces) are completed, then this test may be
“skipped.”
• Otherwise ensure that the specified combinations in requirements or sales
brochures are tested:
– All the different versions of operating systems are tested
– All the different versions of data bases are tested
– The network and different interfaces are tested
– The default data and initialization for different combination of applications
are tested (e.g. HR with Financial)
–
7. Explain Mutation Testing. [NOVEMBER 2013]
Mutation Testing?
Mutation Testing is a testing technique that focuses on measuring the adequacy of

test cases.
• Mutation Testing is NOT a testing strategy like path or data-flow testing. It does
not outline test data selection criteria.
• Mutation Testing should be used in conjunction with traditional testing techniques,

not instead of them.

• Faults are introduced into the program by creating many versions of the program
called mutants.
• Each mutant contains a single fault.
• Test cases are applied to the original program and to the mutant program.
• The goal is to cause the mutant program to fail, thus demonstrating the effectiveness
of the test case.
TEST CASE ADEQUACY
• A test case is adequate if it is useful in detecting faults in a program.
• A test case can be shown to be adequate by finding at least one mutant program that
generates a different output than does the original program for that test case.
• If the original program and all mutant programs generate the same output, the test
case is inadequate.
MUTANT PROGRAMS
• Mutation testing involves the creation of a set of mutant programs of the

program being tested.
• Each mutant differs from the original program by one mutation.
• A mutation is a single syntactic change that is made to a program statement.
CATEGORIES OF MUTATION OPERATORS

• Operand Replacement Operators:
– Replace a single operand with another operand or constant. E.g.,
• if (5 > y) Replacing x by constant 5.
• if (x > 5) Replacing y by constant 5.

• if (y > x) Replacing x and y with each other.
– E.g., if all operators are {+,-,*,**,/} then the following expression a = b * (c - d)

will generate 8 mutants:
• 4 by replacing *
• 4 by replacing -.
• Expression Modification Operators:
– Replace an operator or insert new operators. E.g.,
• if (x == y)
• if (x >= y) Replacing == by >=.
• if (x == ++y) Inserting ++.
• Statement Modification Operators:
– E.g.,
• Delete the else part of the if-else statement.
• Delete the entire if-else statement.
• Replace line 3 by a return statement.
MUTATION TESTING ALGORITHM
• Generate program test cases.
• Run each test case against the original program.
– If the output is incorrect, the program must be modified and re-tested.
– If the output is correct go to the next step ...
• Construct mutants using a tool like Mothra.

• Execute each test case against each alive mutant.
– If the output of the mutant differs from the output of the original program,
the mutant is considered incorrect and is killed.
• Two kinds of mutants survive:
– Functionally equivalent to the original program: Cannot be killed.
– Killable: Test cases are insufficient to kill the mutant. New test cases must be
created.
MUTATION SCORE
• The mutation score for a set of test cases is the percentage of non-equivalent
mutants killed by the test data.
• Mutation Score = 100 * D / (N - E)
– D = Dead mutants
– N = Number of mutants
– E = Number of equivalent mutants
• A set of test cases is mutation adequate if its mutation score is 100%.
8. EXPLAIN IN DETAIL ABOUT INTEGRATION TESTING. [NOVEMBER 2011]
Testing in which software components, hardware components, or both together

are combined and tested to evaluate interactions between them.
Integration testing usually go through several real word business scenarios to
see whether the system can successfully complete workflow tasks.
Integration plan specifies the order of combining the modules into partial
systems.
The entire system is viewed as a collection of subsystems (sets of classes)
determined during the system and object design
Goal: Test all interfaces between subsystems and the interaction of subsystems
The Integration testing strategy determines the order in which the subsystems
are selected for testing and integration.

Why do we do integration testing?
• Unit tests only test the unit in isolation
• Many failures result from faults in the interaction of subsystems
• Often many Off-the-shelf components are used that cannot be unit tested
• Without integration testing the system test will be very time consuming
• Failures that are not discovered in integration testing will be discovered after the
system is deployed and can be very expensive.
Stubs and drivers
• Driver:
Driver
• A component, that calls the TestedUnit
• Controls the test cases
• Stub:
• A component, the TestedUnit Tested

depends on Unit
• Partial implementation
• Returns fake values.
Stub
TYPES OF INTEGRATION TESTING
TOP-DOWN
BOTTOM-UP
MODULE STRUCTURE

A B
leve C H
l1
level 0 E F G I
• A uses C and D; B uses D; C uses E and F; D uses F, G, H and I; H uses I

• Modules A and B are at level 3; Module D is at level 2 Modules C and H are at level
1; Modules E, F, G, I are at level 0
• level 0 components do not use any other components
• level i components use at least one component on level i-1 and no component at a
level higher than i-1
TOP-DOWN INTEGRATION TESTING
The Top-down method begins by testing the main program unit (the root of the
tree) with one lower level node. Any other lower level units/nodes that may be
connected should create as a stub.
Only modules tested in isolation are the modules which are at the highest level
After a module is tested, the modules directly called by that module are merged with
the already tested module and the combination is tested

Requires stub modules to simulate the functions of the missing modules that may be
called.
However, drivers are not needed since we are starting with the modules which is
not used by any other module and use already tested modules when testing modules
in the higher levels
A B
C
H
E F G I
Advantage of using top-down approach
The major advantage of this top-down method is the fact that a system prototype
can be developed early on in the project process.
This is a very attractive property of this integration testing technique as usually the
client of a software company will have little or no software engineering knowledge

and therefore if the client cannot see something tangible then it would be easy for
the client to assume that little or no work is being completed.
In addition to this, having an early prototype will give the client a better feel for how
the final product will look which is useful as any changes that the client might think
of will appear early in the development process (where it would be easier to
implement).
Disadvantage:
One disadvantage of this method is the fact that the programmers will haveto
produce a large number of stubs, this means extra work is needed on topof
producing the actual system by producing units that will later be thrownaway.
Assuming the developer is using the decomposition-tree model the formula below
can be used to calculate the number of stubs that must be produced.
2. BOTTOM – UP APPROACH
The bottom-up method begins by testing one of the leaves6 of the program with its
parent node first.
Any higher nodes that are directly connected to the nodes being tested are constructed
as drivers.
Only terminal modules (i.e., the modules that do not call other modules) are
tested in isolation
Modules at lower levels are tested using the previously tested higher level
modules
Non-terminal modules are not tested in isolation
Requires a module driver for each module to feed the test case input to the
interface of the module being tested
However, stubs are not needed since we are starting with the terminal modules
and use already tested modules when testing modules in the lower levels

A B
C
H
E G I
F
Advantage:
The major advantage of this method of integration testing is that the program
itself is fully functional at every stage.
This is in contrast from the top-down method, where as a prototype can be
created at an early stage, but it will have little or no functionality.
Disadvantage:
Inability to create an early prototype for the client. This could easily lead to
problems down the line, such as the client may question the progress of the

project or when the prototype is finally produced the client might decide on
changes that are difficult to implement at that late stage of the project.
The formula can be used to calculate the number of stubs that must be produced.
number of drivers = (number of nodes - number of leaves).
OTHER APPROACHES TO INTEGRATION TESTING
Sandwich Integration
– Compromise between bottom-up and top-down testing

– Simultaneously begin bottom-up and top-down testing and meet at a
predetermined point in the middle
Big Bang Integration
– Every module is unit tested in isolation

– After all of the modules are tested they are all integrated together at once
and tested
– No driver or stub is needed
However, in this approach, it may be hard to isolate the bugs!

PONDICHERRY UNIVESITY QUESTIONS
2 marks
1. What is basic path testing?[November-13][Pg.No.9, Qn. No.:24]

2. What is the purpose of boundary value analysis? [November 13][Pg no:6 Qn.
No:16]
3. What is the difference between integration testing and system testing?
[April 2013][Pg.no.8, Q.No.23]
4. Define integration testing?[November 11] [Pg.no.6, Q.No.12]
5. Define static testing?[April-12] [Pg.no.6, Q.No.12]
6. What is statement coverage? [April-12] [Pg.no.3, Q.No.7]
7. What is finite state testing? [April-12] [Pg.no.21, Q.No.10]
11 marks
1. Explain mutation testing? [NOVEMBER 2013] [Pg.no.53, QN.NO.7]

2. What are the levels of software testing? Explain.[NOVEMBER 2013] [Pg.no.39,
QN.NO.6]
3. Explain white box testing technique? [NOVEMBER 2011] [Pg.no.18, QN.NO.2]
4. Briefly explain integration testing? [NOVEMBER 2011] [Pg.no.56, QN.NO.8]
5. Discuss about statement coverage? [APRIL 2012] [Pg.no.18, QN.NO.2]
6. Explain system testing? [APRIL 2012] [Pg.no.39, QN.NO.7]

Department of Computer Science and Engineering: Subject Name: Subject Code: CS E71

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Department of Computer Science and Engineering: Subject Name: Subject Code: CS E71

Uploaded by

Copyright:

Available Formats

Sri ManakulaVinayagar Engineering College

Department of Computer Science and Engineering

Verified by: Approved by:

Software quality assurance and testing Page 1

1.What is meant by testing strategy?

2. Mention the characteristics of testing strategies.

All testing strategies have following characteristics.

Testing proceeds in an outward manner. It starts from testing the individual

3. What are the types of software testing strategies?

Analytic testing strategy: It takes a complete overview of requirements, design, and

Software quality assurance and testing Page 2

4. What is white box testing?

5. What are white box testing techniques?

The white box techniques are

6.What are the advantages and disadvantages of white box testing?

Accuracy of implemented code can be determined.

Software quality assurance and testing Page 3

Knowledge of internal structure or working of application helpful in

To test the software application a highly skilled resource is required to carry

7.What is statement coverage?[APRIL 2012]

Statement coverage is one of the code coverage analysis techniques.This technique

8.what is branch coverage or decision coverage?

Branch coverage is closely related to decision coverage . Decision coverage

9.What is multiple condition coverage?

Multiple-condition coverage is a more detailed kind of coverage than condition

Software quality assurance and testing Page 4

10. What is mutation testing?

11.How to create mutant programs?

Original Program Mutant Program

If (x>y) If(x<Y< strong="">)</Y<>

Print "Hello" Print "Hello"

Print "Hi" Print "Hi"

12. What is Dataflow coverage?

Software quality assurance and testing Page 5

13. What is code coverage analysis?

Code coverage is an important type of test effectiveness measurement. Code

14. What is Black box technique?

15 .Mention the techniques of black box testing?

The Black box testing techniques are

16. What is boundary value analysis? [NOVEMBER 2013]

17. Define Robustness testing.

Robustness testing is a testing methodology to detect the vulnerabilities of a

Software quality assurance and testing Page 6

18. Write about Equivalence portioning.

19. What is Syntax testing?

20. What are the steps involved in syntax testing?

Syntax testing consists of the following steps:

1. Identify the target language or format (hidden or explicit).

Software quality assurance and testing Page 7

22.What are the advantages and disadvantages of Black box testing ?

Black box testing White box testing

Implementation Knowledge is not required Implementation Knowledge is required to

Testing is applicable on higher levels of Testing is applicable on lower level of

Software quality assurance and testing Page 8

In Black Box Testing no knowledge In White Box Testing knowledge regarding

24. What is Basic path testing? [November 2013]

In software engineering, basis path testing, or structured testing is a white for

1.Write in detail about Software testing strategies [APRIL 2012]

Software quality assurance and testing Page 9

In many ways, testing is an individualistic process, and the number of different

A STRATEGIC APPROACH TO SOFTWARE TESTING

Software quality assurance and testing Page 10

DO I=1 to TABSIZE WHILE (NOTFOUND); /SEARCH TABLE/