Software teSting

Unit II :Testing Methodologies

Notes edited and compiled by Prof. Mohan S. Khedkar

Specification

• Requirements document – what the software will become

• Typically, the product specification is a written document using

words and pictures to describe the intended product.

• Specification is testable; can be used to find bugs before

software is developed

• Specification can be in written or unwritten form.

Windows Calculator product spec:

The Edit menu will have two selections:

Copy and Paste.

These can be chosen by one of three

methods: pointing and clicking with the
mouse, using access-keys (Alt+C for
Copy and Alt+P for Paste), or using the
standard Windows shortcut keys of
Ctrl+C for Copy and Ctrl+V for Paste.

The Copy function will copy the current

entry displayed in the number text box
into the Windows Clipboard.

The Paste function will paste the value

stored in the Windows Clipboard into
the number text box.
Black Box Testing
• Black Box Testing, also known as Behavioral Testing, is
a software testing method in which the internal structure/
design/ implementation of the item being tested is not known
to the tester. These tests can be functional or non-functional,
though usually functional.
Black Box Testing

• Black box testing – what is supposed to be done functionality

- does not concern how the software accomplish the work

- outside view of the software

- input-output relationships
This method attempts to find errors in the following categories:

• Incorrect or missing functions

• Interface errors
• Errors in data structures or external database access
• Behavior or performance errors
• Initialization and termination errors

EXAMPLE :
A tester, without knowledge of the internal structures of a
website, tests the web pages by using a browser; providing inputs
(clicks, keystrokes) and verifying the outputs against the expected
outcome.
Black Box Testing Advantages
• Tests are done from a user’s point of view and will help in
exposing discrepancies in the specifications.

• Tester need not know programming languages or how the

software has been implemented.

• Tests can be conducted by a body independent from the

developers, allowing for an objective perspective and the
avoidance of developer-bias.

• Test cases can be designed as soon as the specifications are

complete.
Black Box Testing Disadvantages

• Only a small number of possible inputs can be tested and

many program paths will be left untested.

• Without clear specifications, which is the situation in many

projects, test cases will be difficult to design.

• Tests can be redundant if the software designer/ developer

has already run a test case.
White-box testing
• White-box testing (also known as clear box testing, glass box
testing, transparent box testing, and structural testing) is a
method of testing software that tests internal structures or
workings of an application.
• White (clear) box testing – how the software does its job
- look into code details

• In white-box testing an internal perspective of the system,

as well as programming skills, are used to design test cases.

• The tester chooses inputs to exercise paths through the code

and determine the expected outputs.

• The tester needs to have a look inside the source code and
find out which unit/chunk of the code is behaving
inappropriately.
Advantages of White Box testing
• Testing can commence even before the GUI is ready.

• As internal functionality is considered, all the possible

conditions are considered and test cases are generated. Hence
all the functionalities are being tested.

• It minutely verifies whether the program can be successfully

executed with other parts of the application.

• It removes extra lines of code which are not required in the

program thereby optimizing the program and increases the
efficiency.

• It helps in evaluating all the loops and paths.

• Various hidden defects get unearth while conducting clear box
testing.
Disadvantages of White Box testing
• As the internal code of the application has to be considered
while preparing the test cases, skilled testers are required who
have knowledge of programming also. Hence the cost of the
resources is high.
• It is not possible for the tester to look into every bit of the
code and identify the hidden errors. This may result in failure of
the application.
• White box testing is an exhaustive method.
• It takes time to tester to develop the test cases.
• Test cases are a waste if changes in the implementation code
are done frequently.
• If the application is large then complete testing through white
box techniques is not feasible.
Static and Dynamic Testing
• Static Testing – testing a software without running it
- examination
- review

• Dynamic Testing – running and using the software.

• The best analogy - when checking out a used car.

oKicking the tires, checking the paint, and looking under the
hood are static testing techniques.

oStarting it up, listening to the engine, and driving down the

road are dynamic testing techniques.
Static Black Box Testing : Testing Specification
• Static black box testing
- The specification is a document, not a program  static

• carefully examine it for bugs.

• goal: to find bugs early

• Specification can be a written or graphical document or a

combination of both.

• You can even test an unwritten specification by questioning the

people who are designing and writing the software.
High Level Review of Specification
• First Step: Stand back and view it from high level
– Examine the spec for large fundamental problems, oversights, and
omissions.

• Better understanding of the Spec

– whys and hows behind the spec

• Pretend to be the customer

- find out who they are, e.g. research & interview
- gain familiarity of the application field
- customer expectations, quality is measured in terms of these
expectations
- try to understand the spec
• Research existing standards and guidelines
- why standards and guidelines?
. terminology and conventions
. Industrial/government requirements
. Hardware/software interfaces
- make sure correct standards & guidelines are used

• Review and test similar software

- similar systems of feel and look  efficiency
- competitors’ products
- issues to review
. Scale: bigger, smaller, similar, etc.
. Complexity
. Testability: can you test such software
. Quality/reliability
- use the software
Low Level Specification Test
• Specification Attribute Checklist
- A good, well-thought-out specification:
. Complete
. Accurate
. Precise, unambiguous, and clear
. Code-free: what, but not how
. Testable
• Specification terminology checklist - avoid
problem words
. Always, every, all, none, never: absolute
. Certainly, therefore, clearly, obviously, …
. Sometimes, often, usually, …
. Etc., and-so-on, and-so-forth: no way to test
. Good, fast, cheap, efficient, … : not quantified
Is Specification Testable?

• Yes!

• Sample usage scenarios

- For example, the user will …
- All aspects of the scenarios must be covered.
Examining Specification Summary

• Subjective process
- not a step by step process

• Higher level review

- to flush out oversights and omissions

• Low level techniques

- to assure all details are defined
Dynamic Black Box Testing
• 1. It is very easy to start testing the software without knowing and just
rectify the errors as and when encountered.

• 2. Such an approach might work for a little while. If the software is still
under development, it’s very easy to get lucky and find a few bugs right
away.

• 3. Unfortunately, those easy pickings will quickly disappear and a more

structured and targeted approach is required to continue finding bugs
and to be a successful software tester.

• 4. The testing approach has to be designed for it to work perfectly and

for a successful software tester.
Techniques For Dynamic Black Box Testing

1.Test-to-Pass and Test-to-Fail

2.Equivalence Partitioning

3.Data Testing

4.State Testing

5.Other Black-Box Test Techniques

 Test-to-Pass and Test-to-Fail
• 1. When you test-to-pass, you really assure only that the software
minimally works and as a software tester it is the first positive
approach.

• 2.don’t see what you can do to break it. Test the software by
applying the simplest and most straight forward test cases.

• 3. During test to fail, the software should be tested for the boundary
conditions on its various limits.

• 4. If the program is using integer numbers, than the output should

be tested in the limits of the integer variable.
Example of Test-to-Pass

• To test an application which contains a simple textbox to enter

age and requirements say that it should take only integers
values.
Example of Test-to-Fail

It should show a validation error message for all invalid inputs or

system should not allow to enter a non integer values.
Equivalence Partitioning

• In this method, the input domain data is divided into different

equivalence data classes.

• This method is typically used to reduce the total number of test

cases to a finite set of testable test cases, still covering maximum
requirements.

• It is the process of taking all possible test cases and placing them
into classes. One test value is picked from each class while testing.
Let us consider a program that separates integers into positive
or negative. And accepts any number between -5 and + 5.

The range of input integers can be split into the following partitions :
•Negative Integers: Values between -5 and -1
•Zero (neither positive nor negative)
•Positive Integers: Values between 1 and 5
•Values > 5 (Invalid)
•Values < -5 (Invalid)

Valid Valid Valid Invalid Invalid

-5 -1 0 1 5 >5 < -5
•Ticket values 1 to 10 are considered valid & ticket is booked. While value 11 to
99 are considered invalid for reservation and error message will appear, "Only
ten tickets may be ordered at one time."

1.Any Number greater than 10 entered in the reservation column (let say 11) is
considered invalid.
2.Any Number less than 1 that is 0 or below, then it is considered invalid.
3.Numbers 1 to 10 are considered valid
4.Any 3 Digit Number say 100 is invalid.
Data Testing
• 1. The simplest view of software is to divide its world into two parts:
the data and the program.
• 2. The data is the keyboard input, mouse clicks, disk files, printouts,
and so on.
• 3. The program is the executable flow, transitions, logic, and
computations.
• 4. The user information is checked and the data is tabulated with
the expected results.

• Examples of data would be

•The words you type into a word processor
•The numbers entered into a spreadsheet
•The backup files stored on your floppy disk
•The data being sent by your modem over the phone lines
Boundary Conditions

• 1. Boundary conditions are special because programming, by its

nature, is susceptible to problems at its edges.

• 2. The boundary conditions are defined as the initial and the final
data ranges of the variables declared.

• 3. If an operation is performed on a range of numbers, odds are the

programmer got it right for the vast majority of the numbers in the
middle, but maybe made a mistake at the edges.

• 4. The edges are the minimum and the maximum values for that
identifier.
Sub-Boundary Conditions
• 1. They’re the ones defined in the specification or evident when
using the software.

• 2. Some boundaries, though, that are internal to the software aren’t

necessarily apparent to an end user but still need to be checked by
the software tester.

• 3. These are known as sub-boundary conditions or internal

boundary conditions.

• 4. In the given example the sub boundary condition is the value of

factorial
For example

1. #include<stdio.h>
2. void main()
3. {
4. int i , fact=1, n;
5. printf(“enter the number “);
6. scanf(“%d”,&n);
7. for(i =1 ;i <=n;i++)
8. fact = fact * i;
9. printf (“the factorial of a number is ”%d”, fact);
10. }
Default, Empty, Blank, NULL, Zero and None, Invalid, Wrong,
Incorrect and Garbage Data

• 1. The final type of data testing is garbage data. This is where you test-to-
fail.

• 2. You’ve already proven that the software works as it should by testing-

to-pass with boundary testing, sub boundary testing, and default testing.

• 3. Now it’s time to throw the trash at it.

• 4. isn’t necessary, that if you’ve tested everything discussed so far you’ve

proven the software will work.

• 5. In the real word, however, there’s nothing wrong with seeing if the
software will handle whatever a user can do to it.
State Testing

• 1. The data gets tested on the numbers, words, inputs, and outputs
of the software.

• 2. The product or the software should be tested for the program’s

logic flow through its various states.

• 3. A software state is a condition or mode that the software is

currently in.

• 4. example illustrates the software mode of the paint software in

airbrush mode.
Example illustrates the software mode of the paint software in
airbrush mode.
Testing Software’s Logic Flow
• 1. Testing the software’s states and logic flow has the same
problems. It’s usually possible to visit
• 2. Check for the all possible states for test to pass condition
• 3. Creating a State Transition Map
Testing States to Fail
• Testers while performing takes care about the various aspects of
operating system and the various testing methods
• Race Conditions and Bad Timing
1. As operating systems can do multitasking.
2. Multi-tasking means that an operating system is designed to run
separate processes concurrently.
3. These processes can be separate programs such as a spreadsheet
• Repetition, Stress, and Load
1. These tests target state handling problems where the programmer
didn’t consider what might happen in the worst-case scenarios.
2. Repetition testing involves doing the same operation over and over.
3. Stress testing is running the software under less-than-ideal
conditions such as low memory, low disk space, slow CPUs, slow
modems.
4. load testing, you feed it all that it can handle
Other Black-Box Test Techniques
These techniques are the combinations of the various tests and
methods and the specification details that are been listed before in
the chapter.
• 1. Behave like a dumb user: - As a tester behave like you are new to
the field and don't not know anything to do with the product or the
software that you are testing.
• 2. Look for bugs where you have already found them: - As a tester if
you have already worked on the similar project then just jump to
the places where you know that the bugs can be found. Bugs are
likely to occur at the same place.
• 3. Follow Experience, Intuition, and Hunches: - the experience of the
test gives the tester an advantage to find the bugs early and also
how to remove the bugs as early as possible.
 thank You
S/W Testing Notes edited and compiled by Prof. Mohan
S. Khedkar