Fundamentals of SW Testing

FUNDAMENTALS OF
SOFTWARE TESTING
Rahesh Dudani
Certified Quality Manager
Certified Software Quality Professional
Certified Software Test Manager
Certified Lead Auditor - ISO 9001:2000
Certified Lead Auditor – BS 7799:2002
e mail: rahesh.dudani@gmail.com
phone: 2569247 mobile: 9370148115
FUNDAMENTALS OF SOFTWARE TESTING

1 of 44
SICSR
SOFTWARE TESTING
"Primary role of testing is not
demonstration of correct
performance, but the exposure
of hidden defects."
- G. J. Myers
2 of 44
SICSR
SOFTWARE TESTING
• It is not sufficient to
demonstrate that the software is
doing what it is supposed to do
• It is more important to
demonstrate that the software is
not doing what it is not
supposed to do.
3 of 44
SICSR
TRADITIONAL DEFINITIONS OF TESTING
• Establishing confidence that a program does
what it is supposed to do (Hetzel, 1973)
• Confirming that a program performs its
intended functions correctly
• THESE DEFINITIONS ARE
INCOMPLETE AND DESCRIBE ALMOST
THE OPPOSITE OF WHAT TESTING
SHOULD BE!!
4 of 44
SICSR
WHAT TESTING ACTUALLY IS
• Identifying the differences between
expected and actual results
• Process of executing a program (or
part of it) with the intention or goal of
finding defects
• Detecting specification errors and
deviations from the specifications.
5 of 44
SICSR
ESSENTIALS OF SOFTWARE TESTING
• TESTING IS DESTRUCTIVE PROCESS :
A CREATIVE DESTRUCTION
• TESTING NEEDS A SADISTIC
APPROACH
• If a test does not detect a defect actually
present, is it successful or unsuccessful?
• Test that detects a defect is valuable
investment: it has helped in improving the
product..
6 of 44
SICSR
ESSENTIALS OF SOFTWARE TESTING
• It is risky to develop software

and not test it
• High pressures to deliver
software as quickly as possible:
test process must provide
maximum value in shortest time.
7 of 44
SICSR
VERIFICATION
• Disciplined approach to evaluate whether
a software product fulfils the requirements
or conditions imposed on them (are we
building the product right?)
• Method: walkthrough, formal inspection
and review of each software product
• Also called static testing..

8 of 44
SICSR
VERIFICATION
• Done by systematically reading the
contents of a software product with the
intention of detecting defects
• Helps in identifying not only presence of
defects but also their location
• A ‘filter’ applied at various points during
the SDLC to ‘purify’ the product as it
progresses through various phases.
9 of 44
SICSR
VALIDATION
• Disciplined approach to evaluate whether
the final, as-built software product fulfils its
specific intended use (are we building the
right product?)
• Method: testing each software product at
each phase of life cycle using test plans, test
cases for properly selected types of tests
• Also called dynamic testing..

10 of 44
SICSR
VALIDATION
• Done by systematically testing a software
product with the intention of finding defects
• Helps in identifying the presence of defects,
not their location
• Necessary to demonstrate not just that the
software is doing what it is supposed to do, but
also is not doing what it is not supposed to do

11 of 44
SICSR
WALKTHROUGH
• An informal process, initiated by the author of a
software product to a colleague for assistance in
locating defects and for suggesting improvements
» normally not planned
» author explains the product
» colleague comes out with observations
» author provides clarification if required
» author notes down relevant points and takes
corrective actions.
12 of 44
SICSR
INSPECTION
• A thorough, word-by-word checking
of a software product (or part of a
product) with the intention of:
» locating defects
» confirming traceability of relevant
requirements
» checking for conformance to relevant
standards and conventions.
13 of 44
SICSR
REVIEW
• Normally done at the end of a phase of SDLC,
when the author/s feel/s that the product is
error-free and can go to the next phase
» Usually conducted by a manager
» Main purpose is to decide if the product can go the
next phase
» Includes to check if suitable inspections have been
done
» After review, the product is ‘base-lined’.
14 of 44
SICSR
GUIDELINES FOR INSPECTIONS
• Plan the inspection

• Allocate trained resources
• Moderator / inspection leader circulates the
product to be inspected and checklist to
inspection team in advance
• Product is clear-compiled, checked for
spellings, grammar, formatting, etc..
15 of 44
SICSR
• Inspection team of 3 to 5 members:
roles - moderator, author, scribe,
domain expert, user
• Maximum time: 1 to 1 1/2 hour per
inspection; size of product to be
selected accordingly
• Inspect the product; not the person
behind it..
16 of 44
SICSR
• Author / moderator reads the product,
line by line
• Concentrate on locating defects; avoid
discussions on possible solutions
• Classify and record the defects
• Review the inspection process
• Products that can be inspected: SRS,
Design, Code, User Manual, Installation
Manual, Help Screen, Various plans, etc.
17 of 44
SICSR
ECONOMICS OF TESTING
Optimum
testing
Number of Cost of
defects testing
Duration
18 of 44
ESTABLISHING A TESTING POLICY
• Good testing does not just
happen
• It must be planned
• Automation is not the only
answer
• Avoid ‘Anti-testing’ messages.

19 of 44
SICSR
TEST PROCESSES
• Establish test objectives
• Design test cases
• Write test cases
• Review test cases
• Execute the tests
• Examine test results
• Repeat the tests (regression).
20 of 44
SICSR
PRINCIPLES OF SOFTWARE TESTING
• Good testing MUST examine:

» users’ requirements
» design objectives
» user interface
» internal structure
» AND then execute the code..

21 of 44
SICSR
The probability of detecting new
defects is proportional to the number of
detecting more defects
defects already detected:

Probability of
Number of defects detected

22 of 44
• Testing is not something done as a
formality at the end of development; it is
done throughout the SDLC to detect
defects as early as possible and prevent
defect migration to subsequent phases
• SOFTWARE TESTING =
» Verification (Static Testing)
»+
» Validation (Dynamic Testing).

23 of 44
SICSR
TEST METHODS
• Black box testing (also called
functional testing):
» Test to the specs (Do not look
inside the code)
• White box testing (also called
structural / glass box testing):
» Test to the code (Do not look
inside the specs).
24 of 44
SICSR
OTHER IMPORTANT TESTS
• Stress Testing
• Usability Testing
• Security Testing
• Storage Testing
• Recovery Testing
• Procedure Testing
• Compatibility Testing
• Installability Testing
• Documentation Testing
• Configuration Testing
• Parallel Testing.
25 of 44
SICSR
TEST PLAN
• Test plan identifier: unique
• Introduction: objective, background, scope,
references
• Test items: product to be tested, with version
number; document against which each is to
be tested
• Features to be tested: traced to clause / para
no. of referenced document.

26 of 44
SICSR
TEST PLAN
• Suspension criteria
• Resumption criteria
• Test deliverables: test cases, test logs, test
incident reports, test summary
• Environmental needs: hardware, software,
communication lines, special tools, stubs, drivers,
etc.
• Responsibilities: managing, designing, executing,
witnessing, checking, approving, resolving..

27 of 44
SICSR
TEST PLAN
• Skills required
• Training needs, providing training
• Risks and contingencies
• Completion criteria: cost,
duration, defects found, defects
remaining, coverage.

28 of 44
SICSR
• THE SECRETS OF GOOD
TESTING:
» Proper test plans
» Systematically prepared test cases
• Without these, you are
doing ‘MONKEY
TESTING’.
29 of 44
SICSR
CASE AND CAST
TOOLS

30 of 44
SICSR
MANUAL SYSTEMS
• Difficult to obtain compliance

» Inexperienced persons lack perspective/ motivation
» Procedures may be felt cumbersome/irritating
» Human error – forget/by – pass – “just this once”
» Some persons see it as a block to creativity and
against their fundamental right to circumvent it.
• Require more “Training” on procedures
• Suffer more in case of personnel turnover
31 of 44
SICSR
AUTOMATED SYSTEMS
• Can be done at various levels (Organization, Project)
• Beneficial for routine and cumbersome tasks
• Automation is more difficult to circumvent, hence
compliance is better, less non-compliances in audits
• Less chances of error
• Team members have more time for non-routine
activities, can concentrate on work needing human
involvement.

32 of 44
SICSR
CASE / CAST TOOLS
• Tools which automate various
activities related with software
development:
» Project Management, analysis, design,
coding, testing, documentation, etc.
• Help to reduce effort required for
development of a product..
33 of 44
SICSR
CASE / CAST TOOLS
• Help to build quality into a product
• Not the ultimate solution: skilled
people are required to understand,
select and use proper tools effectively
• Can be a single tool for supporting a
specific activity or a combination of a
number of components.
34 of 44
SICSR
• Business Process Engineering and
modeling tools:
» Represent business data objects, their
relationships, processes
• Project planning tools:
» Estimating size, effort, duration, resources
» Network modeling, interdependencies, scheduling
• Risk analysis tools:
» Handling list of risks, their analysis

35 of 44
SICSR
• Project management tools:
» Monitoring of progress of project
» Collecting metrics related to project performance
• Requirements tracing tools:
» Create database of all requirements
» Identify requirements traced to other products
• Metrics management tools:
» Collect and update various metrics like KLOC, defects /
KLOC, effort in person-days / person-months, etc.

36 of 44
SICSR
• Documentation tools:
» Contain templates for various documents
» Provide facilities for updating documents only by
authorized personnel
» Handle version control of documents
• Database management tools:
» Handle project database
• Software Configuration Management tools:
» Configuration identification, change control, library control,
version control, configuration status, configuration audit, etc.

37 of 44
SICSR
• Analysis and design tools:
» Help create models of the system to be built
(Data Flow Diagrams, Control Flow Charts, etc.)
• Prototyping / simulation tools:
» Develop mock-ups of the system to be built like screen
and report layouts
• Interface design and development tools:
» Tool kit for components like menus, icons, buttons,
scrolling mechanisms, etc.

38 of 44
SICSR
• Programming tools:
» Various compilers, editors, application generators,
OO programming environments, query languages,
graphical programming environments, etc.
• Web development tools:
» Generating elements of web pages: text, colours, fonts,
graphics, etc.
• Static testing tools:
» Performing walkthroughs, inspections and reviews of products
» Complexity analysis, code comprehension, syntax and
semantic analysis..
39 of 44
SICSR
• Tools for test planning:
» Templates for test plans
» Test effort and staffing estimates (based on complexity,
size)
» Test schedules, monitoring
• Tools for test design:

» Test data generators
» Requirements based test design
» Coverage analysis: statement coverage, condition
coverage, decision coverage, multiple condition
coverage, path coverage..
40 of 44
SICSR
• Tools for test execution and evaluation:
» Capture / playback
» Coverage analysis
» Test case management
» Collection of defect metrics
» Defect tracking
• Tools for simulation and performance:
» Client / server testing tools: load tests, stress tests
» Web testing tools
» Testing communication applications, networks
» Simulate hardware configuration.
41 of 44
SICSR
ISSUES TO CONSIDER
• Integration with other tools/development platform
• Scaling to support growth (no. of users)
• Security/protection features
• Different types of user interfaces
• Supports multiple site development
• Supports different operating systems
• Need for hardware resources (some tools require
considerable disk space, RAM and CPU power)..
42 of 44
SICSR
ISSUES TO CONSIDER
• Price and licensing terms
• Delivery period
• Caliber of development/support staff :
Training – Conceptual and tool based
• Vendor attributes
• Opinion of customer base
• Utility to the organisation
• Stability
• Customizing to suit your process: or
customizing your process to suit the tool?
43 of 44
SICSR
COSTS OF AUTOMATION
• Automation
» Cost of tool
» Cost of training
» Cost of changing “way of working”
• Non-automation
» Time spent doing the tasks manually
» Efforts in correcting problems due to
• Non compliance
• Human errors
44 of 44
SICSR

Fundamentals of SW Testing

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Fundamentals of SW Testing

Uploaded by

Copyright:

Available Formats

FUNDAMENTALS OF

FUNDAMENTALS OF SOFTWARE TESTING

• It is risky to develop software

FUNDAMENTALS OF SOFTWARE TESTING

FUNDAMENTALS OF SOFTWARE TESTING

FUNDAMENTALS OF SOFTWARE TESTING

• Plan the inspection

FUNDAMENTALS OF SOFTWARE TESTING

• Good testing MUST examine:

FUNDAMENTALS OF SOFTWARE TESTING

defects already detected:

Number of defects detected

FUNDAMENTALS OF SOFTWARE TESTING

FUNDAMENTALS OF SOFTWARE TESTING

FUNDAMENTALS OF SOFTWARE TESTING

FUNDAMENTALS OF SOFTWARE TESTING

FUNDAMENTALS OF SOFTWARE TESTING

• Difficult to obtain compliance

FUNDAMENTALS OF SOFTWARE TESTING

FUNDAMENTALS OF SOFTWARE TESTING

FUNDAMENTALS OF SOFTWARE TESTING

FUNDAMENTALS OF SOFTWARE TESTING

FUNDAMENTALS OF SOFTWARE TESTING

• Tools for test design:

You might also like