Download as pptx, pdf, or txt
Download as pptx, pdf, or txt
You are on page 1of 95

[NEC Group Internal Use Only]

Overview of
ISTQB Foundation Level

NEC Technologies India Pvt. Ltd.


4th July 2016- 5th July,2016
Presented By:
Tanya Maini
Contents:

 Fundamentals Of Testing

 Testing throughout the software life cycle

 Static Techniques

 Test Design Techniques

 Test Management

 Tool Support for Testing


Part1: Fundamentals of
Testing

3 © NEC Corporation 2015 NEC Group Internal Use Only


Testing and its Need

▌ ‘Whenever we are testing something, we are checking whether


it is FIT FOR PURPOSE!’

▌ It is needed as humans make mistakes. Some of them are


deadly!

4 © NEC Corporation 2015 NEC Group Internal Use Only


Testing and its Need

▌ Test activities exist before and after test execution.

▌ Thought process and activities involved in designing early tests


and review of documents helps to prevent defects.

▌ Objectives:
 Finding Defects.
 Gaining confidence about level of quality.
 Providing information for decision making.
 Preventing defects

5 © NEC Corporation 2015 NEC Group Internal Use Only


Why is Testing Necessary?

▌ Software systems form an integral part of life from business


applications to consumer products

▌ Software products that do not work correctly can lad to


problems like:

 Loss of money
Time or business reputation
Injury or death.

6 © NEC Corporation 2015 NEC Group Internal Use Only


Testing through third party

When I can test my own work,


why ask others to come and spot
my flaws?

We are good lawyers for our own


mistakes, but very good judges
of the mistakes of others!!!

7 © NEC Corporation 2015 NEC Group Internal Use Only


How Much Testing is Enough?

▌ It should take account of level of risk including:

 Technical risks
 Safety Aspects
 Business Risks
 Project constraints such as time and budget

Testing should provide sufficient information to stakeholders to


make informed decisions about release of software being tested
or handover to customer.

8 © NEC Corporation 2015 NEC Group Internal Use Only


Hierarchy to failures

9 © NEC Corporation 2015 NEC Group Internal Use Only


Defects placement in SDLC

10 © NEC Corporation 2015 NEC Group Internal Use Only


Cost of Defects in SDLC

11 © NEC Corporation 2015 NEC Group Internal Use Only


Defining Software Testing

12 © NEC Corporation 2015 NEC Group Internal Use Only


Testing Principles (1/8)

13 © NEC Corporation 2015 NEC Group Internal Use Only


Testing Principles (2/8)

Testing shows presence of bugs

▌ Testing reduces the probability of undiscovered defects


remaining in software, but even if no defects are
found, it is not a proof of correctness.

14 © NEC Corporation 2015 NEC Group Internal Use Only


Testing Principles (3/8)

Exhaustive Testing is impossible

▌ Exhaustive Testing means testing all combinations of inputs and


preconditions

▌ If we test all combinations, then testing time and cost will


increase exponentially

▌ Optimum amount of testing needs to be planned based on


assessment of various factors (For Ex: Risk, Customer Needs
etc.)
15 © NEC Corporation 2015 NEC Group Internal Use Only
Testing Principles (4/8)

Early Testing

▌ Cost of defects increase exponentially across the SDLC.

▌ Hence, testing must start as early as possible in the Software


development life cycle.

▌ It is done to find as many early defects and fix them at a low


cost.

16 © NEC Corporation 2015 NEC Group Internal Use Only


Testing Principles (5/8)

Defect Clustering

▌ 80% of the defects are found in 20% of the modules.

▌ With experience, testers should be able to detect such buggy


modules and test them early, so as to find a good amount of
defects.

17 © NEC Corporation 2015 NEC Group Internal Use Only


Testing Principles (6/8)

The Pesticide Paradox

▌ If the same tests are repeated over and over again, eventually
the same test cases will no longer find new bugs

▌ To overcome this, the test cases need to be regularly reviewed


and revised, adding new and different tests to help find new
defects.

18 © NEC Corporation 2015 NEC Group Internal Use Only


Testing Principles (7/8)

Testing is context dependent

▌ Different kinds of software are tested differently

▌ A safety-critical software is tested differently from an e-


commerce site

▌ A storage product is tested differently from an ATM machine’s


software

19 © NEC Corporation 2015 NEC Group Internal Use Only


Testing Principles (8/8)

Absence of error fallacy

▌ If the system built is unusable and does not fulfil the user’s
needs and expectations then finding and fixing defects does not
help.

20 © NEC Corporation 2015 NEC Group Internal Use Only


Useful Terminologies:

▌Maintenance Testing: Verify that no new defects have been


introduced during development of the changes
▌Operational Testing: Assess system characteristics such as
reliability or availability
▌Debugging: Development activity that finds, analyses and
removes the cause of failure
▌Re-testing: Tester ensures that the fix resolves the failure.
▌Exit Criteria: Set of conditions, agreed upon with the
stakeholders for permitting a process to be officially completed
▌Test Basis: All documents from which requirements of a
component can be inferred
▌Test Monitoring and Control: Management task involving
periodically checking the status of project and applying
corrective actions
▌Test Plan: Document specifying scope, approach, resources and
schedule of intended test activities

21 © NEC Corporation 2015 NEC Group Internal Use Only


Fundamental Test Process

22 © NEC Corporation 2015 NEC Group Internal Use Only


Test Planning and Control

▌Test Planning is the activity of defining the objectives and


specifications of test activities in order to meet objectives and
mission.

▌Test Control is the activity of comparing actual progress against


plan and reporting status and deviations.

▌Involves taking actions necessary to meet the mission and


objectives of project.

23 © NEC Corporation 2015 NEC Group Internal Use Only


Test Analysis and Design

▌Test objectives are transformed into tangible test conditions and


test cases

▌Involves:
 Reviewing Test Basis
 Prioritizing test conditions based on analysis of test items
 Designing and prioritizing high level test cases
 Designing test environment setup and identifying any required
tools
 Creating bi directional traceability between test basis and test
cases

24 © NEC Corporation 2015 NEC Group Internal Use Only


Test Implementation and Execution

▌Includes finalizing, implementing, prioritizing, environment


preparation and execution of test cases

▌Tasks involves:
 Creating test data and writing automated test scripts
 Verify test environment has been setup correctly
 Updating bi-directional traceability between test basis and test
cases
 Execution and logging outcome of test execution and comparing
actual with expected result
 Reporting discrepancies as incidents
 Repeating test activities as a result of action taken for each
discrepancy

25 © NEC Corporation 2015 NEC Group Internal Use Only


Evaluating Exit Criteria and Reporting

▌Includes assessment of test execution against defined objectives

▌Tasks involves:
 Checking Test logs against Exit criteria
 Assessing if more tests are needed of if exit criteria specified
should be changed
 Writing a test summary report for stakeholders

26 © NEC Corporation 2015 NEC Group Internal Use Only


Test Closure Activities

▌Involves collection of data from completed test activities to


consolidate experience, test-ware, facts and numbers

▌Tasks involves:
 Checking all planned deliverables have been delivered
 Documenting acceptance of system
 Analyzing lessons learnt to determine changes needed for future
release and projects
 Using information gathered to improve test maturity

27 © NEC Corporation 2015 NEC Group Internal Use Only


Degree of Independence

28 © NEC Corporation 2015 NEC Group Internal Use Only


Quiz Time

What is Testing?

Define Error, Fault, Bug, Defect,


Failure

Why is Regression Testing Required?

Specify 7 principles of Testing.

Define 5 stages of Fundamental Test


Process
29 © NEC Corporation 2015 NEC Group Internal Use Only
Key Learnings Summarization

Flash cards

30 © NEC Corporation 2015 NEC Group Internal Use Only


Part2: Testing throughout
the software life cycle

31 © NEC Corporation 2015 NEC Group Internal Use Only


V-Model

32 © NEC Corporation 2015 NEC Group Internal Use Only


V-Model: Test Levels

▌Component testing: Searches for defects in and verifies the


functioning of software components (e.g. modules, programs,
objects, classes etc.) that are separately testable.

▌Integration testing: Tests interfaces between components,


interactions to different parts of a system such as an operating
system, file system and hard ware or interfaces between systems.

▌System testing: Concerned with the behaviour of the whole


system/product as defined by the scope of a development project
or product. The main focus of system testing is verification
against specified requirements.

▌Acceptance testing: Validation testing with respect to user


needs, requirements, and business processes conducted to
determine whether or not to accept the system.

33 © NEC Corporation 2015 NEC Group Internal Use Only


Component/Unit/Module Testing (1/2)

▌Searches for defects in, and verifies the functioning of software


(e.g. modules, programs, objects, classes, etc.) that are
separately testable

▌Occurs with access to the code being tested with the help of test
framework or debugging tool

▌Stubs and drivers are used to replace the missing software and
simulate the interface between the software components in a
simple manner

▌Involves programmer who wrote the code

34 © NEC Corporation 2015 NEC Group Internal Use Only


Component/Unit/Module Testing (2/2)

▌Test Basis includes:

 Component requirement

 Detailed design

 Code

35 © NEC Corporation 2015 NEC Group Internal Use Only


Integration Testing

▌Tests interfaces between components and interfaces between


system

▌There may be more than one level of Integration Testing:

 Component Integration testing tests interaction between s/w


components and is done after component testing
 System Integration Testing tests interaction between different
systems and may be done after system Testing

Test Basis includes:


 Software and System Design
 Architecture
 Workflows
 Use cases

36 © NEC Corporation 2015 NEC Group Internal Use Only


Integration Testing: Top Down vs. Bottom-Up

37 © NEC Corporation 2015 NEC Group Internal Use Only


Big-Bang vs. Incremental Integration Testing

▌Big-Bang Integration Testing


 All components or systems are integrated simultaneously,
after which everything is tested as a whole
 Less time taking activity
 Ideal for short releases
 Debugging and bug fixing can be complex

▌Incremental Integration testing


 All programs are integrated one by one, and a test is carried
out after each step
 Slower process
 Suitable for longer releases
 Debugging and bug fixing is simple

38 © NEC Corporation 2015 NEC Group Internal Use Only


System testing

The process of testing


an integrated system
to verify that it meets
specified
requirements

 Concerned with the behaviour of whole system/product


 Test environment should correspond to final target or production environment
 Includes tests based on risk or requirement specification, business processes, etc
 System testing should investigate both functional and non-functional
requirements of the system.
 Testers may also need to deal with incomplete or undocumented requirements.
 System testing starts by using the most appropriate specification-based (black-
box) techniques for the aspect of the system to be tested.

39 © NEC Corporation 2015 NEC Group Internal Use Only


Acceptance Testing

▌The goal of acceptance testing is to establish confidence in the


system, part of the system or specific non-functional
characteristics, e.g. usability, of the system

▌Test Basis includes:


 User Requirements
 System Requirements
 Use Cases
 Business Processes
 Risk analysis report

40 © NEC Corporation 2015 NEC Group Internal Use Only


Forms of Acceptance Testing (1/2)

Operational Contract acceptance


acceptance test is testing is performed
acceptance of the against a contract's
User acceptance test acceptance criteria for
system by system
focuses mainly on the producing custom-
administrators like
functionality thereby developed software.
testing of
validating the fitness-
backup/restore, Regulation
for-use of the system
disaster recovery, Acceptance testing is
by the business user
maintenance tasks, performed against any
check of security regulations like
vulnerabilties government, legal, etc

41 © NEC Corporation 2015 NEC Group Internal Use Only


Forms of Acceptance Testing (2/2)

▌COTS(Commercial off the shelf) software often wants to get


feedback from potential or existing customers before the product
is put up for sale commercially

▌Alpha testing is performed at developing organization’s site but


not by developing team

▌Beta Testing(Field Testing) is performed by customers at


their own locations

42 © NEC Corporation 2015 NEC Group Internal Use Only


Types of testing

Functional (Black-Box)Testing Non-Functional Testing

Test of what the system does.


Testing the attributes of a component or
Focused on suitability,
system that do not relate to
interoperability, security, accuracy
functionality, e.g. reliability, efficiency,
and compliance.
usability, maintainability and portability

Structural (White-Box/ Glass-Box)


Testing

Testing based on an analysis of the internal


structure of the component or
System.
Best used after specification-based technique.
Most often used as a way of measuring the
thoroughness of testing through assessment of
coverage.

43 © NEC Corporation 2015 NEC Group Internal Use Only


Confirmation and Regression Testing

Confirmation Testing (Re-


Testing)
 Tester raises defect
Maintenance Testing
 Developer fixes the defect  When a system is
 Tester executes same test operational, environment is
to ensure that defect is often corrected or changed
indeed fixed  Testing performed during
this phase is called
Maintenance Testing

Regression testing:
Purpose is to verify that
modifications in the software
or the environment have not
caused unintended adverse
side effects

44 © NEC Corporation 2015 NEC Group Internal Use Only


Quiz Time

Name different Test Levels

Why is Regression Testing Required?

Specify Non functional quality


characteristics

What is Alpha and Beta Testing?

45 © NEC Corporation 2015 NEC Group Internal Use Only


Part3:Static Techniques

46 © NEC Corporation 2015 NEC Group Internal Use Only


Static Testing

“Testing of a software development artefact, e.g., requirements,


design or code, without execution of these artefacts, e.g.,
reviews or static analysis”

▌Reviews are a way of testing software work product and can be


done a manual activity

▌Benefits include early defect detection and correction,


development productivity improvements, reduced testing cost
and time, fewer defects in later stages

▌Defects easier to find in reviews:


 Deviation from standards
 Requirement Defects
 Design Defects
Incorrect interface specifications
47 © NEC Corporation 2015 NEC Group Internal Use Only
Review Process: Phases (1/3)

48 © NEC Corporation 2015 NEC Group Internal Use Only


Review Process: Phases (2/3)

▌Planning: Starts with a 'request for review' by the author to the


moderator. A moderator is often assigned to take care of the
scheduling (dates, time, place and invitation) of the review. Also,
checks entry criteria for the document

▌Kick-Off: Reviewers receive a short introduction on the


objectives of the review and the documents. Also, to get
everybody on the same wavelength and control time that needs
to be spent on reviewing

▌Preparation: The participants work individually on the


document under review using the related documents,
procedures, rules and checklists provided. Also, potential defects,
questions and comments are noted

49 © NEC Corporation 2015 NEC Group Internal Use Only


Review Process: Phases (3/3)

▌Review meeting: The meeting typically consists of the following


elements (partly depending on the review type): logging phase,
discussion phase and decision phase. Also, severity of the defect
is specified as critical, major or minor

▌Rework: Based on the defects detected, the author will improve


the document under review step by step. Not every defect that is
found leads to rework. It is the author's responsibility to judge if
a defect has to be fixed.

▌Follow Up: The moderator is responsible for ensuring that


satisfactory actions have been taken on all (logged) defects,
process improvement suggestions and change requests.

50 © NEC Corporation 2015 NEC Group Internal Use Only


Roles and Responsibilities

▌Manager: Decides execution of reviews and allocates time in


project schedules

▌Moderator: Leads the review. Responsible for planning the


review, running the meeting and following up after the meeting

▌Author: Person responsible for document to be reviewed

▌Reviewers: Individuals who after necessary preparation identify


and describe findings

▌Scribe: Document all issues, problems and open points identified


during meeting

51 © NEC Corporation 2015 NEC Group Internal Use Only


Review Types (1/3)

52 © NEC Corporation 2015 NEC Group Internal Use Only


Review Types (2/3)

53 © NEC Corporation 2015 NEC Group Internal Use Only


Review Type: Informal Review(3/3)

▌Involves:

 No formal process
 May take the form of pair programming or technical lead
reviewing designs and code
 Results may be documented
 Main Purpose: Inexpensive way to get some benefit

54 © NEC Corporation 2015 NEC Group Internal Use Only


Static Analysis by Tools (1/2)

▌Objective is to find defects in software source code without


actually executing the software and locate defects that are hard
to find in dynamic testing

▌Static analysis tools analyze program code (e.g. control flow and
data flow)

▌Typically used by developers before and during component and


integration testing.

▌Value is:
 Early detection prior to test execution
 Early warning about suspicious aspects of code by calculation of
metrics
 Improve maintainability of code and design
 Prevention of defects
55 © NEC Corporation 2015 NEC Group Internal Use Only
Static Analysis by Tools (2/2)

▌Typical defects discovered by static analysis:

 Referencing a variable with an undefined value


 Unreachable/dead code
 Missing and erroneous logic
 Overly complicated constructs
 Programming standard violations
 Variables that are not used or improperly declared
 Security Vulnerabilities

56 © NEC Corporation 2015 NEC Group Internal Use Only


Quiz Time

Name the types of review

What is the difference between


walkthrough and inspection

Specify different phases of review

Specify some defects discovered by


static analysis

57 © NEC Corporation 2015 NEC Group Internal Use Only


Key Learnings Summarization

Flash cards

58 © NEC Corporation 2015 NEC Group Internal Use Only


Part4:Test Design
Techniques

59 © NEC Corporation 2015 NEC Group Internal Use Only


Test Analysis: Identifying Test conditions

Test analysis is the process of looking at something that can be


used to derive test information.

This basis for the tests is called the 'test basis'. It could be a
system requirement, a technical specification, the code itself (for
structural testing), or a business process. Sometimes tests can
be based on an experienced user's knowledge of the system,
which may not be documented.

From a testing perspective, we look at the test basis in order to


see what could be tested - these are the test conditions. A test
condition is simply something that we could test.

60 © NEC Corporation 2015 NEC Group Internal Use Only


Test Design: Specifying Test cases

Test Cases: should be


Test Conditions: can be
specific and may or may
vague and cover a large
not traverse multiple
range of possibilities.
conditions at once.

Once a given input value has been chosen, the tester needs to
determine what the expected result of entering that input
would be and document it as part of the test case.

In addition to the expected results, the test case also specifies


the environment and other things that must be in place before
the test can be run (the preconditions) and any things that
should apply after the test completes (the postconditions).

61 © NEC Corporation 2015 NEC Group Internal Use Only


Test Implementation: Specifying Test Procedures

▌The next step is to group the test cases in a sensible way for
executing them and to specify the sequential steps that need to
be done to run the test.

▌The document that specifies the executable order of the tests is


called a test procedure, and is often also referred to as a test
script.

▌For example, a test may create a new customer record, amend


that newly created record and then delete it. These tests need to
be run in the correct order, or they won't test what they are
meant to test.

62 © NEC Corporation 2015 NEC Group Internal Use Only


Testing Techniques

63 © NEC Corporation 2015 NEC Group Internal Use Only


Specification-based (black-box) testing techniques

▌Equivalence Partitioning

 The idea behind the technique is to divide (i.e. to partition) a


set of test conditions into groups or sets that can be considered
the same (i.e. the system should handle them equivalently),
hence 'equivalence partitioning

▌Example:
Assume that the application accepts an integer in the range 100
to 999.
 Valid Equivalence Classes: 100 to 999 inclusive
 Non-valid Equivalence Class partitions: less than 100, more
than 999, decimal numbers and alphabets/non-numeric
characters.

64 © NEC Corporation 2015 NEC Group Internal Use Only


Specification-based (black-box) testing techniques

▌Boundary Value Analysis

 Boundary value analysis (BVA) is based on testing at the


boundaries between partitions. We have both valid boundaries (in
the valid partitions) and invalid boundaries (in the invalid partitions).

▌Example:

Assume postal rates for 'light letters' are: $0.25 up to 10 grams;


$0.35 up to 50 grams; $0.45 up to 75 grams; $0.55 up to 100
grams. Which test inputs (in grams) would be selected using
boundary value analysis?
a. 0,9,19,49,50,74,75, 99,100
b. 10,50,75,100,250,1000
c. 0,1,10,11,50,51,75,76,100,101
d. 25,26,35,36,45,46,55,56
65 © NEC Corporation 2015 NEC Group Internal Use Only
Specification-based (black-box) testing techniques

▌State Transition Testing

 Allows the tester to view the software in terms of state,


transition between states, input/event that triggers state
change(transitions) and actions resulting from those transitions

66 © NEC Corporation 2015 NEC Group Internal Use Only


Specification-based (black-box) testing techniques

▌Example:

Specify minimum series of valid transitions to cover every state.

a. SS-S1-S2-S4-S1-S3-ES
b. SS-S1-S2-S3-S4-ES
c. SS-S1-S2-S4-S1-S3-S4-S1-S3-ES
d. SS-S1-S4-S2-S1-S3-ES

67 © NEC Corporation 2015 NEC Group Internal Use Only


Specification-based (black-box) testing techniques

▌Decision Tables Testing


 A decision table is a good way to deal with combinations of
things (e.g. inputs) and to capture system requirements. Also,
referred to as 'cause-effect' table.

Decision table contains the triggering conditions, often


combination of true and false for all input conditions

68 © NEC Corporation 2015 NEC Group Internal Use Only


Specification-based (black-box) testing techniques

▌Use Case Testing

 Use case describes interactions between actors(users or


system)

 Use case testing is a technique that helps us identify test cases


that exercise the whole system on a transaction by transaction
basis from start to finish.

69 © NEC Corporation 2015 NEC Group Internal Use Only


Structure-Based or White-Box Techniques

Structure-based test design techniques are a good way of generating


additional test cases that are different from existing tests. They can help
ensure more breadth of testing, in the sense that test cases that achieve
100% coverage in any measure will be exercising all parts of the
software from the point of view of the items being covered.

 Statement Coverage
 Patch Coverage
 Statement Coverage

Coverage= Number of Coverage items exercised x 100%


Total Number of Coverage Items

70 © NEC Corporation 2015 NEC Group Internal Use Only


Structure-Based or White-Box Techniques

▌Statement Coverage and Statement Testing


It is the assessment of percentage of executable statements that
have been exercised by test suite

▌Decision Coverage and Decision Testing


It is the assessment of percentage of decision outcomes that have
been exercised by test case suite

For Statement coverage, path should cover all nodes ( )


while for Decision coverage, path should cover all edges ( )

SC of 100% does not guarantee DC of 100%,


but DC of 100% guarantee 100% SC

71 © NEC Corporation 2015 NEC Group Internal Use Only


Structure-Based or White-Box Techniques

▌If x=3 then


Display message X
if y=2 then
Display message Y
else
Display message Z
Else
Display message Z

Calculate Statement Coverage and Decision Coverage

Decision Coverage: 3
Statement Coverage: 3
72 © NEC Corporation 2015 NEC Group Internal Use Only
Experience-Based Techniques

▌ Error Guessing:
 Success of error depends on skill of a tester.
 Enumerate a list of possible defects and to design tests that
attack these defects (Also called Fault Attack)
 List can be built based on experience, available defect and
failure data.

▌ Exploratory Testing:
 Testers are involved in minimum planning and maximum
execution.
 Involves concurrent test design and test execution
 Most useful when there are inadequate specifications and
severe time pressure

73 © NEC Corporation 2015 NEC Group Internal Use Only


Part5:Test Management

74 © NEC Corporation 2015 NEC Group Internal Use Only


Test Organization and Independence

▌Benefits of independence includes:


 Independent testers see other and different defects and are
unbiased
 An independent tester can verify assumptions people make
during specification and implementation

▌Drawbacks include:
 Isolation from development team
 Developers may lose a sense of responsibility for quality
 Independent testers may be blamed for delays in release

75 © NEC Corporation 2015 NEC Group Internal Use Only


Tasks for Test Leader and Tester

▌Also called test Manager or test coordinator.


▌Tasks include:
 Coordinate and write a test strategy for the project
 Plan the tests-including selecting test approaches, estimating
time, effort and cost of testing
 Initiate the specification, preparation, implementation and
execution of tests
 Introduce suitable metrics for measuring test progress

▌Tasks for tester includes:


 Create and review tests developed by others
 Create test specifications, setup environment and acquire data
 Implement tests, execute and log the tests, evaluate results and
document deviations/incidents
 Automate tests and measure performance of component

76 © NEC Corporation 2015 NEC Group Internal Use Only


Test Planning and Estimation

▌Test Planning
Is a continuous activity and is performed in all life cycle processes
and activities

▌Tasks includes
 Determine the scope, risks and identify objectives of testing
 Determining overall approach of testing, including Entry and
Exit criteria
 Making decisions about what to test, how test activities should
be done and scheduling test analysis and design activities

Entry Criteria: The set of generic and specific conditions for


permitting a process to go forward with a defined task
Exit Criteria: The set of generic and specific conditions, agreed
upon with the stakeholders for permitting a process to be
officially completed
77 © NEC Corporation 2015 NEC Group Internal Use Only
Test Planning and Estimation

▌Test Estimation

Approaches include:

Metric Based Approach: Estimating testing effort


based on metrics of similar project

Expert Based Approach: Estimating the task


based on estimates made by owner of the task
or by experts

78 © NEC Corporation 2015 NEC Group Internal Use Only


Test Monitoring, Reporting and Control

▌Test Progress Monitoring


Purpose is to provide feedback and visibility of test activities for:
 Percentage of test completion
 Defect information

▌Test Reporting
About effectively communicating our findings to other project
stakeholders
 Test summary report is created for the same

▌Test Control
About guiding and corrective actions to try to achieve the best
possible outcome for the project. Ex:
 Performance testing is normally run on weekday evenings during
off-hours in the production environment.

79 © NEC Corporation 2015 NEC Group Internal Use Only


Risk and Testing

▌Risk is defined as chance of event, hazard, event or situation


resulting in undesirable consequences or a potential problem

▌Project Risks
Risks that surround the project’s capability to deliver its objective
 Organizational Factors
a. Skill, training and staff shortage
b. Problems with tester communicating test results

 Technical issues
a. Problems in defining right requirements
b. Test environment not ready on time

 Supplier issues
a. Failure of third party

80 © NEC Corporation 2015 NEC Group Internal Use Only


Risk and Testing

▌Product Risks
Potential failures in the software or system are called Product risks

▌These include:
 Failure prone software delivered
 Potential that software/hardware could cause harm to company
 Poor software characteristics, data integrity and quality

81 © NEC Corporation 2015 NEC Group Internal Use Only


Incident Management

▌ Incident Report is:


A document reporting on any event that occurred, e.g. during the
testing, which requires investigation. Objectives include:
 Provide developers and other parties with feedback about
identification of problem
 Provide test leaders means of tracking the quality of the system

▌ Must include:
 Summary
 Incident description (inputs, expected results, actual results,
anomalies, date and time, procedure step, environment,
attempts to repeat, testers and observers)
 Impact

82 © NEC Corporation 2015 NEC Group Internal Use Only


Part6:Tools support for
Testing

83 © NEC Corporation 2015 NEC Group Internal Use Only


Tool support for Testing

▌Test Tools can have many purposes like:


 Improve efficiency by automating repetitive tasks
 Supporting manual activities like test planning, test design, test
reporting and monitoring
 Automate activities that can be done manually like performance
testing

▌Tools can be used for many activities like:


 Directly used such as test execution tools, test data generation
tools and result comparison tools
 Managing test process like manage test results, data,
requirement, incidents, reports, etc. and for reporting and
monitoring test execution

▌It can be any tool that aids in testing, e.g. spreadsheet

84 © NEC Corporation 2015 NEC Group Internal Use Only


Tool support for Test Management (1/2)

▌Test Management Tools


 Management of tests
 Scheduling of tests to be executed
 Traceability of tests and logging test results
 Preparing progress reports and tracking defects

▌Requirement Management Tools


 Storing requirement statements
 Identifying undefined, missing or 'to be defined later'
requirements
 Prioritizing requirements for testing purposes
 Checking consistency of requirements

85 © NEC Corporation 2015 NEC Group Internal Use Only


Tool support for Test Management (2/2)

▌Incident Management Tools


 Also known as defect tracking tool, defect-management tool, bug
tracking tool or bug-management tool
 Storing information about the attributes of incidents, i.e. defects,
failures, change requests and manages life cycles of incidents
 Store status (e.g. open, rejected, duplicate, deferred, closed)

▌Configuration Management Tools


 Storing information about versions and builds of the software and
testware
 Traceability between software and testware and different versions
or variants
 Access control (checking in and out).
 Keeping track of which versions belong with which configurations

86 © NEC Corporation 2015 NEC Group Internal Use Only


Tool support for Static testing

▌Provide a cost effective way of finding more defects at an earlier


stage

▌Review Tools:
Storing and sorting review comments
 Communicating comments to relevant people
 Keeping track of comments
 Repository for rules, procedures and checklist to be used

▌Static Analysis Tools:


 Used by developers to find defects prior to dynamic testing
 Calculates metrics such as cyclomatic complexity
 Enforce encoding standards
 Identify anomalies in the code

87 © NEC Corporation 2015 NEC Group Internal Use Only


Tool support for Test Specification

▌Test Design Tools


 Used to generate test inputs from requirements, code, test
conditions
 Generating Expected results if test oracle is present

▌Test Data Preparation Tools


 Enable data to be selected from an existing database or created,
generated, manipulated and edited to be used
 Construct a large number of similar records for volume tests

88 © NEC Corporation 2015 NEC Group Internal Use Only


Tool support for Test Execution and Logging

▌Test Execution tools (Ex: Selenium)


 Also known as 'capture/playback' tools, 'capture/replay' tools or
'record/playback' tools
 Use a scripting language to drive the tool
 Closely tied to the flow and interface presented by the GUI.

▌Test harness/Unit Test Framework


 Provides stubs and drivers
 Supplies inputs and receives outputs generated by the software
 Recording result of each test
 Support for debugging

89 © NEC Corporation 2015 NEC Group Internal Use Only


Tool support for Test Execution and Logging

▌Test Comparators
Determine differences between files, databases or test results
 Dynamic comparison of events that occur during test execution

▌Security tools
 Identifying viruses
 Detecting intrusions such as denial of service attacks
 Simulating various types of external attacks
 Probing for open ports or other externally visible points of attack
 Identifying weaknesses in password files and passwords

▌Coverage Measurement Tools


 Measure the percentage of specific types of code structure (e.g.
statement, branch or decision

90 © NEC Corporation 2015 NEC Group Internal Use Only


Tool support for Performance and Monitoring

▌Dynamic Analysis Tools


 Detects memory leaks
 Identifying Null pointers
 Useful in component and component integration testing

▌Performance/Load/Stress testing Tools


 Generating load and measuring time of specific transactions
 Producing graphs or charts of responses over time

▌Monitoring Tools
 Keeps track of the status of the system
 Analyze, verify and report on usage of specific system resources
 Monitoring network traffic and sends alert

91 © NEC Corporation 2015 NEC Group Internal Use Only


Benefits and Risks of Using Tool

92 © NEC Corporation 2015 NEC Group Internal Use Only


Introducing a Tool to an Organization

▌Important considerations:

 Organization’s maturity and readiness


 Identify areas where tool support will improve testing process
 Evaluation of tools against clear requirements and objective
criteria
 Evaluation of the vendor (training, support and other
commercial aspects) or open-source network of support;
 Identifying and planning internal implementation (including
coaching and mentoring for those new to the use of the tool).
 Estimation of cost benefit ratio

93 © NEC Corporation 2015 NEC Group Internal Use Only


Thank You

94 © NEC Corporation 2015 NEC Group Internal Use Only

You might also like