Test Planning and Strategizing

Risks, Assumptions, Issues and

Dependencies (RAID) – Introduction
LEVEL – SPECIALIST

Press F11 to view this course in Full Screen

 Learning Objectives

At the end of the session, you will be able to:

Describe RAID

List down the factors influencing risks in Testing Projects

Explain different categories of Risks

Explain Risk Management

Indicates the topic in discussion

Indicates topics to be covered
Indicates topics covered

2
 Do You Know

• Do you know the risks associated with test planning?

• Are you aware of common issues in each phase and dependencies with other systems and interfaces?
• Can you list down the various assumptions, issues and dependencies for all test phases?
• Can you classify issues into the severity levels and categories with respect to test elements?

3
 Learning Objectives 1 of 4

At the end of the session, you will be able to:

Describe RAID

List down the factors influencing risks in Testing Projects

Explain different categories of Risks

Explain Risk Management

4
 RAID in Test Planning & Strategizing-
Introduction
RAID stands for Risks, Assumptions, Issues and Dependencies

Risks

• Any specific event which might occur and thus has a negative impact on a project or program.
• Have an associated probability of occurrence along with an impact if it does materialize.

Assumptions

• Belief that is set as true to enable the team to proceed with the project or program.
• Considered during Test Strategizing, Planning and estimation phases.

Issues

• It is a topic/subject/question which arises in the project and has to be dealt with in order to ensure the
project runs smoothly.
• Exist as a problem today where as Risks might turn into issues in the future.

Dependencies

• Exists when an output from one piece of work or project is needed as mandatory input for another project
or piece of work

5
 RAID in Test Planning & Strategizing-
Examples
Risks
Example:
Absence of a test
configuration management
tool for test artifacts
management is a significant
risk in the project since the
SIT and UAT testing results
will be inconsistent due to
frequent test design
changes and modification of
regression test suites that
are done for each build
Inference:
Risk Management process
should be put in place to
handle and mitigate all
testing related risks
6
Assumptions
Example:
A Test Plan is produced by
assuming the availability of
skilled functional testers and
automation testers for a
testing project throughout
the entire duration.
However, if this assumption
turns out to be false then
testing is severely impacted
Inference:
Assumptions can turn out to be
false. Hence it is the
responsibility of testing team
to monitor, manage all
assumptions for minimal
impact to the project
Issues
Example:
Absence of a key skilled
resource for a particular
module during test execution
(and unlikely to attend duties
for prolonged period of time)
and no other back up
identified for the same is an
example of an issue
Inference:
Issues can occur during any
phase of software test life
cycle, it is very important to
track, monitor and action on
them. Issues should be
managed through Issue
Management Process
Dependencies
Example:
Unavailability of test data on
time for testing business
critical integration related
test scenarios since the
project team had a
dependency on a third party
to produce the test data for
the system that they own
and they have failed to
deliver on time.
Inference:
Managing inter-dependencies
is critical to ensuring projects,
regardless of their size, run
smoothly. Dependencies
should be recorded,
monitored, and managed
effectively
 Learning Objectives 2 of 4

Describe RAID.

List down the factors influencing risks in Testing Projects

Explain different categories of Risks

Explain Risk Management

7
 RAID – Factors influencing risks
Factors influencing risks are: Timeline, Supplier, Organization and Specialists

• In a testing project, the software to be tested is not supplied to test execution at the
planned point of time. Also, the timeline available for Testing will be very less.
Timeline
• There should always be a plan for prioritizing tests, alternative orders of testing, parallel
testing, use of more platforms, third parties, or even testing after delivery

• Failure of a third party who should supply deliverable, i.e. software modules,

Supplier components, sub-systems or libraries, or tools for development or testing.

• There may also be problem with supply of third party testing services, contractual
issues, wrong or no deliverable at too high costs

8
 RAID – Factors influencing risks (Contd..)

• Skill and Staff shortages, lack of specialists

Organizational • Personal and training issues - availability of the wrong people without adequate knowledge
• Political issues - problems to define the right requirements, or problems for the testers to
communicate their needs and test results, failure to follow up on low level testing and reviews

• Feasibility i.e. the extent that requirements can be met given existing constraints and the
quality of the design, development and test team.
Specialist
• Low quality teams may produce low quality software, which increases the cost of testing. Low
quality test teams do not find the defects, resulting in increased risk after testing

9
 Common Risks in STLC Phases
Test Analysis
• Delay in finalizing business
requirements/ functional
requirements/ system
requirements
• Frequent scope change i.e.
addition of new
requirements/modification
of existing requirements /
removal of existing
requirements
• Baselined SRS, HLD, LLD
are not available to start
the test design
10
Test Design
• Delay in receiving responses
from business teams for queries
and clarifications
• Unavailability of Test
Configuration Management Tool,
Testing Tools, Defect
management Tools
• Unavailability of skilled QA
testers for specific test design
• Third party systems inputs not
received on time and integration
touch points not known
• Baselined Test Strategy & Test
Plan
Test Execution
• Delay in receiving sign-off on
test strategy/ test case, test
environment set-up, data
refresh in test environment,
deployment of code from
development to test
environment, fixing defects
detected by QA teams will
impact the overall progress/
planned end date
• Delivery of poor quality code
resulting in high count of
defects (than planned)
• Overall project or design or
development team delay
Test Closure
• Open high severity and
high priority defects
• No resolution plan for
deferred defects
• Incomplete testing due to
schedule shrinkages or
frequent changes or
budget challenges
• Delay in providing sign-off
on test execution results or
testing artifacts
 Learning Objectives 3 of 4

At the end of the session, you will be able to:

Describe RAID

List down the factors influencing risks in Testing Projects

Explain different categories of Risks

Explain Risk Management

11
 Categories of Testing Risks

Test Risk Category Test Risk Cause

Testing Schedule Risks • Incorrect test estimations

• Test Resources not tracked properly
• Failure to identify complex functionalities and time required to develop them
• Unexpected project scope extensions
Testing Budget Risks • Incorrect test budget estimation
• Testing cost overruns
• Project scope extensions
Testing Technical Risks • Continuous changing requirements
• Complex requirements from implementation perspective
• Challenges in integration and testing

12
 Categories of Testing Risks

Test Risk Category Test Risk Cause

Testing Operational • Failure to address priority conflicts in testing
Risks • Incorrect testing roles and responsibilities
• Insufficient testing resources
• Lack of testing training
• Poor test resource planning
• Inadequate communication
Testing Program Risks • Testing budget constraints
• Market changes
• Changing customer priorities
• Changes in Government rules and regulations

13
 Examples of Testing Risks
Category Example
Client Management Failure to satisfy testable user requirements/testing expectations
Project & Program Inexperienced test manager/ test leader, testing complexity
Management
Quality & Process Alignment to strategic testing goals / vision
Risk of schedule overrun Chance of missing the schedule target of production move
Risk of missed Requirement Traceability Matrix is not updated with new Change Requests received in the
Functionality/Rework project. This runs the risk of testing missed functionality in the final product
Risk of application not Performance testing of the End to End application could not be conducted due to the
meeting NFRs limitations in test environment. This may lead to the risk of application not meeting NFRs
Risk related to lost Formal report on problem root cause analysis and permanent resolution are not being
opportunities for documented and tracked. This leads inadequate tracking of problems and permanent
process improvement resolutions and thus loss of opportunities for process improvement
Risk of Customer NFRs have not been well defined in this project which may lead to scalability and
dissatisfaction performance related issues during implementation. Thus this runs the risk of customer
dissatisfaction. Risks related to CSAT feedback and implementation at project/account level
Risk related to Project Testing team’s current work volume is more than existing team strength. It might affect
planning quality of work in the long run

14
 Examples of Testing Risks
Category Example
Risk related to The testing team has started with the test design activity but no KT was given to the team.
unsuccessful transition KT Effectiveness not tracked or very low that will impact the test coverage
Risk of Integration Issues Environments and hardware’s as required for integration testing is not yet available
although the project plan shows it has reached the schedule. This will lead to integration
issues and subsequent schedule slippage in the project
Risk of low Productivity Project is delivering through multiple builds and releases. UAT of some release shows very
high defect leakage to client. No defect prevention activity is evidenced in the
project. Thus high rework effort spent to fix bugs will lead to low productivity of the team
as actual effort spent will be much higher than estimated
Risks related Resourcing Risks related to inadequate testing resources planning, skill gap analysis, resource back up
etc. for testing projects
Risks related to Risks with low profitability, wrong pricing models, Penalty clauses etc
Commercial/Financial
Risk related to Client Risks related to test status reporting, test governance, escalation management, client
Engagement communication etc
Legal/Contractual risks Assumptions / Out of scope are not clearly documented in the Project Scope document,
SOW. This may lead to conflicts in expectations during the test execution phase /
acceptance

15
 Learning Objectives 4 of 4

At the end of the session, you will be able to:

Describe RAID.

List down the factors influencing risks in Testing Projects

Explain different categories of Risks

Explain Risk Management

16
 Testing Risks Management- Methodology

• Testing risks methodology is created by identifying the following elements:

Testing Risks Management

Testing Scope Testing Risks Framework
Responsibilities

Testing Scope

Testing Issues
Casual Unknown Testing Events Unknown Testing Events Management
Analysis Known Impact Un-Known Impact Process

Certainty Known Testing Events Known Testing Events Testing Risks

Known Impact Un-Known Impact Management

17
 Testing Risks Management- Framework
• Testing risks management framework paradigm shows the different activities composing project risk management.

• Since communication is the key factor which cut across all these 5 phases, is positioned centrally because it is both the
conduit through which all information flows and, often, is the major obstacle to risk management. In essence, from
this framework, a project may structure a risk management practice best fittingRiskinto its projectsurfaces
Identification management structure.
risks before they
become problems and adversely affect testing.
Corrects for deviations from
planned risk actions. Once
Risks Techniques can be used for surfacing risks by the
testing risk metrics and
Identification application of a disciplined and systematic
triggering events have been
process to raise concerns and issues for
chosen, there is nothing
subsequent analysis.
unique about risk control.

It relies on test processes to Analysis is the conversion of

Tracking consists of Risk Control Risks Analysis
control risk action plans, testing risks related data into risk
monitoring the status and
correct for variations from
actions taken to ameliorate Communicate decision making information.
plans, respond to triggering
risks. Planning It
turns testing
events provides therisks
basis for the Test
information
Manager to work and
into decisions actions
on the right
Appropriate risk metrics are
risks
identified and monitored to It involves developing actions to
enable the evaluation of Risks address individual risks, prioritizing
the status of risks Risk Planning risk actions, and creating an integrated
Tracking
themselves and of risk risk management plan
mitigation plans.

18
 Testing Risks Management - Responsibilities

• Risks management is the responsibility of Test Lead and Test Manager. However, all project stakeholders should
participate in the risk identification and analysis process.

• Overall the extended project team carries out risk management and mitigation activities

Testing Risks Management Activities Responsible Party

Overall direction of testing risks management plan Test Manager / Test Leader
Plan development and execution of testing risk management Test Manager / Test Leader
plan using tools (Ex: Risk Portal) to automate the process of
risks
Management

Provide counsel and assistance Business Analyst, Development Team,

regarding testing risks identification / assessment / analysis / Delivery Assurance
handling

Risk Watch List Test Manager / Test Leader

Preparation and issuance of risk Test Manager / Test Leader
reporting

19
 Testing Risks Management Process –High
Level View
High level overview of testing risks management process:

Continuous Risk Management Process Input Phases

Methodology

Proposal
Proposal
Identify/
Revisit Risk Tracking via Risk Portal
https://Riskportal
risk
areas

Diligence
Due Diligence
risks
project risks
Report Assess Risk Priority Number (RPN) Classification
the risk the risk

of project

Due
Risk Management Council meeting &

Identification of
action plan

Identification

Transition
Transition
Monitor
and Control
Evaluate risk
risk
Risk Mitigation

state
Steady state
Steady
Weekly Risk Report

20
 Testing Risks Management Process – High
Level View
Activities that are involved in each stage of risk life cycle process are:
Risk reporting
Identifying/Revisiting Risk Monitoring Report the Risk
Risk Assessment Risk Control and Evaluation
Risk Areas

 Identification of Risk  Risk Assessment  Review the  Monitor the Tracking and
by the delivery team by the delivery Effectiveness of the actions being taken Reporting of
team. Risk Mitigation to mitigate the risk. Risks with high
Categorize the Risk plan. RPN.
Quantify the  Review
 Identify the process magnitude of Risk  Review Actions mitigation action
area being impacted by via Risk Priority taken on Risks with plan if not
the risk Number (RPN). high RPN effective.
values(>500)
Logging of the Risk in  Plan for mitigation  Re-Evaluate risks
the Risk portal or Risk of Risk Review Actions post mitigation
Log taken on aging plan
 Plan for Closure of Risks.
the Risk Closure of Risks.

Risk could be RPN is a product of Risk Management Council Risks with high RPN Weekly Email for High
Operational, Severity, Probability & (RMC) is a high level (>500) and Aging RPN & Aging risks to
Technological or Detect ability as rated by management team formed to (>100) are monitored be sent to all stake
Financial in nature the delivery team assess High priority & Aging holders
Risks.
21
 Testing Risks Management – Tools and
Techniques
Failure Model – Effect and Analysis (FMEA) - It is a technique used to identify, prioritize & eliminate failures from
system or process before they reach customer

Failure Model Effect Analysis

(Fail to Intent, (Root Cause Consequence (Mitigation means: What are we going to do?
Meaning

Failure to perform certain function
-Immediate Consequence
-Cumulative Consequence)
-1st Defense : Avoid / Eliminate Failure causes
-2nd Defense : Identify / Detect Failure earlier
-3rd Defense : Reduce Impact / consequence
Risk Assessment: RPN Calculation
Action Plans: How, When & Who will mitigate)

Phase Question Output

Identify  What can go Wrong? Failure Descriptions:
Causes  Failure Modes  Effects
3 Phases of FMEA

Analyze  How likely is the Failure? Risk Priority Number (RPN)

 What are the Consequences = Occurrence * Severity * Detection

Act  What can be done?  Design Solutions

 How can we eliminate the  Create Test Plans
cause?  Design Process Changes
 How can we reduce the  Error Proofing etc
severity?

22
 Testing Risks Management – Tools and
Techniques
Brainstorming, Root Cause Identification, Interviewing and Diagramming Techniques are other risk
management techniques.

•Employed as a general data-gathering and creativity technique which identifies

risks, ideas, or solutions to issues.
Brainstorming
•Uses a group of team members or subject-matter experts spring boarding off each
others' ideas, to generate new ideas.

•Technique for identifying essential causes of risk.

•Using data from an actual risk event, enables you to find out what happened and
Root cause identification
how it happened, and understand why it happened, so that you can devise
responses to prevent recurrences.

23
 Testing Risks Management – Tools and
Techniques

Face-to-face meeting comprised of project participants, stakeholders, subject-

matter experts, and individuals who may have participated in similar, past projects
Interviewing
is a technique for gaining first-hand information about and benefit of others'
experience and knowledge.

• Cause and effect diagrams - Cause and effect diagrams or fishbone diagrams
are used for identifying causes of risk
• System or process flow charts - Flow charts illustrate how elements and
Diagramming techniques processes interrelate.
• Influence diagrams - Influence diagrams depict causal influences, time
ordering of events and other relationships between input variables and
output variables.

24
 Summary

The key points covered in this session are:

 Risks: A risk is any specific event which might occur and thus has a negative impact on a project or program.
 Assumptions: An assumption is a belief that is set as true to enable the team to proceed with the project or
program.
 Issues: An issue is a topic/subject/question which arises in the project and has to be dealt with in order to
ensure the project runs smoothly.
 Dependencies: A dependency exists when an output from one piece of work or project is needed as mandatory
input for another project or piece of work
 Timeline crunch, Dependency on Third Party, Lack of training/ process / Specialists are the different factors
which influence the risks
 Common Testing risks occur in different phases of STLC such as Test Analysis, Test Design, Test Execution and
Test Closure
 Schedule, Budget, Operational, Technical and Program Risks are broad categories of Risks

25
 Summary

 Testing risks methodology is created by identifying Testing Scope, Testing Risks Framework and Testing Risks
Management Responsibilities
 Risks Identification, Risks Analysis, Risk Planning, Risks Tracking and Risk Control are different activities of Risks
Management Framework
 Risks management is the responsibility of Test Lead and Test Manager. However, all project stakeholders should
participate in the risk identification and analysis process. Identify/Revisit Risk areas, Assess the Risk, Monitor and
Evaluate risk, Report the Risk are important stages of Risk Management Process.
 Risk Tracking, RPN classification, Action Plan, Mitigation and weekly report are the different methodologies of
Risk Management
 FMEA - It is a technique used to identify, prioritize & eliminate failures from system or process before they reach
customer
 Brainstorming, Root cause identification, Interviewing, Diagramming techniques are Other Risk Management
Techniques

26
Test Planning and Strategizing

Risks, Assumptions, Issues and Dependencies

(RAID) – Introduction

Thank You