Test Planning and Strategizing

Quality Gates

Level- Practitioner

Press F11 to view this course in Full Screen

 Learning Objectives

At the end of the session, you will be able to:

Describe Quality Gates in a software project.

List down the Quality Management Performance activities across SDLC phases.

Explain Quality Gates and Review checklists across Test Phases.

List down the steps to follow for Quality Gates Review.

Indicates the topic in discussion

Indicates topics to be covered
Indicates topics covered

2
 Do You Know

• Are you aware of the purpose of entry and exit criteria, suspension, redemption and acceptance criteria in

each phase?

• Do you know the importance of adhering to quality gates?

3
 Learning Objectives 1 of 4

At the end of the session, you will be able to:

Describe Quality Gates in a software project.

List down the Quality Management Performance activities across SDLC activities.

Explain Quality Gates and Review checklists across Test Phases.

List down the steps to follow for Quality Gates Review.

Indicates the topic in discussion

Indicates topics to be covered
Indicates topics covered

4
 What is a Quality Gate?

A Quality Gate is a..

• Special milestone in a software project, that is located before a phase which is strongly dependent on the outcome
of a previous phase. It is especially useful between phases in which breaches in disciplines must be overcome.
• It demands a set of documents and includes special requirements on these documents, both of which are detailed
in a checklist.
• Decision makers and domain experts are involved in:
o Arriving at project specific checklist
o Taking decision on further proceeding of the project, based on the inputs provided in the checklist.
• The Stage-gate model can also be used to manage multiple projects to identify the most promising projects and
cancel the risky projects. It is not limited to the domain of software development and can be used for any type

of new product development.

5
 Project Quality Gate

Project Quality Gate Involves:

• Formal assessment of the quality and integrity of the project deliverables.

• Formal checklists are used throughout the life of a project.

• Formal sign-off and acceptance occurs at each gate.

• Measure customer satisfaction throughout the project.

• Information is assured to be communicated to the correct stakeholders.

6
 Project Quality Gate-Purpose and Impact

The intention of Q-Gates is to ensure that when we reach significant milestones in the project we check the
Quality of the delivery.
Few of the key factors are:

• Are we on track?

• Are all deliverables complete?

• Are they fit for purpose?

• Do the Partner/Customer agree?

• Is risk managed?

• Can we start the next phase without delay?

7
 Voice of stakeholders

“Requirements and “Visibility into quality is “ Lack of automated static

Specifications lack proper known in testing phase only… code reviews and formal
elicitation, elaboration and No unified view of quality unit testing to measure
documentation“ metrics” code quality”

“Our customers want new

“We've tried quality and we
didn't see any results."
There's a lag time between
the investment and the
payoff, and it varies by
organizational size
“Ownership of Quality is
resident with Testing only”
“We need more
automation to bring down
effort without
compromising on coverage”
features, not better
quality." We need to
convince software vendors
that it's perfectly OK to
spend less time on features
and more time on quality

"In order to get the product out

“Wish I could secure
separate funding for Quality
aspects when projects
transition into functional”
“Wish there is a Unified in a timely manner, we must “We have tools in place to
approach to Quality across sacrifice some quality.“ assist us, but we don't
the organization to reduce "Software vendors would like to enforce their usage”
the effort downstream ” provide a quality product, but
they've got to get it out to
market before someone down
the street does,"

8
 Cost of Quality

Cost of quality is nothing but the cost to fix the defect.

Cost of Quality

Cost of conformance Cost of non-conformance

Costs associated with Consists of expenses surrounding

preventing and appraising the failure to meet customer
the quality of the product requirements

9
 Cost of Quality

Architecture
Requirements Development Testing Release
& Design

Quality
Quality Quality Quality Manager Quality
Manager Manager Manager Manager

Test Lead Test Lead Test Lead Test Lead Test Lead

Testers Testers
Cost to fix a bug 

Test Scenarios/ Defect Fix/Test Test Closure/

Test Test Strategy/Plan Cases/Scripts/data (Re)Execution reporting/Metrics
Requirements
1x 5x 10x 50x 100x

“ Designing and embedding Quality check parameters upstream reduces the overall
cost of Quality and also the testing effort expended per cycle”

10
 Learning Objectives 2 of 4

At the end of the session, you will be able to:

Describe Quality Gates in a software project.

List down the Quality Management Performance activities across SDLC phases.

Explain Quality Gates and Review checklists across Test Phases.

List down the steps to follow for Quality Gates Review.

Indicates the topic in discussion

Indicates topics to be covered
Indicates topics covered

11
 Quality Management Performance across SDLC

ANALYZE DESIGN DEVELOP TEST IMPLEMENT

QG
QG QG
QG QG
QG QG
QG QG
QG QG
QG
Automated Objective and
Requirements Implement Defect Perform Static and
Static Code quantitative measure of
Elicitation Prediction Models Dynamic Analysis
Analysis quality across lifecycle

Defect detection Continuous

Perform Modeling -functional Automated Unit Integration Testing
/Prototyping requirements & Testing with the
design application

Identify Evaluating Root cause analysis

Defect detection- Technical (SIT and UAT) and
Requirements
Defects non-functional Complexity Action Plan
requirements &
design
Prepare Test Plan, Traceability
Risk Prioritization
Estimation and Traceability (Requirements to
of Test Cases
testing Scope (Requirements to Test Cases)
Design)
Regression and
Automation Code Coverage
Profiling Review/Analysis
Traceability (Design
to Code)

Test Case
Optimization

12
 Learning Objectives 3 of 4

At the end of the session, you will be able to:

Describe Quality Gates in a software project.

List down the Quality Management Performance activities across SDLC phases.

Explain Quality Gates and Review checklists across Test Phases.

List down the steps to follow for Quality Gates Review.

Indicates the topic in discussion

Indicates topics to be covered
Indicates topics covered

13
 Quality Gates across Test Phases- Test Initiation

Test Initiation Test Planning Test Design Test Execution Test Closure

Entry Criteria Signed off scope and BRD documents are available

• The test manager/test lead should ensure at least the BRD(Business Requirement Document) is
available to draft the high-level test estimates and plan for testing resource requirements.
Key Activities • To commence the test planning phase, it is mandatory to have the scope finalised and signed off.
They have to review and document the impact of the changes to the BRD.
• The testing team should be in place and they should understand the BRD. They should facilitate any
training that may be needed from the business analyst.
• They should have clearly identified and shared the roles and responsibilities for the entire testing
team. The same should be documented in the Test Strategy.

Key Deliverables High Level Test Estimates

• Scope has been reviewed and finalised

Exit Criteria • Testable requirements and Non-functional requirements (if applicable) have been identified
• High level estimates have been prepared, reviewed and signed off
• Test team has been finalised

14
 Test Initiation Quality Gate Review Checklist

15
 Quality Gates across Test Phases-Test Planning

Test Initiation Test Planning Test Design Test Execution Test Closure

• Signed off functional specifications document is available

Entry Criteria • High level estimates have been prepared, reviewed and signed off

• Test Scenarios identified from Business Requirement Document(BRD) and Functional specifications( FS) documents
should be updated in the RTM. Further, based on the volume and complexity of the identified test scenarios, the
test estimates should be revised and reflected in the overall project estimates.
Key Activities • There should be separate environments earmarked for the testing phase
• The environment provision should be done prior to the test design phase to allow the Testing team to collect and create
the test data required during execution

• Test Strategy
Key Deliverables • Test Plan (if applicable)
• Detailed Test Estimates

• Various types of testing to be performed identified (functional, non-functional etc.)

• Test strategy and Test Plan (if applicable) documents have been prepared, reviewed and signed off
Exit Criteria • Automation and performance testing details have been captured as a part of the test plan (if applicable)
• Regression suite exists (if applicable)
• Environments for testing have been identified and made available
• Detailed test estimates have been prepared, reviewed and signed off
• Requirements traceability matrix has been initiated

16
 Test Planning Quality Gate Review Checklist

17
 Test Planning Quality Gate Review Checklist

18
 Test Planning Quality Gate Review Checklist

19
 Quality Gates across Test Phases-Test Design

Test Design –
Test Initiation Test Planning Scenario & Test Test Execution Test Closure
Cases

• Signed off Test Strategy, Test Plan (if applicable) and detailed test estimates are available
• Regression suite has been made available (if applicable)
Entry Criteria
• Test environment has been booked and made available

• Manual Test Cases for functional and non-functional testing (if applicable)
Key Deliverables • Automation Test Scripts (if applicable)
• Requirements Traceability Matrix

• Test-scenarios have been identified, prepared, reviewed and signed off by BA and test manager/test
lead
• Test cases are created in a standard template and have been reviewed and signed off
• Regressions suite has been finalized (if applicable)
• Test environments are ready with requisite test data
Exit Criteria
• If automation/performance testing are within scope, test scripts have been prepared, reviewed and
signed off
• Updated version of Requirements traceability matrix

20
 Test Design Quality Gate Review Checklist

*BRD – Business Requirements Document

*TSD – Test Scenario Document

21
 Test Design Quality Gate Review Checklist

22
 Test Design Quality Gate Review Checklist

23
 Quality Gates across Test Phases-Test Environment

Test environment decides the software and hardware conditions under which a work product is tested. Test environment set-up is one of the
critical aspects of testing process and can be done in parallel with Test Case Development Stage. Test team may not be involved in this activity if
the customer/development team provides it. In that case, they are required to do a readiness check of the given environment.

Test Initiation Test Planning Test Design - Test Closure

Environment Test Execution

• System Design and architecture documents are available

Entry Criteria • Environment set-up plan is available

• Understand the required architecture, environment set-up

• Prepare hardware and software requirement list
Key Activities • Finalize connectivity requirements and prepare environment setup checklist
• Setup test Environment and test data
• Perform smoke test on the build and Accept/reject the build depending on the test result

• Environment ready with test data set up 

Key Deliverables • Smoke Test Results.

• Environment setup is working as per the plan and checklist

Exit Criteria • Test data setup is complete
• Smoke test is successful

24
 Test Environment Quality Gates Review Checklist

25
 Quality Gates across Test Phases-Test Execution

Test Execution –
Test Initiation Test Planning Test Design Test Closure
ST and SIT

• Unit testing results

• Unit tested codebase with no severity 1 or severity 2 defects
Entry Criteria • Dedicated test environment for ST and SIT with test data loaded and ready
• Signed off ST and SIT test cases along with test data
• Signed off test plan that details coverage criteria

• ST and SIT Test Execution Log

Key Deliverables • ST and SIT Test Results and Reports
• ST and SIT Defect Log

• 100% of the planned test scenarios and test cases are successfully executed
• Test cases/scripts that are required for subsequent tests cycles, updated with missing / incorrect steps
Exit Criteria • No severity 1 or severity 2 defects are open
• Requirements traceability matrix updated

26
 Quality Gates across Test Phases-Regression

Test Execution -
Test Initiation Test Planning Test Design Test Closure
Regression

• All defect fixes have been deployed into test environment

• Regression test scenarios / test cases for the application have been developed and reviewed
Entry Criteria
• Test environment is stable for regression test execution with adequate test data for manual and
automated regression testing

• Regression Test Execution Log

• Regression Test Results and Reports
Key Deliverables • Regression Test Defect Log
• ST, SIT and Regression Test Execution Signoff
• Completed and signed off Test Summary Report

• 100% of planned test scenarios and test cases are successfully executed (for automated and manual)
• Requirements traceability matrix updated.
• No severity 1 or severity 2 defects are open
Exit Criteria • 100% of critical and high severity defects have been fixed, tested in development and retested in
regression test environment
• Updated automation scripts have been stored in the repository
• Test summary reports and metrics reports are completed and delivered

27
 Test Execution Quality Gate Review checklist

28
 Test Execution Quality Gate Review Checklist

29
 Test Execution Quality Gate Review Checklist

30
 Quality Gates across Test Phases-Test Closure

Test Initiation Test Planning Test Design Test Execution Test Closure

• Presence of unresolved defects accepted by the business analyst

• All planned ST, SIT and Regression Test cases have been executed
• Risks of unexecuted and failed test cases have been identified, documented and accepted by key
Entry Criteria
stakeholders
• ST, SIT and regression testing have been completed and signed-off
• Test summary report has been reviewed and signed off

• Test Summary Report

Key Deliverables • Test Closure Report
• Root Cause Analysis Report (if applicable)

• Test closure report has been prepared, reviewed and signed off
• Lessons learnt during the project have been documented
Exit Criteria • Root cause analysis for all defects has been performed
• Regression test cases and Automation test scripts have been updated (if applicable)

31
 Test Closure Quality Gate Review Checklist

32
 Test Closure Quality Gate Review Checklist

33
 Learning Objectives 4 of 4

At the end of the session, you will be able to:

Describe Quality Gates in a software project.

List down the Quality Management Performance activities across SDLC phases.

Explain Quality Gates and Review checklists across Test Phases.

List down the steps to follow for Quality Gates Review.

Indicates the topic in discussion

Indicates topics to be covered
Indicates topics covered

34
 Quality Gate - Readiness Review

The steps to be followed for review are:

1) Prepare Quality Gate scorecard along with milestone slip chart by indicating the project phase deliverables is according to project
plan and assures deliverables will be available for review.

2) Conduct meeting and present/ share the status of the project phase.

3) Refer project specific checklist while reviewing the artifacts.

4) Deliverable owners present deliveries according to the scorecard

5) Customers evaluate content and quality of each deliverable

6) Project quality Manager records result of the review in the scorecard and the deadlines and action owners for open points/tasks.
Sample as given below

1) Acceptance Level 1: Accepted

2) Acceptance Level 2: Conditionally accepted. Quality improvement to the deliverable is needed. A comment describes the next
steps depending on the estimated impact to the project.

3) Acceptance Level 3: Not available or not accepted. Fulfillment is needed. A comment describes the next steps, depending on
the estimated impact to the project.

35
 Quality Gate - Readiness Review

Once the project manager or Project quality Manager record the findings after the review,

• Capture customer satisfaction ratings to success measures identified in the Project

Management Plan and record actions for improvement.

• Summarize the minutes, action items and release decision taken for the Quality Gate

• After Quality Gate Review, verify that follow-up actions have been taken place,

provide scorecard to Phase Acceptance and Closure.

36
 Case Study

Client Background:

‘My bank’ was a leading bank in the U.S.A. They had a regulatory

requirement for which the development was done by Vendor1 and

QA by Vendor2.

Since it was a regulatory requirement, any delay in the project will

invite legal complications and loss of reputation to the Bank. So, it

was planned to be completed on time.

37
 Project Background

The QA team defined the below entry criteria for Test Execution phase

Entry criteria :

 Signed-off test cases

 Signed-off Test plan
 Environment available for execution

38
 Problem Statement 1

All entry criteria were met for execution phase and the QA team commenced the Test Execution as per the
schedule.

During Smoke testing, they found that the Test environment was not stable for Testing. This activity was
pending from the client side.

What should be the action item from the

Project Manager so as to complete the
testing on time without compromising the
quality?

39
 Solution

Project Manager conducted the quality gate meeting with the client and got into an agreement that:

 Testing would be suspended till the availability of suitable Test environment

 Date and Time of environment availability
 Identification and tracking of high risks on a hourly basis by the Project Manager
 Initiation of contingency plan by Project Manager to make up for the delay

40
 Problem Statement 2

During Test execution, the team found a critical bug which impacted lots
of test cases. They realized that it could have been detected at the unit
testing.

It was later discovered that the development team had not completed
the unit testing as planned and they were still testing that piece of code
which had the critical defect.

41
 Lessons Learnt

What was missed here?

Testing team did not include “Availability of signed-off unit test results” as one of the entry criteria
for test execution.

Lessons learnt

• Entry and exit criteria need to be clearly documented in test planning and strategizing
document.
• Phase wise Quality Gate review is important which will help to identify the critical factors that
may impact testing.

42
 Problem Statement 3

In Test Design Phase, the client SME reviewed and provided the comments on the test scenarios and test
cases delivered by the QA team. But he did not provide the sign off for the same. Due to project pressure,
the team commenced with the Test Execution as planned.

At the end of the execution phase, the client SME assigned a set of new cases to be tested by the team. The
scenarios were totally new and the team had not factored in any effort for testing those scenarios. Hence
they could not accommodate the additional test cases.

The SME insisted to execute those new test cases to verify the stability of the enhancement. The team had to
work additional hours and execute the test cases.

43
 Lessons Learnt

What went wrong?

Since the client SME has provided the review comments, the testing team assumed that it can be
considered as an approval for the Test Scenarios and cases. This is not correct. The right approach would
be to get an approval from all the involved stakeholders.

Lessons learnt

“Any artifact delivered by the testing team should be approved by stakeholders. Only in such cases, any
major review comment/change from the client at a later stage can be raised a Change request”.

44
 Summary

Key take a way's of the session are:

Quality Gate is a Special milestone in a software project which is located before a phase that is strongly dependent on

the outcome of a previous phase.

Project Quality Gate means formal assessment of the quality and integrity of the project deliverables

The intention of Q-Gates is to ensure that when we reach significant milestones in the project we should check the

Quality of the delivery

Cost of quality has two main components- the cost of conformance and the cost of non-conformance.

The cost to fix the error in the maintenance phase is 100 times more expensive than to fix in the design phase

45
 Summary (contd..)

 A Quality Gate is a collection of completion criteria and sufficiency standards representing satisfactory

execution of project plan phase.

 Each SDLC phase has certain Entry criteria, Exit Criteria, Key activities and Key Deliverables.

 Project quality Manager records result of the review in the scorecard and the deadlines and action owners

for open points/tasks.

 After Quality Gate Review, Project Quality Manager ensure and verify whether the follow-up actions have

been taken place and provide scorecard to Phase Acceptance and Closure.

46
 Test Your Understanding

1. “Availability of Release notes” is an entry criteria for which of the following phases

a) Test Design
b) Test Execution

c) Test closure

3
 Test Your Understanding

2. What will be the appropriate supporting evidence to be specified in a Quality Gate table for the exit
criteria “ All planned test cases have been executed” in Test Execution phase?

a) Defect Logs
b) Test Logs

c) Test Completion report

4
 Test Your Understanding

3. Say true or False

a) a. Q-Gates Dashboard will depict that whether the project is on track or any slippage.
b) b. All projects need to have defined quality gates which needs to be agreed by all
stakeholders
c) c. Early identification of defect increases the cost of quality.

5
Thank You!