Agenda:

Test Execution

1
 Test Execution

Test Execution Objectives:

1. Definition:

– The processing of a test case by the software under test, producing

an outcome.

2. Purpose :

– To execute the appropriate collections of tests required to evaluate

product quality.

– To capture test results that facilitate ongoing assessment of the

product .

2
 Test Execution

Execution Considerations:

– Execute tests which are deemed highest risks first.

– Tests on which there are many dependent tests will be executed first.

– Test cases central to the architecture.

– Test execution on number of Operating system, browsers, servers

etc.

– Execution Manual or Automated?

3
 Test Execution

Execution Activities:

1. Set up Test platforms.

2. Identify Test cycle strategy.

3. Execute

 Executing the Unit Test

 Executing the Integration Test

 Executing the System Test

4
 Test Execution

Execution Activities (Contd…):

1. Set up test environment :

1. Hardware

2. Software

3. Tools

4. Data

2. Test Cycle Strategy:

1. Number of test cycles as per Test Plan.

5
 Test Execution

Execute Unit Tests:

1. Execute using tools or manually:

1. Execute tests individually

2. Execute a suite

2. Executed by Developer.

3. Defects are not reported in Defect Tracking tool.

4. Entrance Criteria:

1. The unit is compiled successfully.

2. Code has gone through Code inspection process.

3. Unit Test Environment is ready.

6
 Test Execution

Execute Integration Tests:

1. The most effective method for validating successful integration is to:

1. Test the client components.

2. Test the server components.

3. Test the network.

4. Integrate the client, server, and network.

2. Entrance Criteria:

1. Unit testing of the components is complete.

2. No open defect exists in the unit.

7
 Test Execution

Execute System Tests:

Entrance Criteria:
• Unit & Integration Testing must be successful.

• Set up system test environment, mirroring the planned production

environment as closely as possible.

• Test Cases must be Reviewed & Approved Successfully.

• Build must be deployed in the Test Environment.

• Identify test cycles needed to replicate production where batch processing is

involved.

• Assign test cases to test cycles.

• Execute the tests.

8
 Test Execution

Test Execution Process:

Raise Participate Complete

Execute each Send the
Mark Status defects for in Defect the test
of the test daily status
as Pass/Fail the failed Triage cycle execution of
step in test report to
in HP ALM test cases in and explain all the test
case Test Lead
HP ALM the defects cases

9
 Test Execution

Test Execution Process Contd…:

• Once all Test cases are approved and the test environment is ready for testing,
tester conduct smoke/sanity testing to confirm whether the build is testable or
not.

• Each Tester is assigned Test cases directly in HP ALM.

• Testers to ensure necessary access to the testing environment, HP ALM for

updating test status and raise defects.

• If any issues, will be escalated to the Test Lead and in turn to the Project
Manager as escalation.

• If any showstopper during Smoke/Sanity testing will be escalated to the

respective development team for fixes.

10
 Test Execution

Test Execution Process Contd…:

• Tester will prepare a Run chart with day-wise execution details.

• If any failures, defect will be raised as per severity guidelines in HP ALM tool
detailing steps to simulate along with screenshots if appropriate.

• Daily Test execution status as well as Defect status will be reported to all
stakeholders.

• Testing team will participate in defect triage meetings in order to ensure all test
cases are executed with either pass/fail category.

• If there are any defects that are not part of steps but could be outside the test
steps, such defects need to be captured in HP ALM and map it against the test
case level or at the specific step that issue was encountered after confirming
with Test Lead.

11
 Test Execution
Test Execution Process Contd…:
• This process is repeated until all test cases are executed fully with Pass/Fail
status.

• During the subsequent cycle, any defects fixed applied will be tested and results
will be updated in HP ALM during the cycle.

Note:

• The defects identified will be documented in a defects report template or in a

bug reporting tool, and then we report the same to developers.

• Once we receive a modified build from development team, we conduct smoke

test, re-test and regression testing.

• If any new-defects identified again we document and report the same to

developers. This process will continue until all test cases are passed and all
reported defects are fixed and closed.
12
 Test Execution
Review - When to Stop Testing:
Test Manager will consider the following factors:

– Deadlines, e.g. release deadlines, testing deadlines;

– Test cases completed with certain percentage passed;

– Test budget has been depleted;

– Coverage of code, functionality or requirements reaches a specified point;

– Bug rate falls below a certain level; or

– Beta or alpha testing period ends

13
 Test Execution

Concerns in Test Execution:

1. Software is not in a testable mode for this test level.

2. There is inadequate time and resources.

3. Significant problems will not be uncovered during testing.

14
 Test Execution
Exit Criteria for System Testing:

Technical
Exit Criteria Test Team Notes
Team

100% Test Cases & Scripts executed

95% pass rate of Test Cases & Scripts

No open Critical and High severity defects

95% of Medium severity defects have been closed

All remaining defects are either cancelled or documented as

Change Requests for a future release

All expected and actual results are captured and documented

with the test script

All test metrics collected based on reports from HP ALM

All defects logged in HP ALM

Test Closure Memo completed and signed off

Test environment cleanup completed and a new back up of the

environment

15
 Test Execution-Review Questions

1. Exit criteria of Unit Testing is the Entry criteria for the Integration testing.

1. TRUE

2. FALSE

2. All the functionalities are implemented and all the ‘open’ defects are
addressed in the Unit Testing, will indicate ___________.

1. Exit criteria of Integration Testing

2. Exit criteria of System Testing

3. Exit criteria of UAT

4. Exit Criteria of Unit Testing

16
Question and Answer

17