Performance Testing Framework

Author(s): Rajendra Guthala (97822)

Date written (MM/DD/YY): 08/10/10

Declaration

I/We hereby declare that this document is based on my/our personal experiences
and/or experiences of my/our project members. To the best of my/our knowledge,
this document does not contain any material that infringes the copyrights of any
other individual or organization including the customers of Infosys.

Project Details

·         Project(s) involved: Starbucks Coffee

·         H/W Platform: Windows XP

·         S/W Environment: Load Runner, Oracle Applications, Quality Center

·         Application Type: Oracle Applications

·         Project Type: Performance Testing

Target readers: Performance Testing Team and Load Runner Users

Keywords: Performance Testing, Monitoring.

Introduction: This BOK explains process to implementation of Performance

Testing. Also contains details of different tools to measure and control the
performance of the application.

Index:

I. Introduction -1
II. Performance Testing Overview -2
III. Performance Testing Process -3
IV. Performance Tools -4
V. Roles and Responsibilities of Performance Test Team -5
VI. Conclusion and Reference(s) -6

1
Performance Testing Overview:

 Performance Test is the use of software to simulate the load on applications to

determine the speed, scalability, and stability characteristics of a system

 With clear performance test objectives, performance test provides input data to
make sound business decisions such as system/software/hardware/database
configuration

 Different performance test types provides various performance data on a given

system such as validation of the system’s ability to handle the production load
without degradation

 With well defined performance SLAs, Performance Test can reduce costly
performance related production issues by tuning and optimizing the system
before production deployment

Types of Performance Test:

 Baseline Performance Test: Focuses on measuring response times of single

user transactions or stand alone transaction at a % of a full load. Single user tests
will be repeated for select critical business transactions in order to get a baseline
for load testing.

 Load test: Validates the system to load conditions that resemble a live production
environment as closely as possible. It emulates current user or transaction loads
while monitoring the behavior of the various application infrastructure tiers, such
as application servers, database servers or user interface response time. The test
data can be bulk-loaded using a production database snapshot or can be
generated through scripting. Parameterization is used to automatically test
variables such as transaction response time, number of hits per time unit, CPU
utilization, or memory usage.

 Stress/Endurance test: Extends the focus of the load test. It determines the
point at which the application or underlying infrastructure no longer meets
required service levels (Scalability Testing) or ceases to function (Breakpoint
Testing). It includes the validation of concurrent user or system activities

2
Performance Testing Process:

System Analysis/Determining the Load

Scenarios

Identifying Performance Test Scope

Build Performance SLA and Data

Requirements

Secure Performance Environment and

Build Schedule

Develop Base Line Scripts and Develop

Performance Scenarios

Set Performance Monitors and

Determine frequency of Execution

Execute Individual Performance Test

and Execute ETE Performance Test

Collate and Analyze the Results and

Report

3
Performance Tools:
There are large numbers of Performance Testing tools available in the market. We
need to choose the suitable and compatible Test tool based on the requirement and beyond
the project budget.

Example: Load Runner from HP, Rational Performance Tester from IBM, Silk Performer from
Segue etc...

Monitoring Tools:

Monitoring tools should be select based on the environment/technology of the

application. Also we have different tools for Database and Network monitoring.

Examples:

DBTuner – monitoring Database statistics

Stats pack - monitoring Database statistics

Native tools available for Unix OS monitoring

Wily intrascope – Application server monitoring

Orion Network Performance Monitor-Network Monitoring

Performance Test Execution:

The following activities are involved in performance test execution:

 Single User run to validate the test environment and Test Scripts

 Dry run with more no. of Virtual Users before actual Load Test

 Run Actual Load Test under Performance circumstances

 Execute the Performance Scripts for Baseline, Load and Stress Test

Performance Tuning:

 Analyze the report output and identify the issues

 Eliminate the bottlenecks and degradation points to build an optimal System

 Tune the Database and system parameters and code components to meet SLA.

 Co-ordinate with the development people and Functional people to discuss and
resolve open issues.

4
Roles and Responsibilities of Performance Test Team:

Performance Test Lead:

 Define Performance Test Scope: Identify and determine the Performance Test
Scope.

 Build and Approve Business Profile: Build and approve Business profile to be
simulated and measured during performance test.

 Secure Performance Test Environment: Coordinate securing the dedicated

environment for performance test.

 Build Performance Test Schedule: Build and maintain schedules of performance

test Script development, execution and reporting.

 Build Performance Report Summary: Build and provide end to end performance
test results summary.

Performance Tester:

 Collect performance SLAs and Requirements: Collect performance

requirements and SLAs for all business scenarios. Prepare necessary templates
to collect the data and SLA information.

 Develop performance Scripts and Scenarios: Develop performance scripts

based on manual scripts and functional flow of the business scenario. Also
correlate and parameterize the scripts. Develop performance scenarios based
on the business profile using pacing, think time and no.of iterations with
appropriate monitors.

 Execute Tests: Execute individual performance tests and End to End

performance tests. Also execute the scenarios by simulating the transactions
and users with respect to business profile.

 Report Test Results: Build performance test results report by analyzing the test
results individually corresponding to SLA of each scenario.

Functional SME:

 Provide the performance area of concern based on the business criticality and
known issues.

 Provide the functional support to ensure performance scripts are developed to

simulate appropriate business process to measure the actual performance.

 Review the performance test results and provide inputs on potential

optimization to improve and meet the performance SLAs.

5
Technical SME:

 Ensure appropriate performance monitoring tools and configurations are in

place for systems under test. Provide the server information including names,
port number, and bandwidth etc. to be configured in performance test
scenarios.

 Optimize and tune code, data base, network etc. for the system

Conclusion:

Performance testing approach will be varying from project to project. The process
and reporting explained above is the generic approach for any kind of Performance
Testing Project. I have been discussed only the high level information which can be
helpful to get basic idea of how to do performance testing. The selection of
Performance Tool and Monitoring tool are depends on the type of the project and
usage.

Reference:

 Project experience on different environments

 Performance Test Planning Experience in Current project
 Other Performance Approach and Guidance documents available in different
websites.