Agenda

● Performance testing
•Load Testing
•StressTesting
● What is JMeter
● JMeter basic elements
● Building a Script
●Running a Script
●Recording a Script (Bootstrapping)
Sam Varadarajan, for IEJUG
svaradar@gmail.com
Performance Testing
What is Performance testing?
Performance testing is performed to determine how a system performs in terms of
responsiveness and stability under a particular workload. It is typically non-functional,
meaning, we only care about load on the system, scalability etc. Actual functionality is
not tested in this phase. (That will be done in Functional Test phase).

The real goal is to find that maximum load that the system can handle without
performance degradation or crashes.

There are several types of performance testing, but 2 are important:

• Load/Capacity Testing
• Stress Testing

We will be focusing mainly on Load Testing in this presentation.

Load / Capacity Testing
•Load testing is the simplest form of performance
testing. A load test is usually conducted to
understand the behavior of the system under a
specific (expected) sustained load.

•The load can be the expected concurrent number

of users on the application performing a specific
number of transactions within the set duration.

•In simple terms, it is the maximum load the horse

can carry without it tripping or the carriage
toppling!

•During the tests we will collect the the response

times of all the important business critical
transactions. If the database, application server,
etc. are also monitored, then this simple test can
itself point towards any bottlenecks in the
application software.
Stress Testing
•Every system has a capacity limit. When the load
goes beyond the limit, the System starts responding
very slowly and even produce errors. we have seen
this in Databases, Network etc.

•It is more visible on a Web site, as number of users

can drastically (often unpredictably) increase,
depending on content. We saw this recently with
HealthCare.gov! Sometimes, websites crash due to
attacks such as denial of service attack.

•In simple terms, Stress on the system is that last

straw on the camel's back that broke it.

•The purpose of the stress testing is to find the

capacity limit of the system. We do this by increasing
the load on the system beyond a safe limit, so the
system will eventually crash. With the findings, we
can adjust the parameters to stretch this limit.
 General Load Testing Script
Load Testing itself can broken down into following steps:

•Create Test (Can be recorded using some tool)

•Configure workload settings
•Add measurement hooks
•Data drive script (using variables and parameters to generalize it)
•Add logging
•Generate reports/charts from performance data collected.
Load Testing - Manually
This is how they did load testing before.
Just ak a bunch of users to come to a
training room. The users will start using
the application according to a script and
run simultaneously!
We probably generated a very weak load
and didn't really stress it either!
 Load Testing Automated
•All or most of these steps can be automated using several different
programs/scripts.
•There are testing tools that have integrated these into tools called Load Testing
tools.
•Mercury Load Runner and Rational Rose tools are commercial products that have
been around for a long time.
•With the advent of Web applications, several new tools have appeared in both
commercial and open source arena. Following slide shows a list of Load testing
tools available.
Load Testing Tools
Tool Name Company Name Notes

Apache JMeter An Apache Jakarta open source project Java desktop application for load testing and performance measurement.

BlazeMeter BlazeMeter Ltd. BlazeMeter is a JMeter compatible, self-service, load testing platform.

Blitz Spirent Communications Blitz is a service for load and performance testing of websites, mobile, web apps
and REST APIs in the cloud.

Gatling Open Source JVM application with scenarios as code and portable HTML reports.

Loader.io SendGrid Labs Cloud based load testing service

Load Impact Load Impact - AB Cloud based performance testing SaaS tool.

LoadRunner HP Performance testing tool primarily used Licensed.

Load Test (included with Parasoft Performance testing tool

Soatest)

loadUI SmartBear Software Cross-platform load testing tool

Login VSI Login VSI, Inc. Performance testing software for Windows based virtualized desktops. Licensed.

NeoLoad Neotys Load testing tool for web and mobile applications.

OpenSTA Open System Testing Architecture Open source web load/stress testing application, licensed under the Gnu GPL.

Rational Performance Tester IBM Eclipse based large scale performance testing tool

Silk Performer Borland Application performance tool with cloud and local virtual agents. Licensed

Test Studio Telerik Test Studio is a load testing tool

Visual StudioUltimate edition Microsoft Visual Studio Ultimate edition includes a load test tool

WebLOAD RadView Load testing tool for web and mobile applications Licensed.[3]

(Courtesy: Wikipedia)
JMeter
 What is JMeter?
•JMeter is a free, open-source performance measurement tool
written in Java

•JMeter is built by the reputed Apache project community

• JMeter is available as a desktop Java application and can be run

in GUI and batch modes. It can also be run in distributed modes.

•It has a completely extensible architecture. It allows us to

develops various elements that can be reused in different parts
of a test plan. It also allows plugins to be developed, so the tool
can be further extended.
Why JMeter?
 It is an Open source tool.
 Can load and performance test many different server types:
 Web - HTTP, HTTPS
 SOAP
 Database via JDBC
 LDAP
 JMS
 Mail - POP3(S) and IMAP(S)

 User friendly GUI Design. Can be run in batch and distributed modes to handle more load.
 Full multithreading framework allows concurrent sampling by many threads and simultaneous
sampling of different functions by separate thread groups.
 Can mimick Web browser functionality including Caching and Cookies.
 Controllers are configurable and can monitor the server performance.
 Test results can be captured in various format like summary report, Graph, Aggregate report,
Aggregate graph, Results in tree and Results in Table. The results can be stored in files for later
review as well.
 Extensible architecture. Supports plugins, bean shell etc to add more functionality.
Installing Jmeter
• Install Java (1.5 or higher)
• Download JMeter http://jmeter.apache.org/download_jmeter.cgi
• Add path of java installation in environment path variables.
JMeter Test Plan
•A test plan describes a series of steps
JMeter will execute when run.

•A complete test plan will consist of one

or more Thread Groups, logic conrollers,
sample generating controllers, listeners,
timers, assertions, and configuration
elements.
Elements of a Test Plan
The Test Plan is the wrapper for the entire test suite. It wraps around
the following elements in an hierarchical fashion.

•Thread Group
This is really the logic unit of a load test. There can be many thread
groups in a Test Plan.
–Setup number of thread
–Set up ramp up period
–No. of times test executes
•Controllers
–Sampler (Send Request to Server)
–Logical Controller (Customize logic to send request); includes
Loop Controller and One Time Controller.
•Listener
Help to capture and display results in various formats. For e.g.,
–Graph Result
–View Results Tree or Table format and many more.
•Timers
–Add a delay to the next request

Continued….
Elements of Test Plans
• Assertions
– Allow you to assert fact about responses received from HTTP request
• Configuration Elements
– Allow you configure settings
• Preprocessor
– Execute prior to sampler request
• Post Processor
– Execute some action after sampler request
 Order of Execution
1. Config Element
2. Pre-Processor
3. Timer
4. Sampler
5. Post-Processor
6. Assertion
7. Listener
Building a Test Plan
 Thread Group

•Thread group elements are the

beginning points of any test plan.

•All controllers and samplers must

be under a thread group.

•Each thread represents a user 

setting a thread group to 500
simulates 500 users.
Thread Group Contd...
Sampler
•Samplers tell JMeter to send requests to a server and wait for a response.
They are processed in the order they appear in the tree.
Sampler Contd...
Listener
•Listeners can be used to display (or store) performance information JMeter collects during tests.
Listener Contd...
Listener Contd...
Logic Controller
•Logic Controllers let you
customize the logic that
JMeter uses to decide
when to send requests.
•Logic Controllers can
change the order of
requests coming from their
child elements. They can
modify the requests
themselves, cause JMeter
to repeat requests, etc.
 Configuration Elements
•A configuration element
works closely with a
Sampler. Although it does
not send requests (except
for HTTP Proxy Server ), it
can add to or
modifyrequests.
Configuration Element
Contd...
Configuration Element
Contd...
Assertions
•Assertions allow you to assert facts about
responses received from the server being
tested.
• Using an assertion, you can essentially "test"
that your application is returning the results you
expect it to.
 Other Test Elements
● A Pre-Processor executes some action prior to a Sampler Request being made.

● A Post-Processor executes some action after aSampler Request has been made.

●A Timer will cause JMeter to delay a certain amount of time before each sampler
which is in its scope.
 Variables
JMeter allows
parameters,
properties and
variables to keep the
test scripts
generalized.
Syntax for Variable:
${VARIABLE}
• Variables can be
declared and used in
successive layers.
JMeter Test Plan Completed
 JMeter Test Plan Completed
• Once you completed, you can save the
test plan with a meaningful name.
• The script file generated is an XML file
and will be saved with a .JMX
extension.
• You can run this script using command
line mode as well, so you don't have to
load GUI.
Running a Test Plan
 Running in Batch Mode
On command line, in the JMeter\bin directory, you can type:

jmeter -n -t C:\sam\load_test\scripts\LT600_laweb2_test_syt6.jmx -j C:\sam\load_test\log\

jmeter_1212.overnight.log -l C:\sam\load_test\log\jmeter_1212.overnight.jtl

Where,
LT600_laweb2_test_syt6.jmx is the script I saved in the completion step above.

-n makes it non-GUI
-t name of the test script you want to use
-j for the JMeter.log (I change this to match my log file naming above)
-l for the JTL file - in XML format, has the same details we get in listeners above.
We may have some use for this, but for now I am just saving it.

You can put the above command inside a batch file to save typing!
Appendix
Non-Testing Elements
 Non-Testing Elements Contd...
HTTP(S) Script Recording Element
 Recording a Script
• Setup a non-test element, HTTP(S) Test
Script Recorder on the workbench.
• Setup all the inclusion, exclusion lists
and other properties.
• Click on Start.
Now, JMeter is running as a local proxy
server. If you adjust PC's proxy setting
to use this as the proxy server, all
requests will pass through JMeter.
 Recording a Script Contd...
Next you need to setup the browser to use our JMeter listener as a
local proxy
Recording a Script Contd...
 Recording a Script Contd...
• Now that we have captured all the
pages to be visited, we can clean it all
up. This is a good place to start.
• Parametrize and add variables
wherever possible and move the pages
(samplers) to a Thread Group and build
up our test plan.
•
Recording a script Contd...
An easier way is to use BadBoy software to capture the pages and export to
JMeter. Download and install Badboy software from
http://www.badboy.com.au/download/send?svId=124
Recording a Script Contd...
 Real life Test Scenario
• Our App Servers kept crashing on
heavy load days. We had to constantly
restart the servers, bringing down all the
web users along with it. After making
some changes to the application, had to
simulate the huge load on the servers to
test if it will sustain the next tax season.
After reviewing various tools, we chose
JMEter.
 Real life Test scenario
Contd...
• When we upgraded our infrastructure*
recently, we used JMEter to compare
old environment and new environment
for performance. We ran the same set
of scripts against old and new
environments to identify any
bottlenecks.

* We upgrade to PowerBuilder 12.5, Oracle to 11g, Windows servers to

Windows 2008.
 Real life Test scenario
Contd...
• Our Web application did an overkill of logging in the
database. We had a maximum concurrent user limit
set to 300 in Oracle. We wanted to increase that.
• To help with the tweaking, I ran JMeter scripts that
pushed the EAServers to connect to the database
continuously to make the database run out of
handles - sort of stressing the database to its limits.
We were able to tweak the setting and retest for the
load condition.
• Eventually, I refactored the code to reduce these
loggings. Of course, JMeter came to the rescue
again!
Demo
Q&A