D83171GC10 - SG - Oracle WebLogic Server 12c - Performance Tuning Workshop (Student Guide)

Oracle WebLogic Server 12 c:
Performance Tuning Workshop
Student Guide
D83171GC10
Edition 1.0
January 2014
D85092
Authors Copyright © 2014
, Oracle and/or its affiliates. All rights reserved.
Elio Bonazzi Disclaimer
Mark Lindros This document contains proprietary information and is protected by copyright and
other intellectual property laws. You may copy and print this document solely for your
own use in an Oracle training course. The document may not be modified or altered
Technical Contributors in any way. Except where your use constitutes "fair use" under copyright law, you
and Reviewers may not use, share, download, upload, copy, print, display, perform, reproduce,
publish, license, post, transmit, or distribute this document in whole or in part without
TJ Palazzolo the express authorization of Oracle.
Bill Bell
The information contained in this document is subject to change without notice. If you
Mathew Slingsby find any problems in the document, please report them in writing to: Oracle University,
500 Oracle Parkway, Redwood Shores, California 94065 USA. This document is not
Serge Moiseev warranted to be error-free.
Kevin Tate
Takyiu Liu Restricted Rights Notice
Jose Alvarez If this documentation is delivered to the United States Government or anyone using
the documentation on behalf of the United States Government, the following notice is
Ahmed Gaid applicable:
Colin Stevenson
U.S. GOVERNMENT RIGHTS
David Anderson The U.S. Government’s rights to use, modify, reproduce, release, perform, display, or
disclose these training materials are restricted by the terms of the applicable Oracle
Mark Scarton license agreement and/or the applicable U.S. Government contract.
Mike Williams
Trademark Notice
Diganta Choudhury
Eric Gross Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names
may be trademarks of their respective owners.
Editors
Malavika Jinka
Aju Kumar
Graphic Designer
Seema Bopaiah
Publishers
Sumesh Koshy
Giri Venugopal
Contents
1 Course Introduction
Course Objectives 1-2
Target Audience 1-3
Course Prerequisites 1-4
Introductions and Setting Expectations 1-5
Course Schedule 1-6
Course Practices 1-8
Classroom Guidelines 1-9
Facilities in Your Location 1-10
For More Information 1-11
Related Training 1-12
2 Performance Tuning: Overview

Objectives 2-2
Agenda 2-3
Performance Terminology 2-4
Scalability 2-5
Performance Testing 2-6
Maintaining Test Integrity 2-7
Benchmarking 2-8
Load and Stress Testing 2-9
Why Is Testing Important? 2-10
Performance Tuning Methodology 2-11
Performance Testing Concerns 2-12
Performance Testing Methodology Tips 2-13
Metrics 2-14
Results Reporting 2-15
Bottlenecks 2-16
CPU Bound 2-17
I/O Bound 2-18
Disk Bound 2-19
Network Bound 2-20
Database Bound 2-21
Section Summary 2-23
Practice 2-1 Overview: Setting Up the Practice Environment 2-24
iii
Practice 2-2 Overview: Exploring the Course Applications 2-25
Agenda 2-26
Load-Testing Tools 2-27
Choosing Testing Tools 2-28
The Grinder 2-29
The Grinder Architecture 2-30
The Grinder Console 2-32
The Grinder Summary Statistics 2-33
Monitoring Performanceby Using the Administration Console 2-34
Oracle WLS Monitoring Dashboard 2-36

Monitoring Dashboard: Views 2-37
Practice 2-3 Overview: Executing a Benchmark 2-39
Practice 2-4 Overview: Stress Testing an Application and Identifying
Bottlenecks 2-40
3 Tuning Operating System Resources

Objectives 3-2
Agenda 3-3
Processor Metrics 3-4
Memory Metrics 3-6
Network Interface Metrics 3-8
I/O Device Metrics 3-10

Agenda 3-13
What to Monitor in Operating System 3-14
CPU Utilization: What to Look For 3-15
CPU Usage in Linux Environment 3-17
sar Command 3-18
vmstat Command 3-19
iostat Command 3-20
netstat Command 3-21
System Monitor 3-22
Performance Tools and Commands 3-23
Monitoring Disk I/O 3-26
Swapping 3-28
inode Cache 3-29

Practice 3-1 Overview: Monitoring Linux CPU Utilization 3-31
Practice 3-2 Overview: Monitoring Linux Disk I/O 3-32
Practice 3-3 Overview: Monitoring Linux Virtual Memory and Network 3-33
iv
Agenda 3-34
File Handles 3-35
TCP/IP Parameters to Support a Large Number of Connections 3-36
Practice 3-4 Overview: Analyzing and Tuning Linux Kernel Parameters 3-40
4 Tuning HotSpot JVM

Objectives 4-2
Assumptions and Expectations 4-3
Agenda 4-4
What Is Performance? 4-5
Performance Focus for This Lesson 4-7
HotSpot JVM Monitoring Tools 4-8
Section Summary 4-9
Agenda 4-10
Garbage Collection Basics 4-11
Generational Garbage Collection 4-12
Garbage Collectors: Java Heap Options 4-14
Setting Heap Memory Size 4-15
Setting Heap Size Options 4-16
G1 Garbage Collector 4-18
GC Algorithms 4-20
GC Performance Goals 4-22
Focusing on Throughput 4-23

Focusing on Responsiveness 4-24
Evaluating GC Algorithm 4-25
GC Tuning Tips 4-26
Server-Class Machine Detection 4-30
Ergonomics: What It Does 4-31
Practice 4-1 Overview: Tuning JVM Garbage Collection 4-33
Agenda 4-34
Using jps 4-35
Using jcmd 4-37
Using jcmd: Examples 4-38
Using jcmd: Histogram 4-39
Using jinfo 4-40
Using jstat 4-41

Using jstack 4-42
jstack 4-43
v
Practice 4-2 Overview: Using JVM Command Line Tools 4-45
Agenda 4-46
Java Monitoring and Management Architecture 4-47
Java VisualVM 4-48
Java VisualVM Connections 4-49
Remote Monitoring 4-50
Java VisualVM Interface 4-51
Monitoring JVM 4-52
Monitoring Threads 4-53
Profiler Snapshot 4-54

Using VisualGC 4-55
Using jconsole 4-56
Using GCHisto 4-57
Practice 4-3 Overview: Using Java VisualVM 4-58
Practice 4-4 Overview: Using VisualGC, jconsole, and GCHisto 4-59
Mission Control 4-60
Java Discovery Protocol (JDP) 4-61
Mission Control: Architecture 4-62
JVM Browser 4-63
Mission Control: Management Console Overview Tab 4-64
General: Server Info 4-65
MBeans: Attributes 4-66
MBeans: Operations 4-67
MBeans: Notifications 4-68

MBeans: Metadata 4-69
MBeans: Triggers 4-70
Runtime: Server 4-71
Runtime: Memory 4-72
Runtime: Garbage Collection 4-73
Runtime: Memory Pools 4-74
Runtime: Threads 4-75
Best Practices 4-76
Practice 4-5 Overview: Using Mission Control 4-77
Flight Recorder 4-78
Flight Recorder: Benefits 4-79
Data Flow in Flight Recorder 4-80
Initiating Recording Using the Management Console 4-81
Tab Groups of Flight Recorder 4-82

Flight Recorder General Tab 4-83
Flight Recorder Memory: Overview Tab 4-84
Flight Recorder Memory: GC Collections Tab 4-85
vi
Flight Recorder Memory: Object Statistics 4-86
Practice 4-6 Overview: Using Flight Recorder 4-88
5 Monitor and Tune Server Performance

Objectives 5-2
Agenda 5-3
On-Demand Deployment 5-4
Enable Exalogic Optimizations 5-5
Monitor Channel Connections 5-6

Section Summary 5-7
Agenda 5-8
Stuck Threads 5-9
Configuring Stuck Threads Behavior 5-10
Parameters for Stuck Threads at the Server Level 5-12
Actions upon Stuck Threads Condition 5-13
Auto Restart of the Server 5-14
Practice 5-1 Overview: Configuring Server Stuck Thread Behavior 5-16
Practice 5-2 Overview: Configuring Automatic Restarting of a Failed Server 5-17
Practice 5-3 Overview: Configuring WebLogic Overload Protection 5-18
Agenda 5-19
Connection Backlog Buffering 5-20

Agenda 5-23
Secure Sockets Layer (SSL) 5-24
Agenda 5-26
Logging Considerations 5-27
Log Filters Review 5-28
Practice 5-4 Overview: Configuring Log Filters for WebLogic 5-30
6 Monitor and Tune Cluster Performance

Objective 6-2
Agenda 6-3
Cluster: Review 6-4
Benefits of Clustering 6-6

Basic (Single-Tier) Cluster Architecture 6-7
Multitier Cluster Architecture 6-8
Remote Calls and Deployment Packaging 6-9
vii
Architecture Advantages and Disadvantages 6-10
Cluster Communication 6-12
Practice 6-1 Overview: Configuring Multicast and Unicast Clustering 6-15
Agenda 6-16
A Cluster Proxy for a Web Application Cluster 6-17
Proxy Plug-Ins 6-18
Oracle HTTP Server (OHS) 6-19
Hardware Load Balancers 6-21

Agenda 6-24
HTTP Session Persistence 6-25
In-Memory Session Replication 6-26
File Session Persistence 6-27
JDBC Session Persistence 6-28
Oracle Coherence: Overview 6-29
Coherence*Web and WebLogic Clusters 6-30
Session Persistence Comparison 6-31
Practice 6-2 Overview: Tuning Session Persistence 6-33
7 Tuning JDBC Performance

Objectives 7-2
Agenda 7-3
Connection Pools 7-4
Connection Pool Sizing 7-5
Database Listener Timeouts 7-7
JDBC and Application Design 7-8
Section Summary 7-9
Agenda 7-10
Connection Testing 7-11
Shrink Frequency 7-13
Practice 7-1 Overview: Tuning JDBC Connection Pools 7-15
Agenda 7-16
Row Prefetch 7-17
Statement Caching 7-18
Statement Cache Tuning 7-20

Connection Pinned to Thread 7-21
Practice 7-2 Overview: Testing Performance of JDBC Statement Caching 7-23
viii
Practice 7-3 Overview: Testing Performance of Pinned to Thread 7-24
Practice 7-4 Overview: Configuring Batch Updates 7-25
Agenda 7-26
Combining Transactions 7-27
Batch Updates 7-28
Transaction Isolation Levels 7-29
Operations Permitted at Each Isolation Level 7-30
Transaction Isolation Levels 7-31
Agenda 7-33
Exalogic Optimizations 7-34
Agenda 7-36
Oracle Database Tuning 7-37
Practice 7-5 Overview: Tuning an Oracle Database for WebLogic 7-41
8 Configuring Work Managers

Objectives 8-2
Agenda 8-3
WebLogic Server Threads 8-4
Monitoring a Server Thread Pool 8-5
Monitoring Server Threads 8-6
Section Summary 8-7

Agenda 8-8
Work Managers 8-9
Default Work Manager 8-10
Work Manager Scope 8-11
Work Manager Architecture 8-12
Practice 8-1 Overview: Configuring and Using Work Managers 8-14
Practice 8-2 Overview: Configuring Work Manager Constraints 8-15
9 Tuning J2EE Applications

Objectives 9-2
Agenda 9-4
What Is a JavaServer Page? 9-5
JSP Request Processing 9-6

Precompiled JSP 9-7
The appc Application Compiler 9-8
Using the precompile Parameter 9-9
ix
Tuning JSP: Compilation Tuning 9-10
HttpSession Replication Tuning 9-11
keepgenerated Parameter 9-12
Page Check Interval 9-13
verbose Parameter 9-15
Servlet Reload Check Interval 9-16
Using Web Servers for Static Content 9-17
Session Timeout 9-18
Session Invalidation 9-19
Page Directive and HTTP Session 9-20

Using Custom JSP Tags 9-21
Using the WebLogic cache Tag 9-22
Web Application Tuning JSP and Servlet Output Buffer Tuning 9-23
Practice 9-1 Overview: Tuning JSP 9-25
Agenda 9-26
Enterprise JavaBeans 9-27
Stateless Session Beans 9-29
Stateless Session Beans Tuning 9-30
Pool Miss Ratio 9-32
Pool Management 9-33
Determining the Pool Size 9-34
Configuring a Stateless Session EJB Pool 9-35
Message-Driven Beans (MDBs) 9-36

Configuring an MDB Pool 9-37
Configuring to Use Batching with an MDB 9-38
MDB Tuning 9-39
Stateful Session Beans 9-41
Cache Management 9-42
What Happens When the Cache Fills Up? 9-44
Determining the Cache Size 9-45
Idle Timeout and Eligibility 9-46
Cache Type 9-47
Configuring a Stateful Session EJB Cache 9-49
Use Filtering ClassLoaders 9-50
General Application Performance Problems 9-52
EJBs: Coding for Performance 9-53

Practice 9-2 Overview: Tuning Stateless EJBs 9-55
Appendix A: Reference Slides
x
Course Introduction
Copyright © 2014, Oracle and/or its affili ates. All rights reserved.
Course Objectives
After completing this course, you should be able to:

• Use a standard tuning methodology
• Tune an operating system to support WebLogic
• Monitor and tune JVM performance
• Monitor and tune server performance
• Monitor and tune cluster performance
• Monitor and tune JDBC services
• Tune GridLink data sources
• Monitor and tune a persistent store
• Configure a work manager for an application
• Monitor and tune Java EE applications
In addition to these high-level terminal objectives, each lesson has a lower-level set of enabling
objectives.
Oracle WebLogic Server 12c: Performance Tuning Workshop 1 - 2

Target Audience
This course is intended for WebLogic Server administrators.
If you are concerned whether your background and experience match that of the target
audience, ask the instructor.

Course Prerequisites
Required
• Learners should have completed D80149GC10 Oracle
WebLogic Server 12c: Administration I or have equivalent
experience.
Suggested
• System administration experience

Introductions and Setting Expectations
Introduce yourself:
• Name
• Your company and role
• Your experience with WebLogic Server 12c and/or prior
releases
• Anything specific you are looking for in this class
This is a workshop. That means the

content is approximately 25%
focused on lecture and 75% focused
on hands-on practice.
Have everybody say a little about themselves and what they want from this course. This is a
great time to set student expectations as to what they will or will not experience in this course.
It is also a great way for instructors to know which topics are of the most interest to their
students.

Course Schedule
Day One
1. Course Introduction
2. Performance Tuning Methodology
3. Tuning Operating System Resources
4. Tuning HotSpot JVM
Day Two
5. Monitor and Tune Server Performance
6. Monitor and Tune Cluster Performance
7. Tuning JDBC Performance
This schedule is approximate and subject to change.

Course Schedule
Day Three
8. Configuring Work Managers
9. Monitoring and Tuning Java EE Applications

Course Practices
• Most topics are reinforced with a hands-on practice.

• Some practices include a scripted solution to aid any
students who encounter problems or run out of time.

Classroom Guidelines
• The instructor starts each session at the scheduled time.

• Do ask questions, but be respectful of the current topic and
the interests of other students.
• Ensure that cell phones are silent.
These guidelines enable you to get maximum benefit from the course.

Facilities in Your Location
• Enrollment/Registration/Sign in
• Badges
• Parking
• Phones
• Internet
• Restrooms
• Labs
• Lunch
• Kitchen/Snacks
• Hours
• Materials (paper, pens, and markers)
Contact your instructor or the education coordinator for site-specific information. This may not
be applicable for a Live Virtual Class (LVC).

For More Information
Topic Website
Education and training http://education.oracle.com
Product documentation http://www.oracle.com/technology/documentation
Product downloads http://www.oracle.com/technology/software
Product articles http://www.oracle.com/technology/pub/articles
Product support http://www.oracle.com/support
Product forums http://forums.oracle.com
Oracle Learning Library
http://www.oracle.com/oll
Product tutorials and demos
These materials are not intended to be a complete reference for all W ebLogic Server
performance tuning topics and features. After you complete the course, Oracle offers a variety
of resources that you can use to obtain additional information.

Related Training
Course Title
Oracle WebLogic Server: Administration I
Oracle WebLogic Server: Administration II
Oracle WebLogic Server: JMS Administration
Oracle WebLogic Server: Troubleshooting Workshop
Note that some of the courses listed in the slide may be available as traditional or virtual
classroom training, whereas others may be self-paced, online offerings.

Performance Tuning: Overview
Objectives
After completing this lesson, you should be able to do the

following:
• Describe performance terms and concepts
• Configure The Grinder as a testing tool
• Apply performance testing methodology and tools

• Identify bottlenecks by using performance tools

Agenda
• Performance Tuning: Overview

– Terminology
– Performance Testing
– Benchmarking
– Why Is Testing Important?

– Performance Tuning Methodology
• Testing Tools

Performance Terminology
The performance of a software system is often characterized

by its:
Concept Description
Response time A time metric (for example, the time it takes the server to
deliver a web page)
Latency The time between the issuing of a request and the actual work
beginning on that request (Often latency is intertwined with
response time.)
Throughput A rate metric (requests per unit of time) (for example, requests
per second, bytes (B/KB/MB) per second)
Resource utilization A consumption metric (for example, the percentage of CPU

usage)
The performance of a system is measured mainly in terms of the response time, throughput per
unit time, and the resource utilization. However, in multitier systems that use the W ebLogic
Server implementations, the term “latency” often occurs in discussions about response time.
Latency refers to the time between the issuing of a request and the time when the work actually
begins on the request. For example, when a browser issues an HTTP request, it takes some
time for the request to arrive at the target server, and for the target server to start work on the
request. That time spent in transit (while no work is being done on the request) is latency.
Naturally, you will want to reduce the latency, improve the response time, increase the
throughput, and optimize the resource utilization of the system.

Scalability
The term scalability refers to a system’s ability to perform within

specification under increasing:
• User load
• Data load
• Hardware expansion
ci
• Resource contention tr
e
M
e
c
n
a
rm
o
rf
e
P
Increasing Resource
The scalability of the system is related to the ability of the system to accommodate increasing
load without having the system degrading unacceptably in performance.
For more information about the concepts on scalability and configuration information for
WebLogic Server, refer to Oracle® Fusion Middleware Performance and Tuning Guide 12c
Release 1 (12.1.2).
The chart in this slide shows the scalability of two different tests. Perhaps Test A is before
tuning and Test B is after tuning. The chart represents any possible scenario whereby
something increases that affects scalability, and some performance metric is captured to see if
scalability is achieved. Test A clearly shows a decline in performance as the resource is
increased, while Test B shows linear, increased performance.

Performance Testing
Performance testing includes a variety of test types.
Test Type Description
Functional Test Verifies that a specific application or application unit is performing

as expected
Stress Test A type of load test designed to determine a system’s limits
Load Test Determines how a system functions while handling a certain

number of concurrent requests
Soak Test A stress test or load test that is run for a long period of time
Functional testing verifies that an application demonstrates the correct behavior under certain
inputs. Load testing determines whether an application can support a specified load (for
example, 500 concurrent users) with specified response times. Load testing is used to create
benchmarks. Soak testing is load testing over an extended period of time. Soak testing
determines whether an application can meet specified goals for stability and reliability, under a
specified load, for a specified time period.

Maintaining Test Integrity
• Generate loads from machines other than those hosting

the system under test
• Minimize the amount of data collection software running on
the machines hosting the system
• Allow the system to ramp up and self-tune before

recording metrics
• Realistic tests should typically take into account:
– Utilizing the same hardware, OS, and database
configurations as the production system
– Clients connecting over a WAN instead of a LAN
There are certain considerations to keep in mind when undergoing performance testing. The
purpose of a performance test is to determine how the system performs in the actual conditions
within which it will run. Generating the client load on the system using the same hardware
skews the numbers because resource utilization is affected by the processing required to
generate the load. Similarly, if your profiling software is too intrusive or heavy, it can also skew
the numbers because resource utilization is affected again. Java applications, which include
Java application servers, interpret, compile, and optimize Java bytecode when the process is
started initially. The most accurate performance numbers are achieved by giving the JVM and
WebLogic ample time to “warm up” to a steady state before attempting to do your performance
testing. Otherwise, the performance numbers you collect may be skewed again because the
system was not fully optimized or was still spawning threads and performing other initialization
work.
The testing environment and the production environment should be identical in every way
possible. Many companies save money by using scaled-down systems for testing, but lose that
money right back in production because they run into unforeseen problems. A scaled-down
system might become CPU-bound during stress testing, but perhaps the larger production
system is more likely to become disk I/O bound. It is worth the investment and relief of stress
that comes when production is down.

Benchmarking
A benchmark:
• Is a performance measurement of a system
• Is used to compare system performance in different
circumstances
• Includes a workload that remains the same between

measurements
• Is based on the application you are testing
• Is used to create a baseline performance measurement of
your application for comparison and future use
To determine your performance objectives, you must understand the application deployed and
the environmental constraints placed on the system. Gather information about the levels of
activity that components of the application are expected to meet, such as:
• The anticipated number of users
• The number and size of requests
• The amount of data and its consistency
• Your target CPU utilization
Performance objectives are limited by constraints, such as the configuration of hardware and
software, including CPU type, disk size versus disk speed, and memory. There is no single
formula for determining your hardware requirements. The process of determining what type of
hardware and software configuration is required to meet application needs adequately is called
capacity planning. Capacity planning requires assessment of your system performance goals
and an understanding of your application. Capacity planning for server hardware should focus
on maximum performance requirements. In addition, consider how much of your WebLogic
Server machine’s processing power is consumed by processes unrelated to WebLogic Server.

Load and Stress Testing
• Load testing measures performance statistics for a system

at different levels of concurrent request loads.
• Stress testing measures a system’s limits. For example:
– Maximum concurrent users
– Maximum data volume

• Performance testing a WebLogic Server application
requires:
– Measurable performance goals, such as “Must meet Service
Level Agreements (SLA) under peak load”
– A defined methodology and key metrics: Response time
must meet key performance indicators (KPI)
– Load generation or simulation, and measurement tools
You begin a stress testing plan by testing the use cases of the application. Because those use
cases provide a description of how the application is used when in production. Use cases can
often be translated directly into test scripts that can be run against the application.
Three very useful and free tools that you can use to start generating client load to stress a
WebLogic application are:
• The Grinder (http://sourceforge.net/projects/grinder), which is an all Java tool that is
customizable via plug-in modules
• Apache JMeter (http://jakarta.apache.org/jmeter/), which is an open source all Java
software designed to load test functional behavior and measure performance
The stress tests should include micro benchmarks, that is, tests on small parts of the
application. The parts of an application that are used the most should be tested on their own.
That way, if you can tune those parts, your performance will increase quite a bit. Even though
testing the most used parts of an application is important, it does not replace stress testing the
whole application (including database access).

Why Is Testing Important?
Customers who performed higher levels of testing were more

satisfied with product effectiveness.
9.0 Product Effectiveness
n
io
t 8.5
Software Reliability
c Speed of Product
a
f
t 8.0
is Scalability of Product
a
S 7.5
r
e
m 7.0
o
t
s 6.5
u
C
6.0
No test Informal test/formal Formal test using
test based on written automated tools
plan/other
• Results from Q1 2003 Customer Satisfaction Survey

• Product Effectiveness is the average satisfaction ratings of the nine (9)
product functionality items.
In a customer satisfaction survey, a direct correlation was shown between the amount of
testing performed and the customer’s satisfaction with the application.
In every measure including overall product effectiveness, reliability, speed, and scalability,
customers that did more testing and more formalized testing had better results than those that
did not test.

Performance Tuning Methodology
Select parameters
What is the
workload to place
Define Modify to change and
apply one at a
on the system? Workload Settings
time.
What are the

application’s Define Perform Run the same
performance tests again.
requirements? Objectives Tests
Target the entire Select Compare

application or
Subsystems Results
specific aspects.
Run the initial

Establish Results
No
tests to determine
baseline Baseline Good?
performance. Are the numbers
Yes better or worse?
Done
Good system performance depends on good design, good implementation, defined

performance objectives, and performance tuning. Tuning allows you to minimize the impact of
bottlenecks on your performance objectives. Implement mechanisms that provide performance
metrics that you can compare against your performance objectives, allowing you to schedule a
tuning phase before your system fails. You will find that at least one resource (CPU, memory,
or I/O) will be a bottleneck in the system.
The objective of the performance tuning process is to meet your performance objectives in an
iterative manner by changing parameters one at a time. It is not to eliminate all bottlenecks all
at the same time. Resources within a system are finite. It is very important to make
benchmarks realistic; otherwise, there will be unexpected results when the application goes
into the production environment. If an application accesses a database, ensure that it accesses
the database during the stress test. If you remove the database access, benchmark numbers
will be meaningless.
In production, many clients might connect over a W AN. If so, ensure that you include testing
client connections over a WAN during the stress test. Several products simulate WAN client
connections. The benchmarks that you compare should be between systems of a consistent
state.

Performance Testing Concerns
• Testing without a real network connection can give false

measures.
• Low user simulations can be very different from high user
simulations.
• Network throughput may be larger than the deployed

environment.
• Non-persistent message performance is dependent on
processor and memory.
• Persistent message performance is dependent on disk
speed.
• Use long-running tests to soak-test stability and data
storage capacity.
Whatever is being tested should match the ultimate deployment environment. Failure to do so
could skew performance results and put you in a difficult position in production. This slide
shows some examples of things that can go wrong or things to keep in mind.

Performance Testing Methodology Tips
• Perform tests and create benchmarks on:

– Functional areas (subsystems) of the application
– The full (integrated) application
• Test performance early and often, but defer tuning until
forced to do so.
It is important to understand why benchmarking is used. Benchmarks provide performance

statistics for an application. Java applications do not stand alone. Their performance depends
on the operating system, the hardware, the Java Virtual Machine, the just-in-time (JIT)
compiler, and more. Benchmarks can be run on multiple platforms to provide information about
the performance characteristics of an application on each one; this lets you determine the best
one to use. A development project can have a long life cycle, and benchmarking can be used
to determine whether performance requirements are being met. Performing benchmarks
throughout the project life cycle helps avoid discovering, at the end of development, that an
application’s performance is inadequate.
Before fine-tuning an application, operating system parameters, or WebLogic Server
parameters, perform an initial benchmark test to determine a baseline. After tuning, perform
another test to determine the effect on performance. A good benchmark can provide a lot of
information about an application’s performance characteristics; however, if the results are
taken out of context, they can cause fear, uncertainty, and doubt.

Metrics
• Metrics are the units used to express performance

measurements, such as throughput and response time.
• Metrics and their meaning must be understood by all
stakeholders.
• Each benchmark result set should (minimally) include

details about:
– The application’s version and design
– WebLogic Server domain and cluster configuration
– The version and patch level of WebLogic Server
– The operating system configuration
– The hardware configuration
The performance of an application can be affected by factors such as the application code, the
operating system, the hardware platform, and more. Therefore, benchmark results should be
reported with, at least, the following information:
• A detailed description of the application, such as a class diagram and/or use case
diagram
• The type of operating system, including version and patch level
• The version and patch level of WebLogic Server
• The details about the hardware, such as the processor speed, hard drive space, and the
amount of RAM
• The performance characteristics of any external systems
Without providing details about the environment, the benchmark results have little meaning.
Benchmarks given in the wrong context can cause uncertainty and doubt.
The benchmark results are appropriate for the application in the specific test environment. It
should not be assumed that the application would exhibit the same behavior when put into a
different environment. The volume of performance test result data can mount quickly. Consider
using a relational database for storing and analyzing test result data.

Results Reporting
• Well-planned, tabulated benchmark result reports are

essential to good analysis.
• Storing benchmark results in a database is helpful to good
reporting and test comparisons.
Number Avg.
Client Avg. Configuration
of App. Resp.
Load Throughput ID
Servers Time
1 2128ms 2000 100TPS small_clust03
Analysis: Scalability of a
software system
Archiving test result data in a relational database can help you observe long-term trends and
establish reliable baseline data, especially when compared to “live” statistics collected after the
application is deployed and operational. Many commercial performance-testing tools use a
relational database for storing test result data because of these advantages. Also, consider
archiving the version of W ebLogic Server, OS, hardware, network, and other configuration data
along with the application-specific metrics you collect.
The performance-reporting tools also enable you to provide an identifier for each test run to
enable easier comparison.
The table in the slide shows the benchmark results based on the number of application
servers.

Bottlenecks
The results from testing will identify bottlenecks. Any system,

CPU, disk, network, running at 100% may not give valid results.
Bottleneck Type Description
CPU Bound The system cannot process additional workload because the
processor is too busy (at or near 100%).
I/O Bound The processor is not fully used

(< 75%), even when the workload is increased.
Disk Disk is oversaturated.

Network Network is oversaturated.
Database Database is oversaturated.
*For network bottlenecks, take into consideration the MAN and
WAN, if you have remote users.
Generally, your goal is to get to a point where the application server achieves your target CPU
utilization. If you find that the application server CPU is underutilized, confirm whether the
database is bottlenecked. If the database CPU is 100% used, check the query plans of SQL
calls in your application. For example, are your SQL calls using indexes or doing linear
searches? Also, confirm whether there are too many ORDER BY clauses used in your
application that are affecting the database CPU. If you discover that the database disk is the
bottleneck (for example, if the disk is 100% used), try moving to faster disks or to a redundant
array of independent disks (RAID) configuration, assuming the application is not doing more
writes than required.
When you know that the database server is not the bottleneck, determine whether the
application server disk is the bottleneck. The disk I/O on an application server can be optimized
using faster disks or RAID, disabling synchronous Java Message Service (JMS) writes, using
Java Transaction API (JTA) direct writes for transaction logs, or increasing the HTTP log
buffer.
Check the amount of data transferred between the application and the application server, and
between the application server and the database server. This amount should not exceed your
network bandwidth; otherwise, your network becomes the bottleneck.

CPU Bound
The CPU-bound situation is easy to detect because the CPU

runs at or near 100% utilization.
PossibleCauses Solutions
Frequent garbage collection • Monitoring and tuning garbage collection

• Increasing heap size or physical memory
• Redesigning the application
Load too heavy for system • Proper capacity planning so hardware and
software align with expected load
Over-allocated resources (such as • Tuning resources to match the physical

threads vs. CPUs) architecture
Poorly designed Java application • Profiling and tuning the application

Ideally, the CPU should run only at 50%–75%, although there are some applications where
having the CPU running near 85%–95% is fine. After the CPU is bound, the application server
cannot do any more work. Adding threads will not do anything because the CPU is doing its
maximum amount of work. In some cases, the problem may even be caused by allocating too
many threads.

I/O Bound
The I/O bound case has the following properties:

• CPU utilization is not saturated.
• Performance is the same regardless of client load.
• The disk, network, or database is saturated.
Disk Network Database
I/O bound is very common in real-world applications. This is a situation in which the CPUs are
not saturated, but the performance is the same regardless of the number of clients. For
example, 10 clients send 50 transactions per second to the database, but if there are only five
clients, they still send 50 transactions per second to the database. In this scenario, usually the
requests from clients may be waiting for I/O operations, and the performance bottlenecks could
also be the disk, network, or database.

Disk Bound
When an application is determined to be disk bound, there are

several possible causes.
Insufficient physical memory

leading to paging • Profiling
sizes, or for memoryphysical
increasing leaks, decreasing
memory heap
Excessive logging • Profiling application to discover I/O usage

patterns
• Profiling and tuning the offending processes
Poorly designed Java application • Profiling and tuning the application
When the disk is overrun, something on the machine is over-utilizing it. This means that it is not
necessarily the application, although it could be. Common causes of disk bound issues are
excessive logging or if physical memory is insufficient to run everything all at once. First, verify
if virtual memory is in use. If it is, then you either have a memory leak in your application, your
application is not sized correctly, or the amount of physical memory on the system is too low. If
virtual memory is not in use, then you have to profile the system to find the offending processes
that are using the disk and profile those processes for offending operations.

Network Bound
A network can be the bottleneck if it is saturated. Networks can

saturate with as low as 50% bandwidth use.
Network bandwidth is insufficient Buy more bandwidth.

Tune network settings at the OS level.
Network bandwidth is not used efficiently Use larger packets, which make better
use of network bandwidth.
Non-tuned network settings Set timeouts properly to avoid locking up

resources.
Poorly designed application Profile application for excessive remote

calls. Redesign to limit or control network
use.
The network can be your bottleneck if it gets saturated, which is common over W AN links
because there is not much bandwidth. Monitor your network to determine how much bandwidth
is being used. The easiest fix is to buy more bandwidth. You can use a network sniffing tool to
monitor and analyze network performance issues.

Database Bound
The bottleneck for your application is found to be the database.
Too fewfor
waiting connections so clients are
a connection Create more database connections.
Long-running queries Profiley our database for inefficient table

design, SQL, and other known
performance areas. Redesign and tune
as needed. Avoid temp tables, use
indexing, limit query results, and so on.
Hardware performance is insufficient Move database to faster hardware or
move to a dedicated machine so the
database does not compete for
resources.
When the database is the bottleneck, there are many solutions, depending on the cause of the
problem. Sometimes, the number of connections is too low, and concurrent clients block,
waiting for a connection to free up. Then the solution is to create and use more database
connections. Sometimes, queries take too much time. In this case, the solution may be to
create secondary indexes on certain fields. If your database server’s machine is too slow, look
for better hardware, or move the database to a dedicated machine. If these solutions do not
resolve the problem, look into the fine-tuning options for your vendor-specific database.
Many of the solutions for handling a database bottleneck reside with application programmers.
Programmers can make more efficient trips to the database by grabbing all their information in
one query instead of multiple queries. This topic goes beyond the scope of this course.

The AWR is used to collect performance statistics including:
• Wait events used to identify performance problems
• Time model statistics indicating the amount of DB time associated with a process
• Active Session History (ASH) statistics
• Some system and session statistics
• Database object usage statistics
• Resource-intensive SQL statements

Section Summary
In this section, you should have learned how to describe

performance terms and concepts.

Practice 2-1 Overview:
Setting Up the Practice Environment
This practice covers setting up the initial environment used for
all the practices of the course.

Exploring the Course Applications
This practice provides a high-level overview of the applications
used in this course. You do not have to be familiar with the
detailed implementation of these applications, as much as
being familiar with how to use them.

Agenda
• Performance Tuning: Overview

• Testing Tools

Load-Testing Tools
Different commercial and open-source load-testing tools are

available in the market, including:
• Oracle Application Testing Suite
• The Grinder We are using this in this course.
• Apache JMeter
• HP LoadRunner
• RadView WebLOAD
JMeter is an Apache Jakarta project that can be used as a load-testing tool for analyzing and
measuring the performance of a variety of services, with a focus on web applications. JMeter
can be used as a unit-test tool for Java Database Connectivity (JDBC) database connections,
FTP, Lightweight Directory Access Protocol (LDAP), web services, JMS, HTTP, and generic
TCP connections. JMeter also supports assertions to ensure that the data received is correct,
per-thread cookies, configuration variables, and a variety of reports.
LoadRunner is a performance- and load-testing product by Hewlett-Packard (since it acquired
Mercury Interactive in November 2006) for examining system behavior and performance, while
generating actual load. Working in LoadRunner involves using three different tools, which are
part of LoadRunner. They are Virtual User Generator (VuGen), Controller, and Analysis.

Choosing Testing Tools
You must consider many factors when you are choosing a

load-testing tool.
• Installation and configuration
• Deployment for distributed testing
• Reporting
• Accuracy of results
• Cost
• Learning
• Other features
You mainly consider the hardware and software resources in your environment and choose
suitable testing tools. You should also consider the knowledge of the personnel in your
organization, and the cost of training if required.
In this course, you use The Grinder, which is developed in Java and released under the
Berkeley Software Distribution (BSD) license (open source).

The Grinder
• The Grinder:
– Is a load-testing tool
– Is an open-source implementation based on Java and
Python
– Supports a distributed, agent-based load-simulation model
– Provides a graphical console to manage agents and view
aggregated results
– Supports HTTP/S (links, forms, cookies, and so on) but can
be extended to support additional protocols
• The Grinder agent:
– Utilizes a specific number of processes and threads
– Executes a supplied test script with each thread
The Grinder is a Java load-testing framework that makes it easy to run a distributed test using
many load-injector machines. It is freely available under a BSD-style open-source license, and
is based on other open-source technologies such as Jython, HTTPClient, XMLBeans, and
PicoContainer. The Grinder was srcinally developed for the book Professional Java 2
Enterprise Edition with BEA WebLogic Server.
Each test context runs in its own thread. The threads can be split over many processes
depending on the requirements of the test and the capabilities of the load-injection machine.
The Grinder makes it easy to coordinate and monitor the activity of processes across a network
of many load-injection machines from a central console.
Scripts can be created by recording actions of a real user by using TCPProxy. The script can
then be customized manually. Input data (for example, URL parameters or form fields) can be
dynamically generated. The source of the data can be anything including flat files, random
generation, a database, or previously captured output.
The Grinder has

management for special supportUsers
test contexts. for HTTP
can that
writeautomatically handles
their own plug-ins to cookie and connection
a documented interface,
although this is rarely necessary due to its powerful scripting facilities.

The Grinder Architecture
CPU
Agent
Start test
Thread
Thread
1 Report results 3 Thread 2
CPU Test script
HTTP/S System
Console
CPU under test
1 3 Agent
Thread
Thread
Thread 2
Test script
The Grinder has a framework for running test scripts across a number of machines. The
framework comprises three types of processes:
• Worker processes interpret Jython test scripts and perform tests using a number of
worker threads.
• Agent processes manage worker processes.
• The console coordinates the other processes, and collates and displays resulting
statistics.
Because The Grinder is written in Java, each of these processes is a Java Virtual Machine.
For heavy-duty testing, you start an agent process on each of the load-injector machines. The
worker processes that they start can be controlled and monitored using the console. There is
little reason to run more than one agent on each load injector, but you can if you want. Each
worker process sets up a network connection to the console to report statistics. Each agent
process sets up a connection to the console to receive commands, which it passes on to its
worker processes. The console listens for both types of connection on a particular address and
port.

A test is a unit of work against which statistics are recorded. Tests are uniquely defined by a test
number and also have a description. Users specify which tests to run by using a Jython test
script. The script is executed many times in a typical testing scenario. A single execution of a test
script is called a “run.”

The Grinder Console
Manage agents. Distribute

test scripts.
Results for
individual requests
Overall results
for an entire
test
The Start processes, Reset processes, and Stop processes menu items send signals to The
Grinder processes that are listening. These controls are disabled if no agents are connected to
the console. On the Processes tab, you can check whether any agents are connected. The
Start processes send signals to worker processes that they should move into the running state.
Processes that are already running ignore this signal. Processes that are in the finished state
exit. The agent process then rereads the properties file and launches new worker processes in
the running state. The Reset processes signal all the worker processes to exit.
The sample controls determine how the console captures reports from the worker processes. It
is important to understand that these control only the console behavior. They do not adjust the
frequency at which the worker processes send reports. The slider controls the period at which
the console takes a sample. This involves adding up all the reports received over that sample
interval and calculating the TPS as (number of tests that occurred)/(interval length). It is also
the period at which the console graphs and statistics are updated.
Each time the worker processes run, they generate a new set of logs. Logs from previous runs
are “archived” by renaming them. The number of logs that are kept from previous runs can be
controlled with grinder.numberOfOldLogs .

The Grinder Summary Statistics
You can view a summary of the statistics in tabular format in The Grinder’s console. Click the
Results tab to view the statistics of the test in tabular format.
This graphic is a screenshot of The Grinder console showing statistics.

Monitoring Performance by Using
the Administration Console
You can use the Administration console to quickly monitor the performance of WebLogic
Server instances. The Monitoring tab for each server provides pages that help you monitor the
status and performance of resources associated with the server. Some of the important pages
are:
• The General page provides general runtime information about this server and includes
information such as the current lifecycle state of this server, which JVM is used, and
when the server was started.
• The Health page provides a table showing the health state of the resources in the server.
• The Channels page provides a table showing the name, protocol, and number of active
connections being served by this server.
• The Performance page lets you monitor performance information about this server. You
can also use this page to force garbage collection or a thread dump.
• The Threads page provides information about the thread activity for the current server in
the form of two tables. The first table on this page provides general information about the
status of the thread pool. The second table provides information about individual threads.

• The Timers page displays information about the timers in use by this server.
• The Workload page provides statistics for the work managers, constraints, and request
classes that are configured for this server.

Oracle WLS Monitoring Dashboard
Use the WLS Monitoring Dashboard to monitor the status and

performance of all the servers in a domain.
You can also use the Monitoring Dashboard to graphically view the current and historical
operating state of WebLogic Server and hosted applications in a domain. The Monitoring
Dashboard provides a set of tools for organizing and displaying diagnostic data into views,
which surface some of the more critical runtime WebLogic Server performance metrics and the
change in those metrics over time. The Monitoring Dashboard also includes the Metric
Browser. You can use the Metric Browser to select the specific managed bean (MBean) types,
instances, and attributes that you want to monitor in a chart for a selected server. Historical
MBean attribute data is always retrieved from a W LDF archive, which is available if you have
configured the WLDF Harvester to gather metrics data and created that archive. The diagnostic
data displayed by the Monitoring Dashboard consists of runtime MBean attributes with numeric
or Boolean values that are useful to measure, either as their current values or as their changes
over time. These values, referred to in the Monitoring Dashboard as metrics, srcinate from two
sources:
• Directly from active runtime MBean instances: These metrics are sometimes called polled
metrics.
• From the Archive that have been collected by the Harvester: These metrics are also
known as collected metrics.

Monitoring Dashboard: Views
• A view is a collection of one or more charts that display

captured monitoring and diagnostic data.
• Two general types of views can be displayed, created, or
modified in the Monitoring Dashboard:
– Built-in views: A
runtime metrics forset
all of predefined
running viewsServer
W ebLogic of available
instances in
the domain.
– Custom views: Any view created by a user. Custom views
are available only to the user who creates them.
A view is a collection of one or more charts that display captured monitoring and diagnostic
data. Views are displayed in the main display panel of the Monitoring Dashboard. Only one
view is displayed at a time. Two general types of views can be displayed, created, or modified
in the Monitoring Dashboard:
• Built-in views: A set of predefined views of available runtime metrics for all running
WebLogic Server instances in the domain. Built-in views surface some of the more critical
run-time WebLogic Server performance metrics and serve as examples of the Monitoring
Dashboard's view and graphing capabilities. Built-in views are available for every user
logged into the Administration Console and using the Monitoring Dashboard. You cannot
rename, delete, modify, or save a built-in view.
• Custom views: Any view created by a user. Custom views are available only to the user
who creates them. Custom views are automatically persisted and can be accessed again
in subsequent Monitoring Dashboard sessions. You can also create a custom view by
copying (cloning) a built-in view, and then modifying, renaming, or deleting it as with other
custom views.

Section Summary
In this section, you should have learned how to:

• Apply performance testing methodology and tools
• Identify bottlenecks using performance tools
• Configure The Grinder as a testing tool

Executing a Benchmark
This practice covers the following topics:
• Setting up a WebLogic Server domain
• Setting up The Grinder
• Recording load test script
• Performing a load test
• Recording performance metrics
This practice establishes a baseline measurement of performance for an application.

Practice 2-4 Overview: Stress Testing an
Application and Identifying Bottlenecks
• Configuring The Grinder to push the application to its limits
• Performing the stress test
• Recording performance metrics
• Using various tools to determine the bottlenecks of the
application
This practice pushes the application to its limits and students compare performance numbers
with baseline numbers and identify bottlenecks of the system.

Tuning Operating System Resources
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Objectives
After completing this lesson, you should be able to:

• Describe Linux/UNIX performance metrics for the CPU,
memory, network, and I/O subsystems
• List the performance monitoring tools provided by the
Linux/UNIX operating systems

• Use several OS-provided monitoring tools
• Configure a few parameters to improve performance

Agenda
• Linux/UNIX performance metrics

– CPU metrics
– Memory metrics
– Network interface metrics
– I/O device metrics

• Performance monitoring tools
– top, sar, vmstat, ps, iostat, netstat , System
Monitor
• OS parameters that affect WebLogic performance

Processor Metrics
Metric Meaning
CPU utilization Is probably the most straightforward metric. It
describes the overall utilization per processor.
User time Depicts the CPU percentage spent on user
processes, including nice time. High values in
user time are generally desirable because, in this
case, the system performs actual work
System time Depicts the CPU percentage spent on kernel
operations including IRQ and softirq time. High
and sustained system time values can point you
to bottlenecks in the system, which should
generally spend as little time as possible in kernel
time
CPU utilization: This refers to a computer's usage of processing resources, or the amount of
work handled by a CPU. Actual CPU utilization varies depending on the amount and type of
managed computing tasks. Certain tasks require heavy CPU time, while others require less
because of non-CPU resource requirements.
User Time: This is the amount of time the command spends in "User Mode." User Mode is a
"safety" feature because when a program is in User Mode it cannot cause other programs to
crash, or behave unexpectedly.
System Time: System Time is the amount of time the command spends in "Kernel Mode."
There are certain tasks a command or an application must run in Kernel Mode to accomplish;
however, time spent in "Kernel Mode" is indicative of time spent by the Linux/UNIX
"infrastructure" (system services, context switches, I/O synchronization, etc.) rather than CPU
cycles spent satisfying the application computational requests.
Context Switch: When a CPU switches from one process (or thread) to another, it is called
as context switch. When a process switch happens, the kernel stores the current state of the
CPU (of a process or thread) in the memory. The kernel also retrieves the previously stored
state (of a process or thread) from the memory and puts it in the CPU. Context switching is
very essential for multitasking of the CPU, however, a higher level of context switching can
cause performance issues.

Processor Metrics
Metric Meaning
Waiting Total amount of CPU time spent waiting for an I/O
operation to occur. Like the blocked value, a
system should not spend too much time waiting
for I/O operations; otherwise, you should
investigate the performance of the respective I/O
subsystem.
Idle time Depicts the CPU percentage the system was idle
waiting for tasks
Nice time Depicts the CPU percentage spent on re-nicing

processes that change the execution order and
priority of processes
The nice command is used to run a program with modified scheduling priority/nicenesses.
Nicenesses range at least from -20 (resulting in the most favorable scheduling) through 19
(the least favorable). The default behavior is to increase the niceness by 10. Niceness should
not be confused with a scheduling priority, which lets applications determine the order in
which threads are scheduled to run. Unlike a priority, a niceness is merely advice to the
scheduler, which the scheduler is free to ignore.

Memory Metrics
Metric Meaning
Free memory Compared to most other operating systems, the
free memory value in Linux should not be a
cause for concern as the Linux kernel allocates
most unused memory as file system cache; so
subtract the amount of buffers and cache from
the used memory to determine (effectively) free
memory.
Swap Usage Depicts the amount of swap space used
Slabs Depicts the kernel usage of memory. Note that

kernel pages cannot be paged out to disk.
Swap In/Out is a reliable means of identifying a memory bottleneck. Values above 200 to 300
pages per second for a sustained period of time express a likely memory bottleneck.
A Slab is a set of one or more contiguous pages of memory set aside by the slab allocator for
an individual cache. This memory is further divided into equal segments the size of the object
type that the cache is managing.

Memory Metrics
Metric Meaning
Active versus Provides you with information about the active
inactive use of the system memory. Inactive memory is a
memory likely candidate to be swapped out to disk by the
kswapd daemon.
Buffer and The buffers number represents in-memory blocks
cache that result from the kernel accessing the disk,
such as when the kernel needs to read the
contents of files. The cached figure tells you how
much RAM is being used to cache the content of
recently read files. The buffer figure increases
when the file system layer is bypassed while the
cache grows when the file system is used.
There are two states of used memory: inactive and active.

Active memory is memory that is being used by a particular process. Inactive memory is
memory that was allocated to a process that is no longer running. "top" and "free -m" do
not differentiate between these two types of memory. This is why the command free -m
may show only 2% memory not being used, while the top command may only show a few
processes using 1 or 2%.
Cached is the size of the Linux page cache, minus the memory in the swap cache, which is
represented by SwapCached (thus the total page cache size is Cached + SwapCached).
Linux performs all file I/O through the page cache. Writes are implemented as simply marking
as dirty the corresponding pages in the page cache; the flusher threads then periodically write
back to disk any dirty pages. Reads are implemented by returning the data from the page
cache; if the data is not yet in the cache, it is first populated.
On a modern Linux system, Cached can easily be several gigabytes. It will shrink only in
response to memory pressure. The system will purge the page cache along with swapping
data out to disk to make available more memory as needed.
Buffers are in-memory block I/O buffers. They are relatively short-lived. Prior to Linux kernel
version 2.4, Linux had separate page and buffer caches. Since kernel release 2.4, the page
and buffer cache are unified and Buffers is raw disk blocks not represented in the page cache
— i.e., not file data. The Buffers metric is thus of minimal importance. On most systems,
Buffers is often only tens of megabytes.
Network Interface Metrics
Metric Meaning
Packets Informs you about the quantity of packets
received and received and sent by a given network interface
sent
Bytes received Depicts the number of bytes received and sent by
and sent a given network interface
Packets Is a count of packets that have been dropped by
dropped the kernel, either due to a firewall configuration or
due to a lack of network buffers
Overruns Overruns represent the number of times that the
network interface ran out of buffer space
Packet drops in Linux can happen at two layers, either at the NIC level or at the Network
stack level. An abnormal amount of packet drops at NIC level could indicate RX or TX buffers
set to sub-optimal levels.
The overruns metric should be used in conjunction with the packets dropped value to identify
a possible bottleneck in network buffers or the network queue length.

Network Interface Metrics
Metric Meaning
Collisions per Provides an indication of the number of collisions
second that occur on the network that the respective
interface is connected to. Sustained values of
collisions often concern a bottleneck in the
network infrastructure, not the server. On most
properly configured networks, collisions are rare
events.
Errors The number of frames marked as faulty. This is
often caused by a network mismatch or a partially
broken network cable. Partially broken network
cables can be a significant performance issue for
copper-based gigabit networks.
Data packets can collide with one another when being sent on a shared medium or through
repeaters. A network collision occurs when more than one device attempts to send a packet
on a network segment at the same time.Collisions are resolved using carrier sense multiple
access with collision detection (CSMA-CD) in which the competing packets are discarded and
re-sent one at a time. This becomes a source of inefficiency in the network.
Only one device in the collision domain may transmit at any one time, and the other devices
in the domain listen to the network in order to avoid data collisions. Because only one device
may be transmitting at any one time, total network bandwidth is shared among all devices.
CRC Errors: Frames were sent but were corrupted in transit. The presence of CRC errors,
but not many collisions usually is an indication of electrical noise. Make sure that you are
using the correct type of cable, that the cabling is undamaged and that the connectors are
securely fastened.
Frame Errors: An incorrect CRC and a non-integer number of bytes are received. This is
usually the result of collisions or a bad Ethernet device.
FIFO and Overrun Errors: The number of times that the NIC was unable of handing data to
its memory buffers because the data rate the capabilities of the hardware. This is usually a
sign of excessive traffic.

I/O Device Metrics
Metric Meaning
Iowait Time the CPU spends waiting for an I/O
operation to occur. High and sustained values
most likely indicate an I/O bottleneck.
Average queue Amount of outstanding I/O requests. In general, a
length disk queue of 2 to 3 is optimal; higher values
might point toward a disk I/O bottleneck.
Average wait A measurement of the average time in ms it takes
for an I/O request to be serviced. The wait time
consists of the actual I/O operation and the time it
waited in the I/O queue.
iowait: This number shows the % of time the CPU is wasting in waiting for I/O. A part of this
number can result from network I/O, which can be avoided by using an Async IO library. The
remaining part of it is simply an indication of how I/O-bound your system is.
Average Wait: The percentage of time that I/O operations spent waiting in queue in
comparison to actually being serviced. If this figure goes above 50% then each I/O request is
spending more time waiting in queue than being processed. If this ratio skews heavily
upwards (in the >75% range) you know that your disk subsystem is not being able to keep up
with the I/O requests and most I/O requests are spending a lot of time waiting in queue.
Oracle WebLogic Server 12 c: Performance Tuning Workshop 3 - 10

I/O Device Metrics
Metric Meaning
Transfers per Depicts how many I/O operations per second are
second performed (reads and writes). The transfers per
second metric in conjunction with the kBytes per
second value helps you to identify the average
transfer size of the system.
Blocks Depicts the reads and writes per second
read/write per expressed in blocks of 1024 bytes as of kernel
second 2.6
Kilobytes per Reads from and writes to the block device in
second kilobytes represent the amount of actual data
read/write transferred to and from the block device.

Section Summary
In this section, you should have learned how to describe

Linux/UNIX performance metrics for the CPU, memory,
network, and I/O subsystems.

Agenda

– CPU metrics
– Memory metrics

– top, sar, vmstat, ps, iostat, netstat, System
Monitor

What to Monitor in Operating System
Subsystems to be monitored at OS level:

• CPU utilization
• Network traffic
• Disk I/O
• Virtual memory usage
• Processes and kernel locks
• Big picture view of CPU demand

• Per process measurement of CPU utilization
• Measurements of CPU usage
• User (usr) time
• System (sys) time
• Idle time
• Voluntary context switching (VCX)
• Involuntary context switching (ICX)

CPU Utilization: What to Look For
• You can use the jps or jstat command to find the

process IDs of all running Java processes on your
machine.
• Get a measurement of CPU utilization for each process.
• Measurements of CPU usage:

– User (usr) time
– System (sys) time
– Idle time
– Voluntary context switching (VCX)
– Involuntary context switching (ICX)
CPU usage is a very important performance indicator. CPU usage should be measured over
long periods. If the peak CPU usage is sustained at very low values, and the applications are
performing at expected speeds, this indicates that the processor can accommodate more
work. Such a system can be used for scaling up.
You can look at these parameters when monitoring CPU:
• High CPU time: A high (consistently above 60 percent) CPU time indicates a lot of CPU
cycles are affecting performance. A reduction in kernel CPU time will give more CPU
time to the application. Similarly, high sys CPU time could indicate shared resource
contention.
• Idle CPU: The idle CPU can be an indicator of an application's inability to scale.
Combination of high sys or kernel CPU utilization and idle CPU could indicate shared
resource contention as the scalability blocker.

• Frequent context switches affect performance in resource release and reclaim
operations:
- A voluntary context switch occurs when a running thread voluntarily takes itself off
the CPU as a result of some blocking operation such as an I/O wait or the thread
going to sleep.
- An involuntary context switch occurs when an executing thread has either
exhausted its time allocation, or the executing thread was preempted by a higher
priority thread.

CPU Usage in Linux Environment
• CPU usage at near 100 percent for an extended period

indicates that some application is performing inefficiently.
• If peak CPU usage is at low values and the responses are:
– Slow, then the applications are not effectively using the CPU
– Quick, then the system is capable of handling additional load

• You can use the System Monitor graphic tool to monitor
CPU usage.
• Some of the command-line tools in the Linux environment
are:
– sar
– vmstat
– iostat
If the CPU usage remains high (near 100 percent) over long periods, and occasional
performance problems arise, then this could mean:
• The system is very heavily used. Such a system cannot handle any further load.
• Some of the applications are not designed efficiently. By tuning these applications, the
CPU usage can be brought to efficient levels.
One single program can very rarely keep the CPU 100 percent busy (that is, 0 percent idle
and 0 percent wait for more than a few seconds at a time). Even in heavily loaded multiuser
systems, there are occasional 10-milliseconds (ms) periods that end with all threads in a wait
state. If a monitor shows the CPU as 100 percent busy for an extended period, there is a good
chance that some program is in an infinite loop.

sar Command
• Displays the system activity report for the CPU

• Samples system activity counters available in the UNIX
kernel and prepares reports
• Displays the following values to look at in the sar output:
– CPU usage
– Disk activity
The operating system contains several counters that are incremented as various system
actions occur. These include counters for CPU utilization, buffer usage, disk and tape I/O
activity, standard output (terminal) device activity, switching and system-call activity, file
access, queue activity, interprocess communications, and paging.
The sar command, when executed, reads from these counters and displays information to
the user.
Use the sar command to get real-time sampling and display of current statistics.
To collect and display system statistic reports immediately, use the following command:
# sar -u 2 5

vmstat Command
• This command reports virtual memory statistics of the

process, virtual memory, disk, trap, and CPU activity.
• The reported CPU activity is a percentage breakdown of
user mode, system mode, idle time, and waits for disk I/O.
• The values to in
– Processes look
the from the vmstat
run queue ( procs output
r)
are:
– CPU time in user, system, and idle ( cpu us, sy, and id)
Average
since
system
start
The vmstat command quickly provides compact information about various system resources
and their related performance problems. This command reports statistics about kernel threads
in the run and wait queue, memory, paging, disks, interrupts, system calls, context switches,
and CPU activity.
If it is used without any options or only with the interval and optionally, the count parameter,
such as vmstat 2 10, then the first line of numbers reflect the average values since system
started.
You can get detailed help on vmstat from man pages.

iostat Command
• This command generates two types of reports:

– CPU Utilization report
– Device Utilization report
• You get a quick first impression of whether or not the
system has a disk I/O-bound performance problem.

• The typical syntax is:
– iostat <options> <interval> <count>
Use the iostat command for monitoring system input/output device loading by observing
the time the devices are active in relation to their average transfer rates. The iostat
command generates two types of reports: the CPU Utilization report and the Device Utilization
report.
The typical syntax for the iostat command is:
$> iostat <options> <interval> <count>
• Option: The device for which information is needed such as a d (disk), c (CPU), or t
(terminal)
• Interval: The time period in seconds between two samples
• Count: The number of times the data is needed

netstat Command
netstat is one of the most used tools. It displays several

network-related information such as socket usage, routing,
interface, protocol, network statistics, and more. The following
are some of the basic options:
• -a Show all socket information
• -r Show routing information
• -i Show network interface statistics
• -s Show network protocol statistics
Socket information
Proto The protocol (tcp, udp, raw) used by the socket
Recv-Q The count of bytes not copied by the user program connected to this
socket
Send-Q The count of bytes not acknowledged by the remote host
Local Address Address and port number of the local end of the socket. Unless the
--numeric (-n) option is specified, the socket address is resolved to its
canonical host name (FQDN), and the port number is translated into the
corresponding service name.
Foreign Address Address and port number of the remote end of the socket
State The state of the socket. Since there are no states in raw mode and
usually no states used in UDP, this column may be left blank.

System Monitor
The System Monitor application enables you to monitor system processes and the usage of
system resources. You can also use the System Monitor to modify the behavior of your
system. The System Monitor window contains three tabbed sections:
• Processes: Shows active processes and how processes are related to each other. It
provides detailed information about individual processes and enables you to control
active processes.
• Resources: Displays the (recent) history of CPU, memory and swap space, and
network usage.
• File Systems: Shows the disk space usage and availability.
You can start the System Monitor in the following ways:
• On the Applications menu, select System Tools > System Monitor.
• On the command line, execute the command:gnome-system-monitor.

Performance Tools and Commands
Tool Main
Purpose
top Process activity
netstat Network statistics
vmstat System activity, Hardware and System

Information
iostat Average CPU load, disk activity
sar Collect and report system activity
mpstat Multiprocessor usage
ps, pstree Displays the processes
free Memory usage
On systems running Java applications such as WebLogic Server, the output of a ps -A

command might easily fill up the display to the point where it is difficult to get a complete
picture of all running processes.
In this case, the pstree command might come in handy as it displays the running processes
in a tree structure and consolidates spawned subprocesses (for example, Java threads). The
pstree command can help identify srcinating processes. There is another ps variant, pgrep. It
might be useful as well.
The command /usr/bin/free displays information about the total amount of free and used
memory (including swap) on the system. It also includes information about the buffers and
cache used by the kernel.
The mpstat command is used to report the activities of each of the available CPUs on a
multiprocessor server. Global average activities among all CPUs are also reported. The
mpstat utility is part of the sysstat package.
The mpstat utility enables you to display overall CPU statistics per system or per processor.
mpstat also enables the creation of statistics when used in sampling mode analogous to the
vmstat command with a sampling frequency and a sampling count.

Performance Tools and Commands
Tool Main
Purpose
pmap Process memory usage
tcpdump, Wireshark Detailed network traffic analysis
nmon Collect and report system activity
strace System calls
/proc file system Various kernel statistics
Gnome System Monitor Real-time systems reporting and graphing
The pmap command reports the amount of memory that one or more processes are using.
You can use this tool to determine which processes on the server are being allocated memory
and whether this amount of memory is a cause of memory bottlenecks. For detailed
information, use pmap -d option.
tcpdump is a simple but robust utility. It has basic protocol analyzing capability allowing you
to get a rough picture of what is happening on the network. tcpdump supports many options
and flexible expressions for filtering the frames to be captured (capture filter).
nmon, short for Nigel's Monitor, is a popular tool to monitor Linux systems performance
developed by Nigel Griffiths. Since nmon incorporates the performance information for
several subsystems, it can be used as a single source for performance monitoring. Some of
the tasks that can be achieved with nmon include processor utilization, memory utilization, run
queue information, disks I/O statistics, network I/O statistics, paging activity, and process
metrics. In order to run nmon, simply start the tool and select the subsystems of interest by
typing their one-key commands. For example, to get CPU, memory, and disk statistics, start
nmon and type c m d.

The strace command intercepts and records the system calls that are called by a process,
and the signals that are received by a process. This is a useful diagnostic, instructional, and
debugging tool. System administrators find it valuable for solving problems with programs.
The /proc file system is not a real file system, but nevertheless it is extremely useful. It is not
intended to store data; rather, it provides an interface to the running kernel. The proc file
system enables an administrator to monitor and change the kernel on the fly. Most Linux tools
for performance measurement rely on the information provided by /proc.

Monitoring Disk I/O
• You can look for the following parameters to monitor disk

I/O:
– Number of disk accesses
– Latency and average latencies
– Paging
– Inode caches
– File handles
• You can use the following command-line tools to monitor
disk I/O:
– iostat
– sar -b
• You can also use the System Monitor GUI.
The Linux command sar (short for System Activity Report) prints to the standard output
device the selected activity details. You can use the sar –b command effectively to monitor
the I/O operations.
• For example, the following is a portion of output from sar –b command:
11:40:01 AM tps rtps wtps bread/s bwrtn/s
11:50:01 AM 0.73 0.00 0.73 0.00 9.61
12:00:01 PM 0.72 0.00 0.72 0.00 9.53
12:10:01 PM 0.87 0.00 0.87 0.00 12.87
12:20:01 PM 0.72 0.00 0.72 0.00 9.17
12:30:01 PM 0.83 0.01 0.82 2.34 11.98
12:40:01 PM 1.55 0.19 1.36 7.48 31.82
Average: 1.46 0.60 0.87 7.98 16.13

• tps refers to total number of transfers per second that were issued to physical devices.
A transfer is an I/O request to a physical device. Multiple logical requests can be
combined into a single I/O request to the device. A transfer is of indeterminate size.
- rtps refers to number of reads per second.
- wtps refers to number of writes per second.
• bread/s refers to number of blocks read per second. The blocks are 512 bytes in size.
• bwrtn/s refers to number of blocks written per second.

Swapping
• Paging and swapping are ways for moving data in memory

to a storage device such as a hard drive.
• They are mechanisms meant to improve efficient use of
memory.
• However, frequent
performance paging
problems and swapping can cause
because:
– The CPU cycles are spent in moving data from and to disks
– Disk thrashing may occur with frequent swapping
– Page faults can occur when contiguous area is not available
• You can use vmstat to monitor swapping.
Paging and swapping are methods for moving data ni memory to a storage device such as a
hard drive.
Swapping moves complete memory structures for a process to a specified area on the hard
disk. The specified area is referred to as swap area. The process goes from the state of being
in memory to being swapped out entirely. The advantage of swapping is that memory for a
program is allocated contiguously. However, performance can be adversely affected when the
system ends up constantly swapping in and out of disk—a situation often referred to as
thrashing.
Paging involves portioning process memory into pages and writing out pages of process
memory rather than entire process memory. This reduces time and amount of disk i/o
involved. But this would mean that more of the memory still remains occupied even though
the corresponding process cannot be effectively operated upon.
You can reduce swapping by:
•
Using smaller Java heap sizes

• Adding physical memory
• Reducing number of applications running on the machine

inode Cache
• The inode cache is an in-memory hash table used by the

Virtual File System (VFS) defined infs/inode.c.
• The correct sizing of the inode cache is important for
good performance.
• The percentage
having of inode
active pages s that can
in memory werebeoverwritten while
retrieved by still
using
the following command:
sar -v
• If this number is consistently nonzero, theinode cache
should be increased.
The VFS maintains an inode cache to speed up accesses to all the mounted file systems.
The inode cache is implemented as a linked list that stores the inodes that have been
accessed along with pointers to all pages that are part of that file and are currently in memory.
Every time a VFS inode is read from the inode cache, the system saves an access to a
physical device. The entries in the inode cache stay in the cache until the system removes
them to make room for newer entries.
To determine whether theinode cache is large enough, check the following sar columns:
iget/s, namei/s, and dirbk/s. The first column (iget/s) tells how frequently inode had
to be accessed from the disk. The latter two columns indicate the demand on the file system
(for file name-to-inode translation and directory-block scanning). If iget/s values are
frequently too high (greater than five, or so), you may need a larger inode cache. If namei/s
is more than ten times greater than iget/s, you may benefit from a largerinode cache.
Reconfigure and check again.

Section Summary
In this section, you have learned about:

• The performance monitoring tools provided by the
Linux/UNIX operating systems
• How to monitor the usage of system resources such as
CPU, memory, network, and I/O subsystems

Monitoring Linux CPU Utilization
• Running the StressTestOS application to generate CPU
load
• Using various operating system utilities to monitor and
assess the load on the CPU

Monitoring Linux Disk I/O
• Running the StressTestOS application to generate disk I/O
activity
assess the load on the disk

Monitoring Linux Virtual Memory and Network
• Running the StressTestOS application to generate network
activity
assess the load on the network

Agenda

– CPU metrics
– Memory metrics

– top, sar, vmstat, ps, iostat, netstat, System Monitor

File Handles
• The file-max parameter defines the maximum number

of file handles that the Linux kernel will allocate.
• You can modify this using the fs.file-max parameter.
• Update /etc/sysctl.conf with the following entry:
fs.file-max=65535
• Run sysctl –p to load the new value or reboot the
system for the change to take effect.
The /proc/sys/fs/file-maxparameter sets the maximum number of file handles that the
Linux kernel will allocate. This parameter is tuned to improve the number of open files by
increasing the value of /proc/sys/fs/file-maxto 65535.

TCP/IP Parameters to Support a Large
Number of Connections
• In order to support a large number of network connections
where WebLogic Server is accessed by many
simultaneous users, it is crucially important to tune several
TCP/IP parameters:
– The default value for the time wait interval is usually too high
and needs to be adjusted. To determine the number of
sockets in TIME_WAIT, you can use netstat -a | grep
TIME_WAIT | wc -l . When the number approaches the
maximum number of file descriptors per process, the
application’s throughput will degrade.
• To check the current value, you can usecat
/proc/sys/net/ipv4/tcp_keepalive_time and
cat /proc/sys/net/ipv4/tcp_keepalive_intvl.
When the number approaches the maximum number of file descriptors per process, new
connections have to wait for a free space in the application’s file descriptor table.
One of the problems found in servers with a lot of simultaneous TCP connections is the large
number of connections that are open but unused.
TCP has a keepalive function that probes these connections. By default, the keepalive
routines wait for two hours (7200 secs) before sending the first keepalive probe; after that,
then resend it every 75 seconds. If no ACK response is received for nine consecutive times,
the connection is marked as broken. This length of time might be too long for the server and
could result in excess memory usage and a decrease in server performance. Setting it to
1800 seconds (30 minutes), for example, might be more appropriate:
net.ipv4.tcp_keepalive_time=1800.

• When the WebLogic Server is heavily loaded or has many
clients with bad connections with high latency, it can result
in an increase in half-open connections.
• These half-open connections are stored in the backlog
connections queue.
– You should set this value to at least 4096
(net.ipv4.tcp_max_syn_backlog=4096 ). The default is
1024.

• For servers that sustain many connections at the same
time, the TIME-WAIT sockets for new connections can be
reused.
• This is useful in Web servers; you should set:
net.ipv4.tcp_tw_reuse=1. If you enable this
parameter, you should also enable fast recycling of
TIME-WAIT sockets status:
net.ipv4.tcp_tw_recycle=1.
With these parameters enabled, the number of connections is significantly reduced. This is
good for performance because each TCP transaction maintains a cache of protocol
information about each of the remote clients. In this cache, information such as round-trip
time, maximum segment size, and congestion window are stored.

Section Summary
In this section, you should have learned about configuring

operating system parameters (mainly TCP/IP) to improve
performance

Analyzing and Tuning Linux Kernel Parameters
This practice covers changing Linux kernel parameters that
affect the way the system handles a large number of
connections.

Tuning HotSpot JVM

Objectives

following:
• Describe JVM performance characteristics
• Describe JVM garbage collection algorithms
• Monitor and tune JVM garbage collection

• Use HotSpot ergonomics
• Tune JVM heap settings
• Use command-line JVM monitoring tools
• Use GUI JVM monitoring tools

Assumptions and Expectations
This lesson makes the following assumptions:

• Students already know what Java is.
• Students already know what a JVM is and have a basic
understanding of its architecture.
• Students are already familiar with basic concepts of Java

garbage collection.
• This lesson provides an overview of JVM tuning and how it
relates to WebLogic Server. Students should refer to the
three-day JVM Tuning course provided by Oracle
University for complete details.

Agenda
• JVM Performance: Overview

• The JVM and Java Garbage Collection
• Command-Line JVM Tools
• GUI JVM Tools

What Is Performance?
This table shows the different aspects of JVM performance:
Aspect Description
Memory Footprint The amount of memory used by an application and the JVM
Startup Time The time taken for an application to start
Scalability How an application performs as the load and resources

increase
Responsiveness How quickly an application responds with requested data
Throughput The amount of work an application can perform in a specific

period of time
There are several ways in which you can define performance. Some of these aspects of
performance impact the JVM.
Each of these aspects should be a well-defined requirement for an application. In addition, the
importance should be prioritized for the application. This clarification is an important input to
the folks who are tackling the application's performance issues.
Memory Footprint: It is very important to know the environment and ecosystem in which your
application runs. Shared resources can have a significant impact on performance.
Virtual memory swapping should be minimized for any Java application. Consider a scenario
where a garbage collection occurs with a large portion of the Java heap in virtual memory.
Scanning for referenced objects would take much longer than when the heap is in physical
memory. It is very important to configure a system and the Java heap to avoid virtual memory
swapping. Questions to ask yourself include:
• Does the application run on a dedicated system?
• Does the application share the machine with other applications and JVMs?

Startup Time: Startup time is very important for client applications that tend to get started
over and over again. Server applications do not place the same importance on fast startup
time because they are expected to run for longer periods of time after starting up initially.
However, when servers are restarted frequently during the iterative development process, fast
startup time is important.
Scalability: Scalability can apply to not only an application but also to many other aspects of
a system of which the application is a part.
Tip: It is very important to have a development or qualification environment that replicates
production environment situations. This reduces the chance to be caught off-guard with an
application that does not scale well.
Responsiveness: Responsiveness can be measured in a number of ways. But the bottom
line is that
typically a responsive
more application
focused on returns Long
responsiveness. requested
pausedata
timequickly.
are notClient applications are
acceptable.
Throughput: Server applications typically focus more on throughput and less on
responsiveness. High pause times are more acceptable, but still avoided if at all possible.

Performance Focus for This Lesson
This lesson focuses on:

• Java application performance
• Optimizing the JVM for throughput or responsiveness
• Discovering, troubleshooting, and tuning Java performance
issues

HotSpot JVM Monitoring Tools
Management tools provided with HotSpot JVM include:

Tool Description
VisualVM, jconsole, Mission Control, JMX-compliant JVM monitoring and

and Flight Recorder management consoles
jps JVM process status tool
jcmd JVM process status and management

tool
jmap and jhat Tools that dump and display heap and
shared object memory map
jstack Tool that prints Java stack traces of Java
threads
You can use the tools listed in the slide to monitor JVM performance and resource
consumption. You can also use some of the tools for troubleshooting. Java VisualVM,
jconsole, Mission Control, and Flight Recorder are GUI-based tools that require X-Windows
in Linux and UNIX environments. All of the other tools are command line–based tools.

Section Summary
In this section, you should have learned how to describe JVM

performance characteristics.

Agenda

• GUI JVM Tools

Garbage Collection Basics
Al i veo b j e c t U nr e f e r e nc e dO b j e c t s M e mo rys p ac e
Before marking
After marking
Free After normal

memory deletion
After normal
deletion with
compacting
Garbage collection is a way in which Java recollects the space occupied by loitering objects.
In general, a garbage collector is responsible for three tasks:
• Allocating memory for new objects
• Ensuring that any referenced objects (live objects) remain in memory
• Recovering memory used by objects that are no longer reachable (dead objects)
Marking: The first step that the garbage collector performs is called marking. The garbage
collector iterates each object one by one through the application graph, checks whether the
object is being referenced, and if so marks the object as being used. The marked objects will
not be deleted in the sweeping stage.
Normal Deletion: The deletion of objects happens in the sweeping stage. The traditional and
easiest way is to mark the space as free and let the allocator use complex data structures to
search the memory for the required free space.
Compacting: It is obvious that the traditional way of freeing memory has many problems
associated with it. An improved way is by providing a defragmenting system that compacts
memory by moving objects closer to each other and removes fragments of free space, if any.
In this way, the allocation of future objects is much faster.

Generational Garbage Collection
The Java heap is divided into three sections:

Objects are allocated in Eden. One
survivor space is empty at a time. Live
objects are copied from Eden and the
other survivor space until they are old Objects that survive enough GC cycles in the
enough to copy to tenure. young generation are moved here.
GC occurs quickly and frequently here. GC occurs less frequently and can take longer.
Application classes, Java language

classes, and methods are loaded
Survivor Space into PermGen.
Eden S0 S1 Tenured Permanent
Young Generation Old Generation Permanent Generation
Non-generational garbage collectors iterate over every object in the heap and check whether
or not the object has some active references to it. As the number of objects increases in the
heap, this process would take a longer time to complete, and therefore, would be inefficient.
A careful observation of a typical object would tell us the following two characteristics:
• Most allocated objects will die young.
• Few references from older to younger objects exist.
To take advantage of this, the Java HotSpot VM splits the heap into different physical areas,
which are called generations. HotSpot uses a type of garbage collection that is termed
generational. This means that the Java heap is partitioned into generational spaces. The
default arrangement of generations (for all collectors with the exception of the throughput
collector) looks something like the image in the slide.
There are three types of generational spaces:
• Young Generation
• Tenured Generation
• Permanent Generation

Each of these generations hold objects of different ages. Different algorithms are used to
perform garbage collection in different generations, and each algorithm is optimized based on
commonly observed characteristics for that particular generation.
Garbage collections for the young generation occur relatively frequently and, therefore, must
be fast. So the young generation space is usually small and likely to contain objects that are
no longer referenced. The young generation consists of Eden and two smaller survivor
spaces. Most objects (except some large objects) are initially allocated in Eden. The survivor
spaces hold objects that have survived at least one young generation collection. At any given
time, one of the survivor spaces holds such objects, while the other is empty and remains
unused until the next collection.
Objects that survive some number of young generation collections are tenured. This
generation is typically larger than the young generation and its occupancy grows more slowly.
The garbage collections in tenured generation are infrequent, but may take significantly
longer to complete.
The permanent generation is where application Java classes, language classes, and method
objects are stored. If an application loads a very large number of classes, the size of the
permanent generation might need to be increased by using the -XX:MaxPermSize option.

Garbage Collectors: Java Heap Options
-Xmx -XX:MaxPermSize
-Xms -XX:PermSize
-XX:NewSize
r r r
e e e
s s s
e e e
r r r
v
e
To From Eden Tenured v
e
Perm v
e
d d d
-XX:MaxNewSize
Young Generation Old Generation Permanent

Generation
The various options available to size the heap are as follows:

• -Xmx<size> is the maximum size of Java heap (young generation + tenured
generation).
• -Xms<size> is the initial size of Java heap (young generation + tenured generation).
• Applications with emphasis on performance usually set -Xms and -Xmx to the same
value.
• When -Xmx ! = -Xms, Java heap growth or shrinkage requires a full garbage
collection.
• -Xmn<size> is the size of the young generation heap space.

Setting Heap Memory Size
• The –Xms value is the space in memory that is committed

to the VM at startup.
• The JVM can grow up to the size of the–Xmx value.
• The difference between -Xmx and -Xms is virtual memory
(virtually committed).
$ export USER_MEM_ARGS="–Xms1g –Xmx2g"
$ startManagedWebLogic.sh myserver ...
Command Line
• For maximum performance, set minimum and maximum

heap values the same.
$ export USER_MEM_ARGS="–Xms2g –Xmx2g"

$ startManagedWebLogic.sh myserver ... Command Line

Setting Heap Size Options
Property Description
-Xms<size> Initial heap size
-Xmx<size> Maximum heap size
-Xmn<size> Sets initial NewSize and MaxNewSize the same
-XX:PermSize=<size> Initial size of the permanent generation
-XX:MaxPermSize=<size> Maximum size of the permanent generation
-XX:NewSize=<size> Initial size of the young generation
-XX:MaxNewSize=<size> Maximum size of the young generation
-XX:NewRatio=<ratio> Ratio of young generation space to tenured generation space
-XX:SurvivorRatio=<ratio> Ratio of Eden and survivor spaces in young generation
Configuring the Java HotSpot heap size options when starting WebLogic Server increases
performance for most applications. These options may differ depending on your architecture
and operating system.
• -Xms
- This parameter configures the starting and minimum size of the Java heap.
- As a general rule, set this value to be equivalent to the maximum heap size
parameter (Xmx) to avoid expensive growing and shrinking of the heap.
• -Xmx
- This parameter configures the maximum size of the Java heap.
- Set this value to accommodate all the objects of your application. When the
objects in memory require more space than specified by the–Xmx value, you get
“Out of memory” errors.
- If a value is too large, it can deprive memory for other services running on the
computer and can cause paging and thrashing.

• -Xmn
- Applications with emphasis on performance tend to use -Xmn to size the young
generation because it combines the use of -XX:MaxNewSize and
-XX:NewSize and almost always explicitly sets -XX:PermSize and
-XX:MaxPermSize to the same value. A growing or shrinking permanent
generation space requires a full garbage collection.
• -XX:PermSize
- This parameter configures the initial size of the permanent generation.
- Set this value to be equivalent to the maximum permanent generation size to avoid
expensive growing and shrinking of the heap.
• -XX:MaxPermSize
-
This parameter configures the maximum size of the permanent generation.

- If this value is not set high enough, your entire application server can crash with an
out of memory error.
- Unintentionally referenced objects left behind by component classloaders can
cause memory leaks in the permanent generation space.
• -XX:NewSize
- This parameter configures the initial size of the young generation.
- Set this value to be one-fourth the size of the maximum heap size. Increase the
value of this option for larger numbers of short-lived objects.
• -XX:MaxNewSize
- This parameter configures the maximum size of the young generation.
- Set this value to be one-fourth the size of the maximum heap size. Increase the
value of this option for larger numbers of short-lived objects.
• -XX:NewRatio
- Specifies the ratio between the young and tenured generation heap sizes.
- Set the ratio to 3 to set the young generation to 1/4 of the heap—one part young
and three parts old.
• -XX:SurvivorRatio
- Specifies the ratio between survivor and Eden areas in the new generation.
- The New generation area is divided into three subareas: Eden and two survivor
spaces that are equal in size. Set the ratio to 6 to set each survivor space to 1/8 of
the Eden size.
- If survivor spaces are too small, copying collection overflows directly into the old
generation. If survivor spaces are too large, they will be empty. At each GC, the
JVM determines the number of times an object can be copied before it is tenured,
called the tenure threshold. This threshold is chosen to keep the survivor space
half full. Use the -XX:+PrintTenuringDistribution option to show the
threshold and ages of the objects in the new generation.

G1 Garbage Collector
The Garbage-First (G1) collector is a server-style garbage

collector, targeted for multi-processor machines with large
memories.
Meets GC pause time goals with a high
probability, while achieving high throughput
O O E E O • Operates concurrently with applications
threads like the CMS collector
E O O S • Compacts free space without lengthy
GC-induced pause times
E O S O • Provides more predictable GC pause
durations
O E E O O
O S S O O
E Eden Space
O O E O
S Survivor Space
O Old Generation
The heap is one memory area split into many fixed sized regions. Region size is chosen by
the JVM at startup. The JVM generally targets around 2000 regions varying in size from 1 to
32 MB. Regions are not required to be contiguous like the older garbage collectors.
A remember set is kept for each region, which is a list with references to objects. Threads
inform the garbage collector when they change a reference, which can change the remember
set. When a garbage collection occurs, the areas containing the most free space (or garbage)
are collected first, hence the nickname, garbage-first. The idea behind this method is that the
most common scenario for garbage collection will be that several regions will exist that
contain no living objects. The garbage collector can then simply free the regions without
requiring the mark-and-sweep or compacting processes, which speeds up GC performance.

The G1 collector provides settings to define overhead and pause time targets. The collector
then sweeps only the number of areas it calculates it can complete within that target setting.
These regions are mapped into logical representations of Eden, Survivor, and old generation
spaces. The colors in the picture shows which region is associated with which role. Live
objects are evacuated (copied or moved) from one region to another. Regions are designed to
be collected in parallel with or without stopping all other application threads.
Regions can be allocated into Eden, Survivor, and old generation regions. In addition, there is
a fourth type of object known as Humongous regions. These regions are designed to hold
objects that are 50% the size of a standard region or larger. They are stored as a set of
contiguous regions. Finally, the last type of regions would be the unused areas of the heap.

GC Algorithms
These are the HotSpot VM command-line switches that affect

GC algorithms:
Command-Line Switch GC Method Chosen

-XX:+UseSerialGC Serial
-XX:+UseParallelGC Parallel
-XX:+UseParallelOldGC Parallel compacting
-XX:+UseParNewGC Parallel Young collector
-XX:+UseConMarkSweepGC Concurrent mark-sweep
-XX:+UseG1GC Garbage First (G1)
With the advent of Ergonomics, most command-line switches are redundant. However, in
some specific cases, you may need to use these switches. The table shows HotSpot VM
options with the option description and the associated generation.
• Serial: The serial collector is a single-threaded, stop the world, garbage collector. It is
best used on single CPU systems where multithreading collection would not provide
increased performance.
• Parallel: Considering that hardware resources (CPU and memory) are getting to be
relatively inexpensive, the parallel collector, also known as the throughput collector,
enables you to take advantage of available CPUs rather than leaving most of them idle
while only one does garbage-collection work.
• Parallel compacting: The difference between the parallel compaction collector and the
parallel collector is that the parallel compaction collector uses a young algorithm for
garbage collection of the old generation. With the parallel compaction collector, the old
and permanent generations are collected in a mostly parallel fashion with sliding
compaction. By default, only minor collections are executed in parallel and major
collections are performed with a single thread. This option enables parallel compaction
for major and minor collections.

• Parallel young collector: This parallel collector performs parallel garbage collection on
the young generation. It is used in conjunction with the CMS collector for the old
generation.
• Concurrent mark-sweep (CMS): For many applications, fast response time is the most
important performance criteria. To facilitate this, you can set up a CMS collector, also
known as the low-latency collector. Generally, GC for the young generation does not
typically cause long pauses. However, old generation collections, though infrequent, can
impose long pauses. So most of the collection of the old generation using the CMS
collector is done concurrently with the execution of the application.
• Garbage first: The G1 collector is the newest collection algorithm and is meant for high-
performance server-side applications. It optimizes collection time by avoiding expensive
processing associated with other collection methods. It also optimizes collection time by
only collecting what it can within specified performance targets. The permanent
generation is still collected by using the CMS collector.

GC Performance Goals
The following are the different ways to measure GC

performance:
Performance Goal Description
Throughput Percentage of time not spent in GC over a long period of

time
Responsiveness How quickly an application responds in a given scenario
Footprint Overall memory a process uses to execute
WebLogic Server applications almost

focus on footprint performance!
Throughput is the percentage of total time not spent in garbage collection, considered over
long periods of time. Throughput includes time spent in allocation (but tuning for speed of
allocation is generally not needed). Pauses are the times when an application appears
unresponsive because garbage collection is occurring. A responsive application attempts to
reduce the length of these pauses.
Users have different requirements of garbage collection. For example, some consider the
right metric for a web server to be throughput, because pauses during garbage collection may
be tolerable, or simply obscured by network latencies. However, in an interactive graphics
program even short pauses may negatively affect the user experience.
Some users are sensitive to other considerations. Footprint is the working set of a process,
measured in pages and cache lines. On systems with limited physical memory or many
processes, footprint may dictate scalability.
There are numerous ways to size generations. The best choice is determined by the way the
application uses memory as well as user requirements.

Focusing on Throughput
Throughput:
• Has as highest priority the raw throughput of the
information or data being processed
• Maximizes application throughput even at the expense of
responsiveness
• Will tolerate high pause times in order to maximize
throughput
A Java application that focuses on throughput emphasizes the raw throughput of the
information or data being processed. This is the most important quality for the application.
Pause times resulting from JVM garbage collection events are not an issue, or of very little
interest. As long as the overall throughput of the application over a period of time is not
sacrificed, long pause times are allowed. Examples of applications that focus on throughput
include:
• A large phone company printing bills or statements
• A large credit card company printing statements
• A bank calculating interest for accounts

Focusing on Responsiveness
Responsiveness
• Have as highest priority the servicing of requests within a
predefined maximum time
• Raw throughput of data or speed of processing requests
are secondary to max response time goal

• Are sensitive to GC pause time
A Java application that focuses on responsiveness emphasizes how quickly an application

responds in a given scenario rather than focusing on the raw throughput of the application.
Most applications emphasizing responsiveness have a maximum pause time the application
can tolerate. Examples of applications that focus on responsiveness include:
• Applications connected to a user interface such as a web browser or an IDE
• Financial trading applications
• Telecommunication applications
Java applications focusing on responsiveness are sensitive to the time it takes for garbage
collection events to complete.

Evaluating GC Algorithm
C ri t e ri a Description
Pause time Does the collector introduce pauses by stopping the world?
Memory footprint What is the total memory allocated versus the total memory available
to the application?
Virtual memory usage and How efficiently does the garbage collector manage the reference for
interaction pages outside the resident memory during a full GC?
Parallel Does the collector make use of multiple CPUs if available?
CPU usage What is the percentage of CPU spent in garbage collection?
Concurrency Is there a contention for a CPU between application threads and the
garbage collector?
Can this be overcome by use of systems with multiple CPUs?
Compiler Are compile-time operations causing issues?

GC Tuning Tips +
Heap size influences:
An undersized heap with the
• GC frequency and the pause during concurrent collector leads to full
collections GCs with an increase in load and
• The number of short- and long-term objects fragmentation problems.
• Fragmentation and locality problems
An oversized heap leads to

increased collection times
and locality problems.
Size the heap to handle
peak and burst loads.
Sizing the permanent generation is

important for applications that dynamically
Increase memory as you increase generate and load many classes.
the number of processors, because
allocation can be parallelized.
Setting PermSize and MaxPermSize
to the same value is recommended!
Increasing the size for permanent generation can be helpful if there are numerous
dynamically generated classes. However, you should bear in mind that you should increase
-XX:MaxPermSize along with -XX:PermSize. To tune this, you may set-XX:PermSize
as high as -XX:MaxPermSize and then reduce it to an appropriate level when you know
your typical requirements.

GC Tuning Tips +
Configure as much memory as

possible to the virtual machine unless
you have problems with pauses.
Increased object lifespan

Explicit garbage collection increases GC frequency
calls (System.gc() ) force because live objects take
major garbage collections. up heap space. So keep
live objects to a minimum.
Measure the effectiveness of

Use the default serial collector
explicit GC calls by disabling them
using -XX:+DisableExplicitGC for smaller applications.
If you want to disable calls to request GC manually, set -XX:+DisableExplicitGC on the

command line while starting the W ebLogic Server. This disables calls to System.gc()and
the distributed Remote Method Invocation (RMI) GC, but JVM performs garbage collection
when necessary.

GC Tuning Tips +
Applications that rely on

finalization cause lags in The concurrent collector
garbage collection. advantages increase with
Avoid using finalization. the number of CPUs.
Use Intimate Shared Memory

(ISM) and variable page sizes For larger applications hosted
to reduce the smear problem on WebLogic Server, use the
where available: throughput collector:
-XX:+UseISM -XX:+UseParallelGC
For more information about tuning HotSpot JVM, refer to GCPortal at:
http://www.oracle.com/technetwork/articles/javase/gcportal-
136937.html
GCPortal enables a service, Application Modeling, and Performance Tuning from a GC
perspective. It is implemented in J2EE and available as a J2EE W AR module that can run on
any J2EE Application Server. It allows developers to submit log files and analyze application
behavior.
You can use GCPortal to performance tune and size the application to run optimally under
lean, peak, and burst conditions.
To find more information about aggressive options, use -XX:+PrintCommandLine.

Intimate Shared Memory (ISM) increases the page size from 8 KB to 4 MB. It uses real
addresses not virtual addresses. It locks pages in shared memory and eliminates paging to
disk. A performance boost of 5-10% is possible by using ISM. However, this is not guaranteed
and you should monitor the performance of your application to be sure.
ISM warnings:
• Is a Solaris-only feature
• Locks memory and should be used only on dedicated systems
• Memory fragmentation may prevent allocation of contiguous 4-MB pages.
• Use ipcs or ipcrm to discover and delete any locked segments due to abnormal JVM
terminations.
Refer to the following URL for further details about various parameters that you can use with
Java 7 SE:
• http://java.sun.com/javase/7/docs/technotes/tools/windows/java.h
tml#nonstandard

Server-Class Machine Detection +
The JVM can detect if it is running on a server-class machine.
Running on an Intel or SPARC 32-bit platform.

JVM Launch 64-bit defaults to server VM mode.
-server No -client Yes Client Class

set? set? Machine
Yes No
CPUs > 1 No
&& Mem >=
2GB? Setting
Yes PRODUCTION_MODE
Server Class to true causes
Machine
WebLogic to start
using -server option.
In J2SE 5.0 or later, when a machine with at least two CPUs and at least 2 GB of physical
memory is used, it is said to be a server-class machine.
During JVM launch, the JVM detects whether the application or server is running in a
server-class machine; if so, the Java HotSpot Server VM is used.
Server-class detection occurs if you do not specify -server or -client when launching
the application on an Intel or SPARC 32-bit machine running Solaris or Linux. Windows
machines always default to the client VM. All 64-bit platforms default to the server VM.
The PRODUCTION_MODE parameter in setDomainEnv.sh can be set to “true” to start server
mode JVM.
The client VM focuses on faster start up time for iterative development and faster response
time for users. This VM starts up quickly, but may not perform as fast as the server VM over
time.
The server VM starts up more slowly than the client VM, but performs faster over time.

Ergonomics: What It Does
• Evaluates the system and auto-magically chooses defaults

for the HotSpot JVM. No tuning required.
• Relies on definition of “server class machine”
– 2 or more processor cores and 2 or more GB of physical
RAM
• Server class machines will use -server JIT compiler.
• Nonserver class machines are considered "client class."
The J2SE Platform version 1.5 or later offers a feature called ergonomics. The goal of
ergonomics is good JVM performance with a minimum of command-line tuning. When
ergonomics is enabled, the JVM attempts to match the best settings for an application.
On server-class machines, by default, the following are selected:
• Server JIT compiler
• Throughput garbage collector
• Heap sizes
• Initial heap size of 1/64 of physical memory up to 1 GB
• Maximum heap size of 1/4 of physical memory up to 1 GB
If not identified as a server class, then runs as a client class:
• Client JIT compiler
• Default serial collector, same as before

Section Summary

• Tune JVM heap settings
• Describe JVM garbage collection algorithms
• Monitor and tune JVM garbage collection
• Use HotSpot ergonomics

Tuning JVM Garbage Collection
• Configuring different heap settings
• Configuring different garbage collectors
• Loading an application with each setting
• Capturing and comparing performance metrics
• Analyzing performance results and trying to answer some
questions

Agenda

• GUI JVM Tools

Using jps
• Command-line utility to find running Java processes

• Included in the HotSpot JDK
• Capable of local and remote monitoring
jps [-q] [-mlvV] [<hostid> where <hostid> =

<hostname>[:<port>] Command Line
jps is very similar to ps in UNIX. Its purpose is to identify the process ID of Java processes.
If jps is run without specifying a hostid, it looks for instrumented JVMs on the local host. If
jps is started with a hostid, it looks for JVMs on the indicated host, using the specified
protocol and port. This can be used to retrieve the virtual machine identifier (vmid) or Local
Virtual Machine Identifier (lvmid), which can be used as an input for other commands.
Note: See the jps man page on oracle.com for details about-q, -mlvV options:
http://download.oracle.com/javase/7/docs/technotes/tools/share/jps.html

Using jps: Example
• Output from a jps command with no options lists each

Java application’s lvmid followed by the short form of the
application’s class name or JAR file name.
• The -l option outputs the full package name for the
application’s main class or the full path name to the
application’s JAR file.
lvmids
$ jps
19518 NetworkServerControl
22287 Console
22334 Grinder
These are WebLogic Server instances.
5155 jps You use other command-line options with
19719 Server jps to determine which server is which.
19539 Server
Command Line
The list of JVMs produced by the jps command may be limited by the permissions granted to
the principal running the command. The command lists only the JVMs for which the principal
has access rights as determined by operating system–specific access control mechanisms.
This command is important to retrieve thelvmids of the JVMs running so that you can use
them in other commands.
The screenshot shows the result of ajps call.

Using jcmd
• Utility to send diagnostic commands to a running JVM

– Gets list of PIDs like jps
– Provides a lot of query options for running JVMs
• Basic usage
$ jcmd
$ jcmd <pid> command
$ jcmd 0 command Command Line
The jcmd (pronounced "j command") utility is a new diagnostic tool that can look up process
IDs of running JVMs and also send diagnostic commands to a running JVM. The following are
some example jcmd commands.
• jcmd: Returns the PID of all running JVMs on the system
• jcmd 1234 VM.flags : Displays the command-line options used to start the JVM with a
PID of 1234
• jcmd java2 VM.flags : Displays the command-line options used to start the JVM
whose name includes the string "Java2"
• jcmd 0 VM.flags : Displays the command-line options used to start all the running JVM
instances on this machine

Using jcmd: Examples
• Getting a list of options

$ jcmd 1234 help
$ jcmd 1234 help -all
Command Line
• JVM information
$ jcmd 1234 VM.uptime
$ jcmd 1234 VM.command_line
$ jcmd 1234 VM.version
$ jcmd 1234 VM.system_properties Command Line
• Threads Shows deadlocks if they exist
$ jcmd 1234 Thread.print Command Line
• jcmd 1234 help : Lists all the options available on this JVM
• jcmd 1234 help -all : Lists all the options available on this JVM and prints a brief
description of each option
• jcmd 1234 VM.uptime : Displays the amount of time the JVM has been up
• jcmd 1234 VM.command_line : Shows the command used to start the JVM and the
command-line options
• jcmd 1234 VM.version : Shows the version of the JVM
• jcmd 1234 VM.system_properties : Displays the system properties for the
selected JVM

Using jcmd: Histogram
$ jcmd 1234 GC.class_histogram

num #instances #bytes class name
----------------------------------------------
1: 29775 3865168 <constMethodKlass>
2: 29775 3825648 <methodKlass>
3: 1110 3074312 [I
4: 2309 2903976 <constantPoolKlass>
5: 2309 2172472 <instanceKlassKlass>
6: 2210 1985184 <constantPoolCacheKlass>
7: 10778 859312 [C
8: 3286 788120 [B
9: 1177 525640 <methodDataKlass>
10: 2546 315248 java.lang.Class
11: 3836 261224 [S
12: 10376 249024 java.lang.String
Command Line
jcmd 1234 GC.class_histogram: Displays a list of classes taking up the most memory.
You may want to redirect the output to a file because it can be very long.

Using jinfo
jinfo prints Java configuration information for a given Java

process or core file or a remote debug server.
$ jinfo 19719
Attaching to process ID 19719, please wait...
Debugger attached successfully.

Server compiler detected.
JVM version is 24.0-b56
Java System Properties:
java.vendor = Oracle Corporation
. . .
VM Flags:
-Xms256m -Xmx512m -XX:MaxPermSize=256m -Dweblogic.Name=server1 ...
-Dweblogic.ProductionModeEnabled=true ...
-Dwls.home=/u01/app/fmw/wlserver/server
-Dweblogic.home=/u01/app/fmw/wlserver/server
-Dweblogic.management.server=host01:7001
Command Line
Refer to the following URLs for more information:

http://docs.oracle.com/javase/7/docs/technotes/tools/share/jinfo.html
The example in the slide showsjinfo displaying the system properties and flags of a JVM
running WebLogic.
Syntax: jinfo [ option ] pid | executable core |[server-id@]remote-
hostname-or-IP
where
• pid: Java process ID
• executable: Java executable from which the core dump was produced
• core: Core file for which the configuration information is to be printed
• server-id: Unique server ID
• remote-hostname-or-IP: Remote server’s IP or hostname

Using jstat
• Command-line utility that runs on a local or remote JVM

• Included in the HotSpot JDK
$ jstat -<option> [-t] [-h<lines>] <vmid> [<internal>
[<count>]]
Command Line
• Garbage collection options:

-gc, -gccapacity, -gccause, -gcnew, -gcnewcapacity,
-gcold, -gcoldcapacity, -gcpermcapacity, -gcutil
jstat is a command-line tool that displays detailed performance statistics for a local or
remote HotSpot VM. The–gcutil command-line option is used most frequently.
For information about garbage collection options, see the man page at:
http://docs.oracle.com/javase/7/docs/technotes/tools/share/jstat.html
Caution: When using the CMS collector (also known as concurrent collector), jstat reports
two full GC events per CMS cycle, which is obviously misleading. However, young generation
stats are accurate with the CMS collector.

Using jstack
• You can use jstack to print a stack trace of all the

threads currently running in a VM.
• jstack is different from the normal thread dump process
in the following ways:
– Prints thread stack for a VM running remotely
– Prints thread stack from the core file
– Prints thread stack on stdout where the command is
running, whereas thread dump (kill -3 pid or Ctrl-Break)
prints thread stack trace on the VM’s stdout
– Is a vendor-dependent, platform-dependent implementation,
whereas thread dump is not
Syntax:
jstack [ option ] pid | executable core |[server-id@]remote-
hostname-or-IP
• pid: Java process ID
• executable: Java executable from which the core dump was produced
• core: The core file for which the configuration information is to be printed
• server-id: Unique server ID
• remote-hostname-or-IP: Remote server’s IP or hostname
For each Java frame, the full class name, method name, “bci” (byte code index), and line
number, if available, are printed.
Refer to the following URL for more information:
http://docs.oracle.com/javase/7/docs/technotes/tools/share/jstack.html

jstack
jstack prints Java stack traces of Java threads for a given

Java process, core file, or a remote debug server.
$ jstack -l 19719
2013-09-23 09:32:53
Full thread dump Java HotSpot(TM) 64-Bit Server VM (24.0-b56 mixed mode):
. . .
"main" prio=10 tid=0x00007f90ac009800 nid=0x4d08 in Object.wait()
[0x00007f90b1cfc000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000e02c4c18> (a weblogic.t3.srvr.T3Srvr)
at java.lang.Object.wait(Object.java:503)
at weblogic.t3.srvr.T3Srvr.waitForDeath(T3Srvr.java:995)
- locked <0x00000000e02c4c18> (a weblogic.t3.srvr.T3Srvr)
at weblogic.t3.srvr.T3Srvr.run(T3Srvr.java:492)
at weblogic.Server.main(Server.java:74)
Locked ownable synchronizers:- None“
VM Thread" prio=10 tid=0x00007f90ac066800 nid=0x4d0b runnable
"GC task thread#0 (ParallelGC)" prio=10 tid=0x00007f90ac01f800 nid=0x4d09 runnable
"GC task thread#1 (ParallelGC)" prio=10 tid=0x00007f90ac021800 nid=0x4d0a runnable
"VM Periodic Task Thread" prio=10 tid=0x00007f90ac09e000 nid=0x4d12 waiting on
condition
JNI global references: 254 Command Line
A stack trace of all threads can be useful when you are trying to diagnose a number of issues
such as deadlocks or hangs. Deadlocks and hangs may consume a server’s resources more,
which may let the user request deprive or starve. Diagnosing and eliminating these issues
from applications deployed on WebLogic Server increases the performance of the server.
The following are the possible thread states:
• NEW: Thread has been created, but it has not started running yet.
• IN_NATIVE: Thread is running native code.
• IN_VM: Thread is running VM code.
• IN_JAVA: Thread is running (either interpreted or compiled) Java code.
• BLOCKED: Thread is blocked.
• ..._TRANS: If you see any of the preceding states followed by “_TRANS,” it means that
the thread is changing to a different state.
• UNINTIALIZED: Thread is not created. This will normally not happen (unless there is a
serious bug such as memory corruption).
The screenshots show an example of runningjstack.

Section Summary
In this section, you should have learned how to use command

line JVM monitoring tools.

Using JVM Command Line Tools
This practice covers experimenting with different command-line
tools to monitor WebLogic Server JVM processes.

Agenda

• GUI JVM Tools

Java Monitoring and Management Architecture
JMX-based
Management
Java application Application
(Example: Java
Instrumentation VisualVM)
Platform
MBean
Server
Instrumentation
SNMP
Java Virtual Machine MIB
SNMP-based
Management
Operating system Application
(Example: HP
Network Node
Manager)
The graphic shows the Java Management Extensions (JMX)-based management applications
and Simple Network Management Protocol (SNMP)-based management applications
connecting to Java applications facilitating remote management.
Java has greatly expanded JVM monitoring and management support capabilities and
included the Java VisualVM, jconsole, Mission Control, and Flight Recorder tools to take
advantage of these capabilities.
When an application is run with JMX enabled, a JMX Agent process is activated within the
JVM to manage JMX requests. JMX clients can then locate and connect to the JMX agent
and query it for the available management resources.
JMX was later complemented with the JMX Remote API (JSR 160), which allowed remote
clients to access the JMX agent. The JMX Remote API includes access control and secure
sockets layer (SSL) support, which makes it secure enough for use in production.
An MBean is a managed object that follows the design patterns conforming to the JMX
specification. An MBean can represent a device, an application, or any resource that must be
managed. The management interface of an MBean comprises a set of readable and writable
attributes, and a set of invocable operations. MBeans can also emit notifications when
predefined events occur.

Java VisualVM
Java VisualVM is a JMX-compliant GUI tool that enables you to

monitor and manage Java applications and JVM.
Java VisualVM, first made available with JDK version 6, update 7, is a graphical user
interface that provides information about Java applications and the JVM on which the
applications run. Java VisualVM federates several monitoring, troubleshooting, and profiling
utilities such as jmap, jinfo, and jstack to obtain data from the JVM software, and then
reorganizes and presents the information graphically.
You can view different data about multiple Java applications uniformly, whether they are
running locally or on remote machines.
Java VisualVM can be used by Java application developers to troubleshoot applications and
to monitor and improve the applications’ performance. Java VisualVM can allow developers to
generate and analyze heap dumps, track down memory leaks, browse the platform’s MBeans
and perform operations on those MBeans, perform and monitor garbage collection, and
perform lightweight memory and CPU profiling.

Java VisualVM Connections
Java VisualVM can connect to running JVMs:

• Local: Connects to JVMs running on the local system
• Remote: Connects to a JMX agent by using an RMI connector
• Advanced: Connects to a JMX agent by using a non-RMI
connector
When started without any arguments, Java VisualVM automatically detects all JVMs running
locally. In a production environment, it is recommended to run Java VisualVM on a separate
machine so as to minimize contention for system resources. The screenshot shows Java
VisualVM connected to two local JVM instances.
Note: A non-RMI connector could be SNMP based.

Remote Monitoring
• You can use Java VisualVM to monitor and manage

remote applications and JVM.
• Ensure that the jstatd agent is running on the remote
machine that you want to monitor.
• For security,
password when remote
authentication monitoring
over is enabled,
SSL is enabled by default.
You can use Java VisualVM to monitor applications and JVMs running on remote hosts. Java
VisualVM can display general data about the application’s run-time environment and can
monitor memory heap and thread activity; however, Java VisualVM cannot profile remote
applications.
You can add a remote host as follows:
1. Right-click the Remote node in the Applications window.
2. Select Add Remote Host.
3. Enter the host name or IP address in the Add Remote Host dialog box. (You can also
specify a display name that will be used to refer to the host when listed under the
Remote node.)
Connected remote hosts are displayed as nodes and the applications running on the host as
subnodes under the Remote branch of the Applications window.

Java VisualVM Interface
The Applications
window contains a
tree of JVMs that are
visible to Java
VisualVM.
The screenshot shows the Java VisualVM interface after connecting to a HotSpot JVM. There
are two main panels in this interface: the Applications window and the Monitoring Panel.
The Applications window:
• The Applications window is the main point of entry for exploring the details of running
JVMs. This window uses a tree structure to enable you to quickly view the JVMs
(running on the local and any connected remote systems) that are visible to this Java
VisualVM. In Solaris and Linux systems, you can also access core dumps and saved
snapshots from the Applications window.
• You can right-click a node in the Applications window and use the pop-up menu to
perform actions related to that node, including opening application tabs, taking heap and
thread dumps, and opening snapshots in the main window.
The Monitoring Panel contains statistics and graphs on the applications to which you have
connected.

Monitoring JVM
The Monitoring tab provides four detailed graph sections:

• CPU: The CPU graph shows CPU trends in historic intervals in real time, and GC
activity in correlation to CPU statistics.
• Heap/PermGen: The Heap graph displays the total heap size and how much of the
heap is currently used. The PermGen graph displays changes in the permanent
generation area over time.
• Classes: The Classes graph displays an overview of the total number of loaded classes
and shared classes.
• Threads: The Threads graph displays an overview of the number threads in the
application’s JVM. You can use Java VisualVM to take a thread dump if you want to
capture and view more detailed data on application threads at a specific point in time.
In addition, you can manually force a garbage collection or dump heap. When you initiate a
heap dump, a Heap Dump tab appears in the monitored JVM or application. Also, the heap
dump node appears under the application or JVM node.

Monitoring Threads +
The Threads tab presents high-level data on thread activity. The Threads tab is visible if Java
VisualVM can make a JMX connection and retrieve JMX instrumentation from the JVM.
If the target is local and based on Java 6 or later, then the JMX connection is made
automatically. If the application is running on an older version, you may need to explicitly
establish a JMX connection with the JVM software.
By default, the Threads tab displays a timeline of the current thread activity. You can click a
thread in the timeline to view details about that thread on the Details tab.
Using the Thread Dump button, you can force a stack trace while a local application is
running. A thread dump includes thread states for the active Java threads. It does not stop the
application. You can use a stack trace to help diagnose a number of issues such as
deadlocks or when an application hangs. Using Java VisualVM to take a thread dump can be
very convenient where you do not have a command-line console for the application.

Profiler Snapshot
Profiler snapshots capture profiling data at the moment the snapshot is taken. You can take a
profiler snapshot at any time during a profiling session. After you take the snapshot, stop the
profiling session to reduce resource contentions.
When you take a profiler snapshot, a node representing the snapshot appears below the
application node in the Applications window and a temporary snapshot file is written to your
local system in the Java VisualVM user directory.
You can take the following types of profiler snapshots by using Java VisualVM:
• Memory snapshot: A memory snapshot captures profiling data on allocated objects.
You can take a memory snapshot when you are using the profiler to analyze memory
usage.
• CPU snapshot: A CPU snapshot captures data on the performance of the application.
You can take a CPU snapshot when you are using the profiler to analyze application
performance.

Using VisualGC
Not included with the JDK
VisualVM plug-in tab
Visually observe
Includes class loading and JIT
garbage collection!
compilation information
VisualGC is a stand-alone graphical JVM monitor or a VisualVM plug-in. In this course and
typically, VisualGC is used as a VisualVM plug-in. You can install it directly by using the
VisualVM plug-in center. With VisualGC, a picture is worth a thousand words, because you can
see visually exactly what is going on with the garbage collector. In addition to garbage
collection, VisualGC also provides information about class loading and JIT compilation.

Using jconsole
Is included with the JDK
Attach to multiple local or remote JVMs
jconsole is a graphical monitoring and management console that comes with the HotSpot
JDK. jconsole supports both Java Management Extensions (JMX) and MBean technology.
This allows jconsole to monitor multiple JVMs at the same time. In addition, more than one
jconsole session can monitor a single JVM session at the same time.jconsole can
monitor the following JVM features:
• Memory usage by memory pool/spaces
• Class loading
• JIT compilation
• Garbage collection
• Threading and logging
• Thread monitor contention
Note: MBeans are managed beans, which are Java objects that represent resources to be
managed. They can be used with JMX applications. Multiple jconsole sessions can attach
to a single JVM.

Using GCHisto
Open source project: Not included with the JDK

http://gchisto.dev.java.net
Summarizes GC activity
obtained from GC logs
GCHisto is a stand-alone GUI for analyzing GC log data. (There is also a VisualVM plug-in
under development.) You can analyze multiple log files at the same time. GCHisto also
allows the comparison of heap sizes or collector types for JVM tuning by comparing GC logs.
Open Source Project: http://gchisto.dev.java.net

Using Java VisualVM
This practice covers experimenting with Java VisualVM to
monitor WebLogic Server JVM processes.

Using VisualGC, jconsole, and GCHisto
This practice covers experimenting with VisualGC,jconsole,
and GCHisto to monitor WebLogic Server JVM processes.

Mission Control
Mission Control is a group of management and monitoring

tools:
Tool Description
Management Console JMX-compliant monitoring tool
Flight Recorder Records and analyzes the JVM and running applications
Features from JRockit JVM are integrated
Mission Control is a set of tools for managing, monitoring, profiling, and troubleshooting your
Java applications. Mission Control uses data collected from the Hotspot JVM as a part of its
normal operations, thus minimizing the performance overhead. The main tools available in
Mission Control are:
• Management Console: The Management Console views real-time behavior of your
application and JVM. Features include the ability to create rules that trigger on certain
events (for example, an email will be sent if the CPU reaches a 90% load).
• Flight Recorder: Records performance data over a period of time. Typical information
recorded includes the Java heap distribution, garbage collections, method samples,
latency data, and lock profiling information.
• JMX Agent: Provides access to all MBeans deployed in the platform MBean server.
Using these MBeans, you can read attribute information, such as garbage collection
pause times.

Java Discovery Protocol (JDP)
Java Discovery Protocol (JDP) is based on multicast and enables

Mission Control to discover, connect, and manage JVMs remotely.
Set these basic options on the remote
JVM to set up remote connectivity.
-Dcom.sun.management.jmxremote.port=7091
-Dcom.sun.management.jmxremote.rmi.port=7091
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.autodiscovery=true
Mission
Control
Machine Machine
You configure JMX remote connectivity with your JVMs using Java options that enable JDP.
Mission Control is configured to automatically discover JDP-enabled JVMs. This connection
uses SSL and a username and password login by default. The options in the slide bypass this
security for development environments.

Mission Control: Architecture
Management Console Flight Recorder
Start Flight Recording
Recordings
JMX agent
JVM Process
The main focus of Mission Control is to do the necessary instrumentation with the lowest
possible impact on the running system. The technology used also enables the application to
run at full speed after the tool is disconnected from the JVM, which makes Mission Control
uniquely suitable for use in production environments.
The graphic shows that the management console interfaces with the JMX agent. The Flight
Recorder is able to analyze and display results of generated recordings.

JVM Browser
• Invoking jmc brings up the JVM Browser window.

• Use the JVM Browser view to launch tools for a specific
JVM process.
Remote
JVMs
Process ID
Start this tool

Local
JVMs
When you launch Mission Control, the JVM Browser is opened. The JVM Browser is a tree
view that you can use to manage your JVM connections. It automatically discovers locally
running JVMs as well as JVMs on the network that have been started using JDP. Other plug-
ins, such as the Flight Recorder tool, use the JVM Browser to access the connections to
JVMs to record.
Although Mission Control should discover your JVMs automatically, you can also explicitly
create a connection to a JVM. Within the JVM Browser pane, click the Create a new custom
JVM connection button. Enter a Host , Port, and Connection Name. Mission Control can
only connect to JVMs that have enabled remote JMX connectivity.
To start other Mission Control tools such as the Management Console, select a running JVM,
right-click the JVM, and select an application.
To update user preferences for the JVM Browser or other plug-ins, select Window >
Preferences from the main menu.

Mission Control:
Management Console Overview Tab
JVM Browser
Management Console
The Mission Control Management Console is displayed above. To open it, select a running
JVM from the JVM Browser on the left side of the interface. Once selected, right-click and
select Start JMX Console.
The General page is shown with the Overview tab by default. The page shows information
about CPU Usage along with Garbage Collection memory. Any part of the Overview tab can
be customized.

General: Server Info
The Server Information tab shows key information about this JVM and the current operating
system.

MBeans: Attributes
The MBean Browser tab provides access to detailed information about the selected JVM. The
default attributes tab is shown in this screenshot. The selected Operating System attribute
provides information about the current JVM and the operating system on which it is running.

MBeans: Operations
The Operations tab provides access to special attributes that may be queried for information
or changed via the interface. So it provides a way to make changes to a running JVM.

MBeans: Notifications
The Notifications tab allows you to subscribe to attributes that periodically publish data. In the
example shown, data about minor garbage collections is recorded to the interface.

MBeans: Metadata
The Metadata tab shows detailed information about any attribute.

MBeans: Triggers
The Triggers tab allows you to set triggers based on JVM events. Set the conditions for the
trigger as shown in this screenshot. Then, the Action tab allows you to specify how you get
notified, anything from a dialog popup to an email.

Runtime: Server
This is the System tab from the Runtime page. It contains general information about the JVM.

Runtime: Memory
The Memory tab shows detailed information about the heap and memory.

Runtime: Garbage Collection
The Garbage Collection tab shows information about the number of collections and time-
related information.

Runtime: Memory Pools
Like VisualGC,
but simpler
The Memory Pools tab shows real-time information about the various memory components of
the JVM.

Runtime: Threads
The Thread tab shows information about threads in the JVM. You can enable a number of
attractive features using the check boxes at the top of the table. Detailed information per
thread about:
• CPU Usage
• Deadlocks
• Memory Allocation

Best Practices
• To prevent the GUI from contending for resources with

WLS and its applications, run the Management Console
remotely.
• To prevent unauthorized access to the JVM management
server, implement restrictive mechanisms in the firewall.
To minimize resource contention in production environments, you should run the

management console from a machine different from those used to run applications. As a
practice, you may want to dedicate a separate machine for running the management console
and prevent other machines from accessing the management console.

Using Mission Control
This practice covers experimenting with Java Mission Control to

Flight Recorder
• The JRockit Flight Recorder is now integrated into the

Hotspot JVM. It is now called the Java Flight Recorder
(JFR).
• JFR consists of:
– Runtime: The JFR runtime is the JVM engine that performs
the actual recordings.
– GUI: The user interface to view JVM recordings
The Java Flight Recorder is a performance monitoring and profiling tool that makes
diagnostics information always available, even in the wake of catastrophic failure, such as a
system crash.
JFR is a rotating buffer of diagnostics and profiling data that is always available on demand.
You might consider it a sort of time machine that enables you to go back in time to gather
diagnostics data leading up to an event. The data stored in the rotating buffer includes JVM
and application events. Because JFR is always on, using the default configuration will not
result in any performance overhead.
• JFR runtime is the actual recording agent and comprises these components:
- The Flight Recorder agent, which controls buffers, disk I/O, MBeans, and so on.
This component provides a dynamic library that mixes C and Java code and also
provides a JVM-independent pure Java implementation.
- Producers, which insert data in the Flight Recorder buffer. Producers include the
JVM itself, applications.

third-party other Oracle Java applications and, through a Java API, events from
• The JFR graphical user interface is part of the Mission Control suite of diagnostic and
profiling tools. It allows users to view JVM recordings, current recording settings, and
run-time parameters.

Flight Recorder: Benefits
The Flight Recorder enables you to go back historically to

gather diagnostics data leading up to an event:
Flight R ecorder Us e: Description
As a black box Provides an account of events leading up to a failure of

a system or application
For real-time diagnostics When an application detects an error or warning, it can

request Flight Recorder data from the time window
leading up to the error
For diagnostic profiling Profile an application's performance during testing or in
production
Flight Recorder offers a low-to-zero overhead process that has the following benefits:
• Always on: You no longer need to restart a broken application and create a recording.
Because the Flight Recorder runs at the time the problem occurs, it provides you with
information from before the time you were alerted of the problem. Runtime analysis data
is always available in the buffer; providing a record of the problem you can diagnose it
based upon the actual event. This feature helps diagnose intermittent problems because
you do not have to start a recording and hope the problem happens again.
• Allows for third-party event providers: A set of APIs allow the Flight Recorder to
monitor third-party applications, including WebLogic and other Oracle products. This
means that you will receive JVM-level detail combined with logic from the Java
application running on the JVM.
• Reduces total cost of ownership: Because you spend less time diagnosing and
troubleshooting problems, the Flight Recorder helps reduce operating costs, reduce
business interrupts, provide faster resolution time when problems occur, and improve
system efficiency.

Data Flow in Flight Recorder +
When full, is moved to When full, is moved to
Java
events
Thread Global Disk

buffer buffer chunk
JVM
events
Data is sent to the recorder by the JVM (through internal APIs), and by the Java application
(through the Flight Recorder APIs). The Flight Recorder runtime stores this data in small
thread-local buffers that are flushed to a global in-memory buffer. The data in the in-memory
buffer will be moved from the in-memory buffer to a disk buffer when running in persistent
storage mode.
Buffers are critical to the efficacy of the Flight Recorder. The size of the buffer determines
when and how data is removed from the disk buffers. Data can be discarded after evaluating
either its age or the size of the buffer.
You can configure the buffer sizing mode at the command line when you start the Flight
Recorder. You can select from one of two modes:
• Time-bound mode: In the Time-bound sizing mode, the Flight Recorder keeps data for
a certain period of time. Data that has expired will be removed from the disk.
• Space-bound sizing mode: The Flight Recorder will always keep the size of the
buffers
time. bounded, so the maximum age of the events in the buffer may vary from time to

Initiating Recording Using
the Management Console
Start your JVM by using the
-XX:+UnlockCommercialFeatures and -XX:+FlightRecorder options. 1
Run your application 4

under load.
2
The screenshot shows an example of initiating performance data recording by using the
Management Console. After a recording is completed, Mission Control opens the recording in
a new tab and shows the view of performance data. You can save this data for further
analysis and comparison with other recordings.

Tab Groups of Flight R ecorder
Description
General information about the recording and the recorded application and JVM
Information about memory management and garbage collection
Information about methods, optimizations, and exceptions
Information about the threads and locks in the recorded application
Information about file and network reads and writes
Information about underlying hardware and system properties
Generic information about selected events based upon event types selected in the
Events Type View

Flight Recorder General Tab
Time range selection
Heap,
CPU,
and GC
Pause
statistics
Tabs
CPU and heap data
Tabs
The General tab of the Flight Recorder displays high-level information about the recording. It
displays memory usage, CPU usage, and system properties of the JVM. The VM arguments
that you specified on the command line when starting the JVM are also available on the
General tab. You can achieve a very high level of understanding by investigating this data.
For more information about the specific areas, further investigation is required on the subtabs
on the bottom of the page, or the other tab groups.

Flight Recorder Memory: Overview Tab
Machine
memory
and
Java
GC data heap
statistics
Tabs
The Overview tab provides the broad picture of the memory usage during the Flight Recorder
session. The tab has two graphs:
• The Events and Operatives Set: Shows the graph of when a particular event occurred.
By default a set of events, such as garbage collection, are predefined. In addition, you
can also define events based on your application logic.
• Memory Usage: Shows a graphic representation of the memory used during a Flight
Recorder session.
The two tables on the bottom, namely GC Configuration and GC Statistics, provide average
and maximum statistics relating to the memory (heap) usage and garbage collection during
the session.

Flight Recorder Memory: GC Collections Tab
Tabs
Heap,
and GC
Pause
statistics
Tabs
More
detailed
GC data
Tabs
The GC Collections tab gives information about the garbage collected during the recording.
The charts show heap usage.
In the Garbage Collections table, you see a list of garbage collection with the time taken and
the generation of heap affected by garbage collection. When you select a GC in the list, you
can find more details under the appropriate subtab in the Details table.

Flight Recorder Memory: Object Statistics
Classes
using
the heap
Top
growing
objects
Tabs
The Object Statistics tab shows the number of instances and cumulative memory used by
different classes during the Flight Recorder session. The Top Growers table provides details
of the classes that are growing. Unbridled growth of objects can indicate a memory leak.

Section Summary
In this section, you should have learned how to use GUI JVM
monitoring tools.

Using Flight Recorder
This practice covers experimenting with Java Flight Recorder to

Monitor and Tune Server Performance

Objectives

following:
• Configure domain and server parameters to improve
performance
• Configure the server to handle stuck threads

• Set up connection backlog buffering
• Describe secure sockets layer (SSL) configuration
• Configure loggers and log filters

Agenda
• Configuring domain and server parameters to improve

performance
• Configuring the server to handle stuck threads
• Setting up connection backlog buffering
• Describing secure sockets layer (SSL) configuration

• Configuring loggers and log filters

On-Demand Deployment
• You can set WebLogic Server internal applications to start

up when first accessed (on demand).
• This improves server startup time, but first access to the
application is delayed.
• Code snippet to change the

InternalAppsDeployOnDemandEnabled attribute
using WLST:
connect()
edit()
startEdit()
cmo.setInternalAppsDeployOnDemandEnabled(true)
activate()
WebLogic deploys many internal applications during startup. Many of these internal
applications are not needed by every user. You can configure WebLogic to wait and deploy
internal applications on first access (on demand) instead of always deploying them during
server startup. This can conserve memory and CPU time during deployment as well as
improve startup time and decrease the base memory footprint for the server.
In a development domain, the default is for WebLogic to deploy internal applications on
demand. For a production-mode domain, the default is for WebLogic to deploy internal
applications as part of server startup.
In production mode, you can change the InternalAppsDeployOnDemandEnabled
attribute using the Administration Console:
1. Click Lock & Edit to start an edit session.
2. Select the domain to bring up the Configuration > General tab.
3. Change the setting of the “Enable on -demand deployment of internal applications”
check box.
4. Click Save, and then click Activate Changes to activate the changes that will take effect
at the next restart of W ebLogic.

Enable Exalogic Optimizations
This will enable the following core optimizations:

• Pure java NIO Socket Muxer
• Scattered Reads
• Gathered Writes
• JSSE as the SSL Provider
• Lockless Request Manager
• More aggressive self-tuning thread algorithm
Turn on “ExalogicOptimizationsEnabled” flag under Domain > Configuration > General.

Monitor Channel Connections
Ensure that you do not have too many or uneven TCP connection
counts across managed servers.

Section Summary
In this section, you should have learned how to configure

domain and server parameters to improve performance.

Agenda

performance


Stuck Threads
• Every client request makes use of a thread from one of the

execute queues.
• Some operations, such as the following, can make the
thread wait inordinately:
– Computation loop in an application that has run into an
infinite loop
– Database calls that have resulted in a deadlock situation
• Such situations can cause the thread to be stuck with that
operation for whatever time it takes to complete that
operation.
• Performance of the server can degrade significantly if
many such threads get stuck.

Configuring Stuck Threads Behavior
• WebLogic diagnoses a thread as stuck if it is continually

working (not idle) for a set period of time.
• You can tune a server’s thread detection behavior by
changing the:
– Length of time before a thread is diagnosed as stuck
– Frequency with which the server checks for stuck threads
• WebLogic never kills threads. It is not possible to kill any
threads.
– It is possible, however, to force WebLogic to set a managed
server to the FAILED state. The server can be either
automatically restarted by the Node Manager or suspended.
WebLogic diagnoses a thread as stuck if it is continually working (not idle) for a set period of
time. You can tune a server’s thread detection behavior by changing the length of time before
a thread is diagnosed as stuck, and by changing the frequency with which the server checks
for stuck threads.
You can configure stuck thread detection behavior for each WebLogic server by using the
Server Configuration > Tuning tab of the Administration Console:
• In the Configuration > Tuning tab, update these parameters as necessary:
- Stuck Thread Max Time: Amount of time, in seconds, that a thread must be
continually working before a server instance diagnoses a thread as being stuck. By
default, WebLogic considers a thread to be “stuck” after 600 seconds of
continuous use.
- Stuck Thread Timer Interval: Amount of time, in seconds, after which a server
instance periodically scans threads to see whether they have been continually
working for600
interval to theseconds.
configured Stuck Thread Max Time. By default, WebLogic sets this
Note: You configure stuck thread detection parameters on a per-server basis.

Configuring Stuck Threads Behavior
• A STUCK thread is not necessarily stuck, which means

that:
– It is long-running
– At the point in time detected, the logged stack trace is where
the thread was currently executing
— It may be doing something else immediately before or after!
• When you encounter a STUCK thread:
– Take multiple thread dumps at regular intervals
– Looking at the stack trace of the STUCK thread and its
progress over time is key to diagnosing the issue.
– WLDF can be configured to automatically create an incident
that will include the series of thread.

Parameters for Stuck Threads at the Server Level
Based on certain thresholds, the server can identify such

situations and mark the server’s state as FAILED.
Thresholds include:
• Stuck Thread Max Time
• Stuck Thread Count

• Stuck Thread Timer Interval
Max Stuck Thread Time: The number of seconds that a thread must be continually working
before this server diagnoses the thread as being stuck. For example, if you set this to 600
seconds, WebLogic considers a thread to be stuck after 600 seconds of continuous use.
Stuck Thread Count: The number of stuck threads after which the server is transitioned into
a FAILED state. There are options in OverloadProtectionMBean to suspend and shut down a
FAILED server. By default, the server continues to run in a FAILED state. If the Stuck Thread
Count value is set to 0, the server never transitions into the FAILED server irrespective of the
number of stuck threads.
Stuck Thread Timer Interval: The number of seconds after which W ebLogic periodically
scans threads to see whether they have been continually working for the configured maximum
length of time.
The screenshot shows how to set the Max Time and Timer Interval.

Actions upon Stuck Threads Condition
Thresholds include:
• Max Stuck
Thread Time
• Stuck Thread
Count
• Failure Action
The screenshot shows how to set the Failure Action, the Max Stuck Thread Time, and Stuck
Thread Count.
IMPORTANT: To force W ebLogic Server to reboot the server which incurred the stuck thread
threshold (or to suspend it), it is crucial to set the "Stuck Thread Count" parameter to a
number different from zero; otherwise, the "Failure Action" will be disregarded.

Auto Restart of the Server
• If the server goes to a FAILED state, it can be forced to

shut down and can be restarted using the Node Manager.
• This process of restarting the server can be automated.
The screenshot shows how to set the option to perform an auto restart.

Section Summary
In this section, you should have learned how to enable policies

to deal with Stuck Threads.

Configuring Server Stuck Thread Behavior
• Configuring WebLogic Server to transition servers into the
FAILED state when a certain threshold of stuck threads is
exceeded

Practice 5-2 Overview: Configuring Automatic
Restarting of a Failed Server
• Causing a number of threads (exceeding the threshold) to
become stuck
• Observing the server running stuck threads being rebooted
by WebLogic after it transitioned into the FAILED state

Configuring WebLogic Overload Protection
• Configuring WLS memory parameters to detect the
OVERLOAD condition
• Causing a server to transition into the OVERLAOD state

Agenda

performance


Connection Backlog Buffering
This specifies how many TCP connections can be buffered

before refusing additional requests. Configuration options:
• Default value: 300
• Minimum: 0
• Maximum: Operating-system dependent
Connection backlog buffering is the number of connection requests that a WebLogic Server
instance will accept before refusing additional requests. The accept-backlog parameter in
config.xml specifies how many Transmission Control Protocol (TCP) connections can be
buffered in a wait queue.
The fixed-size queue is populated with requests for connections that the TCP stack has
received, but the application has not yet accepted.
The screenshot shows where to configure the Accept Backlog setting in the Administration
Console.
To address connection backlog, you may also have to tune TCP parameters at the operating
system. In a Linux environment, the following parameters may be of significance:
/sbin/ifconfig lo mtu
kernel.msgmni
kernel.sem
kernel.shmmax
fs.file-max
net.ipv4.tcp_max_syn_backlog

Connection Backlog Buffering
• If the Accept Backlog value is set too low, most

connections will be dropped or refused at the client.
• A very high Accept Backlog value creates an overhead on
the server.
• You should
balance try different
between Accept
refusing Backlog frequently
connections values andand
find a
running the server with extraordinary load.
Raise the Accept Backlog value from the default by 25 percent for every “connection refused”
message when accessing WebLogic.
Setting the backlog to 0 prevents accepting incoming connections on some OS.
A guideline for setting Connection Backlog Buffering is 8,192 for a server with 4 GB of
physical memory.

Section Summary
In this section, you should have learned how to tune connection

backlog buffering.

Agenda

performance


Secure Sockets Layer (SSL)
• The following are two areas related to secure sockets layer

(SSL) performance:
– Handshake
– Bulk encryption and decryption
• To enhance
individual SSL
SSL performance,
connections anddecrease the number of
handshakes.
• Use hardware accelerators to offload SSL processing from
the application server to the dedicated SSL accelerator.
SSL is also known as Secure HTTP (HTTPS) because it defines secure wrapping,
authentication, and encryption for HTTP content. From a performance perspective, SSL
encryption and decryption create an additional processing layer, beyond regular HTTP
processing. This layer includes the following two major CPU-intensive phases:
• SSL handshake
- After establishing a TCP connection, SSL creates a security context between
endpoints by using Public Key Interchange (PKI). This is known as an SSL
handshake. In terms of aggregate network traffic, an SSL handshake consumes
processing power that is proportional to the connection rate (measured in
connections per second).
• Encryption
- After a security context is established, an endpoint uses it to encrypt or decrypt
HTTP content. This processing is performed on each byte of HTTP data.
Therefore, it consumes processor cycles proportional to aggregate network
throughput (measured in megabits per second).
Decreasing the number of connections increases performance for secure communication
through SSL connections, as well as non-secure communication through simple TCP/IP
connections.

Section Summary
In this section, you should have learned how to enable SSL

optimizations.

Agenda

performance


Logging Considerations
• Ensure that applications perform minimal

logging.
• Set log severity thresholds to the highest
acceptable levels.
• Disable standard output and HTTP logging.
• Disable any diagnostic logs associated with
WebLogic Diagnostic Framework (WLDF),
Java Message Service (JMS), Java Database
Connectivity (JDBC), and so on.
• Use log filters to further refine which messages are
written to disk or sent to the administration server.
• Logging is asynchronous whereas standard output
is synchronous.
Under each server’s advanced logging configuration attributes, locate the following:
• Log File Severity Level: The minimum severity of log messages going to the server log
file. By default, all messages go to the log file.
• Standard Out Severity Level: The minimum severity of log messages going to the
standard out. Messages with a lower severity than the specified value will not be
published to standard out.
• Domain Log Severity Level: The minimum severity of log messages going to the
domain log from this server’s log broadcaster. Messages with a lower severity than the
specified value will not be published to the domain log.
By default, the HTTP subsystem keeps a log of all HTTP transactions in a text file. The default
location and rotation policy for HTTP access logs are the same as the server log. Disable the
HTTP access log from the Logging > HTTP tab. You can also tune HTTP logging for
individual Web applications using the XML descriptor weblogic.xml.
Finally, confirm that each JMS destination (topic or queue) does not enable message logging.
Select a destination and click its Configuration > Logging tab.

Log Filters Review
Log filters provide control over the log messages that are published. A filter uses custom logic
to evaluate the log message content, which you use to accept or reject a log message (for
example, to filter out messages of a certain severity level, from a particular subsystem, or
according to specified criteria). Only the log messages that satisfy the filter criteria are
published. You can create separate filters for the messages that each server instance writes
to its server log file, standard out, and memory buffer, or broadcasts to the domain-wide
message log.
1. Select the name of the active domain in the Domain Structure panel.
2. Click the Configuration > Log Filters tab. Click New. Enter a value to identify the filter in
the Name field and click Finish.
3. Edit the new filter. Enter the criteria for qualifying messages. You can click E dit to enter
or paste in an expression with the WLDF Query Language syntax, or you can click Add
Expression to construct an expression by using a set of dialog boxes.

Section Summary
In this section, you should have learned how to minimize

logging overhead by enabling Log Filters.

Configuring Log Filters for WebLogic
• Enabling more verbose debugging info
• Improving the accuracy of logging by filtering out unwanted
information
• Ability to switch dynamically (no WLS restart) from less

verbose to more verbose logging

Monitor and Tune Cluster Performance

Objective
After completing this lesson, you should be able to tune

WebLogic Server clusters for performance.

Agenda
• Cluster Architecture and Communication Performance

• Load Balancers
• Session Persistence Performance

Cluster: Review
A cluster:
Domain
• Is a logical group of managed servers
Cluster
from the same domain that run Clients
cooperatively Server
• Supports
availabilityfeatures
for webthat provide high
applications, web
Machine
services, EJBs, and JMS Cluster

Server
• Is transparent to its clients Proxy
Machine
• Can have servers added to it statically
or dynamically
Server
• Requires a cluster proxy to provide load
balancing, if it hosts web applications Machine
A WebLogic Server cluster consists of one or more managed servers from the same domain
running simultaneously and working together to provide increased reliability and scalability. A
cluster appears to clients as one WebLogic Server instance. The server instances that
constitute a cluster can run on one machine or on multiple machines.
A cluster achieves high availability through the replication of services. Because of this
replication, failover is possible. When one server fails, a second server automatically can
resume operation where the first server left off.
Load balancing, the distribution of work across the cluster, ensures that each server in the
cluster helps carry the load.
Scalability is achieved because you can increase a cluster’s capacity by adding server
instances to the cluster, without making any architectural changes.
A cluster also assists in migration. After a system failure on one server, work can be
continued by moving the services that server provided to another server in the cluster (service
level migration), or by moving the entire server to a new hardware (whole server migration).
Once a cluster is created, configured servers can be added to it. A dynamic cluster is based
on a server template. A server template sets server attributes. Once a server template is
assigned to a cluster, servers based on the template are generated and added to the cluster.

Clusters support different types of applications as follows:
• Web applications: Load balancing is achieved through the cluster proxy. This proxy can
be a web server using a WebLogic Server proxy plug-in or a hardware load balancer.
Failover is achieved by replicating or storing the HTTP session state of clients.
• For Enterprise JavaBeans (EJBs), clustering uses the EJB’s replica-aware stub for load
balancing and failover. When a client makes a call through a replica-aware stub to a
service that fails, the stub detects the failure and retries the call on another replica.
• For JMS applications, clustering supports transparent access to distributed destinations
from any member of the cluster.

Benefits of Clustering
Concept Description
More capacity for applications can be provided by

Scalability adding servers, without interruption of service or making
architectural changes.
Work (for example, client requests) is distributed across
Load balancing the members of a cluster.
When a server fails, another one can automatically take
Failover its place. Information on the failed server is replicated
(or stored), so that the new server has access to it.
When a server fails, its “pinned” services can continue
Migration by moving them to another server in the cluster, or by
moving the entire failed server to a new hardware.
A “pinned” service is a service that must run only on a

single instance of WebLogic Server at any given time.
A WebLogic Server cluster provides the following benefits:

• Scalability: The capacity of a cluster is not limited to one server or one machine.
Servers can be added to the cluster dynamically to increase capacity. If more hardware
is needed, a new server on a new machine can be added.
• Load Balancing: The distribution of jobs across the cluster members can be balanced,
so no one server is overloaded.
• Failover: Distribution of applications and their objects on multiple servers enables
easier failover of the session-enabled applications.
• Availability: A cluster uses the redundancy of multiple servers to insulate clients from
failures. If one server fails, another can take over. With the replication (or storage) of
server-specific information, the failover can be transparent to the client.
• Migration: This ensures uninterrupted availability of pinned services or components—
those that must run only on a single server instance at any given time. An example of a
pinned service is the use of the Java Transaction API (JTA).

Basic (Single-Tier) Cluster Architecture
Machine
Cluster
Server 1
Web App EJB
Code Code
Server 2
Web App EJB
Code Code
Cluster
Proxy
Server 3
Clients Web App EJB
Code Code Back-end
Systems
Server 4 and
Web App EJB Databases
Code Code
Firewall
Machine
The basic, single-tier cluster architecture has all W ebLogic Server application code in a single
tier. That single tier includes both web applications and Enterprise JavaBeans.
Remove the “EJB Code” box for systems that do not use EJBs.
Note: If web applications and EJBs are contained within the same deployment unit, then no
serialization/deserialization or network processing is required, thus increasing performance.

Multitier Cluster Architecture
Machine
Cluster A
Server 1
Web App Machine
Code
Cluster B
Server 2 Server 5
Web App EJB
Code Code
Cluster
Proxy
Server 3 Server 6
Clients Web App EJB
Code Code Back-end
Systems
Server 4 and
Machine Databases
Web App
Code
Firewall
Machine
In the multi-tier cluster architecture, two separate WebLogic Server clusters are configured:
• Cluster A for the web application tier
• Cluster B to serve clustered EJBs
If your system does not use EJBs, you would not use the multi-tier cluster architecture in this
way. However, you could use it to load balance calls to JMS.
Note that calls between the web applications and EJBs are serialized and deserialized, which
along with accessing the network decrease performance.

Remote Calls and Deployment Packaging
Slowest
Server 1 Server 2 Network and
Web App EJB serialization
App.war Code processing
Server 1 Serialization
Web App EJB
App.war App.jar
processing
Server 1
EAR
App.ear NO network or
Web App EJB serialization
App.war App.jar processing
Fastest
• Example 1 shows a web application on server 1 communicating with an EJB on server

2. Every request involves serializing the data on server 1, network processing over the
wire, and deserializing the data on server 2. It also involves the same overhead when
sending a reply.
• Example 2 shows a web application on server 1 communicating with an EJB also on
server 1. This avoids the overhead of the network, but still involves serializing the
request on server 1 at the web application and deserializing the data on server 1 at the
EJB. Replies incur the same overhead.
• Example 3 shows a web application and EJB deployed within a single Enterprise
Application (EAR) file. The request and reply avoid the overhead of both the network
and serialization/deserialization processes.
Note: These are not really administration tasks, but are important factors to discuss with
application developers so they know the best way to package the applications you deploy.

Architecture Advantages and Disadvantages
Cluster
Architecture Advantages Disadvantages
Basic • Easier to administer • Cannot load balance
(single-tier) • Less network traffic EJB calls
• EJB calls are local (and therefore
faster)
Multi-tier • EJB calls are load balanced • Harder to administer

• Scaling options (for example, you • Perhaps more
can shift (or add) hardware and hardware and licensing
WebLogic server instances to costs
whichever tier is busier) • EJB calls are remote
• More security options (for example, (and therefore slower)
you could place a firewall in • More network traffic
between the web application tier
and the EJB tier)
Basic (single-tier) advantages:

• Easier administration: There is only one cluster to create, configure, and maintain. Also,
because one cluster hosts web applications and EJBs, you can easily deploy enterprise
applications to the cluster. The web application and EJBs are in the same archive.
• Less network traffic: Clients access WebLogic Server through web applications. Calls
from WebLogic Server to back-end systems and databases still occur, but all calls from
the web application tier to EJBs are within the same JVM.
• Faster EJB performance: Because the calls from the web applications to the EJBs are
within the same instance of WebLogic Server, there is no network overhead. The calls to
the EJBs are local, not remote. This is especially important if the web applications make
frequent EJB calls. Additionally, if the EJBs are packaged within the same enterprise
application as the web applications, then serialization and deserialization processing
cost is also avoided.

Basic (single-tier) disadvantage:
• EJB calls cannot be load balanced. Each call from the web application tier to an EJB is
always to the EJB running on that same instance of WebLogic Server. It is therefore
possible that the server load becomes unbalanced. Let us say that 200 concurrent users
are accessing your applications, and they have been load balanced so that 50 are
accessing each of the four servers in the cluster. It just so happens that the 50 users on
server 1 are performing tasks that call EJBs, while the other 150 users on the other
servers are not. Server 1 will be exceptionally busy, while servers 2, 3, and 4 will not. If
the EJB calls were load balanced to an EJB tier, however, then the “EJB load” would be
spread across all of the servers in the EJB tier cluster.
Multi-tier advantages:
•
EJB calls are load balanced: Each call to an EJB can be load balanced across all the
servers in the EJB cluster. The “unbalanced” situation described above is no longer
possible.
• More scaling options: Separating the web application and EJB tiers onto separate
clusters provides you with more options for scaling the system and distributing the load.
For example, if users spend most of their time using the web applications, and those
applications make few EJB calls, you can use a larger number of WebLogic Server
instances in the web application cluster. If things change, and your applications become
more EJB-intensive, you can shift or add servers to the EJB cluster.
• More security options: With another layer there is an opportunity to add more security.
For example, you could place a firewall in between the web application and EJB
clusters.
Multi-tier disadvantages:
• More difficult administration: There is more than one cluster to create, configure, and
maintain. Also, because one cluster hosts web applications and the other EJBs,
deployment becomes more complicated.
• Perhaps higher costs: With two clusters you may have more instances of WebLogic
Server and more hardware.
• Slower EJB performance: Because the calls from the web applications to the EJBs are
always remote, you must pay the price of remote calls. The applications must be
developed with this in mind so that calls to EJBs are infrequent and “course grained.”
• More network traffic: Because all EJB calls are remote, network traffic increases.

Cluster Communication
Cluster members communicate with each other in two ways:

• One-to-many messages:
– For periodic “heartbeats” to indicate continued availability
– To announce the availability of clustered services
– Note: This communication can use either:

— IP multicast (recommended): A multicast host and port must be
configured.
— IP unicast: No additional configuration is required.
• Peer-to-peer messages:
– For replicating HTTP session and stateful session EJB state
– To access clustered objects that reside on a remote
server (multi-tier architecture)
– Note: This communication uses sockets.
An instance of WebLogic Server uses one-to-many communication to send regular

“heartbeat” messages that advertise its continued availability to other server instances in the
cluster. The servers in a cluster listen for heartbeat messages to determine when a server has
failed.
All servers use one-to-many messages to announce the availability of clustered objects that
are deployed or removed locally. Servers monitor these announcements so that they can
update their local JNDI tree to indicate the current deployment of clustered objects. This is the
maintenance of the so-called “cluster-wide” JNDI tree.
IP multicast enables multiple applications to subscribe to an IP address and port number, and
listen for messages. A multicast address is an IP address in the range 224.0.0.0-
239.255.255.255. IP multicast does not guarantee that messages are received, so
WebLogic Server allows for the possibility that some messages may be missed. Ifyou use
multicast, you must ensure your network propagates multicast messages to all clustered
servers. The multicast time-to-live value can be increased if you find that messages are being
missed. With multicast, you must ensure that no other applications share the multicast
address and port, or servers will have to process extra messages, which introduces extra
overhead.

Firewalls can break multicast transmissions. Although it might be possible to tunnel multicast
transmissions through a firewall, this practice is not recommended. A final worry with multicast
messaging is the possibility of a multicast “storm,” in which server instances do not process
incoming messages in a timely fashion, which leads to retransmissions and increased
network traffic.
IP unicast does not have the network issues of multicast. All devices support TCP/IP and the
protocol in itself has built-in retry logic. However, unicast communication can sometimes not
perform as well as multicast communication. Oracle recommends to use multicast whenever
possible. If it is not possible, or unicast works effectively for your application, it is still fully
supported by Oracle. You can set up a separate network channel for unicast communication,
but it is not required. If no separate channel is defined, each server’s default channel is used
(the default channel is the server’s configured host and port).

Note: Other courses in this series inform students that unicast is Oracle’s recommended
protocol. However, this information changed between when these courses were being
developed and this is the latest information at this time. The WebLogic documentation is
currently being reviewed to address this.
IP sockets provide a simple, high-performance mechanism for transferring messages and
data between two applications. Clustered WebLogic Server instances use IP sockets for:
• Replicating HTTP session state and stateful session EJB state
• Accessing clustered objects that reside on a remote server instance (as in the multi-tier
cluster architecture)

Section Summary
In this section, you should have learned how to describe:

• The pros and cons of different WebLogic cluster
architectures
• The performance characteristics of remote, local, and
inter-deployment unit calls

• The different types of cluster communication

Configuring Multicast and Unicast Clustering
This practice covers the following:
• Configuring and monitoring multicast cluster
communication
• Configuring and monitoring unicast cluster communication

Agenda

• Load Balancers

A Cluster Proxy for a Web Application Cluster
• A cluster proxy provides load balancing and failover for a

web application cluster. It gives the cluster its “single
server” appearance.
• There are basically two kinds of cluster proxies:
– A web server with the WebLogic proxy plug-in
– A hardware load balancer
Cluster
Proxy Advantages Disadvantages
Web server • Low cost (or free) • Only round-robin load balancing
with plug-in • You probably already have available
experience with the web server • Must configure the plug-in
Hardware • More sophisticated load • Cost

load balancer balancing algorithms • Must be compatible with the
• No plug-in configuration WebLogic Server session cookie
Cluster Proxies are how clients interact with a web application cluster, whether they are
hardware or software based. You have two basic choices of cluster proxy: a web server using
a plug-in or a hardware load balancer (such as F5 BIG-IP).
Hardware load balancers must support a compatible passive or active cookie persistence
mechanism. Passive cookie persistence enables WebLogic Server to write a cookie
containing session information through the load balancer to the client. You can use certain
active cookie persistence mechanisms with WebLogic Server clusters, provided the load
balancer does not modify the WebLogic Server session cookie. If the load balancer's active
cookie persistence mechanism works by adding its own cookie to the client session, no
additional configuration is required to use the load balancer with a WebLogic Server cluster.
A WebLogic Server proxy plug-in is available for Netscape Enterprise Server, Apache HTTP
Server, Microsoft Internet Information Server (IIS), and Oracle HTTP Server (which is based
on Apache). These plug-ins provide round-robin load balancing to the servers in the cluster.
They use the W ebLogic Server session cookie information to route requests to the server that
has a client’s session data.

Proxy Plug-Ins
A proxy plug-in:
• Load balances client requests to clustered WebLogic
Server instances in a round-robin fashion
• Avoids routing requests to failed servers in the cluster
• Routes requests based on WebLogic Server session

cookie information
Cluster
Server 1
Web Server Server 2
Plug-in Server 3
Server 4
Clients
Passive cookie
Passive cookie persistence enables the W ebLogic Server to write a cookie containing session
parameter information through the load balancer to the client. When using some hardware
load balancers, you must configure the passive cookie persistence mechanism to avoid
overwriting the WebLogic Server cookie that tracks primary and secondary servers used for
in-memory replication. Specifically, you must set the following values:
• String offset value to the Session ID value plus 1 byte for the delimiter character
• String length to 10 bytes
Active cookie
Active cookie persistence is supported as long as the mechanism does not overwrite or
modify the WebLogic HTTP session cookie. In this case, no additional configuration is
required.
SSL persistence
SSL persistence performs all encryption and decryption of data between clients and the
cluster and uses a plain text cookie on the client to maintain an association between the client
and a server in the cluster.

Oracle HTTP Server (OHS)
OHS is a web server that is:

• A component of the Oracle Web Tier Suite
• Based on Apache HTTP Server
• Installed with the WebLogic Server plug-in module
(mod_wl_ohs) by default
– The plug-in must be configured by using the
mod_wl_ohs.conf file.
• Managed and monitored by using the Oracle Process
Manager and Notification Server (OPMN)
– OPMN manages and monitors non-Java components of
Oracle Fusion Middleware.
– OPMN can be accessed by using theopmnctl command-
line utility.
OHS is based on the Apache web server. OHS supports single sign-on, clustered deployment
and high availability, and Web Cache.
Configuration of Oracle HTTP Server is specified through directives in configuration files in
the same manner as with Apache HTTP Server.
A mod_wl_ohs module is available in OHS. This module enables you to integrate your
WebLogic Server environment with OHS immediately after the configuration of the OHS
instance and the domains.
OHS directories are divided between the Oracle home and the Oracle instance. The Oracle
home directories are read-only, and contain the Oracle Fusion Middleware binaries. The
Oracle instance directories contain the modules, applications, and logs for OHS. Each OHS
component has a root configuration directory found at
<instance>/config/OHS/<component>, which includes the W LS plug-in configuration
file, mod_wl_ohs.conf. Similarly, each component’s log files are found at
<instance>/diagnostics/logs/OHS/<component>.

OPMN provides Oracle Fusion Middleware system (non-Java) components with process
management and failure detection. It consists of the Oracle Process Manager (PM) and the
Oracle Notification Server (ONS). PM is responsible for starting, restarting, stopping, and
monitoring the system processes. ONS is the transport mechanism for failure, recovery,
startup, and other related notifications between components in Oracle Fusion Middleware.

Hardware Load Balancers
When choosing a hardware load balancer, some of the factors

to consider include:
• Support for a compatible passive or active cookie
persistence mechanism and SSL persistence
• Pages-served performance
• Flow of connections to the servers
• Management and configuration tools
• Security
• Load-balancing algorithms
• Other features
Passive cookie
Passive cookie persistence enables the W ebLogic Server to write a cookie containing session
parameter information through the load balancer to the client. When using some hardware
load balancers, you must configure the passive cookie persistence mechanism to avoid
overwriting the WebLogic Server cookie that tracks primary and secondary servers used for
in-memory replication. Specifically, you must set the following values:
• String offset value to the Session ID value plus 1 byte for the delimiter character
• String length to 10 bytes
Active cookie
Active cookie persistence is supported as long as the mechanism does not overwrite or
modify the WebLogic HTTP session cookie. In this case, no additional configuration is
required.
SSL persistence
SSL persistence performs all encryption and decryption of data between clients and the
cluster and uses a plain text cookie on the client to maintain an association between the client
and a server in the cluster.

Hardware Load Balancers
• There is a wide range of load-balancing algorithms:

– Round-robin
– Weight based
– Load based
– Random
• You should skip dead servers in a cluster, thereby
improving response times.
• Use a hardware load balancer directly in front of a
WebLogic Server cluster to minimize the number of hops.
• Hardware load balancers may also be able to absorb
SSL-processing load
Using a hardware load balancer directly in front of a W ebLogic Server cluster results in a
single network hop from the load balancer to a WebLogic Server. This provides better
performance than placing the load balancer in front of a farm of third-party proxy web servers,
which require the standard two hops:
• One from the load balancer to the web server
• The other from the web server to a WebLogic Server
Load balancers often provide SSL acceleration that offloads the SSL processing from the
application server to the dedicated SSL accelerator. This can also significantly improve
performance by decreasing the amount of time required to process secure transactions.

Section Summary
In this section, you should have learned how to describe cluster

proxying techniques and their performance characteristics.

Agenda

• Load Balancers

HTTP Session Persistence
• The WebLogic Server provides clustering support for JSPs

and servlets by persisting the HTTP session state.
• To benefit from HTTP session persistence, ensure that the
session state is persistent by configuring:
– In-memory replication Recommended, as it is the fastest
– File system persistence

– Java Database Connectivity (JDBC) persistence
– Coherence*Web persistence
Web application components such as servlets and JSPs maintain data on behalf of clients by
using an HttpSession instance that is available on a per-client basis. To provide high
availability of web applications, shared access to a single HttpSession object must be
provided. HttpSession objects can be replicated within the WebLogic Server by storing
their data via in-memory replication, file system persistence, database persistence, or
Coherence*Web persistence.

In-Memory Session Replication
Cluster
• The WebLogic Server (WLS)
can replicate: Server 1
– HttpSession objects Primary
– Stateful session EJBs
Server 2
• Session objects(primary
on two servers exist in and
memory
backup). Secondary
• Backup:
Server 3
– Server determined by replication
Secondary
group or machine definition
– Object created immediately after
Server 4
the primary object is created
Primary
• Primary failure makes the backup
object the primary object.
Using in-memory replication, WebLogic Server copies session state from one server instance
to another. The primary server stores the primary session state (the primary server is the
server to which the client is connected when a session is first created). A replica of the
session state is stored on another instance of WebLogic Server in the cluster (the secondary
server). The replica is kept up-to-date so that the data there can be used if the primary server
fails.
By default, session replication is synchronous. The asynchronous option replicates data in
batches to improve cluster performance.
In the event of a failure of the primary object, the backup object will be promoted to the
primary object for all future requests. When a failover situation occurs, another backup object
will be created. This is ideal because the replication of object data has to occur only between
the primary and backup objects (rather than the entire cluster).
The graphic shows that each primary server in the cluster is backed up to another single
server in the cluster.

File Session Persistence
File persistence stores session information in files to a highly

available file system.
All members have access to any
client sessions for failover purposes
Cluster
(each server can act as the
secondary to any other server). Server
Web App
Code
File System
Server
Cluster
Web App
Proxy Code
Clients
Server
Web App
Code
Session state may also be stored in the file system.

For file-based persistence:
• You must create the directory in which to store the files.
• The servers must have the appropriate access privileges.
Any server can act as the secondary server to back up any primary server. Therefore, the
session cookie does not keep track of a secondary server.
Performance is slower than in-memory replication, but faster than database persistence.
The graphic shows that servers in a cluster may store their states in a shared flat file system.

JDBC Session Persistence
Cluster Common access

• All server instances obtained via identical
have access to all Server 1 data sources
sessions. Servlet 1
• Subsequent requests Servlet 2
from the same client

can be handled by any Server 2
server. Servlet 1
Great failover capability Servlet 2

Significant performance
reduction
Server 3
• Changing session Database
Servlet 1
objects causes (slow)
database Servlet 2
HttpSession objects
synchronization. stored in database
Whenever a servlet creates or uses a session object, the servlet stores the session data
persistently in the database. When a subsequent client request enters the cluster, any server
in the cluster can handle the request. Each server in the cluster has identical access to the
persistent store where it can look up the information needed to satisfy the client’s request.
This technique provides for good failover capability because any server in the cluster can
resolve a client’s request, but there is a significant performance reduction due to the many
database synchronizations required in a large web-based system.
Session persistence is not used for storing long-term data between sessions. That is, you
should not rely on a session still being active when a client returns to a site at some later date.
Instead, your application should record long-term or important information in a database.
You should not attempt to store long-term or limited-term client data in a session. Instead,
your application should create and set its own cookies on the browser. Examples of this
include an auto-login feature where the cookie lives for a long period or an auto-logout feature
where the cookie expires after a short period of time. Here, you should not attempt to use
HTTP sessions; instead you should write your own application-specific logic.
The graphic shows that in persistent JDBC replication, each server in the cluster is connected
to a database using a JDBC connection pool. The HttpSession state is stored in the
database.

Oracle Coherence: Overview +
Storage-disabled: Storage-enabled:
No data replication or Data replication and partitioning
partitioning on these nodes across these nodes
Grid
Application Cache Server
Replicated data
Application Cache Server provides a backup
Network for failover.
Application Cache Server
Distributed cache:
Near cache: In-memory cached
Locally cached objects objects across JVMs
Partitioning refers to the ability of Coherence to load-balance data storage, access, and
management across all the servers in the cluster. For example, when using Coherence data
partitioning, if there are four servers in a cluster, each will manage 25% of the data. And if
another server is added, each server will dynamically adjust so that the five servers will
manage 20% of the data. This data load balancing will occur without any application
interruption and without any lost data or operations. Similarly, if one of those five servers were
to fail, each of the remaining four servers would readjust to managing 25% of the data. Once
again, there is no data loss, including the 20% of the data that was being managed on the
failed server.
While the partitioning feature dynamically load balances data evenly across the entire server
cluster, replication ensures that a desired set of data is always available and up-to-date at all
times in the cluster. Replication allows operations running on any server to obtain the data
that they need locally, at basically no cost, because that data has already been replicated to
that server. The only downside of partitioning is that it introduces latency for data access, and
in most applications the data access rate far outweighs the data modification rate. To
eliminate the latency associated with partitioned data access, Coherence can employ “near
caching.” Frequently and recently used data from the partitioned cache are maintained on the
specific servers that are accessing that data within a near cache, and this local data is kept
up-to-date by using event-based invalidation.

Coherence*Web and WebLogic Clusters +
Storage-disabled: Storage-enabled:
Multiple WebLogic clusters of servers Cached HTTP sessions on a
configured to use the Coherence*Web plug-in Coherence distributed cache
Coherence Distributed Data Grid

Cluster Cache Server
These JVMs
can also be
WebLogic
Cache Server managed
Coherence
Cluster Network servers.
Cache Server
Coherence*Web is not a replacement for WebLogic Server's in-memory HTTP state

replication services. However, Coherence*Web should be considered when an application
has large HTTP session state objects, when running into memory constraints due to storing
HTTP session object data, or if you have an existing Coherence cluster and want to offload
HTTP Session storage to a Coherence cluster.
Coherence*Web is configured with local storage disabled. This means that although the
WebLogic server instance is a member of the Coherence cluster, it is not a data storage
member of the cluster. Storage-disabled members rely on other Coherence cluster members
to manage and replicate data.
By default, Coherence*Web creates a single HTTP session across all web applications for
each client and scopes the session attributes to each web application. This means that if a
session is invalidated in one web application, that same session is invalidated for all web
applications in WebLogic Server that are using Coherence*Web. If you do not want this
behavior, a potential work-around is to edit the <cookie-path> element of the
weblogic.xml descriptor.

Session Persistence Comparison
PersistenceType Durability PerformanceR ating

(1-5: 1=fastest)
In-memory Veryd urable. Failureonly 1
occurs if the primary and
secondary machines crash
simultaneously.
Coherence*Web Very durable. Failure only 2
occurs if the primary and
backup cache servers
crash simultaneously.
File Extremelydurable.Failure 4
only occurs if there are
disk or file errors.
Database Extremelydurable.Failure 5
only occurs if the database
fails.

Section Summary
In this section, you should have learned how to describe the

performance and reliability characteristics of WebLogic’s
session persistence features.

Tuning Session Persistence
This practice covers comparing the performance between
in-memory, file, and JDBC session persistence strategies.

Tuning JDBC Performance

Objectives

following:
• Tune the number of database connections
• Configure test connections on reserve efficiently
• Configure to cache prepared and callable statements

• Configure the pinned-to-thread property to increase
performance
• Describe the various Isolation Levels and their impact on
performance
• Enable Exalogic optimizations
• Tuning the Oracle RDBMS: a primer

Agenda

• Cache prepared and callable statements
• Describe Transactions, the various Isolation Levels and
their impact on performance

Connection Pools
• Creating a connection to the database is an expensive

operation.
• Connection pools:
– Remove the overhead of establishing connections
– Improve server
connections performance
among by sharing
multiple users database
accessing the Web
application
Application server
Connection pool
Connection
available
RDBMS
Java Connection
application in use
Database
One of the advantages of using connection pools over direct connections is that the
connections already exist when applications want to connect to a database. This saves
applications the overhead of creating connections. Also the middle tier software (WLS) can
apply load balancing by assigning connections to applications using a database, and then
releasing and making them available for other applications when they are no longer in use.
Load balancing can also include dynamic growing and shrinking of the number of connections
in a connection pool to adapt to changing load conditions. Connection pools add other
advantages such as the potential for applying security to a connection.
The graphic shows that the application server hosts the connection pool and that applications
use the pools on an as-needed basis.

Connection Pool Sizing
• Creating DB connections is time consuming.

– Ideally, set Min = Max to avoid creating connections on
demand
– If DB connections are limited, set Min to handle normal load
and Max to handle peak loads; also, enable pool shrinking
• Most applications need only one connection per DB per
concurrent request being processed
– Typically, that means one per execute thread.
– Consider <max-threads-constraint > to prevent
contention with limited numbers of DB connections
Each JDBC data source has a pool of JDBC connections that are created when the data
source is deployed or at server startup. Applications use a connection from the pool and then
return it when they have finished using the connection. Connection pooling enhances
performance by eliminating the costly task of creating database connections for the
application.
Creating a database connection is a relatively expensive process in any environment.
Typically, a connection pool starts with a small number of connections. As client demand for
more connections grows, there may not be enough in the pool to satisfy the requests. The
WebLogic Server creates additional connections and adds them to the pool until the
maximum pool size is reached.
One way to avoid connection-creation delays for clients using the server is to initialize all
connections at server startup, rather than on demand as clients need them. Set the initial
number of connections equal to the maximum number of connections on the Connection Pool
tab of your data source configuration. However, you still need to determine the optimal value
for the Maximum Capacity as part of your preproduction performance testing.

Connection Pool Sizing
• Monitor Data Source statistics in the Console:

– Active Connections High Count
– Waiting on Connection High Count
– Wait Seconds High Count
• Tune accordingly:
– Maximum Capacity
– Connection Reserve Timeout (default: 10 sec)

Database Listener Timeouts
• When WebLogic Server is under heavy loads (that is, high

CPU utilization), the database listener may timeout and
throw an exception while creating a new connection.
– The exception thrown is a ResourceDeadException and
the driver exception is "Socket read timed out."
• The workaround is to increase the timeout of the database
server by using the following:
– sqlnet.ora: SQLNET.INBOUND_CONNECT_TIMEOUT =
180
– listener.ora:
INBOUND_CONNECT_TIMEOUT_<listener_name> = 180

JDBC and Application Design
JDBC application performance is most affected by application

design and implementation:
• Cache a data source to avoid excessive JNDI lookups.
• Reuse a connection within a single UI transaction.
• Do not cache a connection across multiple UI transactions.

• Avoid non-XA work within distributed transactions.
• Use PreparedStatements and batching.
JDBC application performance gains are determined by how the application is designed. The
number and location of clients, size, and structure of database management system (DBMS)
tables and indexes, and the number and types of queries affect application performance.
Whenever possible, collect a set of data operations and submit an update transaction in one
statement. This approach results in better performance than using separate statements and
commits.
A simple way to boost JDBC application performance and avoid wasting resources:
• JNDI lookups are relatively expensive. So caching an object that requires a lookup in
client code or application code avoids incurring additional performance cost.
• When client or application code has a connection, maximize the reuse of this connection
rather than closing and reacquiring a new connection. Although acquiring and returning
an existing creation is much less expensive than creating a new one, excessive
acquisitions and returns to pools create contention in the connection pool and degrades
application performance.
• Do not hold connections any longer than is necessary to achieve the work needed.
Getting a connection once, completing all necessary work, and returning it as soon as
possible provides the best balance for overall performance.

Section Summary
In this section, you should have learned how to tune connection

pools.

Agenda


Connection Testing
• The WebLogic Server can test a connection from the

connection pool before giving it to a client.
• The Test Connection On Reserve parameter enables
automatic testing of database connection.
• Connections can
using the Test also be tested
Frequency periodically for validity by
parameter.
To access the JDBC connection pool parameters click Configuration > Connection Pool >
Advanced after having selected the JDBC data source you want to modify.
To make sure that the database connections in a data source remain healthy, you should
periodically test the connections. The WebLogic Server provides automatic testing that you
configure with options on the data source so that the WebLogic Server makes sure that
database connections remain healthy. To configure automatic testing options for a data
source, you set the following options either through the Administration Console or through
WebLogic Scripting Tool (WLST) by using JDBCConnectionPoolParamsBean:
• Test Reserved Connections: Select this option to test each connection before
assigning to a client. This may add a slight delay to the request, but it guarantees that
the connection is healthy. You must also set a Test Table Name.
- When Test Connections On Reserve is enabled, the WebLogic Server tests the
connection by using the query specified in Test Table Name when your application
requests a connection from the data source.
- Testing reserved connections can cause a delay in satisfying connection requests,
but it makes sure that the connection is viable when the application gets the
connection. You can minimize the impact of testing-reserved connections by
tuning Seconds to Trust an Idle Pool Connection.

• Test Frequency: Use this attribute to specify the number of seconds between tests of
unused connections. The WebLogic Server tests unused connections, and closes and
replaces any faulty connections. You must also set the Test Table Name.
• Test Table Name: Use this attribute to specify a table name to use in a connection test.
You can also specify the SQL code to run in place of the standard test by entering SQL
followed by a space and the SQL code you want to run as a test. The Test Table Name
is required to enable any database connection testing.
• Seconds to Trust an Idle Pool Connection: Use this option to specify the number of
seconds after a connection has been proven to be okay that the WebLogic Server trusts
the connection is still viable and will skip the connection test, either before delivering it to
an application or during the periodic connection testing process. This option is an
optimization that minimizes the performance impact of connection testing, especially
during heavy traffic.

Shrink Frequency
• The WebLogic Server periodically shrinks the connection

pool to its initial capacity based on the usage.
• The Shrink Frequency parameter is used to specify the
number of seconds to wait before shrinking a connection
pool.
• When set to 0, shrinking is disabled. This can be helpful in
a production environment.
The screenshot shows how to configure the pool shrink frequency by using the Administration
Console.

Section Summary
In this section, you should have learned how to configure test

connections on reserve efficiently.

Tuning JDBC Connection Pools
• Tuning JDBC connection pool settings
• Using Grinder to load the database and capturing
performance metrics
• Tuning JDBC connection pool settings again

• Using Grinder again and comparing the results

Agenda


Row Prefetch
• Row prefetching improves performance by fetching

multiple rows from the server to the client in one server
access.
• The optimal prefetch size depends on the particulars of the
query.
• In general, increasing this number will increase
performance, until a particular value is reached; usually set
between 10 and100.
Note: This is applicable only for external clients, not for clients
in the same Java Virtual Machine (JVM) as the WebLogic
Server.
Row Prefetch Enabled: Enables multiple rows to be “prefetched” (that is, sent from the
server to the client) in one server access. When an external client accesses a database using
JDBC through the WebLogic Server, row prefetching improves performance by fetching
multiple rows from the server to the client in one server access. The WebLogic Server ignores
this setting and does not use row prefetching when the client and the WebLogic Server are in
the same JVM.
Row Prefetch Size: Specifies, if row prefetching is enabled, the number of result set rows to
prefetch for a client. The optimal prefetch size depends on the particulars of the query. In
general, increasing this number will increase performance, until a particular value is reached.
At that point, further increases do not result in any significant performance increase. Very
rarely will increased performance result from exceeding 100 rows.
Minimum value: 2
Maximum value: 65536
The screenshot shows how to configure Row Prefetch by using the Administration Console.

Statement Caching
• The three types of statements in JDBC are:

– Statements (not cached)
– Prepared statements
– Callable statements
• The prepared
cacheable andstatements and callable
improve overall statements
performance are
through
reuse.
• The statement cache type determines how the cache is
refreshed:
– LRU: Replace least recently used
– FIXED: No replacement
When you use a prepared statement or callable statement in an application or Enterprise

JavaBean (EJB), there is considerable processing overhead for the communication between
the application server and the database server. To minimize the processing costs, the
WebLogic Server can cache prepared and callable statements used in your applications.
When an application or EJB calls any of the statements stored in the cache, the WebLogic
Server reuses the statement stored in the cache.
Each connection in a data source has its own individual cache of prepared and callable
statements. However, you configure statement cache options per data source. That is, the
statement cache for each connection in a data source uses the statement cache options
specified for the data source, but each connection caches its own statements.

Statement Cache Type: This is the algorithm used for maintaining the prepared statements
stored in the statement cache. There are two types of statement cache:
• LRU: When you select LRU (Least Recently Used, the default) as the Statement Cache
Type, the WebLogic Server caches prepared and callable statements used on the
connection until the statement cache size is reached. When an application calls
Connection.prepareStatement(), the WebLogic Server checks to see whether
the statement is stored in the statement cache.
- If so, the WebLogic Server returns the cached statement (if it is not already being
used).
- If the statement is not in the cache, and the cache is full (the number of statements
in the cache equals the statement cache size), then the WebLogic Server
determines which existing statement in the cache was the least recently used and
replaces that statement in the cache with the new statement.
• FIXED: The first fixed number of prepared and callable statements are cached. When
you select FIXED as the Statement Cache Type, the WebLogic Server caches prepared
and callable statements used on the connection until the statement cache size is
reached. When additional statements are used, they are not cached. With this statement
cache algorithm, you can inadvertently cache statements that are rarely used.
In many cases, the LRU algorithm is preferred because rarely used statements will eventually
be replaced in the cache with frequently used statements.
Statement Cache Size: The Statement Cache Size attribute determines the total number of
prepared and callable statements to cache for each connection in each instance of the data
source. By caching statements, you can increase your system performance. However, you
must consider how your DBMS handles open prepared and callable statements. In many
cases, the DBMS will maintain a cursor for each open statement. This applies to prepared
and callable statements in the statement cache. If you cache too many statements, you may
exceed the limit of open cursors on your database server. Setting the size of the statement
cache to 0 turns off statement caching.
• Minimum value: 0
• Maximum value: 1024
Usage Restrictions for the Statement Cache: Using the statement cache can increase
performance, but you must consider its limitations before you decide to use it. Note the
following restrictions when using the statement cache:
• Calling a cached statement after a data definition language (DDL) change
- Prepared statements stored in the cache refer to specific database objects at the
time the prepared statement is cached. If you perform any DDL operations on
database objects referenced in prepared statements stored in the cache, the
statements may fail the next time you run them.
- Likewise, if you add, delete, or rearrange columns in a table, prepared statements
stored in the cache are likely to fail when run again.
•
Using setNull in a prepared statement
- If you cache a prepared statement that uses a setNull bind variable, you must
set the variable to the proper data type. If you use a generic data type, data may
be truncated or the statement may fail when it runs with a value other than null.

Statement Cache Tuning
• WLS caches JDBC Statement objects.

• Cache is per connection:
– # of cached statements = # of conns * cache size
– Prepared statements
– Callable statements
• Tune according to hit/miss ratio.

Connection Pinned to Thread
• A data source can dedicate or “pin” a connection to the

first server thread that requests it.
• This capability:
– May increase performance by eliminating potential
contention for connections by threads
– Is not supported with multidata sources or Oracle Real
Application Clusters (RAC)
To minimize the time it takes for an application to reserve a database connection from a data
source and to eliminate contention between threads for a database connection, you can
enable the Pinned-To-Thread property in the connection properties.
When Pinned-To-Thread is enabled, the WebLogic Server pins a database connection from
the data source to an execution thread the first time an application uses the thread to reserve
a connection. When the application finishes using the connection and calls
connection.close(), which otherwise returns the connection to the data source, the
WebLogic Server keeps the connection with the execute thread and does not return it to the
data source. When an application subsequently requests a connection using the same
execute thread, the WebLogic Server provides the connection already reserved by the thread.
There is no locking contention on the data source that occurs when multiple threads attempt
to reserve a connection at the same time and there is no contention for threads that attempt to
reserve the same connection from a limited number of database connections.
In this release, the Pinned-To-Thread feature does not work with multidata sources or Oracle
RAC. These features rely on the ability to return a connection to the connection pool and
reacquire it if there is a connection failure or if a connection identity does not match.

Section Summary
In this section, you should have learned how to configure

Statement Caching and Batch Updates.

Testing Performance of JDBC Statement Caching
• Tuning the statement caching size
• Using Grinder to load the application and compare its
performance against previous runs

Testing Performance of Pinned to Thread
• Enabling the pinned to thread feature
• Tuning different statement cache sizes
• Using Grinder to load the database and comparing the
performance impact of using pinned to thread

Configuring Batch Updates
• Deploying an application that compares regular versus
batch insert statements
• Comparing the performance of each

Agenda


Combining Transactions
• Transaction operations are expensive and can significantly

degrade performance.
• Combining multiple operations into a single transaction can
help improve performance.
• By default, JDBC connections have auto-commit enabled.

– Every JDBC statement is executed as a separate
transaction.
– You can use Connection.setAutoCommit(false) to
turn this off.
– You can use Connection.start() ,
Connection.commit() , and Connection.rollback()
to demarcate transactions.
Transactions require processing overhead, which can reduce your system’s performance
when compared to a transaction-free (but more easily corruptible) application. Transactions
are a lot of work for a database, and databases by default maintain many kinds of resources
to ensure that ACID properties apply to a transaction.
Combine transactions by moving multiple operations into a single transaction.
By default, JDBC connections work in auto-commit mode, which means that every operation
sent to the database automatically executes as a separate transaction. Turning off auto-
commit and explicitly defining transaction boundaries around multiple database calls can
improve performance. To turn off auto-commit, use the
Connection.setAutoCommit(false) command.
The JDBC API also provides aConnection.getAuto Commit() method that returns the
current auto-commit mode. W hen you turn off auto-commit, you will need to use the
transaction demarcation methods ofConnection.start(), Connection.commit() and
Connection.rollback().

Batch Updates
• A typical application might involve multiple data

manipulation statements (insert, update, delete).
• Combining them as appropriate can help improve
performance.
• Statement.addBatch() and can help in batching

Statement.executeBatch()
updates.

Transaction Isolation Levels
• The transaction isolation level controls the activity of a

transaction by balancing:
– The integrity of the data
– The concurrent access speed
• Four levels are available:

– TRANSACTION_READ_UNCOMMITTED
– TRANSACTION_READ_COMMITTED
– TRANSACTION_REPEATABLE_READ
– TRANSACTION_SERIALIZABLE
Two aspects of transactions can be controlled:

• Integrity of the data: Making sure that the data is up-to-date and exact
• Concurrent access: Accelerating the access to data from multiple clients
The higher the integrity of the data, the slower the access speed. These two variables can be
controlled by the transaction isolation level.
But what is meant by integrity? A problem with integrity means that there is a discrepancy
between the data in one transaction and the data in another transaction. Here are a few
scenarios involving a problem with integrity:
• Dirty reads: A row has been modified by another process, and that modification has not
been committed yet.
• Nonrepeatable read: A row has been modified by another process that did commit the
modification.
• Phantom row: A row has been deleted by another process, and that deletion has not
been committed yet.

Operations Permitted at Each Isolation Level
Non-
Dirty Phantom
Level repeatable
Read Read
Read
READ
Permitted Permitted Permitted
UNCOMMITTED
READ
COMMITTED X Permitted Permitted
REPEATABLE
READ X X Permitted
SERIALIZABLE
X X X
Dirty read: You are permitted to read uncommitted, or dirty , data. You can achieve this effect
by just opening an OS file that someone else is writing and reading whatever data happens to
be there. Data integrity is compromised, foreign keys are violated, and unique constraints are
ignored.
Non-repeatable read: This simply means that if you read a row at time T1 and try to reread
that row at time T2, the row may have changed. It may have disappeared, it may have been
updated, and so on.
Phantom read: This means that if you execute a query at time T1 and re-execute it at time
T2, additional rows may have been added to the database, which may affect your results.
This differs from a non-repeatable read in that with a phantom read, data you already read
hasn't been changed, but instead, more data satisfies your query criteria than before.

Transaction Isolation Levels
Choosing the right isolation level can significantly improve

performance.
e Isolation Level
c
n TRANSACTION_SERIALIZABLE D
a a
m ta
r TRANSACTION_REPEATABLE_READ
o In
rf te
e TRANSACTION_READ_COMMITTED g
P r
it
TRANSACTION_READ_UNCOMMITTED y
The four transaction isolation levels available are:

• TRANSACTION_READ_UNCOMMITTED: Dirty reads, nonrepeatable reads, and phantom
reads are allowed. This is the most permissive isolation level.
• TRANSACTION_READ_COMMITTED: No dirty reads are allowed. Nonrepeatable reads
and phantom reads are allowed.
• TRANSACTION_REPEATABLE_READ: No dirty reads and no nonrepeatable reads are
allowed. Phantom reads are allowed.
• TRANSACTION_SERIALIZABLE: No dirty reads, no nonrepeatable reads, and no
phantom reads are allowed. This is the most restrictive isolation level.

Section Summary
In this section, you should have learned how to describe the

various Isolation Levels and their impact on performance.

Agenda


Exalogic Optimizations
• The Sockets Direct Protocol (SDP) is a transport-agnostic

networking protocol for Remote Direct Memory Access
(RDMA) network fabrics.
• SDP-enabled JDBC thin driver
– JDBC URL: Replace "( PROTOCOL = TCP)" with "(PROTOCOL
= SDP)"
• Requires Oracle DB (Exadata) with
– SDP Listener configured ( listener.ora ).
– Use ActiveGridlink data source instead of Multi Datasource.
SDP was srcinally defined by the Software Working Group (SWG) of the InfiniBand Trade
Association. Initially designed for InfiniBand (IB), SDP has now been redefined as a transport-
agnostic protocol for RDMA network fabrics.
SDP defines a standard wire protocol over an RDMA fabric to support stream sockets
(SOCK_STREAM). SDP uses various RDMA network features for high-performance zero-
copy data transfers. The purpose of the Sockets Direct Protocol is to provide an RDMA-
accelerated alternative to the TCP protocol on IP. The goal is to do this in a manner which is
transparent to the application.

Section Summary
In this section, you should have learned how to enable Exalogic

optimizations.

Agenda


Oracle Database Tuning
• Number of processes
– It includes simultaneous users and Oracle background
processes.
– The default setting (150) is usually too low to accommodate
a large number of users.
• System Global Area (SGA)
– It contains data and control information for each instance.
– It is critical to performance.
– Oracle 11g introduced a new feature called Automatic
Memory Management (AMM), which allows for a simplified
and dynamic configuration of the SGA.
• Maximum opened cursor
– It should be increased if you are using JDBC statement
caching.
You should refer to Oracle Database 2 Day + Performance Tuning Guide 11g Release 2
(11.2) documentation for more detailed information about tuning the Oracle database. The
few parameters mentioned here are commonly tuned in an Oracle WebLogic Server
environment.

Oracle Database Tuning
• Database block size

– Smallest unit of I/O
– Set at database creation
– Default to 8192 bytes
• PGA_AGGREGATE_TARGET
– It dictates performance of large sorts.
– It should be used to allow Oracle to automatically adjust and
size the memory needed for expensive operations like group-
by, hash-join, bitmap merge, etc.
– If Automatic Memory Management (AMM) has been set,
PGA_AGGREGATE_TARGET is redundant.
– Moving forward, AMM should be used in preference to
manual memory configuration.
OLTP applications tend to perform better with a smaller DB_BLOCK_SIZE (2048 or 4096
bytes) while data warehouses usually set this parameters to larger values (16 or even 32 Kb)
Setting PGA_AGGREGATE_TARGET to a nonzero value has the effect of automatically
setting the WORKAREA_SIZE_POLICY parameter to AUTO. This means that SQL working
areas used by memory-intensive SQL operators such as sort, group-by, hash-join, bitmap
merge, and bitmap create will be automatically sized. The memory allocated for
PGA_AGGREGATE_TARGET is not subtracted fromthe memory allocated for theSystem
Global Area (SGA)
Best practice: Use Automatic Memory Management (AMM) – Automatic memory
management is configured using two Oracle RDBMS initialization parameters:
MEMORY_TARGET: The amount of shared memory available for Oracle to use when
dynamically controlling the SGA and PGA. This parameter is dynamic, so the total amount of
memory available to Oracle can be increased or decreased, provided it does not exceed the
MEMORY_MAX_TARGET limit. The default value is "0".

MEMORY_MAX_TARGET: This defines the maximum size the MEMORY_TARGET can be
increased to without an instance restart. If the MEMORY_MAX_TARGET is not specified, it
defaults to MEMORY_TARGET setting.
When using automatic memory management, the SGA_TARGET and
PGA_AGGREGATE_TARGETact as minimum size settings for their respective memory
areas. To allow Oracle to take full control of the memory management, these parameters
should be set to zero.

Section Summary
In this section, you should have learned how to tune an Oracle

database for use with WebLogic.

Tuning an Oracle Database for WebLogic
This practice covers configuring some basic Oracle RDBMS
parameters for running with WebLogic.

Configuring Work Managers

Objectives

following:
• Describe how WebLogic handles concurrent client
requests
• Monitor thread pool size and usage

• Prioritize application processing by using work managers
• Tune work managers by using request classes and
constraints
• Update an application to use a work manager
• Configure stuck thread handling

Agenda
• Default W ebLogic Request Handling

• Work Manager Concepts

WebLogic Server Threads
• WebLogic Server (WLS) processes concurrent requests by

assigning each request to a thread from an available pool.
• WebLogic is “self-tuning” and automatically adjusts the
number of threads in the pool as needed.
• Some threads are dedicated to various background tasks.

• By default, all requests are given equal priority.
Client Server
Client
Client
WebLogic uses a single thread pool, in which all types of work are executed. The WebLogic
Server prioritizes work based on rules that you define, and run-time metrics, including the
actual time it takes to execute a request and the rate at which requests are entering and
leaving the pool.
The common thread pool changes its size automatically to maximize throughput. The
incoming request queue monitors throughput over time and, based on history, determines
whether to adjust the thread count. For example, if historical throughput statistics indicate that
a higher thread count increased throughput, WebLogic increases the thread count. Similarly, if
statistics indicate that fewer threads did not reduce throughput, WebLogic decreases the
thread count. This new strategy makes it easier for administrators to allocate processing
resources and manage performance, avoiding the effort and complexity involved in
configuring, monitoring, and tuning custom execution queues.

Monitoring a Server Thread Pool
Print the current stack

trace for each thread.
View the current

thread pool
statistics.
Select a server and click its Monitoring > Threads tab. The first table provides general
information about the status of the thread pool. The second table provides information about
individual threads. The available columns in the first table include:
• Execute Thread Tot al Count: The total number of threads in the pool
• Execute Thread Idle Count: The number of idle threads in the pool. This count does
not include standby threads and stuck threads. The count indicates threads that are
ready to pick up new work when it arrives.
• Pending User Request Count: The number of pending user requests in the priority
queue. The priority queue contains requests from internal subsystems and users. This is
just the count of all user requests.
• Hogging Thread Count: Returns the threads that are being hogged by a request right
now. These threads will either be declared as stuck after the configured timeout or will
return to the pool before that. The self-tuning mechanism will backfill if necessary.
• Throughput: The mean number of requests completed per second

To display the current Java stack trace for active threads, click the Dump Thread Stacks
button.

Monitoring Server Threads
Current status and

statistics for each thread
The second table on the server’s Monitoring > Threads tab provides the status and statistics
for individual threads, including:
• Total Requests: The number of requests that have been processed by the thread
• Current Request: A String representation of the request this thread is currently
processing
• Transaction: The XA transaction on whose behalf the execute thread is currently
working
• User: The name associated with this thread
• Idle: Returns the value “true” if the execute thread has no work assigned to it
• Stuck: Returns “true” if the execute thread is being hogged by a request for much more
than the normal execution time as observed by the scheduler automatically. If this
thread is still busy after the stuck thread max time, it is declared as stuck.

Section Summary

• Describe how WebLogic handles concurrent client
requests
• Monitor thread pool size and usage

Agenda
• Default We bLogic Request Handling

• Work Manager Concepts

Work Managers
• Work managers enable administrators to prioritize

application work.
• WebLogic allocations threads based on the priority of the
work requested.
• Service level agreements

requirements for a specific(SLAs) describe
application performance
or component:
– Percentage of server resources to use
– Maximum response time
– Maximum number of threads to use
• An SLA is implemented on WebLogic by
using a work manager.
WebLogic allows you to configure how your application prioritizes the execution of its work.
Based on rules that you define and by monitoring actual runtime performance, WebLogic can
optimize the performance of your application and maintain SLAs. You tune the thread
utilization of a server instance by defining rules and constraints for your application by
defining a work manager and applying it either globally to WebLogic domain or to a specific
application component. Each distinct SLA requirement needs a unique work manager.
You can associate one set of scheduling guidelines for one application, and another set of
guidelines for other applications. At run time, WebLogic uses these guidelines to assign
pending work and queued requests to execution threads.
You can define multiple work managers. The appropriate number depends on how many
distinct demand profiles exist across the applications you host on WebLogic.
You can configure work managers at the domain level, application level, and module level.

Default Work Manager
Applications use the default work manager if they are not

assigned to a specific work manager.
• All applications are given equal priority.
• A default fair share of threads is given to each application.
• If no work manager is explicitly assigned to an application,

it uses the default work manager.
• The default work manager can be overridden by creating
and configuring a global work manager called default.
To handle thread management and perform self-tuning, WebLogic implements a default work
manager. This work manager is used by an application when no other work managers are
specified in the application’s deployment descriptors. In many situations, the default work
manager may be sufficient for most application requirements. WebLogic’s thread-handling
algorithms assign to each application its own fair share by default. Applications are given
equal priority for threads and are prevented from monopolizing them.
You can override the behavior of the default work manager by creating and configuring a
global work manager called “default.” This enables you to control the default thread-handling
behavior of W ebLogic.

Work Manager Scope
• Work managers are associated with applications by using

XML deployment descriptors.
• Global work managers are defined in the console and are
available to any application.
• Application-scoped work managers

within the same deployment are defined “inline”
descriptors.
Application
Edit the descriptor EJB1 Global WM1

file or use a
deployment plan. EJB2
Configured in
config.xml
WM2
You can create global work managers that are available to all applications and modules
deployed on a server in the WebLogic administration console and in config.xml. An
application uses a globally defined work manager as a template. Each application creates its
own instance that handles the work associated with that application and separates that work
from other applications. This separation is used to handle traffic directed to two applications
that are using the same dispatch policy. Handling each application’s work separately allows
an application to be shut down without affecting the thread management of another
application. Although each application implements its own work manager instance, the
underlying components are shared.
In addition to globally scoped work managers, you can also create work managers that are
available only to a specific application or module. You can define application-scoped work
managers in the WebLogic administration console and in the following descriptors:
weblogic-application.xml, weblogic-ejb-jar.xml, and weblogic.xml. If you do
not explicitly assign a work manager to an application, it uses the default work manager.

Work Manager Architecture
• Request classes describe the guidelines that WebLogic

uses to allocate threads from the pool.
• Constraints define additional boundary conditions that limit
when threads are allocated (can override request class).
• Error-handling policies define how to handle deadlocked or

timed-out threads.
Constraint A
Request Class Work Manager
Constraint B
A request class expresses a scheduling guideline that WebLogic uses to allocate threads to
requests. Request classes help ensure that high-priority work is scheduled before less
important work, even if the high-priority work is submitted after the low-priority work.
WebLogic takes into account how long it takes for requests to each module to complete.
A constraint defines the minimum and maximum number of threads allocated to execute
requests and the total number of requests that can be queued or executed before WebLogic
begins rejecting requests.
In response to stuck threads, you can define an error-handing policy that shuts down the work
manager, moves the application into administration mode, or marks the entire server instance
as failed.

Section Summary
In this section, you should have learned how to describe work

manager basic concepts.

Configuring and Using Work Managers
• Creating a global work manager
• Creating a global request class and global constraint
• Assigning a request class to a work manager
• Assigning a work manager to an application
• Creating and assigning a work manager at the application
level
• Comparing work managers under load

Configuring Work Manager Constraints
• Configuring a work manager to use a minimum threads
constraint
• Configuring a work manager to use a maximum threads
constraint
• Assigning each work manager to different applications
• Comparing work managers under load

Tuning J2EE Applications

Objectives
After completing this lesson, you should be able to tune the

following:
• Web Applications:
– Configure page check and servlet reload intervals
– Consider web servers to serve static content

– Configure performance features for session persistence
– Precompile JavaServer Pages (JSPs)
– Enable use of custom JSP tags

Objectives
After completing this lesson, you should be able to tune the

following:
• Enterprise JavaBeans:
– Configure the optimal size for stateless session pools
– Configure and optimize message-driven beans

– Optimize performance of stateful session beans
– Optimize session persistence and cache

Agenda
• Tuning Web Applications

• Tuning Enterprise JavaBeans

What Is a JavaServer Page?
JavaServer Pages (JSPs):

• Are templates for dynamic content
• Extend HTML with custom Java code
• Are compiled into servlets by the WebLogic Server (WLS)
• Allow for the division of labor into content production and
programming
<H1>Today is:</H1> <%= new java.util.Date() %>
Today is: Mon May 09 17:10:25 EDT 2006
JavaServer Page (JSP) is a JavaSoft standard specification for combining Java with HTML to
provide dynamic content on web pages. JSPs are more convenient to write than HTTP
servlets because they allow you to embed the Java directly into your HTML page layout. JSPs
are part of the Java Enterprise Platform.
JSPs are meant to be the Web’s entry point into the Java Platform for the Enterprise (JPE).
JSPs are capable of using the JavaBean methods, and therefore provide access to EJBs and
databases.
The syntax for using JSPs is relatively simple, especially if you are already familiar with HTML
and Java. As you will see, most of the Java code and HTML parts in your JSP are combined
to form one large service method for a servlet class.
The graphic in the slide shows the relationship between the JSP code and the HTML
rendered on a browser.

JSP Request Processing
Requests for JSPs are handled by a special WLS servlet called

JSPServlet, which handles:
• Translation of JSP into a servlet
• Compilation of a resulting servlet into a class
• Execution of JSP
JSPServlet translates
myServlet.jsp _myservlet.java
No changes
JSPServlet
since last
translation compiles
HTML _myservlet.class
JSPServlet
loads/executes
When a URL is provided to the WebLogic Server that maps to an underlying JSP, that JSP
must be executed. A special W LS servlet, the JSPServlet, performs the translation and
execution. First, the JSP is examined to see whether it has been previously translated into its
underlying servlet representation. If not, the JSP is compiled and then executed to service the
request. If the JSP has been previously executed, it is precompiled and the source is
examined. If the code is determined to be newer than the previously compiled version, the
JSP is recompiled and then executed. In either case, the servlet representation of the
requested JSP is invoked by the web container to process the given request.
The graphic shows that a JSP is compiled into a servlet class on its first invocation. From that
point onward, every request is redirected to the compiled servlet class.

Precompiled JSP
• Precompiling JSPs save:

– Initial time for the server translation and compilation
– Memory required for noncompiled JSPs
– Management related to noncompiled JSPs
• JSP precompilation can be done:

– Manually, using WebLogic compilers, such as
weblogic.appc
– Automatically, using the precompile option in
weblogic.xml
• If the precompile option is turned ON, recompilation
of JSPs occurs each time the server restarts.
By deploying precompiled JSPs, the first access to the JSPs can be performed faster
because the server does not have to spend time in translation and compilation. Also, because
the compiled JSPs are loaded into memory, and uncompiled JSPs need not be loaded into
memory, the memory consumption is reduced. Accordingly, the memory management effort is
also reduced.

The appc Application Compiler
• The appc compiler compiles EJBs and JSPs and

generates the classes needed to deploy to WebLogic
Server.
• The appc compiler provides the following benefits:
– Flexibility of compiling all modules of an application
– Validation checks throughout an entire application as well as
individual modules
– Easier identification and correction of errors
– Reduction in time and effort in repeated compilations—when
deploying to multiple servers
• Syntax for using appc:
$> java weblogic.appc -options <application archive file
or directory>
The appc compiler, while compiling EJBs and JSPs, also validates the deployment
descriptors The appc compiler offers the following benefits:
• The flexibility of compiling an entire application, rather than compiling individual modules
separately and combining them into an EAR.
• Validation checks all modules and validates application-level deployment descriptors
against the various modules. Without appc, a user had to extract the individual
components of an EAR and manually execute the appropriate compiler (jspc or ejbc)
to prepare the module for deployment. appc automates this process and makes
additional pre-deployment validation checks that were not previously possible.
• It is easy to identify and correct errors that appc produces. If an error occurs while
running appc from the command line,appc exits with an error message. By contrast, if
you defer compilation to the time of deployment and a compilation error occurs, the
server fails the deployment and goes on with its work. To determine why deployment
failed, you must examine the server output, fix the problem, and then redeploy.
• By running appc before deployment, you potentially reduce the number of time a bean
is compiled. For example, if you deploy a JAR file to a cluster of three servers, the JAR
file is copied to each of the three servers for deployment. If the JAR file was not
precompiled, each of the three servers will have to compile the file during deployment.

Using the precompile Parameter
• You can configure the WebLogic Server to precompile

your JSPs when a web application is deployed or
redeployed.
• Set the precompile parameter to true in the
<jsp-descriptor> element of the weblogic.xml
deployment descriptor.
• Snippet from WEB-INF/weblogic.xml:
<jsp-descriptor>
<precompile>true</precompile>
</jsp-descriptor>
• In UNIX environments, the following parameter would help

address mixed-case JSP names:
-Dweblogic.jsp.windows.caseSensitive=true
When using the precompile parameter in weblogic.xml, recompilation of JSPs will occur
each time the server restarts and when you target additional servers.
The default value for precompile, if not specified, is false.
The server compiles JSP files at first deployment. When the server starts subsequently, the
server checks the timestamps of JSP and compiled servlets. If none has changed, then the
compilation is skipped and the server starts up quite fast. Consider this case where JSPs
names have multiple case, such asshowAcccount.jsp
• When a file named showAccount.jsp is compiled, the JSP compiler generates a
servlet __showaccount.class(all lowercase characters) by default, but the stale
checker searches for __showAccount.class (with a capital A in the name). This can
be a concern in UNIX environments, where the stale checker doesnot find __
showAccount.class and recompiles all such files again. This can cause delays at
startup.
• To overcome this, you can use -Dweblogic.jsp.windows.caseSensitive=true.
If you use this switch, then the stale checker finds the __showAccount.class
because the compiler generated this case-sensitive class name.
You will not see this difference for filenames that are already all lowercase, such as
welcome.jsp.

Tuning JSP: Compilation Tuning
• You can force WebLogic server to never look for JSP

source files by using JspClassServlet.
• Precompile the JSP files.
• Map the jsp extension to JspClassServlet
<servlet>
<servlet-name>JSPClassServlet</servlet-name>
<servlet-
class>weblogic.servlet.JSPClassServlet</servlet-
class>
</servlet>
<servlet-mapping>
<servlet-name>JSPClassServlet</servlet-name>
<url-pattern>*.jsp</url-pattern>
</servlet-mapping>
• No need to package JSP files in WAR

HttpSession Replication Tuning
• WebLogic server replicates only the session’s

new/changed attributes.
• Keep session objects as small as possible.
• Better to store lots of attributes rather than one
• Put only Serializable objects in the session

– Failure to do so will prevent replication
• Monitor primary distributions across cluster
– Uneven distributions likely means the Load Balancer/Proxy
isn’t properly distributing the load

keepgenerated Parameter
• If the keepgenerated parameter is set to true, the

intermediary Java files generated in the JSP compilation
process are saved.
• In production environments, it is a good idea to set this to
false.
The default value for keepgenerated is false. Unless this parameter is set to true, the
intermediate Java files are deleted after they are compiled.
A snippet from weblogic.xml:
<jsp-descriptor>
<keepgenerated>false</keepgenerated>
</jsp-descriptor>

Page Check Interval
• Whenever a JSP or a servlet have been modified, it must

be recompiled by the server.
• The frequency at which a server checks for modifications
to JSPs or servlets can affect performance.
JSPServlet
page translates
myServlet.jsp check _myservlet.java
No changes since last translation JSPServlet

compiles
HTML _myservlet.class
JSPServlet loads/executes
The graphic in the slide shows that the JSP is compiled into a servlet based on whether the
JSP has been changed or not. The server will determine whether the JSP should be
recompiled based on the page check.

Page Check Interval
• The page-check-seconds attribute determines the

frequency at which a server checks for modifications to
JSPs.
– A lower value increases the frequency.
– A higher value decreases the frequency.
– A value of –1 disables this check by the server.

• In production environments, set this attribute to –1 for
better performance.
The page-check-seconds attribute sets the interval at which the WebLogic Server checks
to see whether JSP files have changed and need recompiling. Dependencies are also
checked and recursively reloaded if changed.
• The value –1 means never check the pages. This is the default value in a production
environment.
• The value 0 means always check the pages.
• The value 1 means check the pages every second. This is the default value in a
development environment.
In a production environment, where changes to a JSP are rare, it may be best to set the
attribute to –1. In rare cases, where it does require an occasional compilation or change, set
this value at a reasonably high value such as 3600, according to your tuning requirements.

verbose Parameter
• When the verbose parameter is set to true, debugging

information is printed out to the browser, the command
prompt, and the WebLogic Server log file.
• Though not a huge factor, this can still impact
performance.
• Set the verbose parameter to an appropriate value.
When set to true, debugging information is printed out to the browser, the command prompt,
and the WebLogic Server log file. The default value for verbose if not specified is true.

Servlet Reload Check Interval
• The servlet container frequently verifies for newly compiled

or reloaded servlets.
• The frequency of check is determined by the servlet-
reload-check-secs parameter in the <jsp-
descriptor> element of the weblogic.xml
deployment descriptor.
The servlet-reload-check-secsparameter sets the interval at which the WebLogic

Server will check to see whether a servlet has been modified, and if it has been modified,
reloads it. Dependencies are also checked and recursively reloaded if changed.
• The value –1 means never check the servlet. This is the default value in a production
environment.
• The value 0 means pages are checked on every request.
• The value 1 means check the servlet every second. This is the default value in a
development environment.
A value specified in the console will always take precedence over a manually specified value.

Using Web Servers for Static Content
• Front application servers and clusters with a dedicated

web tier to serve static content such as HTML and image
files.
• Proxy requests for servlets and JSPs to application
servers.
• Perform additional optimizations at the web tier, such as
caching.
Oracle HTTP Server WebLogic Server
HTML, JavaScript, Web application

images, etc. (Servlets, JSPs)
Web servers provide a hosting environment that is optimized for delivering static content and
files over HTTP. They typically support common server features such as single sign-on,
access control, and high availability. It is also common for web servers to provide an
execution environment for simple applications that generate dynamic content using languages
such as PERL.
The WebLogic Server is in fact a web server, because it can provide these services as well.
However, the WebLogic Server is optimized for processing Java application requests and for
serving dynamic web content generated from servlets and JSPs. Therefore, if your web
application includes a significant number of static file resources such as HTML documents,
images, and JavaScript libraries, you may achieve greater performance and scalability by
offloading these static resources from the WebLogic Server to one or more dedicated web
servers.

Session Timeout
• Whenever an HTTP session object is created, it resides in

memory until:
– The session object is invalidated
– The server removes it after a certain period of time
(session timeout)
– The server comes down

• You can set the session timeout parameter by using the
Administration Console:
A larger session timeout value means that the session object stays in memory for a longer
period. The size of the session object and the session timeout value can impact performance,
if there are a large number of users.
The default value for the session timeout if not specified is 3,600 seconds.

Session I nvalidation
• Set the invalidation-interval-secs parameter to

an appropriate value.
• Tuning this parameter can improve the performance of
applications that have high traffic.
The WebLogic Server does periodic checks for timed-out and invalid sessions. This is done to
delete old sessions and free up memory. Theinvalidation-interval-secsparameter
controls the time between the memory cleanup.
The default value for invalidation-interval-secsif not specified is 60 seconds.

Page Directive and HTTP Session
• The page directive defines attributes that apply to an

entire JSP.
• By default, the JSP engine creates a session object when
a page directive is used in a JSP.
• If there
false asisthe
novalue
need for
forthe
the session
HttpSession in a JSP, assign
attribute.
– Avoids the unnecessary creation of session objects
– Reduces overhead on memory and the garbage collector
– Could increase performance
• An example use of directives:
<%@ page session="false"%>
If you do not specify the attribute session as false, then by default a session object is created.
Consider the case that customer has completed a session, but then inadvertently tries to
connect back to the completed session, then the JSP indicates that the session is no longer
valid.

Using Custom JSP Tags
Oracle provides three specialized JSP tags that you can use in
your JSPs:
• cache: Enables caching the work that is done within the
body of the tag
• process : EnablesJSPs
parameter–based you to control the flow of query
• repeat: Enables you to iterate over many different types
of sets, including Enumerations, Iterators, Collections, and
Arrays of Objects
Oracle provides three specialized JSP tags that you can use in your JSPs: cache, repeat,
and process. These tags are packaged in the weblogic-tags.jar tag library JAR file.
This JAR file contains classes for the tags and a tag library descriptor (TLD). To use these
tags in your JSP:
• Copy the weblogic-tags.jar file from the ext directory of your WebLogic Server
installation to the WEB-INF/lib directory of the web application containing the JSPs.
• Reference this tag library descriptor in the <taglib> element of the Java EE standard
web application deployment descriptor, web.xml. For example:
<taglib>
<taglib-uri>weblogic-tags.tld</taglib-uri>
<taglib-location>
/WEB-INF/lib/weblogic-tags.jar
</taglib-location>
</taglib>
• Reference the tag library in your JSP with the taglib directive. For example:
<%@ taglib uri="weblogic-tags.tld" prefix="wl" %>

Using the WebLogic cache Tag
• Caching data can boost performance significantly.

• The WebLogic cache tag enables caching the work that is
done within the body of the <wl:cache> tag.
• The cache tag supports both output and input caching.
• The tag supports refreshing and flushing the cache at

various scopes.
Example of using the WebLogic cache tag:
<wl:cache name="holidaycache"
key="parameter.holidaytable"scope="application">
// Retrieve Holidays and output it to the page
</wl:cache>
The cache tag enables caching the work that is done within the body of the tag. It supports
both output (transform) data and input (calculated) data. Output caching refers to the content
that is generated by the code within the tag. Input caching refers to the values to which
variables are set by the code within the tag. Output caching is useful when the final form of
the content is the important thing to cache. Input caching is important when the view of the
data can vary independently of the data calculated within the tag. If one client is already
recalculating the contents of a cache and another client requests the same content, it does
not wait for the completion of the recalculation; instead, it shows whatever information is
already in the cache. This is to ensure that the web site does not come to a halt because a
cache is being recalculated. Additionally, the async attribute means that no one, not even the
user that initiates the cache recalculation, waits. You can force the refresh of a cache by
setting the _cache_refresh object to true in the scope that you want affected. For example,
to refresh a cache at session scope, specify the following:
<% request.setAttribute("_cache_refresh", "true"); %>
If you want all caches to be refreshed, set the cache to the application scope.
Flushing a cache forces the cached values to be erased; the next time the cache is accessed,
the values are recalculated. To flush a cache, set its flush attribute to true.

Web Application Tuning
JSP and Servlet Output Buffer Tuning
• Web Container buffers output while building response:
– Data flushed to client once buffer is full
– Cannot forward request once data sent to client
— Set buffer size accordingly
•
ServletResponse.setBufferSize(int)
• <@page contentType="text/html;
buffer="64kb" %>
– WLS default buffer size is 8 kb.
– WLS default buffer size on Exalogic is 64 kb.
• Avoid calling flush() and close() on
ServletOutputStream
Since any values explicitly specified in weblogic.xml override the defaults, be careful when
moving an application from Development to Production since frequent checking for changes
can cause poor performance due to the disk I/O required to look for changes.
Calling flush() or close() (which does not actually close but calls flush()) can force WLS to
chunked transfer encoding if the content length has not been set prior to the call. Although
chunked transfer encoding works fine, there is additional overhead to using it versus the
normal response encoding.

Section Summary

• Configure page check and servlet reload intervals
• Consider web servers to serve static content
• Precompile JSPs
• Configure performance features for session persistence
• Consider using Coherence*Web to store session
information
• Enable use of custom JSP tags

Tuning JSP
• Using the JSP compiler weblogic.appc to precompile
JSPs
• Improving performance by disabling page check

Agenda
• Tuning Web Applications

• Tuning Enterprise JavaBeans

Enterprise JavaBeans
• Enterprise JavaBeans (EJBs) standardize the

development and deployment of server components built
in Java.
• The EJB specification discusses four types of objects:
– Stateless session beans
– Stateful session beans
– Singleton session beans
– Message-driven beans
• Note that entity beans specifications are considered legacy
and are not present in the current Java EE specifications.
The Enterprise JavaBeans (EJBs) specification is a component architecture specification that

defines the structure of beans, the structure of the containers in which they operate, and the
methods for interaction with their clients. Infrastructure services, application server
implementation, and client implementation are defined by the developer and application
server vendor. For more information about EJBs and to view the EJB specification, see
http://docs.oracle.com/javaee/6/tutorial/doc/gijsz.html .
Each of the EJBs has a particular design approach and requirements for its construction.
There are many benefits that result from having a variety of EJB types. The differences in EJB
types allow an application server to optimize performance by being able to make assumptions
about the state and persistence level of the component. The EJBs that have no state
management can have a higher degree of pooling than EJBs that have state. The three levels
of state behavior that a component can assume are:
• Does not maintain state between client requests (stateless session, message-driven
EJBs)
• Maintains state but is not persisted (stateful session EJBs)
• Is persisted (entity data)

Here is an example where the four types of EJBs can be used: Imagine a web application that
represents an eBusiness web site. The user information and preferences could be
represented by an entity EJB (user bean). An object that calculates the taxes based on
passed parameters (price and address) could be a stateless session EJB. The shopping cart
object could be a stateful session EJB. When the user wants to submit an order, he or she
could invoke a servlet, which in turn sends a JMS message (containing the details of the
order). An ordering object (JMS listener) could be a message-driven EJB.

Stateless Session Beans
• Stateless session beans are maintained as a pool in memory.

• If the pool size is three, and if five clients attempt to use one EJB
type, two clients are blocked.
• Stateless EJBs in a pool are identical. So a server can assign
any available EJB to any client.
Client 1
Client 2
Pool of
Client 3 three beans
Client 4
Blocked
Client 5
Application server
Stateless session beans are maintained in memory and provide nonconversational services.
They do not maintain state on behalf of clients. Accordingly, they do not need to be
maintained over server crash.
The EJB container can pool stateless session EJBs that can be used or reused by different
clients. However, no two clients can access the same EJB instance at the same time.
An application server container can pool stateless session EJBs in the server and make the
beans reusable by different clients. Even though multiple clients might request the use of a
stateless session EJB, the container is not required to create an instance for each client. The
container can optimize the number of instances that it maintains to make the server more
scalable.
The EJB specification mandates that a session EJB can be accessed only by one client at
any time. This means that if the container created only one EJB instance and there were 10
clients requesting the use of that instance, only one client would be allowed to have access to
the instance at any time. The other nine clients would have to block and wait for the first client
to finish using the stateless session EJB. So a container may create multiple instances so that
multiple clients can concurrently have their requests handled.
Note that the clients cannot hold on to a bean forever. There is a timeout period (RMI request
timeout) after which the method call fails and the bean becomes free again.

Stateless Session Beans Tuning
The defaults are typically good enough

• <initial-beans-in-free-pool> = 0
• <max-beans-in-free-pool> = 1000
• There is a cost with creating a new instance in the pool
and thus tuning the initial-beans-in-free-pool might be
beneficial if the runtime cost is too much
• This may increase the time to startup but reduces the
runtime cost

Stateless Session Beans Tuning
Tuning based on Pool Miss Ratio

• Pool Miss Ratio = (Pool Total Miss Count / Pool Total
Access Count) * 100
• These values are available as MBeans (console, WLST,
JMX).
• A high pool miss ratio indicates an improperly tuned pool
where instances of specific beans are not found in the pool
at runtime.

Pool Miss Ratio
WLS console:
• Look for Monitoring->EJBs->Stateless
– "Access Total Count" and "Miss Total Count"

Pool Management
Method complete
Free pool
EJBbusy EJBinactivated
Assign from pool
Client
request
<initial-beans-in-free-pool>
EJB does not exist
This graphic shows the WebLogic Server free pool and the processes by which stateless
EJBs enter and leave the pool. “EJB does not exist,” “EJB busy,” and “EJB inactivated”
represent the states of the EJB from the perspective of the WebLogic Server.
On startup, the WebLogic Server automatically creates and populates the free pool with the
quantity of instances that you specify in the bean’s initial-beans-in-free-pool
deployment element in the weblogic-ejb-jar.xml file. By default, initial-beans-in-
free-pool is set to 0.
If you configure a pool, the W ebLogic Server services method calls with an EJB instance from
the free pool, if one is available. The EJB remains active for the duration of the client’s
method call. After the method completes, the EJB instance is returned to the free pool.
Because the WebLogic Server unbinds stateless session beans from clients after each
method call, the actual bean class instance that a client uses may be different from invocation
to invocation. If all instances of an EJB class are active and max-beans-in-free-pool
has been reached, new clients requesting the EJB class will be blocked until an active EJB
completes a method
minutes elapse), call. If the transaction
the WebLogic times
Server throws out (or, for non-transactional
a RemoteException calls,
for a remote if fiveor an
client
EJBException for a local client.
Note: The maximum size of the free pool is limited by the value of the max-beans-in-
free-pool element, available memory, or the number of execute threads.

Determining the Pool Size
• You can improve performance by tuning the number of

bean instances in this free pool.
• Factors to consider to determine how many EJBs should
be in the pool include:
– The number of threads set in the WebLogic Server (WLS)
– The number of concurrent clients
– The number of dependent back-end resources (for example,
database connections)
When you choose values for initial-beans-in-free-pool and max-beans-in-

free-pool, you must weigh memory consumption against slowing down your application. If
the number of stateless session bean instances is too high, the free pool contains inactive
instances that consume memory. If the number is too low, a client may not obtain an instance
when it needs it. This leads to client threads blocking until an instance frees up, slowing down
the application.
Usually, max-beans-in-free-pool should be equal to the number of worker threads in
the server instance, so that when a thread tries to do work, an instance is available.

Configuring a Stateless Session EJB Pool
You can manage the stateless session EJB pool by:

• Capping the number of instances
• Setting an initial pool size
<weblogic-enterprise-bean>
<ejb-name>InsuranceQuoteBean</ejb-name>
<stateless-session-descriptor>
<pool>
<max-beans-in-free-pool>15</max-beans-in-free-pool>
<initial-beans-in-free-pool>5</initial-beans-in-free-
pool>
</pool>
</stateless-session-descriptor>
...
</weblogic-enterprise-bean>
OR
@Stateless(name="InsuranceQuoteBean", maxBeansInFreePool="15",
initialBeansInFreePool="5")
When a client calls a method on a stateless session EJB, the WebLogic Server obtains an
instance from the free pool. The EJB remains active for the duration of the client’s method
call. After the method completes, the EJB instance is returned to the free pool. Because the
WebLogic Server unbinds stateless session beans from clients after each method call, the
actual bean class instance that a client uses might be different from invocation to invocation.
By default, no stateless session EJB instances exists in the WebLogic Server at startup time.
As clients access individual beans, the WebLogic Server initializes new instances of the EJB.
Optionally, you can set theinitial-beans-in-free-poolelement in weblogic-ejb-
jar.xml to automatically create EJB instances in the free pool during startup. This can
improve initial response time when clients access EJBs because initial client requests can be
satisfied by activating the bean from the free pool (rather than initializing the bean and then
activating it). By default, initial-beans-in-free-pool is set to 0.
The maximum size of the free pool is limited by available memory, or the value of themax-
beans-in-free-pool element.
If all instances of an EJB class are active and max-beans-in-free-poolhas been
reached, new clients requesting the EJB class are blocked until an active EJB completes a
method call. If the transaction times out (or, for non-transactional calls, if five minutes
elapses), the WebLogic Server throws a RemoteException.

Message-Driven Beans (MDBs)
Message-driven beans are:

• Asynchronous stateless components
• JMS message consumers
– Clients do not interact directly with MDBs.
Container
Client
sends
JMS destination
Message-driven beans were introduced as part of the EJB 2.0 specification. Message-driven
beans are asynchronous components that decouple (break apart) clients from the beans that
they use.
Synchronous components implement a block-and-wait behavior. For instance, when a client
invokes a method on a stateless session bean, the client application blocks and waits for the
bean to complete execution. The client application cannot continue until the bean has
completed. In an asynchronous model, a client can invoke a bean and then not wait for the
method to complete before continuing to other activities.
JMS clients publish messages to JMS destinations that are hosted on application servers.
Publishing messages is a very lightweight activity for clients. After publishing a message, a
client is not concerned with the routing, direction, or delivery of the message to the receiver.
The JMS server handles the delivery of the message independently of the client’s operations.
Message-driven beans are special EJBs designed to receive messages sent from JMS
clients.
The graphic in the slide shows a client putting messages into a JMS destination. The
messages are consumed by MDBs.
With J2EE 6, EJB methods can be invoked asynchronously, alleviating the need to resort to
MDB just to obtain asynchronous behavior from EJBs.

Configuring an MDB Pool
You can manage the message-driven EJB pool by setting:

• The initial pool size
• The maximum size
• idle-timeout-seconds
<ejb-name>InsuranceQuoteBean</ejb-name>
<message-driven-descriptor>
<pool>
<max-beans-in-free-pool>15</max-beans-in-free-pool>
<initial-beans-in-free-pool>5</initial-beans-in-free-pool>
</pool>
</message-driven-descriptor>
OR
@MessageDriven(name="InsuranceQuoteBean",
maxBeansInFreePool="15", initialBeansInFreePool="5")
You can define the initial and maximum sizes of the message-driven bean pool in the
weblogic-ejb-jar.xmldeployment descriptor.
The idle-timeout-seconds parameter defines the maximum time that an EJB should
remain idle in the free pool before it is removed. After this time has elapsed, WebLogic Server
removes the bean instance from the free pool so long as doing so will not cause the number
of beans in the pool to fall below the number specified in initial-beans-in-free-pool.
Both stateless session beans and message-driven beans can use idle-timeout-seconds
to remove unused bean instances down to initial-beans-in-free-pool. This can help reclaim
memory from inactive sessions for active sessions.

Configuring to Use Batching with an MDB
• Group transactions to reduce writing transaction logs.

• You can configure these parameters in weblogic-ejb-
jar.xml to optimize MDB processing:
– max-messages-in-transaction
trans-timeout-seconds
–
• Each MDB listening on topic uses a dedicated
daemon-polling thread.
• Each MDB listening on queue uses at least one thread
from the dispatch policy.
MDB
batch
process
The WebLogic Server provides a mechanism for grouping onMessage() calls together as a
single transaction. This mechanism can help increase database performance of an EJB
application by grouping all the transactions into a single I/O request. Grouping transactions
requires fewer transaction logs.
You can enable MDB transaction batching by defining the max-messages-in-
transaction element of the weblogic-ejb-jar.xmldeployment descriptor. The max-
messages-in-transactionelement defines the batch size that the WebLogic Server
uses to process onMessage() transactions. However, increasing the batch size can
increase latency. You should start with a small value (for example, 5). You can increase this
value as your application performance allows.
When using MDB batching, more messages are processed per transaction. This may cause
more transactions to time out because more work is being performed in each transaction. You
can increase the transaction timeout by increasing the value of the trans-timeout-
seconds attribute of weblogic-ejb-jar.xml.

MDB Tuning
Maximum MDB concurrency determined by:

• Is MDB using a custom work manager with a max-threads-
constraint?
– Yes
— max-threads-constraint < max-beans-in-free-pool?
— Yes : Max concurrency = max-threads-constraint
— No : Max concurrency = max-beans-in-free-pool
– No
— Is max-beans-in-free-pool > 16?
— Yes: Max concurrency = 16
— No: Max concurrency = max-beans-in-free-pool

MDB Tuning
To handle errors in MDBs, you should:

• For transactional delivery, usesetRollbackOnly() to
force the message back onto the source queue
• For non-transactional delivery, raise a
RuntimeException
acknowledgement. to prevent message
– EJB spec requires destroying the MDB instance
– Use weblogic.ejb.NonDestructiveRuntimeException to
prevent WLS from destroying the MDB instance
• Use Foreign JMS Server for remote destinations
WebLogic Server has the concept of a Foreign JMS Server that allows you to specify the
JNDI information for remote Connection Factories and Destinations and bind it into the local
server’s (or cluster’s) JNDI tree, eliminating the need to put this connectivity/location
information in the MDB itself.

Stateful Session Beans
• Stateful session EJBs:

– Provide conversational interaction
– Store state on behalf of the client
– Are associated with a single client
– Are synchronous
– Are maintained in memory
• Processing time includes passivation and activation.
Stateful session beans are very similar to their stateless session bean counterparts. In fact,
stateful session beans and stateless session beans are implemented in exactly the same
manner. However, stateful session beans are designed to maintain state across multiple client
invocations on behalf of the client. The stateful session bean does this by storing stateful
properties in the attributes of the component itself. An EJB container is then responsible for
ensuring that subsequent client invocations for the same stateful bean are routed back to the
object that hosts the stateful attributes.
Stateful session beans and stateless session beans also have many differences. First,
because stateless session beans do not maintain any state, they are more scalable than their
stateful counterparts. A container does not have to worry about handling excessive memory
consumption or tracking which clients own what beans. Also, because stateful session beans
maintain state, they can consume more memory.
In stateful sessions, the container puts the idle EJB instances to secondary (disk) storage and
releases memory to be used by active EJB instances. This process is called passivation.
When a client calls the passivated bean instance again, the container brings into memory the
previously passivated bean from secondary storage and puts it in memory. This is called
activation. Obviously, frequent activation and passivation can be a performance problem.

Cache Management
You can specify the maximum number of stateful session EJB

instances allowed in memory for each EJB class.
Specified by the
<max-beans-in-cache> tag
Cache
WLS lets you control how many stateful session EJB instances for one EJB type can be in
memory at a time. At this time, the thresholds that you can set are per EJB type. You cannot
set any thresholds that apply for a group of EJB types together. Also, you can control only the
number of EJBs that are allowed in cache—there are no mechanisms to say, “Consider the
cache to be full if these stateful session EJB instances consume 2 MB of memory.”
So, given this level of cache management, you must answer these questions:
• How big should you set your cache?
• What happens to your EJBs when the cache fills up?
The graphic shows that the application server will host stateful session beans up to the
number specified in the max-beans-in-cache setting.

Cache Management
Passivation
EJB activated
EJB passivated
Cache
Activation
Client
request
EJB does not

exist
The WebLogic Server uses a cache of bean instances to improve the performance of stateful
session EJBs. The cache stores active EJB instances in memory so that they are immediately
available for client requests. Active EJBs consist of instances that are currently in use by a
client, as well as instances that were recently in use.
This graphic shows the WebLogic Server cache and the processes by which stateful EJBs
enter and leave the cache. No stateful session EJB instances exist in the WebLogic Server at
startup time. Before a client begins accessing a stateful session bean, it creates a new bean
instance to use during its session with the bean. When the session is over, the instance is
destroyed. While the session is in progress, the instance is cached in memory.
Activation is the transfer of an EJB instance from secondary storage to memory. Passivation
is the transfer of an EJB instance from memory to secondary storage. Tuning max-beans-
in-cache too high consumes memory unnecessarily. The EJB container performs
passivation when the cache becomes full. W hen the EJB session object is needed again, the
bean is activated by the container.

What Happens When the Cache Fills Up?
• WLS has two options to make additional memory available

in the cache:
– Destroy timed-out EJBs.
– Passivate eligible EJBs to the hard drive.
• Passivation is the
the cache and act of
writing removing
it to a bean
a temporary instance from
store.
• A passivated bean is activated when the requesting client
comes back.
• Activation is the act of retrieving the bean instance from
the temporary store to the cache.
When your stateful session EJB cache begins to fill up, the WLS must make more cache
available to those EJBs that need it. The W LS has two options for freeing up the cache for
use by other EJBs: destroying EJBs that are idle and do not need the memory or passivating
eligible EJBs to the hard drive. Either of these actions has the effect of reducing the number
of EJBs currently consuming cache by one and makes that slot available for another EJB
(either one that is currently passivated or a new one that must be created).

Determining the Cache Size
• Tuning this cache can have a significant impact on

performance.
• The criteria that affect the size of a cache include:
– The total amount of Java heap space available in the VM
– The number of different stateful sessions

– The amount of memory that one stateful session bean
consumes while active
– The number of concurrent clients
Determining the appropriate size of your stateful session bean cache requires a little bit of
work and analysis. The size of your cache should be dependent upon the amount of memory
available to be consumed for this type of EJB, as opposed to other EJBs. When all the EJBs
are active simultaneously in the system, their total memory consumption does not exceed the
amount available in the Java heap space.
You must calculate the approximate size of each EJB type, the number of EJB types
deployed in the system, and the total amount of heap space. Calculate a fixed cost of 2 to 3
MB of RAM for administration and overhead. The ideal cache size is one that prevents your
entire system from exceeding the total VM space available.
Consider clustering as well, because bean replication uses space in the cache of all servers
in a cluster environment.
A larger cache can improve performance because it reduces the number of passivation
operations; however, it would use up more memory (heap space). You can control
serialization of data variables using the transient keyword to avoid persistence of
unnecessary data during EJB passivation.

Idle Timeout and Eligibility
• An EJB that exceeds its <idle-timeout-seconds>

value can be destroyed by the WLS.
• An EJB is eligible for passivation if it:
– Has not exceeded its <idle-timeout-seconds>
– Is not currently executing a method

– Is not participating in a transaction
• You can tune the <idle-timeout-seconds> value for
optimal performance.
WLS performs cache sweeps regularly to optimize performance. Ensuring that the amount of
cache taken up by EJBs is at an appropriate level is important for the efficient operation of the
server. If an EJB exceeds its<idle-timeout-seconds>value that is specified in the
META-INF\weblogic-ejb-jar.xmlfile, it can be destroyed by the WLS. An EJB is eligible
for passivation if it has not exceeded its <idle-timeout-seconds>value, it is not
participating in a transaction, and it is not currently executing a method. Thus, only certain
types of EJBs are eligible for passivation at a time.
Question: An EJB is not currently executing a method, but is participating in a transaction.
Why cannot this EJB be passivated?
Answer: In order for a transaction to maintain integrity, a resource that is participating in the
transaction must remain active and not be destroyed until the transaction has committed or
rolled back. Passivating an EJB could destroy a resource (for example, a database
connection); the active transaction would be compromised by passivation.

Cache Type
• The cache type specifies the order in which EJBs are

removed from the cache.
• It includes:
– Not recently used (NRU), known as lazy passivation
– Least recently used (LRU), known as eager passivation

• The idle-timeout-seconds and max-beans-in-
cache elements also affect passivation and removal
behaviors, based on the value of the cache type.
When you configure eager passivation for a stateful session bean by setting the cache type to
LRU, the container:
• Passivates instances to disk:
-As soon as an instance has been inactive for idle-timeout-seconds,
regardless of the value of max-beans-in-cache
- When max-beans-in-cache is reached, even though idle-timeout-
seconds has not expired
• Removes a passivated instance from disk after it has been inactive for idle-timeout-
seconds after passivation. This is referred to as a lazy remove.

When lazy passivation is configured by setting the cache type to NRU, the container avoids
passivating beans because of the associated systems overhead. The container:
• Removes a bean instance from cache whenidle-timeout-seconds expires and
does not passivate it to disk. This is referred to as an eager remove. An eager remove
ensures that an inactive instance does not consume memory or disk resources.
• Passivates instances to disk when max-beans-in-cache is reached, even though
idle-timeout-secondshas not expired. (When the cache type is NRU, reaching
max-beans-in-cache is the only event that causes passivation.)
• Removes a passivated instance from disk after it has been inactive for idle-timeout-
seconds after passivation

Configuring a Stateful Session EJB Cache
You can manage the stateful session EJB cache by:

• Capping the number of instances
• Setting an idle-timeout period
• Setting the passivation strategy for the cached EJBs
<ejb-name>ShoppingCartBean</ejb-name>
<stateful-session-descriptor>
<stateful-session-cache>
<max-beans-in-cache>1000</max-beans-in-cache>
<idle-timeout-seconds>60</idle-timeout-seconds>
</stateful-session-cache>
</stateful-session-descriptor>
OR
@Stateful(name="ShoppingCartBean",
cacheType=Session.CacheType.N_R_U)
@CacheConfig (maxSize=100000, idleTimeoutSeconds=300,
removalTimeoutSeconds=0)
There are no stateful session EJB instances in the WLS at startup. As clients look up and
obtain references to individual beans, the WLS initializes new instances of the EJB class and
stores them in the cache. Ifmax-beans-in-cache is reached and EJBs in the cache are
not being used, the WLS passivates some of those beans even if they have not reached their
idle-timeout-secondslimit.
When an EJB becomes eligible for passivation, it does not mean that the WLS passivates the
bean immediately. Passivation occurs only when the EJB is eligible for passivation and there
is pressure on server resources, or when the WLS performs regular cache maintenance. You
can specify the explicit passivation of stateful EJBs that have reached idle-timeout-
seconds by setting the cache-type element in the weblogic-ejb-jar.xml file. Specify
LRU when you require more aggressive passivation of EJBs. Specify NRU when you have
memory pressures in the cache and want to keep ni memory the beans your client uses the
most.
For cached EJB instances: When resources become scarce and the cache needs memory,
the WLS examines EJB classes that are approaching their max-beans-in-cache limit. Of
those beans, the WLS removes EJB instances that have not been used for idle-timeout-
seconds from the cache (rather than passivating them to disk).

Use Filtering ClassLoaders
Use a Filtering ClassLoader to reduce the number of JAR files

WebLogic will search while trying to load a class or resource:
• Loading class from the current application still has to
search all the way up to the system classloader before
working its way down to the application’s classloader.
• WLS has hundreds of jar files making this a somewhat
expensive operation.
• A Filtering ClassLoader short circuits this by searching the
Application ClassLoader for configured classes.
– It will not search parent class loaders for filtered classes!
– The same concept applies for resources that you know exist
only in the application
• ClassLoading Delegation Model states that parent class loaders are to be searched
before the current class loader for a class.
• If the class being loaded is in an archive in the current application, it still has to search
all the way up to the system class loader and finally work it’s way down to the
application’s class loader. As you can imagine, with the hundreds of archives that now
make up W ebLogic Server, this can be an expensive operation
• The Filtering ClassLoader sits between the System ClassLoader and the Application
ClassLoader and intercepts requests for configured classes. If the classes are not found
in the application, it will not search the parent class loaders (so a
ClassNotFoundExceptionor NoClassDefFoundExceptionwill result).
• The same concept applies for resources that we know exist only in the application.
Prefer-application-packages
• Improve class loading time when you know certain packages are contained in your
application
• Use when you have your own version of some package that also exists in parent class
loader
Prefer-application-resources
• Improve resource lookups when you know certain resources are in your application.

To configure the FilteringClassLoader to specify that a certain package is loaded from an
application, add a prefer-application-packages descriptor element to weblogic-
application.xml which details the list of packages to be loaded from the application. The
following example specifies that org.apache.log4j.* and antlr.* packages are loaded from the
application, not the system classloader:
<prefer-application-packages>
<package-name>org.apache.log4j.*</package-name>
<package-name>antlr.*</package-name>
</prefer-application-packages>
The prefer-application-packages descriptor element can also be defined in weblogic.xml.

General Application Performance Problems
Most application performance issues are typically contention-

related:
• Overuse of synchronization
• Insufficient resources (for example, database connections)
• Overloaded back-end systems:

– Load testing can identify most contention
– Take thread dumps at regular intervals (5-10 secs)
– Attach a profiler, such as HPROF or VisualVM
Use a gradual ramp-up loading so that you can start seeing issues at the progressive
increase of the load. Do not swamp the WebLogic server with a huge traffic in one blow; it will
simplify your analysis.

EJBs: Coding for Performance
Cache JNDI InitialContext, Data Source, JMS Connection

Factory and Destination objects:
• Use the Service Locator pattern to cache lookups.
• Use Dependency Injection.
• Use Coherence to cache frequently used data:

– Can be used explicitly in your application
– Can be used as a JPA Level 2 cache
Cache, Cache, Cache!

InitialContext, EJB Home Interfaces, Datasources, JMS Connection Factories and
Destinations
• Use the Service Locator pattern
The Service Locator pattern centralizes distributed service object lookups, provides a
centralized point of control, and may act as a cache that eliminates redundant lookups. It
also encapsulates any vendor-specific features of the lookup process.
http://www.oracle.com/technetwork/java/servicelocator-137181.html
• Use Coherence
Can be used as a Level 2 cache for JPA by storing entity data in Coherence and thus
allowing applications on multiple servers to share this data.

Section Summary
In this section, you should have learned how to tune:

• Pool size
• Cache size
• Concurrency strategy
• Tune the different types of EJBs
– Singleton Session Beans, Stateless Sessions Beans,
Stateful Session Beans and Message-Driven Beans
• EJB clients

Tuning Stateless EJBs
• Changing the size of initial-beans-in-free-pool
and max-beans-in-free-pool
• Experiencing improved response time of the stateless
session EJBs

Appendix A: Reference Slides
These are the slides that were removed from the srcinal set to convert this course into
a workshop format. Some slides were removed because even if they are still supported
features of WebLogic, they may not have the importance that they used to have. Other
slides were removed because their content is covered directly in the activity guide of
this workshop. Students can use these slides as reference while performing the
practices.
Lesson 2: Performance Tuning Overview
a workshop format.
Oracle WebLogic Server 12c: Performance Tuning Workshop A - 2

Quiz
A benchmark is a performance measurement of a system

meant for use in comparisons.
a. True
b. False
Answer: a

Quiz
The performance of a software system is characterized by its:

a. Response time only
b. Response time and throughput only
c. Response time and resource utilization only
d. Throughput and resource utilization only
e. Response time, throughput, and resource utilization
Answer: e

Quiz
Each benchmark result set should (minimally) include details

about: (Select all that apply.)
a. The application’s version and configuration
b. WebLogic Server license information
c. The version and service pack of WebLogic Server

d. The hardware configuration
e. The operating system configuration
f. WebLogic Server domain information
Answer: a, c, d, e, f

Quiz
Which steps might offer a solution to a problem that has been

identified as being a database access boundary? (Select all
that apply.)
a. Use indexing.
b. Allow more database connections.

c. Fine-tune your database.
d. Have your application connect to the database when it
needs to, as opposed to using the connection pool.
e. Use a fast, dedicated machine.
Answer: a, b, c, e

The Grinder Agent Configuration
• Agents perform the tests.

• Agents are configured using the grinder.properties
file, which has settings for:
– The location of the console
– The test script to run

– The number of processes to start
– The number of threads to start in each process
– The number of times that each thread should run the test
– Think time adjustments (speed up or slow down)
– Output and logging levels
• If the console is not available, agents run immediately.
• Test script files can also be distributed to agents by using
the console.
The Grinder worker and agent processes are controlled by setting properties in the
grinder.properties file. All properties have default values. If you start The Grinder agent
process without a grinder.properties file, the agent communicates with the console by
using default addresses, uses one worker process and one thread, and makes one run
through the test script found in the grinder.py file. The available properties include:
• grinder.processes: The number of worker processes that the agent should start
• grinder.threads: The number of worker threads that each worker process spawns
• grinder.runs: The number of runs of the test script that each thread performs. A
value of 0 means “run forever,” and should be used when you are using the console to
control your test runs.
• grinder.processIncrement: If set, the agent will ramp up the number of worker
processes, starting the number specified every
grinder.processesIncrementInterval milliseconds. The upper limit is set by
grinder.processes.

• grinder.duration: The maximum length of time in milliseconds that each worker
process should run. grinder.duration can be specified in conjunction with
grinder.runs, in which case the worker processes will terminate if either the duration
time or the number of runs is exceeded.

Steps to Use The Grinder
• Set up a test directory.

• Create or record test scripts using a proxy.
– httpscript.py
– httpscript_tests.py
– grinder.properties
• Edit grinder.properties to reflect the load and other
configuration parameters.
• Run The Grinder Console (optional).
• Run Agents and start threads.
• Verify results either on the console or in the log file.

Setting Up a Test Directory for The Grinder
• The test directory:

– Contains the grinder.properties file
– Can be located anywhere on your computer
– Can have a name other than grinder.properties (in
which case, you must indicate the file name when you start
the agent)
• The grinder.properties file contains:
– Configuration information about tests
– Configuration property name/value pairs
The test directory contains a grinder.properties file that maintains information about the
tests to perform. This directory can be located anywhere on your hard drive; it does not have
to be under The Grinder’s directory. The grinder.properties file contains name/value
pairs for various properties.
Refer to The Grinder documentation for a comprehensive list of parameters and default
values.

The Grinder Scripts
• To test an application, create The Grinder test scripts.

• The test scripts, written in the form of Python scripts,
contain the actual steps that you want to test in the
application.
• You can create the scripts by:

– Manually coding the test tasks
– Using the TCPProxy to record the test tasks
TCPProxy
Browser Server
Output
The TCPProxy is a proxy process that you can place in a TCP stream, such as the HTTP
connection between your browser and a server. It filters the request and response streams,
sending the results to the terminal window (stdout). You can control its behavior by
specifying different filters. The TCPProxy’s main purpose is to automatically generate HTTP
test scripts that can be replayed with The Grinder’s HTTP plug-in. Because the TCPProxy lets
you see what is going on at a network level it is also very useful as a debugging tool in its own
right.
The TCPProxy of The Grinder is used by your browser just as any other HTTP proxy server. If
you open http://grinder.sourceforge.net with your browser, it displays the The
Grinder home page, and the TCPProxy outputs all the HTTP interactions between the
browser and the SourceForge site. It is important to remember to remove any “Bypass proxy
server” or “No proxy for” settings that you might have, so that all the traffic flows through the
TCPProxy and can be captured.
Having finished your run-through, click Stop on the TCPProxy console and the generated
script will be written to grinder.py. The grinder.py file contains headers, requests, and a
logical grouping of requests into pages, for the recorded tests. The recorded script can also
be edited manually to suit your needs.

Recording Scripts with TCPProxy
• You can record the scripts using a proxy.

• The Grinder proxy can be started using the following
command:
– java net.grinder.TCPProxy
• Point the browser to the proxy and test your application.
• The proxy records the requests that your browser makes
to the server and generates The Grinder scripts.
• Edit the scripts as required.
TCPProxy
Browser Server
Output
To start the TCPProxy, set the CLASSPATH togrinder.jar, and then invoke java
net.grinder.TCPProxy.
With no additional options, the TCPProxy starts and displays the initialization as an HTTP or
HTTPS proxy with the following parameters:
• Request filters: EchoFilter
• Response filters: EchoFilter
• Local address: localhost:8001
• Engine initialized, listening on port 8001
The example indicates that the TCPProxy is listening as an HTTP proxy on port 8001.
In the browser options dialog box, set the proxy host to be the host on which the TCPProxy is
running and the proxy port to be 8001. After the test scripts are recorded, three new script
files are created: grinder.properties, httpscript.py, and httpscript_tests.py.

Sample grinder.properties File
This snippet shows the following:

• This test is run as one process that starts 500 threads.
• Each thread executes the tests specified in
httpscript.py twice.
grinder.useConsole=false
grinder.processes=1
grinder.threads=500
grinder.runs=2
grinder.logDirectory=JSP_Test_Results
grinder.numberOfOldLogs=0
grinder.script=exercise/grinderscripts/httpscript.py
The Grinder is controlled by setting properties in the grinder.properties file. The sample
file contains the following parameters:
• It does not use the console, so the agent files run to completion on their own.
• There is only one grinder agent process with 500 threads, with each thread executing
the step twice.
• The results are stored in the JSP_Test_Results folder.
• Runs from different versions are overwritten.

httpscript.py File
This file defines the actual tests that would be invoked by the
threads. This is an example of anhttpscript.py file.
from httpscript_tests import *

from net.grinder.script.Grinder import grinder
class TestRunner:
def __call__(self):
tests[0].GET('http://localhost:7011/ShoppingCartApp/')
tests[1].GET('http://localhost:7011/ShoppingCartApp/browse.jsp')
grinder.sleep(2078)
tests[2].POST('http://localhost:7011/ShoppingCartApp/categories',
( NVPair('boxWriting', 'Writing'),
NVPair('btnSubmit', 'Retrieve Items'), ))
This file has three tests:

• Invoke ShoppingCartApp at http://localhost:7011.
• Invoke browse.jsp.
• Invoke Writing and Retrieve Items test categories.

Example of an httpscript_tests.py file:
from HTTPClient import NVPair
from net.grinder.plugin.http import HTTPRequest
from net.grinder.script import Test
tests = {}
headers0 = ( NVPair('Accept', 'image/gif, image/x-xbitmap,
image/jpeg, image/pjpeg, application/x-shockwave-flash,
application/vnd.ms-excel, application/vnd.ms-powerpoint,
application/msword, */*'),
NVPair('Accept-Encoding', 'gzip, deflate'),
NVPair('Accept-Language', 'en-us'),
NVPair('User-Agent', 'Mozilla/4.0 (compatible; MSIE 6.0;
Windows NT 5.1; SV1; .NET CLR 1.1.4322)'), )
request0 = HTTPRequest(headers = headers0)
tests[0] = Test(0, 'GET ').wrap(request0)

Running the Tests
• Start the console on one of the test machines.

– java net.grinder.Console
• For each test machine, start an agent process.
– java net.grinder.Grinder
– The Grinder Console is optional.

• If grinder.useConsole is set to true in the
grinder.properties file, the agent waits for the
console to start the threads.
• If grinder.useConsole is set to false in the
grinder.properties file, the agent
starts the threads.

Test Results
• You can find the results of a load test in the log directory
specified.
– data_host-n.log (individual test invocation data)
– error_host-n.log (errors during the tests)
– out_host-n.log (final statistics summary)
(where host is the machine host name and n is the worker process
number)
• You can view these results either directly in the log file or
on the console.
You can configure The Grinder to retain the log from different runs by setting the
grinder.numberOfOldLogs parameter in the grinder.properties file. When The
Grinder stores the log files, it appends n (the number of running of the test) to the name of the
log.

out_host-n.log
• Scroll down to the very end in out_host-n.log.

• The numbers in the totals row serve as the benchmark for
the test.

The Grinder Console
Start The Grinder and the console from the directory where the grinder.properties file is
located by using the following command:
$ java net.grinder.Console
In the console, select Action > Start processes. If the grinder.receiveConsoleSignals
property is set to true, The Grinder waits until the console starts and you start the processes
from the console.
The console displays the test graphically. It reports the average transactions per second
(TPS), peak TPS, total number of transactions, number of errors, and number of aborts.
Sometimes it takes the console a few moments to display the results.
During a test, you can observe statistics in the console for each test case and the overall
statistics of the tests. The Grinder defines a transaction as an iteration from a client to the
server. It tracks statistics for all requests defined in the grinder.properties file and for
each test. The Grinder tracks the overall TPS, which includes all test cases.
The
for agraph shows the statistics
web application. for all
It tracks the tests, time
average including the tests that
per transaction, access
TPS, the Welcome
average page
TPS, peak
TPS, total number of transactions, the number of errors, and the number of aborts.
Test 1 (POST testdatasource.jsp) shows the statistics for Test 1, which tests a JSP
within a web application.

Creating a View and Adding a Chart to View
Diagnostic information that is collected by the WebLogic Diagnostic Framework is displayed

in a series of views in the Monitoring Dashboard. Only one view is displayed at a time. A view
can contain one or more charts. A chart contains a viewport, legend, labels, and controls for
identifying and displaying metrics. A metric is a runtime MBean property with a numeric or
Boolean value that is useful to measure, either as its current value or as its change over time.
A chart can display metrics from one or more MBean instances from one or more servers in
the domain.
You can create views by using the Monitoring Dashboards. While the built-in views are very
useful, you may want to create your own views suitable to your environment.
To create a view, select the New View icon in the View List tab of Monitoring Dashboard.
To create a chart for a custom view, you can do one of the following:
• Select the view menu and choose New Chart.
• Drag a metric from the Metric Browser, or from an existing chart, to an empty location in
the view.
After the chart is created, select the chart menu, select Chart Type, and choose the type of
chart you want to have for the metrics to display in it. You can add new metric data to a chart
or you can move or copy a metric from one chart to another.

Quiz
What does the grinder.properties file contain? (Select all

that apply.)
a. Configuration information about tests
b. The Grinder CLASSPATH
c. Configuration property name/value pairs
Answer: a, c

Lesson 3: Tuning Operating System Resources
a workshop format.

Other Useful Commands
• iptraf
– Monitors TCP/IP traffic in a real time manner and generates
real time reports. It shows TCP/IP traffic statistics by each
session, by interface, and by protocol.
• acctcom
– Displays selected process accounting record summaries.
This command reports only on processes that have finished.
• topas
– Reports selected statistics about the activity on the local
system
• gprof
– Reports the flow of control among the subroutines of a
program and the amount of CPU time consumed by each
subroutine.

Lesson 4: Tuning HotSpot JVM
a workshop format.

Garbage Collectors: Comparisons
The following table summarizes the trade-offs between the

garbage collectors:
GC F eature Serial ParallelGC CMSGC G1GC

GC
Parallelism No Yes Yes Yes
Concurrency No No Yes Yes
YoungGCs Serial Parallel Parallel Parallel
OldGCs Serial Parallel Paralleland Conc Paralleland Conc
Parallelism: GC is single threaded or multithreaded.

Concurrency: GC is stop the world or runs simultaneously with application threads.
Young GCs: GC on young generation is single-threaded or multithreaded.
Old GCs: GC on old generation is single-threaded or multithreaded, and is stop the
world or runs simultaneously with application threads.

Throughput Goal
• By setting the throughput goal, you can specify how much total
time the garbage collector can spend for GC.
• Specified with the command-line flag: -XX:GCTimeRatio=nnn
java –Xms1g –Xmx2g –XX:GCTimeRatio=24 ... weblogic.Server
startManagedWebLogic.sh
• The ratio of garbage collection time to application time is

1 : nnn. (1 time for gc and nnn time for application)
• The throughput goal is measured in terms of the time spent while
garbage collecting the heap and the time spent outside of
garbage collection.
Setting a throughput goal is best for tuning WebLogic

Server performance! Set -Xms and -Xmx the same instead!
The throughput collector is a generational collector, so there are separate collections for the
young generation and the old generation. Averages and variances are kept separately for
each generation. The maximum pause time goal is applied to the average plus the variance of
the collections of each generation separately. Each generation may separately fail to meet the
pause time goal.
The throughput goal is measured in terms of the time spent collecting garbage and the time
spent outside of garbage collection (referred to as application time). The goal is specified by
the command-line flag:
-XX:GCTimeRatio=<nnn>
The ratio of garbage collection time to application time is 1 : nnn.
For example, if you set -XX:GCTimeRatio=24, this will set a goal of 1/(1+24), 1/25th, or 4%,
which means 4% of the total time is the goal for garbage collection, and the rest will be
application time.
The time spent in garbage collection is the total time for both young generation and old
generation collections combined. If the throughput goal is not being met, the sizes of the
generations are increased in an effort to increase the time the application can run between
collections.

Maximum Pause Time Goal
• Pause time is the time taken by the garbage collector to

recover heap space that is no longer in use by stopping
the application.
• Setting this goal will limit the longest of such application
pauses.
• You specify it with the command-line flag:
-XX:MaxGCPauseMillis=nnn
java –Xms2g –Xmx2g –XX:MaxGCPauseMillis=500 ... weblogic.Server
• When this goal is specified, the garbage collector adjusts

the heap size and various other parameters to keep the
pauses shorter than the specified time.
The throughput collector is a generational collector, so there are separate collections for the
young generation and the old generation. Averages and variances are kept separately for
each generation. The maximum pause time goal is applied to the average plus the variance of
the collections of each generation separately. Each generation may separately fail to meet the
pause time goal.

Footprint Goal
The footprint goal is applicable only after achieving the set

throughput and maximum pause time goals.
Yes Reduce
GC Pause
Generation Size
> Goal? by 5%
No
Increase
Yes
Throughput Generation
< Goal? Sizes by a Total
of 20%
No
The footprint goal is a

Heap Size Set primary goal for tuning
WebLogic Server performance!
The implementation checks (in this order):

• If the GC pause time is greater than the pause time goal, reduce the generation sizes to
better attain the goal.
• If the pause time goal is being met, consider the application’s throughput goal.
• If the application’s throughput goal is not being met, increase the sizes of the
generations to better attain the goal.
• If both the pause time goal and the throughput goal are being met, the size of the
generations is decreased to reduce footprint.
Dynamic generation sizing: 20% for increases and 5% for decreases by default.
Only one generation is reduced at a time. If two generations are not meeting their pause time
goals, the one with the larger pause time is reduced first.
If the throughput goal is not being met, the sizes of both generations are increased in
proportion to its respective contribution to the total collection time.
All of this notwithstanding, the footprint goal is not generally a performance goal for running
WebLogic Server. Instead, you should avoid dynamic heap resizing by specifically setting the
minimum and maximum heap sizes to the same value.

Monitoring GC: -XX:+PrintGC
• You can enable garbage collection statistics gathering in

the HotSpot VM by turning on the -XX:+PrintGC
command-line flag.
java –Xms2g –Xmx2g –XX:+PrintGC ... weblogic.Server
• In J2SE 5.0 and later, you can set this flag dynamically by
changing the verbose attribute of
java.lang.management.MemoryMXBean.
• You can print information for every collection by using the
-verbose:gc flag.
Redirect both the standard error and standard output to a log file to analyze -verbose:gc
output:
For example, on Windows and Solaris:
> java -Xms200m -Xmx512m -XX:+PrintGC ... weblogic.Server >>
logfile.txt 2>&1
The -XX:+PrintGCDetails flag prints additional information about the collections.

Monitoring GC: –XX:+PrintGC
A sample entry of a garbage collection log in the screenshot shows:

[GC 239680K->109448K(269824K), 0.1218750 secs]
• GC: Indicates that it was a minor collection (young generation). If it had said Full GC, it
was a major collection (tenured generation).
• 239680K: The combined size of live objects before garbage collection
• 109448K: The combined size of live objects after garbage collection
• (269824K): The total available space, not counting the space in the permanent
generation, which is the total heap minus one of the survivor spaces
• 0.1218750 secs: The time it took for garbage collection to occur
You can get more detailed output using -XX:+PrintGCDetails and
-XX:+PrintGCTimeStamps.

GC Manually Using the Administration Console
When required, you can manually request a full garbage

collection by:
• Using the WebLogic Administration Console
• Using Java VisualVM
• Calling the gc() method on java.lang.System

[System.gc()]
• Calling the gc() method on
java.lang.management.MemoryMXBean
If you want to disable calls to request GC manually, set -XX:+DisableExplicitGC on the

command line while starting the W ebLogic Server. This disables calls to System.gc() and
the JVM performs garbage collection when necessary.
The screenshot shows how garbage collection can be forced from the Administration
Console.

Ergonomics Behavior
• To achieve the throughput goal defined by the command-

line flag, the garbage collector automatically resizes the
heap.
• To achieve the throughput goal, larger heaps are used.
• To achieve
footprint, the maximum
smaller heaps arepause
used.time goal and a minimum
• The implementation of -XX:+UseAdaptiveSizePolicy
is used by default with the Parallel Collector.
• Multiple goals, such as –XX:GCTimeRatio and
-XX:MaxGCPauseMillis, can be defined in the
command-line while starting the JVM.
The heap size parameters selected by ergonomics and the features of the adaptive size
policy are meant to provide good performance for server applications.
You can define multiple goals for JVM. When multiple goals are defined, the priority of the
goals is in the following order:
• Maximum pause time goal
• Throughput goal
• Minimum footprint goal

Ergonomics: Command-Line Flags
• Use -XX:+PrintCommandLineFlags to indicate what

Ergonomics is choosing.
• Example:
– On 2CPU Intel Oracle Linux x64 with 8 GB RAM:
Starting WLS with line:

/u01/app/jdk/bin/java -server -Xms256m -Xmx512m
-XX:MaxPermSize=256m ...
-Dweblogic.ProductionModeEnabled=true ...
-XX:+PrintCommandLineFlags ... weblogic.Server
-XX:-BytecodeVerificationLocal -XX:-BytecodeVerificationRemote
-XX:InitialHeapSize=268435456 -XX:MaxHeapSize=536870912
-XX:MaxPermSize=268435456 -XX:+PrintCommandLineFlags
-XX:+UseCompressedOops -XX:+UseParallelGC
Server Console Window

Why Use jinfo?
You can use jinfo to find the following information about a

running VM either locally or remotely:
• Version of the VM running
• Compiler options being currently used
• Classpath currently being used

• Locale information
• Command-line options set during the VM startup
In most cases, if you have GUI access to the machines running JVMs, you would more likely
use Java VisualVM. Java VisualVM usesjinfo for getting the details of JVM and presents
them within its GUI interface. However, when you are limited to access through command-line
options only, then jinfo is very useful.

Application Snapshot
The application snapshot enables you to capture and archive the status of JVM at a point in
time. The target JVM does not need to be running to view the snapshot. Snapshots can also
be sent and viewed by other people.
• An application snapshot also captures general information about the JVM such as heap
dumps, thread dumps, and profiler snapshots at the moment the snapshot is taken.
• You take an application snapshot by right-clicking an application node in the
Applications window and choosing Application Snapshot from the pop-up menu.
• Application snapshots are persisted between Java VisualVM sessions.
• When you take an application snapshot:
- An application snapshot node appears under the Snapshots node in the
Applications window
- Application data is saved in a subfolder in the repository or snapshots folder in the
Java VisualVM user directory
-
Application snapshots are listed under the Snapshots node in the Applications
window. Expand the application snapshot node to view the contents of the
snapshot.
• After you take an application snapshot, you can view and modify the contents. You can
then save the application snapshot as a single application snapshot archive (.apps) to
your local system.
Lesson 5: Monitor and Tune Server Performance
a workshop format.

Domain Startup Mode
• Startup mode is applicable for the entire domain.

• Generally, production mode is suitable for high
performance, whereas development mode is more flexible.
M o de Applicability
Development • You are developing your applications.
• You would like to deploy or redeploy applications
automatically.
• You do not have SSL certificates. (You can use the
demonstration digital certificates.)
Production • You are deploying or testing your applications for
production use.
• You want more control over the deployment and
configuration process.
• You need to ensure that proper SSL certificates have
been installed.
You can set up servers in a domain to run either in development mode or in production mode.
In general, production mode requires you to configure additional security features. You can
set up a domain in development mode and then easily move to production mode. After you
have changed to production mode, you cannot change back to development mode without
restarting the server.
To configure all servers in a domain to run in production mode:
1. If you have not already done so, in the Change Center of the Administration Console,
click Lock & Edit.
2. In the left pane of the Console, under Domain Structure, select the domain name.
3. Select Configuration > General and select the Production Mode check box.
4. Click Save, and then, to activate these changes, in the Change Center, click Activate
Changes.
5. Shut down any servers that are currently running.
6. Invoke the domain’s startWebLogic script. The Administration Server starts in the
new mode.
7. If the domain contains Managed Servers, start the Managed Servers. The Managed
Servers also start up in the defined mode.

WebLogic Server Native IO
• WebLogic Server Native IO uses platform-optimized

socket multiplexor to improve server performance.
• You can enable or disable native I/O using the
Administration Console.
The use of native IO is enabled by default. Native IO is platform dependent.

The screenshot shows how to enable native IO. The native IO can provide performance
benefits where there is contention of IO operations with other (non-WLS) applications on the
system.

Quiz
Which statement is true?

a. Performance of the server can degrade significantly if
many client request threads get stuck.
b. Server performance will not be affected if many client
request threads get stuck. The JVM will allocate more to

compensate.
Answer: a

Stuck Threads Work Manager
• You should normally:

– Define a work manager for managing stuck threads
– Use the stuck threads work manager rather than configuring
stuck threads parameters for individual servers
• Using stuck threads work manager enables you to have
flexibility and easily address problems at the:
– Server level
– Application Level
• You can make the stuck thread work manager:
– Stop a work manager that is running the stuck threads
– Move the application to ADMIN mode
– Mark the server instance as failed

Work Manager for Stuck Threads
• By creating work managers for stuck threads, you can

handle stuck threads:
– At the global level for all applications running on the WLS
server
– At the application level for specific application
• In the application deployment descriptor file (or the
deployment plan) you configure reference to the work
manager that handles stuck thread.
An example of an applications configuration file showing work manager for stuck threads:
• Configured in weblogic-ejb-jar.xml file:
...
<work-manager>
<name>stuckthread_workmanager</name>
<work-manager-shutdown-trigger>
<max-stuck-thread-time>30</max-stuck-thread-time>
<stuck-thread-count>2</stuck-thread-count>
</work-manager-shutdown-trigger>
</work-manager>
...

Work Manager Tuning – I
• Create Work Managers to avoid thread starvation, only as

needed.
– Fair Share work manager is typically what you want.
• Monitor:
– Pending Requests
– Max Wait Time
– Current Wait Time

Work Manager Tuning – II

Work Manager Constraint Tuning – I
• Max Threads Constraint:

– Limit the number of concurrent threads processing requests
for the Work Manager.
– Use to configure MDB concurrency
— Default is 16
• Capacity Constraint:
– Limits the number of total requests (currently processing and
queued) to avoid resource issues. When the capacity is
reached HTTP requests get an HTTP 503 response
– RMI clients get a QueueThrottleException response

Work Manager Constraint Tuning – II
• Min Threads Constraint:

– Guarantees thread availability to process requests, thus
avoiding thread starvation.
– Use only if absolutely required
– Keep value very low (e.g., 3 ~ 4)
• Min Threads Constraint work is immediately scheduled if
constraint not yet met
– Setting too high affects work for other work managers
Setting the min thread constraint will essentially cause work for that work manager to
immediately be scheduled as long as there aren’t the specified number of concurrent
requests already processing. That is, if the min thread constraint value is set to 5, WLS
will always schedule work for that work manager immediately until there is 5 concurrent
requests processing—bypassing all other requests sitting in the execute queue!

Chunk Size
• Chunks are temporary buffers used for I/O.

• WebLogic JVMs divide network data into chunks.
• The default chunk size is 4,080 bytes (4 KB).
• You may need to tune the chunk size for network-intensive
applications.
• Tuning the chunk size also reduces the number of socket
reads or writes.
• The command-line property to set the chunk size:
-Dweblogic.Chunksize=n (n = chunk size)
A chunk is a unit of memory that the WebLogic Server network layer, both on the client and
the server side, uses to read data from and write data to sockets. A server instance maintains
a pool of these chunks. For applications that handle large amounts of data per request,
increasing the value on both the client and the server side can boost performance.

Tuning the Chunk Size
• Set a smaller chunk size for high-frequency small-payload

workloads, and a higher value for high-payload workloads.
• Set the chunk size on both the client and the server JVM
for better performance.
• Tune the chunk

Transmission size
Unit to match
(MTU) size the
andnetwork Maximum
operating system
memory page size for better performance.
• Ensure that the chunk size is a multiple of 8.
• Ensure that the chunk size is a multiple of the MTU size.
A chunk is a unit of memory that the WebLogic Server network layer uses to read data from
and write data to sockets. To reduce memory allocation costs, a server instance maintains a
pool of these chunks. For applications that handle large amounts of data per request,
increasing the value on both the client and server sides can boost performance. The default
chunk size is about 4 KB.
Use the following properties to tune the chunk size and the chunk pool size:
• weblogic.Chunksize: Sets the size of a chunk (in bytes). The primary situation in
which this may need to be increased is if request sizes are large. It should be set to
values that are multiples of the network’s maximum transfer unit (MTU), after subtracting
from the value any Ethernet or TCP header sizes. Set this parameter to the same value
on the client and server.
• weblogic.utils.io.chunkpoolsize: Sets the maximum size of the chunk pool.
The default value is 2048. The value may need to be increased if the server starts to
allocate and discard chunks in steady state. To determine whether the value needs to
be increased, monitor the CPU profile or use a memory or heap profiler for call stacks
invoking the constructor weblogic.utils.io.Chunk.
• weblogic.PartitionSize: Sets the number of pool partitions used (default is 4).
The chunk pool can be a source of significant lock contention as each request to access
to the pool must be synchronized. Partitioning the thread pool spreads the load over
multiple partitions and reduces potential for contention over one partition.
Tuning the Chunk Pool Size
• The WebLogic Server caches chunks in an internal chunk

pool for reuse.
• Chunks are pooled to prevent allocation cost.
• The command-line property to set the chunk size is
-Dweblogic.utils.io.chunkpoolsize=n .
• The default chunk pool size is 512.
• Tuning the pool size accommodates workloads with
particular payloads and concurrent client activity.
• You should set a large pool size to accommodate
workloads with many concurrent clients.
Check the creation of chunk class and try to minimize this by setting chunk pool size.

Quiz
Which statements are true? (Select all that apply.)

a. Chunks are temporary buffers used for I/O.
b. WebLogic JVMs divide network data into chunks.
c. The default chunk size is 1,024 bytes (1 KB).
d. You should tune the chunk size for network-intensive
applications.
e. Tuning the chunk size also reduces the number of socket
reads/writes.
f. The command-line property to set the chunk size is:
-Dweblogic.Chunksize=n (n = chunk size)
Answer: a, b, d, e, f

Quiz
If the Accept Backlog value is set too low, most connections will
be dropped or refused at the client.
a. True
b. False
Answer: a

Quiz

a. You will never run low on memory; the hardware has
become so cheap that there is always lots to spare.
b. Low memory can significantly impact the performance of
the server.
c. The WLS server health-checking system will always block
incoming requests to ensure that the server never has to
deallocate an EJB due to lack of memory.
Answer: b

Quiz
Using hardware accelerators to offload SSL processing from

the application server to the dedicated SSL accelerator will
accelerate performance.
Which statement(s) is (are) true?
a. This statement
a dedicated SSLis accelerator.
false. WLS can process SSL faster than
b. The statement is false. There is no such thing as a
dedicated SSL accelerator.
c. This statement is true.
Answer: c

Lesson 6: Monitor and Tune Cluster Performance
a workshop format.

Quiz
Which of the following statements are true? (Select all that

apply.)
a. Remote calls can be expensive because they involve
sending data over the network.
b. Remote calls
the SOAP are expensive because they always involve
stack.
c. Remote calls can be expensive because they involve
serialization and deserialization to send the data.
d. Remote calls are not expensive.
e. None of the above
Answer: a, c

Quiz
Which of the following statements are true? (Select all that

apply.)
a. In a basic cluster, servlets or JSPs in the presentation tier
typically access objects in the object tier, such as EJBs or
JDBC objects.
b. In a basic cluster, load balancing can be taken advantage
of only between the web tier and the object tier.
c. In a basic cluster, both the presentation and object tiers
are on the same cluster.
Answer: a, c

Load Balancers
• Load balancing distributes the processing and

communications activity across a network of servers so
that no single server is overwhelmed.
• In general, there are two types of load balancers:
– Hardware load balancers
– Software load balancers
• The choice of load balancers can have a significant impact
on performance.
In general, hardware load balancers perform better than software load balancers; however,
hardware load balancers can be very expensive when compared to their software
counterparts. Performance can far outweigh the cost depending on the criticality of the
application.
A few examples of load balancers are:
• Hardware load balancer: Cisco, Foundry Networks, and Big IP
• Software load balancer: Oracle HTTP Server, Apache, Netscape, IIS, and WebLogic
Server (via proxy plug-ins)

Quiz
When considering a load balancer, which of the following

should be well thought out? (Select all that apply.)
a. Support for a compatible passive or active cookie
persistence mechanism and SSL persistence
b. Pages-served performance
c. Compilation rate of JSPs
d. Flow of connections to the servers
e. Management and configuration tools
f. Security
g. Load-balancing algorithms
Answer: a, b, d, e, f, g

Oracle Coherence: Overview
Coherence:
• Provides a distributed, in-memory caching solution for
Java
• Is based on a grid of cache servers or nodes
• Automatically distributes or partitions cached data across

the grid based on the number of servers
• Implements replication for high availability
• Includes a decentralized management and monitoring
model based on JMX
• Supports a proxy architecture to provide connectivity
beyond the Coherence cluster
• Supports additional languages such as .NET and C++
One of the primary uses of Oracle Coherence is to cluster an application’s objects and data.
In the simplest sense, this means that all the objects and data that an application delegates to
Coherence are automatically available to and accessible by all servers in the application
cluster. None of the objects or data will be lost in the event of server failure. By clustering the
application’s objects and data, Coherence solves many of the difficult problems related to
achieving availability, reliability, scalability, performance, serviceability, and manageability of
clustered applications.
Oracle Coherence is a JCache-compliant, in-memory caching and data management solution
for clustered Java EE applications and application servers. Coherence makes sharing and
managing data in a cluster as simple as on a single server. It accomplishes this by
coordinating updates to the cached data by using clusterwide concurrency control, replicating
and distributing data modifications across the cluster, and delivering notifications of data
modifications to any servers that request them. Developers can easily take advantage of
Coherence features by using the standard Java Collections API to access and modify data,
and by using the standard JavaBean event model to receive data change notifications.
Coherence provides a clusterwide view of management information through the standard JMX
API, so that the entire cluster can be managed from a single server. The information provided
includes cache sizes along with hit and miss rates.

Coherence*Web: Overview
• Is a plug-in that enhances WebLogic Server’s session

management implementation with a Coherence grid
• Provides a highly scalable session replication solution that
can span WebLogic domains and clusters
• Reduces WebLogic Server’s memory footprint

• Requires no changes to existing application code
• Allows session data to be shared across applications
• Can be enabled on an application basis
Coherence*Web is an application server plug-in dedicated to managing session state in

clustered environments. It brings the scalability, availability, reliability, and performance
characteristics of a Coherence data grid to in-memory session management and storage.
Also, because it is not constrained by the deployment topologies of the application server, it
enables session sharing and management across different web applications, domains, and
even different application server products. Sometimes you may want to explicitly prevent
session data from being shared by different Java EE applications that participate in the same
Coherence cluster, so Coherence*Web supports this approach as well.
If sessions are shared across web applications, you may want to scope individual session
attributes so that they are either globally visible (that is, all web applications can see and
modify these attributes) or scoped to an individual web application (that is, not visible to any
instance of another application). The latter approach may be desired to help avoid
namespace collisions when multiple applications use the same session attribute names.
With Coherence*Web, session data is stored outside of the application server, thereby freeing
server heap space. This architecture also allows you to individually tune and scale the sizes
of your application server clusters and session data grids.

Replication Configuration
• Session persistence is configured using the

<session-descriptor> element in the weblogic.xml
deployment descriptor file.
• Snippet from WEB-INF/weblogic.xml:
<session-descriptor>
<persistent-store-type>
replicated
</persistent-store-type>
</session-descriptor>
In-memory replication:
<persistent-store-type>replicated</persistent-store-type>
File persistence:
<persistent-store-type>file</persistent-store-type>
<persistent-store-dir>Directory Path</persistent-store-dir>
JDBC persistence:
<persistent-store-type>jdbc</persistent-store-type>
<persistent-store-pool>Data Source Name</persistent-store- pool>

Coherence*Web Persistence:
<persistent-store-type>coherence-web</persistent-store-type>

Quiz
Which of the following statements is true?

a. With in-memory replication, the httpSession state is
replicated to every other server in the cluster.
b. With in-memory replication, the httpSession state is
replicated only to other WLS servers running on the same

physical hardware.
c. In-memory replication is deprecated in version 9 and later
of the product.
d. There is no such thing as in-memory replication.
e. None of the above
Answer: e

Lesson 7: Tuning JDBC
a workshop format.

JDBC Drivers
• JDBC drivers are implementation classes for database

operations.
• Drivers fall into two categories:
– Two-tier: Clients directly access the database.
– Three-tier: Clients
and the middle tiermake a request
connects to the to the middle
database tier (Wof
on behalf LS)the
client.
JDBC
WLS Database
driver
Java
application
JDBC
Database
driver
JDBC drivers are simply collections of classes that implement known methods defined by the
JDBC driver specification. These drivers fall into two broad classes: two-tier and three-tier.
Two-tier drivers are used by a client regardless of who that client is, to access a database
directly. In a three-tier driver, the WebLogic Server (WLS) acts as a middleman, optimizing
connections to the underlying database, typically via a connection pool. A connection pool is a
collection of connections to a database. Connections in a connection pool are created at boot
time by the W LS. The connections of a connection pool are configured to use a particular
JDBC driver and connect to a specific database.
The graphic shows that applications can directly connect to a database by using JDBC, and
that WLS can also connect to databases by using JDBC.

JDBC Architecture
Java application
JDBC API
JDBC-ODBC JDBC-Native JDBC-Net All Java
Bridge Bridge Bridge JDBC Driver
(Type 1) (Type 2) (Type 3) (Type 4)
ODBC Native API

Driver (C, C++)
Network
Server
(WLS)
RDBMS
Applications that use JDBC can use one of four different JDBC driver types. The first type of
driver is the JDBC-ODBC bridge. The Java program accesses a database through a client-
configured Open Database Connectivity (ODBC) connection. A client application can also use
a JDBC driver that accesses the client-side-installed native API driver for a particular
database. A Type 2 driver provides a bridge from the Java JDBC API into the native driver
installed on the client. The native driver has the protocols and knowledge to communicate
directly with the database. The Type 3 driver, or a multitier implementation, uses a network
server to relay or proxy the JDBC requests onto the database. The Type 4 driver is an all-
Java implementation of the JDBC native database driver. The database and the JDBC
application perform direct Java-to-Java calls using this driver type. As a distinction, the JDBC
drivers that allow a client machine to communicate directly with the database are called two-
tier implementations, whereas the proxy of the Type 3 driver is called a multitier
implementation.
The graphic shows four types of JDBC drivers. It shows that Type 3 driver uses a Network
Server.

Data Sources
Data sources:
• Are administered factory objects that provide JDBC
connections
• Are bound into the Java Naming and Directory Interface
(JNDI) and configured using the Administration Console

• Make the application code portable across databases
Application server
JNDI Connection pool
1 lookup
Data source Connection RDBMS
Java 2
application getConnection Database
3 Use like “normal” JDBC connection
A data source is a J2EE concept that allows for abstracting a JDBC connection. With a data
source, a developer or administrator can specify an object that represents a database. In the
case of the WebLogic Server, a data source is a wrapper around a connection pool. A user
can then access the data source and use it to create a connection to the underlying database
via the connection pool associated with the data source. The data source concept allows the
developer to be completely unaware of the connection pool or database that is used to
provide the connection. Although the slide depicts only “normal” data sources, the WebLogic
Server supports both transactional and non-transactional data sources. Transactional data
sources have the added advantage that any actions they participate in can be part of a
transaction.
The graphic shows that the Java application uses JNDI to obtain a reference to a data source.
It does this using a lookup(). The Java application then gets a connection from the data
source by using the getConnection() call. The Java application is then able to make calls
on to the database by using that connection.

Quiz

a) You can cache JDBC Statement objects, but you cannot
batch PreparedStatement objects
b) You can cache JDBC PreparedStatement objects, but you
cannot batch Statement objects

c) You cannot cache JDBC Statement objects, but you can
batch PreparedStatement objects
Answer: c

Quiz
Which statements are transaction isolation levels? (Select all

that apply.)
a. T RANSACTION_READ_UNCOMMITTED
b. TRANSACTION_READ_COMMITTED
c. TRANSACTION_REPEATABLE_READ
d. TRANSACTION_SERIALIZABLE
e. TRANSACTION_READ_SERIALIZABLE
f. T RANSACTION_REPEATABLE_UNCOMMITTED
Answer: a, b, c, d

Logging Last Resource T ransactions
• Logging Last Resource is a WebLogic optimization that

allows one non-XA resource to participate in a distributed
transaction:
– Delays the non-XA data source until after all other XA work is
prepared
– Commits the XA transaction based on the outcome of the
non-XA data source
– Tends to yield performance gains for insert, update, and
delete operations, but not read operations
• Additional tables are required for each server to track
Logging Last Resource transactions
(WL_LLR_servername).
The WebLogic Server includes support for the Logging Last Resource (LLR) transaction
optimization through JDBC data sources. Logging Last Resource is a performance
enhancement option that enables one non-XA resource to participate in a global transaction
with the same atomicity, consistency, isolation, durability (ACID) guarantee as XA. The
Logging Last Resource uses a local transaction for its transaction work. The WebLogic Server
transaction manager prepares all other resources in the transaction and then determines the
commit decision for the global transaction based on the outcome of the Logging Last
Resource’s local transaction.
In many cases, a global transaction becomes a two-phase commit (2PC) transaction because
it involves a database operation (using JDBC) and another nondatabase operation, such as a
message-queuing operation (using JMS). In cases such as this where there is one database
participant in a 2PC transaction, the Logging Last Resource Optimization transaction option
can significantly improve transaction performance by eliminating some of the XA overhead for
database processing and by avoiding the use of JDBC XA drivers, which typically are less
efficient than non-XA

When a Logging Lastdrivers.
Resource transaction is committed, the WebLogic Server transaction
manager handles the processing transparently. From an application perspective, the
transaction semantics remain the same, but from an internal perspective, the transaction is
handled differently from standard XA transactions.

Logging Last Resource: Example
5 Commit XA resources
Prepare XA
resources
2
• Start Tx XA Data
• Use data sources Source
1 • Post message
4 Commit non-XA work
• Commit Tx
WLS
Non-XA Data
Application Transaction
Source
(Tx) Manager
3 4
JMS X A
Record Tx for Connection
high availability
2 5
Each WebLogic Server instance maintains a database Logging Last Resource table on the
database to which a JDBC Logging Last Resource data source pools database connections.
These tables are used for storing transaction log records and are automatically created. If
multiple Logging Last Resource data sources are deployed on the same WebLogic Server
instance and connect to the same database instance and database schema, they will also
share the same Logging Last Resource table. The Logging Last Resource table names are
automatically generated unless administrators choose to configure them. The default table
name is WL_LLR_SERVERNAME.
In a global two-phase commit (2PC) transaction with a Logging Last Resource participant, the
WebLogic Server transaction manager follows these basic steps:
1. Receives a commit request from the application
2. Calls a prepare on all other (XA-compliant) transaction participants
3. Inserts a commit record to a table on the Logging Last Resource participant (rather than
to the file-based transaction log)
4. Commits the Logging Last Resource participant’s local transaction (which includes both
the transaction commit record insert and the application’s SQL work)
5. Calls a commit on all other transaction participants
6. After the transaction completes successfully, lazily deletes the datab ase transaction log
entry as part of a future transaction
Configuring the Logging Last Resource
The Logging Last Resource

optimization improves performance by:
• Removing the need for a
distributed driver to connect to
the database
• Reducing the number of
processing steps to complete
the transaction
• Removing the need for
distributed processing at the
database
Options available
only to non-XA
data sources
The Logging Last Resource optimization provides a significant increase in performance for
insert, update, and delete operations. However, for read operations with Logging Last
Resource, performance is somewhat slower than read operations with XA. For best
performance, you may want to configure a non-Logging Last Resource JDBC data source for
read-only operations.
The server will not boot if a Logging Last Resource table is unreachable during boot. Logging
Last Resource transaction records must be available to correctly resolve in-doubt transactions
during recovery, which runs automatically at server startup. If a transaction’s coordinating
server crashes before a Logging Last Resource stores its transaction log record or before a
Logging Last Resource commits, the transaction rolls back. If the server crashes after the
Logging Last Resource is committed, the transaction will eventually fully commit. During
server boot, the transaction coordinator will use the Logging Last Resource to read the
transaction log record from the database and then use the recovered information to commit
any unfinished work on any participating non-Logging Last Resource XA resources.

Lesson 8: Configuring Work Managers
a workshop format.

Stuck Thread Handling
• Threads that are running for a long time are likely to be

deadlocked or in an infinite loop, or indicate an overloaded
server.
• WebLogic Server:
– Periodically checks how long threads have been running
– Logs a warning if a thread becomes stuck
– Creates additional threads to handle new requests
– Sets the server’s state to Failed after a specified number of
threads become stuck
– Can automatically shut itself down if in the Failed state
– NEVER KILLS A THREAD
If an application server continues to accept requests after system capacity is reached,

application performance and stability can deteriorate. WebLogic Server diagnoses a thread as
stuck if it is continually working (not idle) for a set period of time. You can tune a server’s
thread detection behavior by changing the length of time before a thread is diagnosed as
stuck, and by changing the frequency with which the server checks for stuck threads.
If all application threads are stuck, a server instance marks itself failed and, if configured to do
so, exits. You can configure Node Manager, or a third-party, high-availability solution to restart
the server instance for automatic failure recovery. You can configure these actions to occur
when not all threads are stuck, but when the number of stuck threads have exceeded a
configured threshold:
• Shut down the work manager if it has stuck threads. A work manager that is shut down
will refuse new work and reject existing work in the queue by sending a rejection
message. In a cluster, clustered clients will fail over to another cluster member.
• Shut down the application if there are stuck threads in the application. The application is
shut down by bringing it into administration mode. All work managers belonging to the
application are shut down, and behave as described above.

Configuring Stuck Thread Handling
Stuck thread
after 10 minutes
When to set server

to FAILED
Select a server and locate its Configuration > Overload tab. The following fields relate to stuck
thread handling:
• Shared Capacity For Work Managers: The total number of requests that can be
present in the server. This includes requests that are in the queue and awaiting
execution as well as those under execution. The server performs a differentiated denial
of service on reaching the shared capacity. A request with higher priority will be
accepted in place of a lower priority request already in the queue. The lower priority
request is kept waiting in the queue until all high priority requests are executed.
Additional lower priority requests are rejected right away.
• Failure Action: Enable automatic shutdown or suspension of the server on a failed
state. The server self-health monitoring detects fatal failures and marks the server as
failed.
• Max Stuck Thread Time: The number of seconds that a thread must be continually
working before this server considers the thread stuck
• Stuck Thread Count: The number of stuck threads after which the server is transitioned
into FAILED state. Mark the server instance as failed and shut it down if there are stuck
threads in the server. In a cluster, clustered clients that are connected or attempting to
connect will fail over to another cluster member.

Application Stuck Thread Handling
Individual applications can automatically transition into

administration mode when stuck threads are detected.
<application-admin-mode-trigger>
<max-stuck-thread-time>300</max-stuck-thread-time>
<stuck-thread-count>5</stuck-thread-count>
</application-admin-mode-trigger>
weblogic-application.xml
The <application-admin-mode-trigger> element of weblogic-application.xml

specifies the number of stuck threads needed to bring the application into administration mode.
Similar to server-level stuck thread detection, you must specify the amount of time, in seconds,
for which a thread must execute an application request before being considered stuck.

Quiz
For which of the following levels in WebLogic is stuck thread

handling available?
a. Domain and application scope
b. Domain and server scope
c. Server and machine scope

d. Server and application scope
Answer: d

Quiz
In which of the following files is the configuration for

globally-scoped work managers located?
a. config.xml
b. weblogic.xml
c. config/workmanager-<num>.xml
d. weblogic-application.xml
Answer: a

Creating a Work Manager
Using the administration console, you can create global work managers that are used to
prioritize thread execution. To create a global work manager:
1. In the left pane of the con sole, expand Environment and select Work Managers. Click
New.
2. Select Work Manager, and click Next.
3. In the Name field, enter a name for the new work manager. Click Next.
4. In the Available Targets list, select the server instances or clusters on which you will
deploy applications that reference the work manager. Then, click Finish.

Request Classes
ClassType Description
Fair Share A numeric weight that represents the average percentage of
thread execution time while under load (default=50)
Response Time A target average response time, in milliseconds
Context Associate other request

principals (users, groups)classes with specific security
Context
Fair Share
Request Class
Request Class
Work 80%
Group A
Manager
Group B Fair Share
Request Class
20%
There are multiple types of request classes, each of which expresses a scheduling guideline
in different terms. A Work Manager may specify only one request class.
A fair share request class specifies the average thread-use time required to process requests.
For example, assume that WebLogic is running two modules. The Work Manager for Module
1 specifies a fair share of “80” and the Work Manager for Module 2 specifies a fair share of
“20.” During a period of sufficient demand, with a steady stream of requests for each module
such that the number of requests exceeds the number of threads, the WebLogic Server will
allocate 80% and 20% of the thread-usage time to Module 1 and Module 2, respectively.
A response time request class specifies a response time goal in milliseconds. Response time
goals are not applied to individual requests. Instead, the WebLogic Server computes a
tolerable waiting time for requests with that class by subtracting the observed average thread-
use time from the response time goal, and schedules requests so that the average wait for
requests with the class is proportional to its tolerable waiting time.
A context request class aggregates one or more other request classes, and assigns each to a
unique username or group.

Creating a Request Class
3
1
Edit the work manager and

assign the request class.
4
After you have created a global work manager, you typically create at least one request class
or constraint and assign it to the work manager. Each work manager can contain only one
request class, but you can share request classes among multiple work managers. To create a
global request class:
New and then select the type of global request class that you want to create. Click Next.
2. For a fair share request class, enter the numeric weight in the Fair S hare field. For a
response time request class, enter a time in milliseconds in the Response Time Goal
field. When finished, click Next.
3. In the Available Targets list, select the server instances or clusters on which you will
deploy applications that reference this request class. Then, click Finish.
4. Edit your existing work manager. Select your new request class by using the Request
Class field, and click Save.

Constraints
ConstraintType Description
Maximum Threads • The maximum number of concurrent threads that can be

used to service requests
• Can be a static value or can dynamically adjust to the
maximum capacity of a given JDBC data source
Minimum Threads The minimum number of threads that the server should
allocate to avoid deadlocks
Capacity The maximum size of the request backlog before rejecting

new requests
A maximum threads constraint limits the number of concurrent threads executing requests
from the constrained work set. The default is unlimited. For example, consider a constraint
defined with maximum threads of 10 and shared by three entry points. The scheduling logic
ensures that not more than 10 threads are executing requests from the three entry points
combined. You can define a static value for this constraint or in terms of the availability of a
dependent resource, such as a JDBC data source connection pool.
A minimum threads constraint guarantees the number of threads the server will allocate to
affected requests to avoid deadlocks. The default is zero.
A capacity constraint causes the server to reject requests only when it has reached its
capacity. The default is -1. Note that the capacity includes all requests, queued or executing,
from the constrained work set. Work is rejected either when an individual capacity threshold is
exceeded or if the global capacity is exceeded. This constraint is independent of the global
queue threshold.

Creating a Constraint
4
Edit work manager and
3
assign request class
Edit the work manager and

assign constraints.
To create a global constraint and assign it to a work manager:

New and then select the type of global constraint that you want to create. Click Next.
2. Enter the configuration information based on the type of constr aint you are creating. For
a maximum or minimum threads constraint, enter a thread count. Alternatively, for a
maximum threads constraint, use the Data Source field to supply the name of a JDBC
data source. The maximum capacity of the data source is then used as the constraint.
3. In the Available Targets list, select server instances or clusters on which you will deploy
applications that reference this constraint.
4. After you have created a constraint, you must assign it to an existing work manager to
use it. Each work manager can contain only one constraint of each type, but you can
share constraints among multiple work managers.

Referencing Classes or Constraints
A Work Manager is defined by encapsulating one request class

and many constraints to schedule requests.
There has to be one request class or constraint or both defined within a Work Manager. The
Work Manager snippet in the slide uses one request class and one constraint. The Work
Managers cannot have more than one request class; however, it can contain many
constraints.
Snippet from config.xml:
<work-manager>
<name>priority_work_manager</name>
<fair-share-request-class>
<name>very_high_priority</name>
<fair-share>1000</fair-share>
</fair-share-request-class>
<min-threads-constraint>
<name>min_five_Threads</name>
<count>5</count>
</min-threads-constraint>
</work-manager>

Sharing Constraints Among Work Managers
• Constraints can be shared among Work Managers.

• Create independent constraints, and then assign to Work
Managers.
You can create constraints that are independent of Work Managers. Then you can create
Work Managers and assign the constraints to the Work Managers. This way you can share
constraints among Work Managers.

Work Managers and Stuck Threads
• WebLogic can automatically shut down a work manager

when it detects that the work manager has stuck threads.
• For global work managers, you must use WLST.
Configure stuck thread detection for a global work manager:

...
wmShutdown = getMBean('/SelfTuning/MyDomain/WorkManagers/wm1/
WorkManagerShutdownTrigger/wm1')
wmShutdown.setStuckThreadCount(5)
wmShutdown.setMaxStuckThreadTime(300) WLST Script
The WorkManagerShutdownTriggerMBean MBean configures the conditions under which

an associated work manager is automatically shut down. The trigger specifies the number of
threads that need to be stuck for a certain amount of time. A shutdown work manager refuses
new work but attempts to complete pending work. There is currently no interface in the
administration console for configuring this MBean type for global work managers. For
application-scoped work managers, use the<work-manager-shutdown-trigger>
element, which is a child element of <work-manager>.

Assigning Work Managers to Applications
Assign a work manager to a web application or module by

using weblogic.xml:
...
<wl-dispatch-policy>HighPriorityWM</wl-dispatch-policy>
weblogic.xml
Assign a work manager to a specific EJB by using weblogic-

ejb-jar.xml:
...
<ejb-name>AccountManager</ejb-name>
...
<dispatch-policy>HighPriorityWM</dispatch-policy>
</weblogic-enterprise-bean> weblogic-ejb-jar.xml
In your deployment descriptors, you reference one of the work managers, request classes, or
constraints by its name.
An enterprise application (EAR) cannot be directly associated with a work manager, although
it can define its own application-scoped work managers. Instead, individual modules within
the enterprise application can reference global or application-scoped work managers.
For web or web-service applications, use the wl-dispatch-policy element to assign the
web application to a configured work manager by identifying the work manager name. This
web application-level parameter can be overridden at the individual servlet or JSP level by
using the per-servlet-dispatch-policy element.
For EJB applications, use the dispatch-policy element to assign individual EJB
components to specific work managers. If nodispatch-policy is specified, or the
specified dispatch-policy refers to a nonexistent work manager, the server’s default work
manager is used instead.

Quiz
Which of the following is NOT a type of work manager request

class?
a. Context
b. Fair Share
c. Response Time
d. Cluster
Answer: d

Quiz
Identify the true statement or statements.

a. The Stuck Thread Work Manager is configured to always
send an SNMP trap in response to stuck threads
parameters.
b. The Stuckthe
increase Thread
threadWork
pool Manager is configured
to compensate tothreads.
for stuck always
c. The Stuck Thread Work Manager is configured to shut
down the Work Manager by killing all threads in response
to stuck threads parameters.
Answer: a

Lesson 9: Tuning J2EE Applications
a workshop format.

Session Persistence Considerations
• File and Java Database Connectivity (JDBC) session

persistence typically have a significantly larger overhead
than in-memory replication.
• Because WLS replicates sessions at the attribute level,
application developments should attempt to:
– Utilize sessions sparingly (consider cookies)
— Even better, consider Coherence*Web
– Group data that often changes together into an attribute
– Minimize the size of each attribute
The WebLogic Server offers session persistence mechanisms that cater to the differing
requirements of your application. The session persistence mechanisms are configurable at
the Web application layer. The session management strategy that you choose for your
application depends on real-world factors such as HTTP session size, reliability, and session
failover requirements. In terms of pure performance, the in-memory session persistence is a
better overall choice. However, in-memory-based session persistence requires the use of
WebLogic clustering, so it is not an option in a single-server environment.
As a general rule, you should optimize your application so that the WebLogic Server does as
little work as possible when handling session management and persistence. The use of
sessions involves a scalability trade-off in most contexts. Use sessions only for states that
cannot realistically be kept on the client or if URL rewriting support is required. For example,
keep simple bits of states, such as a user’s name, directly in cookies. Similarly, keep
frequently used values in local variables if appropriate.
In your application, try to segregate changing session data into a single session attribute. If
you use a single large attribute that contains all the session data and only 10% of that data
changes, the entire attribute has to be replicated, causing unnecessary network overhead.

Session Persistence Cache Size
• You can control the number of latest sessions in server

memory for file or JDBC persistence.
• The timeout-secs, cache-size, and invalidation-
interval-secs parameters control the volume of
session data before being swapped to the persistent store.
• Snippet from weblogic.xml:
...
<persistent-store-type>file</persistent-store-type>
<cache-size>2056</cache-size>
...
</session-descriptor>
...
You can configure the number of sessions that are held in memory for an application by using
several parameters in the <session-descriptor> element of the weblogic.xml
deployment descriptor file.
The <cache-size> parameter limits the number of cached sessions that can be active in
memory at any one time. It is applicable only for file and JDBC persistence. If you expect high
volumes of simultaneous active sessions, you do not want these sessions to soak up the
RAM of your server because this may cause performance problems swapping to and from
virtual memory. When the cache is full, the least recently used sessions are stored in the
persistent store and recalled automatically when required. By default, the number of cached
sessions is 1,028. To turn off caching, set this to 0.
The <invalidation-interval-secs> parameter sets the time, in seconds, that the
WebLogic Server waits between doing house-cleaning checks for timed-out and invalid
sessions, and deleting the old sessions to free up memory. Use this element to tune the
WebLogic Server for best performance on high-traffic sites. The default value is 60 seconds.

Offload HTTP Sessions to Coherence*Web
• WLS performance can be impacted if too many sessions

remain in memory.
– Consider Coherence*Web for apps with lots of users, large
sessions, or long session timeouts.
– Offloads sessions into a Data Grid.
– Easy to scale by increasing the number of Coherence nodes
in the Grid.
– Allows sharing sessions across applications

Quiz
Can a Stateless Session Bean Maintain State?

a) A Stateless Session Bean cannot maintain state under
any circumstance
b) A Stateless Session Bean can maintain state if it
implements
invocations specific code to preserve its state between
c) The moment a Stateless Session Bean implements code
for state preservation across invocations, it automatically
becomes a Stateful Session Bean
Answer: b
A Stateless Session Bean defined through the @Stateless annotation remains a
Stateless Session Bean for the duration of the application, it is placed in Stateless Bean
pool and is treated by WebLogic as a Stateless Bean. It can implement, however, code
which saves its state to the database and retrieves it when invoked by the same or
another session. In this case state is maintained, but the bean is still a Stateless Bean.

Quiz
Select which statement is true.

a. During passivation, an ejb application is shutdown.
b. During activation, all bean instances are written to file.
c. During passivation unused bean instances are written to
file.
d. During activation bean instances are written to database.
Answer: c

Quiz
Select which statement is true.

a) A Stateless Session Bean can implement a Web Service,
but a Stateful Session Bean cannot.
b) Both Stateful and Stateless Session Beans can implement
a Web Service
c) A Stateful Session Bean can implement a Web Service,
but a Stateless Session Bean cannot.
Answer: a

Singleton Session Bean
A Singleton Session Bean is instantiated once per application

and exists for the lifecycle of the application.
• Designed for circumstances in which a single enterprise
bean instance is shared across and concurrently accessed
by clients
• Singleton Session Beans offer similar functionality to
stateless session beans but differ from them in that there is
only one singleton session bean per application, as
opposed to a pool of stateless session beans, any of which
may respond to a client request.
• Like stateless session beans, singleton session beans can
implement web service endpoints.
Singleton session beans maintain their state between client invocations but are not
required to maintain their state across server crashes or shutdowns.
Applications that use a singleton session bean may specify that the singleton should be
instantiated upon application startup, which allows the singleton to perform initialization
tasks for the application. The singleton may perform cleanup tasks on application
shutdown as well, because the singleton will operate throughout the lifecycle of the
application.

When to use Singleton Session Beans
Singleton Session Beans are appropriate in the following

circumstances:
• State needs to be shared across the application.
• A single enterprise bean needs to be accessed by multiple
threads concurrently.
• The application needs an enterprise bean to perform tasks
upon application startup and shutdown.
• The bean implements a web service.

D83171GC10 - SG - Oracle WebLogic Server 12c - Performance Tuning Workshop (Student Guide)

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

D83171GC10 - SG - Oracle WebLogic Server 12c - Performance Tuning Workshop (Student Guide)

Uploaded by

Copyright:

Available Formats

Oracle WebLogic Server 12 c:

Performance Tuning Workshop

Elio Bonazzi Disclaimer

2 Performance Tuning: Overview

Oracle WLS Monitoring Dashboard 2-36

3 Tuning Operating System Resources

Section Summary 3-12

Section Summary 3-30

4 Tuning HotSpot JVM

Focusing on Throughput 4-23

Using jstat 4-41

Profiler Snapshot 4-54

MBeans: Notifications 4-68

Tab Groups of Flight Recorder 4-82

5 Monitor and Tune Server Performance

Monitor Channel Connections 5-6

Section Summary 5-22

6 Monitor and Tune Cluster Performance

Benefits of Clustering 6-6

Section Summary 6-23

7 Tuning JDBC Performance

Statement Cache Tuning 7-20

8 Configuring Work Managers

Section Summary 8-7

9 Tuning J2EE Applications

JSP Request Processing 9-6

Page Directive and HTTP Session 9-20

Message-Driven Beans (MDBs) 9-36

Section Summary 9-54

Appendix A: Reference Slides

After completing this course, you should be able to:

Oracle WebLogic Server 12c: Performance Tuning Workshop 1 - 2

This course is intended for WebLogic Server administrators.

Oracle WebLogic Server 12c: Performance Tuning Workshop 1 - 3

Oracle WebLogic Server 12c: Performance Tuning Workshop 1 - 4

This is a workshop. That means the

Oracle WebLogic Server 12c: Performance Tuning Workshop 1 - 5

This schedule is approximate and subject to change.

Oracle WebLogic Server 12c: Performance Tuning Workshop 1 - 6

Oracle WebLogic Server 12c: Performance Tuning Workshop 1 - 7

• Most topics are reinforced with a hands-on practice.

Oracle WebLogic Server 12c: Performance Tuning Workshop 1 - 8

• The instructor starts each session at the scheduled time.

Oracle WebLogic Server 12c: Performance Tuning Workshop 1 - 9

Oracle WebLogic Server 12c: Performance Tuning Workshop 1 - 10

Oracle WebLogic Server 12c: Performance Tuning Workshop 1 - 11

Oracle WebLogic Server 12c: Performance Tuning Workshop 1 - 12

After completing this lesson, you should be able to do the

• Apply performance testing methodology and tools

Oracle WebLogic Server 12c: Performance Tuning Workshop 2 - 2

• Performance Tuning: Overview

– Why Is Testing Important?

Oracle WebLogic Server 12c: Performance Tuning Workshop 2 - 3

The performance of a software system is often characterized

Resource utilization A consumption metric (for example, the percentage of CPU

Oracle WebLogic Server 12c: Performance Tuning Workshop 2 - 4

The term scalability refers to a system’s ability to perform within

Oracle WebLogic Server 12c: Performance Tuning Workshop 2 - 5

Performance testing includes a variety of test types.

Test Type Description

Functional Test Verifies that a specific application or application unit is performing

Load Test Determines how a system functions while handling a certain

Oracle WebLogic Server 12c: Performance Tuning Workshop 2 - 6