openSAP cst2 Week 2 All Slides PDF

Week 2: Performance & Management
Unit 1: SAP JVM Insight and Profiling Tools

SAP JVM Insight and Profiling Tools
SAP JVM
▪ The SAP JVM is a certified Java Virtual Machine (JVM)

and Java Development Kit (JDK).
▪ It is designed and developed with supportability, broad
platform support, and reliability as the main paradigms.
▪ As a commercial Java licensee, the SAP JVM is based
on the OpenJDK.
▪ The platform ports, including OS support, JIT compiler,
and bytecode interpreter, were done by SAP.
© 2018 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 2

SAP JVM
The SAP JVM has the following main primary jobs:

▪ Execute Java code
▪ Manage memory
– Allocates memory from the operating system (OS)
– Distributes memory to the Java applications internally
– Manages memory allocations and deallocation

SAP JVM tools
SAP JVM is enhanced with several

supportability features like:
▪ SAP JVM Profiler
▪ SAP JVM Monitoring Tool

SAP JVM Profiler
The SAP JVM Profiler helps analyze Java applications

running in the SAP JVM. It has the following capabilities:
▪ Allocation analysis
▪ Performance hotspot analysis
▪ Method parameter analysis
▪ Synchronization analysis
▪ File I/O analysis
▪ Network I/O analysis
▪ Garbage collection analysis
▪ Class statistics analysis
▪ Thread dump analysis
▪ Heap dumps

SAP JVM Profiler
To connect the Profiler to the SAP JVM, the

jvmmond has to run in the host where the JVM is.
The Profiler itself is just a front end and can run

from another host.
▪ To start the jvmmond, execute it from the filesystem,
under the SAP JVM directory …\sapjvm_X\bin\.
More information about the SAP JVM Profiler is

available in SAP Note 1783031, SAP KBA
2621050 and SAP Community at:
https://wiki.scn.sap.com/wiki/x/LB0B

SAP JVM Profiler
Once the Profiler is connected to the

SAP JVM, you can monitor the JVM
with detailed information on the
following:
▪ General
▪ System
▪ Performance
▪ Garbage Collection
▪ JIT Compiler

SAP JVM Profiler
On the SAP JVM Profiler main screen,

select the Profiling analysis needed.
There are 6 analysis options to perform the

profiling on the AS JAVA.

SAP JVM Monitoring Tool
Using the SAP JVM Monitoring Tool you can

send commands to the JVM:
▪ Force garbage collection
▪ Force maximum compaction GC
▪ Retrieve complete GC history
▪ Stop command line profiling
▪ Change command line flags…
▪ Set core size limit to max
▪ Set trace flags…
And you can also debug Java applications

running in the JVM.

SAP JVM Monitoring Tool
Using the SAP JVM Monitoring Tool you can

also print in its own GUI or dump to an
external file data like:
▪ Stack trace
▪ DLL info
▪ Compressed class-space info
▪ Garbage collection history
▪ Class statistic
▪ Codeblobs
▪ hs_err_info
▪ JIT info
▪…

Thank you.
Contact information:
open@sap.com
© 2018 SAP SE or an SAP affiliate company. All rights reserved.
No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP SE or an SAP affiliate company.
The information contained herein may be changed without prior notice. Some software products marketed by SAP SE and its distributors contain proprietary software components
of other software vendors. National product specifications may vary.
These materials are provided by SAP SE or an SAP affiliate company for informational purposes only, without representation or warranty of any kind, and SAP or its affiliated
companies shall not be liable for errors or omissions with respect to the materials. The only warranties for SAP or SAP affiliate company products and services are those that are
set forth in the express warranty statements accompanying such products and services, if any. Nothing herein should be construed as constituting an additional warranty.
In particular, SAP SE or its affiliated companies have no obligation to pursue any course of business outlined in this document or any related presentation, or to develop or release
any functionality mentioned therein. This document, or any related presentation, and SAP SE’s or its affiliated companies’ strategy and possible future developments, products,
and/or platforms, directions, and functionality are all subject to change and may be changed by SAP SE or its affiliated companies at any time for any reason without notice. The
information in this document is not a commitment, promise, or legal obligation to deliver any material, code, or functionality. All forward-looking statements are subject to various risks
and uncertainties that could cause actual results to differ materially from expectations. Readers are cautioned not to place undue reliance on these forward-looking statements, and
they should not be relied upon in making purchasing decisions.
SAP and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP SE (or an SAP affiliate company)
in Germany and other countries. All other product and service names mentioned are the trademarks of their respective companies.
See www.sap.com/corporate-en/legal/copyright/index.epx for additional trademark information and notices.
Unit 2: Memory Management Overview
Memory Management Overview
Java Virtual Machine
The AS Java memory is managed by the SAP JVM.

Its primary jobs related to memory are:
▪ Allocate memory from the operating system
▪ Distribute the memory among applications running in
the JVM
▪ Allocate memory for Java objects
▪ Deallocate memory from Java objects
▪ Run the garbage collector (GC)

Heap Memory
The heap memory is managed with the Generation

Mechanism in SAP JVM, and it consists of 2 different
Young
memory areas:
▪ Young (New) Generation
– Is smaller than the Old area
– New objects are allocated here
▪ Old (Tenured) Generation
– Is the bigger memory segment
– Old objects, which were previously in the Young area
and were promoted, stay here Old

Garbage collector
What is a garbage collector (GC)?

▪ A type of automatic memory management present in
the JVM
▪ Removes unreferenced (unused) objects from the
heap memory

Garbage collector
How does the GC run and how is it triggered:
▪ Runs independently from the applications
▪ Is triggered by certain events:

– Amount of free memory below a certain threshold
– Object allocation request cannot be fulfilled
(not enough available free space)

Minor GC
Focuses on the Young Generation

▪ Automatically triggered when the Young space is full Young
▪ Smaller memory area to scan = fast operation
(~0.2 seconds)
Old

Full GC
Focuses on the whole heap

▪ Triggered when a minor GC does not free up Young
enough memory in the Young space
▪ Triggered when the amount of free space in the
Old generation is below a certain threshold
▪ Larger area to scan = Time-consuming operation
(~10 seconds)
Old

Java process
Besides the Java heap, which is formed by

Young and Old generations, the Java process
in the operating system also has:
▪ Metaspace
▪ Codecache
▪ Thread stacks
▪ Shared libraries
Meta-
space Thread Shared
Java heap
Code- stack libraries
cache

Metaspace
Previously known as Permanent Generation

(check SAP Note 2121243), it has the
following characteristics:
▪ Holds metadata describing user classes
▪ Applications with a large code base quickly fill
this area
▪ Is covered by the Full GC
Meta-
space Thread Shared
Java heap
cache

Codecache
▪ Stores code generated by the JIT compiler

▪ When the number of compiled classes do not fit the codecache area,
the JIT compiler is disabled until the area is flushed.
More details:
SAP NOTE 1591449: Increasing the SAP JVM's codecache size
and
SAP KBA 2442336: How to enable SAP JVM codecache sweeping
Meta-
space Thread Shared
Java heap
cache

Thread stack and shared libraries
The thread stack area holds information about

the thread execution. Usually, each thread uses
from 1 to 4 megabytes of this memory segment.
In shared libraries you have a native memory

segment with libraries, i.e. DLLs.
Meta-
space Thread Shared
Java heap
cache

Out of memory
When the heap space is not enough to fit all the objects, the Java process is restarted with an
OutOfMemoryError.
In this case, the AS Java restarts with an exit code 666. A heap dump can be generated, containing
all the content of the memory, to be analyzed in order to find the cause of the resource shortage.

Thank you.
open@sap.com
Unit 3: Heap Dump Analysis
Heap Dump Analysis
OutOfMemoryError
An OOM error happens when the total size of the

Java objects that you have, plus any additional
Young
Java object that you are also trying to allocate,
do not fit in the heap memory.
It might be caused by:

▪ Incorrect server sizing
▪ Code memory leak
Old

Heap Dump Analysis
OutOfMemoryError
You can see when an OutOfMemoryError happens as the AS Java server node restarts with an
exit code 666.
The exit code can be monitored mainly offline through:

▪ Logs
▪ SAP MMC

Heap Dump Analysis
JVM parameters
When an OutOfMemoryError happens, a

heap dump containing all the data from the
heap can be generated in the filesystem
automatically. The JVM parameter to enable
it is:
▪ -XX:+HeapDumpOnOutOfMemoryError
The file will be written with extension “.hprof”.

Heap Dump Analysis
JVM parameters
By default, in AS Java, the heap dumps are

written to the server node’s directory
(/usr/sap/<SID>/<InstID>/j2ee/cluster/server
<N>/). You can change this location with the
JVM parameter:
▪ -XX:HeapDumpPath=<directory where to save
the heap dumps>

Heap Dump Analysis
Manual heap dump
The heap dump can also be generated

manually, although this is not recommended
and used in just a few and specific cases. It
can be generated in the SAP JVM Monitoring
Tool:
1. On the top bar of the tool, select Dump
2. Select Dump heap
More information on this be obtained at

https://wiki.scn.sap.com/wiki/x/dgYlDg

Heap Dump Analysis
Memory Analyzer tool
To parse a heap dump, the Eclipse Memory

Analyzer tool (MAT) is used. It can be
downloaded from:
▪ http://www.eclipse.org/mat/
To be used with the SAP NetWeaver Application

Server for Java, extension packs need to be
installed in the MAT. More details: SAP Note
1883568.
The extension packs can be installed following

the steps from this Wiki.

Heap Dump Analysis
Inside the MAT, open the heap dump as follows:

1. Click the File menu at the top
2. Click Open Heap Dump…
3. Select the heap dump file and click OK

Heap Dump Analysis
After selecting the heap dump, the MAT will start

to parse it. Depending on the host resources, the
parsing can take minutes or hours.

Heap Dump Analysis
Reports
While the parsing is being done, the MAT will

pause to question if a report should be
generated:
▪ Leak Suspects Report
– Check all objects for memory leak suspects
▪ Component Report
– Check a set of specific objects for suspected
memory issues
▪ Re-open previously run reports

Heap Dump Analysis
Leak Suspects Report
The report selected on the previous screen is generated

in a ZIP file in the same directory as the heap dump is
stored.

Heap Dump Analysis
The Leak Suspects Report is the recommended

report for OutOfMemoryError issues.
In the report, there is a graph showing the

memory usage and what is using it.

Heap Dump Analysis
The Leak Suspects Report shows the

problem suspects in an ordered manner.
In the Problem Suspect field you can see:

▪ What is using the memory
▪ The keywords that can be used to search for
existing SAP Notes
▪ The CSN component, in case it is necessary to
contact SAP

Thank you.
open@sap.com
Unit 4: Java Threads and Thread Dump Analysis
Java Threads and Thread Dump Analysis
Java threads
▪ The thread system is the backbone of SAP

NetWeaver AS for Java and is needed to do multiple
tasks concurrently
▪ The main goal is to broaden the list of stakeholders
and raise the quality of service of applications
▪ The thread management infrastructure provides a
mechanism for registering thread queues, specifying
the size of the queue and the maximum allowed
concurrency
▪ Resources, like DB connections, SAP JCo
connections, etc. are handled by pools of opened
physical connections, which are logically re-used by
the applications in parallel threads

Application Server ABAP work processes vs Application Server Java threads
▪ The ABAP stack has a fixed number of work

processes which is single-threaded
▪ On an ABAP stack, a single work process can
be debugged and restarted without impacting
other processes/applications
▪ Java threads are created and destroyed on the
fly, on demand
▪ One badly-behaving program processed by a
thread may impact all other threads in the
same Java Virtual Machine (JVM)

Java Thread Managers
▪ Thread Manager: Supplies all system threads in

which AS Java system operations are executed
(core, services, etc.)
▪ Application Thread Manager: Supplies the
threads in which the code of the client
applications is executed
▪ Configuration is done using the SAP Config tool.
▪ Monitoring of the thread system information can
be done by the SAP Management Console (SAP
MMC), the shell console administrator using
telnet, or the SAP NetWeaver Administrator. For
more details, check SAP KBA ##1625458 – How
to check thread usage on SAP NetWeaver
Application Server for Java

Java threads lifecycle
Important thread conditions

▪ Born / Dead
▪ Sleeping (waiting on condition):
▪ Waiting (in Object.wait()):
▪ Thread synchronization: used when
different threads try to access/modify
the same object
▪ Monitor: Java class with synchronized
code in it, used to control the client
threads
▪ Object lock: Every object has a lock
and can be controlled by one thread
at a time

Deadlock situation

Thread dump
Thread dump:
▪ Shows all threads which exist at that moment on the

AS JAVA.
▪ Shows stack trace of all threads and their current
status
▪ Shows thread IDs, locked objects, and amount of
free threads
When to trigger a thread dump:

▪ System performance is affected, server hangs and
does not respond to requests
▪ High CPU consumption of a server node

How to trigger a thread dump
Important tools:
▪ SAP MMC or SAP MC (Applet)
▪ JVMMON (or JVMPROF)
▪ JSMON
▪ SAP Solution Manager RCA work center
▪ SAP NetWeaver Administrator
▪ SAP NetWeaver for Java, support tool
▪ Command Line (kill -3 <pid> on Unix) Task Manager
(Create Dump File on Windows)
Where to find the thread dumps:

▪ The thread dump is by default written to the dev_server<x> and the
std_server<x>.out files in the work directory
▪ When triggered via SolMan, the dumps can be analyzed
immediately.
Thread dump viewer
▪ Helps to automatically detect issues based on

common problem patterns (e.g. no free HTTP
thread, deadlock issues etc.)
▪ Can help in performing additional analysis steps,
e.g. "why there are no free HTTP threads", "what
are all HTTP threads doing”
▪ Interprets the semantics of stack traces and
proposes reasonable next steps
▪ Available for download in SAP Note 1020246 or
use the plug-in in the SAP NetWeaver Developer
Studio (NWDS)

General roadmap for thread dump analysis

Thank you.
open@sap.com
Unit 5: Monitoring
Monitoring
Infrastructure
▪ Monitoring the AS Java system helps the

administrator to identify resource bottlenecks or SAP NW
problems early on and to take any action that is Java
required. Applications
▪ The monitoring in SAP NetWeaver AS Java is based

Kernel ICM
on the standard Java Management Extensions
(JMX).
▪ The external tools connect through the JMXAPI and Server Server
Services Monitoring
Segment
…
can display all current values in the JMX monitors. JMX JMX
▪ There are various tools that can be used to analyze Performance

the monitored data.
sapstartsrv

Monitoring
Tools: SAP NetWeaver Administrator system overview
▪ AS Java overview dashboard: key metrics which

define the server state
▪ Displays metrics (OS, CPU, VM) related to the
Java engine in single dashboard
▪ Entry point for troubleshooting scenarios

Monitoring
Tools: SAP NetWeaver Administrator history reports
▪ NWA: → Availability & Performance → Resource

Monitoring → History Reports or /nwa/java-sys-
reports
▪ Use history reports as a problem detection and
analysis tool to:
a) Monitor the AS Java cluster and Java
applications
b) Track the performance of one or all instances
and cluster nodes of a particular Java system
c) Detect and analyze the causes of problems in
an AS Java in a situation of reduced
performance
d) Perform motivated tuning for productive use
with the AS Java

Monitoring
Tools: SAP NetWeaver Administrator thread dumps and heap dumps
▪ Troubleshooting → Advanced Troubleshooting

▪ The connected user must be assigned to the roles
NWA_SUPERADMIN or Administrator

Monitoring
Tools – JSMON
▪ Command line program for monitoring and

management of SAP NetWeaver AS Java
▪ To inspect the state of a Java instance; few
infrastructure demands
a) Display a list of the processes
b) Enable, disable, and restart the specified
processes
c) Shut down a specified Java instance
d) Enable/disable debugging for a specified
process
e) Dump stack trace
f) Display a list of the ports used by the Java
Engine

Monitoring
Tools – SAP Management Console (SAP MMC)
Prerequisites
▪ The SAP Host Agent is installed on the host
where the application server of the SAP system
or instance runs
▪ You have installed Java Runtime Environment
(JRE) 1.4.2 or higher
▪ Your Web browser supports Java
▪ Your Web browser’s Java plug-in is installed
and enabled to run scripting of Java applets

Monitoring
Tools – SAP Management Console (SAP MMC)
Features
▪ Monitor Java AS processes
▪ Monitor system performance and operations
▪ ICM monitoring
▪ AS Java session monitoring
▪ System state monitoring
▪ Trigger thread dumps
MMC Snapshot
SAP Note 1847251 – How to create an MMC
snapshot: SAP AS JAVA

Monitoring
Knowledge resources
▪ SAP Note 2577844 – AS Java Monitoring and

Logging parametrization best practice
▪ SAP Note 2304490 – SAP Management Console
(applet version) does not start
▪ SAP Note 1625458 – How to check thread usage
on SAP NetWeaver Application Server for Java

Thank you.
open@sap.com
Unit 6: CPU Analysis
CPU Analysis
Introduction
▪ Approximately 80% of the CPU time is spent in 20% of the coding.

▪ SAP JVM offers Performance Hotspot Analysis to identify which execution paths cause the highest
proportion of CPU consumption.
▪ SAP JVM Profiler's approach to discover hotspots is to collect method execution samples on a statistical
basis.
▪ It avoids excessive performance overhead, which makes the profiler ready for action even in a production
environment.

CPU Analysis
Profiling snapshot
▪ The profiling data is archived and managed in a

profiling snapshot.
▪ The snapshot can be analyzed tabularly and
graphically in the Eclipse-based front end of the
profiler or in NWDS.
▪ The profiler identifies expensive method
calls/execution paths.
▪ The user can use this data to resolve issues.

CPU Analysis
Collecting profiling snapshot
▪ SAP Note 1783031 – Analyzing AS Java

performance with SAP JVM Profiler
▪ Included in SAP JVM deliverable at sapjvm_X/bin

CPU Analysis
Evaluating profiler snapshot
▪ Methods (Flat) and Methods (Hierarchical) statistics

are the most useful representations of the profiling
data.
▪ Thread By ID and Thread By Name are thread-
oriented statistics that allow to divide the profiling
data by the different threads.
▪ Components (Flat) and Components (Hierarchical)
are additional statistics that make use of method-
and class filters to expose CPU times spent within
specific system components.

CPU Analysis
Knowledge resources
▪ SAP Note 1808977 – High CPU utilization is observed by

the jlaunch process
▪ SAP Note 1783031 – Analyzing AS Java performance with
SAP JVM Profiler
▪ SAP KBA 2621050 – How to profile and analyze AS JAVA
performance using Eclipse plug-in
▪ Community Wiki High CPU utilization due to excessive logging

Thank you.
open@sap.com

openSAP cst2 Week 2 All Slides PDF

Uploaded by

Copyright:

Available Formats

You might also like

openSAP cst2 Week 2 All Slides PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

openSAP cst2 Week 2 All Slides PDF

Uploaded by

Copyright:

Available Formats

Week 2: Performance & Management

Unit 1: SAP JVM Insight and Profiling Tools

▪ The SAP JVM is a certified Java Virtual Machine (JVM)

© 2018 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 2

The SAP JVM has the following main primary jobs:

© 2018 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 3

SAP JVM is enhanced with several

© 2018 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 4

The SAP JVM Profiler helps analyze Java applications

© 2018 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 5

To connect the Profiler to the SAP JVM, the

The Profiler itself is just a front end and can run

More information about the SAP JVM Profiler is

© 2018 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 6

Once the Profiler is connected to the

© 2018 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 7

On the SAP JVM Profiler main screen,

There are 6 analysis options to perform the

© 2018 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 8

Using the SAP JVM Monitoring Tool you can

And you can also debug Java applications

© 2018 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 9

Using the SAP JVM Monitoring Tool you can

© 2018 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 10

The AS Java memory is managed by the SAP JVM.

© 2018 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 2

The heap memory is managed with the Generation

© 2018 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 3

What is a garbage collector (GC)?

© 2018 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 4

How does the GC run and how is it triggered:

▪ Runs independently from the applications

▪ Is triggered by certain events:

© 2018 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 5

Focuses on the Young Generation

© 2018 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 6

Focuses on the whole heap

© 2018 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 7

Besides the Java heap, which is formed by

© 2018 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 8

Previously known as Permanent Generation

© 2018 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 9

▪ Stores code generated by the JIT compiler

© 2018 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 10

The thread stack area holds information about

In shared libraries you have a native memory

© 2018 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 11

© 2018 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 12

An OOM error happens when the total size of the

It might be caused by:

© 2018 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 2

The exit code can be monitored mainly offline through:

© 2018 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 3

When an OutOfMemoryError happens, a

The file will be written with extension “.hprof”.

© 2018 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 4

By default, in AS Java, the heap dumps are

© 2018 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 5