Best practices for SAP ERP using HP BladeSystem

c-Class blades

Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

Solution configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Server configuration . . . . . . . . . . . . . . . . . . . . . . . . . . 3

SAP solution-based servers . . . . . . . . . . . . . . . . . . . . . 3

Cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Storage management . . . . . . . . . . . . . . . . . . . . . . . . 3

Storage configuration . . . . . . . . . . . . . . . . . . . . . . . . . . 4

Storage array . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

Array configuration and virtual disk layout . . . . . . . . . . . . . . . 4

SAN infrastructure . . . . . . . . . . . . . . . . . . . . . . . . . 6

Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

SAP ERP workload analysis . . . . . . . . . . . . . . . . . . . . . . . 7

Response time components . . . . . . . . . . . . . . . . . . . . . . . 8

Test procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

SAP and server tuning . . . . . . . . . . . . . . . . . . . . . . . . . 9

Tuning SAP memory parameters . . . . . . . . . . . . . . . . . . . 9

Tuning SAP buffer parameters . . . . . . . . . . . . . . . . . . . . 9

Managing the total number of users . . . . . . . . . . . . . . . . 12

Managing the number of total work processes . . . . . . . . . . . . 13

Server sizing to accommodate more than 900 concurrent users . . . . 15

Using the database server as an SAP application server . . . . . . . 15

Storage tuning . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

Impact of customer data size . . . . . . . . . . . . . . . . . . . 18

Managing the number of disk drives in the SAPDATA disk group . . . . 19

Managing the number of EVA disk groups . . . . . . . . . . . . . . 20

Selecting the Vraid type for SAPDATA virtual disks . . . . . . . . . . 21

Selecting the Vraid type of virtual disk for the transaction log . . . . . . 22

Storage sizing to accommodate more than 900 concurrent users . . . . 23

Best practices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

SAP administrators . . . . . . . . . . . . . . . . . . . . . . . . . . 24

Server administrators . . . . . . . . . . . . . . . . . . . . . . . . . 24

Storage administrators . . . . . . . . . . . . . . . . . . . . . . . . 24

Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

We value your feedback . . . . . . . . . . . . . . . . . . . . . . . 26

Appendix A Bill of Materials . . . . . . . . . . . . . . . . . . . . . . . . 27

Appendix B References . . . . . . . . . . . . . . . . . . . . . . . . . . 28

For more information . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

HP technical references . . . . . . . . . . . . . . . . . . . . . . . . 29

Overview
A significant change is taking place in the SAP install base. While many SAP customers continue
to run older R/3-based versions, migrations to the newer NetWeaver-based versions of SAP are
accelerating. Migrations to these newer versions of SAP software require both conversions of the
application environment, as well as the need to evaluate, upgrade, and refresh the servers and
storage infrastructure used to deploy the SAP system landscape.

Once this new hardware infrastructure is chosen, a series of considerations are required to
effectively configure and deploy the new SAP NetWeaver software onto the new infrastructure.
This document provides medium-scale, enterprise-class SAP customers running Windows and
MS-SQL with a comprehensive set of best practices to appropriately configure and productively
deploy SAP NetWeaver and a typical ERP application onto HP c-Class blade servers and a
SAN-based infrastructure.

The following fundamental deployment and operational issues are addressed:

• The best way to configure the SAP landscape across the server blades, optimizing both
performance and availability

• The effects of scaling the environment when a blade is added as an additional application server

• The appropriate configuration for the HP EVA disk array, including:

– Choosing the appropriate number of disks

– Configuring the disks into the appropriate number of EVA disk groups for both the SAP
application components and the database

– Comparing Vraid 1 and Vraid 5

– Varying the size of the database

• The benefits of properly configuring the number and types of SAP work processes

• The benefits of properly tuning SAP shared memory, with emphasis on buffer parameters

Test results of methods and best practices are described in the following sections. This information
is intended to facilitate the productive planning, timely deployment, and seamless management
of a fully-operational SAP NetWeaver landscape on HP c-Class blade servers and SAN-based
infrastructure. Knowledge of this information will ensure:

• Proper deployment of the appropriate server and storage infrastructure

• Effective deployment of the Windows operating system, MS-SQL database, and SAP system
environment

• Accelerated deployment, reduced risk, and minimized total costs

By implementing the best practices provided in this white paper, HP:

• Reduced the baseline response time of the SAP workload by 67% with basic buffer tuning

• Improved I/O response time by 60% by properly sizing the number of disks

• Reduced response time by up to 40% when the SAP work processes were properly configured

In all cases where the number of disks was properly sized, the more economical and space
efficient Vraid 5 deployment provided very satisfactory performance for the database.

2
Solution configuration
Server configuration
Testing examined an SAP ERP 2005 system, which consists of a central instance (CI), a database
instance, and a dedicated dialog instance. Each instance is installed on an identically-configured
HP ProLiant BL460c Server Blade, with a clustered SAP CI and the Microsoft SQL Server
2005 Server Pack 2 (SP2) database. Figure 1 displays an SAP production system, an SAP
development system, or an SAP quality assurance system. The server configuration elements
used during testing are described in following sections. Refer to Table 1 for a complete listing
of the servers used during testing.

SAP solution-based servers

The hardware for the SAP solution-based servers includes three HP ProLiant BL460c Blade
servers running Microsoft Windows Server 2003 R2 Enterprise x64 Edition SP2. Each server has
two 2.66-GHz Intel Xeon X5355 quad-core processors and 32 GB of memory. The SAP ERP
application utilizes eight total CPU cores per server. Each server is equipped with a dual-port
Emulex 4 Gb PCI-E-to-Fibre Channel host bus adapter (HBA) utilizing a Microsoft Storport driver.
An HP Multipath I/O Device Specific Module 2.01.01 for Enterprise Virtual Array (HP MPIO DSM
2.01.01 for EVA) is used for managing the path control from the HBAs to the EVA8100.

The server configuration size is based on methods used by the HP SAP Competency Center for
the accommodation of 18,000 SAP Application Performance Standards (SAPS) by the entire SAP
system. In SAP Quick Size terms, an 18,000 SAPS system is a medium-sized SAP system.

Cluster

Two of the three ProLiant BL460c Blade servers running Microsoft Windows Server 2003 R2
Enterprise x64 Edition SP2 are clustered using Microsoft Cluster Server. A cluster group is created
using the database server and the CI server in the SAP system landscape. The testing is executed
with the SQL Server 2005 database and the SAP CI running on different servers. Clustering the
two servers provides a level of availability in case a server fails.

Storage management

Storage management is provided by a ProLiant BL460c Blade server running Microsoft Windows
Server 2003 R2 Enterprise Edition SP2. The server utilizes HP StorageWorks Command View
EVA 7.0, with the EVAPerf Enterprise monitoring tool, for collecting EVA performance information.

Table 1. Component list—servers

Server type Server use

ProLiant BL460c SAP central instance cluster

ProLiant BL460c SQL Server 2005 database cluster

ProLiant BL460c SAP dialog instance

ProLiant BL460c Storage management

ProLiant BL460c SAP solution manager

ProLiant BL460c Domain controller

3
Figure 1. Server configuration diagram, BladeSystem C7000 Enclosure

Storage configuration
Storage array

An HP StorageWorks EVA8100 2C12D storage array running XCS 6110 firmware is fully
populated with 168 146 GB 15K RPM Fibre Channel disk drives. This array stores the SAP
and the SQL Server 2005 executables, the SAPDATA files (where all the SAP ERP database
files are located), and the database log files.

Array configuration and virtual disk layout

Three initial configurations of EVA disk groups are tested to provide performance comparisons.

The SAPDATA files are placed in a disk group containing at least 32 disks to ensure that there
is enough capacity to accommodate a minimum 1 TB of SAPDATA and the supporting SQL
Server 2005 and SAP files.

In the first configuration, 32 disk drives are used, and all the virtual disks are placed in a single
disk group. This configuration provides the easiest method of administration because of the
simplicity of its design.

In the second configuration, two disk groups are used with a total of 40 disk drives:

• One disk group for the SQL Server 2005 transaction logs, consisting of eight disks

• One disk group for the SAPDATA files, consisting of 32 disks

In this configuration, the sequential I/O log files are separated from the random I/O SAPDATA
files. Availability is improved, because database data files and log files are separated into distinct
disk groups. If the disk group with the SAPDATA files becomes unavailable, the transaction logs
are safe. If the other disk group becomes unavailable, the SAPDATA files are safe.

4
In the third configuration, 40 disk drives are used, and all virtual disks are placed in a single disk
group. This configuration has the same benefits as the first configuration, but provides the SAP
system with more disk resources. More disks in a disk group provide the virtual disks in that disk
group with a greater capacity to handle I/O at an acceptable latency.

The fourth configuration, which is not illustrated, is identical in layout to the second configuration,
except it uses a total of 48 disk drives:

• The disk group containing the SAPDATA files, consisting of 40 disks

• The disk group containing the database transaction logs, consisting of eight disks

Best practice
Use disk groups are populated with disk drives in multiples of eight. This is an EVA best practice that ensures that
internal administration of the EVA disk groups is optimized.

We use Vraid 5 for SAPDATA files because fewer disks are required to store the same amount of
data than Vraid 1. For many workloads, Vraid 5 performs similarly to Vraid 1.

All of the configurations, the SAP executables, the SQL Server 2005 executables, the cluster
quorum, and the Microsoft Distributed Transaction Coordinator (MSDTC), are placed on separate
virtual disks in the disk group containing the SAPDATA files.

To allow the storage activity to be balanced evenly between the EVA controllers, two virtual disks
are used for the SAPDATA files. Each SAPDATA virtual disk contains two SAPDATA files.

Figure 2. EVA8100 configuration diagram

Note
In Figure 2, the boxes with dotted-lined borders represent EVA virtual disks. The virtual disks for SAP executables, SQL
Server 2005 executables, cluster quorum, and MSDTC are not shown.

5
SAN infrastructure

Two unzoned HP BladeSystem 4 Gb SAN switches running v5.3.0d firmware with 4 Gb optical
transceivers are used to create two independent 4 Gb fabrics.

Table 2. Component list—storage

Storage components Component type

Host bus adapters (HBA) (6) Emulex LPe1105 FC Dual Channel 4 Gb,
PCI-E-to-Fibre Channel

SAN switches (2) Brocade 4 Gb SAN Switch for c-Class BladeSystem

Storage array (1) EVA8100 Storage Array

6
Testing
Objectives
This document presents best practices for storage administrators, server administrators, and SAP
administrators, using SAP ERP 2005 with the HP StorageWorks EVA8100 storage array, HP
ProLiant c-Class Blade servers, and the SQL Server 2005 database as part of an SAP system.

This document also provides a performance proof point for an SAP ERP workload with a
combination of HP storage devices and servers. With this information, it is easy to determine if a
given SAP ERP workload provides an acceptable user response time.

Tuning and sizing the servers and storage devices to handle the workload is a vital step in
providing a solution that meets performance expectations. This goal is accomplished by varying
server and storage configurations and tuning the parameters of SAP to optimize performance
for the SAP ERP workload.

SAP ERP workload analysis

This section examines how storage and server tuning influences an SAP ERP workload’s
performance, including:

• Characteristics of the SAP ERP Sales and Distribution workload

• I/O characteristics of the workload from the servers to the virtual disk on the storage array

A virtual disk is a logical unit of storage on the EVA8100 storage array. The size of the virtual
disk is specified. However, the physical storage space for a virtual disk can exist on a number of
different disk drives in a disk group managed by the EVA8100.

For the SAP ERP workload, the virtual disks that must be managed are the SAPDATA (all of the
SAP ERP database files are located here), and the database transaction log virtual disks. Almost
100% of all storage activity involving I/Os per second (IOPS) and workload data throughput occurs
on the SAPDATA and the transaction log virtual disks. Storage activity to the transaction log virtual
disk will always be 100% write, with a typical write size of 16 KB per I/O.

The SAP ERP workload storage activity for the SAPDATA virtual disks follows:

• The ratio of read-to-write host IOPS is 4:1.

• The ratio of read-to-write host throughput is 4:1.

• The read size is 8 KB per I/O.

• The write size is 8−64 KB per I/O, with the vast majority of writes being 8 KB.

• This SAP ERP workload for SAPDATA virtual disks is read-I/O dominated.

The SAP ERP Sales and Distribution workload has several characteristics, such as creating order,

delivery, and invoice records for your business customers.

To execute this workload, follow these steps:

1. Create a customer order with SAP transaction VA01.

2. Create a delivery date for the order with SAP transaction VL01N.

3. Display the order with SAP transaction VA03.

7
4. Update the delivery date for the order with SAP transaction VL02N.

5. List the orders for the customer with SAP transaction VA05.

6. Create an invoice for the order with SAP transaction VF01.

Response time components

The primary goals of this testing are to:

• Understand the impact of various server, infrastructure, and storage configurations

• Optimize total response time for a given hardware configuration with a real-world SAP ERP
user load and configuration

Total response time is defined as the time it takes the SAP system to process a user request.
Optimized response time allows an SAP system to process more transactions during a given
time-period. This section describes the components of total response time.

The primary components of total response time are the individual response times for SAP work
processes that include: Dialog, Update1, and Update2.

Dialog. The Dialog work process responds to an active user request to execute dialog steps from
the SAP user interface. For instance, think of a person sitting at terminal entering information on
an SAP user screen and then performing an action that processes the data.

Update1. The Update1 work process is a time-critical update to the SAP database. Think of this
process as taking the information that the user entered in a Dialog step, and then moving that
information from the user screen into database tables.

Update2. The Update2 work process is a low-priority database update. It is used primarily in
statistics-related database transactions.

We can break down the components of total response even further. The Dialog, Update1, and
Update2 work processes all have the following response-time subcomponents:

• Wait time—The time it takes for SAP to find a free work process

• CPU time—The amount of time that the work process has active control of the CPUs

• Database request time—The time needed to access and perform operations on the database

• Processing time—The total time needed to execute an SAP program

Test procedures
To characterize configuration performance using this SAP ERP workload, real-time user loads are
run, and performance information is captured for SAP, the servers, and the storage devices.

Testing is conducted with the following characteristics of this SAP ERP workload:

• 850 to 1,800 concurrent users are simulated with scripts.

• One or two application servers are utilized simultaneously for testing.

• One SAP instance (also referred to as an application server) exists on each physical server.

The total response time for various numbers of concurrent users is directly measured via a script.

8
SAP and server tuning
The first step to improve and optimize the SAP ERP workload is to determine the SAP parameters
and settings that affect the workload’s performance. This section describes the most important
findings regarding the tuning of SAP and the SAP application servers.

Tuning SAP memory parameters

Unlike operating systems, which define virtual memory as memory available to the OS other
than physical RAM, SAP defines virtual memory as the total pool of memory available to SAP,
regardless of its underlying source. The size of SAP virtual memory is equal to the size of all the
physical memory, plus the operating system page file space (on local disk), allocated to the SAP
instance by the SAP server’s operating system.

SAP virtual memory is further broken down into two sections: shared memory and local memory
(see Figure 3). Shared memory is used by all the work processes of an SAP instance. It is
allocated at instance startup. Local memory is allocated for specific work processes. It is not
shared by all work processes—it is dedicated and allocated exclusively to each individual work
process for that work process’s sole use.

SAP Note 88416, entitled Zero administration memory management as of 4.0A/Windows, details
many appropriate settings for SAP memory parameters on a 64-bit Windows server.

Tuning SAP buffer parameters

SAP buffers are part of the SAP shared memory pool that contain all users’ global objects and
SAP work processes. Figure 3 provides an overview of the SAP memory architecture.

Figure 3. Overview of SAP memory structure

All SAP buffers have associated parameters that require tuning to achieve the best response time.
By using SAP transaction ST02, it is easy to determine which parameters have an impact on the
workload’s performance. Transaction ST02 displays many SAP buffers characteristics.

Figure 4 shows a screenshot of a transaction ST02 window with the SAP buffer parameters
set correctly for best performance.

9
Figure 4. Screenshot of SAP transaction ST02 with SAP buffers tuned correctly for no swaps

Transaction ST02 is run to verify there are no non-zero entries in the Swaps column of data. A
zero entry indicates an SAP instance is not paging to disk, which is the optimal situation because
transfer speeds between SAP and disk are much slower than those between SAP and physical
memory. Consequently, if there are non-zero values in the Swaps column, the SAP instance is
not optimized for workload performance. Most likely, the problem is that either an SAP buffer’s
memory space is almost full, or the buffer has used all of its available free directories. Either of
these situations will negatively impact workload performance.

To test the effect of tuning SAP buffer parameters, a baseline test is performed using the following
attributes:

• One application server

• The default out-of-the-box SAP buffer parameter settings

• The memory parameter settings recommended by SAP Note 88416

After the baseline test is run, transaction ST02 is used to determine which SAP buffer parameters
need tuning. The parameters that require tuning to resolve a paging-to-disk problem (displayed
non-zero values in the Swaps column) are shown in Figure 5.

10
Figure 5. Screenshot of SAP transaction ST02 after baseline test

As Figure 5 shows, performance-degrading memory problems exist, which are highlighted in red
on the screen. The program buffer, Generic Key buffer, and Export/import buffer are all swapping
to disk due to a critical shortage in their number of free directories, free space, or both.

The program buffer, Generic Key buffer, and Export/import buffer must be tuned so they have
no swaps to disk and have ample free directories and space available in memory. The tuning is
performed in SAP transaction RZ10. Five parameters need to be tuned in order to achieve optimal
performance with this SAP ERP workload for up to 1800 concurrent users:

• Size of program buffer: abap/buffersize=500000

• Maximum number of generic key buffered objects: zcsa/db_max_buftab=80000

• Size of generic key table buffer: zcsa/table_buffer_area=120000000

• Maximum number of export/import buffered objects: rsdb/obj/max_objects=40000

• Size of export/import buffer: rsdb/obj/buffersize=81920

Using these values for the buffer parameters ensure that no swapping to disk occurs.

Improvement in total response time resulting from tuning the SAP buffer parameters is shown
in Figure 6. The results are clear:

• Tuning the appropriate SAP buffer parameters to prevent swapping to disk significantly
improves response time.

11
• Tuning the buffer settings provides a 67% reduction in response time over the default buffer
settings.

• Tuning the buffers lowers the processor utilization of the SAP application server.

Figure 6. Comparison of response times based on SAP buffer settings

Managing the total number of users

One of the first goals of this document is to find an appropriate number of concurrent users an
application server can service. Different numbers of users are tested using one application server;
the results are presented in Figure 7.

Figure 7. Comparison of response times based on number of users

12
Up to 900 users can use one application server. Figure 7 shows that both response time and CPU
utilization on the server increase with the number of concurrent users on the system. When
950 users are introduced into the system, however, the server’s user response time increases
dramatically.

At the 900 user level, the overall processor utilization for the application server is 78%. At that
point, it appears the application server has additional capacity for handling more users. However,
the search for additional user capacity comes at a steep price in terms of response time. Running
950 users provides significantly worse response time compared to 850 to 900 users.

The reason that 900 users are accommodated and 950 are not is because the server’s CPU core
0 saturates at over 99% utilization. With 900 users, the 8-core system has an overall utilization
of 78%, but CPU core 0 is at 91% utilization (not shown). With this SAP ERP workload, CPU
core 0 always has the highest percent utilization. The remaining seven CPU cores have nearly
equal processor utilization. The SAP application utilizes CPU core 0 more than the other 7 CPU
cores for this workload.

Best practice
Limit the number of users to 900 per application server with this server configuration. This allows excellent overall CPU
utilization without handicapping the SAP system’s response time by saturating CPU core 0 of the 8-processor application
server.

Managing the number of total work processes

Each SAP instance can be allocated with a maximum of 100 work processes. However, in order
to achieve the best response time, each SAP instance used for an SAP ERP workload must
manage the total number of Dialog, Update1, and Update2 work processes. The more work
processes dedicated to an SAP instance, the more physical memory the SAP instance needs
to manage those work processes.

SAP online help suggests that five work processes per CPU core is the optimal work process
distribution for an SAP instance. A 40-process configuration tests this suggested guideline by
maintaining the recommended 5:1 ratio of SAP work processes to CPU cores.

Testing also sought to determine whether increasing the number of available SAP instance work
processes would improve user response time.

The number of work processes per SAP instance is set using SAP transaction RZ10 and is
monitored with transaction SM50. Figure 8 compares the total response times for nine different
total work process configurations per SAP instance and shows that increasing the number of work
processes per SAP instance (application server) does not necessarily yield a better response time.
In fact, the configuration with the most work processes has the worst total response time.

13
Figure 8. Comparison of response times based on total work processes per SAP instance

The 40-process case demonstrates nearly 40% improvement in total response time compared
to the worst performing cases. The result is an SAP environment that can handle 40% more
user transactions during a given time frame.

Figure 9 shows an increase in the memory needed for allocating more total work processes. The
90-process case uses the most memory and provides the worst total response time.

The results of this testing clearly indicate that the recommendation to maintain a 5:1 ratio of SAP
work processes to CPU cores in a SAP configuration is a valid one. The 40-process case allocates
five work processes per CPU core and produces the best overall performance.

Figure 9. Comparison of memory used based on total work processes per SAP instance

14
Server sizing to accommodate more than 900 concurrent users

By adding more application servers to process the SAP ERP workload, you increase the number
of processor cores available to perform the workload and maximize the possible number of
concurrent users on the SAP system. To avoid serious impacts to the response time of the SAP
system, HP recommends you add another application server to the SAP landscape for every
900 additional concurrent users. Figure 10 shows the impact on user response time when the
number of application servers is increased from one to two, and the user load is increased from
900 concurrent users to 1,800 users.

Figure 10. Comparison of response times using two application servers versus one application server with 900 users per
application server

In both test cases, the processor utilization of each application server is the same.

The response time of each SAP application server does not scale linearly with user load, as
shown with the 70 ms response time with one application server versus the 88 ms response
time with two application servers.

Using the database server as an SAP application server

With SAP ERP 2005, it is easy to set up the SAP system to use the database server as an SAP
application server. However, the SAP instance has to share the CPU and memory resources of
the database server with the SQL Server 2005 database managed by the server. There are some
situations when using the database server as an SAP application server is feasible, as long as you
understand the impact that the SAP instance presents to the database server.

Figure 11 compares the SAP response time and processor utilization on the database server to a
dedicated SAP application server.

15
Figure 11. Comparison of response times for an SAP instance on a dedicated application server and the database server

Figure 11 shows that the database server is able to also serve as an SAP application server, but
does not provide as good a response time as a dedicated SAP application server. The main issue
is that the SAP instance and the database have to share CPU resources, which become limited
under user load. If the database server is used as an SAP application server, the ability to add
more concurrent SAP users to the SAP system is severely limited. Adding more users means that
the database needs more dedicated CPU resources. Adding more concurrent users to an SAP
system increases the processor utilization of the database server.

Best practice
For optimal scalability and performance, place SAP instances only on dedicated SAP application servers.

Storage tuning
To optimize the SAP ERP workload, it is necessary to determine the EVA8100 settings that have
the greatest impact on workload performance. This section describes findings for tuning the
EVA8100 storage array for this SAP ERP workload.

Overall, the EVA8100 array easily handled this workload. The EVA controller utilization is 16%
to 25%, (as shown in Figure 12), which indicates the EVA’s processors are not stressed by this
storage workload.

16
Figure 12. Comparison of EVA processor utilization based on number of users and disks

An SAP ERP workload generates a significant number of IOPS that are proportional to the number
of concurrent users the SAP system is servicing.

HP and SAP have conducted research to correlate the number of SAPS an SAP system can
handle, the CPU utilization of the SAP system, and the number of IOPS that system produces.

For this c-Class blade system, approximately 6,000 IOPS are generated by an application server
servicing 900 concurrent users. When an additional application server is added, servicing an
additional 900 users (1,800 total users accommodated by the SAP system), the system generates
about 8,500 IOPS.

Due to the OLTP nature of the SAP ERP workload and with its predominantly 8 KB I/O size, the
prime concern for sizing the storage system is accommodating IOPS. If you add more concurrent
users (even when adding more application servers), you must also increase the storage resources
dedicated to the SAP system to meet the demands for IOPS with the increased user load.

As with many enterprise applications, the storage system response time to IOPS from the SAP
system should be less than 20 ms. Microsoft characterizes I/O response time to the SQL Server
2005 database as follows:

• A storage system response time of 10-20 ms to SAPDATA files is acceptable.

• A storage system response time of 20-30 ms to SAPDATA files is not acceptable.

• A storage system response time of above 30 ms to SAPDATA files is cause for alarm and
should trigger an investigation.

Testing shows that storage system latency has a direct impact on the response time of the SAP
system to the user. Every effort to improve storage system latency needs to be made with this
SAP ERP workload. Methods for improving the storage system latency are described in the
following sections.

17
Impact of customer data size

For this SAP ERP workload, the database accesses primarily go to the database tables that
contain the customer data being referenced. This is expected behavior, because this SAP ERP
workload has the primary characteristics of an SAP Sales and Distribution workload.

Two different customer data sizes are tested: 300 GB and 750 GB. A comparison of the storage
performance is shown in Figure 13 and Figure 14.

Figure 13. Comparison of storage system latencies based on customer data size using Vraid 5

Figure 14. Comparison of storage system latencies based on customer data size using Vraid 1

Figure 13 and Figure 14 demonstrate that the amount of customer data impacts the response
time of the storage system using Vraid 5 or Vraid 1. A smaller amount of data accessed results in

18
better storage system response time because the seek time need to access a smaller amount of
data on a disk drive is less than the seek time needed for a larger amount of data.

Seek time is defined as the time it takes to read or write data on a particular location of a disk
drive, including the time the read/write head of the disk needs to physically move to the correct
location. If the disk is less occupied with frequently accessed data, the disk drive has to seek
across a smaller number of disk sectors and tracks to read or write the needed data blocks. For
the disk drives used during this testing, the seek time of the disk drive comprises approximately
60% of the overall response time (per the disk drive manufacturer’s specifications). There is a
shorter response time when seeking across the smaller number of disk sectors for 300 GB of
customer data than for 750 GB of data.

Examining Figure 13, a 40-disk SAPDATA disk group with virtual disks using Vraid 5
accommodates 300 GB of accessed customer data at acceptable storage latency. However, if
the customer data size is 750 GB, either Vraid 1 has to be used—or more disks need to be
dedicated to the SAPDATA disk group.

With an 80% read workload (4:1 read/write IOPS ratio) from the database server to the storage
array with 900 concurrent SAP users, the 6,000 host IOPS translate into 7,200 disk IOPS to the
disk drives utilizing Vraid 1, or 180 (7,200/40) disk IOPS per disk drive. In Vraid 5, the same 6,000
host IOPS result in 9,600 disk IOPS being serviced by the disk drives, or 240 (9,600/40) disk
IOPS per disk drive.

Each disk drive in the tested EVA8100 services an average of 164 IOPS (158 write IOPS or 172
read IOPS) at a disk latency of 15 ms. These IOPS numbers, which are published by the disk
drive manufacturer, assume an average seek-time across all possible sectors and tracks of the
disk drive. Under these assumptions, a disk group of 40 disk drives services 6,560 (40 x164) disk
IOPS at a 15-ms latency. Figure 13 and Figure 14 demonstrate that it is possible for a disk drive to
handle more than 164 IOPS with 15 ms latency when the disk drive does not need to seek across
all its sectors and tracks for the customer data.

While an SAP ERP database consists of more than just customer data, the test results confirm that
knowing the access pattern of your SAP ERP application to the underlying database can provide
better storage response time than calculations based on the disk manufacturer’s performance data.

Managing the number of disk drives in the SAPDATA disk group

Adding more disk drives to a disk group improves the storage system latency of that disk group’s
virtual disks if the IOPS capacity of the storage system is already under some stress. For the SAP
application, if the disk group containing the SAPDATA virtual disks has a storage latency of greater
than 20 ms, the number of disk drives in that disk group might need to be increased.

The effects of adding eight disks to the disk group containing the SAPDATA virtual disks are
presented in Figure 15 and Figure 16.

19
Figure 15. Comparison of the number of disks in the SAPDATA disk group using Vraid 5

Figure 16. Comparison of the number of disks in the SAPDATA disk group using Vraid 1

Figure 15 and Figure 16 demonstrate a significant storage system latency improvement by using
40 disks for the SAPDATA disk group instead of 32 disks for both Vraid 5 and Vraid 1. The reason
is that the capacity for handling IOPS at a given latency is 25% better using 40 disks than 32
disks. With 40 disks dedicated to the SAPDATA disk group, the storage system provides excellent
performance with 900 concurrent SAP users regardless of the chosen Vraid type.

Managing the number of EVA disk groups

It is critically important to size the disk group containing the SAPDATA virtual disks appropriately
to achieve desired storage system performance.

20
Best practice
Place virtual disks containing SAPDATA files in a group other than the disk group where the database transaction logs
reside to maximize the availability of SAP and its underlying SQL 2005 database.

Due to the pronounced effect the number of disks in the SAPDATA disk group has on storage
latency, special care must be taken when allocating disk drive resources to the SAP system.

For example, if an SAP administrator wants to accommodate 900 concurrent SAP users, the
administrator requests 40 disk drives to handle the I/O load of the SAPDATA files. Non-optimized
storage performance for 900 SAP users occurs if the storage administrator allocates 40 total
disks—32 disks for the SAPDATA disk group and 8 disks for the transaction logs.

If the SAP administrator requires a system capable of handling 900 SAP users, the administrator
must allocate 48 total disks—40 disks for the SAPDATA disk group and 8 disks for the transaction
log disk group.

Best practice
Size first for IOPS requirements of the SAPDATA disk group, and then allocate 8 disks for the transaction log disk group.
Do not allocate disks for the transaction log disk group by removing disks from the SAPDATA disk group.

Selecting the Vraid type for SAPDATA virtual disks

For random I/O storage workloads that consist of less than 100% read IOPS, Vraid 1 provides
better storage system performance than Vraid 5.

Any degree of performance improvement between Vraid 1 and Vraid 5 directly depends on the
percentage of read IOPS versus the percentage of write IOPS that the workload exhibits. The
greater the percentage of write IOPS, the greater the storage performance improvement by
switching from Vraid 5 to Vraid 1. Heavy write-I/O workloads benefit the most from Vraid 1.

This SAP ERP workload, with predominantly OLTP characteristics, is a read-I/O dominated
workload. However, SAP ERP workload does not exclusively consist of read IOPS, so the potential
benefits of switching from Vraid 5 to Vraid 1 for the SAPDATA disk group must be examined.

Figure 17 compares the disk latencies of Vraid 1 and Vraid 5 when there are 40 disk drives in
the disk group containing the SAPDATA files. The outcome indicates that for both read and write
latencies, switching from Vraid 5 to Vraid 1 results in an improvement in both read and write
latencies. However, the results also show that whether you use Vraid 5 or Vraid 1, the subsequent
read and write latencies indicate that the storage system is performing at a fully acceptable level
for this SAP ERP workload.

21
Figure 17. Comparison of Vraid types using the same number of disk drives

Given the same number of disk drives, Vraid 1 consistently provides significant (40% or more)
improvement in storage system latency. Acceptable storage latencies are achieved for both
Vraid 1 and Vraid 5 with 40 disks. The decision to use Vraid 1 over Vraid 5 is first based on
achieving a storage system response time goal. If a storage system response time is less than
20 ms using Vraid 5, Vraid 5 is a fully acceptable solution for that SAP system. While Vraid
1 provides even better performance than Vraid 5 for this workload, the performance gains must
be weighed against the extra raw storage capacity needed to accommodate a 1 TB SAP ERP
database. With Vraid 5, only 1.25 TB of raw storage is needed for the database, while in Vraid
1, 2 TB of raw storage is needed.

Selecting the Vraid type of virtual disk for the transaction log

The storage activity to the transaction log virtual disk always consists of 100% write IOPS.
Because the storage activity is 100% write IOPS, the number of disk IOPS serviced by a
transaction log virtual disk using Vraid 1 is half the number of the same virtual disk using Vraid 5.
Therefore, the potential for reaching an IOPS bottleneck using Vraid 1 for the disk group is half
that of using Vraid 5. Another reason to choose Vraid 1 is because Vraid 1 provides better data
redundancy for the critical database transaction logs.

This testing compares the use of Vraid 1 with Vraid 5 as the transaction log virtual disk to
determine which configuration has the more favorable impact on total response time. The result
is that there is no difference in the total user response time, whether you use Vraid 1 or Vraid
5 for the database transaction log virtual disk. The Vraid type of the transaction log virtual disk
has no impact on the user response time of the SAP system.

The EVA8100 is not stressed by the transaction log activity regardless of Vraid type. Vraid 1 is the
recommended redundancy for the virtual disk containing the transaction logs.

22
Storage sizing to accommodate more than 900 concurrent users

HP recommends you add another application server to this SAP landscape for every 900
additional concurrent users that need to be serviced. The addition of 900 more concurrent users
has a direct impact on the number of IOPS the storage system needs to service. For an 1,800
concurrent user SAP system, approximately 8,500 IOPS need to be provided by the storage
system at an acceptable latency of less than 20 ms.

Using 40 disks in the SAPDATA disk group, Vraid 1 is able to provide a read latency of 20 ms
and a write latency of 7 ms with 300 GB of customer data. Vraid 5 is not capable of providing
acceptable storage system latencies with only 40 disks. More than 40 disks are needed to achieve
acceptable storage system performance with 1,800 concurrent SAP users with Vraid 5 or with a
customer data size of over 300 GB.

23
Best practices
The following sections outline the best practices deduced from this testing for SAP, server, and
storage administrators.

SAP administrators
• Tune the SAP buffers until no swaps are observed with SAP transaction ST02; that is, until
there is no paging to disks. The tuned buffer settings provide a 67% decrease in response
time over the default buffer settings.

• Use a total of 40 SAP work processes on each application server for best performance. The
40-process case demonstrates a nearly 40% improvement in total response time as compared
to the worst performing cases. Maintain a 5:1 ratio of SAP work processes to CPU cores
on each application server.

• See SAP Note 88416, entitled Zero administration memory management as of 4.0A/Windows.
This SAP note details many appropriate settings for SAP memory parameters on a Windows
server.

Server administrators
• Have no more than 900 concurrent users on a dedicated SAP application server.

• Avoid using the database server as an SAP application server.

• Avoid the situation where CPU core 0 saturates on an SAP application server, by targeting an
overall processor utilization of 80% across all CPU cores on each SAP application server.

Storage administrators
• Size the storage-based-on-IOPS needs first for an SAP ERP workload, and then size
everything else. Due to the I/O size of the SAP ERP workload, disk drive IOPS are the
resource in greatest demand.

• Size all disk groups with multiples of eight disk drives. This is an EVA best practice that
ensures the internal administration of the disk groups is optimized.

• Know that the size of the customer data tables in the SAPDATA files directly impacts the
storage system’s expected latency.

• Ensure the SAPDATA disk group has enough disk drives to accommodate the IOPS
requirements of the SAPDATA files. For one application server serving 900 concurrent users,
a minimum of 32 disk drives are required. For two application servers serving 1800 concurrent
users, a minimum of 40 disk drives is required.

• Know that Vraid 5 is fully acceptable for SAPDATA virtual disks under most circumstances.
Vraid 1 provides better performance for an SAP ERP workload. However, only switch to Vraid
1 if using Vraid 5 does not provide a storage system latency of less than 20 ms.

• Use Vraid 1 for the transaction logs because of its 100% write I/O workload, greater disk
redundancy, and improved fault tolerance.

24
• Deploy the two-disk group configuration only after you have allocated enough disk drives to
the SAPDATA disk group. Do not remove drives from the SAPDATA disk group in order to
create a separate disk group for the database transaction logs.

• Isolate the transaction logs into their own disk group to improve overall availability of the
storage solution.

Issues
SAP Note 88416, Zero administration memory management as of 4.0A/Windows, details many
appropriate settings for SAP memory parameters on a Windows server. The primary memory
setting is PHYS_MEMSIZE, which establishes many other SAP memory settings on a Windows
server. During testing, the default PHYS_MEMSIZE setting is 512 MB on SAP system after
completing installation. However, each SAP application server has almost 32 GB of memory
available for use by the SAP instance. With tests running over 300 concurrent users, the SAP
instance crashes and must be restarted. The PHYS_MEMSIZE setting must be set to match
the amount of memory that an administrator is willing to allocate on a Windows server to an
SAP instance.

If the PHYS_MEMSIZE memory parameter is not set appropriately, the memory in the application
server is not used appropriately by the SAP instance, and the SAP application might crash under a
sustained user load.

25
Conclusion
The test results and related information clearly demonstrate how to properly plan for, successfully
deploy, and productively use a fully-operational SAP NetWeaver production landscape on HP
server/storage infrastructure. In addition, specific detailed examples illustrate exactly how to use
SAP’s CCMS to monitor key aspects of the environment.

Key planning considerations include:

• Selecting and configuring the servers

Use dedicated blades for each function in the landscape. Avoid using the database server as
an SAP application server.

• Understanding the upper-limit-to-user-loads on the SAP application server

In our tests, response times degraded rapidly beyond about 900 concurrent users. Adding a
second application server blade provided the ability to roughly double the number of supported
concurrent users.

• Configuring the disk array

In our tests, it was necessary to size for IOPS, not capacity. Once the number of disks
was properly sized, the EVA-8100 array proved to be capable of running the test workload
with minimal tuning. Parity-based VRAID 5 is entirely appropriate, though Vraid 1 remains
acceptable if ultra-high performance and availability is imperative. Finally, use the two-disk
group configuration to isolate transaction logs only if there are enough disks allocated to
SAPDATA.

Key software tuning considerations include:

• Using Zero Administration Memory Management in Windows

There are specific SAP notes that provide the appropriate details.

• Tuning the SAP buffers until no swaps (that is, no paging to disks) are observed

• Configuring the appropriate number of SAP work processes

Testing confirmed optimal results using a 5:1 ratio for SAP work processes to CPU cores.

The combination of understanding all major considerations, along with knowing exactly what to
do, are key to the successful deployment of an SAP NetWeaver production landscape using HP
servers, storage, and enterprise management. The test-proven techniques developed here serve
as a complete guide that can be used with confidence to ensure success.

We value your feedback

In order to develop technical materials that address your information needs, we need your
feedback. We appreciate your time and value your opinion. The following link will take you to a
short survey regarding the quality of this paper:

http://hpwebgen.com/Questions.aspx?id=12046&pass=41514

26
Appendix A Bill of Materials

QTY Description Part number

HP BladeSystem c7000 Enclosure

1 HP BLc7000 CTO Enclosure 412152-B21

8 HP BLc7000 Encl Single Fan Option 412140-B21

6 HP BLc7000 Encl Pwr Sply IEC320 Option 412138-B21

1 HP BLc7000 Encl Mgmt Module Option 412142-B21

HP ProLiant BL460c Servers

6 HP ProLiant BL460c G1 CTO Chassis 404667-B21

3 HP X5160 BL460c G1 FIO Kit 416660-L21

3 HP X5160 BL460c/xw460c G1 Kit 416660-B21

3 HP X5355 BL460c FIO Kit 435565-L21

3 HP X5355 BL460C Kit 435565-B21

15 HP 8GB FBD PC2-5300 2x4GB Kit 397415-B21

12 HP 72GB 15k 2.5 Single Port HP SAS Drive 431935-B21

6 HP BLc Emulex LPe1105 FC HBA Opt Kit 403621-B21

6 HP SA 641/642/E200 128MB BBWC Kit 351580-B21

HP universal rack and power distribution unit

1 HP Universal Rack 10642 G2 Shock Rack AF002A

1 HP 10K G2 600W Stabilizer Kit AF062A

1 HP 10642 G2 Sidepanel Kit AF054A

2 HP 40A HV Core Only Corded PDU 252663-D75

EVA storage array

1 HP EVA8000 2C12D-A 60Hz 42U Cabinet AD522B

168 HP StorageWorks 146GB 15K FC HDD 364621-B23

8 Storage Works LC/LC 15m Cable 221692-B23

1 EVA8100 Upgrade Kit

Infrastructure − Network switches

2 HP BLc Cisco 1GbE 3020 Switch Opt Kit 410916-B21

Infrastructure − SAN switches

2 Brocade BladeSystem 4/24 SAN Swt Powr Pk AE371A

27
Appendix B References
Thomas, Juergen. SAP with Microsoft SQL Server 2005: Best Practices for High Availability,
Maximum Performance, and Scalability. SQL Server Technical Article. February, 2007.

28
 For more information
HP technical references
• HP StorageWorks Customer Focused Testing

http://www.hp.com/go/hpcft

• HP SAP Solutions

http://www.hp.com/go/SAP

• HP data storage and HP StorageWorks products

http://www.hp.com/go/storage

• HP Blade servers

http://www.hp.com/go/blades

• HP ProLiant servers

http://www.hp.com/go/proliant

©2008 Hewlett-Packard Development Company, L.P. The information contained

herein is subject to change without notice. The only warranties for HP products
and services are set forth in the express warranty statements accompanying
such products and services. Nothing herein should be construed as constituting
an additional warranty. HP shall not be liable for technical or editorial errors or
omissions contained herein.
Microsoft and Windows are U.S. registered trademarks of Microsoft Corporation.
4AA1-5661ENW, March 2008

29