Professional Documents
Culture Documents
Dell Best Practices For Oversubscription of CPU Memory and Storage in VSphere Virtual Environments - 0
Dell Best Practices For Oversubscription of CPU Memory and Storage in VSphere Virtual Environments - 0
2
Oversubscription maximizes the value In vSphere, a physical CPU (pCPU)
of resources—but introduces risk of the refers to:
host not having enough resources to • When hyperthreading is not present or
service all its VMs. enabled: A single physical CPU core
While virtual machines are generally • When hyperthreading is present and
more right-sized than their physical enabled: A single logical CPU core
counterparts were in the past, there is
still room to grow built in, especially Here are two examples:
when particular workloads are idle. • If a host has two eight core processors and
Many administrators see this as an hyperthreading is either not supported or
opportunity to make use of those idle not enabled, that host has sixteen physical
resources in order to maximize virtual CPUs (8 cores x 2 processors).
machine density on a host. However,
with oversubscription, administrators are
• If a host has two eight core processors
and hyperthreading is enabled, that host
Oversubscription
basically assigning to virtual machines has thirty-two physical CPUs (8 cores x 2 is an opportunity
more resources than are actually processors x 2 threads per core).
available on the host. In other words, to maximize VM
if all of the virtual machines suddenly
requested access to all of their allocated
How those resources are presented to
virtual machines
density on a host.
resources, the host would not have In a virtual machine, processors are But it introduces
enough resources to service the needs. referred to as virtual CPUs (vCPUs).
When an administrator adds vCPUs to a risks: if all of the VMs
Resource oversubscription, while it does
increase virtual machine density, carries
virtual machine, each of those vCPUs is
assigned to a pCPU, although the actual
suddenly request
with it some risks. Once a particular pCPU may not always be the same. access to all of their
resource is finally exhausted, if that There must be enough pCPUs available
resource happens to be oversubscribed, to support the number of vCPUs allocated resources,
stability issues can occur and major
performance problems can be
assigned to an individual virtual machine
or that virtual machine will not boot.
the host will not
introduced affecting all of the workloads have enough
running on the host server. However, that doesn’t mean that
administrators are limited to just the resources to service
Before we discuss more about
oversubscription, it’s important to
number of pCPUs in the host. On the
contrary, there is no 1:1 ratio between
the needs.
understand vSphere manages the three the number of vCPUs that can be
basic types of resources: assigned to virtual machines and the
• Processing resources number of physical CPUs in the host.
• Memory resources In fact, as of vSphere 5.0, there is a
• Storage resources maximum of 25 vCPUs per physical core,
and administrators can allocate up to
How vSphere manages 2,048 vCPUs to virtual machines on a
processing resources single host.
How physical resources are represented
on a vSphere host How vSphere manages
In vSphere, administrators assign CPUs memory resources
to virtual machines in order to support vSphere uses a number of techniques
the workload needs of each individual to maximize the use of RAM in a
virtual machine. These virtual processing virtual environment:
resources are pulled from the host’s • Transparent page sharing (TPS)—In most
available physical CPUs. The number of virtual environments, administrators run
physical CPUs that are present in hosts is many copies of the same operating system.
dependent on a couple factors. In these cases, there is a lot of duplication
of memory pages in host memory.
Transparent page sharing is basically a
3
form of memory deduplication—vSphere • Swapping to disk—Swapping to disk
combines multiple identical memory pages is the hypervisor’s last-ditch effort to
into just one and frees the remaining retrieve enough physical RAM to satisfy
pages up for other uses. TPS has an almost the needs of workloads running on a
imperceptible impact on the performance host. Swapping is a process by which the
of the host. hypervisor moves the least used memory
• Memory ballooning—When the VMware pages to disk. Those memory pages are
Tools are installed inside a guest virtual still accessible, but when they are required,
machine, a memory balloon driver is they must be retrieved from disk. Swapping
installed as well. This driver acts as a will noticeably degrade the overall
Windows process, and the OS can use its performance of the host.
normal memory management techniques
to assign idle or unused memory pages to Because vSphere’s other memory
4
is allocated. After all, it’s reasonable acceptable and often recommended
to expect that the virtual machine will method for managing storage capacity.
continue to need additional disk space
as time goes on. Some storage devices have additional
features, such as data compression and
Thin provisioning operates as follows: deduplication, that enable additional
When an administrator provisions the levels of oversubscription. For the
total disk space for the virtual machine, purposes of this paper, however, the
the virtual machine is told that it has focus is on the hypervisor, so only thin
access to the entirety of the allocated provisioning will be discussed.
space. In reality, however, vSphere gives
the virtual machine only the space that it Getting insight into resource usage
is actually consuming. For instance, if an in your environment
administrator allocates 200 GB to a new Now that we have see how resources Swapping to disk
virtual machine, but that virtual machine are managed in a vSphere environment,
is using only 40 GB, the remaining let’s move on to oversubscribing is the hypervisor’s
160 GB remain available for allocation
to other virtual machines. As a virtual
those resources. In order to determine
whether resources are overcommitted,
last-ditch effort to
machine requires more space, vSphere the administrator needs a monitoring retrieve enough
provides additional chunks to that virtual tool such as the free vOPS™ Server
machine, up to the size of the disk that Explorer tool from Dell®. One of its physical RAM to
was originally allocated. several utilities is Environment Explorer,
which provides administrators with a
satisfy the needs of
By using thin provisioning, administrators high-level view of resource usage in workloads running
can create virtual machines with the environment. As shown in Figure 1,
virtual disks of a size that is necessary Environment Explorer shows resource on a host. Swapping
in the long term, without having to
immediately commit the total disk
utilization as a percentage of actual
physical resources, making the utility
will noticeably
space that is necessary to support that a perfect fit for exploring the topic of degrade the overall
allocation. In many tests, it has been resource over-commitment.
shown that thin provisioning carries with performance of
it only a very slight—almost negligible—
performance impact. Accordingly, thin
Now let’s explore some guidelines for
oversubscribing each of the three types
the host.
provisioning has become a common, of resources: processing, memory
and storage.
5
Oversubscribing processing Metrics to watch
resources The following metrics will help you
The common wisdom maintain a vCPU to pCPU ratio that
As mentioned earlier, in vSphere 5, every makes efficient use of resources while
physical processor core can support still allowing workloads to run well:
up to 25 vCPUs. However, for every • Inside virtual machines
additional workload beyond a 1:1 vCPU • CPU Utilization—When average CPU
to pCPU ratio, the vSphere hypervisor usage remains high, it’s time to add an
needs to invoke processor scheduling additional vCPU to the VM.
in order to distribute processor time • On the host:
to virtual machines that need it. For • CPU Ready—CPU Ready is, by far, the
example, if an administrator has created most important gauge of overall host
a vCPU to pCPU ratio of 5:1, then each health standpoint with regard to CPU.
By using thin processor is supporting five vCPUs. CPU Ready indicates the length of time
that a VM is waiting for enough physical
provisioning, Experts in the field disagree about the processors to become available in order
administrators can proper rules of thumb when it comes
to vCPU to pCPU ratio. However, there
to meet its demands. For example, if a
VM is allocated four vCPUs, this metric
create VMs with are two items on which just about will show the length of time that the VM
everyone agrees: waited for four corresponding pCPUs to
virtual disks of a size • Start with one vCPU per virtual machine. become available at the same time.
that is necessary Most experts agree that administrators • CPU Utilization—The overall CPU
should create new virtual machines with usage on the host server is also critical
in the long term, just one vCPU and add virtual vCPUs because it enables an administrator to
as needs dictate. As vCPUs are added, understand just how much work the
without having to the virtual machine is tied to requiring host server is doing.
immediately commit processor time from the host. Whenever
the virtual machine needs to perform an Real-world observations and advice
the total disk space operation, it has to wait for a number of
physical CPUs equal to the number of Virtual resources
that is necessary assigned vCPUs to be available. So, as 4 vCPUs - 200 % of Actual cores
to support that administrators add more vCPUs to a virtual 3 GB Memory - 37 % of Physical
26.6 GB Storage - 13 % of Provisioned
machine, there is an increased risk of
allocation. poorer overall performance.
Figure 2. A lab with a vCPU to pCPU
• The vCPU to pCPU ration is workload
ratio of 2;1
dependent. While 1:1 vCPU to pCPU
assignment is sometimes advocated, other
Online forums are filled with questions
ratios are common. Although vSphere
from users requesting insight into
5 supports a ratio of up to 25:1, your
acceptable vCPU to pCPU ratios in a
ability to achieve a high ratio will depend
real-world environment. While some
on the kinds of workloads you need to
responses continue to advocate for a
support. If the host is supporting lots of
1:1 ratio, from a pure density standpoint,
virtual machines, each with only meager
1:1 should be considered a worst-case
processing needs, the vCPU to pCPU ratio
scenario. Figure 2 shows a lab with a
could be quite high. If, however, the host
ratio of 2:1.
is running a number of processor intensive
workloads, the ratio may be much smaller.
6
Some respondents indicate that they In addition, Environment Explorer
have received guidance that suggests identifies where host processor
no more than a 1.5:1 vCPU to pCPU resources are overcommitted, so you’ll
ratio, but guidance from industry experts know where perform additional analysis
suggests that vSphere real-world to determine if that over-commitment
numbers are in the 10:1 to 15:1 range. is causing performance issues. As the
Still others indicate that VMware itself has “% of actual cores” metric begins to
a recommended ratio range of 6:1 to 8:1. surpass 500 percent, carefully monitor
CPU Ready and general workload
The Dell white paper, “Demystifying performance to ensure that business
CPU Ready (% RDY) as a Performance needs are being met.
Metric,” establishes the following
vCPU:pCPU guidelines: Oversubscribing memory resources
• 1:1 to 3:1 is no problem. The common wisdom Environment
• 3:1 to 5:1 may begin to cause Oversubscribing RAM is one of the more
performance degradation. controversial resource oversubscription Explorer (part of
• 6:1 or greater is often going to cause options. Whereas CPU and storage
resources are often overcommitted,
Dell’s free vOPS
a problem.
there seems to be some conservatism Server Explorer
In addition, keeping the CPU Ready when it comes to overcommitting RAM.
metric at 5 percent or below is tool) provides a
considered a best practice. Metrics to watch
On a host server, administrators need to
high-level view of
The actual achievable ratio in a specific watch the amount of RAM actually in use resource usage.
environment will depend on a number by virtual machines. As the actual RAM in
of factors: use approaches 100 percent, either add
• vSphere version—The vSphere CPU additional RAM to the server or migrate
scheduler is always being improved. The workloads to hosts that have more
newer the version of vSphere, the more available RAM.
consolidation that should be possible.
• Processor age—Newer processors are Real-world observations and advice
much more robust than older ones, so In order to maximize VM density and
organizations with newer processors ensure that the environment remains
should be able to achieve higher operational, it’s important to monitor
processor ratios. the actual memory utilization. That’s
• Workload type—Different kinds of why Environment Explorer displays RAM
workloads on the host will result in different usage (the “% of physical” metric) using
optimal ratios. the amount of RAM actually provisioned
to each virtual machine, rather than
vScope Explorer, another utility included the amount of RAM actually being
in vOPS Server Explorer, can help used by virtual machines once all of
you investigate performance metrics, vSphere’s various memory management
including CPU Ready, at both the host techniques are taken into consideration.
and virtual machine levels, so you can
determine whether your vCPU to pCPU
ratio is too high.
7
The level of over-commitment possible storage volume running out of space,
depends on one primary factor: how the VMs will still think they have available
much memory deduplication can take disk space to use but there won’t be any
place by virtue of the fact that there are space available. This can cause a serious
many similar workloads running on the outages that can result in data loss
host. The greater the level of disparity and costly recovery. So if you use thin
between running workloads, the less provisioning, be sure to monitor carefully.
memory consolidation that can take place
and the less density that can be enjoyed. Metrics to watch
Here are some observations about what To mitigate the risks associated with
others are doing and recommending with thin provisioning, you need to keep a
regard to memory over-commitment: close eye on the amount of free space
8
For More Information
© 2013 Dell, Inc. ALL RIGHTS RESERVED. This document DELL ASSUMES NO LIABILITY WHATSOEVER AND DISCLAIMS
contains proprietary information protected by copyright. No ANY EXPRESS, IMPLIED OR STATUTORY WARRANTY RELATING
part of this document may be reproduced or transmitted in TO ITS PRODUCTS INCLUDING, BUT NOT LIMITED TO, THE
any form or by any means, electronic or mechanical, including IMPLIED WARRANTY OF MERCHANTABILITY, FITNESS FOR
photocopying and recording for any purpose without the A PARTICULAR PURPOSE, OR NON-INFRINGEMENT. IN NO
written permission of Dell, Inc. (“Dell”). EVENT SHALL DELL BE LIABLE FOR ANY DIRECT, INDIRECT,
CONSEQUENTIAL, PUNITIVE, SPECIAL OR INCIDENTAL
Dell, Dell Software, the Dell Software logo and products—as DAMAGES (INCLUDING, WITHOUT LIMITATION, DAMAGES
identified in this document—are registered trademarks of Dell, FOR LOSS OF PROFITS, BUSINESS INTERRUPTION OR LOSS
Inc. in the U.S.A. and/or other countries. All other trademarks OF INFORMATION) ARISING OUT OF THE USE OR INABILITY
and registered trademarks are property of their respective TO USE THIS DOCUMENT, EVEN IF DELL HAS BEEN ADVISED
owners. OF THE POSSIBILITY OF SUCH DAMAGES. Dell makes no
representations or warranties with respect to the accuracy or
The information in this document is provided in connection completeness of the contents of this document and reserves
with Dell products. No license, express or implied, by estoppel the right to make changes to specifications and product
or otherwise, to any intellectual property right is granted by descriptions at any time without notice. Dell does not make
this document or in connection with the sale of Dell products. any commitment to update the information contained in this
EXCEPT AS SET FORTH IN DELL’S TERMS AND CONDITIONS AS document.
SPECIFIED IN THE LICENSE AGREEMENT FOR THIS PRODUCT,
About Dell
Dell Inc. (NASDAQ: DELL) listens to customers and delivers
worldwide innovative technology, business solutions and
services they trust and value. For more information,
visit www.dell.com.
Dell Software
5 Polaris Way
Aliso Viejo, CA 92656
www.dell.com
Refer to our Web site for regional and international
office information.
Whitepaper-vSphere-Environ-US-KS-2013-03-28