Dell EMC PowerEdge Concepts and Features-Downloadable Content

DELL EMC POWEREDGE
CONCEPTS AND
FEATURES
DOWNLOADABLE CONTENT
DOWNLOADABLE CONTENT
Dell EMC PowerEdge Concepts and Features-Downloadable Content
© Copyright 2021 Dell Inc. Page i

Table of Contents
Dell EMC PowerEdge Concepts and Features..................................................................... 7
Portfolio Overview .................................................................................... 8
Portfolio Overview ................................................................................................................ 9

Introduction to Server Portfolio ............................................................................................. 9
Dell EMC PowerEdge Naming Convention .......................................................................... 9
Service Tag and Asset Tag ................................................................................................ 11
PowerEdge 13G Servers ................................................................................................... 11
PowerEdge 13G Specifications .......................................................................................... 14
PowerEdge 13G Control Panel .......................................................................................... 16
PowerEdge 14G Control Panel .......................................................................................... 22
Server Components ................................................................................ 28
Server Components ........................................................................................................... 29

Introduction ........................................................................................................................ 29
Processors ............................................................................................................... 32
Processors ......................................................................................................................... 32
Intel Xeon Scalable Processor Family ................................................................................ 32
AMD Processors ................................................................................................................ 35
PowerEdge 15G Processors .............................................................................................. 36
Processor Settings ............................................................................................................. 37
Disassembly and Assembly of Processors ......................................................................... 38
Memory ..................................................................................................................... 40
Memory.............................................................................................................................. 40
Server Memory Module ...................................................................................................... 40
Page ii © Copyright 2021 Dell Inc.

Memory Comparison.......................................................................................................... 40
CPU and Memory Architecture Comparison....................................................................... 42
15G CPU and Memory Architecture ................................................................................... 42
Memory Layout .................................................................................................................. 44
Memory Settings ................................................................................................................ 48
Memory Modes .................................................................................................................. 53
Memory Spare Mode ......................................................................................................... 56
Intel Persistent Memory ..................................................................................................... 57
Intel Persistent Memory DIMM Layout ............................................................................... 58
Barlow Pass Advantages ................................................................................................... 59
Industry Mixed Memory Combinations ............................................................................... 59
Memory Population Configurations with Barlow Pass ........................................................ 60
Operational Modes............................................................................................................. 61
Memory Mode and Operational Mode Differences ............................................................. 66
Removal and Installation of DIMMs .................................................................................... 66
Power ........................................................................................................................ 68
Power ................................................................................................................................ 68
Power Supply Unit ............................................................................................................. 68
PSU Configuration Modes.................................................................................................. 69
PSU Redundancy Types .................................................................................................... 72
Power Capping .................................................................................................................. 73
Mixing Power Supplies ....................................................................................................... 74
PSU LED Indicator Behavior .............................................................................................. 75
PSU Firmware Updates ..................................................................................................... 76
PSU Blanks........................................................................................................................ 77
Removal and Installation of PSUs ...................................................................................... 78
Cooling...................................................................................................................... 79
Cooling .............................................................................................................................. 79
Fans and Types of Fans .................................................................................................... 79
Types of Fans .................................................................................................................... 80
PowerEdge 15G Fans ........................................................................................................ 84
© Copyright 2020 Dell Inc. Page iii

Removal and Installation of Fan ......................................................................................... 84
Heatsinks ........................................................................................................................... 85
Memory DIMM Blank and GPU Air Shroud ........................................................................ 88
GPU Heatsinks and Fans................................................................................................... 90
Dell EMC Multi Vector Cooling ........................................................................................... 90
Server Exhaust Temperature ............................................................................................. 93
HVAC - Heating, Ventilation, and Air Conditioning ............................................................. 94
Dell EMC PowerEdge Liquid Cooling ................................................................................. 95
Direct Liquid Cooling Technology ....................................................................................... 97
New Features and Solutions .............................................................................................. 98
Liquid Cooling Setup ........................................................................................................ 101
Benefits of Liquid Cooling ................................................................................................ 104
Fresh Air Environment ..................................................................................................... 105
Fresh Air Environment Deployment Restrictions .............................................................. 105
Networking ............................................................................................................. 108

Networking ....................................................................................................................... 108
Network Daughter Card ................................................................................................... 108
Removal and Installation of an NDC in a PowerEdge R640 Server .................................. 109
OCP Card ........................................................................................................................ 110
Removal and Installation of an OCP Card in a PowerEdge R650 Server ......................... 111
SNAP I/O ......................................................................................................................... 112
SNAP I/O Cards for PowerEdge Servers ......................................................................... 113
SNAPI .............................................................................................................................. 115
Procedure to Replace the SNAPI Riser in a Dell EMC PowerEdge C6520 Sled Within a
PowerEdge C6400 Chassis. ............................................................................................ 116
Rear Input Output and LOM ............................................................................................. 116
Removal and Installation of a LOM Card in the PowerEdge R650 Server ........................ 119
Removal and Installation of a Rear I/O Card in the PowerEdge R650 Server .................. 119
Graphic Processing Units (GPUs) ........................................................................ 120

Graphic Processing Units (GPUs) .................................................................................... 120
Accelerator Cards ............................................................................................................ 120
PowerEdge Server Accelerator Support ........................................................................... 123
Page iv © Copyright 2021 Dell Inc.

Use Cases ....................................................................................................................... 124
Compute Unified Device Architecture............................................................................... 126
CUDA Installation Instructions.......................................................................................... 127
System Board Inlet Temperature ..................................................................................... 127
GPU Riser Module Removal - Video ................................................................................ 128
Server Upgrade - GPU Kit ................................................................................................ 129
Expansion Card ...................................................................................................... 130

Expansion Card ............................................................................................................... 130
PCIe Overview ................................................................................................................. 130
PCIe Slots and Processors .............................................................................................. 131
PCIe Server Peripherals .................................................................................................. 132
PCIe Card Form Factor .................................................................................................... 133
Risers .............................................................................................................................. 134
Storage.................................................................................................................... 135
Storage ............................................................................................................................ 135
Introduction to Server Storage ......................................................................................... 135
NVMe............................................................................................................................... 136
Paddle Card - 15G Only ................................................................................................... 137
Removal and Installation of the Paddle Card ................................................................... 138
Internal Dual SD Module .................................................................................................. 139
Removal and Installation of IDSDM ................................................................................. 140
Boot Optimized Storage Solution (BOSS) ........................................................................ 141
RAID ................................................................................................................................ 144
RAID Level Comparison................................................................................................... 157
Hot Spare ........................................................................................................................ 159
Possible RAID Level Migrations ....................................................................................... 160
PERC Overview ............................................................................................................... 163
NVMe Support with PERC 11 .......................................................................................... 179
PERC Configuration Modes ............................................................................................. 180
PERC Card Matrix ........................................................................................................... 183
Removal and Installation of PERC ................................................................................... 185
© Copyright 2020 Dell Inc. Page v

Backplanes ...................................................................................................................... 186
Removal and Installation of Backplane ............................................................................ 187
Introduction to Server Security ........................................................... 188
Introduction to Server Security ......................................................................................... 189

Server Security Overview................................................................................................. 189
Trusted Platform Module .................................................................................................. 191
Trusted Platform Module 2.0 ............................................................................................ 194
Configuring the TPM ........................................................................................................ 195
Silicon-Based Hardware Root of Trust (RoT) ................................................................... 196
BIOS Live Scanning ......................................................................................................... 197
Intel Boot Guard ............................................................................................................... 199
Boot Guard Process......................................................................................................... 201
Automated BIOS Recovery .............................................................................................. 203
Secure Boot ..................................................................................................................... 204
Secure Boot Policy........................................................................................................... 205
Secure Erase for Self Encrypting Disks (SEDs) ............................................................... 206
System Lockdown mode in iDRAC .................................................................................. 207
Dell Technologies Secured Component Verification......................................................... 208
Configuration Validation ...................................................................... 209
Configuration Validation ................................................................................................... 210

Overview .......................................................................................................................... 210
Hardware Configuration Error Messages ......................................................................... 211
HWC8010 and HW8011 Error Interpretations .................................................................. 212
You Have Completed This Content .................................................................................. 215
Appendix ............................................................................................... 217
Glossary ................................................................................................ 231
Page vi © Copyright 2021 Dell Inc.

Processors
Dell EMC PowerEdge Concepts and Features
© Copyright 2021 Dell Inc. Page 7

Portfolio Overview
Page 8 © Copyright 2021 Dell Inc.

Processors
Portfolio Overview
Introduction to Server Portfolio
Movie:
The web version of this content contains a movie.
Click the link below to watch an introduction to the Dell server portfolio.
https://edutube.emc.com/Player.aspx?autoplay=true&vno=ZICcxeiVyyvuY3T8JG72
ng
Dell EMC PowerEdge Naming Convention
Dell EMC PowerEdge servers with common design components are identified by
the server model name.
The server naming convention provides insight into the form factor, class of
system, generation, and the CPU socket count.
Three Number Naming Convention

Processors
Four Number Naming Convention
Important:
• The PowerEdge XE family of servers is purpose-built for
complex, emerging workloads that require high-
performance and large storage. For example, the
PowerEdge XE8545.
• The PowerEdge XR family of servers is ruggedized,
industrial-grade servers intended for extreme
environments. For example, PowerEdge XR11/XR12.
Page 10 © Copyright 2021 Dell Inc

.
Processors
Service Tag and Asset Tag
The Dell EMC service tag is a seven-character identifier that is unique to the
product.
• The service tag of a PowerEdge server is a pullout tab also known as an
Enterprise Service Tag (EST). ESTs are typically located on the front or rear of
the chassis.
• Information about the service tag can also be found on a sticker typically on the
side of the chassis, and in the server BIOS.
All Dell EMC PowerEdge servers have a Service Tag and can have an Asset Tag
added.
The Asset Tag is an empty field within BIOS where you can input your own
identifying information such as the system’s security number or location ID.
Movie:
Important: In addition to the service tag, there is an Express Service

Code which is a 10-digit or more numeric version of the service tag.
The code can be typed into a telephone for call routing.
Click the link below to watch a demo video on locating the service tag in a
PowerEdge server.
https://edutube.emc.com/Player.aspx?autoplay=true&vno=DnD1V|@$@|UUKQ2lRt
WtNqDYMw
PowerEdge 13G Servers

Processors
The PowerEdge 13G servers feature:

• Incredible density across a range of resources enabling highly flexible data
center scaling.
• Intel Xeon processors.
• PowerEdge RAID Controller 9 series cards for server storage expansion.
• Flexible storage options, including in-box storage tiering1.
• Reliability, Availability, and Storage (RAS) characteristics.
Click each tab to view several 13G PowerEdge servers.
PowerEdge R630
PowerEdge R630
1 A tiered storage solution enables organizations to move older, less important, and
infrequently accessed data to less expensive storage solutions, while maintaining
active and important data on high-performing storage media.

.
Processors
PowerEdge R730xd
PowerEdge R730xd
PowerEdge R930
PowerEdge R930
Tip: For more information about the PowerEdge 13G servers, go to

dell.com/support

Processors
PowerEdge 13G Specifications
5 2
1 4
1: Processor: PowerEdge R930 of the 13th generation supports up to x4 Intel Xeon

E7-8800 product family processors.
2: Power Supplies: 1100 W DC power supply
Titanium efficiency 750 W AC power supply
Platinum efficiency 495 W, 750 W, 1100 W AC power supply
3: Expansion and Network: Up to x7 PCIe 3.0 slots, supports a dedicated RAID

card slot and a dedicated NDC slot.
Embedded NIC- 1 GbE x4 (this port is for management only). The NDC is present
in different versions ranging from 4x 1 GB, 2x 1 Gb + 2x 10 GB or 4x 10 GB.
4: Embedded Management: IPMI 2.0 compliant
iDRAC8 with Lifecycle Controller
iDRAC8 Express (default) or Enterprise (upgradable):

• 8 GB vFlash media (optional)
• 16 GB vFlash media (optional)

.
Processors
5: Memory:
Supported in the form of RDIMMs and LRDIMMs.
Up to 3 TB (24 DIMM slots): 4 GB/8 GB/16 GB/32 GB/64 GB/128 GB
DDR4 up to 2400 MT/s
6:
Listed below are the PERC9 cards that are supported in the PowerEdge 13G
systems:
• Internal controllers: PERC S130 (Software RAID), PERC H330, PERC H730,
PERC H730P, HBA330 (no RAID internal HBA)
• External HBAs (RAID): PERC H830
• External HBAs (non-RAID): 12 Gbps SAS HBA
Following are the specifications of PowerEdge 13G servers.
Memory: Supported in the form of RDIMMs and LRDIMMs.
Up to 3 TB (24 DIMM slots): 4 GB/8 GB/16 GB/32 GB/64 GB/128 GB.
DDR4 up to 2400 MT/s.
Storage Controllers: Listed below are the PERC9 cards that are supported in the
PowerEdge 13G systems:
• Internal controllers: PERC S130 (Software RAID), PERC H330, PERC H730,
PERC H730P, HBA330 (no RAID internal HBA)
• External HBAs (RAID): PERC H830
• External HBAs (non-RAID): 12 Gbps SAS HBA
Power Supplies: 1100 W DC power supply
Titanium efficiency 750 W AC power supply
Platinum efficiency 495 W, 750 W, 1100 W AC power supply
Embedded Management: IPMI 2.0 compliant

Processors
iDRAC8 with Lifecycle Controller
iDRAC8 Express (default) or Enterprise (upgradable):
• 8 GB vFlash media (optional) - all systems will get shipped with a vflash but
only systems that have the enterprise license will be able to use them.
• 16 GB vFlash media (optional) - all systems will ship with 8GB but customers
can request the 16GB vflash as an upgrade.
Expansion and Network: Up to x7 PCIe 3.0 slots, supports a dedicated RAID card
slot and a dedicated NDC slot.
Embedded NIC- 1 GbE x4 (his is the dedicated port for the iDRAC) - The NDC is
present in different versions ranging from 4x 1 GB, 2x 1 Gb + 2x 10 GB or 4x 10
GB.
Processor: PowerEdge R930 of the 13th generation supports up to x4 Intel Xeon

E7-8800 product family processors.
PowerEdge 13G Control Panel
PowerEdge T630 Front Control Panel
8
2 5 6
1 3
4 9 10
7
1: Power Button
2: NMI button
3: System Identification Button
4: SD vFlash card slot
5: LCD Menu Buttons
6: LCD Panel

.
Processors
7: Service Tag
8: VGA Connector
9: USB and iDRAC Direct Connector
10: USB Connector
The table below explains components on the front panel of the PowerEdge server.
Part Number Description
1 Power Button
2 NMI Button
3 System Identification Button
4 LCD Menu Buttons
5 LCD Panel
6 VGA Connector
7 SD vFlash card slot
8 Service Tag

Processors
9 USB and iDRAC Direct Connector
10 USB Connector
In comparison to the PowerEdge 13G servers, the PowerEdge 14G servers

feature:
• The Intel Xeon scalable processor that replaces the Xeon E5 and E7 versions
found on the 13G platform.
• An optional new bezel that includes the LCD panel, to support additional airflow
for devices with higher power consumption.
• PowerEdge RAID Controller 10 series cards that support advanced capabilities.
• Randomized passwords for the iDRAC account, rather than the default
credentials that 13G servers support.
• The ability to use M.2 SATA devices as boot cards (BOSS cards), requiring
Operating System support.
PowerEdge R640
PowerEdge R640

.
Processors
PowerEdge R740xd
PowerEdge R740xd
PowerEdge R940
PowerEdge R940
PowerEdge R940xa: Comparison with bezel and without bezel
All the systems will ship with a bezel but the customer has the choice of purchasing
the bezel with or without LCD.

Processors
PowerEdge R940xa with front bezel

dell.com/support

.
Processors
1 2
4 3
1: Management and Automation: PowerEdge 14G systems provide a single-

management layer for all platforms, reducing the time and effort that is needed to
manage large-scale environments. PowerEdge 14G systems support iDRAC9.
iDRAC9 supports the Group Manager feature that enables users to have multiple
console experience and offers simplified basic iDRAC management.
2: Intel C620 chipset: PowerEdge 14G systems include the Intel Lewisburg as the
Platform Controller Hub (PCH) chip. The Integrated Intel® Ethernet with scalable
iWARP RDMA in the Intel® C620 series chipset provides up to four 10 GBPS high-
speed Ethernet ports for high data throughputs and low-latency. Ideal for storage,
data intensive, and connected IoT solutions.
3: The Intel® Xeon® scalable processor family supports 2933 MT/s memory. As an
example, the PowerEdge R740 and R740xd support two DIMMs per channel at
2933 MT/s with these processors.
4: GPU/FPGA mainstream: Expanded GPU capacity with additional accelerator

cards to maximize workload.

Processors
Supports up to 3 x 300 W or 6 x 150 W accelerator cards maximizes workload

acceleration.
Management and Automation: PowerEdge 14G systems provide a single-

management layer for all platforms, reducing the time and effort that is needed to
manage large-scale environments. PowerEdge 14G systems support iDRAC9 (this
feature is only available with the Data Center and Enterprise license).
iDRAC9 supports the Group Manager feature that enables users to have multiple
console experience and offers simplified basic iDRAC management.
Intel C620 chipset: PowerEdge 14G systems include the Intel Lewisburg as the
Platform Controller Hub (PCH) chip. The Integrated Intel® Ethernet with scalable
iWARP RDMA in the Intel® C620 series chipset provides up to four 10 GBPS high-
speed Ethernet ports for high data throughputs and low-latency. Ideal for storage,
data intensive, and connected IoT solutions.
GPU/FPGA mainstream: Expanded GPU capacity with additional accelerator

cards to maximize workload.
Supports up to 3 x 300 W or 6 x 150 W accelerator cards maximizes workload

acceleration.
Processor: The Intel® Xeon® scalable processor family supports 2933 MT/s
memory. As an example, the PowerEdge R740 and R740xd support two DIMMs
per channel at 2993 MT/s with these processors.
PowerEdge 14G Control Panel
Control Panel View
The image below highlights the control panels that are located at the front of a
PowerEdge 14G system.

.
Processors
Left Control Panel Right Control Panel
The front view of the PowerEdge R740 is used as an example.
Control Panel Components
X2
System health
and system ID
Indicators
Status LED
System health
Indicators
Status LED and system ID
Indicators Indicators
iDRAC Quick Sync 2

wireless indicator
(optional)
Left Control Panel with Left Control Panel

Quick Sync without Quick Sync

Processors
In comparison to the PowerEdge 14G servers, the PowerEdge 15G servers

feature:
• Products purpose-built for diverse environments to support workloads that are
accelerated, computational-heavy, based on Artificial Intelligence and other
edge technologies.
• A cyber resilient architecture and a well-established Root of Trust (RoT)2, to
ensure a complete cycle of security for the server.
• Uniquely designed chassis that support ducted fans to improve energy
consumption.
• Support for Open Compute Project (OCP) NIC 3.0 technology.
PowerEdge R750
2 Root of Trust is a concept that starts a chain of trust that ensures systems boot
with a legitimate code at every step of the boot process. RoT is controlled by the
iDRAC9.

.
Processors
PowerEdge R750
PowerEdge XE8545
PowerEdge XE8545
PowerEdge XR11
PowerEdge XR11
PowerEdge R6515
PowerEdge R6515

Processors

dell.com/support
1 2
4 3
1: Direct Liquid Cooling (DLC): DLC is introduced in the PowerEdge 15G systems
and features a leak-sensing technology to identify and resolve issues faster. The
DLC technology is supported only in the PowerEdge R650, PowerEdge R750,
PowerEdge R750xa and PowerEdge C6520 servers.
2: PowerEdge RAID Controllers: Support for PERC 10 and PERC 11 cards for
enhanced RAID performance.
To view the list of PERC types for Dell EMC systems, visit the List of PowerEdge
RAID Controller (PERC) types for Dell EMC systems document in dell.com/support

.
Processors
3: Configuration Validation: The 15G PowerEdge systems support interconnect

flexibility and advanced iDRAC management features to collect precise system
configuration information and report configuration errors.
4: Memory: The 15G PowerEdge servers support the Intel Optane persistent
memory, that support up to 16 DIMMS per CPU.
Intel Optane persistent memory is also known as Barlow Pass. Click here for more
information on Barlow pass and different configuration.
Direct Liquid Cooling (DLC): DLC is introduced in the PowerEdge 15G systems
and features a leak-sensing technology to identify and resolve issues faster. The
DLC technology is supported only in the PowerEdge R650, PowerEdge R750,
PowerEdge R750xa, and PowerEdge C6520 servers.
PowerEdge RAID Controllers (PERC): Support for PERC 10 and PERC 11 cards
for enhanced RAID performance.
To view the list of PERC types for Dell EMC systems, visit the list of PowerEdge
RAID Controller (PERC) types for Dell EMC systems document in
dell.com/support.https://www.dell.com/support/kbdoc/en-in/000131648/list-of-
poweredge-raid-controller-perc-types-for-dell-emc-systems
Memory: The 15G PowerEdge servers support the Intel Optane persistent
memory, that support up to 16 DIMMS per CPU.
Intel Optane persistent memory is also known as Barlow Pass. Click here for more
information on Barlow pass and different configuration.
Configuration Validation: The 15G PowerEdge systems support interconnect

flexibility and advanced iDRAC management features to collect precise system
configuration information and report configuration errors.

Server Components

Processors
Server Components
Introduction
Server components are broadly identified into two categories:

• Hot swap components
• Field Replacement Units (FRUs)
• Customer Replacement Units (CRUs)
Click each tab to learn about the server components.
Hot Swap Components
Hot swap components can be Hot swappable or hot-pluggable.

• Hot swapping: Describes replacing components while the system is still in
operation.
• Hot plugging: Describes the addition of components that would expand the
system while the system is still in operation.
Hot swap components are designated by orange tabs or markings.
Hot swap components enable zero system downtime for failures and serviceability.
Examples of some Hot swap components are fans, disks, and Power Supply Units
(PSUs).
Fans Drives Power Supply Units
Examples of Hot swap components

Double-click the image to enlarge.

Processors
Field Replacement Units
A field replacement unit (FRU) is a server component or assembly that requires the
entire chassis to be powered off to service.
FRUs are replaced by a user or technician without having to send the entire
product or system to a repair facility.
FRU is marked as blue color. Blue color indicates that the system must be
shutdown to replace this component.
Image of an FRU fan.
Customer Replacement Units
Some component parts are designed for easy customer removal and replacement;
such parts are designated as Customer Self-Replaceable (CSR) or Customer
Replaceable Unit (CRU).

.
Processors
When during the troubleshooting, a Dell Technician determines that the repair can
be accomplished with a CSR/CRU designated part, Dell ships the designated part
directly to the customer, which allows customers to replace parts at their own
convenience.

Processors
Processors
Processors
Intel Xeon Scalable Processor Family
The Intel processor uses a metal naming convention to designate the different
levels of available features. The levels are Platinum, Gold, Silver, and Bronze.
Click each image to learn more about the Intel Xeon scalable processor family.
1 2 3 4
1: Intel® Xeon® Platinum processors offer the industry best performance for
mission-critical and hybrid cloud workloads, real-time analytics, machine learning,
and artificial intelligence. The platinum processors offer monumental leaps in I/O,
memory, storage, and network technologies.
The platinum processors support:

• Up to 40 CPU cores.
• 8+ socket configurations.
• Up to 12 TB memory.

.
Processors
2: Intel® Xeon® Gold processors offer high performance, advanced reliability, and
hardware-enhanced security. The gold processors are optimized for demanding
data-centers, hybrid-cloud compute, network, and storage workloads.
The gold processors support:

• Up to four socket configurations.
• Up to 6 TB memory.
3: Intel® Xeon® Silver processors offer the hardware-enhanced performance and

security that is required for data center compute, network, and storage. The silver
processors are optimal for midsized and growing IT organizations.
The silver processors support:

• Up to two socket configurations.
• Up to 1.5 TB memory.
4: Intel® Xeon® Bronze processors provide optimized performance for small

businesses and basic storage servers.
The bronze processors support:

• Up to eight CPU cores.
• Up to two socket configurations.
• Up to 1.5 TB memory.
• Platinum: Intel® Xeon® Platinum processors offer the industry best

performance for mission-critical and hybrid cloud workloads, real-time analytics,
machine learning, and artificial intelligence. The platinum processors offer
monumental leaps in I/O, memory, storage, and network technologies.
The platinum processors support:
− Up to 40 CPU cores.
− 8+ socket configurations.
− Up to 12 TB memory.
• Gold: Intel® Xeon® Gold processors offer high performance, advanced
reliability, and hardware-enhanced security. The gold processors are optimized

Processors
for demanding data-centers, hybrid-cloud compute, network, and storage

workloads.
The gold processors support:
− Up to four socket configurations.
− Up to 6 TB memory.
• Silver: Intel® Xeon® Silver processors offer the hardware-enhanced
performance and security that is required for data center compute, network, and
storage. The silver processors are optimal for midsized and growing IT
organizations.
The silver processors support:
− Up to two socket configurations.
− Up to 1.5 TB memory.
• Bronze: Intel® Xeon® Bronze processors provide optimized performance for
small businesses and basic storage servers.
The bronze processors support:
− Up to eight CPU cores.

− Up to two socket configurations.
− Up to 1.5 TB memory.

.
Processors
AMD Processors
AMD Processor
The Dell EMC PowerEdge server portfolio is powered by the third generation AMD
EPYC™ Processors. The AMD processors system on a chip (SOC) is the next-
generation data center processor supporting socket compatibility with socket
infrastructure. The AMD Milan processor is based on a new enhanced Zen2 CPU
core with integrated I/O controllers.
The AMD Milan processor:
• Offers significant performance improvement from current generation production.
• Has 128 PCIe lanes, eight-channel memory, and dual-socket configurations.
• Lowers cost through an optimal balance of compute, memory, I/O, and security.
• Offers one I/O memory die which removes internal bottleneck for lower latency.
• Has up to 64 CPU cores per processor.
• Interchip global memory interconnect (xGMI2) up to 64 lanes.
• Has Secure Encrypted Virtualization(SEV) which provides 509 unique
hypervisor keys.
• Has two restrictions.
− The RTC/CMOS is built into the CPU, similar to previous PowerEdge AMD
servers. RTC/CMOS will be lost when CPU1 in server is removed or
reinstalled.

Processors
− AMD does not support early boot. No error message is displayed when there
is no memory that is populated in the system.
PowerEdge 15G Processors
The PowerEdge 15G servers also support third-generation Intel Xeon scalable
processors.
The Intel® Xeon® Processor has increased performance and incremental memory
options.
The Xeon scalable processor supports usages from entry designs based on Intel
Xeon Silver processors to advanced capabilities offered in the new Intel Xeon
Platinum processor.
The third-generation Intel Xeon scalable processor supports:
• Increased memory capacity with up to 8 channels and up to 256 GB DDR4
DIMM support.
• Enhanced memory performance with support for up to 3200MT/s DIMMs (2
DPC).
• Intel Optane persistent memory 200 series module (Intel Optane Persistent
Memory 200 Series, up to 512 GB modules) up to 6 TB of total system
memory/socket DDR+PMM.
• Faster I/O with PCI express 4 and up to 64 lanes (per socket) at 16 GT/s.
• Faster Intel Ultra Path Interconnect (UPI) with 3 Intel UPI at 11.2 GT/s
(supported in gold and platinum options).
The 15G server with Intel processors and heatsinks has an additional anti-tilt
feature to prevent tilting of the heatsink assembly. The plastic nuts secure the
heatsinks to the system board.

.
Processors
Unlocked Locked
Anti-tilt feature in a PowerEdge 15G Intel processor and heatsink assembly.
Processor Settings
The Processor Settings option is used to view and configure various processor
settings. The Processor Settings can be accessed through System Setup utility.
Processor Settings.
Go to System Setup Main Menu > System BIOS > Processor Settings.
Some of the available Processor Settings options are:

• Logical Processor: This option is set to Enabled by default. If this option is set
to Enabled, the BIOS displays all the logical processors. If this option is set
to Disabled, the BIOS displays only one logical processor per core.

Processors
• QPI Speed: Controls QuickPath Interconnect (QPI) data rate settings.

• Virtualization Technology: User can enable or disable the additional hardware
capabilities provided for virtualization. This option is set to Enabled by default.
• Address Translation Service (ATS): This option provides an interface
between CPU and DMA Memory Management to a chipset's Address
Translation and Protection Table to translate DMA addresses to host
addresses. This option is set to Enabled by default.
• Adjacent Cache Line Prefetch: This option optimizes the system for
applications that need high utilization of sequential memory access. This option
is set to Enabled by default. User can disable this option for applications that
need high utilization of random memory access.
• Hardware Prefetcher: When set to enable, the processor is able to prefetch
extra cache lines for every memory request.
• X2Apic Mode: x2APIC architecture extends processor addressability and
enhances interrupt delivery performance. User can enable or disable the
X2Apic mode.
• Number of Cores per Processor: Number of cores per processor controls the
number of enabled cores in each processor. This option is set to All by default.
• Processor Core Speed: Processor core speed specifies the maximum core
frequency of the processor.
For more information, visit below link:

https://www.dell.com/support/manuals/en-in/poweredge-
r630/r630_om_pub/processor-settings-details?guid=guid-5fc21783-239f-4469-be5f-
153eb5403270&lang=en-us
Disassembly and Assembly of Processors
Click the play button to view the process of removing and installing the processor
on the Dell EMC PowerEdge server.
Movie:
Link to video:

.
Processors
https://edutube.emc.com/Player.aspx?autoplay=true&vno=GsFeXmu9Q3gwvx0r3gj
+gQ

Memory
Memory
Memory
Server Memory Module
Dell EMC PowerEdge servers run on Error-Correcting Code (ECC) memory. The
ECC memory can test and correct any memory errors without the processor or the
user being aware of these operations. ECC corrects the errors without interrupting
other operations on the server.
Click the image to enlarge.
PowerEdge Servers
Error-Correcting Code (ECC) Memory
a) Both data (M bits) and code generated by Data In traffic are stored.
b) During fetch, new K code bits are generated from M data bits and compared Data Out
with fetched K code bits.
c) If no errors were detected in Compare, then the path to Data Out is taken.
d) Errors detected by Compare are fixed by the Corrector. Corrector
e) Errors detected by Compare but cannot be fixed by Corrector.
Data M bits
Compare Error Signal
Data In
Code K bits
DDR4 Server Memory
Memory Comparison
The table below highlights the differences in memory features across the three
generations of Dell EMC PowerEdge servers.
Server Generation 13G 14G 15G
Number of 12 DIMMs per 12 DIMMs per 16 DIMMs per

Supported DIMMs CPU CPU CPU

.
Memory
RAM Size 1 X 4 GB 1 X 8 GB 1 X 8 GB
DIMM Type RDIMM RDIMM RDIMM

LRDIMM LRDIMM LRDIMM
NVDIMM-N NVDIMM-BP
Supported 4 Channels 6 Channels 8 Channels

Channels per
Processor
Intel Persistent N/A Apache Pass Barlow Pass

Memory 3DXPoint3 3DXPoint4
Supported Transfer 2400 MT/s 2933 MT/s 3200 MT/s

Speed 2133 MT/s 2666 MT/s 2933 MT/s
1866 MT/s 2400 MT/s 2666 MT/s
1600 MT/s 2133 MT/s
1866 MT/s
Note: NVDIMM-N is not supported on Dell EMC 15G servers.
3 The 3DXPoint is a type of nonvolatile memory.

4 The 3DXPoint is a type of nonvolatile memory.

Memory
CPU and Memory Architecture Comparison
14G 15G
13G
CPU CPU ICX
4 Memory channels x 3 slots per channel = 6 Memory channels x 2 slots per channel = 8 Memory channels x 2 slots per channel =
12 DIMMs per CPU 12 DIMMs per CPU 16 DIMMs per CPU
The common CPU architecture features that evolve are:
• The number of memory channels available:

− PowerEdge 15G servers consist of eight memory channels with dual slots
per channel.
− 15G supports flexible memory configurations ranging from system capacities
of 8 GB (minimum) to 4 TB (maximum).
• The number of DIMMs supported:
− PowerEdge 15G servers can support up to 32 DIMMs.

− Each server has eight memory channels per processor, with each channel
supporting up to 2 DIMMs.
15G CPU and Memory Architecture

.
Memory
15G CPU and memory layout.
The 15G CPU and memory architecture:

• Supports Intel third-generation Xeon Scalable processors.
• Manages data flow between the memory channels5 through memory controllers
on the digital circuit.
• Enables memory interleaving for efficient memory accesses across multiple
DIMMs.
• Provides eight memory channels with up to two memory slots per channel and
sixteen memory modules per processor.
5Memory channels are the physical layer on which the data travels between the
CPU and memory modules.

Memory
Memory Layout
Click each tab to view the memory layout supported for each generation of
servers.

.
Memory
13G Memory Layout
13G memory channels and their relation to DIMM slots.
Features of 13G memory layout are:

• Supports up to 12 DIMM slots per CPU.
− Four memory channels per CPU with three memory slots per channel.

Memory
• In a dual-processor configuration, the memory configuration for each processor

should be identical.6
− For single-processor systems, sockets A1 to A12 are available.
− For dual-processor systems, sockets A1 to A12 and sockets B1 to B12 are
available.
• Populating four memory modules per processor (one DIMM per channel) at a
time maximizes the performance.
• Click here for the guidelines on installing the memory modules on a 13G server.
14G Memory Layout
Some of the features of 14G memory layout are:

− Six memory channels per CPU with two memory slots per channel.
available.
• Populating six identical memory modules per processor (one DIMM per
channel) at a time maximizes the performance.
6 If socket A1 is populated for processor 1, then populate socket B1 for processor 2

with an identical DIMM.
7 If socket A1 is populated for processor 1, then populate socket B1 for processor 2

.
Memory
14G R740 (Intel) PowerEdge 14G R7425 (AMD) PowerEdge

Server Motherboard Server Motherboard
Memory Socket Locations Memory Socket Locations
15G Memory Layout
Some of the features of 15G memory layout are:

− Eight memory channels per CPU with two memory slots per channel.
• DIMMs should be populated in from the A1 slot up to the A16 slot in sequential
order.
8If socket A1 is populated for processor 1, then populate socket B1 for processor 2

Memory

available.
15G R750 (Intel) PowerEdge Server Motherboard 15G R7525 (AMD) PowerEdge Server Motherboard
Memory Socket Locations Memory Socket Locations
Memory Settings
The Dell EMC server memory settings can be accessed through the Lifecycle
Controller (LCC) System Setup option.
Important: Administrators can also change memory settings via the

iDRAC9 GUI. The iDRAC9 is often a customer preferred method to
configure memory settings. Using the iDRAC9 GUI avoids taking
systems down to change BIOS settings until a downtime is identified.
Going into the LCC or F2 to the BIOS to configure memory settings
are both seen as a backup for customers. Customers will configure
memory through the iDRAC9 and then schedule the job to edit the
BIOS settings on next reboot.
Click each step to access the system memory settings.

.
Memory
Step 1
Select System Setup from the Lifecycle Controller home page.
Step 2
Select Advanced Hardware Configuration option.

Memory
Step 3
Select System BIOS option.
Step 4
Select Memory Settings option.

.
Memory
Memory Settings
The memory options for the PowerEdge servers include:

• System Memory Testing: Indicates if the system memory tests are run during
system boot.
• Memory Operating Mode: Indicates current memory operating mode. 9Memory
modes can add redundancy and fail-over features to the server.
9 This option is set to Optimizer Mode by default.

Memory
• Node Interleaving: Specifies if Non-Uniform Memory Architecture10 (NUMA) is

supported.
• There are differences in the Memory Settings between 13G, 14G, and 15G
PowerEdge Servers. Click here to learn how to discover the supported memory
configurations for the different types of PowerEdge Servers.
10
If this field is set to Enabled, memory interleaving is supported if a symmetric
memory configuration is installed.
If the field is set to Disabled, the system supports NUMA (asymmetric) memory
configurations. This option is set to Disabled by default.

.
Memory
Memory Modes
The available memory

operation modes are:
- Optimizer Mode
- Mirror Mode
- Advanced ECC Mode
- Spare Mode
- Dell Fault Resilient Mode
System Memory settings page.
The Dell EMC PowerEdge chipset allows different operating modes for the memory
to be set in the BIOS.
The various memory modes available on the Dell EMC PowerEdge servers are:
• Optimizer Mode: The memory controllers run independently of each other.
• Mirror Mode: The system supports memory mirroring if identical memory
modules are installed in two channels.
• Advanced ECC Mode: The two memory channels closest to the processor
(channel 0 & 1) are combined to form a single 128-bit channel.

Memory
• Spare Mode: One rank11 per memory channel is reserved as a spare.

• Dell Fault Resilient Mode: An area of the memory that is established as fault
resilient and is used by a VMware vSphere hypervisor or other services to
maximize the availability.
13G 14G 15G
Optimizer Mode
Mirror Mode
Advanced ECC Mode
Spare Mode
Dell Default Resilient

Mode
Optimizer Mode:
In Optimizer mode, all three channels are populated with memory modules. This
mode permits a larger total memory capacity but does not support SDDC with x8-
based memory modules.
It is recommended to populate all three channels with identical memory but each
channel can have a different size DIMM. The larger DIMM has to be installed in the
first slot and the configuration has to be the same across all three channels. In a
dual-processor configuration, the memory configuration for each processor must be
identical. Optimizer mode is the only mode to support mixed memory sizes. Any
configurations not following the above rules may generate error messages or not
11
A memory rank is a block or area of data that is created using some, or all, of the
memory chips on a module.

.
Memory
POST at all. For more detail read the initial release notes: Installing and configuring
DDR3 Memory
It is recommended to populate all three DIMM slots on servers with three DIMM
slots per channel to take advantage of memory interleaving to get maximum
performance. While a single UDIMM per channel gives slightly better performance
than an RDIMM, RDIMMs give better performance when multiple DIMM per
channel are installed.
Optimizer is used if just one DIMM for each processor is configured. A minimal
single-channel configuration of 1 GB memory modules per processor is also
supported in this mode. Minimum to POST would be one DIMM and in the first slot
and just CPU 1 installed.
Memory Mirroring:
The PowerEdge system supports memory mirroring if identical memory modules

are installed in the two channels closest to the processor (CH 0 & 1) and no
memory that is installed in the channel furthest from the processor (CH 2).
Mirroring must be enabled in the System Setup program. The total available
system memory is one-half of the total populated memory, any operating system
will only show half the physical memory installed.
Advanced ECC (Lockstep) Mode
In Advanced ECC (Lockstep) mode, the two channels closest to the processor (CH
0 & 1) are combined to form one 128-bit channel. This mode supports Single
Device Data Correction (SDDC) for both x4 and x8 based memory modules.
Memory modules must be identical in size, speed, and technology in the slots on
channel 0 and 1. Channel 2 has to be empty or option will not be available in the
System Setup program.
Using the Intel 5500 and 5520 chipset with Intel 55xx and 56xx processors
channels 0 and 1 are combined which enables 8-bit error correction instead of 4-bit
in normal Advances ECC (not lockstep). SDDC gives the ability to recover from
more types of single and multibit memory errors. The third channel and
corresponding memory slots cannot be used but full amount of installed physical
memory will be accessible to the operating system.
Note: 14G and 15G servers do not support advanced ECC mode.

Memory
Dell Fault Resilient mode
Dell Fault Resilient Mode (FRM) is a Memory Operating mode available on the
BIOS settings of high-end yx2x Dell PowerEdge servers and later. This mode
establishes an area of memory that is fault resilient and protects the hypervisor
against uncorrectable memory errors, and safeguards the system from becoming
unresponsive. Systems with ESXi that supports the FRM feature can load the
operating system kernel to maximize system availability and or critical applications
or services.
Memory Spare Mode
Memory Spare mode can be further classified into two types.
• Single-rank sparing mode: It allocates one rank per channel as a spare. This
mode requires a population of two ranks or more per channel.
• Multi-rank sparing mode: It allocates two ranks per channel as a spare. This
mode requires a population of three ranks or more per channel.
When single rank memory sparing is enabled, the system memory available to the
operating system is reduced by one rank per channel. See the example below.
In a dual-processor configuration with twenty-four 16 GB dual-rank memory modules, the available system memory
is:
3/4 (ranks/channel) × 24 (memory modules) × 16 GB = 288 GB
This calculation changes based upon single-rank or muti-rank sparing. In multi-rank sparing, the multiplier changes
to 1/2 (ranks/channel).

.
Memory
Note: If excessive correctable errors occur in a rank or channel, they

are moved to the spare area. The operating system is running to
prevent errors from causing an uncorrectable failure.
Post Package Repair (PPR) is a self-healing memory enhancement
that works to repair the issue by disabling the location/address at the
hardware layer, enabling a spare memory row to be used instead.
Intel Persistent Memory
The Intel Optane Persistent Memory (Barlow Pass) solution retains data during a
power loss, system shutdown, or system errors. Barlow Pass (BPS) uses persistent
memory as storage, rather than traditional memory.
DRAM
(volitle variables)
Memory Bus
CPU
NVDIMM (BPS)
(volitle variables)
Features of the Intel DC Persistent memory Module (DCPMM) are:
• Creates a unique new memory tier to reduce latencies and optimize workloads.
• Provides disruptive storage class memory cell technology (3DxPoint) that
resides on the DDR memory interface.
• Provides large memory footprints of 128 GB, 256 GB, and 512 GB.
• Enables in-memory data to survive a soft reset or a hard reboot (power loss).
• Provides minimal latency and faster storage for large amounts of memory.

Memory
Note: Barlow Pass is the second generation of Intel Optane

Persistent Memory. Apache Pass is the first generation of Intel
Optane Persistent Memory.
Apache Pass supports 14G platforms and Barlow Pass DIMMs
support the 15G platforms. DIMMs are not compatible across
generations.
Intel Persistent Memory DIMM Layout
The image represents the Intel Persistent memory DIMM layout.
Bulk Capacitors
DIMM
Primary Media Media Media Media Media Media
Controller 31.25mm
Side
133.35mm
Data Clock Oscilator
Buffers Persistent Memory DRAM (AIT)
Media Power Management IC
SPi Flash
SPD
DRAM SPi Secondary
Media Media Media Media Media PMIC
Side
Data
Buffers BUFF BUFF BUFF BUFF

.
Memory
Barlow Pass Advantages
DRAM Memory Tier
NVDIMM
Intel Optane
PCIe SSD
Storage Tier
PCIe SSD
SATA SSD
HDD
Tape
The Barlow Pass architecture consists of two-tier memory and storage hierarchy to
address the data performance and storage challenges. The advantages of the
hierarchical approach are:
• Provides a unique combination of affordable large capacity and support for data
persistence.
• Optimizes the resources for efficient data access and storage.
• Provides higher performance (up to 3200 MT/s) with low latency DRAMs.
• Creates larger memory capacity (up to 4TB per CPU) to store and protect data
in DRAM.
• Enables in-memory computing for large datasets.
• Leverages the speed and proximity from the technologies nearer to the CPU.
Industry Mixed Memory Combinations
Barlow pass supports the following mixed memory combinations.

Memory
DIMM Types NVDIMM-N

RDIMM LR-DIMM 3DS LR-DIMM BPS DIMM
(R Type)
RDIMM Supported Not Supported Not Supported Not Supported Supported
LR-DIMM Not Supported Supported Not Supported Not Supported Supported
3DS LR-DIMM Not Supported Not Supported Supported Not Supported Supported
NVDIMM-N
Not Supported Not Supported Not Supported Not Supported Not Supported
(R Type)
BPS DIMM Supported Supported Supported Not Supported Not Supported
Memory Population Configurations with Barlow Pass
Click each tab to review the Barlow Pass memory and processor configuration for
15G systems.
BPS Memory Configuration
The memory modes can be used only if the DIMMs are RDIMMs with capacity of
32GB or lesser.
Config RDIMM + BPS Number Number Number App- Memory

number (Configuration) of CPU of of BPS Direct Mode
per RDIMMs per CPU Mode
system per CPU
BPS1 4+4 1 or 2 4 4 Y Y
BPS2 6+1 1 or 2 6 1 Y

.
Memory
BPS3 8+1 1 or 2 8 1 Y
BPS4 8+4 1 or 2 8 4 Y Y
BPS5 8+8 1 or 2 8 8 Y Y
BPS6 12+2 1 or 2 12 2 Y
BPS Processor Configuration
Each row on the below chart represents a different valid memory configuration for
mixing Barlow Pass (B) and RDIMMs (R).
Per CPU
IMC 2 IMC 3 IMC 1 IMC 0
Channel F Channel E Channel H Channel G Channel C Channel D Channel A Channel B

A6/B6 A14/B14 A2/B2 A10/B1 A8/B8 A16/B1 A4/B4 A12/B12 A11/B1 A3/B3 A15/B15 A7/B7 A9/B9 A1/B1 A13/B1 A5/B5
B R B R R B R B
R R R R B R R
R NV R NV R R NV R R B R NV R
R R B R R B B R R B R R
R B R B R B R B B R B R B R B R
B NV
R R R R R R R
R R R R R R B
Important: Click here to review recommended guidelines for installing

Intel Persistent Memory 200 Series (BPS) in a PowerEdge server.
Operational Modes
Intel Persistent Memory can be configured into two different modes.

Memory
DIMMs can be used in Memory Mode or App Direct Mode12.
Users can configure the memory modes and update it in the BIOS.
Click each tab to learn about different memory modes.
12The mode can be changed through the BIOS settings: F2 -> System BIOS ->
Memory Settings -> Persistent Memory -> Intel Persistent Memory -> Region
Configuration.

.
Memory
Memory Mode
Memory mode uses the DIMM as

memory only.
Application
Features of Memory mode:
• The system memory functions as
volatile system memory.
• Memory mode is transparent to
the OS and Applications.13
• Memory mode do not change any
software/application on the
DRAM as
system.
Cache
Advantages of Memory mode are:
• BPS DIMMs are used as volatile
system memory when configured
in Memory Mode. DCPMM
volatile
• All DDR4 standard DIMMs are
used as cache (near memory).
• Asynchronous DRAM Refresh
(ADR) is required for all modes.
Memory mode architecture.

Intel® Optane™ DC Persistent Memory
Module(DCPMM)
13They access DIMMs as system memory, and will not have control or direct
access to DDR4 DIMMs that are used for caching.

Memory
Memory Mode setting in System BIOS
AppDirect Mode
No change to application
Application
Standard Raw Device Access OS or VMM
OS or VMM
Blocks File System
New: Use NVDIMM driver BTT
Block Translation Table

NVDIMM Driver
Cache lines Cache lines
Persistent Region
AppDirect mode architecture. The NVDIMM in the image represents BPS.

.
Memory
The App Direct mode is the default memory mode on the BIOS. AppDirect mode
uses the DIMM as storage.
Features of App Direct mode:
• Provides larger storage capacity, higher endurance, low latency and traditional
read/write.
• Works with existing file systems to access the files. Two major methods to
access the files are Block method14 and PMEM method15.
• Cache lines are accessed using load or store instructions.
• Application is responsible for flushing data out of CPU cache into persistence
guaranteed memory buffers.
Advantages of AppDirect mode are:
• Provides a new tier between DRAM and Block Storage.

• BPS DIMMs are used as persistent memory when configured in AppDirect
Mode.
• Persistent memory aware file systems, operating system and applications can
access BPS DIMMs in AppDirect mode.
Important: By default, the memory is in memory mode. The user

must change it to persistent memory in the iDRAC or BIOS.
14 The block method is slower and is similar to traditional storage access. The block
size is configurable at the operating system level.
15 PMEM method uses the full technology potential, but requires the application to
be optimized.

Memory
Memory Mode and Operational Mode Differences
In Memory mode, the BIOS and operating system list the capacity of the Optane
memory and not the RDIMMs. The RDIMMS are used as cache for the Optane
DIMMs when running in memory mode.
In AppDirect mode, 632 GB is the full amount of memory available but only 128 GB
of it is volatile. The system uses the rest of the memory as persistent storage.
Memory Mode (4 x 32GB RDIMM + 4 x 128GB BPS)
App Direct Mode(4 x 32GB RDIMM + 4 x 128GB BPS)
The image shows the difference between the memory capacity that is available when running in
Memory mode and AppDirect mode.
Important: Dell EMC offers Intel Optane Persistent Memory (PMem),

a nonvolatile memory that has the same form factor as a standard
DDR4 DIMM. PMem can be configured into two modes: Memory
Mode and App Mode. Click here to review the Dell EMC PMem 100
Series User's Guide and the information on how to install and remove
the persistent memory.
Removal and Installation of DIMMs
Click the play button to view the process to remove and install DIMMs on a 15G
PowerEdge server.
Link to video:
https://edutube.emc.com/Player.aspx?vno=u7HFeGUw5TxmRHnGo3VGFA==&aut
oplay=true

.
Memory
Movie:

Power
Power
Power
Power Supply Unit
Hot-swappable Dell EMC 800 W PSU.
In most cases16, the power supply unit (PSU) is a hot-swappable component that
provides power redundancy support on PowerEdge servers.
All the Dell EMC PowerEdge servers support a minimum of two PSUs.
16The PSUs shipped in the PowerEdge 200-500 server series are not hot-
swappable. Not all Dell EMC PowerEdge support a minimum of two PSUs. Some
low-end PowerEdge servers have a single PSU.

.
Power
The Dell PSUs offer:

• Automatic load balancing and fail-over.
• Power distribution management capabilities.
• Regular firmware updates.
Important: Before performing a hot swap on a PSU, be sure to verify

the failed PSU in the iDRAC.
PSU Configuration Modes
The PSUs on PowerEdge servers can be configured in different modes based on

the customer requirements17.
The redundant power supply is implemented in PowerEdge servers to keep the

servers running when the primary PSU fails to provide the necessary power supply
and to minimize the risk of a complete server shutdown. The Power
Configuration panel uses the iDRAC9 (and newer versions) interface to configure
redundant power supplies for servers. The type of power supply configuration or
the redundancy mode depends on the server chassis and the number of PSUs.
Click each tab to see the different power supply configurations.
17Not all Dell EMC PowerEdge support a minimum of two PSUs. Low end
PowerEdge servers have a single PSU. For example, the PowerEdge R230 does
not support the multiple PSU feature and redundancy.

Power
Grid Redundant
The Grid Redundant configuration is also known as 1 + 1 configuration.
In grid redundant mode, the hot spare18 feature is disabled and the power output is
distributed equally across both power supplies. The Power Factor Correction (PFC)
is disabled by default, to reduce power consumption when the system is on
standby. However, if a single PSU fails, the power drops down.
The purpose of the Grid Redundancy policy is to enable a modular enclosure

system to operate in a mode in which it can tolerate power failures. These failures
may originate in the input power grid, the cabling, or a PSU itself. When a system is
configured for Grid Redundancy, the PSUs are divided into grids: PSUs in slots 1,
2, and 3 are in the first grid while PSUs in slots 4, 5, and 6 are in the second grid.
The system management manages power so that if there is a failure of either grid,
the system continues to operate without any degradation. Grid Redundancy also
tolerates failures of individual PSUs.
18When the hot spare feature is enabled, one of the redundant PSUs is switched to
the sleep state. The active PSU supports 100 percent of the system load, thus
operating at higher efficiency. The PSU in the sleep state monitors the output
voltage of the active PSU. If the output voltage of the active PSU drops, the PSU in
the sleep state returns to an active output state.

.
Power
Dual power supply configuration with grid redundant option.
No Redundancy
The no redundancy configuration is also known as a 2 + 0 configuration. For

example: Adding two power supplies together to create 1140 W (2 x 570 W). With
no redundancy configuration, the hot spare19 feature is enabled and Power Factor
Correction (PFC) is disabled by default, to reduce power consumption when the
system is in standby. However, if a single PSU fails, the power drops down to just
570 W.
19When the hot spare feature is enabled, one of the redundant PSUs is switched to
the sleep state. The active PSU supports 100 percent of the system load, thus
operating at higher efficiency. The PSU in the sleep state monitors the output
voltage of the active PSU. If the output voltage of the active PSU drops, the PSU in
the sleep state returns to an active output state.

Power
Configuration with no redundancy.
PSU Redundancy Types
The PSU redundancy mode depends on the server type and the number of PSUs
in the system.
Redundancy Mode Option Description
1+0 Non-Redundant No redundancy.
1+1 Input or PSU Provides redundancy for

Redundant input failures (failed power
cables, PDU, grid) and
redundancy for PSU
failures.
2+0 Non-Redundant No redundancy.

Provides support for
configurations that exceed
the output capacity of a
single PSU.
The table shows PSU redundancy modes that are available with Dell EMC PowerEdge servers.

.
Power
Power Capping
The Power capping option is used to limit the amount of power consumed by a
server.
When power cap policy is enabled, it enforces a user-defined power limits on the
system. If power-capping is not enabled, the default hardware power-protection
policy is used. This power-protection policy is independent of the user-defined
policy. The system performance is dynamically adjusted to maintain power
consumption close to the specified threshold.
Enabling power capping through iDRAC.

Go to Configuration > Power Management > Power Cap Policy
Advantages of capping are:

• Power budget threshold can be specified to reduce consumption.
• System performance is dynamically adjusted to maintain the power
consumption below the user-specified threshold.
• Reduces peak power consumption of the server.
• Using OpenManage Enterprise Power Manager you can optimizes the overall
data center power efficiency by setting power caps on individual system or by
groups of servers.
• The power limit can be set through the iDRAC UI.

Power
Mixing Power Supplies
Dell EMC PowerEdge servers support mixed PSU configurations. The criteria to
implement mixed PSU configurations are:
14G
15G
• PSUs must have the same type of label20.

• PSUs must have the same maximum output power.
• Mixing PSUs from different generations of PowerEdge servers is not supported.
• Mixing PSUs from different generations or from the same generation but with
different power wattages results in a mismatch condition. When a mismatch
condition occurs the lower wattage PSU is disabled and a warning condition is
triggered.
20 For example, the Extended Power Performance (EPP) label.

.
Power
Important: In the 15G PowerEdge PSUs, the wattage label is present

on the handle wing. In the PowerEdge 13G/14G PSUs, the wattage
label is present on the chassis.
PSU LED Indicator Behavior
The table details the PSU LED indicator behaviors.
PSU LED
A 1400 W PSU with active LED.
LED Color LED behavior Description
Solid Green PSU is in a healthy state
Blinking Amber, 2 sec ON, PSU is in a faulty state

1 sec Off

Power
Blinking Green with 4 Hz PSU mismatch

rate for five times and then
stay OFF.
Blinking Green with 4 Hz Firmware update

rate.
OFF No power
PSU Firmware Updates
The PSU firmware can be updated through the Lifecycle Controller (LCC). Click
here to review how to update a server Power Supply Unit firmware, including a
video walk-through of the procedure.
Example of a PSU errors on iDRAC due to mixed PSU configuration. Double click to enlarge the
image.
The following precautions should be taken while updating the firmware:
• During the update process, the server powers off.

• Powering on may prematurely lead to a corrupted PSU firmware and can brick
the power supply.
− Do not power on the server as the iDRAC and the Lifecycle Controller are
running background operations during this power off.

.
Power
• A firmware update cannot be performed on a mixed PSU configuration. The

PSU may receive the wrong firmware image due to mixed configuration.
• To update PSU firmware of different wattages:
− Update the firmware on a single PSU.
− Remove PSU 1 and apply updates to PSU 2. Next, remove PSU 2 and apply
updates to PSU 1.
• If matching PSUs are plugged into mixed line voltage, the low line PSU triggers
an error.
PSU Blanks
Location of PSU in Dell EMC PowerEdge R750 (15G).

To maintain an efficient airflow for system cooling, all servers with an empty PSU
slot require PSU blank plates. PSU blanks avoid the loss of cooling airflow. If the
PSU blanks are missing, the system temperature might increase and result in
component failures.
In 15G servers, the PSUs are located in the rear of the system. The PSUs are on
the opposite side of each other for better airflow within the chassis.

Power
PowerEdge R740xd PSU blank installed
PSU bay (slot)
Installing the PSU blank: PSU blank
Align the power supply unit blank

with the power supply unit slot
and push it into the power supply
unit slot until it clicks into place.
Removal and Installation of PSUs
Click the play icon to view the process of removing and installing the a PSU in a
PowerEdge server.
Link to video:
https://edutube.emc.com/Player.aspx?vno=piELqAPBoEvL8f7u5bGZVQ==&autopl
ay=true
Movie:

.
Cooling
Cooling
Cooling
Fans and Types of Fans
High-performance fan used in the Dell EMC PowerEdge R6525 server.
A server consists of multiple fans. When a fan fails, the remaining fans take up the
load.
The cooling fans dissipate the heat generated by the functioning of the server21.
These fans cool the processors, expansion cards, and memory modules.
The fans ramp up when:
21 Some servers may not have hot-swappable fans (Hot swapping is the
replacement or adding of components to a system unit system without stopping,
shutting down, or rebooting the system). If no hot-swappable fans are available and
if a fan fails, the iDRAC ramps up the existing fans, similar to systems with hot-
swap fans. However, the failed fan cannot be replaced until the system has been
powered off as the fans must have their cables that are disconnected from the
system board.

Cooling
• The system cover is removed.

• The system is powered on.
• The iDRAC is rebooted.
Types of Fans
The Dell EMC PowerEdge servers use Standard fans, High-Performance fans, and
Very High-Performance fans based on the server configurations22.
Dell EMC PowerEdge servers come with different chassis dimensions and they can
be 1U, 2U, and so on. Based on the chassis dimension and design the fan
dimension may vary as well.
If a system has six fans and one of the fans fails, the iDRAC ramps up the
remaining fans. It keeps the temperature within the chassis at a set level. (It should
be noted that if the temperature is already well below the required level, the iDRAC
may not ramp up the remaining fans.)
Once the failed fan has been replaced, the iDRAC tests the new fan. It slowly
decreases the speed of the existing fans while increasing the speed of the new fan
until they are all operating at the correct speed.
Removal of the chassis covers often results in the fans ramping up as the cover is
used to deflect airflow throughout the system. If the cover is removed a certain
amount of airflow is lost so the iDRAC, upon detecting that the cover has been
removed, it ramps up the fan-in an effort to increase the airflow across the
components to maintain required temperatures.
22 A user can see the thermal restrictions matrix or the technical guide of the server
for more information.

.
Cooling
Also when the system is first powered on, the temperatures take a few seconds to
be recorded. As a fail-safe procedure, the iDRAC ramps the fans up and then bring
them back down as temperature status is analyzed.
Should a fan fail, errors are posted and the remaining fans pick up the additional
workload. However, based on the temperature within the chassis, the remaining
fans may or may not increase their speed.
The orange color denotes a hot-swappable component.
Some servers, especially the lower ranges may not have hot-swap fans. In that
case, if a fan fails, the iDRAC ramps up the existing fans like for systems that have
hot-swap fans, but the failed fan cannot be replaced until the system has been
powered off as the fans must have their cables that are disconnected from the
system board.
Standard Fans (STD)
Standard fan dimension for:
• R6525 is 40 mm (1.57 inches) x 40 mm (1.57 inches) x 56

mm (2.20 inches).
• R7525 is 60 mm (2.36 inches) x 60 mm (2.36 inches) x 38
mm (1.49 inches).
High Performance Fans (HPR) Standard fan

Cooling
High performance and very high-performance silver grade fans.
The HPR fans provide a higher airflow rate. HPR fans are required in 12 x 3.5”,
rear-storage configurations and most GPU configurations.
High-performance fan dimension for:
• R6525 is 40 mm (1.57 inches) x 40 mm (1.57 inches) x 56 mm (2.20 inches).


.
Cooling
High Performance Fans (VHP)
High performance and very high performance gold grade fans.
VHP fans require front 16x NVMe drives or 8x NVMe, 16x SAS drives with GPU
configurations.
Very high-performance fan dimension for:

Important: New cooling fans come with the High-Performance Gold

and Silver Grade label, while the older cooling fans have the High-
Performance label.
Important: High-performance fans can be distinguished by the color

of the label, gold, or silver. Mixing of STD, HPR (Silver), or HPR
(Gold) fan is not supported.

Cooling
PowerEdge 15G Fans
Some of the 15th generation Dell EMC Intel servers use standard, high-
performance silver grade, or high-performance gold grade fans, dependent on the
configuration.
Dell EMC PowerEdge servers such as R250, R350, T350, T150, XR11 and XR12
use non hot pluggable single rotor fans.
Hot pluggable fans used in the Dell EMC PowerEdge R750 servers
Non-hot pluggable fans used in the Dell EMC PowerEdge T150 and R250 servers
Removal and Installation of Fan
Click the play icon to view the process of removing and installing a system fan-in a
Dell EMC PowerEdge R750 server.
Movie:
Link to video:
https://edutube.emc.com/Player.aspx?vno=o4jLJmM21AIp2vsPpWAs8Q==&autopl
ay=true

.
Cooling
Heatsinks
1U standard (STD) heatsink (HSK) T-type HSK 2U high performance (HPR) HSK
The type of heatsink that is used is based on the CPU TDP23 and GPU
configurations.
Some PowerEdge servers have unique fan positioning in the chassis. For example,
the Dell EMC PowerEdge XR11 has two fans that are located towards the middle
of the chassis. It has an extended heatsink design for optimum cooling.
Did you know?
23 Thermal Design Power (TDP) is measured in watts and is the maximum amount
of heat that is generated by a GPU or CPU. There are multiple types of CPU
heatsinks available including standard (STD), T-type, and full height heatsinks.

Cooling
In, certain single CPU configurations (non-GPU or nonrear-drive) only four fans are
required to be installed in the fan bay.24
Two fans installed in the middle of the PowerEdge XR11 Extended heatsink used in the PowerEdge XR11.
chassis.
Unique fan position and extended heatsink that is used in the PowerEdge XR11.
To remove the heatsink: The heatsink and processor are too hot to touch for some
time after the system has been powered off. Allow the heatsink and processor to
cool down before handling them.
1. Ensure that all four Anti-Tilt wires are in the locked position (outward position),
and then using a Torx number T30 screwdriver, loosen.
24In such configurations, only four fans are required to cool the system. For the
other two fan sockets, two fan blanks are required to be installed in fan bays 1 and
2. The number of fans that are required depends on the server model and
configuration.

.
Cooling
• Loosen the first screw three turns.

• Loosen the screw diagonally opposite to the screw you loosened first.
• Repeat the procedure for the remaining two screws.
• Return to the first screw to loosen it completely.
2. Set the Anti-Tilt wires to the unlocked position (inward position).
3. Lift the processor and heatsink module (PHM) from the system and set the
PHM aside with the processor side facing up.

Cooling
Memory DIMM Blank and GPU Air Shroud
Memory DIMM blank used in the R740. GPU air shroud used in the XE8545.

.
Cooling
Single DIMM blank Processor and DIMM blank
DIMM blanks on empty DIMM slots help regulate air flow through the CPU and
DIMM area.
For some servers, the air shroud looks like piano keys25 that drop down into empty
DIMM slots to stop airflow from being wasted. However, systems with midrange
storage require DIMM blanks for empty DIMM slots since the trays do not contain
piano keys.
Some of the PowerEdge servers like the XE8545 have separate GPU air shrouds
for better airflow and heat dissipation.
25 This means that they do not require single DIMM blanks on empty DIMM slots.

Cooling
GPU Heatsinks and Fans
GPU fans located in the front of the chassis. Heatsinks seated on top of the Nvidia A100 GPUs.
Dell EMC specialized servers also known as XE servers can have advanced GPU
configurations. These systems generate a lot of heat and require custom solutions.
The Dell EMC PowerEdge XE8545 supports up to 4x NVIDIA A100s GPUs and
NVLink in an air-cooled chassis.
The GPUs are cooled with the help of hot-pluggable GPU fans and specialized
heatsinks for each of the GPUs.
Important: Click the following link to get a copy of a guide showing

the steps involved in removing the Nvidia A100 GPU assembly. -
https://dell.bravais.com/s/KlmMp4D2b4cYCkDwPpcK.
Dell EMC Multi Vector Cooling
The features of Dell EMC Multi Vector Cooling 2.0 are:

• Patented adaptive closed loop power capping.

.
Cooling
• Patented baseline fan speed algorithm.

• It provides custom delta-t design that enables users to specify the outlet
temperature (Requires iDRAC Datacenter).
• It provides custom PCIe inlet temperature, airflow control and, other custom
cooling capabilities for PCIe devices (Requires iDRAC Datacenter).
Click the tabs to learn about the upgrades on design and thermal configurations.
Design Innovation
The advanced thermal design streamlines the airflow pathway in the chassis and
directs the appropriate volume of air to components that require a constant air
supply.
The design minimizes the fan and system power consumption while maintaining
the system temperature.

Cooling
Thermal Design and Cooling Components
14G PowerEdge 15G PowerEdge

Servers Servers
The latest thermal design and cooling components features are:
• It has a split PSU design to improve airflow through the system.

• It enables optimized PCIe connectivity from CPU to rear PCIe slots.
• It has a smaller PSU form factor (except 2400 W PSU).
• It provides additional “exhaust lanes” for hot airflow from CPUs.
• It provides enhanced performance that is required for cooling higher thermal
design power (TDP) CPUs.

.
Cooling
Server Exhaust Temperature
iDRAC Cooling Configuration settings page where a user can change the exhaust temperature
limit.
Dell EMC PowerEdge servers use advanced thermal control algorithms to maintain
system temperatures at reliable levels while minimizing fan speed26 and system
airflow.
User Exhaust Temperature Control Options:
• Instances occur that require exhaust temperatures to be reduced, typically to

accommodate legacy power distribution units (PDUs), power cables, or Top of
Rack (ToR) switches.
• The fan speed and exhaust temperature settings are available using HII (BIOS-
based iDRAC settings), iDRAC web UI, and RACADM.
Users can apply custom fan speeds when using interfaces such as: iDRAC UI,
BIOS setup (F2), and RACADM.
26This minimization of system fan speeds and airflow can result in high exhaust
temperatures may be of concern to some users.

Cooling
HVAC - Heating, Ventilation, and Air Conditioning
As computing demands grow, so do data centers, and with this growth comes huge
amounts of heat27 that must be managed efficiently. Many data centers start out as
a few racks in a server room, adding more equipment over time. Without taking
cooling factors into account, data center HVAC management can become difficult.
27When data centers are exposed to heat, servers start to slow down or
malfunction altogether. The same thing happens when the server rooms are too
cold. The ideal temperature for the data center depends on the size and amount of
heat that is emitted, but operating within this ideal temperature range is crucial for
overall performance.

.
Cooling
Hot aisle containment (HAC) guides the hot air (red arrows) into a system unit room air handler
(CRAH) which then recirculates the flow into cool air (blue arrows).
Dell EMC PowerEdge Liquid Cooling
3rd Generation Intel Xeon Scalable 3rd Generation AMD EPYC

Processors Processors
Multiple PowerEdge servers with new Intel and AMD processors support the Dell Technologies
DLC.

Cooling
Direct Liquid Cooling (DLC) solution manages the growing thermal challenges. Dell
DLC solutions28 cool the CPU with warm liquid, which has the capacity to transfer
heat up to 4X more than the capacity of air cooling.
Because DLC solutions are more efficient at extracting heat, it reduces the burden
on server system fans and the data center’s cooling infrastructure.
The PowerEdge servers below offer DLC cooling on the newest Intel and AMD
processors:
• C6520
• C6420
• C6525
• R6525
• R7525
• R650
• R750
• R750xa
Important: The DLC solution is not offered as an After Point of Sale

(APOS) kit. Liquid cooling hardware settings cannot be added as an
upgrade post purchase.
28DLC solution is more efficient at extracting heat, reducing the burden on server
system fans and the data center’s cooling infrastructure.

.
Cooling
Discussion Block
Discussion Notes:
• DLC improves overall data center Power Utilization Efficiency (PUE).

• DLC improves the power efficiency of servers and eliminates the need for costly
cooling infrastructures such as chillers and Computer Room Air Conditioning
(CRAC) units. It lowers the overall cost and improves the Total Cost of
Ownership (TCO).
• DLC improves the life of the IT infrastructure.
Direct Liquid Cooling Technology

Coldplat
Manifold
Liquid
flow path
Microchannels
DLC example of a cold plate and coolant loop. Monolithic is used in the 15G rack servers and
modular is used in the Dell EMC PowerEdge C6420 and C6520 servers.
DLC uses the exceptional thermal capacity of liquid to absorb and remove the heat
that is created by new high-power processors. Cold plates are attached directly to
the processors. The coolant captures and removes the heat from the system to a
heat exchanger in the rack or row.
This heat load is removed from the data center using a warm water loop, potentially
bypassing the expensive chiller system. By replacing (or supplementing)
conventional air-cooling with higher-efficient liquid cooling, the overall operational
efficiency of the data center is improved.

Cooling
New Features and Solutions
Leak Sensing Technology
CPU1 Leak Detection Board

Coolant Tubes to external liquid cooling
system
Leak Detection Module

Detection Cable
Turtle Shell
Connect to LC RIO board
CPU2 Leak Detection Board
Liquid Cooling Module (Monolithic architecture) Liquid Cooling Module (Modular architecture)
Leak Sense technology provides customers with the knowledge that potential
issues are found and reported quickly.
If a coolant leak occurs, the system’s leak sensor logs an alert29 in the iDRAC
system.
29Three errors can be reported: small leak (warning), large leak (critical), leak
sensor error (warning – indicates the issue with the leak detection board) on the
iDRAC. These error detections can be configured to take meaningful actions using
tools like OpenManage Enterprise.

.
Cooling
POD Solution
POD solution containing two outer racks with node-level DLC and one middle In-Row Cooler.
A node-level DLC solution captures between 50%-60% of a server’s internal heat

(depending on the configuration).
The Dell EMC rack-level POD solution30 concept is designed for total heat capture.
The POD solution contains front and back containment for racks of DLC servers,
plus an In-Row Cooler that is integrated between the IT racks to capture any
remaining heat.
Liquid cooling systems follow two basic architectures.
Monolithic Architecture
The Dell EMC PowerEdge R650, R750, and R750xa follow the monolithic
architecture.
30 A pod or a cluster is a set of system units that are linked by high-speed networks
into a single unit.

Cooling
In the monolithic architecture, the Liquid Leak Sensor board connects to the
Complex Programmable Logic Device (CPLD) using the Liquid Cooling Rear I/O
board.
SPI-X 1- wire Alert

Cable
Complex Programmable
iDRAC Liquid Cooling Rear I/O Board Liquid Leak Sensor (LLS) Board
Logic Device (CPLD)
C2 Programming
Modular Architecture
The Dell EMC PowerEdge C6520 follows the modular architecture.
In the modular architecture, the Liquid Leak Sensor board connects directly to the
Complex Programmable Logic Device (CPLD).
SPI-X Alert Cable
Complex Programmable Logic Liquid Leak Sensor (LLS) Board

iDRAC
Device (CPLD)
C2 Programming

.
Cooling
Discussion Notes:
• The Liquid Cooling Rear I/O (LC RIO) board is a component specific to the
monolithic architecture only (PowerEdge R650, R750, and R750xa).
• A high-level overview of the liquid cooling process:
− Depending on the platform, the Liquid Leak Sensor (LLS) board, connects to
the immediate upstream entity (LC RIO board for the monolithic architecture
or the CPLD for the modular architecture) using an alert cable.
− The message is then forwarded to the iDRAC using the SPI-X registers, and
the error is logged.
• If a leak develops in a particular cold plate and the detection cable is not
engaged, then the alarm signal will not be received. A disengaged alert cable is
reported as an error in the iDRAC logs.
• Here, the context of a 'modular architecture' does not include blade servers.
The Dell EMC PowerEdge MX750c system does not support a liquid cooling
configuration.
Liquid Cooling Setup
Liquid cooling is supported internally and externally in the latest generation of

PowerEdge servers.
Click each tab to learn more.
Internal Support for DLC (with Leak Detection capabilities)
Within the sled, liquid cooling support is offered by:

• Cold Plates
• Liquid Leak Sensor (LLS) Board
• Liquid Cooling Rear I/O Board (specific to the monolithic architecture only)
• Liquid Cooling Module

Cooling
External Support for Liquid Cooling
External support for liquid cooling is common for both the monolithic and modular
architectures.
Outside the rack, liquid cooling support is offered by:

• Rack manifolds:
− Coolant tubes come out of each sled in the server and connect to a
manifold. The manifold further connects to a Coolant Distribution Unit.
− The racks for liquid-cooled systems must be at least 1200 mm (47.24
inches) deep x 750 mm (29.52 inches) wide to allow the installation of the
manifolds with the Power Distribution Units (PDUs).
• Coolant Distribution Unit (CDU):
− CDUs connect to the rack manifold to pump coolant to the racks and
exchange heat from the servers with facility water.
Liquid Cooling Connection Chain
Facility water at the CDU exchange heat with Manifolds connect Sled-internal liquid
customer site facility water servers to the CDU cooling system
Overview of the DLC with leak detection
Color code for cold water flow.
Color code for hot water flow.

.
Cooling
DLC Ecosystem
The image below shows the high-level overview of the DLC ecosystem.
Rack manifold
Liquid cooled
servers
Coolant
Distribution Unit
Coolant pipe assembly
DLC ecosystem
Discussion Block
Discussion Notes:
• CPU Cold Plates:

The CoolIT cold plates are a passive cooling solution that is managed by
centralized pumping architectures. Designed for use with the Intel Ice Lake
processors, the cold plates replace the traditional heatsink assembly.

Cooling
The cold plates are sold and supported by Dell EMC.

• Rack Manifold:
Made with reliable stainless steel and 100% nondrip quick
disconnects, rack manifolds can be arranged horizontally or vertically for a
manual connection at the front or rear of the rack.
• Heat Exchanger:
Also called as Coolant Distribution Units (CDU), the liquid-to-liquid heat
exchangers can support either one rack or a group of racks for a cooling
solution.
CoolIT Systems Rack DLC product line offers various Heat ExchangeModules
depending on load requirements and availability of facility water, including CHx
(Liquid-to-Liquid), AHx (Liquid-toAir), and custom options.
• Liquid Leak Sensor (LLS) Board:
A Liquid Leak Sensor (LLS) is a mechanism that detects leaks within the liquid
cooling system in the system.
The LLS mechanism can determine if a leak is small (0.02 ml) or large (0.2 ml).
• Liquid Cooling Rear I/O board:
The Liquid Cooling Rear I/O board is a new design, specific to the monolithic
architecture. The Liquid Leak Sensor (LLS) board connects to the Liquid
Cooling rear I/O board using an alert cable.
• Liquid Cooling Module:
The liquid cooling module assembly consists of:

x2 leak detection boards connected to the external liquid cooling system using
x2 coolant tubes.
An alert cable connecting the liquid cooling module to the Liquid Cooling rear
I/O board.
Benefits of Liquid Cooling
The benefits of DLC implementation are:

.
Cooling
• Increased System Cooling Capacity - DLC enables system configurations

that may not be possible with air cooling alone, such as high thermal design
power (TDP) CPUs, dense storage, and add-in cards.
• Improved Energy Efficiency (PUE) – The DLC cold plate solution reduces
energy costs by up to 45% relative to cooled air. It helps extend the life of
existing air infrastructure.
• Higher Compute Density – For the Intel Ice Lake based C6520 system, DLC
cooling supports up to 25% more cores per rack. For the AMD Milan based
C6525 system (with backplane configuration supporting storage drives), DLC
cooling enables 2x the core count over air-cooling alone.
• Swift Serviceability – The CPU DLC cold plate solution attaches with four
screws, making service quick and simple.
Fresh Air Environment
The Dell EMC Chiller-Less Fresh Air solution brings air into the data center from
the outside to support the cooling systems. The Dell Fresh Air Solution:
• It handles operating temperatures up to 45°c (113-degrees fahrenheits).

• It withstands up to 90 percent relative humidity.
• It eliminates compressor-based cooling during certain parts of the year.
Dell has validated a portfolio of servers, storage, networking, and power

infrastructure that has been developed for sustained operation at temperatures
ranging from 5°C (-23°F) to 45°C (113°F) and allowable humidity from 5 to 90
percent.
Dell Fresh Air 2.0 hardware includes specific configurations that can operate at
higher temperature and humidity levels and use clean outside air for air intake
instead of tightly controlled air conditioning (AC) from a cold aisle.
Fresh Air Environment Deployment Restrictions
The general configuration and device restrictions for deployment in a fresh air
environment are listed below.

Cooling
Click each icon to learn more.
Click here to learn more about Dell EMC PowerEdge thermals.
5
2
3
4
1:
• High-power PCIe cards (>75 W that use AUX cable, such as: GPU) Lower
power cards could also be excluded based on system limitations.
• Third-party PCIe card (any power levels).
2: Intel Optane DIMM.
3:
• Platforms with middrive and rear-drive configurations.

• PCIe SSD/NVMe drives.
• M.2/BOSS drives.
4: Tape backup unit (tape drives).
5: Power supply configuration that is not redundant.

.
Cooling
Presence of following devices prevents the system to be fresh air compliant:
• Power supply configuration that is not redundant.

• Intel Optane DIMM.
• High-power PCIe cards (>75 W that use AUX cable, such as, GPU) Lower
• High-power PCIe cards (>75 W that use AUX cable, such as, GPU) Lower
• Third-party PCIe card (any power levels).
• Tape Backup Unit (Tape Drives).
• Platforms with middrives and rear-drive configurations.
• PCIe SSD/NVMe drives.
• M.2/BOSS device.
• Each platform in addition has specific CPU TDP limits (that is CPU > x TDP is
not fresh air compliant) and is listed in platform-specific constraints.
• Status of the system fresh air compliance is reflected in iDRAC interface under
Cooling -> Temperature section.

Networking
Networking
Networking
Network Daughter Card
NDC used in the Dell EMC PowerEdge 12G, 13G, and 14G servers.
Older generation servers used a network interface card (NIC) built into the system
board. When upgrading or changing the NIC technology, users would install a PCIe
network interface controller in one of the PCIe slots in the server.

.
Networking
With the Dell EMC PowerEdge 12G, 13G and 14G servers, the NICs are based on
a daughter card.31 Users can easily change network requirements as they evolve.
A Dell EMC Network Daughter Card (NDC)32 enables the user to choose the right
network fabric without using a valuable PCI slot. It presents an easy upgrade path
from 1 GbE to 25 GbE LAN speeds.
Removal and Installation of an NDC in a PowerEdge R640

Server
Movie:
Link to video:
https://edutube.emc.com/Player.aspx?vno=/g0hzowboonHcUgIVGTkcw==&autopla
y=true
31 The Network Daughter Card (NDC) is a custom form factor mezzanine card that
contains a complete NIC subsystem.
32 The NDC typical includes the features and behavior of a traditional LOM (LAN on
Motherboard) subsystem. It includes the added benefit of flexibility in terms of

providing the choice to customers to choose their favorite network types, speed,
and vendors.

Networking
OCP Card
OCP card in a PowerEdge server.
The Open Compute Project (OCP)33 cards are network cards that connect to the
PCI bus. They are physically smaller than the Industry Standard Architecture (ISA)
expansion card and often connect to a dedicated connector on the system board.
The OCP card was introduced with the Dell EMC PowerEdge 15G servers.
The following are benefits of the OCP card:
• It is a removable networking card.

• Flexibility for customer to choose interconnect (10 GB, 24 GB, 50 GB, and so
on).
• It does not consume a regular PCIe slot.
33The Open Compute Project (OCP) is an organization that shares designs of data
center products and best practices among companies. The designs and projects
include server designs, data storage, rack designs, open networking switches, and
so on

.
Networking
• Replaces Network daughter Card (NDC) from previous generation servers.

• Physically smaller than the ISA expansion card, and often connect to a
dedicated connector on the system board.
Important: The OCP and the NDC cards are not a hot-swappable
component.
Removal and Installation of an OCP Card in a PowerEdge

R650 Server
Click the play button to view the process of removing and installing of an OCP
card in a Dell EMC PowerEdge server.
Movie:
Link to video:
https://edutube.emc.com/Player.aspx?vno=gN92Ty6wDims2t286i4/BQ==&autoplay
=true

Networking
SNAP I/O
The above image shows balanced I/O with SNAP I/O.
A Software-defined Network Accelerated Processing I/O or SNAP I/O network

interface controller consists of a PCIe card.
The SNAP I/O adapters enable both CPUs within a dual-socket server to connect
directly to the network through its own dedicated PCIe interface.
SNAP I/O results in low latency, CPU utilization and higher network throughput.
Some of the advantages of the SNAP I/O are:
• The NIC is directly connected to both CPU sockets.

• The SNAP I/O bypasses the Ultra Path Interconnect (UPI).
• It provides maximum use of the available resources.
Important: Only certain riser configurations in the Dell EMC

PowerEdge 15G platforms have SNAP I/O. For example, they are
available in both Dell EMC PowerEdge R650 and R750 but it depends
the riser configuration that is purchased by the user.
The card used in the 15G AMD servers (Dell EMC PowerEdge R6525
and R7525) is called SNAP I/O. Whereas the card that is used in the
Intel-based 15G servers (Dell EMC PowerEdge C6520) is called
SNAPI. They have the same architecture.

.
Networking
SNAP I/O Cards for PowerEdge Servers
Below are some of the images of the SNAP I/O cards.
SNAP I/O ConnectX-5 Card
The image below is of a SNAP I/O ConnecX-5 dual-port 100 GbE only adapter. It
supports PCIe Gen3/Gen4 x16. It is supported by the iDRAC and the Lifecycle
Controller.
SNAP I/O ConnecX-5.
SNAP I/O ConnectX-6 Card
The bottom-left image is of a primary SNAP I/O card, and the bottom right is an
auxiliary card. The left image is the SNAP I/O ConnectX-6 single port VPI HDR
adapter. It supports PCIe Gen4 x16 and PCIe Gen3 x32 (with auxiliary card). The
iDRAC and Lifecycle Controller do not support this card.

Networking
On the left, is a SNAP I/O ConnectX-6 single port VPI HDR adapter and on the right is an auxiliary
card.
SNAP I/O ConnectX-6 in a Dell EMC PowerEdge Server
The image below shows the SNAP I/O ConnectX-6 card along with the auxiliary
card which is installed in a Dell EMC PowerEdge server.
Primary card in Slot1 and connected to CPU1
Aux card in Slot4 and connected to CPU2
SNAP I/O ConnectX-6 card.

.
Networking
SNAPI
Applications Applications Applications Applications
Network
Socket Direct Card

Configuration
The above image shows SNAPI-capable NIC directly connected to both CPUs bypassing QPI and
UPI34. It frees up bandwidth for applications and improves latency.
Both SNAP I/O and SNAPI35 (also know as socket direct cards) are similar in how
they function. However, they connect to the CPU differently.
• SNAP I/O plugs into the system board.

• SNAPI uses a cable that connects to a riser.
The following components are required to enable SNAPI or SNAP I/O

configuration:
34 The Intel Ultra Path Interconnect (UPI) is a point-to-point processor interconnect.

UPI was developed by Intel which replaced the Intel QuickPath Interconnect (QPI)
in Xeon Skylake-SP platforms starting in 2017.
35 System for NUMA Aligned Partitioned I/O (SNAPI) allows an I/O device to
connect directly with multiple upstream CPU sockets. It bypasses inter-CPU socket
link usage and associated overheads such as NUMA latency penalty.

Networking
• SNAPI capable NIC.

• SNAPI capable risers.
• SNAPI cable that is connected to CPU 2 (In the Dell EMC PowerEdge C6520,
riser 1 B comes with a dedicated SNAPI cable connector that connects into the
system board).
Procedure to Replace the SNAPI Riser in a Dell EMC

PowerEdge C6520 Sled Within a PowerEdge C6400 Chassis.
Click the play button to see a video demonstrating the removal of the riser 1 which
supports the SNAP I/O module in the Dell EMC PowerEdge C6520 server.
Movie:
Link to video:
https://edutube.emc.com/Player.aspx?vno=mLFlKByDShaRqFXSpow4Pg==&autop
lay=true
Rear Input Output and LOM
The rear I/O and the LAN on motherboard (LOM) cards have been introduced with
the Dell EMC PowerEdge 15G servers.

.
Networking
LOM Card OCP Card Rear I/O
RIO and LOM cards on a system board.
The rear I/O card (RIO) consists of:
• iDRAC port36
• Video Graphics Array (VGA)
• USB
• ID Button
• Chassis intrusion switch cable
• Optional serial connector
36 RIO has an iDRAC port, but the iDRAC chipset is on the system board.

Networking
LOM network refers to the Ethernet connectivity provided to the compute sleds by
the I/O modules installed at the back of the PowerEdge servers. LOMs eliminate
the need for a separate network interface card to access a local area network.
The LOM is only available as a two port 1 GB card.
Dell EMC PowerEdge 15G servers support two NIC ports that are embedded on
the LOM card.
Important: When a user purchases a Dell EMC PowerEdge 15G

server with the Direct Liquid Cooling configuration, they get a different
RIO card. The alternate board does not come with a VGA port.
Different types of RIO cards used in a PowerEdge server which supports Direct
Liquid Cooling.
RIO card used in a non-liquid cooling configuration. Custom RIO card used in a Direct Liquid Cooling
configuration.

.
Networking
Removal and Installation of a LOM Card in the PowerEdge

R650 Server
Click the play button to view the process of removing and installing of a LOM card
in a Dell EMC PowerEdge server.
Link to video:
https://edutube.emc.com/Player.aspx?vno=s/qq+d59xdKAk2mnlwv0Pg==&autopla
y=true
Removal and Installation of a Rear I/O Card in the PowerEdge

R650 Server
Click the play button to view the process of removing and installing of a rear I/O
card in a Dell EMC PowerEdge server.
Link to video:
https://edutube.emc.com/Player.aspx?vno=N2oP6pLsbwLi8bfst8oSNg==&autoplay
=true

Graphic Processing Units (GPUs)
Accelerator Cards
Artificial Intelligence (AI) has different computing needs compared to traditional

CPUs. To fill intensive applications performance gaps, accelerators help to remove
server processing roadblocks.
GPUs
A Graphic Processing Unit (GPU) card offload compute-intensive portions37 of the

application to the GPU, while the remainder of the code still runs on the CPU.
Examples of GPU computation in a server:
• Vectored-floating point operations (games or movie rendering process).

• General purpose simulations (intensive number crunching).
37 A GPU typically has thousands of cores that are designed for efficient execution
of mathematical functions.

.
Application Code
Rest of Sequential CPU

Code
(Serial Tasks)
Compute-Intensive Functions
5% of Code
GPU CPU
Rest of Sequential CPU

Code
(Parallel Tasks) (Serial Tasks)
CPUs consist of minimal cores optimized for serial processing, while GPUs consist of thousands of
smaller, more efficient cores designed for parallel performance.
FPGAs
Field Programmable Gate Arrays (FPGA) accelerators are cards can be

dynamically reprogrammed with a data path that exactly matches workloads, such
as data analytics, image inference, encryption, and compression.
In the image example, Intel FPGA Accelerated Network Function moves load
balancing, QoS and classifying tasks away from the CPU load.

DATA IN Intel FPGA Accelerated Network

Functions
Video
Conferencing Load Balancing
AR/VR Quality of Service Network Functions

Classifying
Online Gaming
Forwarding
4K/8K
Multi-casting
Live Streaming
Smart Homes
Intel FPGA card programmed to augment the capabilities of Virtual Network Functions running on a
carrier cloud.
ASICs
Application-Specific Integrated Circuits (ASICs) are cards with silicon devices built
for specific purpose such as graph computing with massively parallel, low-precision
floating-point computing.
Intelligent Processor Unit (IPU) chips in Graphcore are an AI processor.
A Graphcore card with embedded IPUs, specifically designed for artificial intelligence.

.
Tip: The Graphcore IPU supports the PowerEdge R6525 and DSS
8440.
PowerEdge Server Accelerator Support
Dell Technologies supports a broad array of server accelerators in Dell EMC

PowerEdge servers.
Manufactu Model
rer
NVIDIA • Nvidia A100

GPU • Nvidia A40
• Nvidia A30
• Nvidia A10
• Quadro RTX 6000
• Quadro RTX 8000
• T4 Tensor Core
• Tesla V100/S
• Link: https://www.nvidia.com/en-us/data-center/a100/
AMD GPU • MI100

• Link: https://www.amd.com/system/files/documents/amd-cdna-
whitepaper.pdf

Intel FPGA • Stratix 10 SX

• Arria 10 GX
• Link:
https://www.intel.com/content/www/us/en/products/details/fpga/pl
atforms/pac.html
Xilinx • Xilinx U200

FPGA • Link: https://www.xilinx.com/products/boards-and-
kits/alveo/u200.html#solutions
Graphcore Link: https://www.graphcore.ai/products/ipu

IPUs
Example: Click here to view an example of GPU configuration.
Tip: GPU product documentation:

https://www.delltechnologies.com/en-us/servers/server-
accelerators.htm#tab0=0
Use Cases
Click each puzzle piece for information about use cases for GPUs.
Choosing GPUs and other accelerated architectures and products is a key decision
IT teams have in their hands. Once the decision is made, for the appropriate
workloads, then infrastructure strategy and product choices are addressed.

.
The NVIDIA CUDA Toolkit includes GPU-accelerated libraries, debugging and

optimization tools. C/C++ compiler, and runtime library for application deployment.
• Machine and deep learning: Accelerate AI by enabling the parallel processing

power required to speed both training and inferencing workloads.
• Accelerated database: Accelerate speed aggregations, sorts, and grouping
operations to solve complex analytics operations that overload traditional
databases.
• Predictive analysis: Accelerate dynamic correlation and predictive outcomes
with greater speed, accuracy, and scale.
• Streaming data: The Internet of Things (IoT) - accelerate simultaneous
ingestion, exploration, and visualization of streaming data for real-time analysis.
• Financial modeling: Accelerate the HPC and artificial intelligence (AI) industry to
leverage massive datasets to better understand risk and return.
• Modeling and simulation: Provide modeling and simulation for early evaluation,
fast testing of design modifications enabling more iterations.
• Signal processing: Enable providers to model and analyze signal data streams
coming in from computers, radios, videos, and cell phones in real time.
• Visualization: Enhance performance for 3D visualization applications such as
computer-aided design, enabling software to draw models in real time as the
user moves them.
• Seismic processing: Oil and Gas - accelerate extraction information from
massive seismic data stores, speeding time to results and lowering costs.

Compute Unified Device Architecture
CUDA blocks solve a sub- CUDA program with 8 blocks

problem by using finer
pieces with parallel
threads
CUDA runtime allocates these blocks to streaming

multiprocessors (SMs)
Larger GPU with 8 SMs - each SM gets 1 CUDA block
Smaller GPU with 4 SMs - each SM gets 2

CUDA blocks More powerful GPUs have more SMs, enabling performance scalability
CUDA concept showing a four SM GPU and an eight SM GPU. Source:

https://developer.nvidia.com/blog/cuda-refresher-getting-started-with-cuda/
Compute Unified Device Architecture (CUDA) is the NVIDIA parallel computing

platform and programming model.
CUDA divides work into small independent work and solves independently among
the CUDA blocks.
Tip: The NVIDIA CUDA Toolkit is a management software that

provides a development environment for creating high-performance
GPU-accelerated applications. To access the NVIDIA CUDA Toolkit,
go to https://developer.nvidia.com/cuda-downloads
To access the driver download portal of NVIDIA, go to
https://www.nvidia.com/Download/index.aspx
CUDA requires a supported version of Linux with a GCC complier and toolchain, or
Microsoft Windows or Microsoft Visual Studio, depending on the OS used.
The NVIDIA CUDA Toolkit includes GPU-accelerated libraries, debugging and

optimization tools. C/C++ compiler, and runtime library for application deployment.

.
CUDA Installation Instructions
The links that are provided below detail CUDA installation instructions by operating
system.
To access the NVIDIA CUDA

installation guide for Linux, go to
https://docs.nvidia.com/cuda/cuda-
installation-guide-linux/index.html
To access the CUDA installation guide

for Microsoft Windows, go to
https://docs.nvidia.com/cuda/cuda-
installation-guide-microsoft-
windows/index.html
To access the CUDA installation

instructions for VMware, go to
https://docs.vmware.com/en/VMware-
vSphere-Bitfusion/4.0/Example-
Guide/GUID-1E535E93-156A-4E27-
9DE3-0941F1D635EF.html
System Board Inlet Temperature
iDRAC UI, System > Cooling > Temperatures. Double-click image to enlarge.

The system board inlet temperature shouldbetween the minimum and maximum
warning threshold38.
System board temperature that exceeds thresholds degrades the GPU

performance.
If a system board inlet temperature warning message is logged, the GPUs lower
the power consumption39 to avoid thermal damage.
If the inlet temperature exceeds the critical threshold, a message is logged.
GPU Riser Module Removal - Video
Click play button to view the process of removing GPU Riser Module from a
PowerEdge Server,
Movie:
Link to video:
https://edutube.emc.com/Player.aspx?vno=UkyAL1BC2K50Wvk9kGQyKg==&auto
play=true
38 The range is optimal for GPU performance. The iDRAC sets the thermal warning
threshold when the GPU is installed.
39 Lowering the power consumption results in lower GPU performance.

.
Server Upgrade - GPU Kit
The GPU full-length kit, half-length kit, and the GPU power cable kit are kits
available for customers. Depending on the kit ordered, the respective components
are available.
Use SolVe to generate the upgrade procedure for the GPU kit.
SolVe Online home page: https://solveonline.emc.com/solve/home
Caution: Do not install GPUs, network cards, or other PCIe devices

on your system that are not validated and tested by Dell. Damage
caused by unauthorized and invalidated hardware installation will null
and void the system warranty.
Warning: Consumer-Grade GPUs should not be installed or used in

the Enterprise Server products.

Expansion Card
Expansion Card
Expansion Card
PCIe Overview
Lanes PCIe 3.0 Bandwidth / PCIe 4.0 Bandwidth
1.97 / 3.94 GBps
3.94 / 7.88 GBps
7.88 / 15.75 GBps
15.75 / 31.5 GBps
Theoretical bandwidth for PCIe lanes.
A PCIe40 (peripheral component interconnect express) is the bus that connects

supported peripheral components.
The slot size determines the maximum number of PCIe lanes.
Bandwidth scales linearly, so a x4 lane connection has twice the bandwidth of a x1

lane connection.
40 PCIe 3 and PCIe 4 are the bus types common today.

.
Expansion Card
An x1, x4, x8, or x16 card can use a x16 lane slot. A system board can have
multiple slot types and support different PCIe versions.
PCIe Slots and Processors

4 3 2 1
1: Riser 1, x16, CPU1
2: Riser 2 lower, x16, CPU1
2: Riser 2 upper, x16, CPU2

CPU2 CPU1
PowerEdge R750 CPU and PCIe lanes. The PowerEdge R750 supports many riser and PCIe lane
configurations.
The system board processors control the PCIe slots. Also, the system board
chipset may support PCIe slots.
Dell EMC PowerEdge 14G servers support PCIe 3. PowerEdge 15G servers
support PCIe 3 and PCIe 4.
For example: the PowerEdge R750, with the use of expansion card risers, can
support up to 48 x 4.0 PCIe lanes per CPU.

Expansion Card
PCIe Server Peripherals
Dell BOSS card.
Common PowerEdge server peripherals:

• BOSS
• PERC
• Network Interface Card (NIC)
• Accelerators:
− Graphics Processing Units (GPUs)41

− Field Programmable Gate Arrays (FPGAs)42
41 Co‑processors designed to accelerate compute performance.

.
Expansion Card
− Intelligence Processing Unit (IPU)43
PCIe Card Form Factor
The PowerEdge server supports many different PCIe card form factors. The
graphic shows the standard full-length, half-length, and low profile dimensions.
Also, PowerEdge servers may support other form factors such as half-length, and
half-height (HLHH).
Full-length and low profile:

312 mm (12.28")
PCIe Card Form factor Full height: 111.15 mm (4.38")
Low profile: 68.9 mm (2.7")
Half-length: 167.65
mm (6.6")
PCIe standard form factor dimensions.
42 Specific characteristics to run certain types of algorithms up to 1000X faster than

traditional software solutions.
43 Emphasize graph computing with massively parallel, low-precision floating point
computing.

Expansion Card
Risers
Riser examples. Supported risers for the PowerEdge R750xa.
Riser cards enable users to install additional expansion cards for the server.
Riser cards can accommodate all servers, including 1U servers.

.
Storage
Storage
Storage
Introduction to Server Storage
The Dell EMC PowerEdge servers support storage

options to access, manage, secure, and provide
compute services for the data.
The type of storage option varies based on the cost,

performance, size, and storage space. This topic
provides an insight into the different types of storage
devices that a PowerEdge server supports.

Storage
NVMe
Dell NVMe storage drive and NVMe card.
Non-Volatile Memory express (NVMe) is an open logical device interface for

accessing non volatile storage media that are attached through a PCI Express
(PCIe) bus or switch.
Advantages of NVMe storage devices are:

• Enables SAS (Serial Attached SCSI) links to communicate with flash media.
• Provides higher input and output operations per second.
• Eliminates the SCSI bottleneck caused due to array latency.
• Reduces the latency of flash-based infrastructure.
• Lowers power consumption of the server.

.
Storage
Tip: The NVMe devices may require installation of additional devices

such as PCIe extenders.
Paddle Card - 15G Only
Paddle cards are used to connect an NVMe backplane to the system board using
cables. The paddle card interfaces with the system board chipset.
When paddle cards are used, the onboard s150 is controlling the NVMe disks.
Paddle cards are similar to risers, but they do not have the riser cage. Paddle cards
provide efficient data management on systems with many storage devices.
The paddle cards are only available with certain riser configurations. Not all
systems will come with paddle boards, it is dependent on the users configuration.
For example, in the PowerEdge R750, the configuration that supports 24 X 2.5"
hard drives with the backplane has paddle cards for efficient management.
Click the highlighted area to view the paddle card.
Installing the R4 paddle card on PowerEdge R750xa.

Storage
1:
Paddle cards used in Dell EMC PowerEdge servers.
Removal and Installation of the Paddle Card
The video shows the process of removing and installing the paddle card on a Dell
EMC PowerEdge server.
Click the play icon to start the video.
Movie:
Link to video:
https://edutube.emc.com/Player.aspx?vno=s9rdvKggg4PdlphZBPRjNw==&autopla
y=true

.
Storage
Internal Dual SD Module
Internal Dual SD Module with dual SD cards.
The Internal Dual SD Module (IDSDM) provides a redundant SD-card module for
embedded hypervisors. The users configure the IDSDM for storage or as the
operating system boot partition.
Features of the IDSDM card are:
• Maintains mirror mode configuration by using SD cards in both slots to provide

redundancy.

Storage
• Provides high performance and data protection44.

• Provides active mode and standby mode options for dual SD cards.
• Supports single card operation without redundancy.
Important: When the redundancy option is set to Mirror Mode, the

information is replicated from one SD card to another.
The IDSM is not recommended for ESXi 7.0 due to write limitations.
Click here for more information.
The IDSDM also supports a vFlash45 card with an iDRAC Enterprise
license.
vflash only gets shipped with the IDSDM on 14G servers. The
PowerEdge 15G servers no longer support vflash with the iDRAC as its
been deprecated.
Removal and Installation of IDSDM
The video shows the process of removing and installing the IDSDM and vFlash
cards on the Dell EMC PowerEdge server.
44 The data is written on both cards, but the data is read from the first card. If the
first card fails or is removed, then the second card automatically becomes active.
45 vFlash cards provide a shared storage space between the server system and its
iDRAC.

.
Storage
Movie:
Link to video:
https://edutube.emc.com/Player.aspx?vno=Sa4eNZrdwA1doMbXQGmChg==&auto
play=true
Boot Optimized Storage Solution (BOSS)
Dell EMC offers two types of Boot Optimized Storage Solution (BOSS) cards.
Click each tab to learn more about the BOSS cards.
BOSS-S1
BOSS-S1 is a simple RAID solution card designed specifically for booting a

server's operating system.
Features of the BOSS-S1 Adapter Card Features of the BOSS-S1 Modular Card
1. SATA drive connector (2) 1. Modular BOSS-S1 card
2. 80 mm M.2 SATA drive 1 2. SATA drive connector
3. 80 mm M.2 SATA drive 2 3. 80 mm M.2 SATA drive
Features of BOSS-S1 are:
• Supports up to two 6 Gbps M.2 SATA drives.

• M.2 devices are read-intensive with 240 GB or 480 GB capacity.

Storage
• Creates a single virtual disk from the available hard drives.

• Available in the low-profile and full-height form factors.
Click here to learn more about the Dell EMC BOSS-S1 card through the Dell EMC
Boot Optimized Server Storage-S1 User's Guide.
BOSS-S2
Dell Technologies Boot Optimized Storage Solution-S2 (BOSS-S2) is a RAID

solution card that is designed for booting a server's operating system. The Dell
EMC 15G and future servers support BOSS-S2 cards. BOSS-S2 provides
enhancements over the BOSS-S1. Some of the features are:
Features of BOSS-S2 Card
1. M.2 blank
2. M.2 carrier
3. BOSS-S2 module bracket
4. BOSS-S2 module
5. M.2 card
6. BOSS-S2 card
7. Signal cable
8. Power cable
9. Power cable connector
10. Signal cable connector
• Supports up to two 80 mm (3.14 in) M.2 SATA solid-state devices.

• M.2 devices are read-intensive with 240 GB or 480 GB capacity.
• Provides a rear-facing module for quick and easy accessibility to the M.2 SSDs.
• Enables full hot-plug support.
• Connects using PCIe Gen 2.0 with four lanes.
• Supports two SATA generation 3 ports.

.
Storage
Click here to learn more about the Dell EMC BOSS-S2 through the Dell
Technologies Boot Optimized Storage Solution-S2 User's Guide.
Important: Configure and manage BOSS cards through the iDRAC,

System Setup Utility, OMSA, and BOSS CLI.
BOSS Management on iDRAC
BOSS Management on System Setup Utility

Storage
BOSS Management on OMSA
BOSS CLI Commands
Command Description Example Usage
info -o hba Display BOSS-S1 controller mvsetup info -o hba

information
info -o vd Display virtual drive mvsetup info -o vd

information
info -o pd Display physical drive ./mvcli info -o pd

information
smart -p <PD_ID> Display SMART information ./mvcli smart -p 0

of physical drive
event Display controller events ./mvcli event
RAID
Redundant Array of Independent/Inexpensive Disks (RAID) combines multiple hard

drive components into a single logical unit for the purposes of data redundancy and
data protection.

.
Storage
Data is distributed across the drives in several ways known as RAID levels. Based
on the customer requirements, the RAID levels can be configured for optimal
performance. Click here to learn more about available RAID levels and
specifications.
Click each RAID level to learn more.
1 2 3 4 5 6 7
1:
The features of RAID 0 are:
• Enables the system to write data across multiple hard

drives instead of one hard drive.
• Divides each hard drive storage space into 64 KB
stripes.
• Enhanced performance as multiple hard drives are
accessed simultaneously, but does not provide data
redundancy.
The advantages of RAID 0 are:

• Performance boost for read and write operations due to the striping of data
across multiple disks.
• Increases the total size of available space that is presented to the operating
system.
The disadvantages of RAID 0 are:

• Does not provide redundancy/duplication of data.
• The failure of one of the disks leads to entire data loss.
2:

Storage
• Data written to one disk is simultaneously written to

another disk.
• If one disk fails, the contents of the other disk can be
used to run the system and rebuild the failed hard
drive.
• Provides data redundancy as the contents of the disk
are written to a second disk.

• Improves read performance since different blocks of data can be accessed from
all the disks simultaneously.
• A multithreaded process can access block 1 from disk 1 and block 2 from disk 2
at once thereby increasing the read speed.
• Ideal for mission critical storage and hosting operating systems.

• Write performance is reduced since all the drives must be updated whenever
new data is written.
• Disk space is wasted to duplicate the data thereby increasing the cost to
storage ratio.
3:

.
Storage
• Combines distributed parity46 with disk striping.

• Provides redundancy for one hard drive failure without duplicating the contents
of entire hard drive.

• Provides high read performance.
• Efficient use of drive capacity.
• If one disk fails, the data is created from the parity information that was created
for redundancy.

• Longer rebuild of data as compared to RAID 0 due to parity recalculation.
4:
RAID 6 uses the concept of dual parity with block-level disk striping. RAID 6 allows
two disk failures without duplicating the contents of entire physical disks. The disk
46Parity data is redundant data that is generated to provide fault tolerance within
certain RAID levels.

Storage
capacity is calculated by n-2. If there are four disks, then the virtual disk capacity is
the total size of two disks.
The minimum number of disks required to configure RAID 6 is four. RAID 6 can
have a maximum of 32 drives.

• Provides data redundancy.
• The dual parity provides high read performance.

• Write performance decreases due to dual parity calculations.
• Additional cost required due to two disks dedicated parity.
5:
An example of RAID 10.
RAID 10 combines RAID 0 and RAID 1 with a minimum of four disks. In RAID 10,
two disks are striped and mirrored onto two other disks, creating a single array of
disk drives.

• High read performance.
• Good fault-tolerance.

.
Storage
• Provides faster data recovery due to data redundancy.

• Faster rebuild time as compared to other RAID levels.

• Expensive and complex to setup as compared to other RAID levels.
• Essentially uses only half of its storage capacity.
• One drive can fail from each mirrored set, but two drives cannot fail within the
same RAID 1 set. Failure of two drives in the same span leads to data loss.
6: RAID 50 (RAID 5+0), a type of nested RAID level, combines the block-level
striping of RAID 0 with the distributed parity of RAID 5.
The minimum number of disks required to configure RAID 50 is six.

• Multiple disk failures are tolerated before the data loss.
• Provides better fault-tolerance than other RAID levels.

• Failure of two drives in the same span leads to data loss.
7: Features of RAID 60 are:
• Parity is distributed across all drives in the array.

Storage
• Fault Tolerance - Disk errors, dual disk failures.

• Advantage - Data redundancy, high read performance.
• Disadvantage - Write performance decrease due to dual parity calculations.
Extra cost due to 2 disk equivalent devoted to parity.
Important: RAID levels can be configured through the Lifecycle

Controller, System Setup Utility, and OMSA.

.
Storage
RAID 0
Block 1 Block 2
Block 3 Block 4
Block 5 Block 6
Block 7 Block 8
Disk 1 Disk 2
RAID 0 uses the concept of striping that allows data to be written across multiple
hard drives instead of one physical disk. RAID 0 involves the partitioning of each
physical disk storage space into 64 KB stripes.
The minimum number of disks required to configure RAID 0 is two. RAID 0 can
• Performance boost for read and write operations due to the striping of data
across multiple disks.
• Increases the total size of available space that is presented to the operating
system.
• Does not provide redundancy/duplication of data.

• The failure of one of the disks leads to entire data loss.

Storage
RAID 1
Block 1 Block 1
Block 2 Block 2
Block 3 Block 3
Block 4 Block 4
Disk 1 Disk 2
RAID 1 uses the concept of data mirroring. Data is mirrored or cloned to other disks
so that if one of the disks fails, the other one can be used.
The minimum number of disks required to configure RAID 1 is two. RAID 1 can
• Improves read performance since different blocks of data can be accessed from
all the disks simultaneously.
• A multithreaded process can access block 1 from disk 1 and block 2 from disk 2
at once thereby increasing the read speed.
• Ideal for mission critical storage and hosting operating systems.
• Write performance is reduced since all the drives must be updated whenever
new data is written.
• Disk space is wasted to duplicate the data thereby increasing the cost to
storage ratio.

.
Storage
RAID 5
Parity
Generator A A P
0 1 a
B P B
0 b 1
P C C
c 0 1
D D P
0 1 d
Disk 1 Disk 2 Disk 3
RAID 5 uses the concept of distributed parity with block-level disk striping. RAID 5
stripes data blocks across multiple disks like RAID 0 while storing parity
information. The disk capacity is calculated by n-1. If there are three disks, then the
virtual disk capacity is the total size of two disks.
The minimum number of disks required to configure RAID 5 is three. RAID 5 can
• Provides high read performance.

• Efficient use of drive capacity.
• If one disk fails, the data is created from the parity information that was created
for redundancy.
• Longer rebuild of data as compared to RAID 0 due to parity recalculation.

Storage
RAID 6
Parity A A P Q
Generator 0 1 a a
B P Q B
0 b b 1
P Q C C
c c 0 1
Q D D P
d 0 1 d
Disk 1 Disk 2 Disk 3 Disk 4
RAID 6 uses the concept of dual parity with block-level disk striping. RAID 6 allows
two disk failures without duplicating the contents of entire physical disks. The disk
capacity is calculated by n-2. If there are four disks, then the virtual disk capacity is
the total size of two disks.
• Provides data redundancy.

• The dual parity provides high read performance.
• Write performance decreases due to dual parity calculations.

• Additional cost required due to two disks dedicated parity.

.
Storage
RAID 10
RAID 0
RAID 1 RAID 1
Block 1 Block 1 Block 2 Block 2
Disk 1 Disk 2 Disk 3 Disk 4
RAID 10 combines RAID 0 and RAID 1 with a minimum of four disks. In RAID 10,
two disks are striped and mirrored onto two other disks, creating a single array of
disk drives.

• Good fault-tolerance.
• Provides faster data recovery due to data redundancy.
• Faster rebuild time as compared to other RAID levels.
• Expensive and complex to setup as compared to other RAID levels.

• Essentially uses only half of its storage capacity.
• One drive can fail from each mirrored set, but two drives cannot fail within the
same RAID 1 set. Failure of two drives in the same span leads to data loss.

Storage
RAID 50
RAID 0
RAID 5 RAID 5
A A P A A P
0 1 a 2 3 a
B P B B P B
0 b 1 2 b 3
P C C P C C
c 0 1 c 2 3
D D P D D P
0 1 d 2 3 d
Disk 1 Disk 2 Disk 3 Disk 4 Disk 5 Disk 6
RAID 50 (RAID 5+0), a type of nested RAID level, combines the block-level striping
of RAID 0 with the distributed parity of RAID 5.
The minimum number of disks required to configure RAID 50 is six.
• Multiple disk failures are tolerated before the data loss.

• Provides better fault-tolerance than other RAID levels.
• Failure of two drives in the same span leads to data loss.

.
Storage
RAID 60
RAID 0
RAID 6 RAID 6
A A P Q A A P Q
B P Q B B P Q B
P Q C C P Q C C
Q D D P Q D D P
Disk 1 Disk 2 Disk 3 Disk 4 Disk 5 Disk 6 Disk 7 Disk 8
RAID 60 (6+0), a type of nested RAID level, combines the block-level striping of
RAID 0 with the dual distributed parity of RAID 6.
The minimum number of disks required to configure RAID 60 is eight.
• High data redundancy.

• High disk space consumption.
RAID Level Comparison
The table highlights the major differences between each RAID level.
RAID 0 RAID 1 RAID 5 RAID 6 RAID 10 RAID RAID 60

50

Storage
Descrip Data Disk Data Data RAID 0 RAID RAID 0

tion striping mirrorin striping striping and 0 and and
(no data g with with two RAID 1 RAID RAID 6
protectio distribute distribut combine 5 combine
n) d parity ed d combi d
parity ned
blocks
Number 1-32 2 3-32 4-32 4-192 6-192 8-192

of hard
drives
(PERC
9)
Number 1-32 2 3-32 4-32 4-240 6-240 6-240

of hard
drives
(PERC
10)
Fault None Single Single Double One per Disk Disk

Toleran disk disk disk mirror errors, errors,
ce failure failure failure set single dual disk
disk failures
failure
s
Benefit Highest Data Best Highest Highest Provid Data

perform protecti balance data perform es redunda
ance on of cost, protecti ance increa ncy,
through performa on with sed high
redunda nce, and through data fault read
ncy data redunda protectio toleran perform
protectio ncy n ce ance
n

.
Storage
Hot Spare
System Setup Utility displaying the hot spare options.
Hot spares47 are dedicated standby disks. When a hard drive that is used in a
virtual disk fails, the assigned hot spare48 is activated to replace the failed hard
drive without interrupting the system or requiring any intervention. When a hot
spare is activated, it rebuilds the data for all redundant virtual disks that were using
the failed hard drive.
47 A hot spare must be at least as large as the drive it is to replace, and a hot spare
must be the same drive type (SAS/SATA) as the drive it is to replace.
48 Hot spare cannot be assigned to 7200 RPM disks to replace 10 drives. It also
cannot be assigned to hard drives with large size.

Storage
The types of hot spare are:
• Global Hot Spare - Assigned to all virtual disks on the controller.

• Dedicated Hot Spare - Assigned to only one virtual disk on the controller.
• Persistent Hot Spare - Once enabled, any slots with hot spares configured
automatically become persistent hot spare slots49. If a hot spare disk fails or is
removed, a replacement disk that is inserted into the same slot automatically
becomes a hot spare with the same properties.
• Revertible Hot Spare - The replace member functionality allows a previously
commissioned hot spare to revert to a usable hot spare. When a disk failure
occurs within a virtual disk, an assigned hot spare, dedicated, or global, is
commissioned and begins rebuilding until the virtual disk is optimal.
Possible RAID Level Migrations
The table illustrates possible RAID Level Migrations.
Source Target Number of Number Capacity Description

RAID RAID Physical of Expansion
Level Level Disks Physical Possible
(Beginning) Disks
(End)
49The PERC 10 series can be configured so that the system backplane or storage
enclosure disk slots are dedicated as hot spare slots. This feature can be enabled
using the Dell OpenManage storage management application.

.
Storage
RAID 0 RAID 0 1 or more 2 or more Yes Increases

capacity by
adding disks.
RAID 0 RAID 1 1 2 Yes Converts a

nonredundant
virtual disk into
a mirrored
virtual disk by
adding one disk.
RAID 0 RAID 5 1 or more 3 or more Yes Adds distributed

parity
redundancy; at
least one disk
must be added.
RAID 0 RAID 6 1 or more 4 or more Yes Adds dual

distributed
parity
redundancy; at
least two disks
must be added.
RAID 1 RAID 0 2 2 or more Yes Removes

redundancy
while increasing
capacity.
RAID 1 RAID 5 2 3 or more Yes Maintains

redundancy
while adding
capacity.

Storage
RAID 1 RAID 6 2 4 or more Yes Adds dual

distributed
parity
redundancy and
adds capacity.
RAID 5 RAID 0 3 or more 2 or more Yes Converts to a

nonredundant
virtual disk and
reclaims disk
space that is
used for
distributed
parity data; one
disk can be
removed.

capacity by
adding disks.
RAID 5 RAID 6 3 or more 4 or more Yes Adds dual

distributed
parity
redundancy; at
least one disk
needs to be
added.

.
Storage
RAID 6 RAID 0 4 or more 2 or more Yes Converts to a

nonredundant
virtual disk and
reclaims disk
space that is
used for
distributed
parity data; two
disks can be
removed.
RAID 6 RAID 5 4 or more 3 or more Yes Removes one

set of parity
data and
reclaims disk
space that is
used for it; one
disk can be
removed.

capacity by
adding disks.
RAID 10 RAID 4 or more 6 or more Yes Increases

10 capacity by
adding disks; an
even number of
disks must be
added.
PERC Overview
The Dell EMC PowerEdge RAID Controller (PERC) is a series of RAID disk storage
controllers which support SAS, SATA hard drives, and Solid-State Drives (SSDs).
NVMe hardware RAID support is available with the PERC 11 (H755N front,
H755MX and H755 adapter).

Storage
H745 Adapter
1 Heat sink
2 Battery
3 Battery cable connector
4 SAS or SATA backplane connector B
5 SAS or SATA backplane connector A
6 PCIe Connector
H745 Front

.
Storage
1 PCIe cable connector
2 Battery
3 Heat sink
7 Power connector
H345 Adapter

Storage
1 Heat sink
4 PCIe connector
H345 Front

.
Storage
2 Heat sink
5 Power connector
H755 Adapter

Storage
1 Heat sink
2 PCIe connector
3 Battery
4 Backplane connector A
5 Backplane connector B
H755 Front

.
Storage
1 Battery
2 PCIe input connector
3 Power card edge connector
4 Heat sink
H755 NVMe

Storage
1 Battery
4 Heat sink
H755 MX
The PERC11 controller introduces new features that boost performance. PERC11
supports the PCIe Gen4 host interface and the upgraded DDR4 8GB 2666MT/s
cache memory. However, the greatest addition to this generation of technology is

.
Storage
the inclusion of NVMe hardware RAID support. NVMe hardware RAID support is
available on the H755N front, H755MX and H755 adapter form factors.
Important: The PERC H755MX does not support the MX5016s

storage sled. The customers want to use the MX5016s should use the
HBA300MMZ (manages internal disks only) or jumbo PERC
(manages both internal and storage sled disks).
The Dell EMC PowerEdge RAID Controller (PERC) is a series of RAID disk storage
controllers which support SAS and SATA hard drives, and Solid-State Drives
(SSDs). NVMe hardware RAID support is available with the PERC 11 (H755N
front, H755MX and H755 adapter).
Click each PERC card image to learn more.
2 1
3 6
4
7
5 8
1: Features of PERC H755 Adapter

Storage
1 Heatsink
2 PCIe connector
3 Battery

.
Storage
1 Heatsink
2 Battery
6 PCIe connector
3: Features of PERC H745 Front

Storage
2 Battery
3 Heatsink
7 Power connector

.
Storage
1 Heatsink
4 PCIe connector
5: Features of PERC H345 Front

Storage
2 Heatsink
5 Power connector
6: Features of PERC H755 Front SAS

.
Storage
1 Battery
2 PCIe input connector
4 Heatsink
7: Features of PERC H755N Front NVMe

Storage
1 Battery
4 Heatsink
8: PERC H755 MX
NVMe support requires universal backplane.

.
Storage
The PERC H755MX does not support the MX5016s storage sled. The customers
want to use the MX5016s should use the HBA300MMZ (manages internal disks
only) or jumbo PERC (manages both internal and storage sled disks).
The graphic above details the PERC 10 and 11 series.
NVMe Support with PERC 11
OS
SCSI
VD
PERC
Root
Complex
NVMe NVMe
Drives Drives
NVMe RAID
The PERC 11 features that support NVMe are:

• The PERC 11 supports Dell PCIe Gen3 and Gen4 NVMe devices.
• The PERC 11 controller supports up to 8 direct-attached NVMe drives. Links to
devices are x2 width for all NVMe Devices, even for drives that support x4.
• The PERC H755N is a non-transparent bridge that acts as a root port to NVMe
drives.
• A virtual disk (VD) made of NVMe drives is displayed as SCSI VD to the host
operating system.

Storage
• A PERC translates the SCSI instructions and passes the instructions to the
NVMe drives.
• Windows Device Manager lists all the NVMe drives.
Virtual disks
Non-RAID disks
The image shows how the Windows Device Manager lists the NVMe drives.
PERC Configuration Modes
In the Dell EMC 15G servers, PERC has two options for enclosure configuration
mode: Unified Mode and Split Mode.
• The Unified Mode is the default mode.

• Use Split Mode for a high-performance with no failover or High Availability (HA)
functionality.
Click each number to learn how to reset the enclosure mode through the PERC.
Edit enclosure mode
Go to Storage Configuration. Expand Enclosure Configuration.

.
Storage
Select the mode
In Unified Mode, all 24 drives are connected to the front controller.
The split mode is indicated as <X:Y>. By default, the split mode is a <12:12> split.
X slots are assigned to one controller and Y slots are assigned to a different
controller.
Once the mode is selected, click Add to Pending Operations.

Storage
Apply Changes
Once added to pending operations, click Apply Now to initiate the configuration
operation.
After the job is completed, a cold reboot is required to apply the changes.

.
Storage
PERC Card Matrix
Click each tab to view the PERC 10 and PERC 11 cards technical specification.
PERC 10 Specifications
Feature PERC H345 PERC H745 PERC H745P MX
RAID Levels 0, 1, 10 0, 1, 5, 6, 10, 50, 60 0, 1, 5, 6, 10, 50, 60
Cache memory N/A 4 GB DDR4 2133 8 GB DDR4 2133

Mhz cache Mhz cache
Drives type 3 Gbps SATA, 6 3 Gbps SATA, 6 3 Gbps SATA, 6

Gbps SATA or SAS, Gbps SATA or SAS, Gbps SATA or SAS,
and 12 Gbps SAS and 12 Gbps SAS and 12 Gbps SAS
PCIe support Generation 3 Generation 3 Generation 3

Technical Specification of PERC 10 Cards.
PERC 11 Specification
Feature PERC H755 PERC H755 Front PERC H755N Front

Adapter SAS NVMe
RAID Levels 0, 1, 5, 6, 10, 50, 60 0, 1, 5, 6, 10, 50, 60 0, 1, 5, 6, 10, 50, 60
Cache memory 8 GB DDR4 2666 8 GB DDR4 2666 8 GB DDR4 2666

MT/s cache MT/s cache MT/s cache
Drives type 3 Gbps SATA, 6 3 Gbps SATA, 6 Gen3 (8 GT/s) and

Gbps SATA or SAS, Gbps SATA or SAS, Gen4 (16 GT/s)
and 12 Gbps SAS, and 12 Gbps SAS NVMe
Gen3 (8 GT/s), and
Gen4 (16 GT/s)
NVMe
PCIe support Generation 4 Generation 4 Generation 4

Technical Specification of PERC 11 Cards

Storage
For detailed specifications on the PERC cards visit dell.com/support.
Ask: List few PERC cards from PERC 10.6 series which support 14G and 15G
both.
Accept answers and discuss. Ensure that the following points are covered:
• PERC 10.6 supports SAS or SATA Only.

• H345 Controller has no battery nor DDR. The PERC H345 supports only RAID
0,1,10 and SysPDs.
• H740/H745/H840 has two modes, RAID Mode and eHBA mode.
− RAID Mode: All RAID Levels. VDs' start at VD0 and counts up.
− eHBA Mode: RAID 0, 1, 10 and EPD-PTs. VDs' start at VD239 and counts
down.
• The PERC 11.1 supports SAS/SATA or NVMe depending on the configuration
of the system.
• Single RAID mode that supports all RAID levels and EPD-PTs.
− VDs' start at VD239 and counts down.

.
Storage
Removal and Installation of PERC
The video shows the process of removing and installing of the rear-loading PERC
module on the Dell EMC PowerEdge R750xa.
Movie:
Link to video:
https://edutube.emc.com/Player.aspx?vno=Hgc9SuXuvDTi9nC5Z05xxA==&autopla
y=true

Storage
Backplanes
Dell EMC PowerEdge backplane.
A backplane is a high-speed communication circuit board that provides a single

interface for storage devices, better logical control, and faster connection speeds.
Some of the features of the server backplane are:
• HDDs/SSDs connect to the front of the backplane.

• SATA/SAS/NVMe cables with power and I2C data cables connect to the rear of
the backplane.
• Drives fail to power on when the backplane loses power connection.
• System fails to connect to the storage drives if the connection between the
backplane and the system controller is interrupted.
• The backplane configuration may differ based on the system design.

.
Storage
Removal and Installation of Backplane
The video shows the process of removing and installing the backplane on the Dell
EMC PowerEdge R750 server.
Movie:
Important: Make sure to remove all the cables that connect to the
system backplane before removing the backplane from the system.
Link to video:
https://edutube.emc.com/Player.aspx?vno=Nh7UK+Zx7sJKmki4oH/bzQ==&autopl
ay=true

Introduction to Server Security

Storage
Introduction to Server Security
Server Security Overview
Server security focuses on protecting the data and resources that are stored in
servers.
Server security is critical for securing the IT infrastructure.
The security approach is built-in and not bolted-on50.
The latest PowerEdge servers support an enhanced cyber resilient architecture

that uses silicon-based Root of Trust (RoT)51 to increase the server security.
Click each image for more information.
50 The modern IT security approach is engineered and integrated as an entity, not

bolted on as an afterthought.
51 Root of Trust is a concept that starts a chain of trust needed to ensure computers
boot with legitimate code.

Storage
1 3 2 4 5 6 7 8
1: Cryptographically Verified Trusted Booting: Anchors end-to-end server

safety and overall data center security. It includes features like silicon-based RoT,
digitally signed firmware, and automatic BIOS recovery.
2:
3: Secure Boot: Checks the cryptographic signatures of Unified Extensible

Firmware Interface (UEFI) drivers and other code that is loaded before the
Operating System runs.
4: iDRAC Credential Vault: A secure storage space for credentials, certificates,

and other sensitive data that is encrypted with a silicon-based key unique to each
server.
5: Dynamic System Lockdown: Secures any system configuration and firmware

from malicious or unintended changes while alerting users to any attempted system
changes.
6: Enterprise Key Management: Delivers a central key management solution to

manage data-at-rest across the organization.
7: System Erase: Allows users to easily retire or repurpose the latest PowerEdge
servers by securely and quickly wiping data from storage drives and other
embedded nonvolatile memory.
8: Supply Chain Security: Provides supply chain assurance by ensuring there is

no product tampering or counterfeit components before shipping products to the
users.

.
Storage
• Cryptographically Verified Trusted Booting: Anchors end-to-end server

safety and overall data center security. It includes features like silicon-based
RoT, digitally signed firmware, and automatic BIOS recovery.
• Secure Boot: Checks the cryptographic signatures of Unified Extensible
Firmware Interface (UEFI) drivers and other code that is loaded before the
Operating System runs.
• iDRAC Credential Vault: A secure storage space for credentials, certificates,
and other sensitive data that is encrypted with a silicon-based key unique to
each server.
• Dynamic System Lockdown: Secures any system configuration and firmware
from malicious or unintended changes while alerting users to any attempted
system changes.
• Enterprise Key Management: Delivers a central key management solution to
manage data-at-rest across the organization.
• System Erase: Allows users to easily retire or repurpose the latest PowerEdge
servers by securely and quickly wiping data from storage drives and other
embedded nonvolatile memory.
• Supply Chain Security: Provides supply chain assurance by ensuring there is
no product tampering or counterfeit components before shipping products to the
users.
Trusted Platform Module
The Trusted Platform Module (TPM) is a hardware security device that provides the
server with the ability to create cryptographic keys. The cryptographic keys are
used for encryption and decryption.

Storage
TPM employs a specialized chip52 that stores protected key information in a

tamper-proof medium to authenticate the host system hardware.
When TPM is enabled on a device, the resident operating system works together
with the device to encrypt the hard drives. TPMs are passive devices.53 So, they
do not have the intelligence to communicate.
The TPM cannot be removed from one system board and installed on another
system board.
52 The chip includes a unique endorsement key that is baked into the module
during manufacturing, like a digital fingerprint to establish the trustworthiness of
data and applications. This cross-platform solution engages at the lowest level of
system operation, protecting against unauthorized firmware and software
modifications that can undermine system integrity.
53 TPMs only receive commands and return responses.

.
Storage
1 2
3 4
1: Hashing: Used to convert the input (string of characters) of any length to a fixed
size value which represents the original string using an algorithm.
2: Random Number Generation: A technique of generating a sequence of

numbers/symbols that cannot be predicted by anything other than a random
chance.
3: Asymmetric Key Generation: A technique of generating two keys - a public key

and a private key which are used in cryptography.
4: Asymmetric Encryption/Decryption (Public Key Cryptography): This

technique uses the public key (known to all users) and the private key (known only
to the owner) for encryption and decryption.
The TPM security features are:

• Hashing: Used to convert the input (string of characters) of any length to a fixed
size value which represents the original string using an algorithm.
• Random Number Generation: A technique of generating a sequence of
numbers/symbols that cannot be predicted by anything other than a random
chance.

Storage
• Asymmetric Key Generation: A technique of generating two keys - a public

key and a private key which are used in cryptography.
• Asymmetric Encryption/Decryption (Public Key Cryptography): This
technique uses the public key (known to all users) and the private key (known
only to the owner) for encryption and decryption.
Trusted Platform Module 2.0
Introduction to TPM 2.0
TPM 2.0 supports newer hash algorithms,

SHA-256, which can improve driver signing
and key generation performance.
SHA-256 is 256 bits or 32 bytes whereas

SHA-1 is 20 bytes. The SHA-1 algorithm is
being deprecated in favor of stronger
algorithms, such as SHA-256.
The advantages of TPM 2.0 over TPM 1.2

are:
• TPM 2.0 supports the SHA-256 hashing
algorithm, whereas TPM 1.2 supports
only SHA-1, which has known
vulnerabilities.
• TPM 2.0 also supports newer operating
systems: Windows Server 2016 and later
versions as well as Ubuntu 16.04 and
later versions.
TPM 2.0 is not fully supported in legacy BIOS mode because there is no pointer
to TPM logs in legacy BIOS mode.
TPM 2.0 Hierarchies
Each domain or hierarchy of TPM has its own resources and controls.

.
Storage
TPM 2.0 supports three domains:

• Security:
Storage Hierarchy- The storage hierarchy is used by the end user or by the
enterprise IT department for general cryptographic usage.
• Platform:
Platform Hierarchy- The platform hierarchy is under the control of the platform
manufacturer. The hierarchy includes functions that protect platform or firmware
integrity.
• Privacy:
Endorsement Hierarchy- The endorsement hierarchy is used when the user has
privacy concerns. The endorsement administrator has access to some
protected TPM commands and functionalities.
Configuring the TPM
Different settings are used on Windows Server 2012R2, Windows Server 2016 and
later versions to match the operating system capabilities.
Windows Server 2016 and later versions:
The BIOS settings need certain modifications to fully leverage the Windows Server
2016. Modifying the BIOS enables the server for the TPM guarded host

Storage
deployment that is required to run shielded virtual machines. Guarded hosts54 and
shielded Virtual Machines55 are new to Windows Server 2016 and later versions.
Accessing the TPM using the System Setup menu.
Silicon-Based Hardware Root of Trust (RoT)
Silicon-based RoT is the latest security feature implemented on PowerEdge

servers. The concept is rooted in the foundational security and protection of each
PowerEdge server.
54 Guarded hosts can run shielded Virtual Machines (VMs).

55 Shielded VMs are supported by a set of security controls that protect against
rootkits and bootkits.

.
Storage
The PowerEdge 15G servers use a Silicon-based RoT to attest to the integrity of
the code running. The servers ensure that no unauthorized BIOS or firmware codes
run. If the code is replaced with malware, the server cannot execute the code.
The iDRAC is responsible for RoT and verifies BIOS SPI code before allowing host
chipset & CPU to run any code.
RoT Purpose
The silicon-based RoT starts a chain of trust to ensure systems boot with a
legitimate BIOS code. If the performed BIOS code has been verified as legitimate,
those credentials are trusted by the execution of each subsequent code.
RoT Operation
1. On a server, the silicon chip acts to validate that the BIOS is legitimate by
checking its encrypted signature.
2. This encrypted signature (a Dell EMC encryption key) is burned into silicon
during the manufacturing process and cannot be changed.
The only way to make the Root of Trust robust is to do it in hardware. The read-
only encryption keys are burned into PowerEdge servers at the factory. These keys
cannot be changed or erased. When the server powers on, the hardware chip
verifies that the BIOS code is legitimate from Dell EMC using the key that is burned
into silicon in the factory.
Verifying the RoT
A failure to verify that the BIOS is legitimate results in a shutdown of the server and
the user is notified in the log. The BIOS recovery process can be initiated by the
user. If the RoT is validated successfully, the rest of the BIOS modules are
validated by using a chain of trust procedure until control is handed off to the
operating system or the hypervisor.
BIOS Live Scanning
• The BIOS Live Scanning feature enables users to scan the system BIOS once
POST is completed. This task can be run once or can be set up on a schedule.

Storage
• The scan period could be once a week, once a month or once in a year
(adjustable by end user).
• The BIOS Live Scanning is a licensed feature and is available only with iDRAC
Datacenter license.
Image of the iDRAC UI with the BIOS Live Scanning option highlighted.

.
Storage
Intel Boot Guard
The 14G and the 15G PowerEdge servers support the Intel Boot Guard verified
boot feature. Boot Guard protects the server BIOS.
• Basic Input Output System (BIOS) is implicitly a critical element of any solution
stack that includes risks56 while updating.
• The BIOS persists between power cycles, becoming a potentially attractive
target for malicious attacks.
• Attacks against the BIOS are typically hard to detect. Attacks run before the
operating system and other security software loads. This mechanism leaves a
platform or organization exposed to further threat or performance issues.
1: What is the function of Boot Guard?
56 Due to this, some users hesitate to perform scheduled updates during a server
life cycle.

Storage
Boot Guard is a processor feature that prevents the system from running the
firmware images that are not released by the manufacturer. It also allows the BIOS
or UEFI to verify that the BIOS is not compromised before booting.
2: How does Boot Guard operate?
In the Boot Guard verification method, the CPU compares the current BIOS or
UEFI firmware image with an official hash-generated version of the image that is
stored on PowerEdge servers.
3: How to identify Boot Guard occurrence?
If a Boot Guard event is activated, the BIOS or UEFI feature is immediately

activated and an attempt to recover a backup BIOS or UEFI occurs.
Discussion
What is the function of Boot Guard?
Boot Guard is a processor feature that prevents the system from running the
firmware images that are not released by the manufacturer. It also allows the BIOS
or UEFI to verify that the BIOS is not compromised before booting.
How does Boot Guard operate?
Discussion
Dell's BootGuard verification method involves comparing the BIOS image against
the official hash that is generated and stored on Dell's servers.
Basic Input Output System (BIOS) is implicitly a critical element of any solution
stack and since the BIOS persists between power cycles it poses a potentially
attractive target for malicious attacks. Attacks against the BIOS are typically hard to
detect because they run before the operating system and other security software
loads.

.
Storage
Due to the critical nature of the BIOS and the perceived risks of updating, some
customers hesitate to perform scheduled updates during a server lifecycle. This
can leave a platform or organization exposed to even further threat or performance
issues. For this reason, we have implemented multiple new features including Boot
Guard.
Protection at the Chipset: The Dell EMC 14th generation of PowerEdge servers
supports the Intel Boot Guard verified boot feature. The Boot Guard extends the
platform root of trust to the Platform Controller Hub (PCH). The PCH contains One-
Time Programmable (OTP) fuses that are burned by the Dell EMC factory during
the manufacturing process with the selected Boot Guard policy and the hash of the
Master Public Key. The Key Manifest on the BIOS SPI flash is signed by the Dell
EMC Master OEM key, and delegates authority to the Boot Policy Manifest key.
Then the Boot Policy Manifest authorizes the Initial Boot Block (IBB), which is the
first BIOS code module to run at the reset vector. If the IBB fails authentication,
Boot Guard shuts down the system and does not allow it to boot. Each BIOS
module contains a hash value of the next module in the chain and uses the hash
value to validate the next module. The IBB validates (SEC+PEI) before handing off
control to it. The (SEC+PEI) then validates (PEI+MRC) and (PEI+MRC) further
validates the (DXE+BDS) modules. After that point, the UEFI Secure Boot, if
enabled, can extend the root of trust to the remaining BIOS, third-party UEFI
drivers, and operating system loader.
Boot Guard Process
Boot Guard - Protection at the Chipset
1. The Boot Guard extends the platform RoT to the Platform Controller Hub
(PCH).
2. The PCH contains One-Time Programmable (OTP) fuses that are burned at the
Dell EMC factory during the manufacturing process.
3. The OTP contains the selected Boot Guard policy and the hash of the master
public key.

Storage
4. The key manifest on the BIOS SPI flash is signed by the Dell EMC master
Original Equipment Manufacturer (OEM) key and delegates authority to the boot
policy manifest key.
5. Each BIOS module contains a hash value of the next module in the chain and
uses the hash value to validate the next module.
Boot Guard Event in the LCC Log
If the Boot Guard event detects any issue in the BIOS image before booting, it
immediately activates the BIOS or UEFI recovery feature and attempts to recover a
backup BIOS or UEFI.
The Boot Guard event and the subsequent events that perform the BIOS or UEFI
recovery are captured in the Lifecycle Controller log as highlighted in the image.
BIOS recovery logs in the Lifecycle Controller.

.
Storage
Automated BIOS Recovery
If BIOS is corrupted57, a method to recover the BIOS is needed.
A BIOS/UEFI recovery can be initiated in two ways, either through Boot Guard or
when the BIOS detects corruption.
There are two BIOS ROMs in the system, one that is 32MB (for the normal full-
sized BIOS) and another 16MB recovery ROM.
The iDRAC stores a backup BIOS image for BIOS recovery.
The iDRAC orchestrates the entire end-to-end recovery process.
The BIOS recovery occurs in three steps:

1. The iDRAC extracts the recovery BIOS image from a Dell Update Package
(DUP) saved in the control panel during factory install. The DUP programs the
recovery ROM to boot instead of the traditional BIOS.
2. Once the system boots the recovery ROM58, a BIOS update job is launched to
flash the primary ROM with the full BIOS image using the latest BIOS DUP
known to the system.
3. After the BIOS update completes, the recovery ROM is disabled, the system
reboots back to the primary ROM.
57 BIOS corruption can either be due to a malicious attack, due to a power loss
during the update process, or due to any other unforeseen event.
58 Read Only Memory

Storage
Tip: For more information about automated BIOS recovery, go to 15G

Boot Guard and Automatic BIOS recovery page in dell.com/support
Secure Boot
As software security breaches are becoming more frequent and incognitive, system
administrators must deploy a wider variety of defenses, such as Secure Boot.
The UEFI Secure Boot is a technology that secures the boot process by verifying if
the drivers and operating system loaders are signed by the key that is authorized
by the firmware.
1: What is Secure Boot?
Secure Boot is a system BIOS feature that guards against attacks by preventing
the execution of unauthorized code in the preboot environment. It provides an
improved way for the BIOS to authenticate each component in the system using
certificates or policies during the boot process.
2: How does Secure Boot operate?
The BIOS authenticates each module that is run during the boot process using
certificates in the Secure Boot policy. Before the system BIOS loads a module into

.
Storage
memory, Secure Boot checks whether the module has the authorization to run the
system. This is done by launching various code modules, such as: device firmware,
diagnostics, and operating system loaders.
3: How to identify Boot Guard occurrence?
If a Boot Guard event is activated, the BIOS/UEFI feature is immediately activated

and an attempt to recover a backup BIOS/UEFI occurs.
What is Secure Boot?
Secure Boot is a system BIOS feature that guards against attacks by preventing
the execution of unauthorized code in the preboot environment. It provides an
improved way for the BIOS to authenticate each component in the system using
certificates or policies during the boot process.
How does Secure Boot operate?
The BIOS authenticates each module that is run during the boot process using
certificates in the Secure Boot policy. Before the system BIOS loads a module into
memory, Secure Boot checks whether the module has the authorization to run the
system. This is done by launching various code modules, such as device firmware,
diagnostics, and operating system loaders.
How to identify Boot Guard occurrence?
If a Boot Guard event is activated, the BIOS/UEFI feature is immediately activated

and an attempt to recover a backup BIOS/UEFI occurs.
Secure Boot Policy
The Secure Boot policy allows a user to specify the policy or digital signature that
BIOS uses to authenticate. The policy can be classified as:
• Standard: BIOS uses the default set of certificates to validate the drivers and
operating system loaders during the boot process. By default, the Secure Boot
Policy is set to Standard.
• Custom: BIOS uses the specific set of certificates that can be imported or
deleted from the standard certificates to validate the drivers and operating
system loaders during the boot process.

Storage
The Secure Boot policies in the latest technology of Dell EMC servers are
described in terms of various modes.
Setup Mode User Mode Audit Mode Deployed Mode
In this mode, the This works similar to setup This mode is the most
This uses a Dell factory,
customer can provide mode, except for policy secure and requires
randomly-generated,
their private key and violations. If there is a someone to physically
private key to
authenticate their policy violation, this does be at the box to modify
authenticate all cards
environment. not stop the system from any policies.
that are approved by
booting and will record the
Dell EMC.
failure in the LC log.
Secure Boot Policy Modes
Secure Erase for Self Encrypting Disks (SEDs)
Secure Erase is the process of permanently erasing all data on Self-Encrypting

Disks (SEDs) and resetting their security attributes.
Secure Erase is used to reset the security attributes when an SED is inaccessible
due to lost or forgotten paraphrase.
The process of Secure Erase can be identified when data on SEDs is completely
erased and reset to the default state.
Important: Instant Secure Erase (ISE) is used to instantly erase user

data. The PowerEdge 14G and 15G servers offer ISE-capable drives
by default.
ISE drives use the same encryption technology as SEDs but do not
allow the encryption key to be secured. The encryption technology
allows the drive to be repurposed and securely erased using the
cryptographic erase function.

.
Storage
System Lockdown mode in iDRAC
When OpenManage Essentials introduced the notion of configuration drift for both
hardware changes and firmware changes, customers began asking if there was a
way to prevent any changes from happening in the first place. This is how
Lockdown Mode started. The Lockdown option is selected in the iDRAC GUI and
used to prevent any changes to firmware or hardware settings during normal
operations.
When the System Lockdown Mode is enabled, only some configuration changes
are allowed.
Image shows the system lockdown mode in the highlighted box.
Important: The Lifecycle Controller cannot be booted using the F10

key as the key is disabled and therefore no firmware updates can
occur. Users can enter the BIOS on POST by using the F2 key. But,
users cannot make any changes since read only mode is enabled.

Storage
Dell Technologies Secured Component Verification
Dell Technologies Secured Component Verification (SCV) is a supply chain

assurance offering that enables users to verify that the PowerEdge server received
on order, matches the factory manufactured model.
Order Build Collect Store Deliver Validate
System's component data is Dell signed certificate User validates the

Order is sent to Platform certificate is
User places order authenticity of the
collected, encrypted and a stored in a central is part of the iDRAC
for server products the factory for components via the
platform certificate is created location and in the chipset. .
the build. iDRAC.
and signed by a High iDRAC.
Security Module (HSM).
Secured Component Verification process
How does the SCV work?

• To validate components in a certificate containing the unique system
component IDs is generated during factory assembly process.
• This certificate is signed in the Dell factory and is stored in iDRAC9, later used
by the SCV application.
• The SCV application validates the system inventory against the SCV certificate.
Tip: For more information on Secured Component Verification (SCV),

visit the Dell EMC Secured Component Verification Reference Guide
for Servers page in dell.com/support

.
Configuration Validation

Storage
Configuration Validation
Overview
Configuration Validation is a feature in PowerEdge servers

that works to modularize interchangeable pieces of a server,
and allows the user to manage the varying, leveraged parts
into unique configurations.
Configuration Validation:
• Ensures a system is aware of the hardware in the server.

• Holds information about Self-Describing I/O (SDIO) of backplanes, risers and
internal cards (PERC, BOSS and so on).
• Provides detailed information on the source and destination of internal chassis
cable connections.
• Utilizes the unique identifier given to each piece of hardware in system, so
users can recognize each piece individually.
The Configuration Validation feature enables the iDRAC platform to:
• Collect the precise configuration of the hardware components.

• Compare the collected configuration with the expected configuration, which
allows the user to identify errors in platform configuration.
Important: Configuration Validation is available only in the

PowerEdge 15G servers.

.
Storage
Hardware Configuration Error Messages
Configuration Validation displays the following error messages when a component

is misconfigured.
• Error Message - HWC8010: Occurs when there are one or two issues in the
configuration.
• Error Message - HWC8011: Occurs when there are multiple issues in the
configuration.
The product specific user manuals provide additional details about these errors.
Error Messages
Error Message HWC8011.
Error Message HWC8010.

Storage
Error Messages Resolved
Message when error HWC8010 is resolved.
Message when error HWC8011 is resolved.
HWC8010 and HW8011 Error Interpretations
The following table highlights the HWC8010 and HW8011 error messages along
with the interpretation of the error.
Error Definition Interpretation Example

.
Storage
Configuration A configured element • Configuration Error Configuration

Error within the closest is reported if the Error: Slimline
match contains customer or the SL2 and BP4
something that does onsite technician There is a
not match the Dell incorrectly configuration
qualified configures the error between
configuration. system; or if there the SL2 and
is an issue in the BP4.
system during the
factory dispatch.
• The key action is
to verify the cable
routing and fix the
errors.
Configuration The iDRAC interface • The cases that are Configuration

Missing found a configured reported are due to Missing: CPU2
component missing the cables being There is a
within the closest damaged or must missing
match detected. be replaced. configuration in
CPU2.
• The key action is
to guide the
customer to verify
the missing cables.
• If all cables are
present and
connected, then
call out for a
replacement.

Storage
Communication A configured element • The controllers can Communication

Error is present, but communicate over Error:
communication over a cable, but the Backplane 2
the management communication is The controllers
interface is not unsuccessful over have an issue to
working. the management communicate
interface. over backplane
2.

.
Storage
You Have Completed This Content
Click the Save Progress and Exit button in the course menu or below to
record this content as complete.
Go to the next learning or assessment, if applicable.

Appendix
13G Memory Population Guidelines

Dell recommended guidelines for installing memory modules on a 13G server are:
• RDIMMs and LRDIMMs must not be mixed.
• DRAMs with x4 and x8 based memory modules can be mixed.
• Up to three dual-rank or single-rank RDIMMs can be populated per channel.
• Up to three LRDIMMs can be populated per channel regardless of rank count.
• If memory modules with different speeds are installed, they will operate at the
speed of the slowest installed memory module(s) or slower, depending on
system DIMM configuration.
• Populate memory module sockets only if a processor is installed.
• Populate all the sockets with white release tabs first, followed by the black
release tabs, and then the green release tabs.
• Memory modules of different capacities can be mixed, provided other memory
population rules are followed.
• When mixing memory modules with different capacities, populate the sockets
with memory modules with highest capacity first.
− For example, if the customer wants to mix 4 GB and 8 GB memory modules,
populate 8 GB memory modules in the sockets with white release tabs and 4
GB memory modules in the sockets with black release tabs.
should be identical.
• Mixing of more than two memory module capacities in a system is not
supported.
• Populate four memory modules per processor (one DIMM per channel) at a time
to maximize performance.

Appendix

Dell recommended guidelines for installing memory module on a 14G server are:
• All DIMMs must be DDR4.
• RDIMMs and LRDIMMs/NVDIMMs and LRDIMMs should not be mixed.
• NVDIMMs and RDIMMs can be mixed.
• 64 GB LRDIMMs should not be mixed with 128 GB LRDIMMs.
• Up to two RDIMMs and LRDIMMs can be populated per channel regardless of
rank count.
• A maximum of two different ranked DIMMs can be populated in a channel
regardless of rank count.
speed of the slowest installed memory module(s) or slower, depending on the
• Populate all the sockets with white release tabs first, followed by the black
release tabs.
with memory modules with the highest capacity first.
− For example, if the customer wants to mix 8 GB and 16 GB memory
modules, populate 16 GB memory modules in the sockets with white release
tabs and 8 GB memory modules in the sockets with black release tabs.
supported.
• Unbalanced memory configurations will result in a performance loss so always
populate memory channels identically with identical DIMMs for best
performance.

Appendix

Dell recommended guidelines for installing memory module on a 15G server are:
• All DIMMs must be DDR4.
• RDIMMs and LRDIMMs must not be mixed.
• In Optimizer Mode, the DRAM controllers operate independently in the 64-bit
mode and provide optimized memory
performance.
speed of the slowest installed memory module(s) or slower, depending on the
• Populate all the sockets with white release tabs first and then the black release
tabs.
with memory modules with the highest capacity first.
− For example, if the customer wants to mix 8 GB and 16 GB memory
modules, populate 16 GB memory modules in the sockets with white release
tabs and 8 GB memory modules in the sockets with black release tabs.
supported.
• Unbalanced or odd memory configuration results in a performance loss and
system may not identify the memory modules being installed, so always
populate memory channels identically with equal DIMMs for best performance.
• Supported RDIMM / LRDIMM configurations are 1, 2, 4, 6, 8, 12, or 16 DIMMs
per processor.

Appendix
PowerEdge XR11 Fans and Heatsinks
Two fans installed in the middle of the PowerEdge XR11 Extended heatsink used in the PowerEdge XR11.
chassis.
Single DIMM Blanks
Single DIMM blank Processor and DIMM blank

Appendix
iDRAC Liquid Leak Error
The iDRAC GUI reporting a leak detection in the server.
PowerEdge Thermals - FAQs

Why does my system fan speed change?
In general, fan speed changes due to change in system ambient temperature or

hardware component temperatures. Fan speeds increase in order to maintain
system reliability by ensuring components operate below their maximum
temperature specifications. Most frequently, fan speeds increase due to the
workload running on the CPU. Other hardware components with increased cooling
requirements include GPU and FPGA cards, third-party PCIe adapters, high-end
networking adapters, and high-density storage configurations.
Why are my fans running at very high speed or at full speed?
Fans may be running at high or full speed for various reasons. The workload
running on the server can result in high CPU utilization and thus an increase in
cooling requirement. If the system is idle and fans are still at full speed, then either
a hardware option (such as a high-power card or a third-party PCIe adapter)
presents in the server requires that full fan speed, or there is a failure of sensor
communication, a fan failure, or operation of a server without chassis cover and/or
air regulating shroud. Some systems require blanks for nonpopulated hard drive
slots, DIMM slots and/or CPU. Cooling for certain components may be
compromised if these blanks are missing, resulting in higher fan speeds.
Why cannot I lower my fan speeds?
Thermal algorithms define the minimum system fan speeds based on ambient
temperature, system configuration and system utilization. Allowing the user to
reduce fan speed could put system cooling at risk, potentially causing system
thermal-related failures. The only instance in which the user can reduce system fan

Appendix
speed is when a third-party PCIe adapter card is part of the configuration for which
a thermal algorithm provides cooling based on limited information from the card.
This response may result in overcooling of the card. In this case, the user can turn
off the fan response that is associated with this card, or define a custom airflow
value for the card (iDRAC Web interface or RACADM). Turning off third-party Card
fan response may reduce the fan speed if other components within the system are
not requesting a higher fan speed than the response requested by the third-party
PCIe adapter card. It is not recommended to turn off this response unless the user
is aware of the cooling requirement of the adapter card.
I hear a fan spinning but my server is not powered ON. Is that expected?
Some server platforms are designed to allow one particular fan-in the system to
power ON when the system is in standby (AUX) state (AC plugged in, but power
button not pressed). This fan may run under some system inlet ambient conditions
to ensure cooling for onboard network devices that may be active in system AUX
state.
How many fans are supported on this server?
Some platform configurations require only a limited subset of fans to be present in

the server. For example, some 2-socket platforms may allow a lower fan count
when configured with a single CPU. If a configuration does not require full fan
population, a fan blank must be installed to prevent air recirculation within the
system. Most configurations generally require full fan population. If the
configuration has fewer fans than required, a fan failure log is generated in iDRAC.
Conversely, if there are more fans present than required, the additional fans may
operate at their lowest speed, with no additional response to system utilization. It is
not recommended to populate more or fewer fans than the required quantity for
optimum cooling operation of the server. Some platforms also support different fan
types (Standard Performance vs. High Performance) and are represented as Type
1 or 2. A label on the individual fan carrier indicates if the fan is standard or high
performance. There is no indication of the installed fan type in the iDRAC user
interfaces. This feature may be added in a future iDRAC release. It is not
recommended to swap standard fans with high-performance fans, unless allowed
by the platform configuration.
What is fan Pulse Width Modulation (PWM)?

Appendix
Fan speeds are expressed in Revolution Per Minute (RPM) but the input signal that
drives the fan to run at different speed is expressed as PWM (Pulse Width
Modulation). PWM can be any number between 0% and 100%. It should be noted
that a PWM of 0% generally does not mean that a fan is OFF. 0% is typically
defined as the fan’s lowest operational speed. Conversely, at 100% PWM, fans run
at the maximum RPM. The relationship between fan PWM and RPM is linear.
What custom cooling options are available to users?
Various custom thermal settings are available and accessible using iDRAC
interfaces like RACADM, iDRAC UI, and BIOS HII browser. These thermal options
include, Custom Thermal Profiles (Maximum Performance, Maximum Performance
per Watt, Sound Cap); custom fan speed options (minimum fan speed, fan speed
offsets); and reduced Exhaust Temperature settings. In addition, custom airflow
settings can be applied to third-party PCIe adapter cards through RACADM and
iDRAC UI interfaces. The easiest way to access these options is to connect to the
iDRAC Web UI of the server and go to Cooling -> Configure Fans -> Fan
Configuration.
What is the Sound Cap option in Custom Thermal Profiles?
Sound Cap is a new feature of PowerEdge 14G servers. Sound Cap was
developed in response to customer requests and is for specialized environments in
which minimizing acoustical output is a higher criteria than peak raw performance.
Sound Cap limits, or “caps”, CPU power consumption and thus fan speed, resulting
in a lower acoustical ceiling. Its application is unique for acoustical deployments
and may result in reduced system performance.
Why are PCIe adapter cards installed based on a slot priority requirement in
the server?
There are various reasons that slot restrictions exist for certain cards. Some
common ones are:
• Certain slots are limited by PCIe lane width (like x4, x8, x16).
• Mechanically, a card may fit only in certain locations. This can be based on
such as whether the card is single wide vs. double wide, or standard-length or
full-length card.

Appendix
• Cabling that is connected to the card that requires the card to be in a certain
location for optimum cable routing.
• Cooling limitations in certain slots, such as airflow limitations may cause a
Cooling or Thermal priority.
Where can I find more information about PCIe adapter card cooling on the
server?
The best place to look for this information is within the iDRAC Web UI. From the
iDRAC home screen, select Cooling -> Fan Overview -> Configure Fans. Then
scroll down to see the “PCIe Airflow Settings”. This section displays all the PCIe
adapter slots present in the system and the maximum airflow in LFM (Linear Feet
per Minute) available at each slot (when all fans are at full speed). This section also
indicates if a particular PCIe adapter card is considered a third-party Card, and if
so, what LFM is being provided. The user has the option to customize the airflow
based on the card specifications. This feature is new with PowerEdge 14G servers
and is an industry first.
Why is the top cover of my system hot and is that an indication of potential
cooling problem? OR Why are CPU temperatures high? OR Why is the air
coming out of the server so hot?
The system top cover may get hot in local regions above the CPU heatsinks or
near the back of the system. This occurs most commonly in dense systems and in
1U servers. The localized heating of the top cover is due to the close proximity of
the cover to the CPU heatsink or to the heated exhaust air at the rear of the
system. The surface- and exhaust temperatures should not exceed safety limits of
70°C. Components such as CPUs, GPUs, and general board components are
designed to run at higher temperatures without impact component or system
reliability. Users wanting to review or adjust system temperatures or exhaust air
temperature can use Custom Thermal Settings through various iDRAC interfaces to
increase fan speed (and thus system cooling) by applying any one of the Fan
Speed Offset, Minimum Fan Speed, and/or Custom Exhaust Temperature options.
Does my platform support GPUs?
Many high-power compute GPU devices that are passively cooled require platform-
specific configuration restrictions, and those are allowed only on a limited number
of platforms. Lower power (such as less than 75 W) PCIe adapters are supported
on all platforms. See platform-specific limitations to ensure compliance.

Appendix
Why are different CPU heat sinks required in some configurations?
Some platforms require different CPU heat sinks based on the installed CPU TDP
or other specific hardware options. For example, shorter heat sinks and a different
air shroud are required in the R740 and R740xd to allow for proper GPU cooling.
See the individual platform details for specific information.
Different RIO Cards
RIO card used in a non-liquid cooling configuration. Custom RIO card used in a Direct Liquid Cooling
configuration.
PowerEdge 750xa GPU Configurations

The Dell EMC PowerEdge 750xa supports two GPU configurations in the chassis
front end.
• Four GPUs configuration
− In this configuration, each riser has two GPUs.
• Two GPUs configuration
− In this configuration, each riser has one GPU. The empty slots should be
installed with a dummy GPU module.

Appendix
− For example, if GPUs are installed in slot 31 and slot 33, then dummy GPU
modules must be installed in slot 32 and slot 34. Similarly, if two GPUs are
installed in slot 33 and slot 34, then dummy GPU modules must be installed
in empty slots 31 and slot 32.
Left GPU Module Right GPU Module
Rear Facing Module
The Online Course Contains an Interaction Here.
[Detailed description of the Interaction for Guides]
BOSS Management on iDRAC

Appendix
BOSS Management on System Setup Utility
BOSS Management on OMSA
BOSS CLI Commands
Command Description Example Usage
info -o hba Display BOSS-S1 controller mvsetup info -o hba

information

Appendix
info -o vd Display virtual drive mvsetup info -o vd

information
info -o pd Display physical drive ./mvcli info -o pd

information
smart -p <PD_ID> Display SMART information ./mvcli smart -p 0

of physical drive
event Display controller events ./mvcli event
RAID in Lifecycle Controller

Appendix
RAID in System Setup Utility
RAID in OMSA

Appendix
NVMe Support
NVMe Drives PCIe Generation
Intel Gen 4
P5500/P5600
Kioxia Gen 4
CD6/CM6
Samsung Gen 4
1733/1735

Glossary
Parity
Parity is small data that is used to connect to larger data blocks and recover data
when a disk failure occurs.


Dell EMC PowerEdge Concepts and Features-Downloadable Content

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Dell EMC PowerEdge Concepts and Features-Downloadable Content

Uploaded by

Copyright:

Available Formats

DELL EMC POWEREDGE

© Copyright 2021 Dell Inc. Page i

Dell EMC PowerEdge Concepts and Features..................................................................... 7

Portfolio Overview .................................................................................... 8

Portfolio Overview ................................................................................................................ 9

Server Components ................................................................................ 28

Server Components ........................................................................................................... 29