Professional Documents
Culture Documents
Efficient Data Center Virtualization With QLogic 10GbE Solutions From HP
Efficient Data Center Virtualization With QLogic 10GbE Solutions From HP
SN0054584-00 A
Reference GuideEfficient Data Center Virtualization with QLogic 10GbE Solutions from HP
Information furnished in this manual is believed to be accurate and reliable. However, QLogic Corporation assumes no responsibility for its use, nor for any infringements of patents or other rights of third parties which may result from its use. QLogic Corporation reserves the right to change product specifications at any time without notice. Applications described in this document for any of these products are for illustrative purposes only. QLogic Corporation makes no representation nor warranty that such applications are suitable for the specified use without further testing or modification. QLogic Corporation assumes no responsibility for any errors that may appear in this document.
ii
SN0054584-00 A
Table of Contents
Preface
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Key Benefits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v v
2 3
5 A
SN0054584-00 A
iii
Reference GuideEfficient Data Center Virtualization with QLogic 10GbE Solutions from HP
List of Figures
Figure Page 1-1 Comparison of a Legacy Deployment Versus an Efficient Deployment . . . . . . . . . . 1-1 1-2 Typical Legacy 1GbE Topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-3 1-3 Efficient 10GbE Topology. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-6
List of Tables
Table 1-1 1-2 1-3 2-1 2-2 2-3 1GbE and 10GbE Components for Reference Network Topology . . . . . . . . . . . . . . Network Bandwidth Sizing for Legacy 1GbE Reference Architecture . . . . . . . . . . . Network Bandwidth Sizing for the Efficient 10GbE Reference Architecture . . . . . . Hardware Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Software Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Management Components. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Page 1-2 1-4 1-7 2-1 2-1 2-2
iv
SN0054584-00 A
Preface
Overview
Designing an efficient data center to support high performance and scalable virtualized server deployment creates a number of challenges when using a legacy 1-gigabit Ethernet (GbE) network I/O infrastructure. This document identifies the challenges of the 1GbE legacy network infrastructures for virtualized data centers and presents a reference architecture solution using QLogics 10Gb Intelligent Ethernet Adapter with HP infrastructure. The reference architecture discussed in this document uses proven HP and QLogic 10GbE networking technology solutions currently available in the market. System architects can leverage the reference architecture to plan, migrate, and optimize their designs, maximizing data center operational efficiency and savings. As data centers continue to increase the number of virtualized hosts on a physical server, the demands for higher efficiency, performance, and consolidation creates critical challenges with 1GbE legacy networks: Low network bandwidth caps the scaling of virtualized hosts and services. Inefficient usage of server PCI slots limits I/O consolidation. Increased cabling, switch ports, and adapters add management complexity and costs. Peak network performance is reduced for on-demand applications and services.
The HP-QLogic 10GbE network solution eliminates the I/O bottlenecks, delivering significant operational savings, high performance, reduced management complexity, and improved scalability for virtualized data center deployments.
Key Benefits
Efficient data center virtualization has the following key benefits: Reduced management complexity Improved operational efficiency High-performance 10GbE network Scalable virtual hosts and services
SN0054584-00 A
vi
SN0054584-00 A
System Architecture
SN0054584-00 A
1-1
Table 1-1 compares the components used for the legacy deployment with the components used for the efficient deployment.
Table 1-1. 1GbE and 10GbE Components for Reference Network Topology
Legacy Deployment with 1GbE Model
Server NICs LAN switch HP DL380 G7 Legacy 1 GbE NIC ports Legacy 1GbE Ethernet switch 1 14 2 switches; 14 switch ports
Component
Units
Units
Network Topology
There are two network topologies described in this document: Typical Legacy 1GbE Topology on page 1-3 Efficient 10GbE Topology on page 1-5
1-2
SN0054584-00 A
SN0054584-00 A
1-3
The ESX virtual standard switch is configured with multiple port groups (16) to support the various networking functions and application groupings. Each port group is supplied with one or more physical 1GbE connections. Virtual LAN (VLAN) tags (AF) are implemented on the port groups for segmenting network traffic. Each VM connects to the data center network through one or more ESX virtual NICs (VNICs), and each VNIC connects to one or more virtual switch (vswitch) port groups. Network bandwidth and high-availability requirements of the various traffic types (see Table 1-2) on the 1GbE physical uplinks are either configured for failover for redundancy or static link aggregation control protocol (LACP) for higher bandwidth.
Table 1-2. Network Bandwidth Sizing for Legacy 1GbE Reference Architecture
Traffic Types for VMware ESX
Management traffic VMotion traffic IP storage traffic NFS traffic Fault tolerance (FT) traffic VM traffic (for 12 VMs) Total
Available Bandwidth
1Gbps 1Gbps 1Gbps 1Gbps 1Gbps 6Gbps 11Gbps
The management network is served by the ESX VMkernel network interface connected to a pair of 1GbE uplinks connected by port group 6 on the ESX virtual standard switch. The management traffic is carried over VLAN F through a redundant pair of 1GbE links (failover configuration) connected to a redundant set of 1GbE switches. While the solid green line is the primary I/O path, the dashed green line depicts a failover path for the network traffic (see Figure 1-2). Two legacy 1GbE switches interconnected to each other create a high-availability configuration that prevents network traffic disruption if one of the 1GbE switches fail. The 1GbE architecture has limited network bandwidth; it is unable to scale to meet the peak network bandwidth requirements for on-demand applications. The reason for this is that most PCIe slots are already occupied, and the network cannot scale with more VMs and application services, which demand additional network bandwidth that cannot increase because of the limitation of additional PCIe slots.
1-4
SN0054584-00 A
There are 14 cables originating from one server using seven switch ports per switch, increasing operational cost and management complexity.
SN0054584-00 A
1-5
The architecture uses the two 10GbE uplinks aggregated (static LACP) to provide a single consolidated 20GbE uplink for the ESX standard virtual switch. The network bandwidth sizing table (see Table 1-3) categorizes various traffic types and the allocation of the 20Gb network bandwidth.
Table 1-3. Network Bandwidth Sizing for the Efficient 10GbE Reference Architecture
Traffic Types for VMware ESX
Management traffic VMotion traffic IP storage traffic NFS traffic FT traffic Virtual machine traffic (for 12 VMs) Total 2 2
Available Bandwidth
1Gps 4Gps 2Gps 2Gps 2Gps 8Gps 19Gps
VMware ESX traffic types (such as VMotion) that were limited to 1-2Gbps in the legacy 1GbE deployment can now leverage the increased available bandwidth, resulting in a more efficient VMotion for live migration. IP storage, NFS traffic, and the storage access needs of the virtual machines are no longer limited to the number of 1GbE adapters dedicated to the traffic type. Two HP 5800 series 10GbE network switches replace the legacy 1GbE switches and provide high-speed and high-availability network traffic configuration (see Figure 1-3). The obvious advantage of the 10GbE solution is that it reduces the overall physical connections, simplifying infrastructure management. Fewer physical ports reduce the wiring complexity and improve infrastructure reliability. With just one PCIe slot occupied and significantly higher bandwidth than the 1GbE legacy solution, the 10GbE architecture can scale to accommodate more VMs and higher network bandwidth requirements for peak demands. This consolidation improves the efficiency of the architecture.
1-6
SN0054584-00 A
HP Part Number
Description
Intelligent Ethernet Adapter HP NC523SFP Switch HP 5800-24G JC100A Fixed four RJ-45 10/100/1000 ports and four SFP+ 10GbE ports 593717-B21 HP ProLiant Ethernet Server adapter
Description
Version
Intelligent Ethernet Adapter Driver-network Switch Platform firmware HP ProCurve switch firmware and boot code
4.0.719
A5800_5.20.R1211
SN0054584-00 A
2-1
Description
Version
Intelligent Ethernet Adapter QLogic vCenter Plugin QLogic CIM Provider vCenter Plugin
QConvergeConsole Management CIM Provider for HP-branded QLogic-based Intelligent Ethernet Adapters A bootable FreeDOSTM operating system along with utilities required to Flash HP P3+-based adapters
HP NC-Series QLogic P3P Flash Update Kit Switch Microsoft Win dows -based network management
1.00
5.0
2-2
SN0054584-00 A
The NC523SFP is ideal for enterprise customers in a VMware ESX environment because it provides the following advantages: The best combination of high speed and low latency for virtualization makes it the ideal choice when migrating from a 1GbE solution to a 10GbE solution. It reduces the number of cables and switch ports needed by using a larger 10GbE pipe in the infrastructure.
SN0054584-00 A
3-1
The ability to provide a higher virtual machine density by meeting and exceeding the network bandwidth demands of an increasing number of virtual machines.
For more information, visit the following Web sites: http://www.hp.com/go/nc523sfp http://www.qlogic.com/go/hp
3-2
SN0054584-00 A
Design Considerations
SN0054584-00 A
4-1
VMware vSphere ESX offers multiple components that relate to Ethernet networking: Hypervisor The hypervisor runs on physical servers and abstracts processor, memory, storage, and networking resources. HP-QLogic NC523SFP 10GbE Server Adapter drivers run in the context of the hypervisor. Virtual standard switch The virtual standard switch is a software component running under the control of the hypervisor. It provides the necessary network connectivity for virtual machines (through the VNIC interface) and back-end services (through the VMkernel network interface). Virtual center management server The virtual center management server allows for configuring, provisioning, and managing virtualized IT infrastructure. VMware back-end services These services include VMware VMotion (which enables virtual machine migration), VMware FT (which provides a higher level of availability), and VMware IP storage and NFS (which provide access to storage and NFS servers). A VMware vSphere ESX environment involves the following traffic types: Management Management traffic goes through the VMkernel management interface on VMware ESXi. This port is used for all management and configuration and is the port by which VMware ESX communicates with the VMware vCenter server. This port generally has very low network usage, but it should always be available and isolated from other traffic types through a management VLAN. VMware VMotion The VMkernel port is used for migrating a running virtual machine from one VMware ESX host to another. Using a 10GbE connection, VMotion under vSphere 4.1 can use up to 8Gbps of aggregate bandwidth with multiple virtual machines being migrated simultaneously. This traffic is typically implemented on a separate VLAN specific to VMware VMotion, with no outside communication required.
4-2
SN0054584-00 A
Fault-tolerant logging The VMkernel port for fault-tolerant logging transfers the input network I/O for the fault-tolerant virtual machine plus the read disk traffic to the secondary fault-tolerant virtual machine. Traffic varies according to the network and storage behavior of the application. End-to-end latency between the fault-tolerant virtual machines should be less than 1ms, and the dedicated 2Gbps of network bandwidth is considered sufficient. This traffic is typically implemented on a separate VLAN specific to fault-tolerant logging, with no outside communication required.
IP Storage The VMkernel port provides access to a disk subsystem over the IP network. In VMware ESXi 4.0, two VMkernel ports can be bonded to allow IP storage traffic over both physical network interfaces. Traffic varies according to I/O; typically 2Gbps of network bandwidth is considered sufficient. This traffic is typically implemented on a storage-specific VLAN common to initiators and targets, although targets may reside on another VLAN accessible through a Layer 3 gateway.
Network file system The VMkernel port communicates with NFS files in VMware ESX. Traffic varies according to I/O, and 12Gbps of network bandwidth is considered sufficient. This traffic is typically implemented on an NFS-specific VLAN, although files may reside on another VLAN accessible through a Layer 3 gateway.
Virtual machines Virtual machines vary in number, may be distributed over more than one VLAN, and may be subject to different policies defined in port profiles and virtual port groups. The reference architecture design assumes an aggregate virtual machines network bandwidth demand to be 68Gbps.
SN0054584-00 A
4-3
With the HP-QLogic 10Gb network infrastructure solution, the transition from the legacy design is very simple and straightforward. After the VMware ESX driver for the NC523SFP has been loaded, the only task is to map the new 10GbE uplinks to the VMware vswitch and create a static LACP (802.3ad) of the two ports of the NC523SFP. Each of the two ports of the NC523SFP are physically connected to two independent HP 5800 switches; this architecture provides for resilience to any single point of failure. While the use of two 10GbE ports helps to reduce the solution cost, some organizations may prefer the use of four 10GbE ports to provide additional bandwidth, additional redundancy, or simply to interface with existing network infrastructure. NOTE: HP supports up to a maximum of two physical NC523SFP adapters in a supported server. The availability of an aggregate bandwidth of 40Gbps bidirectional across the two ports of the NC523SFP provides for all the network bandwidth requirements of the virtual machines and back-end services, as described in Table 1-3. If planned appropriately, the migration from 1GbE to 10GbE in a VMware ESX environment provides the benefits described in the following sections.
Increased Performance
10GbE delivers 10 times the bandwidth of 1GbE in a single pipe. Now, rather than using 8 to 10 1GbE ports in addition to on-board LOMs in each server, organizations can deploy just two 10GbE NICs and meet the network bandwidth demands of the virtual infrastructure. Increased bandwidth per virtual machine 10GbE allows each virtual machine to use bandwidth of more than 1Gbps (and its increments), helping to increase performance and eliminate the network as a bottleneck. Faster VMotion Using a 10GbE connection, VMotion under vSphere 4.1 can use up to 8Gbps of aggregate bandwidth, which enables virtual machines to move to new hosts, quickly reducing downtime. Faster access to IP storage Using a faster 10GbE link, access speeds to IP storage and NFS servers are enhanced, improving overall virtual machine performance.
4-4
SN0054584-00 A
Simplified Management
Adopting 10GbE in virtualized environments reduces the number of Ethernet adapter ports to be managed, components to troubleshoot, and components that can fail. Ease of management Consolidating a number of gigabit Ethernet links onto a single 10GbE connection dramatically decreases the number of management points, including NICs and their firmware/driver, cables, and switch ports. NIC management can be executed in any of three ways: through QConvergeConsole, through the command line interface, or through the Windows network management utility. Simplified troubleshooting Decreasing the number of devices reduces the number of failure points and devices to troubleshoot, contributing to higher availability.
Increased Scalability
Moving to 10gbE enables the data centers virtual infrastructure to scale and accommodate a virtual machine sprawl and associated I/O infrastructure demands. Higher virtual machine density 10GbE provides increased headroom that gives applications running in virtual machines room to scale and the ability to host more virtual machines per ESX host without network bottlenecks. The increased bandwidth of 10GbE improves scalability for both back-end traffic and production traffic to virtual machines. Free PCIe slots Deploying 10GbE frees up server PCIe slots that were occupied by multiple 1GbE adapters, proving room for expansion.
SN0054584-00 A
4-5
4-6
SN0054584-00 A
Conclusion
Deploying HP-QLogic 10GbE technology in virtualized environments based on VMware vSphere allows enterprises to reduce the complexity and increase the scalability of the network infrastructure. The 10GbE products from HP and QLogic enable enterprise customers to meet the performance requirements of virtual machines without the large numbers of physical server connections required in 1GbE networks. Increased bandwidth; fewer cables, ports, and network adapters to manage; fewer PCIe slots used; and the capability to consolidate network I/O on a single wire complete the argument for deploying the HP-QLogic 10GbE technology in an efficient data center.
SN0054584-00 A
5-1
5Conclusion
5-2
SN0054584-00 A
Implementation Resources
Use the following references for more information about the HP server, Intelligent Ethernet Adapter, and switch.
User Guide
HP NC523SFP 10Gb 2-port Ethernet Server Adapter User Guide: http://bizsupport1.austin.hp.com/bc/docs/support/SupportManual/c02653034/c02 653034.pdf
Other Resources
QLogic Converged Network Adapter User's Guide (8200 and 3200 Series): http://filedownloads.qlogic.com/files/Manual/79506/User_Guide_Converged _Network_Adaper_8200_and_3200_Series_A.pdf ServerQuickSpecs for HP ProLiant DL380 G7 Server: http://h20195.www2.hp.com/v2/GetPDF.aspx/c02171016.pdf Intelligent Ethernet Adapter: QuickSpecs for HP NC522SFP Dual Port 10GbE Server Adapter: http://h18000.www1.hp.com/products/quickspecs/13230_div/13230_di v.pdf HP ProLiant Ethernet AdaptersOverview and Features: http://www.qlogic.com/OEMPartnerships/HP/Pages/HPOEMProducts. aspx
SN0054584-00 A
A-1
A-2
SN0054584-00 A
www.qlogic.com
International Offices UK | Ireland | Germany | France | India | Japan | China | Hong Kong | Singapore | Taiwan
2011 QLogic Corporation. Specifications are subject to change without notice. All rights reserved worldwide. QLogic, the QLogic logo, and QConvergeConsole are registered trademarks of QLogic Corporation. ESX, vCenter, VMotion, vSphere, and VMware are trademarks or registered trademarks of VMware, Inc. FreeDOS is a trademark of Jim Hall. HP, ProLiant, ProCurve, and StorageWorks are registered trademarks of Hewlett-Packard Company. Intel and Xeon are registered trademarks of Intel Corporation. Microsoft and Windows are registered trademarks of Microsoft Corporation. PCI Express and PCIe are registered trademarks of PCI-SIG Corporation. All other brand and product names are trademarks or registered trademarks of their respective owners. Information supplied by QLogic Corporation is believed to be accurate and reliable. QLogic Corporation assumes no responsibility for any errors in this brochure. QLogic Corporation reserves the right, without notice, to make changes in product design or specifications.