Professional Documents
Culture Documents
Mz710 Nic v100r001 User Guide 06
Mz710 Nic v100r001 User Guide 06
V100R001
User Guide
Issue 06
Date 2017-10-20
and other Huawei trademarks are trademarks of Huawei Technologies Co., Ltd.
All other trademarks and trade names mentioned in this document are the property of their respective
holders.
Notice
The purchased products, services and features are stipulated by the contract made between Huawei and the
customer. All or part of the products, services and features described in this document may not be within the
purchase scope or the usage scope. Unless otherwise specified in the contract, all statements, information,
and recommendations in this document are provided "AS IS" without warranties, guarantees or
representations of any kind, either express or implied.
The information in this document is subject to change without notice. Every effort has been made in the
preparation of this document to ensure accuracy of the contents, but all statements, information, and
recommendations in this document do not constitute a warranty of any kind, express or implied.
Website: http://e.huawei.com
Purpose
This document describes the MZ710 in terms of its functions, appearance, and configuration
methods.
Intended Audience
This document is intended for:
l Huawei presales engineers
l Channel partner presales engineers
l Enterprise presales engineers
Symbol Conventions
The symbols that may be found in this document are defined as follows.
Symbol Description
Change History
Changes between document issues are cumulative. The latest document issue contains all the
changes made in earlier issues.
Issue 06 (2017-10-20)
This issue is the sixth official release.
Issue 05 (2016-03-31)
This issue is the fifth official release.
Issue 04 (2015-11-09)
This issue is the fourth official release.
Issue 03 (2015-07-17)
This issue is the third official release.
Issue 02 (2015-03-20)
This issue is the second official release.
Issue 01 (2014-11-19)
This issue is the first official release.
Contents
2 Features............................................................................................................................................4
2.1 Feature List..................................................................................................................................................................... 4
2.2 Feature Description.........................................................................................................................................................4
1 Overview
1.1 Functions
1.2 Appearance
1.1 Functions
The MZ710 is an Ethernet network interface card (NIC). It is used for E9000 compute nodes
and provides 40GE ports for compute nodes to connect to switch modules in the chassis.
The MZ710 uses the Mellanox ConnectX-3 Pro (CX3 Pro) chip and supports NIC
applications. The 40GE ports on the MZ710 support 40G/10G auto-negotiation, and 40GE
and 10GE port applications. The MZ710 supports the Remote Direct Memory Access over
Converged Ethernet (RoCE) feature to address low-latency network applications.
1.2 Appearance
The MZ710 can be installed in slot Mezzanine1 (Mezz1 for short) or Mezzanine2 (Mezz2 for
short) on a half-width E9000 compute node or in slot Mezz1, Mezz2, Mezzanine3 (Mezz3 for
short), or Mezzanine4 (Mezz4 for short) on a full-width E9000 compute node.
The MZ710 provides network ports for connecting to switch modules:
l When the MZ710 is installed in slot Mezz1 or Mezz3, its two 40GE ports connect to
switch modules in slots 2X and 3X.
l When the MZ710 is installed in slot Mezz2 or Mezz4, its two 40GE ports connect to
switch modules in slots 1E and 4E.
2 Features
l If flow control is configured on the NIC, it must also be configured on the corresponding switch
module or external switch.
l Certain functions are related to OS and switch features. For details, contact Huawei technical
support.
The MZ710 does not tag or untag packets, but transparently transmits them. VLAN IDs are
specified by the operating system (OS) on an E9000 compute node.
SR-IOV
The MZ710 supports the SR-IOV feature. The NIC supports a maximum of 126 VFs, and
each physical 40GE port on the NIC supports a maximum of 63 VFs.
The VFs derived from each physical function (PF) can be assigned to virtual machines
(VMs). In this way, the mapping between VFs and VMs is set up.
PXE
The 40GE ports on the MZ710 support the Preboot Execution Environment (PXE) feature.
PXE is used for remote boot over an Ethernet or IP network. It enables users to remotely
connect to the PXE server to load OSs.
RoCE
The MZ710 supports the RoCE feature. This feature uses the Remote Direct Memory Access
(RDMA) technology to reduce the packet processing and forwarding latency of the NIC, to
reduce the CPU usage, and to implement low-latency data transmission over the data center
network. With the RoCE feature, the end-to-end read and write delay of the NIC can reach 4
us when the packet length is 128 bytes. The MZ710 supports 16M I/O channels (equivalent to
IB QPs), and provides priority-based scheduling and flow control to support low-latency,
high-bandwidth network transmission. The MZ710 supports OpenFabrics Enterprise
Distribution for Linux (OFED for Linux) and Mellanox OFED for Windows (WinOF).
Figure 3-1 Connections between the MZ710s on a half-width compute node and the ports on
I/O modules
The MZ710 provides two ports, and only 4X of each 8X SerDes is used.
Table 3-2 describes the I/O modules to which the MZ710 can connect.
CX710 2X/3X X
1E/4E X
CX712 2X/3X X
1E/4E X
NOTE
The MZ710 provides two network ports, which share the PCIe function. Therefore, you can query only
one PCIe function in the system.
l Run the ifconfig a command to view the two NIC ports, which are displayed in the
format of eth[num]. See eth6 and eth7 in Figure 4-2.
l Run the ethtool i command. The two network ports on the MZ710 shares the PCie
function. Therefore, eth6 and eth7 have the same PCIe function, as shown in bus-info in
Figure 4-3.
4.2.1 SUSE
To install the MZ710 driver in SUSE Linux Enterprise Server (SLES) 11, perform the
following steps:
Step 3 Run the modinfo command to view the version information of the MZ710 driver Ethernet
(mlx4_en) on SLES 11. See Figure 4-5.
----End
To uninstall the MZ710 driver from SLES 11, perform the following steps:
----End
4.2.2 Windows
The MZ710 driver file on Windows is MLNX_VPI_WinOF-4_70_All_xxx_x64.exe, where
xxx indicates the Windows version.
To uninstall the driver for an MZ710 port, perform the following steps:
Step 1 Run the modprobe 8021q command to load the 802.1q module.
Step 2 Run the ifconfig -a and yast2 lan commands to locate the MZ710 ports.
Step 3 Run the vconfig add eth[num] [vlan id] command to add VLAN IDs to the MZ710 ports.
Step 4 Run the ifconifg eth[num] [ip address] netmask [netmask] command to set IP addresses for
the VLANs, which are equivalent to the IP addresses of eth[num] subports.
#ifconfig eth13.100 192.168.100.10 netmask 255.255.255.0
#ifconfig eth13.200 192.168.200.10 netmask 255.255.255.0
Step 5 Add the VLAN configuration to the configuration file. The OS will automatically load the
VLAN configuration after the server resets or powers off and powers on.
1. Run the echo command to configure the 802.1q module to be loaded upon startup.
#echo "modprobe 8021q">>/etc/rc.local
NOTE
----End
In SLES 11, configure port bonding in the OS configuration file for the MZ710. The MZ710
supports seven port bonding modes, including active-backup, balancing, and Link
Aggregation Control Protocol (LACP).
To configure port bonding for the MZ710 in SLES 11, perform the following steps:
Step 1 Run the ifconfig-a and yast2 lan commands to locate the MZ710 ports.
Step 5 Open the modprobe.conf.local file in /etc. Add the following information to the end of the
modprobe.conf.local file:
#vi /etc/modprobe.conf.local
alias bond0 bonding
options bonding miimon=100 mode=0 use_carrier=0
0: balance-rr (round-robin)
1: active-backup
2: balance-xor
3: broadcast
4: 802.3ad
5: balance-tlb
6: balance-alb
NOTE
If you want to set the bonding mode to 802.3ad, set the peer device port connected to the MZ710 to the
link aggregation mode (static) in compliance with the IEEE 802.3ad specifications. The other bonding
modes have no special configuration requirement for the peer device.
Step 6 Run the /etc/init.d/network restart or rcnetwork restart command to restart the network
application.
----End
Step 1 Run the mst start command to start the mst service, and run the mst status command to
query the device name. See Figure 4-7.
Figure 4-7 Starting the mst service and querying the device name
Step 2 Run the mlxconfig -d /dev/mst/mt4103_pciconf0 q command to check whether the SR-IOV
function is enabled for the NIC. See Figure 4-8.
NOTE
The SR-IOV function is disabled by default. When this function is disabled, the values of SR-IOV_EN
and NUM_OF_VFS are N/A in the command output, as shown in Figure 4-8.
l If the SR-IOV function has been enabled, go to Step 4.
l If the SR-IOV function has not been enabled, go to Step 3.
Step 4 Run the mlxconfig -d /dev/mst/mt4103_pciconf0 q command to check whether the SR-IOV
function is successfully enabled. See Figure 4-10.
Step 5 Open the menu.lst file in /boot/grub/, and add intel_iommu=on at the end of the kernel line
in this file. See Figure 4-11.
Step 7 Run the lspci |grep Mellanox command. You can see information about the new eight VFs, as
shown in Figure 4-13.
#fig46817064
Step 8 In the Virtual Machine Manager window, assign VFs to VMs. See Figure 4-14.
NOTE
Step 9 Select the VM rhel6.4, and click Open. The details about the VM are displayed. You can add
or delete devices for the VM. The following describes how to add a VF for the VM. Figure
4-15 shows the VM window.
Step 10 Click Add Hardware. In the Add New Virtual Hardware window, select the VFs to be
added, as shown in Figure 4-16.
NOTE
----End
OS operating system
PF physical function
PG priority group
QP queue pair
TC traffic class
VF virtual function
VM virtual machine