vNIC

Valentin Enescu

© 2013 IBM Corporation

 System Networking
NIC vs vNIC

What is a NIC?

A Network Interface Controller (NIC) is a component within a blade server that

allows the server to be connected to a network. The NIC provides the physical point
of connection, as well as internal software for encoding and decoding network
packets.

What is a vNIC?

A vNIC is a virtualized network interface that presents the same media access control (MAC)
interface that an actual interface would provide. Multiple VNICs can be configured on top of the same
interface, allowing multiple consumers to share that interfaces. Each vNIC appears as a regular,
independent NIC to the server operating system or a hypervisor, with each vNIC
using some portion of the physical NIC’s overall bandwidth.

Why vNIC?

Virtualizing the NIC helps to resolve issues caused by limited NIC slot availability. By
virtualizing a NIC, its resources can be divided into multiple logical instances.

2 IBM Confidential © 2013 IBM Corporation

 System Networking

Emulex NICs
IBM switches supports the Emulex Virtual Fabric Adapter (VFA) 2-port 10Gb LOM and Emulex Virtual Fabric Adapter

(Fabric Mezz)
pNIC mode

3 IBM Confidential © 2013 IBM Corporation

 System Networking

Emulex NICs
To change from pNIC to vNIC you must enter BIOS-System Settings-Network-NIC menu

4 IBM Confidential © 2013 IBM Corporation

 System Networking

Emulex NICs
vNIC mode

5 IBM Confidential © 2013 IBM Corporation

 System Networking

Virtualizing the NIC for Multiple Virtual Pipes on Each Link

6 IBM Confidential © 2013 IBM Corporation

 System Networking

vNIC IDs on the Switch

On stand-alone (non-stacked) switches, each vNIC is identified by port and vNIC

number:

<port number or alias>.<vNIC pipe number (1-4)>

For example:
INTA1.1, INTA1.2, INTA1.3, and INTA1.4 represent the vNICs on port INTA1.
INTA2.1, INTA2.2, INTA2.3, and INTA2.4 represent the vNICs on port INTA2.

Whenever switches are stacked, the switch csnum ID is also required:

<switch csnum>:<port number or alias>.<vNIC pipe number (1-4)>

For example:
2:INTA2.3 refers to port INTA2, vNIC 3, switch number 2.

7 IBM Confidential © 2013 IBM Corporation

 System Networking

Adapters to switches connections

Server bay 10 Bay 1

LOM INTA10

P0

P1
INTA10
Bay 2

Mezz

P1
INTA10
Bay 3
P2 INTB10

P3

P4
INTA10

Bay 4
INTB10

8 IBM Confidential © 2013 IBM Corporation

 System Networking
vNIC Interface Names on the Server

For Emulex Virtual Fabric Adapter 2-port 10Gb LOM:

9 IBM Confidential © 2013 IBM Corporation

 System Networking
vNIC Interface Names on the Server
For Emulex Virtual Fabric Adapter (Fabric Mezz), when replacing the LOM card:

10 IBM Confidential © 2013 IBM Corporation

 System Networking
vNIC Interface Names on the Server

For Emulex Virtual Fabric Adapter (Fabric Mezz), when adding it with the LOM Card:

11 IBM Confidential © 2013 IBM Corporation

 System Networking

vNIC Interface Names on the Server

In this, the x in the vNIC ID represents the internal switch port and its corresponding
server node.

12 IBM Confidential © 2013 IBM Corporation

 System Networking

Key vNIC features

• Up to four vNICs are supported on each internal switch port.

• Each vNIC can accommodate one of the following traffic types: regular Ethernet,
iSCSI, or Fibre Channel over Ethernet (FCoE).

• vNICs with traffic of the same type can be grouped together, along with regular
internal ports, external uplink ports, and trunk groups, to define vNIC groups for
enforcing communication boundaries.

• In the case of a failure on the external uplink ports associated with a vNIC group,
the switch can signal affected vNICs for failover while permitting other vNICs to
continue operation.

• Each vNIC can be allocated a symmetric percentage of the 10Gbps bandwidth

on the link (from NIC to switch, and from switch to NIC).

13 IBM Confidential © 2013 IBM Corporation

 System Networking

vNICs

For a vNIC to be used it must be first enabled on switch.

Our switches support egress bandwidth metering for vNIC traffic. By default, each of the four vNICs has
25% share of NIC capacity if not specified otherwise, meaning 2.5 Gbps.

Not enabled vNICs bandwidth is not counted in the sum for that NIC

The NIC is responsible for enforcing the bandwidth metering for traffic coming to switch from the vNIC.
The NIC is configured by the switch with use of DCBX messages

Janice(config)#vnic port int1 index 1

Janice(vnic-config)#bandwidth 30
Janice(vnic-config)#enable

14 IBM Confidential © 2013 IBM Corporation

 System Networking

vNIC Groups
vNIC groups can contain:

•vNICs
•Internal ports
•External ports

Each vNIC group is essentially a separate virtual network within the switch. Each vNIC group is assigned
its own unique
VLAN. The vNIC group vlan is automatically assigned to all group members.

When vNIC is added to a vNIC group a configuration DCBX message is sent to server to configure vNIC
bandwidth and vNIC VLAN.

In respect to how uplink is used and data flow within vNIC group there are 2 modes in which the switch
can operate:

15 © 2013 IBM Corporation

IBM Confidential
 System Networking
vNIC Groups

vNIC Groups in Dedicated Mode

In the dedicated mode, only one vNIC group is assigned to an uplink port.

vNIC Groups in Shared Mode

In the shared mode, multiple vNIC groups can be assigned to an uplink port.

16 IBM Confidential © 2013 IBM Corporation

 System Networking

Vnic Groups
vNIC group rules:

Once a VLAN is used for a vNIC group it is used only for vNIC purposes

A VLAN already configured on switch cannot be used as vNIC group VLAN

vNIC groups may have one or more vNIC members. However, any given vNIC can be a member
of only one vNIC group.

All vNICs on a given port must belong to different vNIC groups.

Only one individual external port or one static trunk (consisting of multiple external ports) may be
added to any given vNIC group.

By default, STP is disabled on uplink ports when uplink is added. It can be re-enabled.

The inner VLAN tag is not processed in any way in vNIC groups: The inner tag cannot be stripped
or added on port egress, is not used to restrict multicast traffic, is not matched against ACL filters,
and does not influence Layer 3 switching.
17 IBM Confidential © 2013 IBM Corporation
 System Networking
vNIC configuration example
Configure the virtual pipes for the vNICs attached to each internal port:

>> vNIC Global Configuration# port INTA1 (Select port INTA1)

>> Port INT1 vNICs# vnic 1 (Select vNIC 1 on the port)
>> vNIC INT1.1# ena (Enable the vNIC pipe)
>> vNIC INT1.1# bw 60 (Allow 60% egress bandwidth)
>> vNIC INT1.1# ../vnic 2 (Select vNIC 2 on the port)
>> vNIC INT1.2# ena (Enable the vNIC pipe)
>> vNIC INT1.2# bw 40 (Allow 40% egress bandwidth)
>> vNIC INT1.2# /cfg/virt/vnic/port INTA2 (Select port INTA2)
>> Port INT2 vNICs# vnic 1 (Select vNIC 1 on the port)
>> vNIC INT2.1# ena (Enable the vNIC pipe)
>> vNIC INT2.1# ../vnic 2 (Select vNIC 2 on the port)
>> vNIC INT2.2# ena (Enable the vNIC pipe)

Add ports, trunks, and virtual pipes to their vNIC groups:

Configure the external trunk to be used with vNIC group 2:
>> # /cfg/l2/trunk 1
>> Trunk group 1# add EXT3
>> Trunk group 1# add EXT4
>> Trunk group 1# ena
Enable the vNIC feature on the switch:
>> Trunk group 1# /cfg/virt/vnic/on
18 IBM Confidential
>> vNIC INT3.1# /cfg/virt/vnic/vnicgrp 1 (Select vNIC group 1)
>> vNIC Group 1# vnicvlan 1000
>> vNIC Group 1# addvnic INTA1.1
>> vNIC Group 1# addvnic INTA2.1
>> vNIC Group 1# addport INTA4
>> vNIC Group 1# addport EXT1
>> vNIC Group 1# failover ena
>> vNIC Group 1# ena
(Select a trunk group) >> vNIC Group 1# ../vnicgrp 2
(Add ports the trunk) >> vNIC Group 2# vnicvlan 1774
>> vNIC Group 2# addvnic INTA1.2
(Enable the trunk group) >> vNIC Group 2# addvnic INTA2.2
>> vNIC Group 2# addvnic INTA3.2
vNIC INT3.1 is not enabled.
Confirm enabling vNIC3.1 [y/n]: y
>> vNIC Group 2# addport INTA5
>> vNIC Group 2# addtrnk 1
>> vNIC Group 2# failover ena
>> vNIC Group 2# ena
(Specify the VLAN)
(Add vNIC pipes to the group)
(Add ports to the group)
(Enable vNIC failover for the group)
(Enable the vNIC group)
(Select vNIC group 2)
(Specify the VLAN)
(Add vNIC pipes to the group)
(Shortcut to enable vNIC)
(Add port INTA5 to the group)
(Add trunk group 1 to the group)
(Enable vNIC failover for the group)
(Enable the vNIC group)
© 2013 IBM Corporation
 System Networking

vNIC Teaming Failover

For NIC failover in a non-virtualized environment, when a service group’s external uplink ports fail or
are disconnected, the switch disables the affected group’s internal ports, causing the server to failover
to the backup NIC and switch.

However, in a virtualized environment, disabling the affected internal ports would disrupt all vNIC
pipes on those ports, not just those that have lost their external uplinks.

To avoid this, our switch provides a failover feature that acts at vNIC level, it signals only the affected
vNICs through DCBX messages. This allows the unnaffected vNICs to continue without disruption.

For a trunk uplink, the uplink is considered up

when at least one port in the trunk is up.

By default, vNIC Teaming Failover is disabled, but can be

enabled for each vNIC group using the following commands:

>> # /cfg/virt/vnic/vnicgrp <group number>

>> # failover ena|dis (Enable or disable vNIC failover)

19 IBM Confidential © 2013 IBM Corporation

 System Networking

Miscellaneous

vNIC is supported on Janice/Scooter/Skeeter/Gryphon/GryphonFC/Bingo/Pegasus/Compass/

CompassFC

LACP as uplink and uplink shared mode are available on Janice/Gryphon/GryphonFC/Compass/

CompassFC

There is a maximum number of 32 vNIC groups that can be configured

20 IBM Confidential © 2013 IBM Corporation