KNOWLEDGE BASE

Knowledge Base Article: 000016599

How to configure LACP between a Cisco switch and an Isilon cluster running OneFS 6.5 or later
(000016599)
Version:8

Audience: Level 30 = Customers Article Type: How To

Last Published: Mon Jun 22 18:50:02 GMT 2015 Validation Status: Final Approved

Summary:

Goal: Introduction
This article describes how to set up LACP between a Cisco Gigabit switch and an Isilon cluster running OneFS 6.5 or
later.
Link aggregation is also known as NIC aggregation. For a more comprehensive discussion of link/NIC aggregation,
including options and interface names, see the OneFS Administration Guide starting with the "NIC aggregation settings"
section.
IMPORTANT!
Isilon does not recommend configuring the cluster for link aggregation and connecting it to multiple switches unless the
switches support one of these technologies: Cross-Stack EtherChannels, Multi-Chassis EtherChannels (MEC) or Virtual
Port Channels (VPC).
NOTE
When configuring Link Aggregation Control Protocol (LACP) between switches, network administrators typically use
trunking mode. Trunking is a switch feature that must be set up on both sides of the interface. However, Isilon clusters do
not use trunking and must be configured differently.
Requisite tools or skills
To perform these tasks, you should be familiar with the OneFS web administration interface, be knowledgeable about
your network configuration, and competent running Cisco IOS commands to configure a Cisco switch.

Resolution: Procedure
IMPORTANT!
Removing and adding links to address pools happens immediately after you submit the changes, which disrupts existing
user sessions. To avoid disruption, implement the changes below during a planned maintenance window.
NOTE
Aggregated links on an Isilon cluster are passive, meaning that network traffic can flow across the links set even if the
Cisco switch is not yet configured for LACP. Cisco switch LACP mode must be configured as active. After LACP is
configured on the switch, link aggregation is enabled on the Isilon cluster. To minimize link interruption, Isilon
recommends configuring the Isilon cluster before configuring the switch. Aggregated interfaces are created automatically
on the node and only become LACP links after the aggregate interface is put into a pool and the switch sends an LACP
message.
NOTE
Cisco Nexus switches can interfere with interfaces being used in LACP mode, resulting in interfaces not powering up
when they should. To keep the cluster's interfaces online and functioning properly with LACP, disable Control Plane
Policing (CoPP) on the Cisco Nexus switch.
Phase 1: Configure aggregated interfaces on the Isilon cluster
1. In the OneFS web administration interface, do one of the following:
In OneFS 7. 0 and 7.1, click Cluster Management > Network Configuration.
In OneFS 6.5 and earlier, click Cluster > Networking.
2. In the Subnets section, click the link for the subnet you want to edit (for example, subnet0).
3. In the IP Address Pools section, click the + sign next to the pool you want to edit and then click Edit next to Pool
members.
4. In the Configure Pool Interface Members dialog, select Link Aggregation Control Protocol (LACP) from the
Aggregation mode menu.
5. Do one or both of the following to create the desired configuration of aggregated NICs:
In the Interfaces in current pool pane, click the desired interfaces (for example, ext-1, Node 1), then click the left arrow
button to remove the active external interfaces from the pool.
In the Available interfaces pane, click the desired aggregate interfaces (for example, ext-agg-1), then click the right
arrow button to add the aggregate interfaces to the pool.

IMPORTANT!
Aggregated interfaces must be of the same type. For example, you cannot mix a 1 GigE interface and a 10 GigE
interface.
6. Click Submit.
7. Verify that the configuration is functioning correctly:

a.
7.

a. Log in to the OneFS web administration interface of a different node on the cluster, ideally a node that is not in a pool
with aggregated interfaces.

NOTE
Verifying the configuration from a node other than one that is configured with aggregated interfaces ensures that you do
not inadvertently change settings and cause one of the interfaces to go down.
b. Do one of the following:
In OneFS 7.0 and 7.1, click Cluster Management > Network Configuration.
In OneFS 6.5 and earlier, click Cluster > Networking.
c. In the Subnets section, click the link for the subnet you changed.
d. Confirm that the Settings are correct. If not, correct them.
e. In the IP Address Pools section, for Pool members, click Edit and confirm it displays the correct information. If not,
correct the pool membership.

The following picture shows a sample LACP pool configuration.

8. Repeat steps 1-7 for all nodes to which you want to add link aggregation.
Phase 2: Configure the Cisco switch
CAUTION!
The procedure below does not represent a complete session connected to the switch and provides limited configuration
details only. Do not attempt this procedure unless you are competent configuring your Cisco switch. For detailed session
and configuration information, see the documentation for your Cisco switch.
NOTE
Cisco switches can configure switch ports in either Layer 2 Ethernet mode or Layer 3 IP mode. The Isilon cluster
requires ports to be configured in Layer 2 Ethernet mode (the IP addresses for the links are assigned by OneFS).
Perform the following steps for each switch:
1. Connect to the switch using a serial cable and log on to the switch. For specific instructions on how to do this, see the
Cisco switch documentation.

2. Create a virtual Layer-2 port channel for each node interface in the aggregate, and assign one or two physical interfaces
to the port channel.

NOTE
Each node has its own port channel. Both interfaces on a node (ext-1 and ext-0) belong to the same port channel. You
cannot have interfaces from different nodes in the same port channel.
Although Cisco allows up to eight interfaces per port channel, Isilon normally assigns up to two: one for each interface on
the node that is connected to a switch.
a. Run the following three commands, where <port-channel-number> is a number you assign to the port channel,
<node> is the logical node number (LNN), and <interface> is the physical interface on the node. The switchport
command makes it a Layer-2 port channel.

Suggestion: It is helpful, but not required, to make the port channel number the same as the node number.

interface port-channel <port-channel-number>

description LACP channel for Isilon Node <node>-<interface>

switchport
 For example, for node 1 on the ext-1 interface, run the following commands:

interface port-channel 1

description LACP channel for Isilon Node 1-ext-1

switchport
b. Repeat step 2a for each node and interface in the cluster that you created link aggregation on. You need to have a
different port channel for every node. For example, for node 2 on the ext-1 interface, you might run the following
commands:

interface port-channel 2

description LACP channel for Isilon Node 2-ext-1

switchport
3. Optional: If needed, create VLANs by running the following commands, where <vlan number> is the VLAN you want to
create:

interface vlan <vlan number>

For example:

interface vlan 650

4. If more than one VLAN is traversing LACP link, you must configure trunking mode on all switch port channels that carry
more than one VLAN. Run the following command for each port channel that carries more than one VLAN:

interface port-channel <port-channel-number>

switchport mode trunk

For example:

interface port-channel 2

switchport mode trunk

5. Attach each physical switch interface to its port channel that you created in the previous steps, as follows:

Run the following commands for each physical switch interface port included in the channel group (or LACP bundle) for
each node. In the commands, <switch-interface-number> is the physical interface number on the switch, <node>
is the logical node number (LNN), <interface> is the physical interface on the node, and <port-channel-number>
is the port channel you assigned in step 2. The port channel number becomes the channel group number:

interface <switch-interface-number>

description LACP channel for Isilon Node <node>-<interface>

channel-group <port-channel-number> mode active

For example, the commands below attach physical switch interface Gigabit Ethernet switch port 3 to port channel 1 and
defines the description as node 1, ext-1 interface.

interface gi1/0/3

description LACP channel for Isilon Node 1-ext-1

channel-group 1 mode active

6. To view the port channel interface on a switch, run the following command on the switch:

show run interface port-channel <port-channel-number>

For example, for the virtual switch interface port-channel 1, run:

show run interface port-channel 1

The output looks similar to the following. The word switchport in the output indicates that it is a Layer 2 port channel.

interface port-channel 1

description LACP channel for Isilon Node 1

switchport

switchport mode trunk

7. To view the physical interface, VLAN, channel group number, and mode of a switch port, run the following command:

show run interface <switch_interface number>

Example:
 show run interface gi1/0/3

The output looks similar to the following:

interface gi1/0/3

description LACP channel for Isilon Node 2

channel-group 1 mode active

8. Commit the changes to ensure they persist after a switch reboot, then log off the switch. Consult your switch
documentation for instructions.

9. If you have two switches, repeat the steps on the second switch. Remember that both interfaces on a node must belong
to the same port channel and channel group.

Product: Isilon Switches, Isilon,Isilon OneFS7.0,Isilon OneFS6.5,Isilon OneFS7.1

External Source: Primus

Primus/Webtop solution ID: emc14003130