QUICK START GUIDE

Installing an FNA cache

April 2019

Copyrights and Trademarks

© 2019 Facebook, Inc. All rights reserved.
 Table of Contents
Physical Installation ........................................................................................................................ 3
1. Rack the Servers................................................................................................................. 3
2. Rack the Switch .................................................................................................................. 3
3. Connect the Network Cables from the Servers to the Switch ......................................... 3
4. Install Optics into Ports ..................................................................................................... 3
5. Connect the Switch to the Router .................................................................................... 4
6. Power on the FNA .............................................................................................................. 4
7. Verify Server Operation ..................................................................................................... 5
8. Check Optical Light Levels................................................................................................. 5
Network Configuration .................................................................................................................. 6
1. Configure Point-to-Point Link ........................................................................................... 6
2. Configure Point-to-Point Link ........................................................................................... 6
HPE ProLiant DL380 Generation 10 Overview .............................................................................. 8

FNA Quick Start Guide | 2

The illustrations in this guide will show a typical 4-server cache deployment. Caches can be
deployed with any number of servers from 2 to 20, so please consider these illustrations as an
example and follow the instructions taking into account the number of servers actually deployed.

Physical Installation
1. Rack the Servers
a. Install the servers into a compatible rack. Installing servers in a two-post rack requires
appropriate conversion hardware.
b. Connect the power cables from each server to the Power Distribution Units (PDUs).
AC power supplies are illustrated. For DC power installations, please consult an
electrician.

2. Rack the Switch

a. Install the switch into the rack with the airflow as port side exhaust (switch ports must
face the rear of the rack as shown in Figure 2-1).

Figure 2-1 Figure 2-2

3. Connect the Network Cables from the Servers to the Switch

a. Connect the network cable (green sticker facing up) to the Network Interface
Controller (NIC) port as seen in Figure 2-1 and 2-2.
b. Connect a server at the highest position in the rack (fna001) to QSFP+ port 32 on the
switch. Utilize the illustration as reference for port connectivity for the remaining
servers (four are illustrated in Figure 2-1).

4. Install Optics into Ports

a. Install QSFP-SFP adapters, optical transceivers and QSFP optics into ports 1, 2, 3 and 4
as shown in Figure 2-1.

FNA Quick Start Guide | 3

 b. For Clusters over four servers, follow the port assignments illustrated in Figure 4-1.

Figure 4-1

5. Connect the Switch to the Router

a. Install QSFP-SFP adapters, optical transceivers and QSFP optics into ports 1, 2, 3 and 4
as shown in Figure 2-1.

6. Power on the FNA

a. Press the power buttons on all four servers as shown in Figure 6-1.
b. Connect two power cables to the network switch as shown in Figure 6-1 (the switch
powers on when connected to power).
c. The switch’s system status LED displays as blinking green while powering up, then
steady green for normal and amber indicates a fan disconnect or malfunction. Port
LEDs display green for up, yellow for software disabled and flashing yellow for failed
diagnostics (refer to Figure 6-2).
d. If USB install is needed, refer to the FNA USB Boot Disk Creation and IP
Reconfiguration Guide on the Network Partner Portal (NPP).

FNA Quick Start Guide | 4

 Figure 6-1 Figure 6-2

7. Verify Server Operation

a. Verify server operation by checking the behavior of the LEDs as illustrated on each
server.

Figure 8-1

8. Check Optical Light Levels

a. Check the optical light levels to ensure the cluster has adequate signal stretch. Light
levels need to be between -2 dB and -7 dB.

FNA Quick Start Guide | 5

Network Configuration
1. Configure Point-to-Point Link
The point-to-point connection is configured on the Link-Aggregation Control Protocol (LACP)
interface. Make sure the following Parameters are configured as follows:

i.Set a static route to the allocated subnets via the link-aggregation

interface.
ii. Set the appropriate maximum transmission unit (MTU) for your router
software:
1. For CISCO IOS XR router software, set the MTU to 1514. CISCO IOS
SR software includes both the L2 and L3 overhead in the interface
MTU command.
2. For all other router software, set the MTU to 1500.
The FNA must have internet connectivity BEFORE BGP peering can be established.
The BGP peering session does not need to be routed before the FNA has internet
connectivity.

The FNA kit is preconfigured with the IP addresses that were specified in the order.
If IP addresses need to be changed at any time, please contact FNA Operations via
email;
fna-ops@fb.com.

2. Configure Point-to-Point Link

iii. Use the IPv4 and IPv6 addresses originally indicated in the FNA order form.
iv. (Optional) Enable Graceful Restart (GR). Enabling GR is not required, but it
is recommenced.
v. Enable External BGP (EBGP) Multihop.
vi. Use the peering Autonomous System Number (ASN): 63293
vii. Configure your router’s BGP settings.
viii. FNA’s prefix should be announced to all Facebook peering connections
ix. All servable Facebook netblocks should be announced for network traffic
optimization (completed by the latency-based routing sampling). 1
x. After the BGP settings are saved, the FNA cluster will begin downloading
application data. Within two business days, BGP peering will be established:
1. The connection will be coming from the 16th IP address in each
allocated subnet (for example, 192.168.1.16 for IPv4, and
2001:db8::10 for IPv6).
2. FNA will not advertise any routes/prefixes.
xi. When installation is complete, contact fna-ops@fb.com

FNA Quick Start Guide | 6

FNA Quick Start Guide | 7
HPE ProLiant DL380 Generation 10 Overview

Figure 9-1

Figure 9-2

FNA Quick Start Guide | 8

FNA Quick Start Guide | 9