GigE for the MSR

Fred Kuhns
fredk@arl.wustl.edu

fredk@arl.wustl.edu Washington
WASHINGTON UNIVERSITY IN ST LOUIS
 Ethernet Forwarding Scenario 1
Packet arrives with
IP: 192.163.204.2
Destination Addr:
MAC: 08:00:20:7C:E3:25
destination
IP: 192.163.204.3
host
MAC: 08:00:20:7C:F2:45
on local
IP: 192.163.150.3
MAC: 08:00:20:54:6C:4A
192.168.204.2 network. Output port must
IP hdr Host Host Host
P3data map destination IP address
to MAC address.
Ethernet Ethernet
MSR P1
Switch Switch

Port 1: P0
IP: 192.163.204.2 Use theRouter
Address P1 Resolution
MAC: 00:00:5E:04:00:01 Host
Protocol
Port 0:
to Map 192.168.204.2 IP: 192.163.150.2
to192.163.204.4
IP: 08:00:20:7C:E3:25. MAC: 00:40:33:A3:4C:04
MAC: 00:01:03:7C:23:03
Encapsulation
Port 1: datagram in
IP: 192.163.150.1
Ethernet frame and send.
MAC: 00:01:03:7C:56:34

Fred Kuhns - 1/9/01 Washington

WASHINGTON UNIVERSITY IN ST LOUIS
2
 Ethernet Forwarding Scenario 2
Packet arrives with
IP: 192.163.204.2
Destination Addr:
MAC: 08:00:20:7C:E3:25
destination
IP: 192.163.204.3
host
MAC: 08:00:20:7C:F2:45
NOT on
IP: 192.163.150.3
MAC: 08:00:20:54:6C:4A
192.168.150.2 locally attached network.
IP hdr Host Host Host
P3data OutputForwards
port musttosend
finalto
the next hop router.
destination host
Ethernet Ethernet
MSR P1
Switch Switch

Port 1: Next hopP0 router IP address

IP: 192.163.204.2 P1
MAC: 00:00:5E:04:00:01 must be used in the ARP
Router Host
request:
Port 0: Map 192.168.204.4IP: to192.163.150.2
IP: 192.163.204.4 MAC: 00:40:33:A3:4C:04
00:01:03:7C:23:03.
MAC: 00:01:03:7C:23:03
Port 1:
Encapsulate
IP: 192.163.150.1 datagram in
Ethernet frame and send.
MAC: 00:01:03:7C:56:34

Fred Kuhns - 1/9/01 Washington

WASHINGTON UNIVERSITY IN ST LOUIS
3
 Ethernet Frame Format
Destination (6 B)
Ethernet
Header

Destination Address cont.

Source Address - (6 B)
Source Address cont. Ether Type (2 B)
Version H-length TOS Total length
Identification Flags Fragment offset
Header

TTL Protocol IP Header checksum

IP

IP Source Address
IP Destination Address

Datagram
Transport Header

IP
Fred Kuhns - 1/9/01 Washington
WASHINGTON UNIVERSITY IN ST LOUIS
4
 IP Encapsulation in Ethernet Frames
• Ethernet frame size: 64 - 1518 Bytes
• if type  1500, then IEEE frame, otherwise Ethernet V2.
Ethernet Encapsulation, RFC 894
type Pad
dst address (6) src address (6) Data (46-1500) (0-46) FCS (4)
0800

IEEE 803.2/802.2 encapsulation, RFC 1042

802.2 LLC/SNAP
len Pad
dst address (6) src address (6) Data (38 - 1492) FCS (4)
(2) (0-46)

0  len  1500

802.2 LLC 802.2 SNAP

DSAP SSAP ctl Org Code type
AA AA 03 00 0800

Fred Kuhns - 1/9/01 Washington

WASHINGTON UNIVERSITY IN ST LOUIS
5
 ARP Frame
Destination Address (6B)
Source Address (6B)
Ether Type (2B)
Hardware Address Space (2B)
Protocol Address Space (2B)
Byte length of Hardware address = 6 (1B)
Byte length of Protocol address = 4 (1B)
Operation Code 1/2(2B)
Hardware Address of Sender (6 B)
Protocol Address of Sender (4 B)
Hardware Address of Destination (6 B)
Protocol Address of Destination (4 B)
Fred Kuhns - 1/9/01 Washington
WASHINGTON UNIVERSITY IN ST LOUIS
6
 ARP Message Formats
ARP Message (28 Bytes for Request or Reply)
Host A Eth Host A IP
<eth-B> <ip-A>
ARP Request Request (01)

dst address src address type has pas hl pl op sha spa tha tpa FCS

pad
ff:ff:ff:ff:ff:ff <eth-A> 08060001 0800 6 4 01 <eth-A> <ip-A> <??> <ip-B> xx

18 Byte Pad
ARP Reply Reply (02)

dst address src address type has pas hl pl op sha spa tha tpa FCS

pad
<eth-A> <eth-B> 806 1 800 6 4 02 <eth-B> <ip-B> <eth-A> <ip-A> xx

Host B Eth Host B IP

<eth-B> <ip-A> FCS
Ethernet Header (14 B) Ethernet Data - Pad with zeros to 46 Bytes (4B)

Ethernet Frame with ARP Request/Reply - 64 Bytes

Fred Kuhns - 1/9/01 Washington
WASHINGTON UNIVERSITY IN ST LOUIS
7
 IP over ATM (rfc 791 and 2684)
Version H-length TOS Total length
Identification flags Fragment offset
IP Header

TTL protocol Header checksum

Source Address

Datagram
Destination Address

IP
Options ??

IP data (transport header and

transport data)
AAL5 Trailer

AAL5 padding (0 - 40 bytes)

CPCS-UU (0) CPCS-UU (0) Length (IP packet + LLC/SNAP)

CRC

Fred Kuhns - 1/9/01 Washington

WASHINGTON UNIVERSITY IN ST LOUIS
8
 IP Header Fields (rfc 791)
• Version - support IPv4 (4) TOS Precedense Field:
111 - Network Control
110 - Internetwork Control
• Header Length - Length in 32 bit words 101 - Critic/ECP
100 - Flash Override
(>= 5) 011 - Flash
010 - Immediate
• TOS - Prec. D T R 0 0 001 - Priority
000 - Routine
Remaining TOS Fields:
• Total Length - Length of datagram in D - 1 = Low delay
T - 1 = High Throughput
octets R - 1 = High Reliability

• Id - Assists in reassembling fragments

• Flags - 0 DF MF DF - 1 = Don’t Fragment,
MF - 1 = More Fragments

• Fragment Offset - Where fragment

belongs, offset is in octets

Fred Kuhns - 1/9/01 Washington

WASHINGTON UNIVERSITY IN ST LOUIS
9
 IP Header Fields
• TTL - router must decrement, if 0 then discard
packet
• Protocol - UDP/TCP/ICMP/RSVP to name a few
• Header Checksum - 16 bit one’s complement of
the one’s complement sum of all 16 bit words in
header
• Source Address - Sending hosts IP address
• Destination Address - Destination hosts IP
address

Fred Kuhns - 1/9/01 Washington

WASHINGTON UNIVERSITY IN ST LOUIS
10
 Packet Routing Within MSR
IP processing for FPX
Ingress Egress
1. Broadcast and Multicast
destination address SPC SPC Current VCI Support
2. IP options 1) 64 Ports (PN)
plugins plugins
3. ICMP messages 2) 16 sub-ports (SP)
4. Packet not recognized FIPL FIPL
IP IP Ethernet: Base VC used for
proc proc directly attached hosts,
subports are for hext hop
shim shim shim shim
routers
demux update
WUGS update demux

FPX InVC OutVC FPX

Link Interface

Link Interface
add shim

rem shim
shim
FIPL
... ... proc.

out port + IntBase in port + IntBase

(64 ... 127) (64 ... 127)
From Inbound VC = SPI + ExtBase
previous hop ATM uses VCs
0 <= SPI <= 15 Outbound VC = SPI + ExtBase as link layer
router or Currently support at most 4 0 <= SPI<= 15
endstation Inbound VCs: One for Ethernet or address.
currently support at most 4
Four for ATM
Fred Kuhns - 1/9/01 Washington
WASHINGTON UNIVERSITY IN ST LOUIS
11
 GigE Link Interface
Map multicast If ARP table lookup
or broadcast to fails, send ARP request
Endsystem, broadcast Send to pkt->dst
ethernet address to broadcast address,
or multicast address if bcast or mcast
drop packet. No retries
map to eaddr ARP Table
Pkt VC = 50 are made.
else (M Entries)

To Next Hop or Endstation

resolve w/ARP
IP MAC

IP1 MAC1
No ARP entry aging!
... ...
IP Header
IPM MACM
Ethernet
FPX/SPC

data IP Header
From

AAL5 trailer
VIN Table - 4 entries data
VC MyIP NhIP
50 MyIP0 0 Add Ethernet header
51 MyIP0 NhIP0 using the derived
52 MyIP1 NhIP1 destination address
53 MyIP2 NhIP2 and out source address.
To a next hop router Protocol is IP.
NH #1 = Base + 1 = 51 if VC != 50, Software creates
NH #2 = Base + 2 = 52 Lookup VC in VIN table at boot
VIN table time by writing to
NH #3 = Base + 3 = 53 returns IP used interface.
for ARP lookup
(support N = 4)

Fred Kuhns - 1/9/01 Washington

WASHINGTON UNIVERSITY IN ST LOUIS
12
 Ethernet Assigned Numbers
• RFC1700 obsoleted by online database at IANA:
– http://www.iana.org/assignments/ethernet-numbers
• Ethernet Address - 6 octets:
– 3 high-order octets = Organizationally Unique Identifier
(OUI)
– 3 low-order octets = the interface number
• Multicast bit = lsb of the MSB (xxxx xxx1)
– first byte odd => multicast or broadcast
– first byte even => unicast address
– multicast address = ((OUI | 0x0100) << 24) & Group_ID
• Ethernet Broadcast: FF:FF:FF:FF:FF:FF
Fred Kuhns - 1/9/01 Washington
WASHINGTON UNIVERSITY IN ST LOUIS
13
 IP and Ethernet Multicast
• IANA has allocated address block with OUI = 00:00:5E
– Used for unicast addresses for ”IETF standard track protocols “
– Half of Multicast addresses reserved for IP, remaining for “special
use”. Leaves 23 bits for multicast addresses:
• 01:00:5E:00:00:00 to 01:00:5E:7F:FF:FF
– Could use this block for our interface, see ethernet numbers
• IP Multicast
– Class D address, 0xE0000000 + 28 Bit Group ID
– 224.0.0.0 to 239.255.255.255 (0xE0000000 - 0xEFFFFFFF)
• IP to Ethernet Mapping
– RFC1112 - Host Extensions for IP Multicasting
– Non-unique mapping: 28 bit IP group to 23 bit Ethernet group
• 32 IP multicast groups per mapped ethernet multicast address.

Fred Kuhns - 1/9/01 Washington

WASHINGTON UNIVERSITY IN ST LOUIS
14
 Multicast: IP to Ethernet Mappings
• Network Byte Ordering, Internet Standard Bit order:
(Big-Endian)
Multicast Bit Internet Bit
0 MSB 24 LSB 47
0000 0001 0000 0000 0101 1110 0xxx xxxx xxxx xxxx xxxx xxxx
Block of Ethernet Multicast Address 23 bits
0 8
1110 xxxx xxxx xxxx xxxx xxxx xxxx xxxx
msb lsb
Class D (Multicast)
LSB
Not Used in IP to Ethernet Mapping

Fred Kuhns - 1/9/01 Washington

WASHINGTON UNIVERSITY IN ST LOUIS
15
 IP Broadcast
• No Direct Impact on GigE Interface
• IP Broadcast : default, we will not forward directed
broadcasts.
– limited versus:
• {-1, -1}. Must not be forwarded, Destination address only
– Directed broadcast:
• {Network-Number, -1}, destination address only.
– Subnet Directed Broadcast:
• {Network-Number, Subnet-Number, -1}
– Directed Broadcast to all subnets:
• {Network-Number, -1, -1}
Fred Kuhns - 1/9/01 Washington
WASHINGTON UNIVERSITY IN ST LOUIS
16
 Unicast - If we use the IANA Block

Multicast Bit set to 0

0 MSB 23 LSB 47
0000 0000 0000 0000 0101 1110 0000 0100 xxxx xxxx xxxx xxxx
IANA Block of Ethernet Addresses
16 bits
ARL Interface Number

Fred Kuhns - 1/9/01 Washington

WASHINGTON UNIVERSITY IN ST LOUIS
17
 GigE Link Interface
ARP Table (M Entries)
IP MAC receive ethernet frame: eth
IP1 MAC1 if (eth->type == ARP)
if (eth->arp->has != Ethernet/0001) Drop Frame
From Next Hop or Endstation

... ...
if (eth->arp->pas != IP/0800) Drop Frame
IPM MACM update {eth->arp->spa, eth->arp->sha} in ARP table
if (eth->arp->tpa NOT in {MyIP0, MyIP1, MyIP2})
Drop Frame // target IP not ours to FPX/SPC
if (eth->arp->op == Request/01) { Base VC
Ethernet swap source and target ARP info

To FPX/SPC
IP Header set operation to Reply
set ether header src and dst address IP Header
data send reply
data
}
// Already handled eth->arp->op == Reply/02 AAL5 trailer
// when updated cache above
else if (eth->type == IPv4)
remove ethernet header, padding and CRC
add AAL5 trailer and required padding
break into cells and send on default Base VC
else
Error, drop packet

*Unicast MAC address filtering

Fred Kuhns - 1/9/01 Washington

WASHINGTON UNIVERSITY IN ST LOUIS
18
 Notes
• Packet Received on ATM interface:
– If received on Base_VC (i.e. 50) then
• map IP destination (ip->dst_addr) to ethernet representation.
• Unicast uses ARP table, multicast and broadcast use appropriate mapping.
– Otherwise,
• lookup VC in VIN table: Table entry index = RX_VC - Base_VC.
• ARP the resulting Next Hop IP address.
– This permits a simple mechanism for “tunneling” traffic to a
gateway. This allows us to support directed broadcast and provides
a convenient mechanism for testing.
• Packet received on Ethernet interface:
– if IPv4 then send all (unicast, multicast and broadcast) to input
port processor on the Base_VC (i.e. 50)
Fred Kuhns - 1/9/01 Washington
WASHINGTON UNIVERSITY IN ST LOUIS
19
 ARP Cache
• IP Address = Network_Prefix.Host or simply Net.Host
– Assume a prefix length of at least 24 bits, leaves 8 bits for the host
– An interface can have at most 3 unique IP addresses
• Interface may communicate with at most 256 hosts per network
• Implement ARP cache as a table with 768 entries (3 * 256)
• See next slide ARP Table
VIN Table IP Ethernet
Entry Prefix Local IP Next Hop
IP0,0 Ether0,0
Number Mask Address IP Address
0 Mask0 MyIP0 NH0
... ... Net 0
IP0,255 Ether0,255
1 Mask1 MyIP1 NH1
2 Mask2 MyIP2 NH2 IP1,0 Ether1,0
... ... Net 1
Net 0 = Mask0 & MyIP0 IP1,255 Ether1,255
IP2,0 Ether2,0
Net 1 = Mask1 & MyIP1 ... ... Net 2
Net 2 = Mask2 & MyIP2 IP2,255 Ether2,255
Fred Kuhns - 1/9/01 Washington
WASHINGTON UNIVERSITY IN ST LOUIS
20
 Implementing the ARP Table
‘get next packet’: VIN Table
// received frame from ATM interface Entry Prefix Local IP Next Hop
if (RX_VC == Base_VC) Number Mask Address IP Address
ipdst = ip->dst_addr; 0 Mask0 MyIP0 NH0
else 1 Mask1 MyIP1 NH1
ipdst = VIN_Table[RX_VC- Base_VC].NextHop
2 Mask2 MyIP2 NH2
// ipdst == IP Address of host we must send packet to
// determine network
for (i = 0; i < 3; i++) { ARP Table
if ((ipdst & Maski) == (MyIPi & Maski)) { IP Ethernet
index = (i << 8) | (ip->dst_addr & ~Maski)
IP0,0 Ether0,0
break; }
... ...
if i == 3 ; drop packet, goto get next packet
// i corresponds to the Network Number (0 - 2) IP0,255 Ether0,255
if (ArpTable[index].EtherAddress != 00:00:00:00:00:00) { index IP1,0 Ether1,0
construct ethernet frame ... ...
send packet
IP1,255 Ether1,255
goto ‘get next packet’
} else { IP2,0 Ether2,0
send ARP Request for ipdst ... ...
drop packet, goto ‘get next packet’} IP2,255 Ether2,255
don’t need to store IP address
Fred Kuhns - 1/9/01 Washington
WASHINGTON UNIVERSITY IN ST LOUIS
21
 Notes and Issues
• GigE Control Interface for Software configuration.
1. Reset interface to defaults
2. Clear ARP cache
3. Read ARP table
4. Read VIN table
5. Read ethernet address
6. set VIN table entries and other registers
• Set BASE VC (currently 50)
• Set Entries in the VIN table
• Add static ARP entries??

Fred Kuhns - 1/9/01 Washington

WASHINGTON UNIVERSITY IN ST LOUIS
22
 Notes and Issues
• Comprehensive testing scenarios need defining
• verify multicast and broadcast
• VC to control line card

Fred Kuhns - 1/9/01 Washington

WASHINGTON UNIVERSITY IN ST LOUIS
23
 References
• RFC 1122 - Requirements for Internet Hosts
– Must send and receive using RFC-894 - compliant
– Should receive RFC-1042 mixed with RFC-894 - we do not
– May send using RFC-1042 - we do not
– Must use ARP
– Must flush out-of-date ARP cache entries - not compliant
– Must prevent ARP floods - we only try once
– Should have configurable ARP cache timeout - no
– Should save at least one (latest) unresolved (by ARP) packet - no
– Must report broadcasts to IP layer - compliant
– IP layer Must pass TOS to link layer - via the header
– Must Not report no ARP entry as “destination unreachable” -
compliant

Fred Kuhns - 1/9/01 Washington

WASHINGTON UNIVERSITY IN ST LOUIS
24
 References
• RFC-826 : Address Resolution Protocol
– Maps <protocol, address> to 48 bit Ethernet address
– our processing differs in minor ways
• RFC 1700 : Assigned Numbers
– Ethertype values defined by RFC 1700
– IP to ethernet multicast address mapping defined
• RFC-1812 : Requirements for IPv4 Routers
– Must not believe ARP reply if contains multicast or broadcast
address - not compliant
– Must be compliant with RFC 1122 - Partial
• Support Ethernet V2 only
– RFC 894: IP encapsulation in Ethernet V2 - Supported
– RFC 1042: IP encapsulation in 802.3 frames - Not Supported
Fred Kuhns - 1/9/01 Washington
WASHINGTON UNIVERSITY IN ST LOUIS
25