Gigabit Ethernet, 10 Gigabit Ethernet

Gigabit Ethernet,
10 Gigabit Ethernet
Metro Ethernet
Agenda
• Introduction
• Ethernet Technology
• 10/100 Ethernet
• 1/10 Gigabit Ethernet
– Gigabit Ethernet Technology & 10 Gigabit Ethernet Technology
• Metro Ethernet
• Consortium
• Competing Technologies
• Products / Equipment
• Ethernet Block Diagrams
• Xilinx Solutions for 1/10 GbE Products
• Summary
10 Gigabit Ethernet 2
What is Ethernet?
• Developed by Xerox in cooperation with DEC & Intel
in 1976
• Ethernet specification is the basis for the
IEEE 802.3 standard
– Specifies the layer 1 (physical) and layer 2 (data-link) of the OSI reference
model
– Ethernet uses a bus or star topology
• Simple, very high speed, low cost
– Scalable data rates: 10/100 Mbps, 1/10 Gbps
– Low-cost technology will always win: VHS vs. Betamax
Ethernet History
• 1970s - ALOHA • 1980 - Ethernet - DIX v1.0
• 1973 - Robert Metclafe’s • 1982 - Ethernet - DIX v2.0
research at Xerox
• 1983 - IEEE 802.3 Standard
– 2.94 Mbps, CSMA/CD
Approved
• 1976 - Star Computer defined
a 10 Mbps version • 1995 - IEEE 802.3u - 100
• 1979 - DIX Consortium Mbps
– DEC - Transceivers • 1998 - IEEE 802.3z – 1000
– Intel - Controller chips Mbps
– Xerox - Ethernet • 2002* - IEEE 802.3ae – 10
• 1980-83: Standards Path Gbps
– From DIX to 802.3
Ethernet Dominance
• Ethernet won the LAN wars: default layer 2 protocol
– Quality standards - interoperable products
– Several manufacturers with competing standards
– Lower costs with aggressive chip development
– Evolved with the market needs - features, performance
– Ethernet has displaced Token Ring, VGAnyLAN, FDDI, ATM
• Power of installed base
– Estimated 250 million Ethernet interfaces
– > 95% of Internet traffic is transported over Ethernet
• Ethernet is moving beyond the enterprise to WAN, MAN, SAN,
home and access technologies (Long-Reach Ethernet/Ethernet in
the First Mile)
Shifting the Decimal Point -
Ethernet Migration
• 10 Mb/s to 100 Mb/s
– Shrink the collision domain
– Only two repeaters
• 100 Mb/s to 1000 Mb/s
– Extend carrier to allow meaningful collision domain
– Only one repeater
• 1 Gb/s to 10 Gb/s
– Make 10 GbE full duplex only, no collision domain
Ethernet Economics
Ethernet
Technology
Basics and Framing
Ethernet Basics
• Defines layers 1 and 2 specifications
– Physical layer (layer 1)
• Provides the electrical, mechanical & procedural specs for the
transmission of bits through a communication link, medium or
channel
– Data link layer (layer 2)
• Ensures error control & synchronization between two entities
• Includes Medium Access Control (MAC) & Logical Link Control
(LLC) sub-layers
PHY Layer Functions
• Circuit establishment and release
• Bit synchronization
• Service data unit
• Data transfer sequencing - serialization & latency
• Fault condition notification
• Network management
• Medium specific control functions
– Electrical signals, light pulses, frequencies, …
– Wire destinations, connector types, …
– Signal strengths, timings, latency, distances, …
Analog
10/100 Ethernet PHY Digital
Manchester 10 Mb/s TX
Chip Board
Encode Filter
4B/5B Scrambler Transmitter
M 125 MHz
Auto TX
I Negotiation
25 MHz
Freq Synch
20 MHz
I
De- Baseline Adoptive
5B/4B
+
scrambler Restore Equalization
Clk 10Clk
Recovery Recovery
Manchester
Decoder Link Polarity
Data Link Layer Functions
• Data link connection • Error detection
establishment and release • Identification & parameter
• Service data units exchange
• Framing • Flow control
• Data transfer • Physical layer services
• Frame synchronization • Network management
• Frame sequencing
Ethernet MAC
• Provides all functions necessary to attach an Ethernet physical
layer to the host interface
– Any physical layer chip that supports
• MII standard can attach to the 10/100 MAC
• GMII standard can attach to the 1 GbE MAC
• XGMII standard can attach to the 10 GbE MAC
Processor
Host Interface
Statistic
FIFO Generator
10/100 MAC
Ethernet PHY
Ethernet MAC
Ethernet MAC
Tx Tx Data Flow
Buffer Transmit
MAC
Handler
Management
Register Media
Counter PHY
Independent
Interface
Receive
Rx Rx Data Flow MAC
Buffer Handler
Address
Match
Serial
Interface
MAC Types
• There are 2 MAC protocols defined for Ethernet
– Half-Duplex, Full-Duplex
• Half-Duplex is the traditional form of Ethernet
– Refers to data transmission in one direction at a time
– Uses CSMA/CD protocol
– Assumes that all the "normal" rules of Ethernet are in effect on
the local network
• Full-Duplex bypasses the CSMA/CD protocol
– Full-duplex mode allows two stations to simultaneously
exchange data over a point to point link that provides
independent transmit and receive paths
CSMA/CD (Half-Duplex)
• Carrier Sense Multiple Access w/ Collision Detect
• The network is monitored for presence of a transmitting
station (carrier sense)
– Transmission is deferred if an active carrier is detected
– Station continues to monitor the network until carrier ceases
• If an active carrier is not detected, and the period of no
carrier is equal to or greater than the interframe gap, then
the station immediately begins transmission of the frame
CSMA/CD - Collisions
• While the transmitting station is sending the frame, it monitors the
medium for a collision
• If a collision is detected, the transmitting station stops sending the
frame data & sends a 32-bit jam sequence
– Jam sequence is transmitted to ensure that the length of the collision is
sufficient to be noticed by other transmitting stations
– After sending the jam sequence the transmitting station waits a random
period of time - called "back-off”
• If repeated collisions occur, then transmission is repeated
– But the random delay is increased with each attempt
– Process repeats until station transmits frame without collision
• Once a station successfully transmits a frame, it clears the collision
counter used to increase the back-off time after repeated collisions
CSMA/CD Flow
Station is ready
to send
New Attempt
Wait according to
Backoff Strategy
Sense Channel
Channel Busy
Channel Free
Transmit data & Collision Detected Transmit Jam

sense channel Signal
No Collision Detected
Transmit Complete
Variations of CSMA Protocol
(Half-Duplex)
• 1-persistent CSMA • Non-persistent CSMA
– When a station has frames to – When the channel is busy, the
transmit, it first listens to the station simply gives up and tries
channel, if the channel is idle, at a later time
the frame is sent • P-persistent CSMA
– If the channel is busy, the station – When the channel is busy, the
waits & transmit its frame as
station will keep listening until
soon as the channel is idle
the channel becomes idle (like
– If a collision occurs, the stations 1- persistent)
waits a random interval and
– Then the station transmits the
starts over again
frame with a probability of p
– Station transmits with a – The station backs off with the
probability of 1 whenever it finds
probability of q = 1 - p
the channel idle
Slot Time (Half-Duplex)
• The "slot time" is a key parameter for half-duplex Ethernet
network operation
• Defined as 512 bit times for Ethernet operating at 10/100
Mbps, and 4096 bit times for 1 Gbps
– The 512 bit slot time establishes the minimum size of an
Ethernet frame as 64-bytes
– The 4096 bit slot time establishes the minimum size of a Gigabit
Ethernet frame as 512-bytes
• The minimum transmission time for a complete frame
must be at least one slot time
Slot Time (Half-Duplex)
• Slot time establishes a limit on size of a network in terms
of the maximum cable segment lengths & number of
repeaters that can be in a path
– If the size of a network grows too big, a phenomenon known as
"late collisions" can occur
– Late collisions are considered a failure in the network
• The slot time ensures that if a collision is going to occur, it
will be detected within the first 512 bits (4096 for GbE) of
the frame transmission
Full-Duplex (FD) MAC Types
• Based on the IEEE 802.3x standard, FD MAC type
bypasses the CSMA/CD protocol
• FD mode allows two stations to simultaneously exchange
data over a point to point link
• Aggregate throughput of the link is effectively doubled
– FD 100 Mb/s station provides 200 Mb/s of bandwidth
• FD operation is restricted to point to point links connecting
exactly two stations
• Used in 1/10 Gb E only
Ethernet (IEEE802.3) Frames
Preamble - A series of alternating 1's and 0's used by the
62 bits Ethernet receiver to acquire bit synchronization.
Start Of Frame Delimiter - Two consecutive 1 bits used to acquire

2 bits
byte alignment.
6 bytes
Destination Ethernet Address - Address of the intended receiver.
The broadcast address is all 1's.
6 bytes Source Ethernet Address -The unique Ethernet address of

the sending station.
Length or Type field - For IEEE 802.3 this is the number of bytes
2 bytes of data. For Ethernet I&II this is the type of packet.
46 to
1500 Data - Short packets must be padded to 46 bytes.
bytes
Frame Check Sequence - The FCS is a 32 bit CRC calculated
4 bytes
using the AUTODIN II polynomial.
Ethernet (802.3) Frames
• Preamble
– A sequences of 64 bits used for synchronization
– Give components in the network time to detect the presence of a signal
– Begin reading the signal before the frame data arrives
• Destination & Source MAC Addresses
– The Destination MAC Address field identifies the station or stations that are to receive the
frame
– The Source MAC Address identifies the station that originated the frame
• Length/Type
– If the value of this field is less than or equal to 1500, then the Length/Type field indicates
the number of bytes in the subsequent MAC Client Data field
– If the value of this field is greater than or equal to 1536, then the Length/Type field
indicates the nature of the MAC client protocol (protocol type)
• MAC Client Data
– Contains the data transferred from the source station to the destination station or stations
– If the size < 46 bytes, then use of the subsequent "Pad" field is necessary to bring the
frame size up to the minimum length
Ethernet (802.3) Frames
• Pad
– If necessary, extra data bytes are appended in this field to bring the frame length up to its
minimum size
• Frame Check Sequence
– Contains a 4-byte CRC value used for error checking
– Source station performs CRC when assembling a MAC frame from the Destination MAC Address
through the Pad fields
– Destination station performs CRC when receiving a frame
• Interframe Gap (IFG)
– A minimum idle period between transmission of Ethernet frames
– Provides a brief recovery time between frames to allow devices to prepare for
reception of the next frame
– The minimum interframe gap is 96 bit times
• = 9.6 microseconds for 10 Mb/s Ethernet
• = 960 nanoseconds for 100 Mb/s Ethernet
• = 96 nanoseconds for 1 Gb/s Ethernet
• = 9.6 nanoseconds for 10 Gb/s Ethernet
Ethernet Frame Format Extensions
• Extension Field
– Added to the end of the Ethernet frame to ensure it would be long enough for
collisions to propagate to all stations in the network
– The extension field is appended as needed to bring the minimum length of the
transmission up to 512 bytes
– It is required only in half-duplex mode, as the collision protocol is not used in full-
duplex mode
• Frame Bursting
– Allows a station to transmit a series of frames without relinquishing control of the
transmission medium
– Burst mode applies to half-duplex mode only
– Improves the performance of GbE when transmitting short frames
Ethernet Frame Format Extensions
• Jumbo Frames
– Increase the maximum size of the MAC Client Data field from 1500-bytes to
9000-bytes
• Larger frames would provide a more efficient use of the network bandwidth while
reducing the number of frames that have to be processed
• VLAN Tagging
– Frame format extensions to support Virtual Local Area Network (VLAN)
Tagging
– The VLAN protocol permits insertion of an identifier, or "tag", into the Ethernet
frame format
– VLAN Tagging provides various benefits
• Easing network administration
• Allowing formation of work groups
• Enhancing network security
– The 4-byte VLAN tag is inserted between the “Source” MAC Address field
and the “Length/Type” field
VLAN Concept
Source: Intel
VLAN Frame
Start 802.1Q MAC
Destination Source Tag Control Length
Preamble Frame Tag Client Padding FCS
Address Address Information Type
Delimiter Type Data
7-bytes 1-bytes 6-bytes 6-bytes 2-bytes 2-bytes 2-bytes 0-n bytes 0-p bytes 4 bytes
• 1st 2 bytes - 802.1Q Tag Type which is set to 0x8100

– replaces the normal “Length/Type Field”
• Next 2 bytes
– User Priority Field (3 bits) - indicates frame priority level
– Canonical Format Indicator (1 bit) - indicates presence of a Routing
Information Field
– VLAN Identifier (12 bits) - the VID uniquely identifies the VLAN to which the
Ethernet frame belongs
VLAN Protocol (802.1q)
• 802.3ac standard defines the frame format extension to
support VLAN tagging on Ethernet networks
– New max frame became 1522 instead of 1518 (+ 4 bytes)
• 802.1q standard defines the VLAN protocol
– Allows insertion of a 4-byte identifier, or “tag”, into the Ethernet
frame format to identify the VLAN to which the frame belongs
8B/10B Coding Technique
• Invented by IBM - for low cost implementations - Ethernet
• Simplifies the requirements on bandwidth & clock jitter for devices
– Has an excellent DC balance property with a maximum run-length of 5 and a
good transition density
• Input/output bit rate of 8/10 improves transmission reliability
– Built-in special characters for commands, synchronization, and delineation
• Drawback
– 25% overhead and less bandwidth efficiency
– Line rate of 12.5 Gbps is needed to implement a transmission rate of 10
Gbps with 8b/10b
– For a serial implementation this is a significant disadvantage because
currently there are a large number of devices that support 10 Gbps, but very
few that support 12.5 Gbps
Scrambled Encoding
• SONET/WAN applications
• Allows a lower line rate for extended reach
– Virtually has no overhead & better efficiency
• Simple - can be done in hardware
• Drawbacks
– Maximum run-length is non-deterministic, has no guarantee of DC balance,
and no built-in special characters
– Cost of the devices with this encoding scheme is typically higher
• Makes using this technique less attractive for a LAN
unless extended reach is needed
• 8B/10B encoding most likely will be used in LAN/MAN while
scrambled encoding will likely be specified in the WAN
10/100 Ethernet
10/100 Ethernet MAC
• Provides
– All functions necessary to connect to the host bus
– Both bus master and slave functions
– Host buffer chaining capability for higher system performance
– Internal FIFO management necessary for efficient bus utilization
– MAC is on the lower half of the data link layer which deals with
getting the data on and off the wire
• Half-duplex - based on CSMA/CD protocol
10/100 Ethernet Interfaces
• MII - Media Independent Interface
– Standard specification for the interface between the network controller chips
& associated media interface chips
– MII automatically senses 10 and 100 Mbps Ethernet speeds
• AUI - Attachment Unit Interface
– 802.3 interface between the MAU and the NIC
– Also refers to the rear panel port to which an AUI cable might attach
• MAU - Media Attachment/Access Unit
– Provides interface between AUI port of station & common medium of
Ethernet
– Can be built into the station or be a separate device
– Performs PHY layer functions including conversion of digital data from
Ethernet interface, collision detection, injection of bits onto the network
1/10 GbE
Faster, Cheaper, Simpler

GbE & 10 GbE - Still Ethernet
• An IEEE 802.3 standards project
– Evolution from a 10 Mbps shared coaxial connection to a
dedicated 10 Gbps optical connection
• Ethernet ubiquity leads to rapid development & adoption
– Leverages features of its Ethernet predecessors
• No change to Ethernet frames
• Support Ethernet enhancements (e.g., link aggregation)
• Specify implementation interfaces to enable interoperability
– Minimizes user learning curve and support costs
• Same management architecture
• Compatibility with familiar tools
Gigabit Ethernet - Summary
• IEEE 802.3z • Support star-wired topologies
• 1000 Mbps • Support fiber & copper
• 802.3 Ethernet frame format – At least 2 km over SMF
– Preserves minimum and – At least 500 m over MMF
maximum frame size of 802.3 – At least 25 m over copper
• Full & half-duplex operation • Collision domain diameter of
• Meets all 802 requirements 200m
except Hamming distance • Accommodate 802.3x flow
control
Bytes 8 6 6 2 0 - 1500
Preamble Destination Source Length of Protocol Header, Data, and Pad
Address Address Data Field
IEEE 802.3 Frame Format
Customer Challenges
• Enterprise
– Multiple locations in one metro area
– Need to support high-bandwidth applications: imaging,
CAD/CAM, storage
• Service Providers
– Need high-speed connections between POPs
– Need flexible bandwidth, just-in-time provisioning
• Metro/MANs
– Very low cost
– Emergence of metro Ethernet services
1/10 GbE Vision
• End-to-End Ethernet
– Backbone - Coast-to-Coast, Continent-to-Continent
• Fiber, 10Gbps and up
– Metro - City-to-City, Town-to-Town
• 1Gbps migrating to 10Gbps
– Access - Last/First Mile - Ethernet to the home
• 56Kbps migrating to 1Gbps
– LAN - Desktop-to-Desktop, Floor-to-Floor
• 10/100 Mbps migrating to 1Gbps
• Integrated Services (Voice/Video/Data)
1 GbE Applications
• Ideal backbone interconnect technology for use between
10/100 Base-T LAN switches
• High-performance servers & high-end desktop PCs
• Data centers
10 GbE Applications
• Local Area Networks • Storage Area Networks
– High-speed campus backbone – Database servers
– Server farms (& back-up)
– Technical/scientific/medical
– Data centers
imaging/supercomputing
– Remove LAN bottlenecks
– Eliminate 1 Gbps link aggregation – High-resolution streaming media
issues – Local & remote data mirroring
– ERP applications – Centralized back-up
– Enterprise switches
– Storage service providers
– Edge routers
– Test Equipment • Metro Area Networks
• Wide Area Networks – Private/shared access networks
– Inter/Intra PoP connections – Inter-campus connectivity
– Distance learning – Video-on-demand, broadcast
video, video conferencing
– Telephony, Telemedicine
10 GbE Applications
The Vision: End-to-End
Ethernet
Courtesy: Octavio Morales, World Wide Packets
10-GbE in the WAN
Courtesy: Paul Botorff, Nortel networks
10GbE in the WAN
http://www.cisco.com/warp/public/cc/techno/lnty/etty/ggetty/tech/10gig_wp.htm
10GbE in the MAN
http://www.cisco.com/warp/public/cc/techno/lnty/etty/ggetty/tech/10gig_wp.htm
DataCenter Network
Gigabit and 10-Gigabit Ethernet
Courtesy: B. Tolley/3Com; Nan Chen/Nortel
ISP/NSP Network
Gigabit and 10-Gigabit Ethernet
Courtesy: B. Tolley/3Com; Nan Chen/Nortel
10-GbE in SANs
Courtesy: Arnie Thompson, Network Appliance
Ethernet-based SANs
Courtesy: Arnie Thompson, Network Appliance
Significant Market Impact
• 10 GbE market will reach $3.6 billion in 2004
– 10 GbE revenue in WAN/MAN will grow to 2500% to $1.8 billion in 2003
– 10 GbE LAN will start significant penetration in 2003
• 10-Gb Ethernet Applications
– Communications equipment
• Edge switches and terabit routers – packet-based line
cards
– Bridge/ Layer2 switch
– Router/ Layer3 switch
– Storage equipment - iSCSI line cards
– Test equipment
– PL4 to XGMII bridge
Capacity Convergence
Courtesy: Paul Botorff, Nortel networks
Gigabit Ethernet
Technology
Gigabit Ethernet OSI model
Open Systems Interconnect
(OSI) Reference Model
7. Application Example Protocols
TELNET, HTTP, SMTP

6. Presentation File transfer, mail, API's
Software Layers 5. Session
IETF UDP/TCP, SNMP

4. Transport Error Control, Endto End transmission,
retransmission.
IP
3. Network ARP, routing
2. LLC Multiplexing/Demultiplexing
2. Data Link Gig Ethernet MAC

Hardware Layers Framing, error detection, Link flow control
IEEE802 Gig Ethernet PCS/PMA
1. Physical
Line Drivers/Receivers, encoder/decoders, timing
1000BASE-LX
Building a 1000Base-X
Ethernet System
UDP,IP Line Coding, Serdes, Transceiver
TCP/IP Bus Management Frame Generation, stats
FIFO,Flow Control Auto-Neg, Clk-rec,
v4 gathering, flow control Sync, cml/pecl
8B/10B
PPC OPB Bus
P P Fiber/Copper
8
Management Generic MAC GMII PCS TBI M M MDI
10
A D LX/SX/CX
FIFO 8 8
User
Gen
Logic ®
16
OSI Layer 3 & 4 OSI Layer 2 OSI Layer 1

( Network/Transport Layer) (Data Link Layer) (Physical Layer)
Software Hardware Hardware
Building a 1000Base-X
Ethernet system
Frame Generation, Stats gathering, Flow Control, Transceiver

UDP,IP Bus Management
TCP/IP FIFO,Flow Control Line Coding, Auto-Neg, Syncronization, 8B/10B,
Disparity checks, Serdes Gen and Clock Recovery.
PPC OPB Bus

P Fiber/Copper
8
Management Generic M MDI
MAC/PCS/PMA D LX/SX/CX
FIFO 8
User
Gen
Logic
16
Applications/Target Market
Xilinx
Xilinx
Xilinx
Xilinx
Xilinx
Applications/Target Market
Xilinx
Connecting 100 Mbps workgroup to Gigabit Ethernet
I0 Gigabit Ethernet
Technology
What is 10 GbE?
• Next step in Ethernet data rate
– Preserves Ethernet frame format
– Compatible with previous generations
– Increases performance range to 1000:1
– 10x the performance at 3 to 4x the cost of 1 GbE
• The technology to continue deployment of Ethernet into
MAN/WAN environments
– Low cost MAN/WAN technology
• Uses only full-duplex transmission for MAC
GbE vs. 10 GbE
• Gigabit Ethernet (802.3z) • 10 Gigabit Ethernet (802.3ae)
– Copper & Optical Fiber media only – Optical Fiber media only
– Half (CSMA/CD) & Full Duplex only – Full Duplex only
– Carrier extension & frame burst – Throttle MAC speed
– 8b/10b coding scheme – New coding scheme (64b/66b)
– Leverage Fibre Channel PMDs – New optical PMDs
– Up to 5 km distance – Up to 40 km distance
– Direct attachment to
SONET/SDH gear
What’s Different at 10 Gbps?
• Full-duplex only (no CSMA/CD)
– Previous speeds also defined half-duplex
• Fiber optic media only (no copper cable options)
– All previous speeds included copper media
– Multi mode fiber bandwidth limits distance
– Multiple PMD choices for single mode fiber
• WAN PHY (SONET-friendly)
– WAN interface sub-layer defined for compatibility with SONET OC-192 -
maps to OC-192 frames
– Rate adaptation to SONET payload capacity
• New line code (64b/66b)
• New Ethernet MAN/WAN capability
10 GbE Layer Diagram
Source: 10 Gigabit Ethernet Alliance
10 GbE Components
• PHY standards
– A LAN PHY operating at a data rate of 10.3125 Gbps
– A WAN PHY operating at a data rate of 9.95328 Gbps
• Compatible with OC-192c/SDH VC-4-64c payload rates
• Optical transceivers - PMD interfaces
• MAC
10 GbE LAN PHY
• LAN operates at 10.3125 Gbps data rates
• Uses simple encoding mechanisms to transmit data on
dark fiber and dark wavelengths
10 GbE WAN PHY
• Operates at 9.95328 Gbps • Accomplished by addition of
data rates SONET/SDH sublayer
– Seamless compatibility with – Implemented as an optional sub-layer
SONET OC-192c/SDH STM-64 that maps encoded data stream of
DWDM/TDM the LAN PHY into the payload
• Asynchronous service envelope of a SONET frame
preserved • Supports SONET/SDH overhead
• Enables low cost access to fields & fault processing
SONET optical infrastructure
Why WAN PHY?
• It is SONET friendly
– Enables using SONET infrastructure for layer-1 transport
• SONET ADMs, DWDM transponders, optical regenerators
– Require some SONET features
• OC-192 link speed, SONET framing, minimal path/section/line overhead
processing
• … But it isn’t SONET compliant
– Connects to SONET access devices, but not directly into SONET
infrastructure
– Avoids more costly aspects of SONET
• No TDM support (concatenated OC-192c only)
• Does not require meeting SONET grid laser specifications, jitter
requirements, stratum clocking
• Minimal OAM&P
10 GbE WAN PHY
Encapsulation
• Ethernet packets in SONET frames
– A WAN PHY operating at a data rate compatible with the payload rate of OC-
192c/SDH VC-4-64c
– Define a mechanism to adapt the MAC/PLS data rate to the data rate of the
WAN PHY
10 GbE WAN-PHY Seamless
Attachment
Courtesy: Paul Botorff, Nortel Networks
10 GbE Components - PMD
• PMD objectives are PHY independent
– The 3 serial (850nm, 1310nm, 1550nm) PMDs support both the
LAN & WAN PHY
– There is no relationship between the link distance and the LAN
or WAN PHY
• Optical transceivers - PMD interfaces
– 850nm Serial, MMF, 65M
– 1310nm WWDM, MMF (300M), SMF (10km)
– 1310nm Serial, SMF 10km
– 1550nm Serial, SMF 40km
PMD Options
Courtesy: Bob Grow, Intel
Ethernet LAN Architecture
Courtesy: Bob Grow, Intel
Understanding the
Terminology
Courtesy: Jonathan Thatcher, World Wide Packets
10 GbE PHY Architectures
XAUI
PCS PMA PMD
Serial LAN MAC XGXS XGXS 64b/66b Serial Serial
PCS WIS PMA PMD

Serial WAN MAC XGXS XGXS 64b/66b Serial Serial
PCS PMA PMD

WWDM LAN MAC XGXS 8b/10b WWDM WWDM
MAC - Media Access Control

PCS - Physical Coding Sub-layer
PHY - Physical Layer Device
PMA - Physical Media Attachment
PMD - Physical Media Dependent
WWDM - Wide Wave Division Multiplexing
WIS - WAN Interface Sub-layer
XAUI - 10 Gigabit Ethernet Attachment Unit Interface
XGMII - 10 Gigabit Media Independent Interface
XGXS - XGMII Extender Sub-layer
XSBI - 10 Gigabit Sixteen-Bit Interface
10GbE Sub-layers (1)
• XGXS: XGMII Extender Sub-layer
– Extend physical chip-to-chip interconnect to 20”
– 4 x 3.125 Gb/s XAUI
– 8b/10b encode, 10b/8b decode
– Lane-by-lane synchronization (Compensates for clock disparity
between lanes)
– Loop back mode for system test
• PCS: Physical Coding Sub-layer
– Improve transition density and framing
– 64b/66b encode, 66b/64b decode
– Scrambler & de-scrambler
10GbE Sub-layers (2)
• PMA: Physical Media Attachment
– Serialize and deserialize to 10 Gbps
– 16:1 SERDES
– Clock synthesis
– Low output jitter
– CDR - clock and data recovery
• PMD - Optics: Physical Media Dependent
– O/E and E/O conversion
– Optical transmitter and receiver
10 GbE Interfaces (1)
XGMII XAUI XSBI
Fiber
MAC XGXS XGXS PCS PMA PMD Optics
• XGMII (10G Media Independent I/F)

– 156.25 Mbps, DDR (3” maximum distance)
– 74 pins: Tx/Rx (32 data, 4 control, 1 clock)
• 4 Byte-wide lanes with 1 control bit per lane
• XSBI (10G Sixteen-Bit I/F)
– Based on OIF SFI-4 interface
– 644.53 Mbps, LVDS
– 34 pins: Tx/Rx (16 data (differential signals), 1 clock)
10 GbE Interfaces (2)
• XAUI (10G Attachment Unit I/F)
– AUI - Borrowed from the Ethernet Attachment Unit Interface, X - represents
the Roman Numeral tem for 10 Gbps
– Interface extender - Extends XGMII reach (3” vs. 20” maximum distance)
– 4 differential lanes @ 3.125 Gbps
– Employs 8b/10b EMI-reduced transmission encoding
– XAUI reduces the 74 pin XGMII to 16 pins Tx / Rx (4 lanes, differential/serial)
– A path for cost reduction of system products
• Enables pluggable optical transceivers
• Enables lower IC costs (i.e., pins = cost)
• Longer trace lengths enable higher density ICs (e.g., multiple MACs) and greater
flexibility in board layout
– XAUI is identical to Infiniband & 10 Gigabit Fibre Channel
10 GbE MAC
• MAC - Unchanged from GbE
– MAC data rate of 10.000 Gb/s
• Serial PHY: 10.3 GBaud (64b/66b code)
• Parallel PHY: 4 X 3.125 GBaud (8b/10b code)
• Performs rate control for WAN PHY
• Not used for LAN PHY types
– Full duplex standard only
– Maintains 802.3 frame format and size
– New rate adaptation capability
– Extend inter-frame gap
MAC
• MAC pacing matches the MAC/PLS data rate to the WAN
PHY data rate
– The MAC transmits a longer Inter-Packet Gap between packets
when the WAN PHY is used
– The number of the bytes added to each IPG is proportional to
the length of the previous packet
– These extra bytes of IPG are removed during 64B/66B encoding
to effectively reduce the data rate
10 GbE Benefits
• Brings Ethernet high • Logical path forward to aid in
volume/low cost cost model to the design of future proofed
10 Gbps networks networks
• Scales network backbones in • Aggregates 1 GbE (1000Base-
enterprise core and in SP X, 1000Base-T)
Internet data centers • Enable end-to-end networking
• Leverages 250 million – Integration of LAN, MAN & WAN
Ethernet ports installed base into one seamless network
• Supports all IP services • Compatibility with SONET/
– Data, packetized voice & video SDH OC-192 installed base
• Provides advanced – Leverages existing OC-
192c/SDH-64 core Internet rates
management capabilities for faster time-to-market
Metro Ethernet
10 GbE in the Metropolitan Area

Networks
MAN Dynamics
• Extremely heavy traffic between campuses
– Large campuses have several thousand users per site
– Inter-LAN traffic often reaches multi-gigabit range today
– This traffic will continue to grow
• Interconnecting campus LANs on a single, shared, high-
capacity MAN ring can seamlessly link all sites on a VPN
under a common Ethernet protocol
– No translations required between sites
– Very high traffic concentration between ring nodes
What is Metro Ethernet?
• A Metro Ethernet Network (MEN) offers Ethernet access
and Ethernet based services across a MAN
• Some service providers have introduced a MEN-like
technology for the Wide Area Network (WAN)
Ethernet in the Metro
• Packet traffic is becoming the dominant traffic type in networks worldwide
• Rapid growth in sites demanding high speed access and flexible services
• Ethernet is a mature, predictable, & well understood data networking technology
• Significantly lowers total cost of ownership, which benefits carriers & their
customers
Advantages of 10 GbE
in the Metro
• LAN and WAN-compatibility
– Provides opportunity to implement dependable IP VPN services
• Secure, reliable, scalable applications
• Implementation of a single management infrastructure that can be
scaled to meet the anticipated customer base size
• Facilitating customer control of effective security policies and
applications
• Reduced latency with guaranteed SLAs (service-level agreements)
• Reliable & less expensive alternative to private/leased line
communication networks
Metro Ethernet Transport
Lowering the Cost
of the Network
• Reducing unnecessary layers
of equipment significantly
• Lowers equipment cost
• Lowers operational cost
• Lowers complexity
• Simplifies architecture
Consortium
10 Gigabit Ethernet Alliance

Metro Ethernet Forum
10GEA - 10 Gigabit
Ethernet Alliance
• 100+ Members - Xilinx is a Participating Member
• Mission
– Promote industry awareness, acceptance, and advancement of
technology and products based on the emerging 10 Gigabit
Ethernet standard
– Accelerate industry adoption by driving technical consensus and
providing technical contributions to the IEEE 802.3ae Task
Force
– Provide resources to establish and demonstrate multi-vendor
interoperability of 10 Gigabit Ethernet products
MEF - Metro Ethernet Forum
• Non-profit organization dedicated to accelerating the
adoption of optical Ethernet as the technology of choice in
metro networks worldwide
• Comprised of 67 members as of January 2002
– Leading service providers, major incumbent local exchange
carriers, top network equipment vendors
• Mission - Accelerate adoption of optical Ethernet as the
technology of choice in metro networks worldwide
MEF - Key Objectives
• Build consensus and unite service providers, equipment
vendors, end customers on optical Ethernet
• Facilitate implementation of optical Ethernet standards to
allow delivery of Ethernet services and make Ethernet-
based metro transport networks carrier class
• Enhance worldwide awareness of the benefits of optical
Ethernet services and Ethernet-based metro transport
networks
• Enable Ethernet applications and services, building on the
physical transport specified by e.g. 10GEA
Competition
High-Speed Network for
Data, Voice and Multimedia
Equipment
Interconnection Devices
• Repeater: PHY device that restores data and collision
signals: a digital amplifier
• Hub: Multi-port repeater + fault detection
• Bridge: Data link layer device connecting two or more
collision domains. MAC multicasts are propagated
throughout “extended LAN”
• Router: Network layer device. IP, IPX, AppleTalk. Does
not propagate MAC multicasts
• Switch: Multi-port bridge with parallel paths
What is a Repeater?
• Repeaters are low-level devices that amplify or regenerate
weak signals
• Repeaters are needed to provide current to drive long
cables
• Repeaters are used to join network segments together to
increase the total length of the network
• Act at the physical layer and allow all traffic to cross LAN
segments
What is a Hub?
• A place of convergence where data arrives from one or
more directions and is forwarded out in one or more other
directions
• Hub is a repeater with fault detection functionality
• Connects high-performance stations/devices to Ethernet
LAN and provides high-performance inter-LAN
connectivity using switching technology
• A hub usually includes a switch of some kind

What is a Bridge?
• Connect a LAN to another LAN that uses the same
protocol (e.g., Ethernet or Token Ring)
• Works at the data-link level of a network, copying a data
frame from one network to the next network along the
communications path
• Bridges can make minor changes to the frame before
forwarding it (such as adding and deleting some fields
from the frame header)

What is a Switch?
• Network device that selects a path or circuit for sending a
unit of data to its next destination
– Switch is a simpler & is faster than a router
• Simultaneous switching of packets between its ports increases the
aggregate LAN bandwidth dramatically
• Network device processing packets at layer 2 & 3
– Layer 2 Switch:
• Filters & forwards at the data link layer of the OSI model
• Uses MAC address to determine where frames are sent
– Layer 3 Switch:
• Routes packets at wire speed using Layer 3 information
What is a Router?
• Determines next network point to which a packet should
be forwarded on the way to its final destination
– Routers use the network layer information within each packet to
route it from one destination/LAN to another
• Must recognize all the different layer 3 protocols that are used on
the networks it is linking together
– Routers communicate with one another to determine the best
route through the complex connections of many LANs to
increase speed and cut down on network traffic

What is a NIC?
• NIC - Network Interface Card
• NIC is an expansion card used to connect a PC, server, or
workstation to a LAN
• NIC provides an interface between the network and the
PC’s bus
• Most NICs are designed for a particular type of network,
protocol, and media
• The NIC segments outgoing messages into packet
formats specified by the LAN protocol for transmission

Ethernet Block
Diagrams
Core Router
1/10GbE Line Card
Backplane / Switch Fabric Card
Control Plane Network
Processor Memory Co-Processor (s)
Backplane Switch
10GbE NPU / Backplane I/F Fabric
Optics PHY
MAC ASIC I/F
Memory
xN
OC-48 / OC-192 POS/ATM Line Card Central Management Board
Control Plane Network 10GbE SONET ATM

Processor Memory Co-Processor (s) Mgr. Mgr. Mgr.
Framer / NPU / Backplane SDRAM / SRAM / FLASH

Optics PHY
Mapper ASIC I/F
Buffer I/O
CPU
xN
Mgr. Mgr.
System Bus

Edge Router
OC-12 / OC-3 POS or ATM
Backplane / Switch Fabric Card
Framer Backplane
Optics PHY
Mapper I/F
Backplane Switch
I/F Fabric
10 / 100 / 1000 Ethernet Linecard
Memory
10/100/Gig Backplane
PHY
MAC I/F
Central Management Board

T/E Linecard
TE T/E Backplane SONET ATM

I/O Mgr.
TX / RX Framer I/F Mgr. Mgr.
SDRAM / SRAM / FLASH

DSL Linecard
xDSL HDLC Backplane Buffer I/O

CPU
TX / RX Controller I/F Mgr. Mgr.
System Bus

Enterprise Router
Memory
10/100 10/100 SDRAM
RJ-45 I/F
PHY MAC
DMA Flash /
CPU
Controller SRAM
UART I/F Driver RS232

10/100 10/100
RJ-45
LAN PHY MAC
Side
T/E
T/E Framer
TX/RX
xDSL WAN
HDLC Side
802.11a/b TX / RX
802.11a/b
Baseband (to CO)
Radio
Controller ATM OC-3
ATM SAR
PHY
UTOPIA
PCI Bus (32/33 or 64/66) or

Proprietary Bus

SOHO Router
802.11a/b
802.11a/b
Baseband Memory
Radio SDRAM
Controller I/F
DMA Flash /
10/100 10/100 CPU
RJ-45 MII
MAC
Controller SRAM
PHY
LAN
Side
UART I/F Driver RS232
Home
AFE
PNA
T1/E1 Clock Generator
T/E & DLLs
USB T/E Framer
USB TX/RX
UTP Device
Transceiver DSL
Controller xDSL WAN
HDLC
TX / RX Side
(to CO)
IEEE 1394 /Firewire Cable
QAM Decoder
& FEC
RS-232
PCI Bus (32/33 or 64/66) or

Proprietary Bus

GbE Switch
Magnetics GbE GbE Packet
PHY MAC Memory
Address
GbE GbE
Magnetics
PHY MAC
Cache
Switch
Fabric Network
ASSP Management
CPU
Memory
I/F
GbE GbE
Magnetics
PHY MAC Memory

10 GbE Router Line Card
Control
Plane Memory System Bus
Processor
Memory
Memory
Controller
Controller
P P Classifi- NPU Traffic BP BP

Integ. 10GbE Sche-
10 GbE M C cation or Queue SER / Optics
Optics MAC Mgr.
duler
A S Engine ASIC DES
XAUI XGMII Security Parallel

Parallel
Proc. I/F
I/F
System
Clock
Other Line Card,

Switch Fabric, or
Network Mgmt. Card

GigE Router Line Card
Control
Processor
Memory
Memory
Controller
Controller
Serial I/F
GbE NPU or Backplane
Magnetics PHY SERDES Driver
MAC ASIC SERDES
Receiver
GMII Parallel
Parallel
I/F
I/F
System
Clock
Other Line Card,

Switch Fabric, or
Network Mgmt. Card

10/100 Ethernet Router
Line Card
Control
Processor
Memory
Memory
Controller
Controller
Serial I/F
10/100 NPU or Backplane
Magnetics PHY SERDES Driver
MAC ASIC SERDES
Receiver
MII Parallel
Parallel
I/F
I/F
System
Clock
Other Line Card,

Switch Fabric, or
Network Mgmt. Card

Xilinx Solutions for
1/10 Gigabit Ethernet
Virtex-II Pro SystemIO + 1/10
GbE MAC
Virtex-II Series Expanded to
Include Virtex-II Pro
• Virtex-II Logic, Routing, Features
– Upward compatible, same design tools
– Embedded Multipliers, SelectIO-Ultra (with
840Mbps LVDS), DCI/XCITE, DCM
• Up to 16, 3.125 Gbps serial transceivers
– Channel bonding, 8b/10b encoding
– Supports high-speed interfaces GbE,
10GbE (XAUI), PCI/PCI-X, Infiniband,
RapidIO, HyperTransport, FlexBus 3/4,
POS-PHY 3/4
• Up to four IBM 405 PowerPC®
– 32-bit RISC CPU: 420 DMIPS @ 300 MHz
– The leading embedded CPU architecture in
telecom & networking infrastructure
– IBM CoreConnect™ on-chip bus
Virtex-II Pro and IP Solutions will Further Enable Next
Generation Networking and Telecom Products
Virtex-II Pro Features
50 :
50 :
BRAM
18 Bit
36 Bit
Impedance
18 Bit Controller
Embedded DSP Embedded Dual-Port RAM - XCITE Digitally-Controlled

functionality - up to for Data Buffering Impedance (DCI) for
500 Billion MAC/s simpler PCB layout
4000
CLKIN CLK0 4000XL Virtex

CLK90 3500
CLKFB CLK180
CLK2X 3000
RST CLK270
Delay (ps)
CLK2X180
2500
DSSEN
PSINCDEC CLKDV 2000
CLKFX180
PSEN CLKFX 1500
Virtex-II
PSCLK STATUS[7:0]
LOCKED 1000
PSDONE
500
Φαστεστ ΦΠΓΑ ον τηε
clock signal Πλανετ
control signal 0
0 200 400 600 800 1000
Digital Clock LUTs Reached

SelectIO™
Management (DCM) - Active Interconnect
25 IO types including
Precise Clock Technology with a 300 MHz
840 Mbps LVDS
Generation System Clock

Virtex-II Pro Family

Virtex-II Pro RocketIO
Technology
Transceiver Module
8B/10B
TXDATA FIFO Serializer Transmit TX
Encode Buffer +
TX-
TX Clock Generator
Loopback
Transmitter
31.1 MHz – REFCLK 20X Multiplier
156.3 MHz
Comma Detect Receiver
and Word
Alignment
RX Clock Generator
Elastic 8B / 10B Receive RX+

RXDATA Deserializer
Buffer RX-
Buffer Decode
• Up to sixteen multi-gigabit serial transceivers

• Support 622 Mbps to 3.125 Gbps
• Most complete feature set
Physical Media Attachment
(PMA)
• Programmable features
– 300 mV to 800 mV output
Transceiver Module
– 50 ohm or 75 ohm on-chip
8B/10B
termination
TXDATA FIFO Serializer Transmit TX+
Encode Buffer TX- – 4 levels of pre-emphasis to
TX Clock Generator
Transmitter support up to 20’’ of FR-4
42 MHz – REFCLK 20X Multiplier Loopback pc-board trace
156.3 MHz Receiver
Comma Detect
and Word
Alignment RX Clock Generator – Comma detect
RXDATA
Elastic
Buffer
8B / 10B
Decode
Deserializer Receive
Buffer
RX+
RX-
• Supports AC or DC coupling
• Local loop-back capability

Physical Coding Sublayer
(PCS)
• 8B/10B line coding logic
Transceiver Module • Transmit FIFO for clock
8B/10B
synchronization
TXDATA FIFO Serializer Transmit TX+
Encode
TX Clock Generator
Buffer TX-
• Receive Elastic Buffer
Transmitter for channel bonding
Loopback
42 MHz – REFCLK 20X Multiplier
156.3 MHz
Comma Detect
and Word
Receiver • Flexible FPGA interface
Alignment RX Clock Generator
Elastic 8B / 10B RX+

– 1, 2, and 4 byte
RXDATA Deserializer Receive
Buffer RX-
Buffer Decode
All of the above can be optionally bypassed

Virtex-II Pro Helps You Manage the
Transition from Parallel to Serial I/Fs
IOB
Input SelectI/O-Ultra™
Reg DDR mux
OCK1 Reg
technology for parallel
ICK1 interfaces
Reg
OCK2 3-State Reg
ICK2
• 25 I/O Standards
Reg DDR mux
OCK1
• XCITE Technology
PAD
Reg
• 840 Mbps LVDS
OCK2 Output
• Dedicated DDR
Registers
Gb Serial Gb Serial DDR P

Transceiver Transceiver FF N
32b @ 32b @
78 MHz 78 MHz IOB
• Helps preserve investment in legacy designs

Rocket I/O Multi-Gigabit Serial
Transceivers • Eases transition from parallel to serial technology
Up to Sixteen 3.125 Gbps transceivers • Parallel interface designs will not go away

Complete GE MAC Offerings
• First in programmable industry to provide complete
Gigabit Ethernet (GE) MAC solutions
– 1-GE MAC with GMII interface
– 10-GE MAC with XGMII interface
• Part of the comprehensive Platform FPGA SystemIO
solution for emerging system connectivity standards

1-Gb Ethernet
MAC Core
1-GE MAC Technical Details
- Facts & Features
1 GIG ETHERNET FEATURES
1 GIG ETHERNET FACTS.
Static optional modes
Core Specifics
¾ PCS/PMA: Full Duplex/1000Base-X only.
Supported Family Virtex,Virtex-II, ¾ GMII: Full/Half Duplex/1000Base-X or T.
Spartan2e, Virtex2Pro ¾ Statistics: 41counters based on the IETF
Performance 125MHz EtherStats -- RFC2819.
¾ Host
Global Clock Buffers 2 ¾ Dist_mem vs Block mem
¾ Family
Virtex II Slice Count ~471min
~1777max Dynamic Configuration register:
Special Features MGT, SRL16, DCM ¾ VLAN
Dist & Block RAM ¾ Flow Control: Pause (asymmetrically or
Runs on Software Ver. 4.1I (E.35) symetrically enabled)
¾ Jumbo frames: 9K
I/O connections 200 ¾ Half duplex or Full Duplex
¾ CRC: In-band or MAC generated
¾ Host CLK: programmable.
¾ IFG: Sets IFG stretch capability (test equipment)

Technical Details- MAC with PCS/PMA Block
Diagram (V-II Pro Only)
gtx_clk FPGA LOGIC
(125MHz) GT_Ethernet_1 (MGT)
MAC PCS
Physical Coding Sublayer Physical Media attachment
Tx Conexant IP
TX Txdata[7:0]
Tx Transmit TX+
Frame FIFO Serializer TX-
Txcharisk Buffer
Gen k-codes 8B / 10B
PAUSE Txchardispmode TX Clock Generator
& Encode
Txchardispval
Loop-back
Transmitter
Disp.
RX Txbuferr 20X Multiplier
Client Flow
I/F Control Auto Receiver
62.5 MHz (REF CLK)
Neg. RX Clock Recovery
8B / 10B Deserializer
Receive
Rx Rxdata[7:0] Buffer
RX+
Decode Comma Det. RX-
Frame Rxcharisk
Rx Rxnotintable Elastic
checks
Sync Rxdisperr Buffer
Rxbufstatus(1)
Rxchariscomma SIGNAL_DETECT
HOST Config Stats MDIO MDIO

1-Gb Ethernet MAC w/ GMII
• Single-speed half- and/or full-duplex 1-Gigabit Ethernet Media
Access Controller (GMAC)
– Fully compliant with IEEE 802.3-2000 standard
• Supports Virtex™-II (-4), Virtex-E (-7) and Spartan-IIE (-7) FPGAs
– 8b Gigabit Media Independent Interface (GMII) running 125MHz for 1Gbps
bandwidth
– Can interface to industry standard ASSP PHY chips
– 8b internal data path and back-end interface, 125MHz operation

Additional GMAC Features
• Optional support for VLAN frames
– Virtual LAN allows network administrators to reduce the time it
takes to implement moves, adds, and changes
• Optional support for JUMBO frames (9k max)
• Powerful statistics gathering
– Allows remote network administration
• Configured and monitored through an independent micro-
neutral interface
• Optional flow control through MAC Control pause frames
• Integrated MDIO port for management of PHY layers

1-Gb Ethernet MAC IP Core
Back-End
Interface GMAC CORE
GMAC Internal CORE GMII
TX Data 8 8
Transmit TXD[7:0]
TX Valid TX TX_ER
Engine
I/O TX_EN
TX_CLK
Pause Req Flow Deference
Pause Value 16 COL
Control and Backoff CRS
8
RX Data Receive 8
RX Valid RXD[7:0]
Engine RX
RX_DV
I/O RX_ER
Statistics RX_CLK
Management I/F and Configuration
Compliant IEEE 802.3-2000

Gigabit Ethernet
Applications
• Communications Equipment
– GbE Network Interface Cards (NICs)
• PCI-, PCI-X- and Cardbus-based
– Edge switches and terabit routers – packet-based line cards
• Storage Equipment
– iSCSI line cards
• Numerous bridge opportunities
– PL3 to Gb Ethernet

10-Gb Ethernet
MAC Core
10-Gb Ethernet MAC IP Core
Back-End
Interface XGMAC CORE
XGMAC Internal CORE XGMII

64 64
TX Data Transmit TXD[31:0]
TX Ctrl TX
Engine TXC[3:0]
I/O TX_CLK
Pause Req Flow

Pause Value 16 Control
64
RX Data Receive 64
RXD[31:0]
RX Ctrl Engine RX
RXC[3:0]
I/O RX_CLK
Statistics
Management I/F & Configuration
Designed to IEEE 802.3ae Draft D3.2

10GE MAC
Technical Details
10 GIGABIT ETHERNET MAC FACTS
Core Specifics
Supported Family Virtex-II (-5)
XC2V1000-XC2V8000
Performance 32b XGMII interface 312.5Mbps per
HSTL Class 1 drivers
64b internal data path and back-end
interface, 156.25MHz SDR operation
Global Clock Buffers 3
Virtex-II Slice Count 3629 slices
Special Features Integrate digital clock management
Runs on Software Ver. 4.1i
I/O connections 77 external, 249 internal

10-GE MAC with XGMII
• Single-speed full-duplex 10-Gbps MAC controller
– Accelerates low-risk migration to next-generation networks
– Compliant with IEEE P802.3ae draft D3.2 (to be ratified 3/02)
• Xilinx is active member in IEEE 802.3ae committee
– Supports LAN (10.3Gbps) & WAN (OC192 9.953Gbps) rates
• Interoperates with industry-standard PHY ASSPs
• Available now as a LogiCORE™ product from Xilinx
– Downloadable over the Internet
– Occupies 33% (3269 slices) of Virtex-II 2V2000

Additional 10-GE
MAC Features
• Optional support for VLAN frames
– Virtual LAN allows network administrators to reduces the time it
takes to implement moves, adds and changes
• Optional support for JUMBO frames (9k max)
• Powerful statistics gathering
– Allows network administrators to manage the network
• Configured and monitored through a uP-independent
interface
• Optional flow control through MAC Control pause frames
• Integrated MDIO port for management of PHY layers

10Gb Ethernet MAC w/ XGMII
Glossary
Client P P
MAC : Medium Access Control
Interface MAC PCS WIS M M PCS : Physical Coding Sublayer
(FIFO) A D
WIS : WAN Interface Sublayer
XGMII : 10 Gig Medium Independent Interface
OSI Layer 1 (PHY) PMA : Physical Media Attachment Sublayer
XGMII
XGMII PMD : Physical Medium Dependent Sublayer
• Compliant with IEEE802.3ae, ver. D3.2 (to be ratified 03/02)

– Single speed full-duplex 10 Gbps MAC controller
• 32b XGMII interface 312.5Mbps, per HSTL Class 1 drivers
– 64b internal data path and back-end interface, 156.25MHz SDR operation
– Can interface to industry standard ASSP PHY chips
– Supports LAN (10.3Gbps) & WAN (OC192 9.953Gbps) Line rates

10GE MAC with XGMII
Interface
Back-End
Interface 10 GE MAC + XGMII CORE
XGMII
XGMAC RS/XGMII
RS/XGMII
64
TX Data Transmit
TX Ctrl Engine TX
8
I/O
Flow
16
Control
64
RX Data Receive RX
RX Ctrl
8 Engine I/O
XGMII
MDIO I/F Management MDIO I/F

Virtex-II, Virtex-II Pro

10 GE MAC with XAUI
Interface
Back-End XAUI Floorplan –
10 GE MAC + XAUI CORE 2VP7
Interface
XAUI
XGMAC PCS/PMA
PCS/PMA
64
TX Data Transmit
TX Ctrl Engine
8
Rocket I/O
Pause Req Flow Transceivers
Pause Value 16 Control
64
RX Data Receive
RX Ctrl Engine
8
XGMII
MDIO I/F Management

Virtex-II Pro only

The 10 Gigabit Ethernet
Family
In FPGA
Media Access Control (MAC)
Full Duplex
10 Gigabit Media Independent Interface (XGMII) or

10 Gigabit Attachment Unit Interface (XAUI)
WWDM Serial Serial

LAN PHY LAN PHY WAN PHY
(8B/10B) (64B/66B) (64B/66B + WIS)
WWDM Serial Serial Serial Serial Serial Serial

Can interface PMD PMD PMD PMD PMD PMD PMD
1310 nm 850 nm 1310 nm 1550 nm 850 nm 1310 nm 1550 nm
to
10GBASE-LX4 10GBASE-LR 10GBASE-SW 10GBASE-EW
10GBASE-SR 10GBASE-ER 10GBASE-LW
10GBASE-X 10GBASE-R 10GBASE-W
40 Gbps Switch Fabric
PM8355 = 4-Channel 2.0
to 3.125 Gbps
Transceiver
MGTs
MGTs
MGTs
MGTs
MGTs GMAC
GMAC/PCS/PMA
Virtex-II Pro
Virtex-II Pro
MGTs XGMAC
XGMAC/XAUI

Simple 10GbE Line Card
To Backplane
XAUI- Buffer XGMAC XAUI

like
10GBASE-R Serial LAN PHY

Memory XenPak
E/O Fiber
XAUI 64B/66B SERDES
PCS

10GbE Optical Modules
• XenPak MSA current market favourite to succeed

• 1.5” x 4.5” footprint
• XAUI interface to MAC
• http://www.xenpak.org/

10GBASE-X (WWDM)
• 10GBASE-X PCS is functionally identical to XAUI
– Some management registers move around in the address space
– Signal detect pins used
from optics
XenPak
E/O
E/O
Optical
MUX
E/O
XGMAC XAUI
E/O

A 10 Gigabit Ethernet LAN/WAN
Line Card Design Example
XenPak XAUI POS-PHY L4 CSIX RapidIO
Backplane
ASSP ASSP FPGA ASSP FPGA ASSP
Integrated 10GE 10GE MAC NPU CSIX Bridge Terabit
Optics PHY Switch
OC-192 POS LAN/WAN Fabric
• Implementation review
– 10 Gigabit Ethernet MAC (10GE)
– CSIX
– RapidIO
– POS-PHY Level 4 (PL4)

10 Gigabit Ethernet
Interfaces
Name Number of Data Rate Number of Useful Comments
Lines Per Line Clock and Transmission
Control Lines Distance
XGMII 32 TX 312.5 Mbps 4 Tx Ctrl < 7cm (3”) 74 Signal

10Gb/s 32 RX 1 Tx Clk Lines
Media 4 Rx Ctrl
Independent 1 Rx Clk
Interface
XSBI 16 TX 622 Mbps 1 Tx Clk <20 cm (8”) + 64 Signal
10Gbps 16 RX 1 Rx Clk 1 connector lines
Serial Bus LVDS
Interface
XAUI 4 TX 3.125Gbps None >50 cm (20”) 16 signal
10Gbps 4 RX + 2 connectors lines
Attachment Differential
Unit Interface

10 Gigabit Ethernet
Interfaces

Media 4 Rx Ctrl
Interface
Serial Bus LVDS
Interface
Unit Interface

XGMII
• Logical Interface between
MAC and PHY
Xilinx XGMAC
– Convenient partition for
specification
• Support multiple PHY types XGMII
• Support existing management
interface and register set PCS
PMA
PMD
Fiber

XGMII
• 4 lane parallel interface
– 32 bits Data and 4 Control bits TX
– 32 bits Data and 4 Control bits RX
• Control bit allows embedded delimiters
• DDR signalling, source centred
– 156.25MHz clock (one TX, one RX)
– 312.5 Mbps on each data/control line.

XGMII Control Codes
TXC TXD Description
0 0x00 through 0xFF Normal data transmission
1 0x07 Idle
1 0x9C Sequence (only valid in lane 0)
1 0xFB Start (only valid in lane 0)
1 0xFD Terminate
1 0xFE Error

XGMII Example Waveform
TX_C LK
TXC<3:0> 0xF 0x1 0x0 0xC 0xF
TXD<7:0> I S Dp fram e dat a I
TXD<15:8> I Dp frame da ta I
TXD<23:16> I Dp frame da ta T I
TXD<31:24> I Dp SF D frame da ta I
I: Idle c ontrol c hara cte r, S: St art control c harac ter, Dp: pream ble Da ta oc tet, T: Te rminat e c ontrol c harac te r

Fault Reporting (1)
• Local Fault and Remote Fault codes are defined
– Both start with “Sequence” (D=0x9C, C=1) in lane 0
• Allow “coarse-grained” fault finding prior to further
investigation
• Local fault can be generated anywhere in the PHY
• Remote fault only generated in the MAC in response to a
Local Fault message

Fault Reporting (2)
Normal Operation
Device A Device B
TX Idle Sequence Idle Sequence RX
MAC/XAUI MAC/XAUI
RX Idle Sequence Idle Sequence TX
• Simultaneous A and B power up and reset

• Link Status OK in both A & B
• MAC Transmission Enabled in both A & B
• No negotiation or handshakes

Fault Reporting (3)
Link Fault
Device A Device B
TX Idle Sequence No signal RX
MAC/XAUI MAC/XAUI
RX Remote Fault Remote Fault TX
• PHY in Device B detects fault, passes LF message to MAC

• Link Status FAIL in Device B
• Device B signals Remote Fault to Device A
• Link Status FAIL in Device A
• MAC Transmission halted in both A & B

XGMII Clock Management
GTX_CLK
IBUFG
XGMAC Core
DCM BUFG
CLKIN CLK0 FDDRRSE
FB CLK90 XGMII_TX_CLK
BUFG
TX_CLK
BUFG DCM IBUFG

RX_CLK CLK0 CLKIN XGMII_RX_CLK
FB

XGMII Weaknesses
• Single ended I/O standard
– HSTL Class I limited to 3” track length (unterminated)
• Lots of pins
– 78 pins for entire interface
• Physical layout is therefore difficult at best

10 Gigabit Ethernet
Interfaces

Media 4 Rx Ctrl
Interface
Serial Bus LVDS
Interface
10Gbps 4 RX +2 lines
Attachment Differential connectors
Unit
Interface

XAUI
• 10 Gigabit eXtended Attachment Unit Interface
– Used to extend the 10G Media Independent Interface
– 4 lane serial, embedded clock
• battle-tested 8B10B encoding on each lane
– Allows interconnects of over 500mm (20”) on standard FR-4
material
• Even longer on specialized materials e.g. GETek, RT-Duroid
– Protocol and PHY Independent
– Hot Swappable

Why XAUI is Important
Example: Current 10GE solution with XGMII extender
Virtex-II
TXC
TXD
36 PCS/PMA
MAC Physical Coding Four Serial Channels PMD
Media Access Sublayer/ Physical Bonded Together Physical Medium
Control RXC Media Attachment Dependent
RXD 36
Interface
Parallel Serial
XGMII XAUI
Issue: 2” max Up to 20”
• Allows up to 20” trace plus 2 connectors
• Greater layout flexibility
• Allows back plane implementations
Virtex-II Pro Integrates XAUI
V-II PRO XAUI
TX
MAC
+
PCS/PMA PMD
RX
Benefits
• Lowers pin count and device count
• Solves jitter and skew issues

XAUI Electrical Interface
• At 3.125Gbps, λ/4 = 1”
– Starting to look like Microwave Engineering again
• 100 ohm differential impedance
• Intel suggest stripline configuration
Cohn, Seymour B., "Characteristic Impedance of the Shielded-Strip Transmission Line,"

IRE Transactions on Microwave Theory and Techniques, Vol MTT-2, No. 3, July 1954, pp.
52-57.

Mapping from XGMII to XAUI
Data Control
TXD[31:0] TXC[3:0]
31 24 23 16 15 8 7 6 5 4 3 2 1 0 3 2 1 0
Byte Byte Byte Byte Lane 0= TXD[7:0]+TXC[0]

Lane 3 Lane 2 Lane 1
7 6 5 4 3 2 1 0
8+control
HGFEDCBA,K
8B10B Encoder
10
abcdefghij
0 1 2 3 4 5 6 7 8 9

XAUI Control Codes
Codegroup Description
Dxx.y Normal data
transmission
K28.5 /K/ (Sync)
K28.0 /R/ (Skip)
K28.3 /A/ (Align)
K28.4 /Q/ (Sequence)
K27.7 /S/ (Start)
K29.7 /T/ (Terminate)
K30.7 /E/ (Error)

MGTs Advantage
• 8B10B encode/decode with
error detection Transceiver Module
• Comma detection TXDATA

8B/10B
Encode
FIFO Serializer Transmit
Transmit
Buffer
Buffer
TX+
TX-
• RX elastic buffer/channel TX Clock

TX Clock Generator
Generator
Loopback
Transmitter
bonding 40 MHz -- REFCLK 20X Multiplier

156 MHz Receiver
Comma Detect
• A state-of-the-art PMA
and Word RX
Alignment RXClock
ClockGenerator
Generator
Elastic 8B / 10B Deserializer Receive RX+
(Ser/Des)
RXDATA Deserializer
Buffer Decode Buffer RX-

Idle Generation
• Idle XGMII is translated to ||A||, ||K|| and ||R||
• “Simple” rules for mapping into these codes
– First idle after data alternates between ||A|| and ||K||, unless an
||A|| is not due, then ||K|| is sent
– ||R|| is the second idle after data
– ||A|| codes are separated by no less than 16 columns and no
more than 31 columns
– If not sending an ||A||, either ||R|| or ||K|| is sent, randomly
distributed

Idle Generation (2)
XGMII
T/RXD<7:0 > I I S Dp D D D --- D D D D I I I I I I
T/RXD<15: 8> I I Dp Dp D D D --- D D D T I I I I I I
T/ RX D<23 :1 6> I I Dp Dp D D D --- D D D I I I I I I I
T/ RX D<31 :2 4> I I Dp Ds D D D --- D D D I I I I I I I
PCS
LANE 0 K R S Dp D D D --- D D D D A R R K K R
LANE 1 K R Dp Dp D D D --- D D D T A R R K K R
LANE 2 K R Dp Dp D D D --- D D D K A R R K K R
LANE 3 K R Dp Ds D D D --- D D D K A R R K K R
Le gend :
Dp re pres ents a da ta ch arac ter co ntaining the prea mble pattern
Ds re pres ents a da ta ch arac ter co ntaining the SFD pa ttern

Channel Synchronization
• Each line has to find the “byte boundary”
• Uses Comma characters in 10B stream (0011111xxx or
1100000xxx)
– This doesn’t appear anywhere else
• Follow state machine in 10GbE specification
• Not the same as the state machine within the MGTs
– Must be implemented in CLB logic

Channel Bonding
• Aligns the lanes to each other
• Can deal with up to 84UI of lane-to-lane skew (26.9 ns)
Lane 0 K K R A K R R K K R K R R
Lane 1 K R K K R A K R R K K R K
Lane 2 R K K R A K R R K K R K R
Lane 3 R K K K R A K R R K K R K

Channel Bonding (2)
• Alignment achieved according to state machine in
specification
– Must be implemented in CLB Logic
• Aligns on ||A|| code groups
• Causes read pointers in MGT elastic buffers to (hopefully)
centre on the same point
• LF sent up to MAC until lanes are deskewed

Link Initialization (1)
Device A Device B
TX Idle Sequence+RF Idle Sequence+RF RX
MAC/XAUI MAC/XAUI
RX No Signal No Signal TX
• Device A powered up and reset
• Device B powered down
• Device A detects fault
• Device A signals RF to Device B
• Link Status = FAIL in Device A
• MAC Transmission halted in Device A

Device A Device B
TX Idle Sequence+RF Idle Sequence+RF RX
MAC/XAUI MAC/XAUI
RX No Signal Idle Sequence TX

• Device B powers up and resets
• Device B signals Idle to Device A
• Device A hasn’t synchronised and deskewed yet
• Link Status = FAIL in both A and B
• MAC Transmission halted in both A and B

Device A Device B
TX Idle Sequence Idle Sequence+RF RX
MAC/XAUI MAC/XAUI
RX Idle Sequence Idle Sequence TX
• Device A RX detects idle sequence, synchronises and deskews

• Device A stops sending RF to Device B
• Device A Link Status = OK
• MAC transmission enabled in Device A
• When Device A idle reaches Device B, normal operation

XAUI Clock Management
REFCLK
IBUFG
XGMAC Core
BUFG GT_XAUI_2
GT_XAUI_2
REFCLK
GT_XAUI_2
REFCLK
DCM BUFG GT_XAUI_2
REFCLK
CLKIN CLK0 REFCLK
TXUSRCLK
FB TXUSRCLK2
TXUSRCLK
RXUSRCLK
TXUSRCLK2
TXUSRCLK
TX_CLK RXUSRCLK2
RXUSRCLK
TXUSRCLK2
TXUSRCLK
RXUSRCLK2
RXUSRCLK
TXUSRCLK2
RXUSRCLK2
RXUSRCLK
RXUSRCLK2
RX_CLK

10 Gigabit Ethernet
Interfaces

Media 4 Rx Ctrl
Interface
Sixteen Bit LVDS
Interface
Unit Interface

XSBI
• 10 Gigabit Sixteen Bit Interface
• Interface to the serial PHY serdes
• 16 bits LVDS in each direction
• 640 MHz SDR is a challenge...
• …but APD have a reference design in the pipe

Summary
Summary
• 10GbE is much more cost effective than SONET
– In the end economics always influence key decisions
– Native Ethernet at 10Xs the performance/cost will displace SONET
infrastructure over the long run especially when the traffic is IP
• Simple, scalable, manageable, reliable, interoperable, multi-
protocol support, high performance, application-enabler, cost
effective
• Ethernet leveraged into other technologies
– Ethernet over SONET, Wireless Ethernet - IEEE 802.11b/a,
Resilient Packet Ring (RPR)

Summary
• Service providers are delivering optical IP/Ethernet-based
solutions
– Ethernet provides just-in-time bandwidth, usage based billing,
application aware networks, w/ customer control
– Requirements exist today
• Storage application providers, streaming video, medical imaging
• Ethernet challenge for SPs
– Key missing element in Ethernet is OAM support
– Pricing 10/100/1000 Mbps Ethernet services vs. T1/T3/OC3
• Xilinx solutions provide layer 1 and 2 solutions for Gigabit
Ethernet and 10 Gigabit Ethernet (XGMII, XAUI, XSBI)

Extra
IEEE 802.3 PHY
Specifications
Max. Cable Length
Standard IEEE Data Rate Medium Topology
Half Duplex Full Duplex
1Base5 802.3e 1 Mb/s Two pairs of twisted telephone cable Star 250M N/A
10Base5 802.3 10Mb/s Single 50-ohm coaxial cable (thick Ethernet) Bus 500 M N/A
10Base2 802.3a 10Mb/s Single 50-ohm RG 58 coaxial cable (thin Ethernet) Bus 185M N/A
10Broad36 802.3b 10Mb/s Single 75-ohm CATV broadband cable Bus 1800M N/A
FOIRL 802.3d 10Mb/s Two Optical Fibers Star 1000M >1000
10Base-T 802.3i 10Mb/s Two pairs of 100-ohm Category 3 or better UTP cable Star 100M 100M
10Base-FL 802.3j 10Mb/s Two optical fibers Star 2000M >2000M
10Base-FB 802.3j 10Mb/s Two Optical Fibers Star 2000M N/A
10Base-FP 802.3j 10Mb/s Two Optical Fibers Star 1000M N/A
100Base-TX 802.3u 100Mb/s Two pairs of 100-ohm Category 5 UTP cable Star 100M 100M
100Base-FX 802.3u 100Mb/s Two Optical Fibers Star 412M 2000M
100Base-T4 802.3u 100Mb/s Four pairs of 100-ohm Category 3 or better UTP cable Star 100M N/A
100Base-T2 802.3y 100Mb/s Two pairs of 100-ohm Category 3 or better UTP cable Star 100M 100M
1000Base-LX 802.3z 1Gb/s Long wavelength laser (1300nm) over 62.5um multi-mode fiber Star 316M 550M
1000Base-LX 802.3z 1Gb/s Long wavelength laser (1300nm) over 50um multi-mode fiber Star 316M 550M
1000Base-LX 802.3z 1Gb/s Long wavelength laser (1300nm) over 10um Single mode fiber Star 316M 5000M
1000Base-SX 802.3z 1Gb/s Short wavelength laser (850nm) over 62.5um multi mode fiber Star 275M 275M
1000Base-SX 802.3z 1Gb/s Short wavelength laser (850nm) over 50um multi mode fiber Star 316M 550M
1000Base-CX 802.3z 1Gb/s Specialty shielded balanced copper jumper cable assemblies Star 25M 25M
1000Base-T 802.3ab 1Gb/s Four pairs of 100-ohm Category 5 or better cable Star 100M 100M

XAUI - Interface Extender
Between MAC & PCS


Gigabit Ethernet, 10 Gigabit Ethernet

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Gigabit Ethernet, 10 Gigabit Ethernet

Uploaded by

Copyright:

Available Formats

Gigabit Ethernet,

10/100 Ethernet PHY Digital

4B/5B Scrambler Transmitter

Transmit data & Collision Detected Transmit Jam

Start Of Frame Delimiter - Two consecutive 1 bits used to acquire

6 bytes Source Ethernet Address -The unique Ethernet address of

• 1st 2 bytes - 802.1Q Tag Type which is set to 0x8100

Faster, Cheaper, Simpler

Courtesy: Octavio Morales, World Wide Packets

Courtesy: Paul Botorff, Nortel networks

Courtesy: B. Tolley/3Com; Nan Chen/Nortel

Courtesy: B. Tolley/3Com; Nan Chen/Nortel

Courtesy: Arnie Thompson, Network Appliance

Courtesy: Arnie Thompson, Network Appliance

Courtesy: Paul Botorff, Nortel networks

TELNET, HTTP, SMTP

Software Layers 5. Session

IETF UDP/TCP, SNMP

2. Data Link Gig Ethernet MAC

OSI Layer 3 & 4 OSI Layer 2 OSI Layer 1

Frame Generation, Stats gathering, Flow Control, Transceiver

PPC OPB Bus

Connecting 100 Mbps workgroup to Gigabit Ethernet

Source: 10 Gigabit Ethernet Alliance

Courtesy: Paul Botorff, Nortel Networks

Courtesy: Bob Grow, Intel

Courtesy: Bob Grow, Intel

Courtesy: Jonathan Thatcher, World Wide Packets

PCS WIS PMA PMD

PCS PMA PMD

MAC - Media Access Control

• XGMII (10G Media Independent I/F)

10 GbE in the Metropolitan Area

10 Gigabit Ethernet Alliance

10 Gigabit Ethernet 100

10 Gigabit Ethernet 101

10 Gigabit Ethernet 103

10 Gigabit Ethernet 104

OC-48 / OC-192 POS/ATM Line Card Central Management Board

Control Plane Network 10GbE SONET ATM

Framer / NPU / Backplane SDRAM / SRAM / FLASH

10 Gigabit Ethernet 106

Central Management Board

TE T/E Backplane SONET ATM

SDRAM / SRAM / FLASH

xDSL HDLC Backplane Buffer I/O

10 Gigabit Ethernet 107

UART I/F Driver RS232

PCI Bus (32/33 or 64/66) or

10 Gigabit Ethernet 108

PCI Bus (32/33 or 64/66) or

10 Gigabit Ethernet 109

10 Gigabit Ethernet 110

P P Classifi- NPU Traffic BP BP

XAUI XGMII Security Parallel

Other Line Card,

10 Gigabit Ethernet 111

Other Line Card,

10 Gigabit Ethernet 112

Other Line Card,

10 Gigabit Ethernet 113

Embedded DSP Embedded Dual-Port RAM - XCITE Digitally-Controlled

CLKIN CLK0 4000XL Virtex