Professional Documents
Culture Documents
10 Gbe
10 Gbe
10 Gigabit Ethernet
Metro Ethernet
Agenda
Introduction
Ethernet Technology
10/100 Ethernet
1/10 Gigabit Ethernet
Gigabit Ethernet Technology & 10 Gigabit Ethernet Technology
Metro Ethernet
Consortium
Competing Technologies
Products / Equipment
Ethernet Block Diagrams
Xilinx Solutions for 1/10 GbE Products
Summary
10 Gigabit Ethernet
What is Ethernet?
Developed by Xerox in cooperation with
in 1976
Ethernet specification is the basis for the
IEEE 802.3 standard
Specifies the layer 1 (physical) and layer 2 (data-link) of the OSI reference
model
Ethernet uses a bus or star topology
10 Gigabit Ethernet
Ethernet History
1970s - ALOHA
1973 - Robert Metclafes
research at Xerox
2.94 Mbps, CSMA/CD
10 Gigabit Ethernet
Ethernet Dominance
Ethernet won the LAN wars: default layer 2 protocol
10 Gigabit Ethernet
1 Gb/s to 10 Gb/s
Make 10 GbE full duplex only, no collision domain
10 Gigabit Ethernet
Ethernet Economics
10 Gigabit Ethernet
Ethernet
Technology
Basics and Framing
Ethernet Basics
Defines layers 1 and 2 specifications
Physical layer (layer 1)
Provides the electrical, mechanical & procedural specs for the
transmission of bits through a communication link, medium or
channel
Data link layer (layer 2)
Ensures error control & synchronization between two entities
Includes Medium Access Control (MAC) & Logical Link Control
(LLC) sub-layers
10 Gigabit Ethernet
10 Gigabit Ethernet
10
Analog
Digital
Transmitter
Scrambler
4B/5B
M
I
I
Chip
10 Mb/s TX
Filter
125 MHz
5B/4B
Descrambler
25 MHz
TX
Freq Synch
Baseline
Restore
Clk
Recovery
Manchester
Decoder
10 Gigabit Ethernet
11
20 MHz
Adoptive
Equalization
10Clk
Recovery
Link Polarity
Auto
Negotiation
Board
10 Gigabit Ethernet
12
Error detection
Identification & parameter
exchange
Flow control
Physical layer services
Network management
Ethernet MAC
Provides all functions necessary to attach an Ethernet physical
layer to the host interface
Any physical layer chip that supports
MII standard can attach to the 10/100 MAC
GMII standard can attach to the 1 GbE MAC
XGMII standard can attach to the 10 GbE MAC
Processor
Host Interface
Statistic
Generator
FIFO
10/100 MAC
Ethernet PHY
10 Gigabit Ethernet
13
Ethernet MAC
Ethernet MAC
Tx Data Flow
Tx
Buffer
Transmit
MAC
Handler
Management
Register
Counter
Rx
Buffer
Rx Data Flow
Media
Independent
Interface
Receive
MAC
Handler
Address
Match
Serial
Interface
10 Gigabit Ethernet
14
PHY
MAC Types
There are 2 MAC protocols defined for Ethernet
Half-Duplex, Full-Duplex
15
CSMA/CD (Half-Duplex)
Carrier Sense Multiple Access w/ Collision Detect
The network is monitored for presence of a transmitting
station (carrier sense)
Transmission is deferred if an active carrier is detected
Station continues to monitor the network until carrier ceases
10 Gigabit Ethernet
16
CSMA/CD - Collisions
While the transmitting station is sending the frame, it monitors the
medium for a collision
If a collision is detected, the transmitting station stops sending the
frame data & sends a 32-bit jam sequence
Jam sequence is transmitted to ensure that the length of the collision is
sufficient to be noticed by other transmitting stations
After sending the jam sequence the transmitting station waits a random
period of time - called "back-off
17
CSMA/CD Flow
Station is ready
to send
New Attempt
Wait according to
Backoff Strategy
Sense Channel
Channel Busy
Channel Free
Collision Detected
No Collision Detected
Transmit Complete
10 Gigabit Ethernet
18
Transmit Jam
Signal
19
Non-persistent CSMA
When the channel is busy, the
station simply gives up and tries
at a later time
P-persistent CSMA
When the channel is busy, the
station will keep listening until
the channel becomes idle (like
1- persistent)
Then the station transmits the
frame with a probability of p
The station backs off with the
probability of q = 1 - p
20
10 Gigabit Ethernet
21
22
2 bits
6 bytes
6 bytes
2 bytes
Length or Type field - For IEEE 802.3 this is the number of bytes
of data. For Ethernet I&II this is the type of packet.
46 to
1500
bytes
4 bytes
10 Gigabit Ethernet
23
Length/Type
If the value of this field is less than or equal to 1500, then the Length/Type field indicates
the number of bytes in the subsequent MAC Client Data field
If the value of this field is greater than or equal to 1536, then the Length/Type field
indicates the nature of the MAC client protocol (protocol type)
10 Gigabit Ethernet
24
25
Frame Bursting
Allows a station to transmit a series of frames without relinquishing control of the
transmission medium
Burst mode applies to half-duplex mode only
Improves the performance of GbE when transmitting short frames
10 Gigabit Ethernet
26
VLAN Tagging
Frame format extensions to support Virtual Local Area Network (VLAN)
Tagging
The VLAN protocol permits insertion of an identifier, or "tag", into the Ethernet
frame format
VLAN Tagging provides various benefits
Easing network administration
Allowing formation of work groups
Enhancing network security
The 4-byte VLAN tag is inserted between the Source MAC Address field
and the Length/Type field
10 Gigabit Ethernet
27
VLAN Concept
Source: Intel
10 Gigabit Ethernet
28
VLAN Frame
802.1Q
Start
Tag Control Length
Destination Source
Tag
Preamble Frame
Information Type
Address Address
Type
Delimiter
7-bytes
1-bytes
6-bytes
6-bytes
2-bytes
2-bytes
2-bytes
MAC
Client
Data
Padding
FCS
0-n bytes
0-p bytes
4 bytes
Next 2 bytes
User Priority Field (3 bits) - indicates frame priority level
Canonical Format Indicator (1 bit) - indicates presence of a Routing
Information Field
VLAN Identifier (12 bits) - the VID uniquely identifies the VLAN to which the
Ethernet frame belongs
10 Gigabit Ethernet
29
10 Gigabit Ethernet
30
Drawback
25% overhead and less bandwidth efficiency
Line rate of 12.5 Gbps is needed to implement a transmission rate of 10
Gbps with 8b/10b
For a serial implementation this is a significant disadvantage because
currently there are a large number of devices that support 10 Gbps, but very
few that support 12.5 Gbps
10 Gigabit Ethernet
31
Scrambled Encoding
SONET/WAN applications
Allows a lower line rate for extended reach
Virtually has no overhead & better efficiency
32
10/100 Ethernet
10 Gigabit Ethernet
34
10 Gigabit Ethernet
35
1/10 GbE
Faster, Cheaper, Simpler
37
Preamble
Destination
Address
Source
Address
Length of
Data Field
0 - 1500
Protocol Header, Data, and Pad
38
Customer Challenges
Enterprise
Multiple locations in one metro area
Need to support high-bandwidth applications: imaging,
CAD/CAM, storage
Service Providers
Need high-speed connections between POPs
Need flexible bandwidth, just-in-time provisioning
Metro/MANs
Very low cost
Emergence of metro Ethernet services
10 Gigabit Ethernet
39
40
1 GbE Applications
Ideal backbone interconnect technology for use between
10/100 Base-T LAN switches
High-performance servers & high-end desktop PCs
Data centers
10 Gigabit Ethernet
41
10 GbE Applications
Local Area Networks
10 Gigabit Ethernet
42
10 GbE Applications
10 Gigabit Ethernet
43
10 Gigabit Ethernet
44
45
http://www.cisco.com/warp/public/cc/techno/lnty/etty/ggetty/tech/10gig_wp.htm
10 Gigabit Ethernet
46
http://www.cisco.com/warp/public/cc/techno/lnty/etty/ggetty/tech/10gig_wp.htm
10 Gigabit Ethernet
47
DataCenter Network
Gigabit and 10-Gigabit Ethernet
10 Gigabit Ethernet
48
ISP/NSP Network
Gigabit and 10-Gigabit Ethernet
10 Gigabit Ethernet
49
10-GbE in SANs
10 Gigabit Ethernet
50
Ethernet-based SANs
10 Gigabit Ethernet
51
52
Capacity Convergence
10 Gigabit Ethernet
53
Gigabit Ethernet
Technology
10 Gigabit Ethernet
55
Software Layers
7. Application
Example Protocols
6. Presentation
5. Session
IETF
Hardware Layers
IEEE802
10 Gigabit Ethernet
56
4. Transport
UDP/TCP, SNMP
Error Control, Endto End transmission,
retransmission.
3. Network
IP
ARP, routing
2. LLC
Multiplexing/Demultiplexing
2. Data Link
1. Physical
Building a 1000Base-X
Ethernet System
UDP,IP
TCP/IP
v4
PPC
User
Logic
Bus Management
FIFO,Flow Control
OPB
Bus
Management
FIFO
Generic
MAC
GMII
Line Coding,
Auto-Neg,
Sync,
8B/10B
PCS
TBI
10
Serdes,
Clk-rec,
cml/pecl
P
M
A
Gen
16
OSI Layer 3 & 4
OSI Layer 2
OSI Layer 1
( Network/Transport Layer)
(Physical Layer)
Software
Hardware
Hardware
10 Gigabit Ethernet
57
Transceiver
P
M
D
MDI
Fiber/Copper
LX/SX/CX
Building a 1000Base-X
Ethernet system
UDP,IP
TCP/IP
PPC
User
Logic
Bus Management
FIFO,Flow Control
OPB
Bus
Management
FIFO
Gen
16
10 Gigabit Ethernet
58
Generic
MAC/PCS/PMA
Transceiver
P
M
D
MDI
Fiber/Copper
LX/SX/CX
Applications/Target Market
Xilinx
Xilinx
Xilinx
Xilinx
Xilinx
10 Gigabit Ethernet
59
Applications/Target Market
Xilinx
10 Gigabit Ethernet
60
I0 Gigabit Ethernet
Technology
What is 10 GbE?
Next step in Ethernet data rate
10 Gigabit Ethernet
62
10 Gigabit Ethernet
63
10 Gigabit Ethernet
64
65
10 GbE Components
PHY standards
A LAN PHY operating at a data rate of 10.3125 Gbps
A WAN PHY operating at a data rate of 9.95328 Gbps
Compatible with OC-192c/SDH VC-4-64c payload rates
10 Gigabit Ethernet
66
10 Gigabit Ethernet
67
Asynchronous service
preserved
Enables low cost access to
SONET optical infrastructure
10 Gigabit Ethernet
68
Accomplished by addition of
SONET/SDH sublayer
Implemented as an optional sub-layer
that maps encoded data stream of
the LAN PHY into the payload
envelope of a SONET frame
69
10 Gigabit Ethernet
70
71
10 Gigabit Ethernet
72
PMD Options
10 Gigabit Ethernet
73
10 Gigabit Ethernet
74
Understanding the
Terminology
10 Gigabit Ethernet
75
Serial WAN
WWDM LAN
MAC
MAC
MAC
XGXS
PCS
64b/66b
PMA
Serial
WIS
XGXS
XGXS
PCS
64b/66b
XGXS
PCS
8b/10b
PMA
WWDM
PMD
WWDM
XGXS
10 Gigabit Ethernet
76
PMD
Serial
PMA
Serial
PMD
Serial
77
10 Gigabit Ethernet
78
XAUI
XGXS
XSBI
XGXS
PCS
PMA
PMD
79
Fiber
Optics
80
10 GbE MAC
MAC - Unchanged from GbE
MAC data rate of 10.000 Gb/s
Serial PHY: 10.3 GBaud (64b/66b code)
Parallel PHY: 4 X 3.125 GBaud (8b/10b code)
Performs rate control for WAN PHY
Not used for LAN PHY types
Full duplex standard only
10 Gigabit Ethernet
81
MAC
MAC pacing matches the MAC/PLS data rate to the WAN
PHY data rate
The MAC transmits a longer Inter-Packet Gap between packets
when the WAN PHY is used
The number of the bytes added to each IPG is proportional to
the length of the previous packet
These extra bytes of IPG are removed during 64B/66B encoding
to effectively reduce the data rate
10 Gigabit Ethernet
82
10 GbE Benefits
Brings Ethernet high
volume/low cost cost model to
10 Gbps networks
Scales network backbones in
enterprise core and in SP
Internet data centers
Leverages 250 million
Ethernet ports installed base
Supports all IP services
Data, packetized voice & video
Provides advanced
management capabilities
10 Gigabit Ethernet
83
Metro Ethernet
10 GbE in the Metropolitan Area
Networks
MAN Dynamics
Extremely heavy traffic between campuses
Large campuses have several thousand users per site
Inter-LAN traffic often reaches multi-gigabit range today
This traffic will continue to grow
Interconnecting campus LANs on a single, shared, highcapacity MAN ring can seamlessly link all sites on a VPN
under a common Ethernet protocol
No translations required between sites
Very high traffic concentration between ring nodes
10 Gigabit Ethernet
85
10 Gigabit Ethernet
86
10 Gigabit Ethernet
87
Advantages of 10 GbE
in the Metro
LAN and WAN-compatibility
Provides opportunity to implement dependable IP VPN services
10 Gigabit Ethernet
88
10 Gigabit Ethernet
89
10 Gigabit Ethernet
90
Consortium
10 Gigabit Ethernet Alliance
Metro Ethernet Forum
10GEA - 10 Gigabit
Ethernet Alliance
100+ Members - Xilinx is a Participating Member
Mission
Promote industry awareness, acceptance, and advancement of
technology and products based on the emerging 10 Gigabit
Ethernet standard
Accelerate industry adoption by driving technical consensus and
providing technical contributions to the IEEE 802.3ae Task
Force
Provide resources to establish and demonstrate multi-vendor
interoperability of 10 Gigabit Ethernet products
10 Gigabit Ethernet
92
10 Gigabit Ethernet
93
94
Competition
10 Gigabit Ethernet
96
Equipment
Interconnection Devices
Repeater: PHY device that restores data and collision
signals: a digital amplifier
Hub: Multi-port repeater + fault detection
Bridge: Data link layer device connecting two or more
collision domains. MAC multicasts are propagated
throughout extended LAN
Router: Network layer device. IP, IPX, AppleTalk. Does
not propagate MAC multicasts
Switch: Multi-port bridge with parallel paths
10 Gigabit Ethernet
98
What is a Repeater?
Repeaters are low-level devices that amplify or regenerate
weak signals
Repeaters are needed to provide current to drive long
cables
Repeaters are used to join network segments together to
increase the total length of the network
Act at the physical layer and allow all traffic to cross LAN
segments
10 Gigabit Ethernet
99
What is a Hub?
A place of convergence where data arrives from one or
more directions and is forwarded out in one or more other
directions
Hub is a repeater with fault detection functionality
Connects high-performance stations/devices to Ethernet
LAN and provides high-performance inter-LAN
connectivity using switching technology
A hub usually includes a switch of some kind
10 Gigabit Ethernet
100
What is a Bridge?
Connect a LAN to another LAN that uses the same
protocol (e.g., Ethernet or Token Ring)
Works at the data-link level of a network, copying a data
frame from one network to the next network along the
communications path
Bridges can make minor changes to the frame before
forwarding it (such as adding and deleting some fields
from the frame header)
10 Gigabit Ethernet
101
What is a Switch?
Network device that selects a path or circuit for sending a
unit of data to its next destination
Switch is a simpler & is faster than a router
Simultaneous switching of packets between its ports increases the
aggregate LAN bandwidth dramatically
102
What is a Router?
Determines next network point to which a packet should
be forwarded on the way to its final destination
Routers use the network layer information within each packet to
route it from one destination/LAN to another
Must recognize all the different layer 3 protocols that are used on
the networks it is linking together
Routers communicate with one another to determine the best
route through the complex connections of many LANs to
increase speed and cut down on network traffic
10 Gigabit Ethernet
103
What is a NIC?
NIC - Network Interface Card
NIC is an expansion card used to connect a PC, server, or
workstation to a LAN
NIC provides an interface between the network and the
PCs bus
Most NICs are designed for a particular type of network,
protocol, and media
The NIC segments outgoing messages into packet
formats specified by the LAN protocol for transmission
10 Gigabit Ethernet
104
Ethernet Block
Diagrams
Core Router
1/10GbE Line Card
Control Plane
Processor
Optics
PHY
10GbE
MAC
Network
Co-Processor (s)
NPU /
ASIC
Switch
Fabric
Backplane
I/F
Backplane
I/F
Memory
xN
Central Management Board
Optics
PHY
Memory
Framer /
Mapper
Network
Co-Processor (s)
NPU /
ASIC
10GbE
Mgr.
Backplane
I/F
10 Gigabit Ethernet
106
ATM
Mgr.
CPU
xN
SONET
Mgr.
System Bus
Buffer
Mgr.
I/O
Mgr.
Edge Router
OC-12 / OC-3 POS or ATM
Optics
PHY
Backplane
I/F
Switch
Fabric
Backplane
I/F
10 / 100 / 1000 Ethernet Linecard
PHY
10/100/Gig
MAC
Memory
Backplane
I/F
T/E Linecard
TE
T/E
Framer
TX / RX
Backplane
I/F
I/O Mgr.
SONET
Mgr.
ATM
Mgr.
HDLC
Controller
Backplane
I/F
CPU
System Bus
10 Gigabit Ethernet
107
Buffer
Mgr.
I/O
Mgr.
Enterprise Router
RJ-45
10/100
PHY
Memory
I/F
10/100
MAC
DMA
Controller
CPU
RJ-45
LAN
Side
802.11a/b
Radio
10/100
PHY
UART
10/100
MAC
802.11a/b
Baseband
Controller
T/E Framer
T/E
TX/RX
HDLC
xDSL
TX / RX
ATM SAR
ATM OC-3
PHY
UTOPIA
PCI Bus (32/33 or 64/66) or
Proprietary Bus
10 Gigabit Ethernet
108
SDRAM
Flash /
SRAM
I/F Driver
WAN
Side
(to CO)
RS232
SOHO Router
802.11a/b
Baseband
Controller
802.11a/b
Radio
RJ-45
LAN
Side
10/100
PHY
10/100
MAC
MII
Memory
I/F
CPU
DMA
Controller
UART
AFE
UTP
Flash /
SRAM
I/F Driver
RS232
Home
PNA
USB
Transceiver
USB
Device
Controller
T/E Framer
T/E
TX/RX
HDLC
xDSL
TX / RX
QAM Decoder
& FEC
RS-232
10 Gigabit Ethernet
SDRAM
109
Cable
T1/E1
DSL
Clock Generator
& DLLs
WAN
Side
(to CO)
GbE Switch
Magnetics
GbE
PHY
GbE
MAC
Magnetics
GbE
PHY
GbE
MAC
Packet
Memory
Address
Cache
Switch
Fabric
ASSP
Network
Management
CPU
Memory
I/F
Magnetics
10 Gigabit Ethernet
110
GbE
PHY
GbE
MAC
Memory
System Bus
Memory
Memory
Memory
Controller
Controller
10 GbE
Integ.
Optics
P
M
A
P
C
S
10GbE
MAC
XAUI XGMII
Classification
Engine
NPU
or
ASIC
Security
Proc.
Traffic
Queue
Mgr.
Scheduler
BP
SER /
DES
BP
Optics
Parallel
Parallel
I/F
I/F
System
Clock
Other Line Card,
Switch Fabric, or
Network Mgmt. Card
10 Gigabit Ethernet
111
System Bus
Memory
Memory
Memory
Controller
Controller
GbE
MAC
PHY
Magnetics
GMII
NPU or
ASIC
Backplane
SERDES
SERDES
Serial I/F
Driver
Receiver
Parallel
Parallel
I/F
I/F
System
Clock
Other Line Card,
Switch Fabric, or
Network Mgmt. Card
10 Gigabit Ethernet
112
System Bus
Memory
Memory
Memory
Controller
Controller
10/100
MAC
PHY
Magnetics
MII
NPU or
ASIC
Backplane
SERDES
SERDES
Serial I/F
Driver
Receiver
Parallel
Parallel
I/F
I/F
System
Clock
Other Line Card,
Switch Fabric, or
Network Mgmt. Card
10 Gigabit Ethernet
113
115
BRAM
18 Bit
36 Bit
Impedance
Controller
18 Bit
Embedded DSP
functionality - up to
500 Billion MAC/s
CLK0
CLK90
CLK180
CLK2X
RST
CLK270
CLK2X180
DSSEN
CLKDV
PSINCDEC
CLKFX180
PSEN
CLKFX
STATUS[7:0]
PSCLK
LOCKED
PSDONE
4000
CLKIN
CLKFB
clock signal
control signal
10 Gigabit Ethernet
Virtex
4000XL
3500
Delay (ps)
3000
2500
2000
Virtex-II
1500
1000
500
0
0
Digital Clock
Management (DCM) Precise Clock
Generation
116
XCITE Digitally-Controlled
Impedance (DCI) for
simpler PCB layout
200
400
600
800
1000
LUTs Reached
Active Interconnect
Technology with a 300 MHz
System Clock
SelectIO
25 IO types including
840 Mbps LVDS
10 Gigabit Ethernet
117
Encode
FIFO
Serializer
TX
+
TX-
Transmit
Buffer
Transmitter
31.1 MHz
156.3 MHz
REFCLK
Comma Detect
and Word
Alignment
RXDATA
20X Multiplier
Receiver
RX Clock Generator
Elastic
8B / 10B
Buffer
Decode
Deserializer
118
Loopback
TX Clock Generator
Receive
Buffer
RX+
RX-
Encode
FIFO
Serializer
TX+
TX-
Transmit
Buffer
42 MHz
156.3 MHz
RXDATA
Transmitter
20X Multiplier
REFCLK
Receiver
Comma Detect
and Word
Alignment
RX Clock Generator
Elastic
8B / 10B
Buffer
Decode
10 Gigabit Ethernet
Loopback
TX Clock Generator
119
Deserializer
Receive
Buffer
RX+
RX-
Supports AC or DC coupling
Local loop-back capability
Encode
FIFO
Serializer
TX+
TX-
Transmit
Buffer
42 MHz
156.3 MHz
RXDATA
Transmitter
20X Multiplier
REFCLK
Receiver
Comma Detect
and Word
Alignment
Loopback
TX Clock Generator
RX Clock Generator
Elastic
8B / 10B
Buffer
Decode
Deserializer
Receive
Buffer
RX+
RX-
120
SelectI/O-Ultra
technology for parallel
interfaces
Input
Reg DDR mux
OCK1
Reg
ICK1
Reg
OCK2
3-State
Reg
ICK2
OCK1
Reg
OCK2
32b @
78 MHz
Transceivers
Up to Sixteen 3.125 Gbps transceivers
10 Gigabit Ethernet
121
Output
DDR
FF
Gb Serial
Gb Serial
Transceiver Transceiver
32b @
78 MHz
PAD
25 I/O Standards
XCITE Technology
840 Mbps LVDS
Dedicated DDR
Registers
P
N
IOB
10 Gigabit Ethernet
122
1-Gb Ethernet
MAC Core
~471min
~1777max
MGT, SRL16, DCM
Dist & Block RAM
4.1I (E.35)
I/O connections
200
Special Features
10 Gigabit Ethernet
124
FPGA LOGIC
MAC
GT_Ethernet_1 (MGT)
PCS
Physical Coding Sublayer
Tx
TX
PAUSE
RX
Frame
Tx
Gen
k-codes
&
Disp.
Client
Flow
I/F
Control
Txdata[7:0]
FIFO
Txcharisk
8B / 10B
Txchardispmode
Encode
Txchardispval
Frame
checks
HOST
Config
10 Gigabit Ethernet
Stats MDIO
125
Rx
Sync
MDIO
Transmit
Buffer
Transmitter
20X Multiplier
Neg.
Rx
Serializer
TX+
TX-
TX Clock Generator
Txbuferr
Auto
Loop-back
gtx_clk
(125MHz)
Receiver
RX Clock Recovery
Rxdata[7:0]
Rxcharisk
Rxnotintable
Rxdisperr
Rxbufstatus(1)
Rxchariscomma
8B / 10B
Deserializer
Decode
Comma Det.
Receive
Buffer
RX+
RX-
Elastic
Buffer
SIGNAL_DETECT
10 Gigabit Ethernet
126
127
GMAC CORE
GMII
TX Data
TX Valid
Transmit
Engine
8
TX
I/O
Pause Req
Pause Value 16
RX Data
RX Valid
Flow
Control
Deference
COL
CRS
and Backoff
Receive
Engine
8
RX
I/O
Management I/F
Statistics
and Configuration
128
TXD[7:0]
TX_ER
TX_EN
TX_CLK
RXD[7:0]
RX_DV
RX_ER
RX_CLK
Gigabit Ethernet
Applications
Communications Equipment
GbE Network Interface Cards (NICs)
PCI-, PCI-X- and Cardbus-based
Edge switches and terabit routers packet-based line cards
Storage Equipment
iSCSI line cards
10 Gigabit Ethernet
129
10-Gb Ethernet
MAC Core
XGMAC CORE
XGMII
64
Transmit
Engine
Pause Req
Pause Value 16
RX Data
RX Ctrl
64
Management I/F
64
TX
I/O
TXD[31:0]
TXC[3:0]
TX_CLK
Flow
Control
Receive
Engine
Statistics
& Configuration
64
RX
I/O
RXD[31:0]
RXC[3:0]
RX_CLK
131
10GE MAC
Technical Details
10 GIGABIT ETHERNET MAC FACTS
Core Specifics
Supported Family
Virtex-II (-5)
XC2V1000-XC2V8000
Performance
10 Gigabit Ethernet
Special Features
Runs on Software Ver.
I/O connections
132
10 Gigabit Ethernet
133
Additional 10-GE
MAC Features
Optional support for VLAN frames
Virtual LAN allows network administrators to reduces the time it
takes to implement moves, adds and changes
134
MAC
PCS
WIS
P
M
A
XGMII
XGMII
P
M
D
Glossary
MAC : Medium Access Control
PCS : Physical Coding Sublayer
WIS : WAN Interface Sublayer
XGMII : 10 Gig Medium Independent Interface
PMA : Physical Media Attachment Sublayer
PMD : Physical Medium Dependent Sublayer
10 Gigabit Ethernet
135
TX Data
TX Ctrl
XGMII
64
Transmit
Engine
8
16
RX Data
RX Ctrl
RS/XGMII
RS/XGMII
TX
I/O
Flow
Control
64
Receive
Engine
RX
I/O
XGMII
Management
MDIO I/F
10 Gigabit Ethernet
136
MDIO I/F
TX Data
TX Ctrl
PCS/PMA
PCS/PMA
XAUI
64
Transmit
Engine
Pause Req
Pause Value 16
RX Data
RX Ctrl
XAUI Floorplan
2VP7
Rocket I/O
Transceivers
Flow
Control
64
Receive
Engine
XGMII
Management
MDIO I/F
10 Gigabit Ethernet
137
In FPGA
Can interface
to
10GBASE-LX4
10GBASE-X
10 Gigabit Ethernet
Serial
LAN PHY
(64B/66B)
138
10GBASE-LR
10GBASE-SR 10GBASE-ER
10GBASE-R
Serial
WAN PHY
(64B/66B + WIS)
Serial Serial Serial
PMD
PMD
PMD
850 nm 1310 nm 1550 nm
10GBASE-SW 10GBASE-EW
10GBASE-LW
10GBASE-W
MGTs
MGTs
MGTs
MGTs
10 Gigabit Ethernet
139
MGTs
GMAC
GMAC/PCS/PMA
Virtex-II Pro
Virtex-II Pro
MGTs
XGMAC
XGMAC/XAUI
XAUIlike
Buffer
XGMAC
XAUI
Memory
XAUI
10 Gigabit Ethernet
140
64B/66B
PCS
SERDES
E/O
Fiber
10 Gigabit Ethernet
141
10GBASE-X (WWDM)
10GBASE-X PCS is functionally identical to XAUI
Some management registers move around in the address space
Signal detect pins used
from optics
E/O
XenPak
E/O
Optical
E/O
XGMAC
XAUI
E/O
10 Gigabit Ethernet
142
MUX
Integrated
Optics
OC-192 POS
ASSP
10GE
PHY
LAN/WAN
FPGA
10GE MAC
Implementation review
10 Gigabit Ethernet
143
ASSP
NPU
CSIX
FPGA
CSIX Bridge
RapidIO
ASSP
Terabit
Switch
Fabric
Backplane
ASSP
POS-PHY L4
XAUI
XenPak
10 Gigabit Ethernet
Interfaces
Name
Number of
Useful
Comments
Clock and
Transmission
Control Lines Distance
XGMII
10Gb/s
Media
Independent
Interface
XSBI
10Gbps
Serial Bus
Interface
XAUI
10Gbps
Attachment
Unit Interface
32 TX
32 RX
312.5 Mbps
4 Tx Ctrl
1 Tx Clk
4 Rx Ctrl
1 Rx Clk
74 Signal
Lines
16 TX
16 RX
LVDS
622 Mbps
1 Tx Clk
1 Rx Clk
<20 cm (8) +
1 connector
64 Signal
lines
None
10 Gigabit Ethernet
144
4 TX
3.125Gbps
4 RX
Differential
10 Gigabit Ethernet
Interfaces
Name
Comments
Useful
Number of
Transmission
Clock and
Control Lines Distance
XGMII
10Gb/s
Media
Independent
Interface
XSBI
10Gbps
Serial Bus
Interface
XAUI
10Gbps
Attachment
Unit Interface
32 TX
32 RX
312.5 Mbps
4 Tx Ctrl
1 Tx Clk
4 Rx Ctrl
1 Rx Clk
74 Signal
Lines
16 TX
16 RX
LVDS
622 Mbps
1 Tx Clk
1 Rx Clk
<20 cm (8) +
1 connector
64 Signal
lines
None
10 Gigabit Ethernet
145
4 TX
3.125Gbps
4 RX
Differential
XGMII
Logical Interface between
MAC and PHY
Convenient partition for
specification
Xilinx XGMAC
XGMII
PCS
PMA
PMD
Fiber
10 Gigabit Ethernet
146
XGMII
4 lane parallel interface
32 bits Data and 4 Control bits TX
32 bits Data and 4 Control bits RX
10 Gigabit Ethernet
147
10 Gigabit Ethernet
148
TXD
0x00 through 0xFF
0x07
0x9C
0xFB
0xFD
0xFE
Description
Normal data transmission
Idle
Sequence (only valid in lane 0)
Start (only valid in lane 0)
Terminate
Error
TXC<3:0>
0xF
0x1 0x0
TXD<7:0>
TXD<15:8>
Dp
0xC
0xF
fram e dat a
Dp
frame da ta
TXD<23:16> I
Dp
frame da ta
TXD<31:24> I
Dp SF D
frame da ta
I: Idle c ontrol c hara cte r, S: St art control c harac ter, Dp: pream ble Da ta oc tet, T: Te rminat e c ontrol c harac te r
10 Gigabit Ethernet
149
10 Gigabit Ethernet
150
Device B
TX Idle Sequence
Idle Sequence RX
MAC/XAUI
MAC/XAUI
RX Idle Sequence
Idle Sequence TX
10 Gigabit Ethernet
151
Device A
Device B
TX Idle Sequence
No signal RX
MAC/XAUI
MAC/XAUI
RX Remote Fault
Remote Fault TX
10 Gigabit Ethernet
152
XGMAC Core
DCM
CLKIN
FB
BUFG
CLK0
FDDRRSE
CLK90
XGMII_TX_CLK
BUFG
TX_CLK
BUFG
RX_CLK
DCM
CLK0
IBUFG
CLKIN
FB
10 Gigabit Ethernet
153
XGMII_RX_CLK
XGMII Weaknesses
Single ended I/O standard
HSTL Class I limited to 3 track length (unterminated)
Lots of pins
78 pins for entire interface
10 Gigabit Ethernet
154
10 Gigabit Ethernet
Interfaces
Name
Number of
Lines
Data Rate
Per Line
Number of
Useful
Comments
Clock and
Transmission
Control Lines Distance
XGMII
10Gb/s
Media
Independent
Interface
XSBI
10Gbps
Serial Bus
Interface
XAUI
10Gbps
Attachment
Unit
Interface
32 TX
32 RX
312.5 Mbps
4 Tx Ctrl
1 Tx Clk
4 Rx Ctrl
1 Rx Clk
74 Signal
Lines
16 TX
16 RX
LVDS
622 Mbps
1 Tx Clk
1 Rx Clk
<20 cm (8) +
1 connector
64 Signal
lines
None
>50 cm (20)
+2
connectors
16 signal
lines
10 Gigabit Ethernet
155
4 TX
3.125Gbps
4 RX
Differential
XAUI
10 Gigabit eXtended Attachment Unit Interface
Used to extend the 10G Media Independent Interface
4 lane serial, embedded clock
battle-tested 8B10B encoding on each lane
Allows interconnects of over 500mm (20) on standard FR-4
material
Even longer on specialized materials e.g. GETek, RT-Duroid
Protocol and PHY Independent
Hot Swappable
10 Gigabit Ethernet
156
TXC
TXD
Media Access
Control RXC
RXD
36
36
PCS/PMA
Physical Coding
Sublayer/ Physical
Media Attachment
Parallel
XGMII
Issue: 2 max
157
Dependent
Interface
Serial
XAUI
Up to 20
10 Gigabit Ethernet
PMD
Four Serial Channels
Physical Medium
Bonded Together
XAUI
TX
PMD
PCS/PMA
RX
Benefits
Lowers pin count and device count
Solves jitter and skew issues
10 Gigabit Ethernet
158
10 Gigabit Ethernet
159
Data
TXD[31:0]
24 23 16 15
8 7 6 5 4 3 2 1 0
Control
TXC[3:0]
3 2 1 0
Byte
Byte Byte Byte Lane 0= TXD[7:0]+TXC[0]
Lane 3 Lane 2 Lane 1
7 6 5 4 3 2 1 0
8+control
HGFEDCBA,K
8B10B Encoder
10
abcdefghij
0 1 2 3 4 5 6 7 8 9
10 Gigabit Ethernet
160
10 Gigabit Ethernet
161
MGTs Advantage
10 Gigabit Ethernet
162
Transceiver Module
8B/10B
TXDATA
Encode
FIFO
Serializer
Transmit
Transmit
Buffer
Buffer
TX Clock
Clock Generator
TX
Generator
Transmitter
20X Multiplier
REFCLK
Receiver
Comma Detect
and Word
Alignment
RXDATA
TX+
TX-
Loopback
RX
Generator
RXClock
Clock
Generator
Elastic
8B / 10B
Buffer
Decode
Deserializer
Deserializer
Receive
Buffer
RX+
RX-
Idle Generation
Idle XGMII is translated to ||A||, ||K|| and ||R||
Simple rules for mapping into these codes
First idle after data alternates between ||A|| and ||K||, unless an
||A|| is not due, then ||K|| is sent
||R|| is the second idle after data
||A|| codes are separated by no less than 16 columns and no
more than 31 columns
If not sending an ||A||, either ||R|| or ||K|| is sent, randomly
distributed
10 Gigabit Ethernet
163
S Dp D
D --- D
T/RXD<15: 8>
Dp Dp D
D --- D
T/ RX D<23 :1 6>
Dp Dp D
D --- D
T/ RX D<31 :2 4>
Dp Ds D
D --- D
PCS
LANE 0
S Dp D
D --- D
LANE 1
R Dp Dp D
D --- D
LANE 2
R Dp Dp D
D --- D
LANE 3
R Dp Ds D
D --- D
Le gend :
Dp re pres ents a da ta ch arac ter co ntaining the prea mble pattern
Ds re pres ents a da ta ch arac ter co ntaining the SFD pa ttern
10 Gigabit Ethernet
164
Channel Synchronization
Each line has to find the byte boundary
Uses Comma characters in 10B stream (0011111xxx or
1100000xxx)
This doesnt appear anywhere else
10 Gigabit Ethernet
165
Channel Bonding
Aligns the lanes to each other
Can deal with up to 84UI of lane-to-lane skew (26.9 ns)
Lane 0
Lane 1
Lane 2
Lane 3
K
K
R
R
K
R
K
K
R
K
K
K
A
K
R
K
K
R
A
R
R
A
K
A
R
K
R
K
K
R
R
R
Lane 0
Lane 1
Lane 2
Lane 3
10 Gigabit Ethernet
166
K
R
K
R
K
K
K
K
R
K
K
K
K
K
K
K
K
K
R
K
R
R
R
R
R
R
K
R
A
A
A
A
R
K
R
K
K
K
K
K
R
R
R
R
R
R
R
R
K
K
K
K
K
K
K
K
R
R
R
R
K
K
K
K
R
R
R
R
R
R
R
R
10 Gigabit Ethernet
167
Device B
TX Idle Sequence+RF
Idle Sequence+RF RX
MAC/XAUI
MAC/XAUI
RX No Signal
10 Gigabit Ethernet
168
No Signal TX
Device B
TX Idle Sequence+RF
Idle Sequence+RF RX
MAC/XAUI
MAC/XAUI
RX No Signal
Idle Sequence TX
10 Gigabit Ethernet
169
Device B
TX Idle Sequence
Idle Sequence+RF RX
MAC/XAUI
MAC/XAUI
RX Idle Sequence
Idle Sequence TX
10 Gigabit Ethernet
170
GT_XAUI_2
DCM
CLKIN CLK0
FB
TX_CLK
RX_CLK
10 Gigabit Ethernet
171
BUFG
GT_XAUI_2
REFCLK
GT_XAUI_2
REFCLK
GT_XAUI_2
REFCLK
REFCLK
TXUSRCLK
TXUSRCLK2
TXUSRCLK
RXUSRCLK
TXUSRCLK2
TXUSRCLK
RXUSRCLK2
RXUSRCLK
TXUSRCLK2
TXUSRCLK
RXUSRCLK2
RXUSRCLK
TXUSRCLK2
RXUSRCLK2
RXUSRCLK
RXUSRCLK2
10 Gigabit Ethernet
Interfaces
Name
Number of
Useful
Comments
Clock and
Transmission
Control Lines Distance
XGMII
10Gb/s
Media
Independent
Interface
XSBI
10Gbps
Sixteen Bit
Interface
XAUI
10Gbps
Attachment
Unit Interface
32 TX
32 RX
312.5 Mbps
4 Tx Ctrl
1 Tx Clk
4 Rx Ctrl
1 Rx Clk
74 Signal
Lines
16 TX
16 RX
LVDS
622 Mbps
1 Tx Clk
1 Rx Clk
<20 cm (8) +
1 connector
64 Signal
lines
None
>50 cm (20)
16 signal
+ 2 connectors lines
10 Gigabit Ethernet
172
4 TX
3.125Gbps
4 RX
Differential
XSBI
10 Gigabit Ethernet
173
Summary
Summary
10GbE is much more cost effective than SONET
In the end economics always influence key decisions
Native Ethernet at 10Xs the performance/cost will displace SONET
infrastructure over the long run especially when the traffic is IP
Simple, scalable, manageable, reliable, interoperable, multiprotocol support, high performance, application-enabler, cost
effective
Ethernet leveraged into other technologies
Ethernet over SONET, Wireless Ethernet - IEEE 802.11b/a,
Resilient Packet Ring (RPR)
10 Gigabit Ethernet
175
Summary
Service providers are delivering optical IP/Ethernet-based
solutions
Ethernet provides just-in-time bandwidth, usage based billing,
application aware networks, w/ customer control
Requirements exist today
Storage application providers, streaming video, medical imaging
176
Extra
10 Gigabit Ethernet
178
1 Mb/s
10Mb/s
10Mb/s
10Mb/s
10Mb/s
10Mb/s
10Mb/s
10Mb/s
10Mb/s
100Mb/s
100Mb/s
100Mb/s
100Mb/s
1Gb/s
1Gb/s
1Gb/s
1Gb/s
1Gb/s
1Gb/s
1Gb/s
Medium
Topology
Star
Bus
Bus
Bus
Star
Star
Star
Star
Star
Star
Star
Star
Star
Star
Star
Star
Star
Star
Star
Star
10 Gigabit Ethernet
179
10 Gigabit Ethernet
180
10 Gigabit Ethernet
181