Introduction to PCI Express

Comparison with different version of PCIE

PCIE Protocol Details
Pci DMA Controller
References

An introduction to PCI Express

Proma Goswami

August 19, 2014

Proma Goswami An introduction to PCI Express

 Introduction to PCI Express
Comparison with different version of PCIE
PCIE Protocol Details
Pci DMA Controller
References

Table of contents

1 Introduction to PCI Express

Charecteristics of PCIE
2 Comparison with different version of PCIE
Comparison
Operation
3 PCIE Protocol Details
Protocol
PCIE Link
4 Pci DMA Controller
5 References

Proma Goswami An introduction to PCI Express

 Introduction to PCI Express
Comparison with different version of PCIE
PCIE Protocol Details Charecteristics of PCIE
Pci DMA Controller
References

PCI Express

PCI Express (Peripheral Component Interconnect Express),

officially abbreviated as PCIe, is a high-speed serial computer
expansion bus standard designed to replace the older PCI, PCI-X,
and AGP bus standards.

Proma Goswami An introduction to PCI Express

 Introduction to PCI Express
Comparison with different version of PCIE
PCIE Protocol Details Charecteristics of PCIE
Pci DMA Controller
References

PCI Express is a point to point connection ,i.e, it connects

only two devices no other device can share the connection.
PCIE is based on high speed serial communication.
PCIE is based on individual lanes which can be grouped to
create higher bandwidth connections.

Proma Goswami An introduction to PCI Express

 Introduction to PCI Express
Comparison with different version of PCIE
PCIE Protocol Details Comparison
Pci DMA Controller
References

Comparison with different version of PCIE

Slot Clock(GHz) No of Bits Data/ Cycle Bandwidth
PCIe 1.0 x1 2.5 1 1 250 MB/s
PCIe 1.0 x4 2.5 4 1 1000 MB/s
PCIe 1.0 x8 2.5 8 1 2000 MB/s
PCIe 1.0 x16 2.5 16 1 4000 MB/s
PCIe 2.0 x1 2.5 1 1 500 MB/s
PCIe 2.0 x4 2.5 4 1 2000 MB/s
PCIe 2.0 x8 2.5 8 1 4000 MB/s
PCIe 2.0 x16 2.5 16 1 8000 MB/s
PCIe 3.0 x1 2.5 1 1 1000 MB/s
PCIe 3.0 x4 2.5 4 1 4000 MB/s
PCIe 3.0 x8 2.5 8 1 8000 MB/s
PCIe 3.0 x16 2.5 16 1 16000 MB/s

Proma Goswami An introduction to PCI Express

 Introduction to PCI Express
Comparison with different version of PCIE
PCIE Protocol Details Comparison
Pci DMA Controller
References

Operation Modes

A connection between two a PCIe device and a PCIe switch is

called a link. Each link is composed of one or more lanes, and each
lane is capable of transmitting one byte at a time in both directions
at once. This full-duplex communication is possible because each
lane is itself composed of one pair of signals: send and receive.So
this lane is single bit ,full duplex,high speed serial communication.

Proma Goswami An introduction to PCI Express

 Introduction to PCI Express
Comparison with different version of PCIE
PCIE Protocol Details Comparison
Pci DMA Controller
References

In order to transmit PCIe packets, which are composed of multiple

bytes, a one-lane link must break down each packet into a series of
bytes, and then transmit the bytes in rapid succession. The device
on the receiving end must collect all of the bytes and then
reassemble them into a complete packet. This disassembly and
reassembly happens must happen rapidly enough to where it’s
transparent to the next layer up in the stack. This means that it
requires some processing power on each end of the link. This is
because each lane is only one byte wide, very few pins are needed
to transmit the data.Actually serial transmission scheme is a way
of turning processing power into bandwidth. One of PCIe’s
features is the ability to aggregate multiple individual lanes
together to form a single link. In other words, two lanes could be
coupled together to form a single link capable of transmitting two
bytes at a time, thus doubling the link bandwidth.

Proma Goswami An introduction to PCI Express

 Introduction to PCI Express
Comparison with different version of PCIE
PCIE Protocol Details Comparison
Pci DMA Controller
References

Switch Switch

Lane(x3)
Lane (x1)

Proma Goswami An introduction to PCI Express

 Introduction to PCI Express
Comparison with different version of PCIE
Protocol
PCIE Protocol Details
PCIE Link
Pci DMA Controller
References

Protocol

Till now we were concerned with the system level impact of PCIe.
We did not look at the protocol itself. The following material will
make an attempt to explain the details of PCIe protocol, its layers
and the functions of each of the layers in a brief way.
PCI Express is a high performance, general purpose I/O
interconnect defined for a wide variety of future computing and
communication platforms.

Proma Goswami An introduction to PCI Express

 Introduction to PCI Express
Comparison with different version of PCIE
Protocol
PCIE Protocol Details
PCIE Link
Pci DMA Controller
References

PCIE Link

A Link represents a dual-simplex communications channel between

two components. The fundamental PCI Express Link consists of
two, low-voltage, differentially driven signal pairs: a Transmit pair
and a Receive pair.

Proma Goswami An introduction to PCI Express

 Introduction to PCI Express
Comparison with different version of PCIE
Protocol
PCIE Protocol Details
PCIE Link
Pci DMA Controller
References

PCIE Fabric Topology

CPU

PCI Express
PCI Express Memory
Endpoint Root

PCI Express
PCI Express
PCI Express to
PCI/PCI-x Bridge

Switch
PCI /PCI x
PCI Express
PCI Express
PCI Express PCI Express

Legacy Endpoint Legacy Endpoint Legacy Endpoint Legacy Endpoint

Figure : PCIE Fabric Topology

Proma Goswami An introduction to PCI Express

 Introduction to PCI Express
Comparison with different version of PCIE
Protocol
PCIE Protocol Details
PCIE Link
Pci DMA Controller
References

PCIE Fabric Topology

Root Complex
A Root Complex (RC) denotes the root of an I/O hierarchy
that connects the CPU/memory subsystem to the I/O.
Endpoints
Endpoint refers to a type of Function that can be the
Requester or Completer of a PCI Express transaction either on
its own behalf or on behalf of a distinct non-PCI Express
device (other than a PCI device or Host CPU), e.g., a PCI
Express attached graphics controller or a PCI Express-USB
host controller. Endpoints are classified as either legacy, PCI
Express, or Root Complex Integrated Endpoints.
PCI Express to PCI/PCI-X Bridge
A PCI Express to PCI/PCI-X Bridge provides a connection
between a PCI Express fabric and a PCI/PCI-X hierarchy.
Proma Goswami An introduction to PCI Express
 Introduction to PCI Express
Comparison with different version of PCIE
Protocol
PCIE Protocol Details
PCIE Link
Pci DMA Controller
References

PCI Express Layering Overview

PCI Express can be divided into three discrete logical layers: the
Transaction Layer, the Data Link Layer, and the Physical Layer.
Each of these layers is divided into two sections: one that
processes outbound (to be transmitted) information and one that
processes inbound (received) information.

Proma Goswami An introduction to PCI Express

 Introduction to PCI Express
Comparison with different version of PCIE
Protocol
PCIE Protocol Details
PCIE Link
Pci DMA Controller
References

PCI Express Layering

PCI Express uses packets to communicate information between

components. Packets are formed in the Transaction and Data Link
Layers to carry the information from the transmitting component
to the receiving component. As the transmitted packets flow
through the other layers, they are extended with additional
information necessary to handle packets at those layers. At the
receiving side the reverse process occurs and packets get
transformed from their Physical Layer representation to the Data
Link Layer representation and finally (for Transaction Layer
Packets) to the form that can be processed by the Transaction
Layer of the receiving device. Figure below shows the conceptual
flow of transaction level packet information through the layers.

Proma Goswami An introduction to PCI Express

 Introduction to PCI Express
Comparison with different version of PCIE
Protocol
PCIE Protocol Details
PCIE Link
Pci DMA Controller
References

PCI Express Layering

Transaction
Transaction

Data Link Data Link

Physical Physical

Logical Sub Block

Logical Sub Block

Electric Sub Block Electric Sub Block

RX Tx Rx Tx

Proma Goswami An introduction to PCI Express

 Introduction to PCI Express
Comparison with different version of PCIE
Protocol
PCIE Protocol Details
PCIE Link
Pci DMA Controller
References

Layers of the protocol

Transaction Layer
This is the top layer that interacts with the software above.
Functions of Transaction Layer:
Mechanisms for differentiating the ordering and processing
requirements of Transaction Layer Packets (TLPs)
Credit-based flow control
TLP construction and processing
Association of transaction-level mechanisms with device
resources including Flow Control and Virtual Channel
management

Proma Goswami An introduction to PCI Express

 Introduction to PCI Express
Comparison with different version of PCIE
Protocol
PCIE Protocol Details
PCIE Link
Pci DMA Controller
References

Layers of the protocol

Data Link Layer
The Data Link Layer acts as an intermediate stage between the
Transaction Layer and the Physical Layer. Its primary responsibility
is to provide a reliable mechanism for exchanging Transaction
Layer Packets (TLPs) between the two components on a Link.
Functions of Data Link Layer:
Data Exchange:
Error Detection and Retry:
Initialization and power management:
5.4.3 Physical Layer The Physical Layer isolates the Transaction
and Data Link Layers from the signaling technology used for Link
data interchange. The Physical Layer is divided into the logical
and electrical subblocks.
Proma Goswami An introduction to PCI Express
 Introduction to PCI Express
Comparison with different version of PCIE
Protocol
PCIE Protocol Details
PCIE Link
Pci DMA Controller
References

Layers of the protocol

Logical Sub-block Takes care of Symbol Encoding, framing, data

scrambling, Link initialization and training, Lane to lane de-skew

Proma Goswami An introduction to PCI Express

 Introduction to PCI Express
Comparison with different version of PCIE
Protocol
PCIE Protocol Details
PCIE Link
Pci DMA Controller
References

Layers of the protocol

Electrical Sub-block The electrical sub-block section defines the
physical layer of PCI Express 5.0 GT/s that consists of a reference
clock source, Transmitter, channel, and Receiver. This section
defines the electrical-layer parameters required to guarantee
interoperability among the above-listed PCI Express components.
This section comprehends both 2.5 GT/s and 5.0 GT/s electricals.
In many cases the parameter definitions between 2.5 and 5.0 GT/s
are identical, even though their respective values may differ.
However, the need at 5.0 GT/s to minimize guardbanding, while
simultaneously comprehending all phenomena affecting signal
integrity, requires that all the PCI Express system components -
Transmitter, Receiver, channel, and Refclk, be explicitly defined in
the specification. For this reason, each of these four components
has a separate specification section for 5.0 GT/s.
Proma Goswami An introduction to PCI Express
 Introduction to PCI Express
Comparison with different version of PCIE
PCIE Protocol Details
Pci DMA Controller
References

Impact of PCIE DMA performance

PCIE architecture has no central direct memory access controller.

PCI-Express Protocol Overhead
Maximum Payload Size, i.e. the maximum allowed size of a
single packet
Signal Integrity on the link, causing retransmission of packets
(packet replays)
Poor flow control update from the host computer system,
causing long stall times (i.e. times where the endpoint is not
allowed to send packets

Proma Goswami An introduction to PCI Express

 Introduction to PCI Express
Comparison with different version of PCIE
PCIE Protocol Details
Pci DMA Controller
References

PCI Express DMA performance MPS

MPS defines how many bytes of user data (= payload) is

contained in a PCI-Express data packet.
The higher this value is, the less is the protocol overhead,
since Packet header and Packet Footer remain the same.
The actual MPS value is negotiated during link training.
Since FPGA endpoints can implement big MPS values, the
host chipset determines the MPS value

Proma Goswami An introduction to PCI Express

 Introduction to PCI Express
Comparison with different version of PCIE
PCIE Protocol Details
Pci DMA Controller
References

References

References
www.arstechnica.com
www.pcisig.com
www.pcstats.com
www.digitalhomedesignline.com

Proma Goswami An introduction to PCI Express