Introduction to

Pipelining
 Pipelining:Daily Works!!!
Laundry Example
• W, X, Y, Z each have one load of
clothes to wash, dry, and fold
A B C D

• Washer takes 30 minutes

• Dryer takes 40
minutes
• Folder takes 20 minutes
 Sequential Laundry
Time
6 PM 7 8 9 10 11 Midnight

30 40 20 30 40 20 30 40 20 30 40 20
90
T
a
s
A 90
k

O
B 90

r
d C 90
e
r
D
Sequential laundry takes 6 hours for 4 loads
If they learned pipelining, how long would laundry take?
 Pipelined Laundry
6 PM 7 8 9 10 11 Midnight

Time

30 40 40 40 40 20
90
T
a
s
A
90
k

O B
90
r
d
e C
90
r

D
Pipelined laundry takes 3.5 hours for 4 loads
 Pipelining Lessons
• Pipelining doesn’t help
6 PM 7 8 9 latency of single task, it
helps throughput of entire
Time
workload
• Pipeline rate is limited by
30 40 40 40 40 20 the slowest pipeline stage
• Multiple tasks operating
T A simultaneously
a
s • Potential speedup = Number
k B Draining pipe stages
• Unbalanced lengths of pipe
O Filling stages reduce speedup
r C • Time to “fill” pipeline and
d time to “drain” it reduces
e speedup
r D
 What is pipelining?

• A pipeline is a series of stage, where some work is done at each

stage.

• Its an implementation technique whereby multiple instruction are

overlapped in execution.

• Implementation technique that exploits parallelism among the

instruction in a sequential instruction stream.

• A pipelined processor consists of sequence of processing

circuits called segment or stages through which a stream of
operands can be passed.
Why is pipelining desirable?
• Used to improved performance beyond what can be
achieved with non pipelined processing.
• Yields a reduction in the average execution time per
instruction.
 Pipeline Designer Goal
• To balance the length of each pipeline stage
• If stage are well balanced, then the time per instruction on
the pipelined machine is equal to (T)

T = time per instruction on the pipelined machine

Number of pipe stages

Then, the speed up from pipelining equals the number of pipe

stages
 Types of Pipelines
• Instructional Pipeline
– Where different stages of an instruction fetch and
execution are handled in a pipeline

• Arithmetic Pipeline
– Where different stages of an arithmetic operation are
handled along the stages of pipeline.
Instructional Pipeline
 Contd…
• The processing of an instruction need not be divided into only two steps. To gain
further speed up, the pipeline must have more stages
• consider the following decomposition of the instruction
execution
- Fetch Instruction (FI): Read the next expected instruction into a
buffer.
- Decode Instruction ((DI): Determine the opcode and the operand
specifiers.
-Calculate Operand (CO): calculate the effective address of each
source operand.
- Fetch Operands(FO): Fetch each operand from memory.

- Execute Instruction (EI): Perform the indicated operation.

- Write Operand (WO): Store the result in memory.

The timing diagram …
Implementation Pipelining using
DLX
 5 Steps of DLX Instr. Execution:
Step1
Step 1: Instruction fetch cycle (IF)
– Read instruction from memory and store into IR
• IR ← Mem[PC]

– Calculate the next instruction address

• NPC ← PC+4
• 1 instruction is stored in consecutive 4 bytes

Add NPC
+4
PC

Instr. IR
Memory
 5 Steps of DLX Instr. Execution:
Step2
Step 2: Instruction decode/register fetch cycle (ID)
– Read source registers to A and B
A ← Regs[IR6..10 ]
B ← Regs[IR11..15 ]

– Make 16 bits sign extension of A

Reg
16-bit immediate field to make a IR File
B
32-bit immediate value
Imm ← ((IR16 )16 ## IR16..31 ) Rd
b

– Decoding is done in parallel: OP

fixed-field decoding Sign
Imm
b ← Rd 16 Ext 32
 5 Steps of DLX Instr. Execution:
Step 3
Step 3: Execution/effective address cycle (EX):
– Memory reference: Effective Address calculation
» ALUOutput ← A + Imm

– Register-register ALU instruction: Perform ALU operation with R’s

» ALUOutput ← A func B; func B

– Register-Immediate ALU instruction: Perform ALU operation with

immediate operand
» ALUOutput ← A op Imm

– Branch: Effective Address calculation for branch target address

Determine condition code
» ALUOutput ← NPC + Imm; Cond ← (A op 0)
 Step 3 EX
Zero? Cond

NPC

MUX
A
ALU ALUOut

B MUX

Imm

OP
 5 Steps of DLX Instr. Execution:
Step 4
Step 4: Memory access/branch completion cycle (MEM):
– Memory reference : Access memory either
• for LD: LMD ← Mem[ALUOutput] or
• for ST: Mem[ALUOutput] ← B

– Branch : Test Condition

• if (cond) PC ← ALUOutput, NPC

MUX
PC
else PC ← NPC; ALUOut

Cond

Data
LMD
Memory
B
 5 Steps of DLX Instr. Execution:
Step 5
Step 5: Write-back cycle (WB):
Reg-Reg ALU : Store the result into the destination register
Regs[IR16..20 ] ← ALUOutput;

Reg-Immediate ALU : Store the result into destination register

Regs[IR11..15 ] ← ALUOutput;

Load instruction: Store the data read from memory to the

destination register
Regs[IR11..15 ] ← LMD;

LMD

MUX
Register
File
ALUOut

OP
 5 Steps of DLX Datapath
IF Stage ID Stage EX Stage MEM WB Stage
Stage

MUX
Add Zero?

+4

MUX
ALU
PC

Instr. Reg ALU Output

Memory File Data LMD
MUX

MUX
Memory

SMD

Sign
Ext 32
16
 A Simple Implementation
• A multi-cycle implementation
– needs temporary registers-- NPC, IC, A, B, Imm,
Cond, ALUOutput, LMD
– CPI improvements

• A single-cycle implementation
– one long clock cycle
– very inefficient for most machines that have a
reasonable variation among the amount of work
– requires the duplication of FU that could be shared in
a multi-cycle implementation
 Visualizing Pipeline
Time(clock cycles)
CC1 CC2 CC3 CC4 CC5 CC6 CC7 CC8 CC9

ALU
IM Reg DM Reg
Instruction Order

IM Reg ALU DM Reg Draining

IM Reg ALU DM Reg

ALU
IM Reg DM Reg
Filling

ALU
IM Reg DM Reg
Saving Information Produced
by Each Stage of Pipeline
• Information need to be stored at the end of a clock cycle,
otherwise it will be lost
• Each pipeline stage produces information(data, address, and
control) at the end of the clock cycle
• Thus, we need a storage(called inter-stage buffer) at end of
each pipeline stage
 Inter-Stage Buffer
in DLX Pipeline
• F/D Buffer
– IR, NPC
• D/EX Buffer
– A, B, Imm, b(destination Reg address to store result),
OP(OP-code), cond
– NPC
• EX/M Buffer
– ALUout(arithmetic result or effective address)
– NPC, cond, b, OP
• M/W Buffer
– LMD(data for LD)
– ALUout(arithmetic result), b, OP
 Pipelined DLX Datapath
- Multicycle -
IF Stage ID Stage EX Stage MEM WB
Stage Stage

MUX
Add Zero?

+4

MUX
PC

Instr. Reg

M/W Buffer
ALU
F/D Buffer

D/EX Buffer

EX/M Buffer
Memory File Data LMD
MUX

MUX
Memory

SMD

Sign
16 Ext 32
 Basic Performance Issues
• Pipelining increases the CPU instruction throughput.

• Does not reduce the execution time of an individual instruction

rather slightly increase the execution time due to overhead in the control
of the pipeline.

• Increase in throughput means that a program run faster and has lower
execution time.

• Imbalance among the pipe stage reduces performance since clock can
run no faster than the time needed for the slowest pipeline stage.

• Pipeline overhead arises from pipeline register delay and clock skew.
 Contd…
• Buffering between stages marginally increase Cycle time
• Harzards reduce the CPI.

What is Hazard???
- is a risk in which pipeline operation stall(stop) for one or
more clock cycle.
- it prevent next instruction from executing during its
designated clock cycle
 Pipeline Hazards
• There are three classes of hazards:
– Structural
• Happen due to simultaneous request for the same
resources by two or more instruction
– Eg. IF and MEM both required memory port.
– Data
• Instruction depends on result of prior instruction still
in the pipeline
– Control
• Happen due to branch and jump instruction.
Structural Hazard
Data harzard
 Data Harzard Solution
• Types:
– Interlock: H/w detect data dependency and stall depent
instructions.
 Contd..
- Forwarding or Bypassing: forward the result as soon as
available to EX
 Contd…
• Instruction Scheduling: Reorder instruction. Such that
dependent instruction are 2-3 cycle apart.
– Useful for covering load delays and branch delays
– Useful in hiding delays due to long latency FP operation
 Data Hazard Classification
• True data dependency - (RAW)
• Anti dependency - WAR
• Output dependency - WAW
Control Hazard

Next Class……