PIPELINING

 Implementation technique : multiple instructions

are
overlapped in execution

 Each step in the pipeline completes a part of the

instruction, each step operates in parallel with other
steps

 Instructions enter at one end and exit at other end

 Throughput determined by how often

an instruction exits
pipeline.
Pipelined vs. Single-Cycle Instruction
Execution: the Plan

Program
executio 2 4 6 8 1 1 14 16 1
n order Tim 0 2 8
(in e
instructions) Instructio Data Single-
lw $1, Reg ALU Reg
100($0)
n
fetch
access
cycle
Instructio Data
lw $2, 8 Reg ALU Reg
n acces
200($0) ns fetch s
Instructio
lw $3, 300($0) 8 n
ns fetch ...
8 ns

Assume 2 ns for memory access, ALU operation; 1 ns for

register access: therefore, single cycle clock 8 ns; pipelined
clock cycle 2 ns.
executio 2 4 6 8 10 12 14
Program Tim
n order
e
(in
instructions) Instructio Data
lw $1, Reg ALU Reg
n acces
100($0) fetch
Instructio
s
Data
Pipeline
lw $2, 2 Reg ALU Reg
200($0) ns
n access d
fetch
Instructio Data
lw $3, 2 Reg ALU Reg
n acces
300($0) ns fetch s
2 2 2 2 2
ns ns ns ns ns
Time between instructions (pipelined) =

Time between instructions (non

pipelined)/
(number of pipe
stages)

Pipelining improves performance by increasing

instruction throughput, as opposed to decreasing the
execution time of an individual instruction
Pipelining: Keep in Mind

 Pipelining does not reduce latency of a single task,

it
increases throughput of entire workload
 Pipeline rate limited by longest stage

potential speedup = number pipe stages

unbalanced lengths of pipe stages reduces
speedup
 Time to fill pipeline and time to drain it – when
there is
slack in the pipeline – reduces speedup
 Pipelining
MIPS
 What makes it easy with MIPS?

all instructions are same length

so fetch and decode stages are similar for all
instructions

just a few instruction formats

simplifies instruction decode and makes it
possible in one stage

memory operands appear only in load/stores

so memory access can be deferred to exactly one
later
stage

operands are aligned in memory

one data transfer instruction requires one memory
 Pipelining MIPS

 What makes it hard?


structural hazards: different instructions, at
different stages, in the pipeline want to use the
same hardware resource

control hazards: succeeding instruction, to put
into pipeline, depends on the outcome of a
previous branch instruction, already in pipeline

data hazards: an instruction in the pipeline
requires data to be computed by a previous
instruction still in the pipeline
 Structural Hazards

 Structural hazard: inadequate hardware to simultaneously

support all instructions in the pipeline in the same clock
cycle
 E.g., suppose single – not separate – instruction and data
memory in pipeline below with one read port
 then Program
a structural hazard
execution 2 4 between
6 8first 10
and fourth
12 14 lw
Time
instructions
order
(in instructions)
Instruction Data
lw $1, Reg ALU Reg
100($0)
fetch access
Pipelined
Instruction Data
lw $2, 2 ns Reg Reg
200($0)
fetch
ALU
access Hazard if single
Instruction Data memory
lw $3, 2 ns Reg Reg
fetch access
300($0) ALU
Instruction ALU Data Reg
lw $4, Reg
2 ns fetch access
400($0)
2 ns

2 ns

2 ns

2 ns

2 ns
Pipelined Datapath

 the 5 steps in instruction execution

 Instruction Fetch & PC Increment (IF)

 Instruction Decode and Register Read
(ID)
 Execution or calculate address (EX)
 Memory access (MEM)
 Write result into register (WB)
 Review - Single-Cycle Datapath “Steps”

ADD

4 ADD

PC <<2
ADDR RD 32 Instruction I
1 3
5 5 5
Instruction 6 2
Memory RN1 RN2 WN
RD1 Zer
Register File ALU o
WD
RD2 M
U ADDR
X
Data
E MemoryRD M
U
1 X X
6 32 WD
T
N
D

EX
IF ID Execute/ Address MEM WB
Instruction Fetch Instruction Decode Calc. Memory Access Write Back
Pipelining the datapath requires values passed from one
pipestage to next - need pipeline registers.

Pipeline registers carry both data and control from one

stage to next.

Any instruction is active exactly one stage of the pipeline

at a time and actions taken on behalf of an instruction
occur between a pair of pipeline stages.
 Pipelined Datapath

Pipeline registers wide enough to hold data

ADD coming in

4 ADD
64 128
PC bits bits 97 64
<<2
ADDR RD
Instruction I bits bits
32 16 32
5 5 5
Instruction
Memory RN1 RN2
WN
Zero
RD1 ALU
Register File
WD
M
RD2 U ADDR
X
Data
E Memory RD M
U
16 X 32 X
T WD
N
D

IF/ID ID/EX EX/ MEM/

MEM WB
 Pipelined Datapath

Pipeline registers wide enough to hold data

ADD coming in

4 ADD
64 128
bits bits
PC <<2 97 64
ADDR RD
Instruction I bits bits
32 16 32
Instruction 5 5 5

Memory RN1 RN2

WN
Zero
RD1 ALU
Register File
WD
M
RD2 U ADDR
X
Data
E Memory RD M
U
16 X 32 X
T WD
N
D

IF/ID ID/EX EX/MEM MEM/

WB
Only data flowing right to left may cause hazard…, why?
IF for Load, Store, …
ID for Load, Store, …
EX for Load
MEM for Load
WB for Load

Wrong
registe
r
numb
er
 Corrected Datapath

IF/ID ID/EX EX/MEM MEM/WB

ADD
ADD
4 64 133
bits bits 102 69
<<2 bits
bits
PC
ADDR RD 5
RN1 RD1
32 Zero
Instruction RN2
ALU
5
Memory Register
5
WN File RD2 M
WD U ADDR
X
Data
E Memory RD M
U
16 X 32 X
T WD
N
5 D

Destination register number is also passed through ID/EX,

EX/MEM and MEM/WB registers, which are now wider by 5
bits
Events happening in the Pipeline stages

IF/ID.IR Mem [PC]

IF/ID.NPC, PC ( if ((EX/MEM .opcpde ==branch) &

EX/MEM.cond){EX/MEM.ALUoutput}
else {PC+4});

ID/EX.A Regs[IF/ID.IR[rs]]
ID/EX.B Regs[IF/ID.IR[rt]]

ID/EX.NPC IF/ID.NPC; ID/EX.IR IF/ID.IR

ID/EX.IMM Sign extend (IF/ID .IR [immediate field])
Recall Single-Cycle – Control
Signals Effect of control
bits
Signal Effect when deasserted Effect when asserted
Name

RegDst The register destination number for the The register destination number for the
Write register comes from the rt field (bits Write register comes from the rd field (bits 15-
20-16) 11)
RegWrite None The register on the Write register input is
written with the value on the Write data
input
AlLUSrc The second ALU operand comes The second ALU operand is the sign-
from the second register file output extended, lower 16 bits of the instruction
(Read data 2)
PCSrc The PC is replaced by the output of the adder The PC is replaced by the output of the adder
that computes the value of PC + 4 that computes the branch target
MemRead None Data memory contents designated by the
address input are put on the first Read data
output
MemWrite None Data memory contents designated by the
Memto- Reg Mem Mem
address input are replaced by the value of the
Deter- Instruction RegDst ALUSrc Reg Write Read Write
Write data
Branch ALUOp1 ALUp0
input
minin R-format
MemtoReg 1 to the register
The value fed 0 0 data input
Write 1 0 value0fed to the
The 0 register Write
1 0
data input
g lw comes from 0the ALU 1 1 1 1 from
comes 0 the data
0 memory 0 0
contr sw X 1 X 0 0 1 0 0 0
ol bits beq X 0 X 0 0 0 1 0 1
 Pipeline
 Control
Initial design – motivated by single-cycle datapath
control –
use the same control signals
 Observe:
 No separate write signal for the PC as it is written
modified
Will
by be
every cycle hazard
 No separate write signals for the pipeline registers as detectio
n unit!!
 they are written
No separate every
read cycle
signal for instruction memory as it is read
every
clock cycle

No separate read signal for register file as it is read every
clock cycle
 Need to set control signals during each pipeline stage
 Since control signals are associated with components
active during a single pipeline stage, can group
control lines into five groups according to pipeline
stage
Pipelined Datapath with
Control I
PCSrc

0
M
u
x
1

IF/ID ID/EX EX/MEM MEM/WB

Add

Add
4 Ad result
d Branch
Shift
RegWrite left
2
Read MemWrite
Instruction

PC Address register 1 Read

Read data 1
ALUSrc MemtoReg
register 2 ZeZre
Instruction Registers Read oro
memory Write 0 Read
data 2 Address 1
register M ALU ALU
data M
u re Data
Write x u
sult memory x
data 1 0
Write
data
Instruction
[15– 0] 32 6
Sign ALU
16
extend control MemRead
Same Instruction
control [20– 16]
0
M
signals as Instruction ALUOp
[15– 11] u
the single- x
1
cycle RegDst
datapath
Pipeline Control
Signals
There are five stages in the
pipeline

instruction fetch / PC Nothing to control as instruction
memory read and PC write are
increment always enabled

instruction decode / register
fetch

execution / address
calculation

memory access
Execution/Address Write-back

write backlines
Calculation stage control Memory access stage
control lines
stage control
lines
Reg ALU ALU ALU Branc Mem Mem Reg Mem
Instruction Dst Op1 Op0 Src h Read Write write to Reg

R-format 1 1 0 0 0 0 0 1 0
lw 0 0 0 1 0 1 0 1 1
sw X 0 0 1 0 0 1 0 X
beq X 0 1 0 1 0 0 0 X
Pipeline Control
Implementation
Pass control signals along just like the data – extend
each pipeline register to hold needed control bits for
succeeding stages
WB

Instruction
Control M WB

EX M WB

IF/ID ID/EX EX/MEM MEM/WB

 Note: The 6-bit funct field of the instruction required in

the EX stage to generate ALU control can be retrieved as
the 6 least significant bits of the immediate field which
is sign-extended and passed from the IF/ID register to
the ID/EX register
 Pipelined Datapath with
Control II 0
M
ID/EX

uPCSrc WB
x EX/MEM
1
Control M WB
MEM/WB

EX M WB
IF/ID

Add

Add
4 Add result

RegWrite
Branch
Shift
left 2

MemWrite
ALUSrc

MemtoReg
Read
Instructio

PC Address register 1
Read
Read data 1
register 2 Zero
n

Instruction Registers Read ALU ALU

memory Write 0 Read
data 2 result Address 1
register M data
Data M
u
Write memory u
x x
data 1
0
Write
data

Instruction
[15– 0] 16 32 6
Sign ALU MemRead
extend contro
Control Instruction
l
[20– 16]
signals 0
M
ALUOp

emanate Instruction
[15– 11]
u
x
from the 1
RegDst
control
pipeline
portions of
registers
Pipelined Execution and
Control
 Instruction
sequence:

lw $10, 20($1)
sub $11, $2, $3
and $12, $4, $7
or $13, $6, $7
add $14, $8, $9
lw $10, 20($1)
sub $11, $2, $3
and $12, $4, $7
or $13, $6, $7
add $14, $8, $9
I F : lw $ 1 0 , 2 0 ( $ I D : b e fo r e < 1 E X : before<2 M E M : b e fo r e < 3 W B : b e fo re < 4 >
1) > > >

IF/ID ID/EX EX/MEM MEM/WB

0
M 00 00
u WB
x
1 000
000 00
Control WB
M 0 0 0
0000 0
00 M
EX 0 0 WB 0

Add

Add
4 Addresult

RegWrite
Shift Branch
left 2

MemWrite
ALUSrc
Read

MemtoReg
Instruction

PC Address register 1 Read

Read data 1
register 2 Zero
Instruction Registers Read
memory Write 0 ALU ALU Read
data 2 Address 1
register M result data
Data M
u
Write x memory u
data x
1
0
Write
data

Instruction
[15– 0] Sign ALU MemRead
extend control

Instruction
[20– 16]
0 ALUOp
M
Instruction u
[15– 11] x
C lo c k 1 1
RegDst
IF : sub $11, $2, $ ID : lw $10, 20($ E X : before<1 M E M : b e fo r e < 2 W B : b e fo re < 3 >
3 1) > >

IF/ID ID/EX EX/MEM MEM/WB

0
M 11 00
u WB
x
1 lw 000
010 00
Control WB
M 0 0 0
0
0001 00 M WB 0
EX 0 0

Add

4 Add
Add result

RegWrite
Shift Branch
left 2

MemWrite
ALUSrc
1

MemtoReg
Read
Instruction

PC Address register
1 Read $1
X data 1
Read
register 2 Zero
Instruction Registers Read $X
memory Write 0 ALU ALU Read
data 2 Address 1
register M result data
Data M
u
Write memory u
x x
data 1
0
Write
data

Instruction
20 [15– 0] Sign 20 ALU MemRead
extend control

Instruction
10 [20– 16] 10
0 ALUOp
M
Instruction u
X [15– 11] X x
C lo c k 2 1
RegDst
IF: and $12, $4, $ ID : sub $11, $2, $ E X : lw $ 1 0 , . . . M E M : b e fo r e < 1 W B : b e fo r e < 2
5 3 > >

IF/ID ID/EX EX/MEM MEM/WB

0
M 10 11
u WB
x
1 sub
000 010 00
Control M WB
0 0 0
1100
00
EX 1 M
0
0 WB 0

Add

4 Add
Add result

RegWri
Shift Branch
left 2

te
ALUSrc

MemW
2 Read
Instruction

M emtoRe g
rite
PC Address register $1
1 Read $2
3 data 1
register
Read Zero
Instruction Registers Read $3
memory 2 ALU ALU
Write 0 Address Read
data 2 1
register M result data
Data M
u
Write memory u
x x
data 1
0
Write
data

Instruction
X [15–0] Sign X 20 ALU MemRead
extend contro
l
Instruction
X [20–16] X 10
0 ALUOp
M
Instruction u
11 [15–11] 11 x
C lo c k 3 1
RegDst
IF: or $13, $6, $ ID: and $12, $2, $ E X: sub $11, . . . M E M : lw $ 1 0 , . . . W B : b e fo r e < 1
7 3 >

IF/ID ID/EX EX/MEM MEM/WB

0
M 10 10
u WB
x
1 and
000 000 11
Control M WB
1 0 0
1100 10 1 WB 0
EX 0 M 0

Ad
d
4 Add
Add result

RegWrite
Shift Branch
left 2

MemWrite
ALUSrc
4 Read

MemtoReg
Instruction

PC Address register 1 $2
Read $4
5 data 1
Read
register Zero
Instruction Registers Read $5
2 $3
memory Write 0 ALU ALU Read
data 2 Address 1
registe M result data
Data M
r u
memory u
x x
Write 1
data 0
Write
data

Instruction
X [15–0] Sign X ALU MemRead
extend control

Instruction
X [20–16] X
0 ALUO
M p 10
Instruction u
12 [15–11] 12 11 x
C lo c k 4 1
RegDs
t
IF: add $14, $8, $ ID: or $13, $6, $ EX : and $12, . . . M E M : sub $1 W B : lw $ 1 0 , . . .
9 7 1, . . .

IF/ID ID/EX EX/MEM MEM/WB

0
M 10 10
u WB
x
1 or
000 000 10
Control M WB
1 0 1
1100
10
EX 0
0
M 0 WB 1

Ad
d
Add
4 Addresult

RegWrite
Branch
Shift
left 2

MemWrite
ALUSrc
6 Read

MemtoReg
Instruction

PC Address register $4
1 Read $6
7 data 1
Read
register Zero
Instruction Registers Read $7 $5
memor 10 2 ALU ALU
Write 0 Address Read
data 2 1
y register M result data
Data M
u
Write memory u
x x
data 1
0
Write
data

Instruction
X [15–0] Sign X AL MemRead
extend U
contro
l
Instruction
X [20–16] X
0 ALUOp
M 11 10
Instruction u
13 [15–11] 13 12 x
C lock 1
5
RegDst
I F : a fte r< 1 > ID: add $14, $8, $ EX : or $13, . . . M E M : and $12, . . . W B: sub $1
9 1, . . .

IF/ID ID/EX EX/MEM MEM/

0
M WB
10 10
u WB
x
1 add
000 000 10
Control M WB
1 0 1
1100 10 0 WB 0
EX 0 M 0

Ad
d
4 Add
Addresult

RegWrite
Shift Branch
left 2

MemWrite
ALUSrc
8

MemtoReg
Read
Instruction

PC Address register $6
1 Read $8
data
9
Read
register 1 Zero
Instruction Registers Read $9
2 $7
memory 11 Write 0 ALU ALU Read
data Address 1
register M result data
2 Data M
u
Write memor u
x x
data 1 y
0
Write
data

Instruction
X [15–0] Sign X AL MemRead
extend control
U

Instruction
X [20–16] X
0 ALUO
M p 12 11
Instruction u
14 [15–11] 14 13 x
C lock 1
6 RegDst
I F : a fte r< 2 ID : a fte r< 1 > EX: add $1 M EM : or $1 W B: and $1
> 4, . . . 3, . . . 2, . . .

IF/ID ID/EX EX/MEM MEM/WB

0
M 00 10
u WB
x
1
000 000 10
Control M WB
1 0 1
0000
10
EX 0
0
M 0 WB 0

Ad
d
4 Add
Add result

RegWrite
Shift Branch
left

MemWrite
2 ALUSrc
Read

MemtoReg
Instruction

PC Address register Read $8

1 data 1
Read
register 2 Zero
Instruction Registers Read $9
memory 12 Write 0 ALU ALU Read
data Address 1
register M result data
2 Data M
u
Write x memory u
data x
1
0
Write
data

Instruction
[15–0] Sign ALU MemRead
extend control

Instruction
[20–16]
0 ALUOp
M 13 12
Instruction u
[15–11] 14 x
C lo c k 7 1
RegDst
I F : a fte r< 3 > I D : a fte r< 2 > E X : a fte r< 1 > M EM : add $1 W B: or $1
4, . . . 3, . . .

IF/ID ID/EX EX/MEM MEM/

0
M WB
00 00
u WB
x
1
000 000 10
Control M WB
0 0 1
0000
00 0 WB 0
EX 0 M 0

Ad
d
4 Add
Add result

RegWrite
Shift Branch
left

MemWrite
2 ALUSrc

MemtoReg
Read
Instruction

PC Address register
1 Read
data 1
Read
register 2 Zero
Instruction Registers Read
memory 13 Write 0 ALU ALU Read
data Address 1
register M result data
2 Data M
u
Write x memory u
data x
1
0
Write
data

Instruction
[15–0] Sign ALU MemRead
extend control

Instruction
[20–16]
0 ALUOp
M 14 13
Instruction u
[15–11] x
C lo c k 8 1
RegDst
 Pipelined Execution and
Control
I F : a fte r < 4 > I D : a fte r< 3 > E X : a f te r < 2 > M E M : a f te r < 1 > W B: add $1
4, . . .

IF/ID ID/EX EX/MEM MEM/WB

0
M 00
u 00
WB
x
1 000
000 00
Control WB
M 0 0 1
0000 0
00 M
EX 0 0 WB 0

Add

4 Add
Add result
RegWrite
Shift Branch
left 2

MemWrit
ALUSrc
Read
Instruction

MemtoRe
PC Address register 1

e
Read
Read data 1
register 2 Zero

g
Instruction Registers Read ALU ALU
memory 14 Write 0 Read 1
data 2 result Address data
register M M
u Data
Write x memory u
data x
1
0
Write
data

Instruction
[15–0] Sign ALU MemRead
extend control

Instruction
[20–16]
0 ALUOp
M 14
Instruction u
[15–11] x
C lo ck 1
9 RegDst