Computer Architecture

Pipeline: Hazards

Pipeline hazards prevent next instruction from executing during

designated clock cycle.

Hazards reduce the performance from the ideal speedup gained

by pipelining.
 Pipelining Outline

• Introduction
– Defining Pipelining
– Pipelining Instructions
• Hazards
– Structural hazards \
– Data Hazards
– Control Hazards
• Performance
 Pipeline Hazards

• Where one instruction cannot immediately

follow another
• Types of hazards
– Structural hazards - attempt to use the same resource by
two or more instructions
– Control hazards - attempt to make branching decisions
before branch condition is evaluated
– Data hazards - attempt to use data before it is ready
• Can always resolve hazards by waiting
 Structural Hazards

• Attempt to use the same resource by two or

more instructions at the same time
• Example: Single Memory for instructions and
data
– Accessed by IF stage
– Accessed at same time by MEM stage
• Solutions
– Delay the second access by one clock cycle, OR
– Provide separate memories for instructions & data
» This is called a “Harvard Architecture”
» Real pipelined processors have separate caches
 Pipelined Example -
Executing Multiple Instructions

• Consider the following instruction sequence:

Lw (r)
Sw (r)
Add (r)
Sub (r)
 Executing Multiple Instructions
Clock Cycle 1

LW

IF/ID ID/EX EX/MEM MEM/WB

ADD
ADD
4
<<2
PC
ADDR RD RN1 RD1
32 5
ALU Zero
Instruction RN2
5
Memory Register
WN File RD2
5
M
WD U ADDR
X
Data
E Memory RD M
U
16 X 32 X
T WD
N
5
D
 Executing Multiple Instructions
Clock Cycle 2

SW LW

IF/ID ID/EX EX/MEM MEM/WB

ADD
ADD
4
<<2
PC
ADDR RD RN1 RD1
32 5
ALU Zero
Instruction RN2
5
Memory Register
WN File RD2
5
M
WD U ADDR
X
Data
E Memory RD M
U
16 X 32 X
T WD
N
5
D
 Executing Multiple Instructions
Clock Cycle 3

ADD SW LW

IF/ID ID/EX EX/MEM MEM/WB

ADD
ADD
4
<<2
PC
ADDR RD RN1 RD1
32 5
ALU Zero
Instruction RN2
5
Memory Register
WN File RD2
5
M
WD U ADDR
X
Data
E Memory RD M
U
16 X 32 X
T WD
N
5
D
 Executing Multiple Instructions
Clock Cycle 4

SUB ADD SW LW

IF/ID ID/EX EX/MEM MEM/WB

ADD
ADD
4
<<2
PC
ADDR RD RN1 RD1
32 5
ALU Zero
Instruction RN2
5
Memory Register
WN File RD2
5
M
WD U ADDR
X
Data
E Memory RD M
U
16 X 32 X
T WD
N
5
D
 Executing Multiple Instructions
Clock Cycle 5

SUB ADD SW LW

IF/ID ID/EX EX/MEM MEM/WB

ADD
ADD
4
<<2
PC
ADDR RD RN1 RD1
32 5
ALU Zero
Instruction RN2
5
Memory Register
WN File RD2
5
M
WD U ADDR
X
Data
E Memory RD M
U
16 X 32 X
T WD
N
5
D
 Executing Multiple Instructions
Clock Cycle 6

SUB ADD SW

IF/ID ID/EX EX/MEM MEM/WB

ADD
ADD
4
<<2
PC
ADDR RD RN1 RD1
32 5
ALU Zero
Instruction RN2
5
Memory Register
WN File RD2
5
M
WD U ADDR
X
Data
E Memory RD M
U
16 X 32 X
T WD
N
5
D
 Executing Multiple Instructions
Clock Cycle 7

SUB ADD

IF/ID ID/EX EX/MEM MEM/WB

ADD
ADD
4
<<2
PC
ADDR RD RN1 RD1
32 5
ALU Zero
Instruction RN2
5
Memory Register
WN File RD2
5
M
WD U ADDR
X
Data
E Memory RD M
U
16 X 32 X
T WD
N
5
D
 Executing Multiple Instructions
Clock Cycle 8
SUB

IF/ID ID/EX EX/MEM MEM/WB

ADD
ADD
4
<<2
PC
ADDR RD RN1 RD1
32 5
ALU Zero
Instruction RN2
5
Memory Register
WN File RD2
5
M
WD U ADDR
X
Data
E Memory RD M
U
16 X 32 X
T WD
N
5
D
 One Memory Port Structural Hazards
Alternative View - Multicycle Diagram

Time (clock cycles)

Cycle 1 Cycle 2 Cycle 3 Cycle 4 Cycle 5 Cycle 6 Cycle 7

I Load Ifetch

ALU
Reg DMem Reg

n
s

ALU
Reg
t Instr 1
Ifetch Reg DMem

r.

ALU
Ifetch Reg DMem Reg
Instr 2
O
r
d

ALU
Ifetch Reg DMem Reg
Instr 3
e
r
 One Memory Port Structural Hazards
Alternative View - Multicycle Diagram

Time (clock cycles)

Cycle 1 Cycle 2 Cycle 3 Cycle 4 Cycle 5 Cycle 6 Cycle 7

I Load Ifetch Memory Conflict

ALU
Reg DMem Reg

n
s

ALU
Reg
t Instr 1
Ifetch Reg DMem

r.

ALU
Ifetch Reg DMem Reg
Instr 2
O
r
d

ALU
Ifetch Reg DMem Reg
Instr 3
e
r
 One Memory Port Structural Hazards

Time (clock cycles)

Cycle 1 Cycle 2 Cycle 3 Cycle 4 Cycle 5 Cycle 6 Cycle 7

I Load Ifetch

ALU
Reg DMem Reg

n
s

ALU
Reg
t Instr 1
Ifetch Reg DMem

r.

ALU
Ifetch Reg DMem Reg
Instr 2
O
r
Stall Bubble Bubble Bubble Bubble Bubble
d
e
r

ALU
Ifetch Reg DMem Reg
Instr 3
 Structural Hazards
We want to compare the performance of two machines. Which machine is faster?
• Machine A: Dual ported memory - so there are no memory stalls
• Machine B: Single ported memory, but its pipelined implementation has a clock
rate that is 1.05 times faster
Assume:
• Ideal CPI = 1 for both
• Loads are 40% of instructions executed
 Speed Up Equations for Pipelining

CPIpipelined  Ideal CPI  Average Stall cycles per Inst

Ideal CPI  Pipeline depth Cycle Timeunpipelined

Speedup  
Ideal CPI  Pipeline stall CPI Cycle Timepipelined

For simple RISC pipeline, CPI = 1:

Pipeline depth Cycle Timeunpipelined

Speedup  
1  Pipeline stall CPI Cycle Timepipelined
 Pipelining Summary

• Speed Up <= Pipeline Depth; if ideal CPI is 1, then:

Pipeline Depth Clock Cycle Unpipelined

Speedup = X
1 + Pipeline stall CPI Clock Cycle Pipelined

• Hazards limit performance on computers:

– Structural: need more HW resources
– Data (RAW,WAR,WAW)
– Control
 Data Hazards
• Data hazards occur when data is used before
it is ready
Time (in clock cycles)

Value of CC 1 CC 2 CC 3 CC 4 CC 5 CC 6 CC 7 CC 8 CC 9
register $2: 10 10 10 10 10/– 20 – 20 – 20 – 20 – 20
Program
execution
order
(in instructions)
sub $2, $1, $3 IM Reg DM Reg

and $12, $2, $5 IM Reg DM Reg

or $13, $6, $2 IM Reg DM Reg

add $14, $2, $2 IM Reg DM Reg

sw $15, 100($2) IM Reg DM Reg

The use of the result of the SUB instruction in the next three instructions causes a
data hazard, since the register $2 is not written until after those instructions read it.
 Data Hazards
Execution Order is:
Read After Write (RAW)
InstrI
InstrJ InstrJ tries to read operand before InstrI writes it

I: add r1,r2,r3
J: sub r4,r1,r3

• Caused by a “Dependence” (in compiler nomenclature). This

hazard results from an actual need for communication.
 Data Hazards
Execution Order is:
Write After Read (WAR)
InstrI
InstrJ InstrJ tries to write operand before InstrI reads i
– Gets wrong operand

I: sub r4,r1,r3
J: add r1,r2,r3
K: mul r6,r1,r7
– Called an “anti-dependence” by compiler writers.
This results from reuse of the name “r1”.
 Data Hazards
Execution Order is:
Write After Write (WAW)
InstrI
InstrJ tries to write operand before InstrI writes it
InstrJ – Leaves wrong result ( InstrI not InstrJ )

I: sub r1,r4,r3
J: add r1,r2,r3
K: mul r6,r1,r7
• Called an “output dependence” by compiler writers
This also results from the reuse of name “r1”.
 Data Hazard Detection in MIPS (1)
Read after Write
Time (in clock cycles)

Value of CC 1 CC 2 CC 3 CC 4 CC 5 CC 6 CC 7 CC 8 CC 9
register $2: 10 10 10 10 10/– 20 – 20 – 20 – 20 – 20
Program
execution IF/ID ID/EX EX/MEM MEM/WB
order
(in instructions)
sub $2, $1, $3 IM Reg DM Reg

and $12, $2, $5 IM Reg DM Reg

or $13, $6, $2 IM Reg DM Reg

add $14, $2, $2 IM Reg DM Reg

sw $15, 100($2) IM Reg DM Reg

 Data Hazards

• Solutions for Data Hazards

– Stalling
 Data Hazard - Stalling

0 2 4 6 8 10 12 16 18

W
add $s0,$t0,$t1 IF ID EX MEM s0 $s0
written
here

STALL
BUBBLE BUBBLE BUBBLE BUBBLE BUBBLE

STALL
BUBBLE BUBBLE BUBBLE BUBBLE BUBBLE

sub $t2,
$s0,$t3 R
IF s0 EX MEM WB

$s0 read
here
 Data Hazards - Forwarding
• Key idea: connect new value directly to next stage
• Still read s0, but ignore in favor of new result

0 2 4 6 8 10 12 16 18

ID W
add $s0,$t0,$t1 IF ID EX MEM s0
new value
of s0

R
sub $t2,$ s0,$t3 IF s0 EX MEM WB
 Data Hazards - Forwarding
• STALL still required for load - data avail. after MEM
• MIPS architecture calls this delayed load, initial
implementations required compiler to deal with this

0 2 4 6 8 10 12 16 18

ID W
lw $s0,20($t1) IF ID EX MEM s0

new value
of s0

STALL
BUBBLE BUBBLE BUBBLE BUBBLE BUBBLE

R
sub $t2,
$s0,$t3 IF s0 EX MEM WB
• Introduction
– Defining Pipelining
– Pipelining Instructions
• Hazards
– Structural hazards
– Data Hazards
– Control Hazards \
• Performance
 Control Hazards

Control hazard occurs when the pipeline makes wrong decisions on

branch prediction and therefore brings instructions into the pipeline
that must subsequently be discarded. The term branch hazard also
refers to a control hazard.

Branches are different portions of the same pipeline that run at different
times. Most pipelines are composed of more than one branch.
 Control Hazards

Control Hazards. Control hazards can cause a

greater performance loss for pipeline than
data hazards. When a branch is executed, it may
or may not change the PC (program counter) to
something .

A control hazard is taken place when we need

to find the destination of a branch, and can’t
fetch any new instructions until we know that
destination.
 Static Branch Prediction
For every branch encountered during execution predict whether the
branch will be taken or not taken.

Predicting branch not taken:

1. Speculatively fetch and execute in-line instructions following the
branch
2. If prediction is incorrect, flush the pipeline of speculated
instructions
• Convert these instructions to NOPs by clearing pipeline registers
• These have not updated memory or registers at time of flush

Predicting branch taken:

3. Speculatively fetch and execute instructions at the branch target
address
4. Useful only if target address known earlier than branch outcome
• May require stall cycles till target address known
• Flush pipeline if prediction is incorrect
• Must ensure that flushed instructions do not update memory/registers
 Control Hazard - Stall

0 2 4 6 8 10 12 16 18

add $r4,$r5,$r6 IF ID EX MEM WB

beq $r0,$r1,tgt IF ID EX MEM WB

STALL
BUBBLE BUBBLE BUBBLE BUBBLE BUBBLE

sw $s4,200($t5) IF ID EX MEM WB
beq
writes PC new PC
here used here
 Control Hazard - Correct Prediction

0 2 4 6 8 10 12 16 18

add $r4,$r5,$r6 IF ID EX MEM WB

beq $r0,$r1,tgt IF ID EX MEM WB

tgt:
sw $s4,200($t5) IF ID EX MEM WB

Fetch assuming
branch taken
 Control Hazard - Incorrect Prediction

0 2 4 6 8 10 12 16 18

add $r4,$r5,$r6 IF ID EX MEM WB

beq $r0,$r1,tgt IF ID EX MEM WB

tgt:
sw $s4,200($t5) IF
(incorrect - STALL) BUBBLE BUBBLE BUBBLE BUBBLE

or $r8,$r8,$r9 IF ID EX MEM WB

“Squashed”
instruction
 Summary - Control Hazard Solutions
• Stall - stop fetching instr. until result is
available
– Significant performance penalty
– Hardware required to stall
• Predict - assume an outcome and continue
fetching (undo if prediction is wrong)
– Performance penalty only when guess wrong
– Hardware required to "squash" instructions
• Delayed branch - specify in architecture that
following instruction is always executed
– Compiler re-orders instructions into delay slot
– Insert "NOP" (no-op) operations when can't use
– This is how original MIPS worked
Thank You