Pipeline Hazards
Hazards:
Situation when next instruction cannot execute in
the following cycle due to certain dependencies
Structural Hazards:
Hardware cannot support the
combination of instructions
Example: If we had same memory for data and
instructions
Data Hazards:
Data hazards occur when the pipeline
must be stalled because one step must wait for another to
complete.
Control Hazards:
Control hazard occur when branch
decision has to be taken when one instruction is executing
with others following it
Data Hazards
add $s0, $t0, $t1
sub $t2, $s0, $t3
The add instruction doesnt write its result until
the fifth stage, meaning that we would have to
waste three clock cycles in the pipeline
Solution: As soon as the ALU creates the sum
for the add, we can supply it as an input for the
subtract.
Adding extra hardware to retrieve the missing
item early from the internal resources is called
forwarding or bypassing
Forwarding
Note:
only
Forwarding can occur in the forward direction
Load-use data hazard
Solution:
Software that reorders the instruction or
hardware detection
Reordering code to avoid
stalls
Code for C language:
a = b + e;
c = b + f;
Corresponding Assembly Language code:
lw $t1, 0($t0)
lw $t2, 4($t0)
add $t3, $t1,$t2
sw $t3, 12($t0)
lw $t4, 8($t0)
add $t5, $t1,$t4
sw $t5, 16($t0)
Find the hazards in the preceding code segment and
reorder the instructions to avoid any pipeline stalls.
Reordering code to avoid stalls
Cont
Solution
lw $t1, 0($t0)
lw $t2, 4($t0)
lw $t4, 8($t0)
add $t3, $t1,$t2
sw $t3, 12($t0)
add $t5, $t1,$t4
sw $t5, 16($t0)
Observation
MIPS instruction set allows one result
per instruction only
Forwarding is harder if there are
multiple results to forward per
instruction or they need to write a
result early on in instruction
execution.
Data Hazards
sub $2, $1,$3
and $12,$2,$5
or $13,$6,$2
add $14,$2,$2
sw $15,100($2)
Where are the data hazards in the code above?
Last four instructions are dependent on the first
instruction add and sw will get the correct values of
$2 while and instruction and or instruction will get
previous value of $2
Solution: Forwarding
Pipelined Dependencies
Dependence Detection
Discuss only the forwarding to EXE stage of the next instruction
Notation to describe data hazards
1a. EX/MEM.RegisterRd = ID/EX.RegisterRs
1b. EX/MEM.RegisterRd = ID/EX.RegisterRt
2a. MEM/WB.RegisterRd = ID/EX.RegisterRs
2b. MEM/WB.RegisterRd = ID/EX.RegisterRt
Hazard between following two instructions can be detected when sub
instruction is in the MEM stage and and Instruction is in the EXE stage
sub $2, $1,$3
and $12,$2,$5
Or $13, $6, $2
add $14,$2,$2
sw $15,100($2)
Data Hazard1: Sub-and: EX/MEM.RegisterRd = ID/EX.RegisterRs = $2
Data Hazard2: Sub-or: EX/MEM.RegisterRd = ID/EX.RegisterRt = $2
No Hazard: Sub-add
No Hazard: Sub-sw
Dependence Detection
Cont
Base line: If we can take the inputs to the ALU from any pipeline register
rather than just ID/EX, then we can forward the proper data.
Writing the Forwarding Unit
Hazard can occur with the immediately previous
instruction called EX Hazard
Hazard can occur with the instruction preceding the
immediately previous instruction called MEM Hazard
We need to design Forwarding unit such that the
accurate data is forwarded to the ALU input.
Forwarding unit must decide when to pass
1. Register value read in the previous cycle
2. ALU result of the previous instruction
3. ALU result of the instruction previous than the
last one
Dependence Detection and Forwarding Unit
Writing the Forwarding Unit
Cont
We need to write separate Testconditions inside the forwarding unit
for EX and MEM hazards
Writing the Forwarding Unit
Cont..
EX-Hazard Detection Test
MEM-Hazard Detection Test version
1
Writing the Forwarding Unit
Cont
Conditional Forwarding from MEM stage
Consider the forwarding code
add $1,$1,$2
add $1,$1,$3
add $1,$1,$4
Forwarding for instruction 3 above should be done
from instruction 1 or instruction 2?
Instruction 2 i.e. MEM stage rather than the WB
stage of instruction 1
So.. Modify the MEM hazard conditions
accordingly from the previous slide
Writing the Forwarding Unit
Cont
MEM Hazard Detection Test Version
2
Modified Data Path
Recall Load-use data hazard
Solution:
Software that reorders the instruction or
hardware detection
Inserting a Stall in the
Pipeline
Two steps again
Hazard Detection
Stalling the pipeline if forwarding not possible
Load-use hazard detection Test
If the instruction in the ID stage is stalled, then the instruction in the IF
stage must also be stalled; otherwise, we would lose the fetched
instruction
Preventing these two instructions from making progress is accomplished
simply by preventing the PC register and the IF/ID pipeline register from
changing
Base Line: Deasserting all nine control signals (setting them to 0) in the
EX, MEM, and WB stages will create a do nothing or nop(NO
OPERATION) instruction.
Inserting a Stall in the Pipeline
Cont
The hazard detection unit controls the writing of the PC and IF/ID
registers plus the multiplexor that chooses between the real
control values and all 0s.
The hazard detection unit stalls and deasserts the control fields if
the load-use hazard test above is true.
Control Hazards
Which instruction should be fetched after
fetching the branch instruction?
Branch decision is unknown in the next
cycle
The pipeline cannot possibly know what
the next instruction should be, since it
only just received the branch instruction
from memory
Control Hazards Cont
Solution 1
Solution 1: Moving the branch related
hardware to ID stage
One stall is still required i.e. the next
instruction should not be fetched when the
branch instruction is being decoded
Solution 1 Cont
Moving the branch decision up
requires two actions to occur earlier
1. Computing the branch target address
Move the branch adder from the EX stage to
the ID stage
2. Evaluating the branch decision
Complex implementation
Forwarding and data hazard detection hardware
required
Solution 1 Cont
During ID, we need complete the equality comparison so
that if the instruction is a branch, we can set the PC to the
branch target address.
Operands required during the equality comparison may
depend on the previous instructions executing in the other
stages of the pipeline Forwarding required again
Introduction of new forwarding logic
Operands of a branch can come from either the ALU/MEM or
MEM/WB pipeline registers.
Data hazard can occur and a stall might be required
R-Type instruction followed by the depending branch instruction
would require 1 more stall
Load followed by the depending branch instruction would require
2 more stalls
Solution 1 Cont
Solution 1 Cont
Solution 2
Solution 2: Predict the branch decision Statically
Predict always that branches will be untaken. When
youre right, the pipeline proceeds at full speed. Only
when branches are taken does the pipeline stall.
If branches are untaken half the time this optimization
halves the cost of control hazards
Improvement? Take predictions for some branches as
taken and for other as untaken
Example: At the bottom of loops are branches that jump
back to the top of the loop. Since they are likely to be
taken and they branch backward, we could always
predict taken for branches that jump to an earlier
address.
Solution 2 Cont
Solution 3
Solution 3: Dynamic Hardware predictors
Make their guesses depending on the behavior of
each branch and may change predictions for a
branch over the life of a program
Keep a history for each branch as taken or
untaken, and then use the recent past behavior
to predict the future
Branch predictors have approximately 90%
accuracy when amount and type of history
becomes extensive
Solution 3 Cont
Dynamic branch predictor
1-bit branch prediction buffer: Contains 1 bit indicating
whether the branch was taken last time or not
2-bit branch prediction buffer: prediction must be
wrong twice before it is changed
Correlating predictors: both a local branch, and the
global behavior of recently executed branches
together yields greater prediction accuracy
Tournament predictors: Use multiple predictors,
tracking, for each branch, which predictor yields the
best results. Select can operate favoring favoring
whichever of the predictors has been more accurate
Solution 3 Cont
Branch prediction buffer can be
accessed with the instruction address
during the IF pipe stage.
If the instruction is predicted as taken,
fetching begins from the target as soon
as the PC is known after the ID stage
Else sequential fetching continues.
Solution 4
Delayed Decision: Compilers try to place an instruction
after the branch instruction which is independent of the
branch decisions
Code reordering - Same as we discussed before
