Lecture 9: Branch Prediction
Basic idea, saturating counter, BHT, BTB, return address prediction, correlating prediction
Reducing Branch Penalty
Branch penalty in dynamically scheduled processors: wasted cycles due to pipeline flushing on mispredicted branches Reduce branch penalty:
1. Predict branch/jump instructions AND branch
direction (taken or not taken) branches)
2. Predict branch/jump target address (for taken 3. Speculatively execute instructions along the
predicted path
2
What to Use and What to Predict
Available info:
What to predict:
Current predicted PC Past branch history (direction and target) Conditional branch inst: branch direction and target address Jump inst: target address Procedure call/return: target address
PC
pred_PC
IM
Predictors
May need instruction predecoded
PC & Inst
pred info feedback PC
Mis-prediction Detections and Feedbacks
Detections: At the end of decoding
At commit (most cases)
Target address known at decoding, and not match Flush fetch stage Wrong branch direction or target address not match Flush the whole pipeline
FETCH RENAME REB/ROB SCHD EXE
predictors
(at EXE: MIPS R10000)
Feedbacks: Any time a mis-prediction is detected At a branchs commit (at EXE: called speculative update)
WB
COMMIT
4
Branch Direction Prediction
Predict branch direction: taken or not taken (T/NT)
taken Not taken BNE R1, R2, L1 L1:
Static prediction: compilers decide the direction Dynamic prediction: hardware decides the direction using dynamic information
1. 2. 3. 4. 5.
1-bit Branch-Prediction Buffer 2-bit Branch-Prediction Buffer Correlating Branch Prediction Buffer Tournament Branch Predictor and more
Predictor for a Single Branch
General Form
1. Access
PC
state
2. Predict Output T/NT
3. Feedback T/NT
1-bit prediction
T Predict Taken NT
Feedback
NT T
Predict Taken
Branch History Table of 1-bit Predictor
BHT also Called Branch Prediction Buffer in textbook Can use only one 1-bit predictor, but accuracy is low BHT: use a table of simple predictors, indexed by bits from PC Similar to direct mapped cache More entries, more cost, but less conflicts, higher accuracy BHT can contain complex predictors
K-bit
Branch address
2k
Prediction
1-bit BHT Weakness
Example: in a loop, 1-bit BHT will cause 2 mispredictions Consider a loop of 9 iterations before exit:
for (){ for (i=0; i<9; i++) a[i] = a[i] * 2.0; } End of loop case, when it exits instead of looping as before First time through loop on next time through code, when it predicts exit instead of looping Only 80% accuracy even if loop 90% of the time
2-bit Saturating Counter
Solution: 2-bit scheme where change prediction only if get misprediction twice: (Figure 3.7, p. 249)
T Predict Taken
11 01
NT
T
T NT T NT
10
Predict Taken
Predict Not Taken
00
NT
Predict Not Taken
Blue: stop, not taken Gray: go, taken Adds hysteresis to decision making process
Branch Target Buffer
Branch Target Buffer (BTB): Address of branch index to get prediction AND branch address (if taken)
Example: BTB combined with BHT
Branch PC
PC of instruction FETCH =? No: branch not predicted, proceed normally (Next PC = PC+4)
Note: must check for branch match now, since cant use wrong branch address
Predicted PC
Extra Yes: instruction is prediction state branch and use bits predicted PC as next PC
10
Return Addresses Prediction
Register indirect branch hard to predict address
Many callers, one callee Jump to multiple return addresses from a single address (no PC-target correlation)
SPEC89 85% such branches for procedure return Since stack discipline for procedures, save return address in small buffer that acts like a stack: 8 to 16 entries has small miss rate
11
Correlating Branches
Code example showing the potential
If (d==0) d=1; If (d==1)
Assemble code
BNEZ R1, L1 DADDIU R1,R0,#1 L1: DADDIU R3,R1,#-1 BNEZ R3, L2 L2:
Observation: if BNEZ1 is not taken, then BNEZ2 is taken
12
Correlating Branch Predictor
Idea: taken/not taken of recently executed branches is related to behavior of next branch (as well as the history of that branch behavior) Then behavior of recent branches selects between, say, 2 predictions of next branch, updating just that prediction (1,1) predictor: 1-bit global, 1-bit local
Branch address (4 bits)
1-bits per branch local predictors
Prediction
1-bit global branch history (0 = not taken)
13
Correlating Branch Predictor
General form: (m, n) predictor m bits for global history, n bits for local history Records correlation between m+1 branches Simple implementation: global history can be store in a shift register Example: (2,2) predictor, 2-bit global, 2-bit local
Branch address (4 bits)
2-bits per branch local predictors
Prediction
2-bit global branch history (01 = not taken then taken)
14
Accuracy of Different Schemes
(Figure 3.15, p. 206)
20%
Frequencyofof Mispredictions Frequency Mispredictions
18%
16%
14%
4096 Entries 2-bit BHT Unlimited Entries 2-bit BHT 1024 Entries (2,2) BHT
11%
12%
10%
8%
6% 6% 6% 5% 4%
6%
5%
4%
2%
1% 0%
1%
0% nasa7 matrix300 tomcatv doducd spice fpppp gcc espresso eqntott li
4,096 entries: 2-bits per entry
Unlimited entries: 2-bits/entry
1,024 entries (2,2)
15
Estimate Branch Penalty
EX: BHT correct rate is 95%, BTB hit rate is 95% Average miss penalty is 15 cycles How much is the branch penalty?
16
Accuracy of Return Address Predictor
17
