Compiler techniques for exposing ILP
Instruction Level Parallelism
Potential overlap among instructions Few possibilities in a basic block
Blocks are small (6-7 instructions) Instructions are dependent
Goal: Exploit ILP across multiple basic blocks
Iterations of a loop
for (i = 1000; i > 0; i=i-1) x[i] = x[i] + s;
Basic Scheduling
for (i = 1000; i > 0; i=i-1) x[i] = x[i] + s;
Sequential MIPS Assembly Code
Loop: LD ADDD SD SUBI BNEZ F0, 0(R1) F4, F0, F2 0(R1), F4 R1, R1, #8 R1, Loop
Pipelined execution: Loop: LD F0, 0(R1) stall ADDD F4, F0, F2 stall stall SD 0(R1), F4 SUBI R1, R1, #8 stall BNEZ R1, Loop stall
1 2 3 4 5 6 7 8 9 10
Scheduled pipelined execution: Loop: LD F0, 0(R1) 1 SUBI R1, R1, #8 2 ADDD F4, F0, F2 3 stall 4 BNEZ R1, Loop 5 SD 8(R1), F4 6
Loop Unrolling
Loop: Pros: Larger basic block More scope for scheduling and eliminating dependencies Cons: Increases code size LD ADDD SD SUBI BEQZ LD ADDD SD SUBI BEQZ LD ADDD SD SUBI BEQZ LD ADDD SD SUBI BNEZ F0, 0(R1) F4, F0, F2 0(R1), F4 R1, R1, #8 R1, Exit F6, 0(R1) F8, F6, F2 0(R1), F8 R1, R1, #8 R1, Exit F10, 0(R1) F12, F10, F2 0(R1), F12 R1, R1, #8 R1, Exit F14, 0(R1) F16, F14, F2 0(R1), F16 R1, R1, #8 R1, Loop
Comment: Often a precursor step for other optimizations
Exit:
Loop Transformations
Instruction independency is the key requirement for the transformations Example
Determine that is legal to move SD after SUBI and BNEZ Determine that unrolling is useful (iterations are independent) Use different registers to avoid unnecessary constrains Eliminate extra tests and branches Determine that LD and SD can be interchanged Schedule the code, preserving the semantics of the code
1. Eliminating Name Dependences
Loop: LD ADDD SD LD ADDD SD LD ADDD SD LD F0, 0(R1) F4, F0, F2 0(R1), F4 F0, -8(R1) F4, F0, F2 -8(R1), F4 F0, -16(R1) F4, F0, F2 -16(R1), F4 F0, -24(R1) Loop: LD ADDD SD LD ADDD SD LD ADDD SD LD F0, 0(R1) F4, F0, F2 0(R1), F4 F6, -8(R1) F8, F6, F2 -8(R1), F8 F10, -16(R1) F12, F10, F2 -16(R1), F12 F14, -24(R1)
Register Renaming
ADDD
SD SUBI BNEZ
F4, F0, F2
-24(R1), F4 R1, R1, #32 R1, Loop
ADDD
SD SUBI BNEZ
F16, F14, F2
-24(R1), F16 R1, R1, #32 R1, Loop
2. Eliminating Control Dependences
Loop: LD ADDD SD SUBI BEQZ LD ADDD SD SUBI BEQZ LD ADDD SD SUBI BEQZ LD ADDD SD SUBI BNEZ F0, 0(R1) F4, F0, F2 0(R1), F4 R1, R1, #8 R1, Exit F6, 0(R1) F8, F6, F2 0(R1), F8 R1, R1, #8 R1, Exit F10, 0(R1) F12, F10, F2 0(R1), F12 R1, R1, #8 R1, Exit F14, 0(R1) F16, F14, F2 0(R1), F16 R1, R1, #8 R1, Loop
Intermediate BEQZ are never taken Eliminate!
Exit:
3. Eliminating Data Dependences
Loop: LD ADDD SD SUBI LD ADDD SD SUBI LD ADDD SD SUBI LD ADDD SD SUBI BNEZ F0, 0(R1) F4, F0, F2 0(R1), F4 R1, R1, #8 F6, 0(R1) F8, F6, F2 0(R1), F8 R1, R1, #8 F10, 0(R1) F12, F10, F2 0(R1), F12 R1, R1, #8 F14, 0(R1) F16, F14, F2 0(R1), F16 R1, R1, #8 R1, Loop
Data dependencies SUBI, LD, SD Force sequential execution of iterations
Compiler removes this dependency by: Computing intermediate R1 values Eliminating intermediate SUBI Changing final SUBI
Data flow analysis Can do on Registers Cannot do easily on memory locations 100(R1) = 20(R2)
4. Alleviating Data Dependencies
Unrolled loop:
Loop: LD ADDD SD LD ADDD SD LD ADDD SD LD ADDD SD SUBI BNEZ F0, 0(R1) F4, F0, F2 0(R1), F4 F6, -8(R1) F8, F6, F2 -8(R1), F8 F10, -16(R1) F12, F10, F2 -16(R1), F12 F14, -24(R1) F16, F14, F2 -24(R1), F16 R1, R1, #32 R1, Loop
Scheduled Unrolled loop:
Loop: LD LD LD LD ADDD ADDD ADDD ADDD SD SD SUBI SD BNEZ SD F0, 0(R1) F6, -8(R1) F10, -16(R1) F14, -24(R1) F4, F0, F2 F8, F6, F2 F12, F10, F2 F16, F14, F2 0(R1), F4 -8(R1), F8 R1, R1, #32 16(R1), F12 R1, Loop 8(R1), F16
Some General Comments
Dependences are a property of programs Actual hazards are a property of the pipeline Techniques to avoid dependence limitations Maintain dependences but avoid hazards Code scheduling hardware software Eliminate dependences by code transformations Complex Compiler-based
Loop-level Parallelism
Primary focus of dependence analysis Determine all dependences and find cycles
for (i=1; i<=100; i=i+1) { x[i] = y[i] + z[i]; w[i] = x[i] + v[i]; } for (i=1; i<=100; i=i+1) { x[i+1] = x[i] + z[i]; } for (i=1; i<=100; i=i+1) { x[i] = x[i] + y[i]; y[i+1] = w[i] + z[i]; }
x[1] = x[1] + y[1]; for (i=1; i<=99; i=i+1) { y[i+1] = w[i] + z[i]; x[i+1] = x[i +1] + y[i +1]; } y[101] = w[100] + z[100];
Dependence Analysis Algorithms
Assume array indexes are affine (ai + b)
GCD test:
For two affine array indexes ai+b and ci+d: if a loop-carried dependence exists, then GCD (c,a) must divide (d-b) x[8*i ] = x[4*i + 2] +3 (2-0)/GCD(8,4)
General graph cycle determination is NP a, b, c, and d may not be known at compile time
Software Pipelining
Start-up
Finish-up
Iteration 0
Iteration 1
Iteration 2
Iteration 3
Software pipelined iteration
Example
Iteration i LD F0, 0(R1) LD F0, 0(R1) LD F0, 0(R1) Iteration i+1 Iteration i+2
ADDD F4, F0, F2
SD 0(R1), F4
ADDD F4, F0, F2 SD 0(R1), F4
ADDD F4, F0, F2 SD 0(R1), F4 16(R1), F4
Loop:
LD
F0, 0(R1)
Loop:
SD
ADDD F4, F0, F2 SD SUBI 0(R1), F4 R1, R1, #8
ADDD F4, F0, F2 LD SUBI F0, 0(R1) R1, R1, #8
BNEZ R1, Loop
BNEZ R1, Loop
Trace (global-code) Scheduling
Find ILP across conditional branches Two-step process
Trace selection
Find a trace (sequence of basic blocks) Use loop unrolling to generate long traces Use static branch prediction for other conditional branches
Trace compaction
Squeeze the trace into a small number of wide instructions Preserve data and control dependences
Trace Selection
A[I] = A[I] + B[I]
LW LW
F
R4, 0(R1) R5, 0(R2)
A[I] = 0?
ADD
SW
R4, R4, R5
0(R1), R4
BNEZ R4, else
B[I] = X
....
SW J Else: .... X 0(R2), . . . join
C[I] =
Join:
.... SW 0(R3), . . .
Summary of Compiler Techniques
Try to avoid dependence stalls Loop unrolling
Reduce loop overhead
Software pipelining
Reduce single body dependence stalls
Trace scheduling
Reduce impact of other branches
Compilers use a mix of three All techniques depend on prediction accuracy
Food for thought: Analyze this
Analyze this for different values of X and Y
To evaluate different branch prediction schemes For compiler scheduling purposes
add r1, r0, 1000 # all numbers in decimal add r2, r0, a # Base address of array a loop:
andi r10, r1, X beqz r10, even lw r11, 0(r2) addi r11, r11, 1 sw 0(r2), r11
even:
addi r2, r2, 4 subi r1, r1, Y bnez r1, loop
