AI 302

Embedded Systems
SPRING 2023

Lecture 07 - Performance Measurements

Single and Multiple Cycle Processor Designs
Instructor: Dr. Tarek Abdul Hamid
What is Performance?
 How can we make intelligent choices about computers?

 Why some computer hardware performs better at some programs, but performs
less at other programs?

 How do we measure the performance of a computer?

 What factors are hardware related? software related?

 How does machine’s instruction set affect performance?

 Understanding performance is key to understanding underlying organizational

motivation

2 Embedded Systems Dr. Tarek Abdul Hamid

Response Time and Throughput
 Response Time
 Time between start and completion of a task, as observed by end user
 Response Time = CPU Time + Waiting Time (I/O, OS scheduling, etc.)

 Throughput
 Number of tasks the machine can run in a given period of time

 Decreasing execution time improves throughput

 Example: using a faster version of a processor
 Less time to run a task  more tasks can be executed

 Increasing throughput can also improve response time

 Example: increasing number of processors in a multiprocessor
 More tasks can be executed in parallel
 Execution time of individual sequential tasks is not changed
 But less waiting time in scheduling queue reduces response time
3 Embedded Systems Dr. Tarek Abdul Hamid
Book’s Definition of Performance

 For some program running on machine X

1
PerformanceX =
Execution timeX

 X is n times faster than Y

PerformanceX Execution timeY

= =n
PerformanceY Execution timeX

4 Embedded Systems Dr. Tarek Abdul Hamid

What do we mean by
Execution Time?
 Real Elapsed Time

 Counts everything:

 Waiting time, Input/output, disk access, OS scheduling, … etc.

 Useful number, but often not good for comparison purposes

 Our Focus: CPU Execution Time

 Time spent while executing the program instructions

 Doesn't count the waiting time for I/O or OS scheduling

 Can be measured in seconds, or Can be related to number of CPU clock cycles

5 Embedded Systems Dr. Tarek Abdul Hamid

Clock Cycles
 Clock cycle = Clock period = 1 / Clock rate

Cycle 1 Cycle 2 Cycle 3

 Clock rate = Clock frequency = Cycles per second

 1 Hz = 1 cycle/sec 1 KHz = 103 cycles/sec
 1 MHz = 106 cycles/sec 1 GHz = 109 cycles/sec
 2 GHz clock has a cycle time = 1/(2×109) = 0.5 nanosecond (ns)

 We often use clock cycles to report CPU execution time

CPU cycles
CPU Execution Time = CPU cycles × cycle time =
Clock rate
6 Embedded Systems Dr. Tarek Abdul Hamid
Improving Performance
 To improve performance, we need to
 Reduce number of clock cycles required by a program, or
 Reduce clock cycle time (increase the clock rate)

 Example:
 A program runs in 10 seconds on computer X with 2 GHz clock
 What is the number of CPU cycles on computer X ?
 We want to design computer Y to run same program in 6 seconds
 But computer Y requires 10% more cycles to execute program
 What is the clock rate for computer Y ?
 Solution:
 CPU cycles on computer X = 10 sec × 2 × 109 cycles/s = 20 × 109
 CPU cycles on computer Y = 1.1 × 20 × 109 = 22 × 109 cycles
 Clock rate for computer Y = 22 × 109 cycles / 6 sec = 3.67 GHz

7 Embedded Systems Dr. Tarek Abdul Hamid

Clock Cycles Per Instruction (CPI)
 Instructions take different number of cycles to execute

 Multiplication takes more time than addition

 Floating point operations take longer than integer ones

 Accessing memory takes more time than accessing registers

 CPI is an average number of clock cycles per instruction

I1 I2 I3 I4 I5 I6 I7 CPI = 14/7 = 2
1 2 3 4 5 6 7 8 9 10 11 12 13 14 cycles

 Important point

Changing the cycle time often changes the number of cycles required for
various instructions (more later)

8 Embedded Systems Dr. Tarek Abdul Hamid

Performance Equation
 To execute, a given program will require …

 Some number of machine instructions

 Some number of clock cycles

 Some number of seconds

 We can relate CPU clock cycles to instruction count

CPU cycles = Instruction Count × CPI

 Performance Equation: (related to instruction count)

Time = Instruction Count × CPI × cycle time

9 Embedded Systems Dr. Tarek Abdul Hamid
Factors Impacting Performance

Time = Instruction Count × CPI × cycle time

I-Count CPI Cycle

Program X X

Compiler X X

ISA X X X

Organization X X

Technology X
Using the Performance Equation
 Suppose we have two implementations of the same ISA
 For a given program
 Machine A has a clock cycle time of 250 ps and a CPI of 2.2
 Machine B has a clock cycle time of 500 ps and a CPI of 1.0
 Which machine is faster for this program, and by how much?

 Solution:
 Both computers execute same count of instructions = I
 CPU execution time (A) = I × 2.2 × 250 ps = 550 × I ps
 CPU execution time (B) = I × 1.0 × 500 ps = 500 × I ps

550 × I
 Computer B is faster than A by a factor = = 1.1
500 × I

11 Embedded Systems Dr. Tarek Abdul Hamid

Determining the CPI
 Different types of instructions have different CPI
Let CPIi = clocks per instruction for class i of instructions
Let Ci = instruction count for class i of instructions

n ∑ (CPI × C ) i i

∑ (CPI × C )
i=1
CPU cycles = CPI = n
i i
i=1 ∑C
i=1
i

 Designers often obtain CPI by a detailed simulation

 Hardware counters are also used for operational CPUs
12 Embedded Systems Dr. Tarek Abdul Hamid
Example on Determining the CPI
 Problem
A compiler designer is trying to decide between two code sequences for a
particular machine. Based on the hardware implementation, there are three
different classes of instructions: class A, class B, and class C, and they require one,
two, and three cycles per instruction, respectively.
The first code sequence has 5 instructions: 2 of A, 1 of B, and 2 of C
The second sequence has 6 instructions: 4 of A, 1 of B, and 1 of C
Compute the CPU cycles for each sequence. Which sequence is faster?
What is the CPI for each sequence?
 Solution
CPU cycles (1st sequence) = (2×1) + (1×2) + (2×3) = 2+2+6 = 10 cycles
CPU cycles (2nd sequence) = (4×1) + (1×2) + (1×3) = 4+2+3 = 9 cycles
Second sequence is faster, even though it executes one extra instruction
CPI (1st sequence) = 10/5 = 2 CPI (2nd sequence) = 9/6 = 1.5

13 Embedded Systems Dr. Tarek Abdul Hamid

Second Example on CPI
Given: instruction mix of a program on a RISC processor
What is average CPI?
What is the percent of time used by each instruction class?
Classi Freqi CPIi CPIi × Freqi %Time
ALU 50% 1 0.5×1 = 0.5 0.5/2.2 = 23%
Load 20% 5 0.2×5 = 1.0 1.0/2.2 = 45%
Store 10% 3 0.1×3 = 0.3 0.3/2.2 = 14%
Branch 20% 2 0.2×2 = 0.4 0.4/2.2 = 18%
Average CPI = 0.5+1.0+0.3+0.4 = 2.2
How faster would the machine be if load time is 2 cycles?
What if two ALU instructions could be executed at once?
14 Embedded Systems Dr. Tarek Abdul Hamid
MIPS as a Performance Measure
 MIPS: Millions Instructions Per Second
 Sometimes used as performance metric
 Faster machine  larger MIPS
 MIPS specifies instruction execution rate

Instruction Count Clock Rate

MIPS = =
Execution Time × 106 CPI × 106
 We can also relate execution time to MIPS

Inst Count Inst Count × CPI

Execution Time = =
MIPS × 106 Clock Rate
15 Embedded Systems Dr. Tarek Abdul Hamid
Drawbacks of MIPS
Three problems using MIPS as a performance metric

1. Does not take into account the capability of instructions

 Cannot use MIPS to compare computers with different instruction sets

because the instruction count will differ

2. MIPS varies between programs on the same computer

 A computer cannot have a single MIPS rating for all programs

3. MIPS can vary inversely with performance

 A higher MIPS rating does not always mean better performance

 Example in next slide shows this anomalous behavior

16 Embedded Systems Dr. Tarek Abdul Hamid

MIPS example
 Two different compilers are being tested on the same program for a 4 GHz
machine with three different classes of instructions: Class A, Class B, and Class
C, which require 1, 2, and 3 cycles, respectively.

 The instruction count produced by the first compiler is 5 billion Class A

instructions, 1 billion Class B instructions, and 1 billion Class C instructions.

 The second compiler produces 10 billion Class A instructions, 1 billion Class B

instructions, and 1 billion Class C instructions.

 Which compiler produces a higher MIPS?

 Which compiler produces a better execution time?

17 Embedded Systems Dr. Tarek Abdul Hamid

Solution to MIPS Example
 First, we find the CPU cycles for both compilers
 CPU cycles (compiler 1) = (5×1 + 1×2 + 1×3)×109 = 10×109
 CPU cycles (compiler 2) = (10×1 + 1×2 + 1×3)×109 = 15×109
 Next, we find the execution time for both compilers
 Execution time (compiler 1) = 10×109 cycles / 4×109 Hz = 2.5 sec
 Execution time (compiler 2) = 15×109 cycles / 4×109 Hz = 3.75 sec
 Compiler1 generates faster program (less execution time)
 Now, we compute MIPS rate for both compilers
 MIPS = Instruction Count / (Execution Time × 106)
 MIPS (compiler 1) = (5+1+1) × 109 / (2.5 × 106) = 2800
 MIPS (compiler 2) = (10+1+1) × 109 / (3.75 × 106) = 3200
 So, code from compiler 2 has a higher MIPS rating !!!

18 Embedded Systems Dr. Tarek Abdul Hamid

Amdahl’s Law
 Amdahl's Law is a measure of Speedup
 How a computer performs after an enhancement E
 Relative to how it performed previously

Performance with E ExTime before

Speedup(E) = =
Performance before ExTime with E

 Enhancement improves a fraction f of execution time by a factor s and the

remaining time is unaffected

ExTime with E = ExTime before × (f / s + (1 – f ))

1
Speedup(E) =
(f / s + (1 – f ))
19 Embedded Systems
Dr. Tarek Abdul Hamid
Example on Amdahl's Law

 Suppose a program runs in 100 seconds on a machine, with multiply responsible

for 80 seconds of this time. How much do we have to improve the speed of
multiplication if we want the program to run 4 times faster?
 Solution: suppose we improve multiplication by a factor s
25 sec (4 times faster) = 80 sec / s + 20 sec
s = 80 / (25 – 20) = 80 / 5 = 16
Improve the speed of multiplication by s = 16 times
 How about making the program 5 times faster?
20 sec ( 5 times faster) = 80 sec / s + 20 sec
s = 80 / (20 – 20) = ∞ Impossible to make 5 times faster!

20 Embedded Systems Dr. Tarek Abdul Hamid

Example
Consider two different implementations, M1 and M2, of the same instruction set.
There are five classes of instructions (A, B, C, D and E) in the instruction set. M1 has
a clock rate of 4 GHz and M2 has a clock rate of 6 GHz.

a) Assume that peak performance is defined as the fastest rate that a computer
can execute any instruction sequence. What are the peak performances of M1
and M2 expressed in instructions per second?

b) If the number of instructions executed in a certain program is divided equally

among the classes of instructions, except that for class A, which occurs twice as
often as each of the others, how much faster is M2 than M1?
21
Embedded Systems Dr. Tarek Abdul Hamid
Solution
a) For peak performance, the machine will be executing the fastest set of
instructions. M1 will be executing instructions of class A only, and M2 will be
executing instructions that belong to class A, B, or C.

Peak performance of M1 = 4 × 109 / 1 = 4 × 109 IPS = 4000 MIPS

Peak performance of M2 = 6 × 109 / 2 = 3 × 109 IPS = 3000 MIPS

b) Average CPI
= on M1 = (2 × 1 +× 2 + 3 + 4 + 3) / 6 = 14 / 6 = 2.33
Average CPI on M2 = (2 × 2 + 2 + 2 + 4 + 4) / 6 = 16 / 6 = 2.67

Since the instruction count is the same:

Performance of M2 Clock Rate of M2 CPI on M1

Performance of M1
= Clock Rate of M1
×
CPI on M2

M2 is faster than M1 by a factor of (6 GHz / 4 GHz) × (2.33 / 2.67) = 1.31

22 Embedded Systems
Dr. Tarek Abdul Hamid
Benchmarks
 Performance best obtained by running a real application

 Use programs typical of expected workload

 Representatives of expected classes of applications

 Examples: compilers, editors, scientific applications, graphics, ...

 SPEC (System Performance Evaluation Corporation)

 Funded and supported by a number of computer vendors

 Companies have agreed on a set of real programs and inputs

 Various benchmarks for …

CPU performance, graphics, high-performance computing, client-server

models, file systems, Web servers, etc.
 Valuable indicator of performance (and compiler technology)

23 Embedded Systems Dr. Tarek Abdul Hamid

Performance and Power
 Power is a key limitation

 Battery capacity has improved only slightly over time

 Need to design power-efficient processors

 Reduce power by

 Reducing frequency

 Reducing voltage

 Putting components to sleep

 Energy efficiency

 Important metric for power-limited applications

 Defined as performance divided by power consumption

24 Embedded Systems Dr. Tarek Abdul Hamid

Things to Remember
 Performance is specific to a particular program
 Any measure of performance should reflect execution time
 Total execution time is a consistent summary of performance
 For a given ISA, performance improvements come from
 Increases in clock rate (without increasing the CPI)
 Improvements in processor organization that lower CPI
 Compiler enhancements that lower CPI and/or instruction count
 Algorithm/Language choices that affect instruction count
 Pitfalls (things you should avoid)
 Using a subset of the performance equation as a metric
 Expecting improvement of one aspect of a computer to increase performance
proportional to the size of improvement

25 Embedded Systems Dr. Tarek Abdul Hamid

The Performance Perspective
 Recall, performance is determined by:

 Instruction count I-Count

 Clock cycles per instruction (CPI)

 Clock cycle time

CPI Cycle
 Processor design will affect

 Clock cycles per instruction

 Clock cycle time

 Single cycle datapath and control design:

 Advantage: One clock cycle per instruction

 Disadvantage: long cycle time

26 Dr. Tarek Abdul Hamid

Embedded Systems
Levels in Processor Design
 Circuit design
 Keywords: transistors, wires….. etc.
 Results in gates, flip-flops….. etc.

 Logical design
 Putting gates (AND, NAND, …) and flip-flops together to build basic blocks
such as registers, ALU’s etc (cf. CS 221)

 Register transfer
 Describes execution of instructions by showing data flow between the basic
blocks

 Processor description (the ISA or MIPS)

 System description
 Includes memory hierarchy, I/O, multiprocessing etc

27 Embedded Systems Dr. Tarek Abdul Hamid

Review of MIPS Instruction Formats
 All instructions are 32-bit wide
 Three instruction formats: R-type, I-type, and J-type

Op6 Rs5 Rt5 Rd5 sa5 funct6

Op6 Rs5 Rt5 immediate16

Op6 immediate26

 Op6: 6-bit opcode of the instruction

 Rs5, Rt5, Rd5: 5-bit source and destination register numbers
 sa5: 5-bit shift amount used by shift instructions
 funct6: 6-bit function field for R-type instructions
 immediate16: 16-bit immediate value or address offset
 immediate26: 26-bit target address of the jump instruction
28
Embedded Systems Dr. Tarek Abdul Hamid
MIPS Subset of Instructions
 Only a subset of the MIPS instructions are considered

 ALU instructions (R-type): add, sub, and, or, xor, slt

 Immediate instructions (I-type): addi, slti, andi, ori, xori

 Load and Store (I-type): lw, sw

 Branch (I-type): beq, bne

 Jump (J-type): j

 This subset does not include all the integer instructions

 But sufficient to illustrate design of datapath and control

 Concepts used to implement the MIPS subset are used to construct a broad
spectrum of computers

29 Dr. Tarek Abdul Hamid

Embedded Systems
Register Transfer Level (RTL)
 RTL is a description of data flow between registers
 RTL gives a meaning to the instructions
 All instructions are fetched from memory at address PC

 Two types of components:

 Combinational : the output is a function of the input (e.g., adder)
 Sequential : state is remembered (e.g., register)

30 Embedded Systems Dr. Tarek Abdul Hamid

Synchronous design
What is the meaning of Synchronous design ?
 Use of a periodic clock
 edge-triggered clocking determines when signals can be read and when the
output of circuits is stable
 Values in storage elements can be updated only at clock edges
 Clock tells when events can occur, e.g., when signals sent by control unit are
obeyed in the ALU

Stor. Elem 1 Stor. Elem 2

Comb.logic

Clock cycle
Note: the same storage
element can be read /written
in the same cycle
31 Embedded Systems
Dr. Tarek Abdul Hamid
Clocking Methodology
 Clocks are needed in a sequential logic to
 We assume edge-
decide when a state element (register) should
triggered clocking
be updated
 All state changes occur
 To ensure correctness, a clocking methodology
on the same clock edge
defines when data can be written and read
 Data must be valid and
stable before arrival of
clock edge
 Edge-triggered clocking

Register 2
allows a register to be
Register 1

Combinational logic read and written during

same clock cycle
clock

rising edge falling edge

32 Embedded Systems Dr. Tarek Abdul Hamid

Processor design:
data path and control unit
CPU
Combinational
Memory control ALU
Hierarchy
PC
Registers
state Sequential

Memory
Data path
bus

33 Embedded Systems Dr. Tarek Abdul Hamid

Processor design
 Data path
 How does data flow between various basic blocks
 What operations can be performed when data flows
 What can be done in one clock cycle

 Control unit
 Sends signals to data path elements
 Tells what data to move, where to move it, what operations are to be
performed

 Memory hierarchy
 Holds program and data

34 Embedded Systems Dr. Tarek Abdul Hamid

Data path basic building blocks.
Storage elements

 Basic building block (at the RT level) is a register

 In our mini-MIPS implementation registers will be 32-bits
 A register can be read or written

Input bus

Register Write enable signal

Output bus

35 Embedded Systems Dr. Tarek Abdul Hamid

Register file

 Array of registers (32 for the integer registers in MIPS)

 ISA tells us that we should be able to:
 read 2 registers, write one register in a given instruction (at this point we want
one instruction per cycle)
 Register file needs to know which registers to read/write

Read register number bus 0

Write register number
Write enable Read register number bus 1

Read data output bus 0

Write data input bus Register file
Read data output bus 1

36 Embedded Systems Dr. Tarek Abdul Hamid

Memory

 Conceptually, like register file but much larger

 Can only read one location or write to one location per cycle

Read memory address

Write memory address
Read control signal
Write enable

Write data bus Memory Read data bus

37 Embedded Systems Dr. Tarek Abdul Hamid

Combinational elements

Multiplexor (MUX): selects the value of one of Input busses

its inputs to be routed to the output

Select control Mux

signal

Demultiplexor (deMUX or SEL): routes its Output bus

input to one of its outputs
Output busses

Select control Sel

signal
Input bus

38 Embedded Systems Dr. Tarek Abdul Hamid

Arithmetic and Logic Unit
(ALU - combinational)

 Computes (arithmetic or logical operation) output from its two inputs

Zero result bit

Input bus 0
ALU Output bus

Input bus 1

ALU control
(opcode/function)

39 Embedded Systems Dr. Tarek Abdul Hamid

Putting basic blocks together
(skeleton of data path for arith/logical operations)

Zero result bit

Write register number Read register number bus 0

Write enable Read register number bus 1

Read data 0
Register file ALU

Read data 1
ALU control
(opcode/function)
Write data input bus

40 Embedded Systems Dr. Tarek Abdul Hamid

Introducing instruction fetch
Zero result bit

Read Reg #0
Read data 0
Read Reg #1
ALU
Write Reg #

Reg. FileRead data 1

ALU control
(opcode/function)
Write data

Instruction address
Instr. memory
PC

41 Embedded Systems Dr. Tarek Abdul Hamid

PC has to be incremented
(assume no branch)

4
Adder

Instruction address Instruction

Instr. memory
PC

42 Embedded Systems Dr. Tarek Abdul Hamid

Load-Store instructions
Read enable
Instruction
Read data 0
Read Reg #0
Read Reg #1
ALU Data
Write Reg # R/W address
memory
Reg. File

32-bit “store” data

Sign

16-bit offset extend Write enable

Data from “load”

43 Embedded Systems Dr. Tarek Abdul Hamid

Data path for straight code
(reg-reg,imm,load/store)
Read enable
Instruction
Read data 0
Read Reg #0
Read Reg #1
Read data 1 ALU Data
Write Reg # R/W address
memory
Reg. File

32-bit
Sign.
“store” data
16-bit offset ext Write enable

Data for result register

Mux

44 Embedded Systems Dr. Tarek Abdul Hamid

Branch data path

ALU

4 Sftl 2
Adder

32-bit
Instruction

Inst.
PC 16-bit
memory Sign.
ext

45 Embedded Systems
Dr. Tarek Abdul Hamid
Drawbacks of
Single Cycle Processor
 Long cycle time
 All instructions take as much time as the slowest

ALU Instruction Fetch Reg Read ALU Reg Write

longest delay
Load Instruction Fetch Reg Read ALU Memory Read Reg Write

Store Instruction Fetch Reg Read ALU Memory Write

Branch Instruction Fetch Reg Read ALU

Jump Instruction Fetch Decode

 Alternative Solution: Multicycle implementation

 Break down instruction execution into multiple cycles
46 Embedded Systems
Dr. Tarek Abdul Hamid
Multicycle Implementation
 Break instruction execution into five steps

 Instruction fetch

 Instruction decode and register read

 Execution, memory address calculation, or branch completion

 Memory access or ALU instruction completion

 Load instruction completion

 One step = One clock cycle (clock cycle is reduced)

 First 2 steps are the same for all instructions

Instruction # cycles Instruction # cycles

ALU & Store 4 Branch 3
Load 5 Jump 2

47 Embedded Systems Dr. Tarek Abdul Hamid

Performance Example

 Assume the following operation times for components:

 Instruction and data memories: 200 ps

 ALU and adders: 180 ps

 Decode and Register file access (read or write): 150 ps

 Ignore the delays in PC, mux, extender, and wires

 Which of the following would be faster and by how much?

 Single-cycle implementation for all instructions

 Multicycle implementation optimized for every class of instructions

 Assume the following instruction mix:

 40% ALU, 20% Loads, 10% stores, 20% branches, & 10% jumps
48
Embedded Systems Dr. Tarek Abdul Hamid
Solution
Instruction Instruction Register ALU Data Register
Total
Class Memory Read Operation Memory Write
ALU 200 150 180 150 680 ps
Load 200 150 180 200 150 880 ps
Store 200 150 180 200 730 ps
Branch 200 150 180 530 ps
Jump 200 150 decode and update PC 350 ps

 For fixed single-cycle implementation:

 Clock cycle = 880 ps determined by longest delay (load instruction)

 For multi-cycle implementation:

 Clock cycle = max (200, 150, 180) = 200 ps (maximum delay at any step)
 Average CPI = 0.4×4 + 0.2×5 + 0.1×4+ 0.2×3 + 0.1×2 = 3.8

 Speedup = 880 ps / (3.8 × 200 ps) = 880 / 760 = 1.16

Performance Example
We want to compare the performance of a single-cycle CPU design with a multicycle
CPU. Suppose we add the multiply and divide instructions. The operation times are as
follows:
Instruction memory access time = 190 ps,
Data memory access time = 190 ps
Register file read access time = 150 ps,
Register file write access = 150 ps
ALU delay for basic instructions = 190 ps,
ALU delay for multiply or divide = 550 ps

Ignore the other delays in the multiplexers, control unit, sign-extension, etc.

Assume the following instruction mix: 30% ALU, 15% multiply & divide, 20% load, 10%
store, 15% branch, and 10% jump.

a) What is the total delay for each instruction class and the clock cycle for the single cycle
CPU design.
b) Assume we fix the clock cycle to 200 ps for a multi-cycle CPU, what is the CPI for
each instruction class and the speedup over a fixed-length clock cycle?
Solution
a)

Clock cycle = max delay = 1040 ps.

Solution

b)
CPI for Basic ALU = 4 cycles
CPI for Multiply & Divide = 6 cycles
CPI for Load = 5 cycles
CPI for Store = 4 cycles
CPI for Branch = 3 cycles
CPI for Jump = 2 cycles

Average CPI = 0.3 * 4 + 0.15 * 6 + 0.2 * 5 + 0.1 * 4 + 0.15 * 3 + 0.1 * 2 = 4.55

Speedup of multi-cycle over single-cycle = (1040 * 1) / (200 * 4.55) = 1.1