+

William Stallings
Computer Organization
and Architecture
9th Edition
Chapter 3
A Top-Level View of Computer
Function and Interconnection
LEARNING OBJECTIVES

After studying this chapter, you should be able to:

❑ Understand the basic elements of an instruction
cycle and the role of interrupts.
❑ Describe the concept of interconnection within a
computer system.
❑ Understand the difference between synchronous
and asynchronous bus timing.
❑ Explain the need for multiple buses arranged in a
+ hierarchy.
❑ Assess the relative advantages of point-to-point
interconnection compared to bus interconnection.
❑ Present an overview of QPI.
❑ Present an overview of PCIe.
+
Computer Components

◼ Contemporary computer designs are based on concepts

developed by John von Neumann at the Institute for
Advanced Studies, Princeton

◼ Referred to as the von Neumann architecture and is based on

three key concepts:
◼ Data and instructions are stored in a single read-write memory
◼ The contents of this memory are addressable by location, without
regard to the type of data contained there
◼ Execution occurs in a sequential fashion (unless explicitly
modified) from one instruction to the next

◼ Hardwired program
◼ The result of the process of connecting the various components in
the desired configuration
+
Sequence of
Data arithmetic Results

Hardware
and logic
functions

and Software (a) Programming in hardware

Approaches
I nstruction I nstruction
codes interpreter

Control
signals

General-purpose
Data arithmetic Results
and logic
functions

(b) Programming in software

Software
• A sequence of codes or instructions Software
• Part of the hardware interprets each instruction and
generates control signals
• Provide a new sequence of codes for each new
program instead of rewiring the hardware
Major components:
• CPU I/O
• Instruction interpreter Components
• Module of general-purpose arithmetic and logic
functions
• I/O Components
+ • Input module
• Contains basic components for accepting data
and instructions and converting them into an
internal form of signals usable by the system
• Output module
• Means of reporting results
 Memory Memory buffer MEMORY
address register (MBR)
register (MAR) • Contains the data
• Specifies the to be written into
address in memory memory or
for the next read or receives the data
write read from memory

MAR

I/O address I/O buffer

register (I/OAR) register (I/OBR)
• Specifies a • Used for the
+ particular I/O exchange of data
device between an I/O
module and the
CPU MBR
 CPU M ain M emory
0
System 1
2
PC M AR Bus
I nstruction
I nstruction
I nstruction
Computer
IR M BR
Components:
Execution
I /O AR
Data Top Level
View
unit Data
I /O BR Data
Data

I /O M odule n–2
n–1

PC = Program counter
Buffers IR = I nstruction register
M AR = M emory address register
M BR = M emory buffer register
I /O AR = I nput/output address register
I /O BR = I nput/output buffer register

Figure 3.2 Computer Components: Top-Level View

+
Basic Instruction Cycle

Fetch Cycle Execute Cycle

Fetch Next Execute

START HALT
I nstruction I nstruction

Figure 3.3 Basic I nstruction Cycle

+
Fetch Cycle
◼ At the beginning of each instruction cycle the processor
fetches an instruction from memory

◼ The program counter (PC) holds the address of the

instruction to be fetched next

◼ The processor increments the PC after each instruction

fetch so that it will fetch the next instruction in sequence

◼ The fetched instruction is loaded into the instruction

register (IR)

◼ The processor interprets the instruction and performs the

required action
Action Categories

•Data transferred from •Data transferred to or

processor to memory from a peripheral
or from memory to device by transferring
processor between the processor
and an I/O module

Processor- Processor-
memory I/O

Data
Control
processing

•An instruction may •The processor may

specify that the perform some
sequence of execution arithmetic or logic
be altered operation on data
+ Characteristics of a
hypothetical machine
0 3 4 15
Opcode Address

(a) Instruction format

0 1 15
S Magnitude

(b) Integer format

Program Counter (PC) = Address of instruction

Instruction Register (IR) = Instruction being executed
Accumulator (AC) = Temporary storage

(c) Internal CPU registers

0001 = Load AC from Memory

0010 = Store AC to Memory
0101 = Add to AC from Memory

(d) Partial list of opcodes

+
Example
of
Program
Execution
+
Instruction Cycle State Diagram

I nstruction Operand Operand

fetch fetch store

M ultiple M ultiple
operands results

I nstruction I nstruction Operand Operand

Data
address operation address address
Operation
calculation decoding calculation calculation

Return for string

I nstruction complete, or vector data
fetch next instruction

Figure 3.6 I nstruction Cycle State Diagram

+
Classes of Interrupts

Program Generated by some condition that occurs as a result of an instruction

execution, such as arithmetic overflow, division by zero, attempt to
execute an illegal machine instruction, or reference outside a user's
allowed memory space.
Timer Generated by a timer within the processor. This allows the operating
system to perform certain functions on a regular basis.
I /O Generated by an I/O controller, to signal normal completion of an
operation, request service from t he processor, or to signal a variety of
error conditions.
Hardware failure Generated by a failure such as power failure or memory parity error.
 Program Flow Control
User I/O User I/O User I/O
Program Program Program Program Program Program

1 4 1 4 1 4

I/O I/O I/O

Command Command Command
WRITE WRITE WRITE
5
2a
END
2 2

Interrupt Interrupt
2b Handler Handler

WRITE WRITE 5 WRITE 5

END END
3a

3 3

3b

WRITE WRITE WRITE

(a) No interrupts (b) Interrupts; short I/O wait (c) Interrupts; long I/O wait

= interrupt occurs during course of execution of user program

Figure 3.7 Program Flow of Control Without and With I nterrupts

+
Transfer of Control via Interrupts
User Program Interrupt Handler

i
Interrupt
occurs here i+1

M
+
Instruction Cycle With Interrupts

Fetch Cycle Execute Cycle I nterrupt Cycle

I nterrupts
Disabled
Check for
Fetch Next Execute
START I nterrupt;
I nstruction I nstruction I nterrupts Process I nterrupt
Enabled

HALT
 Time

+ 1 1

4 4
I/O operation
I/O operation;
processor waits 2a concurrent with
processor executing

5 5

2b
2
4 Program
4 3a
I/O operation
concurrent with
processor executing
Timing:
Short I/O
I/O operation;
processor waits 5

5 3b
Wait
(b) With interrupts
3

(a) Without interrupts

Figure 3.10 Program Timing: Short I /O Wait

 Time

+ 1 1

4 4

I/O operation; 2 I/O operation

processor waits concurrent with
processor executing;
then processor
waits
5

2
5 Program
4 Timing:
Long I/O
4
3 I/O operation
concurrent with

Wait
I/O operation; processor executing;
processor waits then processor
waits

5
5

3 (b) With interrupts

(a) Without interrupts

Figure 3.11 Program Timing: Long I /O Wait

 Instruction Cycle State Diagram
With Interrupts

I nstruction Operand Operand

fetch fetch store

M ultiple M ultiple
operands results

I nstruction I nstruction Operand Operand

Data I nterrupt
address operation address address I nterrupt
Operation check
calculation decoding calculation calculation

No
I nstruction complete, Return for string interrupt
fetch next instruction or vector data

Figure 3.12 I nstruction Cycle State Diagram, With I nterrupts

 I nterrupt
User program handler X

Transfer of
I nterrupt
Control
handler Y

(a) Sequential interrupt processing

I nterrupt Multiple
Interrupts
User program handler X

I nterrupt
handler Y
+

(b) Nested interrupt processing

Figure 3.13 Transfer of Control with M ultiple I nterrupts

+ Time Sequence of E p
x l
Multiple Interrupts a e
User program
Printer Communication m
interrupt service routine interrupt service routine
t=0

15
0 t=
t =1

t = 25

t= t = 25 Disk
40 interrupt service routine

t=
35

Figure 3.14 Example Time Sequence of M ultiple I nterrupts

+
I/O Function
◼ I/O module can exchange data directly with the processor

◼ Processor can read data from or write data to an I/O module

◼ Processor identifies a specific device that is controlled by a
particular I/O module
◼ I/O instructions rather than memory referencing instructions

◼ In some cases it is desirable to allow I/O exchanges to occur

directly with memory
◼ The processor grants to an I/O module the authority to read from
or write to memory so that the I/O memory transfer can occur
without tying up the processor
◼ The I/O module issues read or write commands to memory
relieving the processor of responsibility for the exchange
◼ This operation is known as direct memory access (DMA)
+ Computer
Modules
The interconnection structure must support the
following types of transfers:

Memory Processor I/O to or

I/O to Processor
to to from
processor to I/O
processor memory memory

An I/O
module is
allowed to
exchange
data
Processor Processor
directly
reads an Processor reads data Processor
with
instruction writes a from an I/O sends data
memory
or a unit of unit of data device via to the I/O
without
data from to memory an I/O device
going
memory module
through the
processor
using direct
memory
access
A communication pathway Signals transmitted by any
connecting two or more one device are available for
devices reception by all other
• Key characteristic is that it is a devices attached to the bus
I
n
shared transmission medium • If two devices transmit during the
same time period their signals will
overlap and become garbled
n
e
Typically consists of multiple
t
communication lines Computer systems contain a
number of different buses B c
• Each line is capable of
transmitting signals representing
that provide pathways
between components at e
binary 1 and binary 0
various levels of the u t
r
computer system hierarchy

s i
c
System bus
o
o
• A bus that connects major The most common computer
computer components (processor,
memory, I/O) interconnection structures
are based on the use of one
or more system buses n
n
Data Bus
◼ Data lines that provide a path for moving data among system
modules

◼ May consist of 32, 64, 128, or more separate lines

◼ The number of lines is referred to as the width of the data bus

◼ The number of lines determines how many bits can be

transferred at a time

◼ The width of the data bus

is a key factor in
determining overall
system performance
+ Address Bus Control Bus

◼ Used to designate the source or ◼ Used to control the access and the
destination of the data on the use of the data and address lines
data bus
◼ If the processor wishes to ◼ Because the data and address lines
read a word of data from are shared by all components there
memory it puts the address of must be a means of controlling their
the desired word on the use
address lines
◼ Control signals transmit both
◼ Width determines the maximum command and timing information
possible memory capacity of the among system modules
system
◼ Timing signals indicate the validity
◼ Also used to address I/O ports of data and address information
◼ The higher order bits are
used to select a particular ◼ Command signals specify operations
module on the bus and the to be performed
lower order bits select a
memory location or I/O port
within the module
 Bus Interconnection Scheme

CPU Memory Memory I/O I/O

Control lines

Address lines Bus

Data lines

Figure 3.16 Bus Interconnection Scheme

 Local Bus Cache
Processor

M ain
M emory
Local I /O
controller C
System Bus
o a
Network Expansion

n t
bus interface Serial
SCSI
M odem

Expansion Bus
B
(a) Traditional Bus Architecture f i
M ain
u
i o
M emory

Cache

s
Local Bus System Bus
Processor /Bridge

SCSI FireWire Graphic Video LAN

g n
High-Speed Bus

FAX Expansion
bus interface Serial
u s
r
M odem

Expansion Bus

(b) High-Performance Architecture

+ Elements of Bus Design
Type Bus Width
Dedicated Address
Multiplexed Data
M ethod of Arbitration Data Transfer Type
Centralized Read
Distributed Write
Timing Read-modify-write
Synchronous Read-after-write
Asynchronous Block
 T1 T2 T3

Clock Timing of
Status
lines
Status signals Synchronous
Address
lines
Stable address Bus
Address
enable
Operations
Data
Valid data in
Read lines
cycle
Read

Data
Valid data out
Write lines
cycle
Write

Figure 3.18 Timing of Synchronous Bus Operations

 Status
Status signals
lines

Address
lines Stable address

Read
Timing of
Asynchronous
Data
lines Valid data

Acknowledge

(a) System bus r ead cycle

Bus
Operations
Status
Status signals
lines

Address
lines Stable address

Data
lines Valid data

Write

Acknowledge

(b) System bus write cycle

+
Point-to-Point Interconnect

Principal reason for change At higher and higher data

was the electrical rates it becomes
constraints encountered increasingly difficult to
with increasing the perform the synchronization
frequency of wide and arbitration functions in a
synchronous buses timely fashion

A conventional shared bus

on the same chip magnified
Has lower latency, higher
the difficulties of increasing
data rate, and better
bus data rate and reducing
scalability
bus latency to keep up with
the processors
+Quick Path Interconnect
QPI
◼ Introduced in 2008

◼ Multiple direct connections

◼ Direct pairwise connections to other components
eliminating the need for arbitration found in shared
transmission systems

◼ Layered protocol architecture

◼ These processor level interconnects use a layered

protocol architecture rather than the simple use of
control signals found in shared bus arrangements

◼ Packetized data transfer

◼ Data are sent as a sequence of packets each of which

includes control headers and error control codes
I /O device

I /O device
I /O Hub

Multicore
Configuration
DRAM

DRAM
Core Core
A B
Using
QPI
DRAM

DRAM
Core Core
C D
I /O device

I /O device
I /O Hub

QPI PCI Express M emory bus

 QPI Layers

Packets
Protocol Protocol

Routing Routing

Flits
Link Link

Physical Phits Physical

Figu r e 3 .2 1 QPI La ye r s
+
Physical Interface of the Intel QPI
Interconnect
COM PONENT A
I ntel QuickPath I nter connect Port
Fwd Clk

Rcv Clk
Transmission Lanes Reception Lanes

Fwd Clk
Rcv Clk

Reception Lanes Transmission Lanes

I ntel QuickPath I nter connect Port

COM PONENT B

Figu r e 3 .2 2 Ph ysica l I n t e r fa ce of t h e I n t e l QPI I n t e r con n e ct

+
QPI Multilane Distribution

#2n+1 #n+1 #1 QPI

lane 0

bit stream of flits #2n+2 #n+2 #2 QPI

lane 1

#2n+1 #2n #n+2 #n+1 #n #2 #1

#3n #2n #n QPI

lane 19

Figure 3.23 QPI M ultilane Distribution

+
QPI Link Layer

◼ Flow control function

◼ Performs two key ◼ Needed to ensure that a
functions: flow control and sending QPI entity does not
error control overwhelm a receiving QPI
entity by sending data faster
◼ Operate on the level of than the receiver can process
the flit (flow control the data and clear buffers for
unit) more incoming data
◼ Each flit consists of a 72-
bit message payload
◼ Error control function
and an 8-bit error
control code called a ◼ Detects and recovers from
cyclic redundancy check bit errors, and so isolates
(CRC) higher layers from
experiencing bit errors
+
QPI Routing and Protocol Layers

Routing Layer Protocol Layer

◼ Packet is defined as the unit of
◼ Used to determine the course transfer
that a packet will traverse
across the available system ◼ One key function performed at
interconnects this level is a cache coherency
protocol which deals with
◼ Defined by firmware and making sure that main
describe the possible paths memory values held in
that a packet can follow multiple caches are consistent

◼ A typical data packet payload

is a block of data being sent to
or from a cache
+
Peripheral Component
Interconnect (PCI)
◼ A popular high bandwidth, processor independent bus that can
function as a mezzanine or peripheral bus

◼ Delivers better system performance for high speed I/O

subsystems

◼ PCI Special Interest Group (SIG)

◼ Created to develop further and maintain the compatibility of the PCI
specifications

◼ PCI Express (PCIe)

◼ Point-to-point interconnect scheme intended to replace bus-based
schemes such as PCI
◼ Key requirement is high capacity to support the needs of higher data rate
I/O devices, such as Gigabit Ethernet
◼ Another requirement deals with the need to support time dependent data
streams
+ Core Core

PCIe Gigabit
Ethernet
PCI e
M emory

Configuration
Chipset
PCI e–PCI PCI e
M emory
Bridge

PCI e

PCI e PCI e
Switch

PCI e PCI e

Legacy PCI e PCI e PCI e

endpoint endpoint endpoint endpoint

Typical configuration that

supports the use of PCIe
Figu r e 3 .2 4 Typica l Con figu r a t ion Usin g PCI e
+
PCIe Protocol Layers

Transaction layer
packets (TLP)
Transaction Transaction

Data link layer

packets (DLLP)
Data Link Data Link

Physical Physical

Figu r e 3 .2 5 PCI e Pr ot ocol La ye r s

+
PCIe Multilane Distribution

B4 B0 128b/ PCI e
130b lane 0
byte stream

B5 B1 128b/ PCI e
130b lane 1
B7 B6 B5 B4 B3 B2 B1 B0

B6 B2 128b/ PCI e
130b lane 2

B7 B3 128b/ PCI e
130b lane 3

Figure 3.26 PCI e M ultilane Distribution

 D+ D–
8b

Differential
Scrambler Receiver

8b 1b Clock recovery
circuit

128b/130b Encoding
Data recovery
circuit
PCIe
130b 1b
Transmit
Parallel to serial Serial to parallel
and
1b 130b
Receive
Block
Transmitter Differential
128b/130b Decoding
Driver

Diagrams
128b

D+ D–
Descrambler
(a) Transmitter
8b

(b) Receiver
 Receives read and write requests from
+ ◼
the software above the TL and creates
request packets for transmission to a
destination via the link layer

PCIe ◼ Most transactions use a split transaction

technique
Transaction Layer (TL) ◼ A request packet is sent out by a
source PCIe device which then waits
for a response called a completion
packet
◼ TL messages and some write
transactions are posted transactions
(meaning that no response is
expected)

◼ TL packet format supports 32-bit

memory addressing and extended
64-bit memory addressing
+
The TL supports four address
spaces:
◼ Memory ◼ I/O
◼ The memory space includes ◼ This address space is used
system main memory and
PCIe I/O devices
for legacy PCI devices, with
reserved address ranges
◼ Certain ranges of memory
addresses map into I/O used to address legacy I/O
devices devices

◼ Configuration ◼ Message
◼ This address space enables ◼ This address space is for
the TL to read/write control signals related to
configuration registers interrupts, error handling,
associated with I/O devices and power management
 PCIe TLP Transaction Types
Address Space TLP Type Purpose
Memory Read Request
Transfer data to or from a location in the
M emory Memory Read Lock Request system memory map.
Memory Write Request
I/O Read Request Transfer data to or from a location in the
I /O
I/O Write Request system memory map for legacy devices.
Config Type 0 Read Request
Config Type 0 Write Request Transfer data to or from a location in the
Configuration
Config Type 1 Read Request configuration space of a PCIe device.
Config Type 1 Write Request
Message Request Provides in-band messaging and event
M essage reporting.
Message Request with Data
Completion
M emory, I /O, Completion with Data
Returned for certain requests.
Configuration Completion Locked
Completion Locked with Data
 Number

+
of octets
1 STP framing 1 Start

Appended by PL
2 Sequence number
DLLP

Created
by DLL
4

2 CRC

12 or 16 Header 1 End

Created by Transaction Layer

Appended by Data Link Layer

PCIe
Appended by Physical Layer
Protocol
Data
0 to 4096 Data

Unit
Format
0 or 4 ECRC

4 LCRC

1 STP framing

(a) Transaction Layer Packet (b) Data Link Layer Packet

Figu r e 3 .2 8 PCI e Pr ot ocol D a t a Un it For m a t

+
TLP Memory Request Format
32 bits

Traffic T E
R Fmt Type R R Attr R Length
Class E P
Last First
16 octets

Requestor I D Tag
DW BE DW BE

Address [63:32]

Address [31:2] R

Figure 3.29 TLP M emory Request Format

+ Summary A Top-Level View of
Computer Function
and Interconnection
Chapter 3
◼ Point-to-point interconnect
◼ Computer components
◼ QPI physical layer
◼ Computer function
◼ QPI link layer
◼ Instruction fetch and
execute ◼ QPI routing layer
◼ Interrupts ◼ QPI protocol layer
◼ I/O function ◼ PCI express
◼ Interconnection structures ◼ PCI physical and logical
◼ Bus interconnection architecture
◼ Bus structure ◼ PCIe physical layer
◼ Multiple bus hierarchies ◼ PCIe transaction layer
◼ Elements of bus design ◼ PCIe data link layer