Distributed Computer System

Lecture 5

Classification of Distributed Systems

(1- Share memory:” according to Connection.”)

What is the Dynamic Interconnection Systems?

• Can provide a connection according to the need of the application.
• Have switch elements (multiplexers).
• Access to channel is switching, so the same channel can be used to connect
multi-nodes.

• It has many types, such as:

[1] Buses-based interconnection network.

a) Single bus system
b) Multiple bus system

[2] Switch-based interconnection network

a) Single stage network “ex: Cube, and plus–Minus2i (PM2I)”.
b) Multi-stage network “ex: baseline and omega”.
c) Crossbar network.

In this section, we introduce another taxonomy for interconnection

networks (INs). In general, interconnection networks can be classified as static
versus dynamic networks. In static networks, direct fixed links are established
among nodes to form a fixed network, while in dynamic networks, connections
are established as needed. Switching elements are used to establish connections
among inputs and outputs. Depending on the switch settings, different
interconnections can be established. Shared memory systems can be designed
using bus-based or switch-based dynamic INs.
Static networks form all connections when the system is designed rather than
when the connection is needed. In a static network, messages must be routed
along with established links.
Dynamic INs establishes a connection between two or more nodes on the fly
as messages are routed along the links. The number of hops in a path from
Dr. Ameer Mosa Al-sadi 1|Page
 source to the destination node is equal to the number of point-to-point links a
message must traverse to reach its destination. In either static or dynamic
networks, a single message may have to hop through intermediate processors on
its way to its destination.

Basic properties of the dynamic system

• Timing:
1. Synchronous (with clock).
2. Asynchronous (handshake, interlocking).

• switching:
1. Circuit (path is allocating for each transmission), (low latency).
2. Packet (data are encapsulating into the packet, transfer independence).

• Control:
1. Centralization.
2. Distributed.

Classification of Distributed Systems

(1- Share memory:” according to Connection.”)

[1] Buses-based interconnection network

1. A passive system is connected to transfer data between communicated
nodes. “They are not responsible for moving the signal along.
2. Buses use timing multiplex; transfer media is sharing.
Carrier Sense Multiple Access/ Collision Detection (CSMA/CD).
3. In given time communication only two nodes (single bus).
4. Low cost, simple realization.
5. Low transmission, bad salability.
6. Weak (never) stability toward fail.

The simplest IN for shared memory systems is the bus. However, the bus may get
saturated if multiple processors are trying to access the shared memory (via the
bus) simultaneously. A typical bus-based design uses caches to solve the bus
contention problem. Other shared memory designs rely on switches for
interconnection.

Dr. Ameer Mosa Al-sadi 2|Page

 [1-a] Single bus system
A single bus is considered the simplest way to connect multiprocessor, systems by
Single bus system.
In its general form, such a system consists of N processors, each having its cache,
connected by a shared bus. The use of local caches reduces processor memory
traffic. All processors communicate with a single shared memory.
The single bus network complexity, measured in terms of the number of buses
used, is O(1), while the time complexity, measured in terms of the amount of input
to output delay is O(N).

1. Low transmission, bad salability “the delay complexity is O(N)”.

2. Its computational complexity is O(1).

[1-b] Multiple bus System

The use of multiple buses to connect multiple processors is a natural
extension to the single shared bus system. A multiple bus multiprocessor system
uses several parallel buses to interconnect multiple processors and multiple
memory modules.

1. The delay complexity is O(mN).

2. Its computational complexity is O(m).

(m) represents the number of multiple buses used, while (N) represents the
number of processors (memory modules) or input/output of the network.

Dr. Ameer Mosa Al-sadi 3|Page

The above three Figures show multiple buses with:
(a) full bus–memory connection;
(b) single bus-memory connection;
(c) partial bus–memory connection.
Read more in ADVANCED COMPUTER ARCHITECTUREAND
PARALLEL PROCESSING ”2.2.2 Multiple Bus Systems.”

Dr. Ameer Mosa Al-sadi 4|Page

 [2] Switch-based interconnection network
In this type of network, connections among processors and memory modules are
made using simple switches. Three basic interconnection topologies are: single-
stage, multistage and crossbar.

The simplest switching element that can be used is the 2x2 switching element (SE).
The Figure below illustrates the four possible settings that a SE can assume. These
settings are called straight, exchange, upper-broadcast, and lower-broadcast. In the
straight setting, the upper input is transferred to the upper output, and the lower
input is transferred to the lower output. In the exchange setting the upper input is
transferred to the lower output and the lower input is transferred to the upper
output. In the upper-broadcast setting, the upper input is broadcast to both the
upper and the lower outputs. In the lower-broadcast, the lower input is broadcast to
both the upper and the lower outputs.

 Switch model:
1. Have (a) input and (b) output.

2. In praxis used (a=b=𝟐𝒌 ),

where k= count control signal.
3. Examples: Baseline, Omega.

[2-a] Multi stage linking network

 MINs provide a number of simultaneous paths between the processors and
the memory modules.
 IN general, MIN consists of a number of stages each consisting of a set of
2×2 switching elements.
 Stages are connected using Inter-Stage Connection (ISC) Pattern.
 These patterns may follow any of the routings in MINs.
 The routing of a message from a given source to a given destination is based
on the destination address (self-routing).
 The delay complexity is O(log N), because the MIN requires log N clocks to
make a connection
 Its computational complexity is O(N log N).
 Blocking network.
 Centralize control.

Dr. Ameer Mosa Al-sadi 5|Page

 MIN general structure as below:

1. MINs provide a number of simultaneous paths between the processors and

the memory modules.
2. IN general, MIN consists of a number of stages each consisting of a set of
2×2 switching elements.
3. Stages are connected using Inter-Stage Connection (ISC) Pattern.
4. These patterns may follow any of the routings in MINs.
5. The routing of a message from a given source to a given destination is based
on the destination address (self-routing).
6. The delay complexity is O(log N), because the MIN requires log N clocks to
make a connection
7. Its computational complexity is O(m).
8. Blocking network.
9. Centralize control.

Multistage interconnection networks (MINs) were introduced as a means to

improve some of the limitations of the single bus system while keeping the cost
within an affordable limit.
The most undesirable single bus limitation that MINs is set to improve is the
availability of only one single path between the processors and the memory
modules. Such MINs provide a number of simultaneous paths between the
processors and the memory modules.
A general MIN consists of a number of stages each consisting of a set of 2×2
switching elements. Stages are connected to each other using Inter-Stage
Connection (ISC) Pattern. These patterns may follow any of the routings in MINs,
the routing of a message from a given source to a given destination is based on the
destination address (self-routing). And it is blocking network.

Dr. Ameer Mosa Al-sadi 6|Page

 Baseline model
0 0
1 1
2 M

M
2 2
3
(M/2)-2
(M/2)-1
M×M
M/2
(M/2)+1
M M

2 2
M-2 M-2
M-1 M-1

Read more: https://hal.inria.fr/file/index/docid/149527/filename/RR.pdf

Omega connection
• To connect N processing element to N memory, Omega network requires
(N/2) switches at each stage, and 𝒍𝒐𝒈𝟐 𝑵 stages
• So it has [ (N/2) 𝒍𝒐𝒈𝟐 𝑵 ] components.
• Omega MIN requires (𝐥𝐨𝐠 𝑵) clocks to make a connection.

The omega is a simple dynamic network that connects each of the inputs on the left side
to some, but not all, outputs on the right side through a single layer of binary switches
represented by the rectangles. The binary switches can direct the message on the left-side input
to one of two possible outputs on the right side. If we cascade enough single-stage networks
together, they form a completely connected multistage interconnection network (MIN), the
omega MIN connects eight sources to eight destinations.

Dr. Ameer Mosa Al-sadi 7|Page

These are dynamic INs because the connection is made on the fly, as needed. In
order to connect a source to a destination, we simply use a function of the bits of
the source and destination addresses as instructions for dynamically selecting a
path through the switches.
For N processing element, an Omega network contains N/2 switches at each stage,
and 𝑙𝑜𝑔2 𝑁 stages. The omega MIN, on the other hand, connects N × N pairs with
N/2 (log 2 N) components. The omega MIN, on the other hand, requires log N
clocks to make a connection. The diameter of the omega MIN is, therefore, log N.

[2-b] Crossbar network

1. Single Network with N² unary switch.
2. Switch element must solve conflict for request a bout current access.
3. Have possibility wire are complex and expensive.
4. Realization is only for a small system.
5. Cross multiplexer provides better transmission properties.
6. The crossbar contains a switching element (SE) at the intersection of any
two lines extended horizontally or vertically inside the switch.
7. The crossbar can provide simultaneous connections among all its inputs and
all its outputs.
8. N² components are needed to connect N × N pairs.
9. Major advantage, it needs one clock to make a connection between source
and destination.
10.Delay complexity, is O(1).
11.Network complexity O(𝑵𝟐 ).
12.Non-blocking network.
13.Centralize control strategy.

Dr. Ameer Mosa Al-sadi 8|Page

 A crossbar network represents the other extreme to the limited single bus
network. While the single bus can provide only a single connection, the crossbar
can provide simultaneous connections among all its inputs and all its outputs. The
crossbar contains a switching element (SE) at the intersection of any two lines
extended horizontally or vertically inside the switch.
The crossbar switch clearly uses more binary switching components; for example,
𝑁 2 omponents are needed to connect N × N source/destination pairs.
The major advantage of the crossbar switch is, its potential for speed. In one clock,
a connection can be made between source and destination. The diameter of the
crossbar is one.

Blocking vs. non-blocking

Both multistage networks (” baseline and omega”,) limit the number of alternate
paths between any source/destination pair. This leads to limited fault tolerance
and network traffic congestion. If the single path between pairs becomes faulty,
that pair cannot communicate. If two pairs attempt to communicate at the same
time along a shared path, one pair must wait for the other. This is called blocking,
and such MINs are called blocking networks. While a network that can handle all
possible connections without blocking like a cross switch network is called a
non-blocking network.

Problem
 Hot-spot:

1. Generated addresses are not equally distributed.

2. Multiple nodes want to communicate through small group switch.

 Dead-lock:

1. Routing unknown “unknown where we have to send the message”.

2. Or conflict on the switch in the network.

 Live-Lock:

1. Continue routing unknown (still conflict).

Dr. Ameer Mosa Al-sadi 9|Page