Scribd
Upload
133 views

Sequential, Parallel and Distributed Algorithms

An algorithm is a sequence of steps that takes inputs and produces outputs. There are three main types of algorithms based on architecture: sequential, parallel, and distributed. [1] Sequential algorithms run steps consecutively, [2] parallel algorithms divide a problem into sub-problems that can be solved simultaneously, and [3] distributed algorithms use multiple interconnected processors each with their own memory.

Uploaded by

Namae Montejo Aniasco
Copyright
© © All Rights Reserved
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
133 views

Sequential, Parallel and Distributed Algorithms

An algorithm is a sequence of steps that takes inputs and produces outputs. There are three main types of algorithms based on architecture: sequential, parallel, and distributed. [1] Sequential algorithms run steps consecutively, [2] parallel algorithms divide a problem into sub-problems that can be solved simultaneously, and [3] distributed algorithms use multiple interconnected processors each with their own memory.

Uploaded by

Namae Montejo Aniasco
Copyright
© © All Rights Reserved
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 18

SEQUENTIAL,PARALLEL

AND DISTRIBUTED
ALGORITHMS

1
What is an Algorithm?
-An algorithm is a sequence of steps that take inputs
from the user and after some computation, produces an
output.
-An algorithm is a sequence of instructions followed
to solve a problem. While designing an algorithm, we
should consider the architecture of computer on which
the algorithm will be executed.

As per the architecture, there are two types of


computers:
Sequential Computer and Parallel
Computer
Depending on the architecture of computers, we have two types of
algorithms:
SEQUENTIAL ALGORITHM AND PARALLEL
ALGORITHM
Sequential Algorithm
-An algorithm in which some consecutive steps of
instructions are executed in a chronological order to
solve a problem.
- In computer science, a sequential algorithm or serial
algorithm is an algorithm that is executed sequentially
– once through, from start to finish, without other
processing executing – as opposed to concurrently or
in parallel.
"Sequential algorithm" may also refer specifically to
an algorithm for decoding a convolutional code.

Convolutional code is a type of error-correcting


code that generates parity symbols via the sliding
application of a boolean polynomial function to a data
stream. The sliding application represents the
'convolution' of the encoder over the data, which gives
rise to the term 'convolutional coding'.
Parallel Algorithm − the problem is divided into sub-
problems and are executed in parallel to get individual
outputs. Later on, these individual outputs are combined
together to get the final desired output.
-In computer science, a parallel algorithm, as opposed
to a traditional serial algorithm, is an algorithm which
can be executed a piece at a time on many different
processing devices, and then combined together again at
the end to get the correct result.

Note! It is not easy to divide a large problem into sub-


problems. Sub-problems may have data dependency among
them.Therefore,theprocessorshavetocommunicatewitheach
othertosolvetheproblem.
- It has been found that the time needed by the
processors in communicating with each other is more
than the actual processing time. So, while designing a
parallel algorithm, proper CPU utilization should be
considered to get an efficient algorithm.

To design an algorithm properly, we must have a


clear idea of the basic model of
computation in a parallel computer.
Model of Computation
Both sequential and parallel computers operate
on a set (stream) of instructions called algorithms.
These set of instructions (algorithm) instruct the
computer about what it has to do in each step.
Depending on the instruction stream and data
stream, computers can be classified into four
categories −
Four categories depending on the instruction
stream and data stream, computers:
• Single Instruction stream, Single Data stream
(SISD) computers
• Single Instruction stream, Multiple Data stream
(SIMD) computers
• Multiple Instruction stream, Single Data stream
(MISD) computers
• Multiple Instruction stream, Multiple Data stream
(MIMD) computers
SISD Computers (Single Instruction stream, Single
Data stream)
SISDcomputers contain one control unit, one processing
unit, and one memory unit.

In this type of computers, the processor receives asingle


stream of instructions from the control unit and
operates on a single stream of data from the memory
unit.
SIMD Computers (Single Instruction stream,
Multiple Data stream)
SIMD computers contain one control unit, multiple
processing units, and shared memor y or
interconnection network.
Here, one single control unit sends instructions to all
processing units. During computation, at each step, all
the processors receive a single set of instructions from
the control unit and operate on different set of data from
the memory unit.

MISD Computers (Multiple Instruction


stream, Single Data stream)
As the name suggests, MISD computers
contain multiple control units, multiple
processing units,and one common memory
unit.
MISD Computers

Here, each processor has its own control unit and they
share a common memory unit. All the processors get
instructions individually from their own control unit
and they operate on a single stream of data as per the
instructions they have received from their respective
control units. This processor operates simultaneously.
12
MIMD Computers (Multiple Instruction stream,
Multiple Data stream)
MIMD computers have multiple control units,
multiple processing units, and a shared memory or
interconnection network.

Here, each processor has its own control unit, local


memory unit, and arithmetic and logic unit.
They receive different sets of instructions from their
respective control units and operate on different sets
of data.

Note!Moststandardcomputeralgorithmsaresequentialalgorithms,
andnotspecificallyidentifiedassuch,assequentialnessis a
backgroundassumption.
OTHER ALGORITHM:

Distributed Algorithm
A distributed algorithm is an algorithm designed to
run on computer hardware constructed from
interconnected processors. Distributed algorithms are
used in many varied application areas of distributed
computing, such as telecommunications,scientific
computing, distributed information processing, and
real-time process control. Standard problems solved
by distributed algorithms include leader
election, consensus, distributed search, spanning
tree generation,mutual exclusion, and resource
allocation.
The choice of an appropriate distributed algorithm to
solve a given problem depends on both the
characteristics of the problem, and characteristics of
the system the algorithm will run on such as the type
and probability of processor or link failures, the kind of
inter-process communication that can be performed, and
the level of timing synchronization between separate
processes.
When to know if algorithm is parallel?
An algorithm is parallel if there are several
processes (tasks, threads, processors) working
on it at the same time. Often the tasks run in the
same address space, and can
communicate/reference results by others freely
(low cost).
When to know if algorithm is distributed?

An algorithm is distributed if it is parallel and


the tasks run on separate machines (separate
address spaces), one task has no direct access
to the work of the others. It has to request
needed data, or just wait until it is sent to it.

You might also like