CH – 1 : Introduc on:

The Demand for Computa onal Speed:

a. Increasing Complexity:
 As computa onal tasks become more complex, tradi onal sequen al compu ng
methods o en become inadequate for achieving mely results.
 Tasks such as weather modeling, fluid dynamics simula ons, and gene c analysis
require vast amounts of computa onal power to process data and generate accurate
predic ons.
b. Real- me Processing:
 Many modern applica ons require real- me processing to meet user demands and
ensure mely decision-making.
 Examples include online transac on processing systems, real- me analy cs for
financial markets, and control systems for autonomous vehicles.
c. Big Data:
 The prolifera on of data in today's digital world necessitates efficient methods for
processing and analyzing large datasets.
 Tasks such as data mining, machine learning, and predic ve analy cs rely on parallel
compu ng to handle the volume, velocity, and variety of big data.
d. Simula on and Modeling:
 Various scien fic and engineering disciplines rely on simula ons and modeling to
understand complex phenomena and op mize designs.
 Examples include climate modeling, computa onal fluid dynamics, structural analysis,
and molecular simula ons.
 Parallel compu ng enables faster execu on of simula ons by distribu ng
computa onal tasks across mul ple processors or nodes.
Parallel Computers:
Parallel computers are compu ng systems designed to execute mul ple instruc ons or tasks
simultaneously, thereby improving computa onal performance by exploi ng parallelism.
These systems consist of mul ple processing units that work together to solve a problem
more quickly than a single processor could achieve alone. Here are the key points about
parallel computers:

Made By Rohit Makwana

Types of Parallel Computers:
a. Shared Memory Systems:
 In shared memory systems, mul ple processors share a single, global address space.
 Processors can access any memory loca on directly, simplifying programming but
poten ally leading to conten on and synchroniza on issues.
 Examples include mul -core processors found in personal computers and symmetric
mul processing (SMP) servers.
b. Distributed Memory Systems:
 Distributed memory systems consist of mul ple independent processors, each with
its own local memory.
 Communica on between processors is achieved through message passing, where
data is explicitly sent and received between processes.
 Examples include clusters of worksta ons, where each worksta on acts as a node
connected via a network, and massively parallel processing (MPP) systems used in
supercomputers.
c. Hybrid Systems:
 Hybrid systems combine features of both shared memory and distributed memory
architectures to achieve scalability and performance.
 Clusters of symmetric mul processing (SMP) nodes interconnected via a high-speed
network represent a common hybrid architecture.
 Such systems offer the benefits of shared memory within each node while leveraging
distributed memory communica on between nodes.
3. Cluster Compu ng:
Cluster compu ng is a computa onal model that involves connec ng mul ple independent
computers (referred to as nodes or hosts) via a network to work together as a single, unified
system. In a cluster compu ng environment, each node typically runs its own opera ng
system and possesses its own processing power, memory, and storage resources. These
nodes communicate with each other over a network, sharing data and coordina ng tasks to
achieve parallel processing and improve overall computa onal performance.
Here are the key points about cluster compu ng:
a. Node Interconnec on:
 Clusters consist of mul ple compu ng nodes interconnected via a network
infrastructure.
Made By Rohit Makwana
  Networks can vary in topology and bandwidth, ranging from tradi onal Ethernet
networks to high-speed interconnects like InfiniBand.
b. Parallel Processing:
 Tasks are divided into smaller subtasks and distributed among the nodes in a cluster
for concurrent execu on.
 Parallel processing enables faster computa on by leveraging the collec ve processing
power of mul ple nodes.
c. Load Balancing:
 Efficient load balancing ensures that computa onal tasks are evenly distributed
among cluster nodes to maximize resource u liza on and minimize execu on me.
 Load balancing algorithms dynamically adjust task assignments based on node
performance and workload characteris cs.
d. Scalability:
 Clusters offer scalability, allowing organiza ons to increase compu ng capacity by
adding more nodes to the cluster as computa onal demands grow.
 Scalability is essen al for handling varying workloads and accommoda ng future
growth without significant architectural changes.
e. Fault Tolerance:
 Cluster compu ng systems o en incorporate mechanisms for fault tolerance to
ensure con nuous opera on in the presence of hardware failures or network issues.
 Techniques such as redundancy, checkpoin ng, and job replica on help mi gate the
impact of failures on overall system performance and reliability.
Advantages:
1. Scalability: Cluster compu ng offers scalability, allowing organiza ons to increase
compu ng capacity by adding more nodes to the cluster. This scalability makes it
suitable for handling varying workloads and accommoda ng growth.
2. Cost-Effec veness: Clusters can be built using commodity hardware, which tends to
be more cost-effec ve compared to specialized supercomputers. This makes cluster
compu ng accessible to a wider range of organiza ons with budget constraints.
3. High Performance: By leveraging parallel processing across mul ple nodes, cluster
compu ng can achieve high performance and improve computa onal throughput,
making it suitable for demanding computa onal tasks.

Made By Rohit Makwana

 4. Fault Tolerance: Clusters typically incorporate fault-tolerance mechanisms, such as
redundant hardware configura ons and job replica on, to ensure con nuous
opera on in the event of node failures.
5. Flexibility: Cluster compu ng offers flexibility in terms of hardware and so ware
configura ons. Organiza ons can customize clusters to meet specific requirements
and adapt them to different types of workloads.
Disadvantages:
1. Complexity: Se ng up and managing a cluster compu ng environment can be
complex, requiring exper se in networking, system administra on, and parallel
programming. Configura on, maintenance, and troubleshoo ng may pose
challenges.
2. Communica on Overhead: Communica on between nodes in a cluster introduces
overhead, including latency and network bandwidth limita ons. Excessive
communica on overhead can impact overall system performance.
3. Resource Conten ons: In shared resource environments, mul ple nodes may
contend for shared resources such as network bandwidth, storage, and memory,
leading to poten al performance bo lenecks and resource conten on issues.
4. Scalability Limits: While clusters offer scalability, there may be prac cal limits to the
number of nodes that can be added to a cluster due to factors such as network
conges on, management complexity, and diminishing returns in performance gains.
5. Single Point of Failure: Despite fault-tolerance mechanisms, clusters may s ll have
single points of failure, such as network switches or master nodes, which can affect
overall system reliability if they fail.

What is Distributed Compu ng explain with their advantages and disadvantages :

Distributed Compu ng:
Distributed compu ng refers to a computa onal model in which tasks are divided among
mul ple computers (nodes) connected via a network. Each node operates independently
and collaborates with other nodes to achieve a common goal. Here are the key aspects,
advantages, and disadvantages:
Concept:
 Decentralized Processing: Unlike tradi onal centralized compu ng, distributed
compu ng distributes processing tasks across mul ple nodes, allowing them to work
concurrently.

Made By Rohit Makwana

  Inter-Node Communica on: Nodes communicate with each other through message
passing or remote procedure calls, exchanging data and coordina ng their ac ons.
 Scalability: Distributed compu ng systems can scale horizontally by adding more
nodes to the network, enabling them to handle larger workloads and accommodate
growth.
Advantages:
1. Scalability: Distributed compu ng systems can scale easily by adding more nodes,
allowing them to handle increasing computa onal demands efficiently.
2. Fault Tolerance: The decentralized nature of distributed compu ng provides
resilience against node failures. If one node fails, the system can con nue to operate
using other nodes, ensuring high availability.
3. Resource U liza on: Distributed compu ng enables be er resource u liza on by
distribu ng tasks across mul ple nodes, reducing the burden on individual resources
and improving overall efficiency.
4. Geographical Distribu on: Distributed compu ng allows resources to be
geographically distributed, enabling collabora on and data sharing across different
loca ons.
5. Flexibility: Distributed compu ng systems are flexible and adaptable, as they can be
configured to meet specific requirements and accommodate diverse workloads.
Disadvantages:
1. Complexity: Developing and managing distributed compu ng systems can be
complex, requiring exper se in network programming, concurrency, and fault
tolerance mechanisms.
2. Communica on Overhead: Communica on between nodes can introduce overhead,
including latency and network bandwidth limita ons, which may affect overall system
performance.
3. Consistency and Coordina on: Ensuring data consistency and coordina ng ac ons
across distributed nodes can be challenging, especially in distributed databases or
transac onal systems.
4. Security Risks: Distributed compu ng introduces addi onal security risks, such as
unauthorized access, data breaches, and network vulnerabili es, which must be
addressed through robust security measures.
5. Cost: Se ng up and maintaining distributed compu ng infrastructure may involve
higher costs compared to centralized systems, including hardware, networking, and
administra on expenses.
Made By Rohit Makwana
Made By Rohit Makwana