Here’s a seatwork activity for the topic "Introduction to Parallel and Distributed Computing" that

encourages students to understand key concepts and apply them to practical examples.

Seatwork: Introduction to Parallel and Distributed Computing

Part 1: Definitions and Key Concepts
1.​ Define the following terms:

o​ Parallel Computing - Parallel Computing operates as a computer system which

splits workload across multiple processors to execute tasks simultaneously. The
technique consists of splitting problems into smaller parts for simultaneous
solving before uniting their outputs.
o​ Distributed Computing - A distributed computing solution exists as a framework
which unites multiple computing nodes to collaborate on shared processing
duties in order to reach common objectives. Such systems join networks to
process tasks while having dispersed locations throughout the system.
o​ Scalability - A system or network shows scalability through its efficient workload
handling capability by adding resources such as hardware devices or processors
among other resources. Technology expansion occurs through two methods:
horizontal addition of new machines or vertical updates of existing devices.
o​ Load Balancing - Load Balancing functions through workload distribution
methods that prevent resources from facing unbearable quantities. This system
improvement method increases both system performance and resource
utilization while simultaneously strengthening overall system reliability.
o​ Concurrency - A system maintains concurrency when it executes multiple tasks
simultaneously and these processes run part of their operations during
overlapping time frames. Multiple tasks proceed independently of each other
even though they may not execute simultaneously.
o​ Synchronization - Synchronization functions as a method in concurrent
computing to synchronize executed processes and threads so they run in a
coordinated temporal order without data conflicts.
o​ Fault Tolerance - The system maintains correct operation even after suffering
component failures because of its fault tolerance capability. The combination of
redundancy with error detection systems and recovery features allows this
accomplishment.
o​ Grid Computing - The distributed computing system known as Grid Computing
unites computers located in different locations across the globe to work through
extensive computational challenges shared between them.
o​ Cloud Computing - The internet-based computing model gives organizations
access to servers and storage and applications through a flexible payment
 system. Users gain access to scalable and managed IT resources through an
on-demand system.
2.​ Explain the difference between Parallel Computing and Distributed Computing. How do
these two types of computing systems differ in terms of architecture and application?

The present computing age demands both high operational efficiency as well as fast
performance solutions for managing extensive issues. The challenges in complexity and
total workload transcension find solutions through parallel computing together with
distributed computing methods. These distinct procedures focus on performance
enhancement yet operate through different systems and executing functions. It is vital to
recognize these contrasting features because they enable effective selection of
appropriate computing solutions for unique requirements.

Architectural Differences
By nature parallel computing requires different processors or cores inside one system to
share responsibilities when running simultaneous computing tasks. Each processor
under this method can access shared memory to process data at high speed.
High-speed memory buses serve as the medium which allows processors to
communicate during coordinated operations. All processors interconnecting in one
system need tight synchronization to guarantee execution consistency.
Multiple independent machines known as nodes united through networking systems
participate jointly to achieve a single goal in distributed computing systems. The system
comprises different machines that keep their own processing power and memory while
using TCP/IP networks to communicate. Tasks under distributed computing operate
independently because its loosely coupled structure eliminates the need for constant
synchronization between nodes. Distributed computing functions better due to its
designed architecture that combines scalability and fault tolerance because node failures
affect only limited system portions.
Application Differences
Issues which need immediate computer speed and regular task communication have
optimal resolution through parallel computing methods. The fields of scientific
simulations along with machine learning model training and graphics rendering achieve
maximum performance benefits through parallel processing because they need to
execute complicated mathematical computations at the same time. Supercomputers in
HPC environments depend on parallel processing to boost their ability for quick
problem-solving in meteorological forecasting and molecular molecular structure
exercises.
Distributed computing better handles systems which operate at large scales together
with multiple users present for simultaneous service or manage extensive data volumes.
The internet-based scalable services provided by Amazon Web Services (AWS) and
Google Cloud are powered by distributed computing technology

Part 2: Case Study Application

Scenario:​
A research institution is working on a large data analysis project involving millions of records.
They need to process and analyze the data in a shorter time frame. They have two possible
solutions:

●​ Using a Parallel Computing system with multiple cores on a single machine.

●​ Using a Distributed Computing system with multiple machines connected over a
network.

Questions:
1.​ What are the advantages and disadvantages of using a Parallel Computing system
versus a Distributed Computing system for this data analysis project?

According to my research, parallel computing achieves fast performance through

system utilization of multiple cores inside a single machine. The necessary executions
run faster when network communication is not required thus increasing operational
speed and efficiency in latency reduction. The management and implementation
processes for parallel computing systems become simpler because everything operates
from a single computing platform. The task management across several machines
becomes less complex because of this approach. Parallel computing faces its main
limitation through physical restrictions that single machine hardware imposes. The
hardware capabilities of a processor reach their full potential when the system reaches
its core size and memory capacity limit. The database expansion eventually reaches a
limit which creates processing challenges since users need to upgrade to high-end
equipment to follow data expansion. Among the important issues in parallel computing is
the requirement to ensure fault tolerance. Failure of the single system will result in full
computation loss which requires the analysis to restart completely.
2.​ Based on the scenario, which approach (parallel or distributed) would be more suitable
for this task? Justify your choice with at least two reasons.

The research institution requires swift and efficient processing of millions of records so
distributed computing provides the best solution. One key reason is scalability. A parallel
computing system using one machine has pre-determined processing limits that its
hardware capabilities enforce. The institution can improve its computing power through
distributed computing by easily adding new machines which makes it a more
dependable solution for processing large datasets.
The reliability performance along with fault tolerance capacities represents a
fundamental decision factor. The duration of extensive data analysis projects extends
over significant time periods without exception because software failures occur inevitable
during execution. The total restart requirement of parallel computing analysis following
machine failure results in wasted time together with expended resources. The distributed
computer system operates when specific machines fail because it provides superior
reliability for data processing operations. A research institution depends on this
resilience because it requires both unbroken analysis performance and secure data
integrity.
 3.​ Describe the role of scalability in this scenario. How would scalability impact the
performance of each system?

The long-term operational effectiveness of the selected system mainly depends on its
scalability characteristics. A research institution that implements parallel computing will
deal with data size limitations because of its setup design. The machine's processing
ability together with memory capacity restricts further operation so institutions must
purchase additional expensive equipment that may become uneconomical to continue
running the system.
A distributed computing system demonstrates dynamic scalability by allowing
additional machine installation to meet growing requirements. The institution can
establish a basic setup then grow its computational power step by step while its data
grows larger. Long-term flexibility combined with cost effectiveness arises from
horizontal scaling which involves adding new machines instead of extensive vertical
upgrades of existing machines. The system provided by distributed computing allows
institutions to maintain flexibility through scalable growth because it adapts to resource
demands without requiring substantial infrastructure adjustments.
Part 3: Practical Application
1.​ Parallel Computing Example:​
Imagine you are tasked with writing a program that performs matrix multiplication on a
large matrix. How would you parallelize the matrix multiplication process? Describe the
steps involved and how the task can be divided among multiple processors.

Matrix multiplication stands as the primary operation in computing for me because its
use extends into scientific research along with machine learning and graphics
processing. The length of computations increases with rising matrix dimensions. The
distribution of workload across multiple processors by parallel computing presents itself
as the optimal solution to deal with this matter because it leads to better performance
alongside reduced computational time.

I believe row-wise distribution stands as the most powerful approach to parallelize matrix
multiplication because it allocates distinct rows of 𝐴 matrix to each processor and performs 𝐶
row calculations. A and computes the corresponding rows in 𝐶. The method functions
effectively in environments that utilize shared memory. Element-wise distribution works as an
effective approach by distributing each processor to calculate separate 𝐶 elements.The massive
number of parallel threads that GPUs can run effectively makes this approach an optimal choice
for them. When dealing with complex large-scale calculations block-wise distribution provides
the most suitable solution because it segments matrices into distinct blocks to minimize
communication delays.
2.​ Distributed Computing Example:​
Suppose you need to create a distributed application to process requests from users in
different geographic locations. Describe how you would distribute tasks across multiple
machines and ensure load balancing. What are some potential challenges in managing
the distributed system, and how would you address them?
 Distributed computing presents an effective methodology to process
large-scale applications which assist diverse users from various geographic
locations. To create a distributed application which handles user requests
efficiently I would develop a system design which distributes workload properly to
achieve smooth operation.
The distribution of tasks would be achieved by creating multiple servers in different
regions to provide users with local access for improved latency. A load balancer
constitutes a critical element for this system because it enables fair distribution of
incoming requests between operational machines.
Optimal response times are achieved through this setup which keeps any single server
from receiving too many requests but avoids machine overload at the same time. The
distribution of tasks will proceed with round-robin scheduling which relocates requests to
successive servers or least connections directing connections to the least busy available
server. Distribution management for systems creates multiple operational difficulties.
The primary problem in distributed systems comes from network delays since data
needs to pass between multiple machines at different locations. I would deploy cache
strategies which will maintain often-requested data near users for speedier delivery. The
system needs to maintain fault tolerance because when a server ends its operation the
rest of the system should continue functioning nominally.
The system would function with backup servers that instantly become active when
machines stop working through failover and redundancy solutions. Consistency of data
becomes complicated because distributed systems maintain different server locations
containing duplicate pieces of information.
To maintain up-to-date information for users I would apply eventual consistency in
databases coupled with replication and synchronization techniques.The power of
distributed computing for large-scale applications remains significant though it needs
proper advanced design decisions. A well-designed distributed system which efficiently
manages distributed tasks and deals with essential challenges delivers fast reliable
scalable services to global users.
Part 4: Reflection
Write a short reflection (100-150 words) on how Parallel and Distributed Computing impact
the efficiency and capabilities of modern computing systems. Consider areas like cloud
computing, big data, scientific simulations, and real-time applications.
My assessment is that parallel and distributed computing brought a revolutionary
transformation to modern computing since they enhanced both performance speed and capacity
expansion. With parallel computing models multiple computational operations execute in parallel
on a single machine that keeps it suitable for high-speed requirements in scientific simulations
alongside real-time processing requirements. The absence of these modern technologies would
result in slower progress for artificial intelligence development together with weather prediction
models and financial software designs. Google Cloud and AWS manage millions of user
requests through distributed computing as their core service. Big data frameworks distribute
tasks across clusters as they process extensive information by using such frameworks as
Hadoop and Spark. The ongoing development of computer technologies amazes me by
expanding the capabilities of machines at an exceptional rate. Parallel and distributed
computing systems will continue to be critical because they produce better speed with more
power alongside scalable operations.

Part 5: Research Application

Find an example of a real-world application (e.g., in scientific computing, finance, or healthcare)
that uses either parallel or distributed computing. Describe the application and explain why the
chosen approach (parallel or distributed) is used in that context. Provide a brief explanation (2-3
sentences) of the benefits it provides.

One real-life instance of distributed computing is forecasting the weather, as this area uses
HPC systems for the massive amounts of data that need to be processed. Institutions such as
the European Centre for Medium-Range Weather Forecasts (ECMWF) and the National
Weather Service (NWS) are examples that work with distributed systems to perform satellite
data analysis, climate simulation, and weather prediction globally.
In this case, distributed computing is required because the amount of data from several sources
can only be processed with multiple machines, which makes it impossible to forecast weather
from a singular centralized server. This makes for easier computations which can be more
accurately parsed and produced much quicker.
Some advantages of distributed systems include fault tolerance in case of hardware failure,
speed which caters to real-time requirements and the system can always scale, which is very
important for meteorologists working with global climate data.

Submission Instructions:
●​ Submit your answers by the end of the class.

●​ Be prepared to present your case study findings to the class and engage in a discussion.

This seatwork provides a mix of conceptual questions, practical examples, and real-world
applications that will help students engage with the topic of Parallel and Distributed Computing
and understand their real-world impact.