Faculty of Engineering Savitribai Phule Pune University
Home
Savitribai Phule Pune University
Fourth Year of Computer Engineering (2019 Course) Home
410250: High Performance Computing
Teaching Scheme: Credit Examination Scheme:100
TH: 4 Hours/Week 3 Mid-Semester (TH) : 30
End- Sem (TH): 70
Prerequisites Courses: -Microprocessor (210254), Principles of Programming
Languages(210255), Computer Networks and Security(310244)
Companion Course: Laboratory Practice V(410254)
Course Objectives:
To understand different parallel programming models
To analyze the performance and modeling of parallel programs
To illustrate the various techniques to parallelize the algorithm
To implement parallel communication operations.
To discriminate CUDA Architecture and its components.
To Understand Scope of Parallel Computing and its search algorithms.
Course Outcomes:
CO1: Understand various Parallel Paradigm
CO2: Design and Develop an efficient parallel algorithm to solve given problem
CO3: Illustrate data communication operations on various parallel architecture
CO4: Analyze and measure performance of modern parallel computing systems
CO5: Apply CUDA architecture for parallel programming
CO6: Analyze the performance of HPC applications
Course Contents
Unit I Introduction to Parallel Computing 09 Hours
Introduction to Parallel Computing: Motivating Parallelism, Modern Processor: Stored-
program computer architecture, General-purpose Cache-based Microprocessor architecture. Parallel
Programming Platforms: Implicit Parallelism, Dichotomy of Parallel Computing Platforms,
Physical Organization of Parallel Platforms, Communication Costs in Parallel Machines. Levels of
parallelism, Models: SIMD, MIMD, SIMT, SPMD, Data Flow Models, Demand-driven
Computation, Architectures: N-wide superscalar architectures, multi-core, multi-threaded.
#Exemplar/Case
Studies Case study: Multi-core System
*Mapping of Course
Outcomes for Unit I CO1
Unit II Parallel Algorithm Design 09 Hours
Global System for Mobile Communications (GSM) architecture , Mobile Station, Base Station
System, Switching subsystem, Security, Data Services, HSCSD, GPRS - GPRS system and
protocol architecture 2.3 UTRAN, UMTS core network; Improvements on Core Network, 802.11
Architecture 802.11a, 802.11b standard
#Exemplar/Case IPoC: A New Core Networking Protocol for 5G Networks.
Studies
Syllabus for Fourth Year of Computer Engineering ` #72/128
Faculty of Engineering Savitribai Phule Pune University
*Mapping of Course CO2
Outcomes for Unit II
Unit III Parallel Communication 09 Hours
Basic Communication: One-to-All Broadcast, All-to-One Reduction, All-to-All Broadcast and
Reduction, All-Reduce and Prefix-Sum Operations, Collective Communication using MPI: Scatter,
Gather, Broadcast, Blocking and non blocking MPI, All-to-All Personalized Communication,
Circular Shift, Improving the speed of some communication operations.
#Exemplar/Case Case study: Monte-Carlo Pi computing using MPI
Studies
*Mapping of Course CO3
Outcomes for UnitIII
Unit IV Analytical Modeling of Parallel Programs 09 Hours
Sources of Overhead in Parallel Programs, Performance Measures and Analysis: Amdahl's and
Gustafson's Laws, Speedup Factor and Efficiency, Cost and Utilization, Execution Rate and
Redundancy, The Effect of Granularity on Performance, Scalability of Parallel Systems, Minimum
Execution Time and Minimum Cost, Optimal Execution Time, Asymptotic Analysis of Parallel
Programs. Matrix Computation: Matrix-Vector Multiplication, Matrix-Matrix
Multiplication.
#Exemplar/Case Case study: The DAG Model of parallel computation
Studies
*Mapping of Course
Outcomes for UnitIV CO4
Unit V CUDA Architecture 09 Hours
Introduction to GPU: Introduction to GPU Architecture overview, Introduction to CUDA C-
CUDA programming model, write and launch a CUDA kernel, Handling Errors, CUDA memory
model, Manage communication and synchronization, Parallel programming in CUDA- C.
#Exemplar/Case Case study: GPU applications using SYCL and CUDA on NVIDIA
Studies
*Mapping of Course
Outcomes for Unit V CO5
Unit VI High Performance Computing Applications 09 Hours
Scope of Parallel Computing, Parallel Search Algorithms: Depth First Search(DFS), Breadth First
Search( BFS), Parallel Sorting: Bubble and Merge, Distributed Computing: Document
classification, Frameworks – Kuberbets, GPU Applications, Parallel Computing for AI/ ML
#Exemplar/Case Case study: Disaster detection and management/ Smart Mobility/Urban
Studies planning
*Mapping of Course CO6
Outcomes for Unit
VI
Learning Resources
Syllabus for Fourth Year of Computer Engineering ` #73/128
Faculty of Engineering Savitribai Phule Pune University
Text Books:
1. Ananth Grama, Anshul Gupta, George Karypis, and Vipin Kumar, "Introduction toParallel
Computing", 2nd edition, Addison-Wesley, 2003, ISBN: 0-201-64865-2
2. Seyed H. Roosta, “Parallel Processing and Parallel Algorithms Theory and Computation‖”,
Springer-Verlag 2000 ,ISBN 978-1-4612-7048-5 ISBN 978-1-4612-1220-1
3. John Cheng, Max Grossman, and Ty McKercher, “Professional CUDA C Programming”,
John Wiley & Sons, Inc., ISBN: 978-1-118-73932-7
Reference Books :
1. Kai Hwang,, "Scalable Parallel Computing", McGraw Hill 1998.
2. George S. Almasi and Alan Gottlieb, "Highly Parallel Computing", The Benjamin and
Cummings Pub. Co., Inc
3. Jason sanders, Edward Kandrot, “CUDA by Example”, Addison-Wesley, ISBN-13: 978-
0-13-138768-3
4. Pacheco, Peter S., “An Introduction to Parallel Programming”, Morgan Kaufmann
Publishers ISBN 978-0-12-374260-5
5. Rieffel WH.EG, Polak, “Quantum Computing: A gentle introduction”, MIT Press,
2011, ISBN 978-0-262-01506-6
6. Ajay D. Kshemkalyani , Mukesh Singhal, “ Distributed Computing: Principles,
Algorithms, and Systems”, Cambridge March 2011, ISBN: 9780521189842
e Books :
1. http://prdrklaina.weebly.com/uploads/5/7/7/3/5773421/introduction_to_high_performance_co
mputing_for_scientists_and_engineers.pdf
2. https://www.vssut.ac.in/lecture_notes/lecture1428643084.pdf
NPTEL/YouTube video lecture link
● https://nptel.ac.in/courses/106108055
● https://www.digimat.in/nptel/courses/video/106104120/L01.html
@The CO-PO Mapping Matrix
CO/
PO1 PO2 PO3 PO4 PO5 PO6 PO7 PO8 PO9 PO10 PO11 PO12
PO
CO1 2 1 - - - - - - - - - -
CO2 2 1 - - - - - - - - - -
CO3 2 1 - - - - - - - - - -
CO4 1 2 - 2 - - - - - - - -
CO5 1 2 - 2 - - - - - - - 1
CO6 2 2 - 2 - - - - - - - 1
Syllabus for Fourth Year of Computer Engineering ` #74/128
