PARALLEL COMPUTING Semester 7
Course Code BCI714B CIE Marks 50
Teaching Hours/Week (L:T:P: S) 3:0:0:0 SEE Marks 50
Total Hours of Pedagogy 40 Total Marks 100
Credits 03 Exam Hours 03
Examination nature (SEE) Theory
Course objectives:
This course will enable to,
• Explore the need for parallel programming
• Explain how to parallelize on MIMD systems
• To Interpret how to apply MPI library and parallelize the suitable programs
• To analyze how to apply OpenMP pragma and directives to parallelize the suitable programs
• To demonstrate how to design CUDA program
Teaching-Learning Process (General Instructions)
These are sample Strategies that teachers can use to accelerate the attainment of the various course outcomes.
1. Lecturer method (L) need not to be only traditional lecture methods, but alternative effective teaching
methods could be adopted to attain the outcomes.
2. Use of Video/Animation to explain functioning of various concepts.
3. Encourage collaborative (Group Learning) Learning in the class.
4. Ask at least three HOT (Higher order Thinking) questions in the class, which promotes critical
thinking.
5. Adopt Programming assignment, which fosters student’s Analytical skills, develop design thinking
skills such as the ability to design, evaluate, generalize, and analyze information rather than simply
recall it.
MODULE-1
Introduction to parallel programming, Parallel hardware and parallel software – Classifications
of parallel computers, SIMD systems, MIMD systems, Interconnection networks, Cache coherence,
Shared-memory vs. distributed-memory, Coordinating the processes/threads, Shared-memory,
Distributed-memory.
Textbook1: Chapter-1, Chapter 2
MODULE-2
GPU programming, Programming hybrid systems, MIMD systems, GPUs, Performance –
Speedup and efficiency in MIMD systems, Amdahl’s law, Scalability in MIMD systems, Taking
timings of MIMD programs, GPU performance.
Textbook1: Chapter2
MODULE-3
Distributed memory programming with MPI – MPI functions, The trapezoidal rule in MPI,
Dealing with I/O, Collective communication, MPI-derived datatypes, Performance evaluation of MPI
programs, A parallel sorting algorithm.
Textbook1: Chapter 3
MODULE-4
Shared-memory programming with OpenMP – openmp pragmas and directives, The trapezoidal
rule, Scope of variables, The reduction clause, loop carried dependency, scheduling, producers and
consumers, Caches, cache coherence and false sharing in openmp, tasking, tasking, thread safety.
Textbook1: Chapter 5
MODULE-5
GPU programming with CUDA - GPUs and GPGPU, GPU architectures, Heterogeneous computing,
Threads, blocks, and grids Nvidia compute capabilities and device architectures, Vector addition,
Returning results from CUDA kernels, CUDA trapezoidal rule I, CUDA trapezoidal rule II: improving
performance, CUDA trapezoidal rule III: blocks with more than one warp.
Textbook1: Chapter 6
Course outcomes (Course Skill Set):
At the end of the course, the student will be able to:
● Explain the need for parallel programming.
● Illustrate parallelism in MIMD system.
● Make use of MPI library to parallelize the code to solve a problem.
● Interpret OpenMP pragma and directives to parallelize the code to solve the given problem.
● Demonstrate a CUDA program for the given problem.
Assessment Details (both CIE and SEE)
The weightage of Continuous Internal Evaluation (CIE) is 50% and for Semester End Exam (SEE) is 50%. The
minimum passing mark for the CIE is 40% of the maximum marks (20 marks out of 50) and for the SEE minimum
passing mark is 35% of the maximum marks (18 out of 50 marks). A student shall be deemed to have satisfied the
academic requirements and earned the credits allotted to each subject/ course if the student secures a minimum of
40% (40 marks out of 100) in the sum total of the CIE (Continuous Internal Evaluation) and SEE (Semester End
Examination) taken together.
Continuous Internal Evaluation:
● For the Assignment component of the CIE, there are 25 marks and for the Internal Assessment Test
component, there are 25 marks.
● The first test will be administered after 40-50% of the syllabus has been covered, and the second test will be
administered after 85-90% of the syllabus has been covered
● Any two assignment methods mentioned in the 22OB2.4, if an assignment is project-based then only one
assignment for the course shall be planned. The teacher should not conduct two assignments at the end of
the semester if two assignments are planned.
● For the course, CIE marks will be based on a scaled-down sum of two tests and other methods of
assessment.
Internal Assessment Test question paper is designed to attain the different levels of Bloom’s taxonomy as per the
outcome defined for the course.
Semester-End Examination:
Theory SEE will be conducted by University as per the scheduled timetable, with common question papers for the
course (duration 03 hours).
1. The question paper will have ten questions. Each question is set for 20 marks.
2. There will be 2 questions from each module. Each of the two questions under a module (with a maximum of 3
sub-questions), should have a mix of topics under that module.
3. The students have to answer 5 full questions, selecting one full question from each module.
4. Marks scored shall be proportionally reduced to 50 marks.
Suggested Learning Resources:
Textbook:
1. Peter S Pacheco, Matthew Malensek – An Introduction to Parallel Programming, second
edition, Morgan Kauffman, 2021.
Reference Books:
1. Michael J Quinn – Parallel Programming in C with MPI and OpenMp, McGrawHill, 2004.
2. Calvin Lin, Lawrence Snyder – Principles of Parallel Programming, Pearson
3. Barbara Chapman – Using OpenMP: Portable Shared Memory Parallel Programming,
Scientific and Engineering Computation
4. William Gropp, Ewing Lusk – Using MPI:Portable Parallel Programing, Third edition, Scientific
and Engineering Computation
Web links and Video Lectures (e-Resources):
1. Introduction to parallel programming: https://nptel.ac.in/courses/106102163
Activity Based Learning (Suggested Activities in Class)/ Practical Based learning
Programming Assignment - 25 marks
1. Write a OpenMP program to calculate n Fibonacci numbers using tasks.
2. Write a OpenMP program to find the prime numbers from 1 to n employing parallel for directive. Record both
serial and parallel execution times.
3. Write a MPI Program to demonstration of MPI_Send and MPI_Recv.
4. Write a MPI program to demonstration of deadlock using point to point communication and avoidance of
deadlock by altering the call sequence
5. Write a MPI Program to demonstration of Broadcast operation.
6. Write a MPI Program demonstration of MPI_Scatter and MPI_Gather
7. Write a MPI Program to demonstration of MPI_Reduce and MPI_Allreduce (MPI_MAX, MPI_MIN, MPI_SUM,
MPI_PROD)
