Scribd
Upload
3 views

01-Parallel Computing

Uploaded by

Alex Siryani
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
3 views

01-Parallel Computing

Uploaded by

Alex Siryani
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 7

Parallel Computing

Outlines - Introduction

 Cost versus Performance


 What is Parallel Computing?
 The Scope of Parallel Computing
 Issues in Parallel Computing

1
Cost versus Performance
Cost versus Performance curve and its evolution over the decades.

2000s
1990s

1980s

1970s

1960s
Performance

Cost
3

What is Parallel Computing?


 Example: Library and workers to distribute books

1) Dividing a task among workers by assigning them a


set of books is an instance of task partitioning.

2) Passing books to each other is an example of


communication between subtasks.

2
The Scope of Parallel Computing
 Applications such as weather prediction and
pollution monitoring.

 Satellites collect billions of bits per seconds


of data relating to pollution level and the
thickness of ozone layer.

Example: Weather Modeling & Forecasting

 Assumptions:
1) Modeling of weather over an area of 3000 x 3000 miles.

2) Area is being modeled up to a height of 11 miles.

3) 3000 x 3000 x 11 cubic mile domain is partitioned into


segments of size 0.1 x 0.1 x 0.1 cubic miles which is
approximately 1011 different segments.
4) Modeling the weather over a two-day period and the
parameters need to be computed once every half hour.

3000 x 3000 x 11 = 99000000 cubic miles

99000000 / (0.1 x 0.1 x 0.1) = 99000000000 ≈ 1011 Segments


6

3
Assumptions: Weather … Continues
5) The computation of parameters inside a segment
uses initial values and values from neighboring
segments.
 Assume that this computation takes 100 instructions,
then a single updating of parameters in entire domain
requires
 1011 segments x 100 instructions = 1013 instructions
 Since this has to be done approximately 100 times
(two days every half hour, that is 96), then total
number of operations (instructions) is 1015.
24 Hours x 2 x 2 days = 96 ≈ 102 times
7
1013 instructions x 102 times= 1015 instructions

Example: Weather Modeling … (Continues)

 On serial supercomputer capable of performing


one billion instructions per second, this weather
modeling would take approximately 280 hours
that is: 1000000000 instructions per second = 109 instruction per second
 1015 instructions / 109 = 10(15-9) = 106 seconds
 106 seconds / 60 second = 16,666.7 minutes
 16,666.7 minutes / 60 minute = 280 hours
 280 hour / 24 hour = 11.67 days

4
Example: Weather Modeling … (Continues)

 Taking 280 hours (11.67 days) to predict weather


for next 48 hours (2 days) is unreasonable.

 Parallel processing makes it possible to predict


weather not only faster but also more accurate.

Example: Weather Modeling … (Continues)

 If we have parallel computer with 1000 workstation class


processors, then we can partition 1011 segments of
domain among these processors.
 Each processor computes parameters for 108 segments
 1011 segments / 103 processors = 108 segments for each processor
 Assuming that the computing power of this computer is
100 million instructions per second, the problem can be
solved in less than 3 hours:
 100000000 instructions per second = 108 is the power of each processor
 108 segments x 100 (instructions per segment) = 108 x 102 =10 (8+2) = 1010
instructions for each processor
 1010 instructions x 100 (2 days) = 10(10+2) = 1012 instructions
 1012 instructions / 108 instruction per second = 10(12-8) = 104 seconds
 104 seconds / 3600 (60 second x 60 minute) sec. ≈ 2.7 hours.
 So, the whole process takes 2.7 hours because processors are working in parallel
10

10

5
Issues in Parallel Computing
1) Design of Parallel Computers:
 Large number of processors.
 Supporting fast communication.
 Supporting data sharing.
2) Design of Efficient Algorithms:
 Issues in designing parallel algorithms are very
different from those in designing their sequential
computers
 Partition: decompose task into several parallel tasks
 Load Balancing: distribute load on processors as evenly as possible
 Communication: How processors can communicate efficiently to
perform the whole parallel task
11

11

Issues in Parallel … (Continues)


3) Methods for Evaluating Parallel Algorithms:
 Given a parallel computer and a parallel algorithm
running on it, we need to evaluate the performance of
the resulting system.
 Performance analysis allows us to answer questions
such as:
a) How fast can a problem be solved using parallel processing?
b) How efficient are the processors used?

12

12

6
Issues in Parallel … (Continues)
4) Parallel algorithms are implemented on parallel
computers using a programming language.
 Examples: Pthreads, High Performance Fortran (HPF)
5) Parallel Programming Tools:
 To facilitate programming of parallel computers.
 Examples: MPI and PVM (using Fortran, C/C++, C#,
and Java).

13

13

Issues in Parallel … (Continues)

6) Portable Parallel Programs


 In a sense that the parallel program can be executed
under different operating systems and different
architectures
7) Automatic Programming of Parallel Computers:
 Parallel compilers are expected to allow us to program a
parallel computer like a serial computer.

14

14

You might also like