Lecture 1 Introduction To PDC

Parallel and Distributed Computing
CS 3006 (BCS-7A | BDS-7A)

Lecture 1
Danyal Farhat
FAST School of Computing
NUCES Lahore
Introduction to Parallel and
Distributed Computing
Motivation for Parallel and Distributed Computing
• Uniprocessor are fast but
Some problems require too much computation
Some problems use too much data
Some problems have too many parameters to explore
• For example
Weather simulations, gaming, web Servers, code breaking
Need for Parallelism?
• Developing parallel hardware and software has traditionally been
time and effort intensive
• If one is to view this in the context of rapidly improving
uniprocessor speeds, one is tempted to question the need for
parallel computing
• Latest trends in hardware design indicate that uni-processors may
not be able to sustain the rate of realizable performance
increments in the future
• A number of fundamental physical and computational limitations
• The emergence of standardized parallel programming
environments, libraries, and hardware have significantly reduced
time to develop (parallel) solution
Moore’s Law
• Number of transistors incorporated in a chip will approximately
double every 18 months.
Moore’s Law (Cont.)
Doubling the transistors does doubles the speed?
• True for single core chips but now the increase in number of
transistor per processor is due to multi-core CPU’s
• In multi-core CPUs, speed will depend on the proper utilization
of these parallel resources
Will Moore’s law hold forever?
• Adding multiple cores on single chip causes heat issues
• Increasing the number of cores, may not be able to increase
speeds [Due to inter-process interactions]
• Transistors would eventually reach the limits of miniaturization
at atomic levels
Need for Parallel and Distributed Computing
• Need for:
Good parallel hardware
Good system software
Good Sequential algorithms
Good Parallel algorithms
• Parallel and Distributed Systems
Parallel and Distributed Systems
• Parallel and distributed computing is going to be more and
more important
Dual and quad core processors are very common
Up to six and eight cores for each CPU
Multithreading is growing
• Hardware structure or architecture is important for
understanding how much it is possible to speed up beyond a
single CPU
• Also capability of compilers to generate efficient code is very
important (e.g. converting for loop into threads)
• It is always difficult to distinguish between HW and SW
influences
Distributed System vs Distributed Computing
Distributed Systems:
• “A collection of autonomous computers, connected
through a network and distribution middleware.”
• Enables computers to coordinate their activities and to
share the resources of the system
• The system is usually perceived as a single, integrated
computing facility
• Mostly concerned with the hardware-based accelerations
Distributed System vs Distributed Computing (Cont)
Distributed Computing:
• “A specific use of distributed systems to split a large and
complex processing into subparts and execute them in
parallel to increase the productivity.”
• Computing mainly concerned with software-based
accelerations (i.e., designing and implementing algorithms)
Parallel vs Distributed Computing
Parallel Computing:
• The term is usually used for developing concurrent
solutions for following two types of the systems:
Multi-core Architecture
Many core architectures (i.e., GPU’s)
Distributed Computing:
• The type of computing mainly concerned with developing
algorithms for the distributed cluster systems
• Here distributed means a geographical distance between
the computers without any shared-Memory
Parallel vs Distributed Computing (Cont.)
Parallel Computing: Distributed Computing:
• Shared memory • Distributed memory
• Multiprocessors • Clusters and networks of
• Processors share logical workstations
address spaces • Processors do not share
• Processors share physical address space
No sharing of physical memory
memory
• Also referred to the study • Study of theoretical
of parallel algorithms distributed algorithms or
neural networks
Practical Applications of P&D Computing
Scientific Applications:
• Functional and structural characterization of genes and
proteins
• Applications in astrophysics have explored the evolution of

galaxies, thermonuclear processes, and the analysis of
extremely large datasets from telescope
Practical Applications of P&D Computing (Cont.)
Scientific Applications:
• Advances in computational physics and chemistry have
explored new materials, understanding of chemical
pathways, and more efficient processes
Example: Large Hydron Collider (LHC) at European Organization for Nuclear
Research (CERN) generates petabytes of data for a single collision
• Weather modeling for simulating the track of natural
hazards like extreme cyclones (storms), flood prediction
Commercial Applications:
• Some of the largest parallel computers power the Wall
Street!
• Data mining-analysis for optimizing business and
marketing decisions
• Large scale servers (mail and web servers) are often
implemented using parallel platforms
• Applications such as information retrieval and search are
typically powered by large clusters
Computer Systems Applications:
• Network intrusion detection: a large amount of data needs
to be analyzed and processed
• Cryptography (the art of writing or solving codes) employs
parallel infrastructures and algorithms to solve complex
codes – SHA 256
• Graphics processing – Each pixel represents a binary
• Embedded systems increasingly rely on distributed control
algorithms - Modern automobiles use embedded systems
Thank You!

Lecture 1 Introduction To PDC

Uploaded by

Copyright:

Available Formats

Lecture 1 Introduction To PDC

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Lecture 1 Introduction To PDC

Uploaded by

Copyright:

Available Formats

Parallel and Distributed Computing

CS 3006 (BCS-7A | BDS-7A)

• Applications in astrophysics have explored the evolution of

You might also like