Introduction to Distributed Systems
Distributed And Parallel System
BY: M. TAHIR MUMTAZ
1
Contents
1. Definition of Distributed System
2. Examples/Application of Distributed System
3. Advantage and Disadvantages of Distributed System
4. Definition Of Parallel System
5. Examples/Applications of Parallel System
6. Advantages and Disadvantages of Parallel System
7. References/Sources
DEFINITION
A distributed system is a collection of
independent computers, interconnected via a
network, capable of collaborating on a task.
A distributed system can be characterized as
collection of multiple autonomous computers
that communicate over a communication
network and having following features:
No common Physical clock
Enhanced Reliability
Increased performance/cost ratio
Access to geographically remote data and resources
Scalability
3
Distributed Systems
A collection of independent computers
that appear to the users of the system
as a single computer
A collection of autonomous computers,
connected through a network and
distribution middleware which enables
computers to coordinate their
activities and to share the resources of
the system, so that users perceive the
system as a single, integrated
computing facility
A Distributed System
4
Examples of Distributed System
Telephone Networks and Cellular
Networks
Computer Networks Such as internet
or intranet
ATM(bank) Machines
Distributed database and distributed
database management system
Network of Workstations
Mobile Computing etc.
5
6
Why Distributed Systems?
Design Requirements
Performance issues
Responsiveness
Throughput
Quality of service
Correctness
Reliability, availability, fault tolerance
Security
Performance
Adaptability
Advantages Of Distributed System
Information Sharing among Distributed
Users
Resource Sharing
Extensibility and Incremental growth
Shorter Response Time and Higher Output
Higher Reliability
Better Flexibility’s in meeting User’s needs
Better price/performance ratio
Scalability
Transparency
7
Disadvantages of Distributed System
Difficulties of developing distributed
software
Networking Problem
Security Problems
Performance
Openness
Reliability and Fault Tolerance
8
Basic problems and
challenges
Transparency
Scalability
Fault tolerance
Concurrency
Openness
These challenges can also be seen as the goals or desired
properties of a distributed system
Transparency
Concealment from the user and the application
programmer of the separation of the components of a
distributed system
Access Transparency - Local and remote resources are accessed
in same way
Location Transparency - Users are unaware of the location of
resources
Migration Transparency - Resources can migrate without name
change
Replication Transparency - Users are unaware of the existence of
multiple copies of resources
Failure Transparency - Users are unaware of the failure of
individual components
Concurrency Transparency - Users are unaware of sharing
resources with others
Scalability
Addition of users and resources without suffering a
noticeable loss of performance or increase in
administrative complexity
Adding users and resources causes a system to grow:
Size - growth with regards to the number of users or resources
System may become overloaded
May increase administration cost
Geography - growth with regards to geography or the distance
between nodes
Greater communication delays
Administration – increase in administrative cost
Openness
Whether the system can be extended in various ways without
troublesome existing system and services
Hardware extensions
adding peripherals, memory, communication interfaces
Software extensions
Operating System features
Communication protocols
Openness is supported by:
Public interfaces
Standardized communication protocols
Concurrency
In a single system several processes are interleaved
In distributed systems - there are many systems with one or
more processors
Many users simultaneously invoke commands or applications,
access and update shared data
Mutual exclusion
Synchronization
No global clock
Fault tolerance
Hardware, software and networks fail
Distributed systems must maintain availability even at low
levels of hardware, software, network reliability
Fault tolerance is achieved by
Recovery
Redundancy
Issues
Detecting failures
Recovery from failures
Redundancy
Fault tolerance
Omission and Arbitrary Failures
Parallel System
A system is said to be a Parallel System in which
multiple processor have direct access to shared
memory which forms a common address space.
Usually tightly-coupled system are referred to as
Parallel System. In these systems, there is a single
system wide primary memory (address space)
that is shared by all the processors. On the other
hand Distributed System are loosely-coupled
system.
Parallel computing is the use of two or more
processors (cores, computers) in combination to
solve a single problem.
9
Examples
Loosely-Coupled Systems
Most distributed systems are “loosely-
coupled:
Each CPU runs an independent autonomous
OS.
Hosts communicate through message
passing.
Computer don’t really trust each other.
Some resources are shared, but most are not.
The system may look differently from different
hosts.
Typically, communication times are long.
Tightly-Coupled Systems
A “tightly-coupled” system usually refers to a
multiprocessor.
Runs a single copy of the OS with a single job queue
has a single address space
usuallyhas a single bus or backplane to which all
processors and memories are connected
processors communicate through shared memory
A Parallel System
10
Applications of Parallel System
An example of Parallel computing
would be two servers that share the
workload of routing mail, managing
connections to an accounting
system or database, solving a
mathematical problem etc
Supercomputers are usually placed
in parallel system architecture
Terminals connected to single
server
11
EXAMPLE: The Earth Simulator Supercomputer
from (2002-2004)
12
Advantages of Parallel System
Provide Concurrency(do multiple
things at the same time)
Taking advantage of non-local
resources
Cost Savings
Overcoming memory constraints
Save time and money
Global address space provides a user-
friendly programming perspective to
memory
13
Disadvantages of Parallel System
Primary disadvantage is the lack of
scalability between memory and
CPUs.
Programmer responsibility for
synchronization constructs that ensure
"correct" access of global memory.
It becomes increasingly difficult and
expensive to design and produce
shared memory machines with ever
increasing numbers of processors.
14
Parallel vs. Distributed System
Parallel Systems Distributed Systems
Memory Tightly coupled system Weakly coupled system
shared memory Distributed memory
Control Global clock control No global clock control
Processor Order of Tbps Order of Gbps
interconnection
Main focus Performance Performance(cost and scalability)
Scientific computing Reliability/availability
Information/resource sharing
15
Sources / References
Websites en.wikipedia.org
books.google.com
www.seminarprojects.com
http://publib.boulder.ibm.com
www.webopedia.com
https://computing.llnl.gov/tutorials
www.cis.upenn.edu
Books Distributed System by Coulouris
Distributed Computing by D. Kshemkalyani,Mukesh
Singhal
16
Thank You
