CSCD604: Distributed Systems

Session 1 – Introduction

By
Solomon Mensah (PhD)

College of Education
School of Continuing and Distance Education
2014/2015 – 2016/2017
 Agenda
• What is a Distributed System?
• Examples of Distributed systems
• Concurrency vs Parallelism
• Focus of the course
• Centralized vs. Distributed systems
• Motivation
• Challenges
• Basic Network Topologies
• Case studies: Google and PASS
• Two Generals’ Problem
 What is a Distributed System?
• Definition 1:
– is one in which components located at networked computers
communicate and coordinate their actions only by passing
messages.
• Definition 2:
– “is one in which the failure of a computer you didn’t even know
existed can render your own computer unusable” [Lesile Lamport,
1987].
• Definition 3:
– is a set of nodes (computers), connected by a network which appear
to its users as a single coherent system.
 What is a Distributed System?
• Definition 4:
– is a system whose components are located on different
networked computers, which communicate and coordinate
their actions by passing messages to one another.
• Definition 5:
– is a network that consists of autonomous computers that are
connected using a distribution middleware.
• They help in sharing different resources and capabilities to
provide users with a single and integrated coherent network.
 Distributed Computing and Examples

• Distributed computing is a field of computer science

that studies distributed systems.
• Examples of distributed systems/ applications:
– Web search engines – Google, Yahoo, Bing, etc
– Multi-player online games
– Financial trading systems
– Email
– Social networks
 Case Study: Google Search Engine

Case Study Assignment I:

• How does the Google search engine contribute to a distributed
system and what are the underling methodologies behind how
the Google search engine works?
• Refer to Chapter 21 of recommended textbook
• Coulouris, G., Dollimore, J., Kindberg, T., and Blair, G. (2012).
Distributed System: Concepts and Design, 5th ed. Addison-Wesley.
ISBN 10: 0-13-214301-1
 Concurrency vs Parallelism
• Concurrency is the task of running and managing the multiple
computations at the same time. While parallelism is the task of
running multiple computations simultaneously.
• Concurrency relates to an application that is processing more
than one task at the same time.
• Concurrency is an approach that is used for decreasing the
response time of the system by using the single processing unit.
• Concurrency creates the illusion of parallelism, however actually
the chunks of a task aren’t parallelly processed, but inside the
application, there are more than one task that is being processed
at a time.
• It doesn’t fully end one task before it begins ensuing.
 Concurrency vs Parallelism
• Parallelism is related to an application where tasks
are divided into smaller sub-tasks that are processed
seemingly simultaneously or parallel.
• It is used to increase the throughput and
computational speed of the system by using multiple
processors.
• It enables single sequential CPUs to do lot of things
“seemingly” simultaneously.
 Focus of this course
• Focus on
– Concepts of distributed systems
– Models/Algorithms
– Resource sharing
• main motivation for constructing distributed systems
– Case studies
 Why study distributed systems?
• It is interesting and challenging
– Partial failures
• Node failures
– If one node fails, the others can be able to function
– Dropped messages over network communication
– Concurrency
• Nodes execute in parallel
– Parallel computing
 Why study Distributed systems?
• It is important and useful
– Societal importance
• Internet/ www
• Small devices (mobile phones)
– Technical importance
• Improve scalability
• Improve reliability
• Inherent distribution
 Centralized vs Distributed Systems

• Centralized System:
– System in which major functions are performed by a single
physical computer
• Originally, everything on single computer
• Later: client/server model

• Distributed System:
– physically separate computers working together on some task
• Early model: multiple servers working together
– Probably in the same room or building
– Often called a “cluster”
• Later models: peer-to-peer/wide-spread collaboration
 Centralized vs Distributed Systems

Server

Client/Server Model

Peer-to-Peer Model
 Distributed Systems: Motivation
• Why do we want distributed systems?
– Cheaper and easier to build lots of simple computers
– Users can have complete control over some components
– Collaboration:
• Much easier for users to collaborate through network resources
(such as network file systems)

• The promise of distributed systems:

– Higher availability: one machine goes down, use another
– Better durability: store data in multiple locations
– Enhance security: each piece easier to make secure
 Distributed Systems: Challenges/Issues
• Reality has been disappointing
– Worse availability: depend on every machine being up
• Lamport: “a distributed system is one where I can’t do
work because some machine I’ve never heard of isn’t
working!”
– Worse reliability: can loose data if any machine crashes
– Worse security: anyone in the world can break into system

• Coordination is more difficult

– Must coordinate multiple copies of shared state information (using
only a network)
– What would be easy in a centralized system becomes a lot more
difficult in distributed system
 Distributed Systems:
Goals/Requirements
• Transparency: the ability of the system to mask its complexity
behind a simple interface
• Possible transparencies:
– Location: Can’t tell where resources are located
– Migration: Resources may move without the user knowing
– Replication: Can’t tell how many copies of resource exist
– Concurrency: Can’t tell how many users exist
– Parallelism: System may speed up large jobs by splitting them
into smaller pieces
– Fault Tolerance: System may hide various things that go wrong
in the system
• Transparency and collaboration require a medium for different
processors to communicate with one another
 Basic Networking Topologies

• Network: physical connection that allows two or more

computers to communicate
• Packet: unit of transfer, sequence of bits carried over the
network
– Network carries packets from one CPU to another
• Protocol: agreement between two parties as to how
information is to be transmitted
 Case Study: PASS used in Space Shuttle
• Developed by IBM in 1981
– PASS – Primary Avionics System Software
– Developed to provide a basic flight capability of the space
shuttle
– Built on 4 different nodes for fault-tolerance

– Assignment II:
• Explain how PASS built on 4 different nodes qualifies to be a
distributed system and how it contributed in improving reliability
in the space shuttle.
 Two Generals’ Problem in distributed
systems
• What is the Two Generals’ Problem or Byzantine Generals’
Problem?
– Applicability to distributed systems
• Two nodes need to agree on a value
• Communicate by messages using unreliable channel

– Agreement is Core problem …

 Two Generals’ Problem in distributed
systems
• Two generals (g1, g2) need to coordinate an
attack
– Must agree on time to attack
– They’ll win only if they attack simultaneously
– Communicate through messengers
– Messengers may be killed on the way
 Two Generals’ Problem in distributed
systems
• Let’s try and solve the problem for g1 and g2
– g1 sends time of attack to g2
– Problem: how to ensure g2 received msg?
– Solution: let g2 acknowledge receipt of msg
– Problem: how to ensure g1 acknowledgement?
• What if time of attack is not agreed by g1 and g2?
• Can they reach an agreement?
 References
• Coulouris, G., Dollimore, J., Kindberg, T., and Blair, G.
(2012). Distributed System: Concepts and Design, 5th
ed. Addison-Wesley. ISBN 10: 0-13-214301-1
[Recommended text book]
• Tanenbaum, A.S., and van Steen, M. 2007. Distributed
Systems: Principles and Paradigms, 2nd ed. Prentice-Hall.
ISBN: 978-0-13-239227-3
 Resource Material
–Resource materials can be found at the
following link:

– SAKAI: https://sakai.ug.edu.gh/
Thank you