DISTRIBUTED COMPUTING

Iftikhar Hussain Babur

System Analyst
Quaid-e-Azam Academy for Educational Development
Govt. of the Punjab, Wahdat Road Lahore
Iftibabur@gmail.com

Superior University, Gold Campus

 Course Outline
Course Title: Distributed Computing
Course Code: COS601130
Credit Hours: 3
Objectives:
The main objectives of this course are to:
1. To learn the principles, architectures, algorithms and programming models used in
Distributed systems.
2. To examine state-of-the-art distributed systems, such as Google File System.
3. To design and implement sample distributed systems.
The course introduces the main principles underlying distributed
systems: processes, communication, naming, synchronization,
consistency, fault tolerance, and security.
On the completion of the unit, students will understand the fundamentals
of distributed computing and be able to design and develop distributed
systems and applications.

Superior University, Gold Campus

 Course Outline
Recommended Books:

1. Tanenbaum, A.S.&Steen,.M.V.(2002)Distributed Systems:

Principles and Paradigms. Prentice-Hall.
2. C. George et el.(2011). Distributed Systems: Concepts and Design.
Addison Wesley.
3. K. Hwang & Z. Xu(1998).Scalable Parallel Computing. McGraw-
Hill.
4. F. Berman, G. Fox, & T. Hey.(2003).Grid Computing. Wiley.
5. Michael P.& Papazoglou.(2007). Web Services: Principles and
Technology. Pearson Prentice Hall.

Superior University, Gold Campus

 Assessment Criteria

Assessment Type Weightage

Assignments (04) 10%
Quiz (Surprise) 04 10%
Mid Term 25%
Project / Presentation 10%
Participation 5%
(100 percent for timely assignments, All
Quizzes, 100 Percent attendance)
Final Term 40%
Distributed Computing

A distributed system is a collection of independent

computers that appears to its users as a single
coherent system.
Important Aspects

• A distributed system consists of components (i.e.,

computers) that are autonomous

• Users (people or programs) think they are dealing with a

single system

• Autonomous components need to collaborate

Characteristics of Distributed Systems

• Differences between various computers and the ways in

which they communicate are mostly hidden from users

• Users and applications can interact with a distributed

system in a consistent and uniform way, regardless of
where and when interaction takes place.

• Distributed systems should also be relatively easy to

expand or scale.
Characteristics of Distributed Systems

• A distributed system will normally be continuously

available

• Layers of software support heterogeneous computers

and networks while offering a single-system view -
sometimes called middleware.
A distributed system organized as middleware.
Four networked computers and three applications:

• Application B is distributed across computers 2 and 3.

• Each application is offered the same interface

A distributed system organized as middleware.
• Distributed system provides the means for components
of a single distributed application to communicate with
each other, but also to let different applications
communicate.
• It also hides the differences in hardware and operating
systems from each application.
Goals

Four goals that should be met to make building a

distributed system worth the effort:

• Should make resources easily accessible

• Should reasonably hide the fact that resources are
distributed across a network;
• Should be open
• Should be scalable.
1. Making Resources Accessible
Main goal of a distributed system –
• make it easy for the users (and applications) to access
remote resources
• to share them in a controlled and efficient way.

• Resources - anything: printers, computers, storage

facilities, data, files, Web pages, and networks, etc.
• Accessibility Issues
• security
• unwanted communication
2. Distribution Transparency
Goal - hide the fact that its processes and resources are
physically distributed across multiple computers – systems
should be transparent
Different forms of transparency in a distributed system
(ISO, 1995).
 2. Distribution Transparency
Transparency Description
Access Hide differences in data representation and how a
resource is accessed
Location Hide where a resource is located
Migration Hide that a resource may move to another location

Relocation Hide that a resource may be moved to another

location while in use
Replication Hide that a resource is replicated
Concurrency Hide that a resource may be shared by several
competitive users
Failure Hide the failure and recovery of a resource
Self Reading Task
Degree of Transparency
Issues:
• Timing:
e.g. requesting an electronic newspaper to appear in your mailbox
before 7 A.M. local time, as usual, while you are currently at the
other end of the world living in a different time zone.
• Synchronization:
e.g. a wide-area distributed system that connects a process in San
Francisco to a process in Amsterdam limited by laws of physics - a
message sent from one process to the other takes about 35
milliseconds.
• it takes several hundreds of milliseconds using a computer network.
• signal transmission is not only limited by the speed of light, but also by
limited processing capacities of the intermediate switches.
Self Reading Task

• Performance:
e.g. many Internet applications repeatedly try to contact a server
before finally giving up. Consequently, attempting to mask a
transient server failure before trying another one may slow down
the system as a whole.

• Consistency:
e.g. need to guarantee that several replicas, located on different
continents, need to be consistent all the time - a single update
operation may now even take seconds to complete, something that
cannot be hidden from users.
Self Reading Task

• Context Awareness:
e.g. notion of location and context awareness is becoming
increasingly important, it may be best to actually expose
distribution rather than trying to hide it. - consider an office worker
who wants to print a file from her notebook computer. It is better to
send the print job to a busy nearby printer, rather than to an idle
one at corporate headquarters in a different country.

• Limits of Possibility:
Recognizing that full distribution transparency is simply impossible,
we should ask ourselves whether it is even wise to pretend that we
can achieve it.
3. Openness
Goal: offer services according to standard rules that describe the syntax
and semantics of those services. e.g.
• computer networks - standard rules govern the format, contents, and
meaning of messages sent and received.
• distributed systems - services are specified through interfaces, which
are often described in an Interface Definition Language (IDL).
• Interface definitions written in an IDL nearly always capture only the
syntax of services
• specify names of the available functions with types of parameters,
return values, possible exceptions that can be raised, etc.
• allows an arbitrary process that needs a certain interface to talk to
another process that provides that interface
• allows two independent parties to build completely different
implementations of those interfaces, leading to two separate distributed
systems that operate in exactly the same way.
3. Openness: Properties of specifications
• Complete - everything that is necessary to make an implementation has been
specified.
• Neutral - specifications do not prescribe what an implementation should look
like
Lead to:
• Interoperability - characterizes the extent by which two implementations of
systems or components from different manufacturers can co-exist and work
together by merely relying on each other's services as specified by a
common standard.
• Portability - characterizes to what extent an application developed for a
distributed system A can be executed, without modification, on a different
distributed system B that implements the same interfaces as A.
Goals: an open distributed system should also be extensible. i.e.
• be easy to configure the system out of different components (possibly from
different developers).
• be easy to add new components or replace existing ones without affecting
those components that stay in place.
4. Scalability
Scalability of a system is measured with respect to:
1. Size - can easily add more users and resources to the system.
2. Geographic extent - a geographically scalable system is one in which the
users and resources may lie far apart.
3. Administrative scalability - can be easy to manage even if it spans many
independent administrative organizations.
Scalability Limitations of Size

Concept Example
Centralized services A single server for all users
Centralized data A single on-line telephone book

Centralized algorithms Doing routing based on complete information

 Distributed Systems Fallacies
• The Network is Reliable
• Latency is Zero
• Bandwidth is infinite
• Network is secure
• Topologies does not change
• There is one administrator
• Transport cost is zero
• The network is homogeneous

Superior University, Gold Campus

Distributed Systems Fallacies
 The Network is Reliable
Networks are complex, dynamic, and often
unpredictable. Many reasons could lead to a network
failure or network-related issues: a switch or a power
failure, misconfiguration, an entire datacenter becoming
unavailable, DDoS attacks, etc. Due to this complexity
and general unpredictability, networks are unreliable.

So how do you make your distributed system reliable,

ensuring it continues to work as expected, even in the
context of unreliable networks?

Superior University, Gold Campus

Distributed Systems Fallacies
 The Network is Reliable
It’s crucial to accept and
treat failure as a matter of
course. You should design
your system so that it’s
able to mitigate failures
that will inevitably occur,
and continue to operate as
expected in spite of
adversities.

Superior University, Gold Campus

Distributed Systems Fallacies
 The Network is Reliable

 Identify critical components and Single Point of Failure (SPoF)

 A single point of failure (SPOF) is a potential risk posed by a
flaw in the design, implementation or configuration of a
circuit or system. SPOF refers to one fault or malfunction
that can cause an entire system to stop operating.
 Chaos Monkey in Netflix
 Chaos Monkey is a software tool that was developed by
Netflix engineers to test the resiliency and recoverability of
their Amazon Web Services (AWS). The software simulates
failures of instances of services running within Auto Scaling
Groups (ASG) by shutting down one or more of the virtual
machines.
Superior University, Gold Campus
Distributed Systems Fallacies
 Latency is Zero
Latency may be close to zero when you’re running apps in your
local environment, and it’s often negligible on a local area
network. However, latency quickly deteriorates in a wide area
network. That’s because in a WAN data often has to travel further
from one node to another, since the network may span large
geographical areas (this is usually the case with large-scale
distributed systems).

As you’re designing your system, you should bear in mind that

latency is an inherent limitation of networks. We should never
assume that there will be no delay or zero latency between data
being sent and data being received.

Superior University, Gold Campus

Distributed Systems Fallacies
 Latency is Zero
Latency is primarily constrained by distance and the speed of light.
Of course, there’s nothing we can do about the latter. Even in
theoretically perfect network conditions, packets cannot exceed the
speed of light. However, we can do something when it comes to
distance: bring data closer to clients through edge computing. If
you’re building a cloud-based system, you should choose your
availability zones carefully, ensuring they are in proximity to your
clients, and route traffic accordingly.

Superior University, Gold Campus

Distributed Systems Fallacies
 Bandwidth is Infinite
While latency is the speed at which data travels from
point A to point B, bandwidth refers to how much data
can be transferred from one place to another in a certain
timeframe.

Superior University, Gold Campus

Distributed Systems Fallacies
 Bandwidth is Infinite
Computing technology has, without a doubt, made
significant advancements since the 90s, when the 8
fallacies were coined. Network bandwidth has also
improved, and we’re now able to send more data across
our networks.
However, even with these improvements, the capacity of
networks is not infinite (partially because our appetite for
generating and consuming data has also increased).
When a high volume of data is trying to flow through the
network, and there is insufficient bandwidth support,
various issues can arise:
Superior University, Gold Campus
Distributed Systems Fallacies
 Bandwidth is Infinite
• Queuing delays,
• bottlenecks, and
• network congestion.
• Packet loss leading to inferior quality of service
guarantees, i.e., messages getting lost or being
delivered out of order.
• Abysmal network performance and even overall
system instability.
There are multiple ways to improve your network
bandwidth capacity.

Superior University, Gold Campus

Distributed Systems Fallacies
 Bandwidth is Infinite

Comprehensive monitoring. It’s essential to track and

check usage in your network, so you can quickly identify
issues and take the appropriate remedial measures.
Multiplexing. Multiplexing a technique that improves
bandwidth utilization by allowing you to combine data
from several sources and send it over the same
communication channel/medium.

Superior University, Gold Campus

Distributed Systems Fallacies
 Bandwidth is Infinite

Lightweight data formats. You can preserve your network

bandwidth by using data exchange formats built for speed
and efficiency, like JSON (JavaScript Object Notation) is a
lightweight data-interchange format.. Another option
is MessagePack, a compact binary serialization format
that creates even smaller messages than JSON.
Network traffic control. You need to think about using
mechanisms such as throttling/rate limiting, congestion
control, exponential backoff, etc.

Superior University, Gold Campus

Distributed Systems Fallacies
 Network is secure
There are many ways a network can be attacked or compromised:
bugs, vulnerabilities in operating systems and libraries, unencrypted
communication, oversights that lead to data being accessed by
unauthorized parties, viruses and malware, cross-site scripting
(XSS), and DDoS attacks, to name just a few (but the list is endless).
Even though true (absolute) security in the world of distributed
computing is a fallacy, you should nonetheless do whatever is in
your power to prevent breaches and attacks when you design,
build, and test your system. The aim is for security incidents to
happen as infrequently as possible and to have limited impact and
ramifications.

Superior University, Gold Campus

Distributed Systems Fallacies
 Network is secure

Threat modeling. It’s advisable to use a structured process to

identify potential security threats and vulnerabilities, quantify the
seriousness of each, and prioritize techniques to mitigate attacks.
Defense in depth. You should use a layered approach, with different
security checks at network, infrastructure, and application level.
Security mindset. When designing your system, you should keep
security in mind and follow best practices and industry
recommendations and advice.

Superior University, Gold Campus

Distributed Systems Fallacies
 Topology does not change
In a nutshell, network topology refers to the manner in which the
links and nodes of a network are arranged and relate to each other.
In a distributed system, network topology changes all the time.
Sometimes, it’s for accidental reasons or due to issues, such as a
server crashing. Other times it’s deliberate — we add, upgrade, or
remove servers.
As you design your distributed system, it’s important to not rely on
the network topology being consistent and not expect it to behave
in a certain way at all times. There are numerous types of network
topologies you may choose to use, each with its own advantages
and disadvantages.

Superior University, Gold Campus

Distributed Systems Fallacies
 Topology does not change

For example, in a ring topology, each node connects to precisely

two other nodes. Data travels from node to node, with each of
them handling every packet. Some of the advantages of using a ring
topology:
• You do not require a central node to manage connectivity.
• It’s relatively easy to reconfigure.
• It can be made full-duplex (dual ring topology), allowing data to
flow in both directions, clockwise and counterclockwise.
On the downside, a single ring topology is quite vulnerable to
failure. If one node goes down, it can take down the entire network.

Superior University, Gold Campus

Distributed Systems Fallacies
 There is One Administrator
There might be only one administrator in the case of a very small
system, or perhaps in the context of a personal project. Beyond
that, there is usually more than one administrator of a distributed
system in nearly all real-life scenarios. For example, think of modern
cloud-native systems that consist of many services developed and
managed by different teams. Or consider that customers that use
your system also need administrators on their side to manage the
integration.
When you engineer your system, you should make it easy (well, as
easy as possible) for different administrators to manage it. You also
need to think about the system's resiliency and make sure it's not
impacted by different people interacting with it.

Superior University, Gold Campus

Distributed Systems Fallacies
 Transport Cost is Zero
Just as latency isn’t zero, transporting data from one point to
another has an attached cost, which is not at all negligible.
First of all, networking infrastructure has a cost. Servers, network
switches, load balancers, proxies, firewalls, operating and
maintaining the network, making it secure, not to mention staff to
keep it running smoothly — all of these cost money. The bigger the
network, the bigger the financial cost.
In addition to finances, we must also consider the time, effort, and
difficulty involved in architecting a distributed system that works
over a highly available, dependable, and fault-tolerant network. It’s
often less risky, simpler, and more cost-effective to offload this
complexity to a fully managed and battle-tested solution that’s
designed specifically for this purpose.
Superior University, Gold Campus
Distributed Systems Fallacies
 The Network is Homogeneous

Often, not even your home network is homogenous. It’s enough to

have just two devices with different configurations (e.g., laptops or
mobile devices) and using different transport protocols, and your
network is heterogeneous.
Most distributed systems need to integrate with multiple types of
devices, adapt to various operating systems, work with different
browsers, and interact with other systems. Therefore, it’s critical to
focus on interoperability, thus ensuring that all these components
can “talk” to each other, despite being different.

Superior University, Gold Campus