Parallel &

Distributed
Computing
SALAH-UD-DIN AYUBI
ASSOCIATE PROFESSOR
GPGC ABBOTTABAD
 Distributed Computing System
The working definition we use for a
distributed computing system states that
itis a number of autonomous processing
elements (not necessarily homogeneous) that are
interconnected by a computer network and that
cooperate in performing their assigned tasks.
The “processing element” are actually a
computing device that can execute a program
on its own.
 Distributed Computing System
 What is being distributed?
 Processing Logic i.e., processing logic or processing
elements
 Functions of computer system delegated to various
pieces of hardware or software.
 Data used by a number of applications, distributed
over various sites.
 Control of execution of various tasks, distributed
among different sites.
 Distributed Computing System
 Why do we distribute at all?
 Because Distributed processing better corresponds to the
organizational structure of today’s widely distributed
enterprises.
 Distributed applications structures, such as
Web-based applications,
electronic commerce business over the Internet
 multimedia applications such as news-on-demand or medical
imaging,
manufacturing control systems
 Goals of Distributed System
Four basic goals to achieve through
distribution
Making Resources Accessible
Distribution Transparency
Openness
Scalability
 Goals of Distributed System
 Making Resources Accessible:
 Make remote resource accessible to users to share in
a controlled way
 Resources can be hardware (printer, scanner,
computer etc.) or software (data, file, webpages etc.)
 making resource accessible bring a lot of benefits
for users e.g.,
 Collaboration through emails, exchanging files,
videos, audio, and documents.
 Goals of Distributed System
 Making Resources Accessible:
 With increased connectivity and sharing issue
related to security also arises.
 Users’ data like password, credit card numbers, may
be eavesdropped
 Privacy may compromise by accessing the personal
profile, medical record, contacts details etc.
 Unwanted communication
 Goals of Distributed System
Distribution Transparency:
A distributed system that is able to present itself to
users and applications as if it were only a single
computer system is said to be transparent.
 Goalis to hide the fact that the processes and
resources are physically distributed
 Differenttypes of transparency required to exist in
a distributed system
 Goals of Distributed System
Distribution Transparency:
Transparenc Description
y
Access Hide differences in data representation and how
a resource is accessed
Location Hide where a resource is located
Migration Hide that resource may move to another location
Relocation Hide that resource may move to another location
while in use
Replication Hide that resource is replicated
Concurren Hide that resource may be shared by various
cy competitive users.
 Goals of Distributed System
Distribution Transparency:
Different operating systems with data
Representations, file naming convention, data
manipulation (Little endian, Big Endian)
No clue about the web server of a website
No clue about physical existence of a home page if
moved or remains there always.
Application does not know the movement of resource
while its operation execution (e.g., mobile users)
 Goals of Distributed System
Distribution Transparency:
Hiding the fact that several copies/replicas of
same resources are serving the users.
To achieve this all replicas should have same
name.
Concurrent user should not know that someone else
is also using the same resource.
For example, two users ticket booking software
Masking a failure in a way that the failure of
resource is not noticed by the user
 Goals of Distributed System
Openness:
 Systemis offering the services in according to
standard rules having certain Syntax and Semantics
 Communication protocols, services provided on
standard interfaces
 Definition
of interfaces through Interface Definition
Language IDL
 Interface
is used for process-to-process
communication
 Goals of Distributed System
 Openness:
 Two different processes use same interface
 Interface definition should be complete and neutral
 Completeness and neutrality provides interoperability
and portability
 Interoperabilityis extent to which Two separate
implementation of a system co-exist and work together.
 Portabilitymeans application developed for system A
can be used for system B without modification
 Goals of Distributed System
Openness:
 Easy to configure
 Easyto extend, means changing the components
should not affect existing one.
 Goals of Distributed System
Scalability:
 It has at-least three dimensions.
 In size means easily add more users or resources
 Geographically users and resources lie far apart
 Administratively it can be easy to manage the
system if it spans separate organizations.
 System scalability in any of these dimension
exhibit loss of performance.
 Goals of Distributed System
 Scalability:
 Achieving scalability may arise some problems
 For a centralized system increase in number of users
may lead to a bottleneck situation on server running
services, data and algorithm
 Solution to this problem is distribute the server
 Sometimes centralized server is unavoidable.
 e.g.,
confidential information like medical records, bank
accounts etc.
 Goals of Distributed System
Scalability:
 Centralized service also raises problems e.g.,
 Domain Name Service (DNS) in a single table.
 Centralized algorithm is also not a good idea.
 Collecting complete routing information from
different routes and sending it to a central server
to update routing information and then
disseminating throughout the network
 Types of Distributed Systems

 TwoBroad categories
 High Performance
Computing (HPC)
 Supercomputers with
Massively Parallel
Processors (MPP)
 Cluster computing
 High-throughput
Computing (HTC)
 Peer-to-Peer (P2P)
networks for file
sharing and Content
Delivery network.
 Types of Distributed Systems
Distributed Computing System:
Cluster Computing: multiple computers closely
connected with each other through high-speed
LAN having same operating system.
Grid Computing: federation of computer
Systems under different administrative control
with very different hardware, software and
network technology.
 Distributed Computing System
Cluster Computing:
Cluster computing is used for parallel
programming in which single program is run
on multiple machines.
Master Node manages Other nodes Provides
interface to system
Runs middleware
 Distributed Computing System
 Grid Computing:
 Grid computing has a high degree of
heterogeneity.
 No assumptions are made regarding hardware,
operating system, networks, administrative
domain and security policy etc.
 Collaboration among different people or
institutions form a virtual organization.
 Specialized software have been evolved to realize
the Grid architecture.
 Distributed Computing System
Grid Computing:
These software bring
administrative
organizations and users
related to those grids
close together to work as
virtual organization.
Thus, most of the efforts
are put together to have a
full supportive
architecture.
 Distributed Computing System
 Grid Computing:
 Fabric layer: provides
interface to local resource at a
specific site.
 Actually, interfaces for
sharing of resources within
virtual organization.
 Functions for querying the
state and capabilities of
resources along with functions
for actual resource
management (e.g., locking
mechanism).
 Distributed Computing System
 Grid Computing:
 Connectivity layer: consists of
communication protocol for
supporting grid transactions
spanning multiple resources.
 E.g., transferring data among
resources.
 Access a resource from remote
location
 Authentication of users and
resources.
 Actual users are not
authenticated rather programs
working on behalf of user are
authenticated.
 Distributed Computing System
 Grid Computing:
 Resource Layer: responsible
for managing a single
resource.
 Uses function of connectivity
layer to directly call the
interface at fabric layer.
 Offer functions to perform
specific operation e.g.,
creating process or reading
data or resource
configuration.
 Distributed Computing System
 Grid Computing:
 Collective layer: deals with
handling access to multiple
resources. E.g.,
 Resource discovery
 Allocation and scheduling of tasks on
multiple resources
 Data replication
 Connectivity layer consists of small
standard collection of protocol
 Collective layer consists of many
different protocols for different
purposes to support broad
spectrum of services
 Distributed Computing System
 Grid Computing:
 Application layer: consists of
applications operate within
virtual organization to ensure
grid computing environment.

 Collective,connectivity and
resource layers are collectively
middleware for grid computing.
 Middleware means grid is a
common site.
 Now access to different layer is
also provided at grid (OGSA)
 Types of Distributed Systems
 Distributed embedded system
 Distributed Pervasive computing is best
representation of distributed embedded system
 Home Systems
A transition from stable to instable system
 Small, battery powered, mobile having wireless
connectivity only.
 By the time these restriction are going to be
nullified by advanced computing devices.
 Types of Distributed Systems
Distributed embedded system
Important feature: lack of human
administrative control
However, owner can configure them
Or they can automatically discover their
environment
 Types of Distributed Systems
 Distributed embedded system
 Pervasive application have following requirements:
 Embrace Contextual Changes
 Devicemust know that environment is continuously
changing all the time
 Encourage Ad-hoc composition
 Easy configuration for a suite of applications
 Recognize sharing as the default
 Easy to read, store, manage and share information
 Types of Distributed Systems
 Distributed embedded system
 Home Systems
 Consist of one or more personal computers (TVs,
audio and video equipment, gaming devices,
smart phones, PDAs, and other personal
wearables, kitchen appliances, surveillance
cameras, clocks, controllers for lighting.
 UPnP (update without human interventions)
 Recommender programs
 Types of Distributed Systems
 Distributed embedded system
 Health care Systems
 Devices related to electronic health care
 Sensor organized in BAN (Wireless)
 Minimal hindrance in human movements
 Continuous data monitoring and automatic
reporting to physicians or to a large storage
 Aggregation of data is required before saving to
a server
 Types of Distributed Systems
 Distributed embedded system
 Health care Systems
 Where and how should monitored data be stored?
 How can we prevent loss of crucial data?
 What infrastructure is needed to generate and propagate
alerts?
 How can physicians provide online feedback?
 How can extreme robustness of the monitoring system be
realized?
 What are the security issues and how can the proper
policies be enforced?
 Types of Distributed Systems
 Distributed embedded system
 Sensor Networks
 They are used for processing information.
 A mesh network
 The questions to be addressed:
 Howdo we (dynamically) set up an efficient tree in a
sensor network?
 How does aggregation of results take place? Can it be
controlled?
 What happens when network links fail?
 Types of
Distributed
Systems
Distribute
d
embedded
system
Sensor
Networks