Distributed System
Distributed System
Distributed System
system
Definition
A distributed system is a collection of indepent
entities that cooperate to solve problem that
cannot be individually solved
processor
memory
Message passing
Message passing
processor
processor
memory
memory
Message passing
2. No shared memory
This is a key feature that requires message
passing for communication this feature
implies( )the absence()
Of the common physical clock
3. Geographical separation
Its not necessary for the processors to be on a
wide area network(WAN) recently the
Network/cluster of work station(NOW/COW)
configuration connecting processor on LAN
The NOW configuration is becoming popular
because of the low-cost and high speed. The
google search engine based on the NOW
architecture
4. Autonomy and
heterogeneity
The processors are loosely coupled in that they
have different speeds and each can be running a
different operating system but they cooperate
with another by offering services or solving a
problem
2. Resource sharing()
Resources such as peripherals, complete data sets in
databases, special libraries, as well as data (variable/files)
cannot be fully replicated at all the sites because it is often
neither practical Therefore, such resources are typically
distributed across the system. For example, distributed
databases such as DB2 partition the data sets across several
servers in addition to replicating them at a few sites
3. Access to geographically
remote data and resources
(
)
In many scenarios, the data cannot be replicated
at every site participating in the distributed
execution because it may be too large or too
sensitive to be replicated. For example, payroll
data within a multinational corporation is both too
large and too sensitive to be replicated at every
branch office/site. It is therefore stored at a
central server which can be queried by branch
offices. Similarly, special resources such as
supercomputers exist only in certain()
locations, and to access such supercomputers,
4. Enhanced reliability()
A distributed system has the inherent
potential( ) to provide increased
reliability because of the possibility of replicating
resources and executions, as well as ( ) the
reality that geographically distributed resources
are not likely to crash/malfunction( ) at
the same time under normal circumstances.
Reliability entails several aspects:
Availability: the resource should be accessible at
all times;
integrity: the value/state of the resource should
be correct
fault-tolerance:the ability to recover from
system failures
When the system meet one of the failure model
5. Increased performance/cost
ratio
(( /
By resource sharing and accessing geographically
remote data and resources, the performance/cost
ratio is increased. Although higher throughput has
not necessarily been the main objective behind
using a distributed system any task can be
partitioned across the various computers in the
distributed system
6. Scalability()
As the processors are usually connected by a
wide-area net- work, adding more processors does
not effected for the communication network.
7. Modularity and
incremental
expandability(
)
Heterogeneous processors may be easily added
into the system without affecting the
performance, as long as those processors are
running the same middleware algo- rithms.
Similarly, existing processors may be easily
replaced by other processors.
Types of Multiprocessors
There are two types of multiprocessors on the
way their memory is organized.
1.Tightly coupled multiprocessors.
2.loosely coupled multiprocessors.
1)Tightly coupled
Multiprocessor
systemscontain
multiple CPUs that are
connected at the bus
level. These CPUs
mayhave access to a
central shared memory
or may
participate( )in a
memory hierarchy with
both local and shared
memory (NUMA).
1)Looselycoupledmultiproce
ssor system(often
referred to as
clusters) are
basedon multiple
standalone()
single or dual
process or
interconnected
viaa high speed
communication
2) Tightly-coupled
(2) Loosely coupled
systems perform
system is just
better & are
Opposite of tightly
physically smaller
coupled system.
than loosely-coupled 3) Less expensive.
system
(4)In a loosely3) More expensive.
coupled system,
4) In a tightly-coupled
the opposite is
system, the delay
true: The interExperienced , when a machine message
message is sent from delay is large and
one computer to
the data rate is
another is short and
low.
data rate is high;
that is, the number
5) For example,
two CPU chips on
the same printed
circuit board and
connected by
wire etched onto
the board are
likely to be
tightly Coupled.
Relation to parallel
multiprocessor/multicomputer
systems
The main question is the system still a distributed
system, or does it become a parallel
multiprocessor system? To better answer these
questions, we first examine the architecture of
parallel systems
Characteristics of parallel
systems
A parallel system may be classified as belonging
to one of three types:
1. A multiprocessor system: is a parallel system in
which the multiple processors have direct access to
shared memory which forms a common address space.
The architecture is shown in the following Figure . Such
processors usually do not have a common clock.
pc1
pc2
pc3
Shared
memor
y
Characteristics of parallel
systems
A multiprocessor system usually corresponds
()to a uniform memory access (UMA)
architecture in which the access latency, waiting
time, to complete an access to any memory
location from any processor is the same.
The processors are in very close physical
proximity and are connected by an
interconnection network
The communication across processors is
traditionally through read and write operations on
the shared memory
the processors usually run the same operating
system, and both the hardware and software are
very tightly coupled.
Characteristics of parallel
systems
Examples of multiprocessor system
Omega network and butterfly(banyan)
network :
each of which is a multi-stage network formed of
2 2 switching elements. Each 2 2 switch
allows data on either of the two input wires to be
switched to the upper or the lower
output wire. In a single step, however, only one
data unit can be sent on an output wire. So if the
data from both the input wires is to be routed to
the same output wire in a single step, there is a
collision
Each 2 2 switch is represented as a rectangle in
the following figure
Characteristics of parallel
systems
Characteristics of parallel
systems
2. A multicomputer parallel system: Is a parallel
system in which the multiple processors do not
have direct access to shared memory. The
memory of the multiple processors may or may
not form a common address space
The processors are in close physical proximity and
are usually very tightly coupled (homogenous
hardware and software), and connected by an
interconnection network. Such computers usually
do not have a common clock
A multicomputer system that has a common
address space usually corresponds to a nonuniform memory access (NUMA) architecture in
which the latency to access various shared
memory locations from the different processors
Characteristics of parallel
systems
Examples of multicomputer system
Wrap-around 2D-mesh(torus) and Hypercube :
For a k k mesh which
will contain k2 processors, the maximum path
length between any two processors is 2(k/2 1).
While A k-dimensional hyper- cube has 2k
processor-and-memory units . Each such unit is a
node
in the hypercube, and has a unique k-bit label.
Each of the k dimensions is associated with a bit
position in the label. The labels of any two
adjacent( )nodes are identical( )except
for the bit position corresponding to the
dimension in which the two nodes differ.
Characteristics of parallel
systems
Characteristics of parallel
systems
3. Array processors: Belong to a class of parallel
computers that are physically very tightly
coupled, and have a common system clock (but
may not share memory and communicate by
passing data using messages).
The primary and most effective use of parallel
systems is for obtaining a higher throughput by
dividing the computational workload among the
processors. The tasks that are most
amenable( ) to higher speedups on
parallel systems
Flynns taxonomy
Flynn identified four processing
modes, based on whether the
processors execute the same or
different instruction streams at the
same time, and whether or not the
processors processed the same
(identical) data at the same time
OSI
Open Systems
Interconnect
The 7-layers
7. Application
6. Presentation
5. Session
4. Transport
3. Network
2. Data Link
1. Physical
Mnemonics
7.) (A)pplication
(A)ll
6.) (P)resentation
(P)eople
5.) (S)ession
(S)eem
4.) (T)ransport
(T)O
3.) (N)etwork
(N)eed
(D)ata
1.) (P)hysical
(P)rocessing
Division of Layers
7. Application
6. Presentation
Upper Layers
5. Session
4. Transport
Middle Layer
3. Network
2. Data Link
1. Physical
Lower Layers
7. Application N
1. Physical
C
A
P
S
U
L
A
T
I
O
Node b
7. Application
1.Physical
D
E
E
N
C
A
P
S
U
L
A
T
I
O
n
7-layers
Application layer:
In this layer the user will control on it directly
The Purpose of using this layer it is convert
User application to network service interface
Example E-mail services
Presentation Layer:
This layer response for Data formatting and
give it to the next layer in suitable way
The purpose of this layer is to formats data for
exchange between points of communication
Ex: Between nodes in a network
7-layers
Session Layer:
Sometimes called the dialog controller, this
layer establishes, maintains, and terminates
sessions between applications
This layer is response for define the devices
and its name and gives repot about the
connection
In this layer defines checkpoints for
acknowledgements during sessions between
applications
7-layers
Transport Layer:
in this layer the data will separate into
segments and it is Repackage proper(
)and efficient delivery of packages the
package should be
Without error
In sequence
Without duplication
7-layers
Network layer:
It is responsible to address messages and
translate the logical address to the physical
address that the network understand it
7-layers
Data Link Layer:
The purpose of this layer is Manages the flow of
data over the physical media and its responsible
for error-free transmission over the physical media
In this layer the data will be separate into smaller
parts called Frame
in this layer two parts will add into data first one
called Header the second one is called Trailer
header
Data
Trailer
7-layers
Physical Layer:
Purpose deals with the transmission of 0s and 1s
over the physical media
Translation of bits into signals
Computer Networks
LAN ,WAN
WAN
LAN
Large geographic
areas (e.g., cities,
states, nations)
Stands For
Covers
Definition
Computer Networks
LAN ,WAN
WANs have a lower
data transfer rate
.compared to LANs
Data transfer
rates
The Internet is a
good example of a
WAN
The network in an
office building can
be a LAN
Example
Computers
connected to a
wide-area network
are often connected
through public
networks, They can
also be connected
through leased lines
.or satellites
Layers 3 devices
Routers, Multi-layer
Switches
Connection
Components
Computer Networks
LAN ,WAN
WANs tend to be
less fault tolerant as
they consist of large
.number of systems
Experiences more
data transmission
errors as compared
to LAN
Experiences fewer
data transmission
errors
Typically owned,
controlled, and
managed by a
single person or
.organization
If there is a need to
set-up a couple of
Fault Tolerance
Data
Transmission
Error
Ownership
Set-up costs
Computer Networks
LAN ,WAN
Have a large
geographical range
generally spreading
across boundaries
and need leased
telecommunication
lines
Have a small
geographical range
and do not need
any leased
telecommunication
lines
Maintaining WAN is
difficult because of
its wider
geographical
coverage and
higher maintenance
.costs
Because it covers a
relatively small
geographical area,
LAN is easier to
maintain at
.relatively low costs
Low bandwidth is
available for
.transmission
High bandwidth is
available for
.transmission
More congestion
Less congestion
Geographical
Spread
Maintenance
costs
Bandwidth
)Congestion)
Ethernet
Ethernet is a certain type of a local area network (LAN)
which was developed in 1972 in the renowned
PARC-research facility of Xerox in Palo Alto by
Robert Metcalfe. In the meantime the companies
Intel, DEC and Xerox have specified a common
standard that has been established in the IEEEstandard 802.3.
Medium
Medium
Medium
Ethernet Technologies:
10Base2
Thin coaxial cable in a bus topology
Collision
CSMA/CD
Collision
Access method:
CSMA/CD
Back-Off Algorithm
If a collision has occurred, the stations
try to send again after a certain period
of time.
After the first collision there a two
different back-off times available, from
which one is chosen at random.
Transmission probability is 50%
After the second consecutive collision
there are four different back-off times
available, from which one is chosen at
random.
The transmission probability now is 75%
Segmentation
Segmentation
Segmentation
waiting for
result
Request
Server
Reply
provide
Time
service
Client Server
Architectures
The client server can be organize in two type of
machine
1. A client machine containing only the programs
that implementing on the user interface level
2. A server machine containing the rest that is the
programs implementing the processing and
data level
The problem with this organization is not
really distributed
Everything is handled by the server, while
the client is no more than a dumb terminal
User interface
REQUEST
operation
Application
Request
server
data
Database server
Return result
wait for data
Return
data
At first time the client that in user interface level will send request to the
sever to get the result
After sending request the client get suspend until the result get back
When the request reach to the Application server the server get resume
Call remote
Procedure
Server
time
Return
from call
Call local procedure
and return result
Asynchronous RPC
When a client calls a remote procedure the client will
block until a reply is returned this request-reply
behavior is unnecessary when there is no result to
return and only leads to blocking the client while it
could have proceeded and have done useful work
To support such situations,RPC system provide
facilities that called asynchronous RPCs by which
client immediately continues after issuing the
RPC request . With asynchronous RPCs the
server immediately sends a reply back to the
client the moment the RPC request is received
The reply acts as an acknowledgement to the
client
The client will continue without blocking as
soon as received the server acknowledgement
Asynchronous RPC
The following figure shows the interaction
between client and server using asynchronous
RPC
Client
Wait for acceptance
Call remote
Return from
call
procedure
Request
Server
Accept Request
call local
procedure
Time
Message-Oriented
Communication
There are two type of communication
Persistent versus transient
Synchronous versus asynchronous
Persistent: messages are held by the
middleware comm. service until they
can be delivered (e.g., email)
Message-Oriented
Communication
Asynchronous: (non-blocking) sender
resumes execution as soon as the message is
passed to the communication/middleware
software
Message-Oriented
Communication