ARYA GROUP OF COLLEGES, JAIPUR Distributed System CS/ VISem

ARYA GROUP OF COLLEGES

Arya College of Engineering & Research Centre, Jaipur

Department of
Computer Science & Engineering

Subject Name with code: Distributed System (DS) (6CS5-11)

UNIT-III-Distributed Scheduling

Presented by: Ms. Rashi Jain

Assistant Professor, CSE
12/04/2022 1
Ms.Rashi jain
ARYA GROUP OF COLLEGES JAIPUR Distributed System CS/VI Sem

UNIT-III

Distributed Process
Scheduling
2

12/04/2022
Ms.Rashi Jain
ARYA GROUP OF COLLEGES JAIPUR Distributed System CS VI Sem

SUBJECT DETAILED SYLLABUS

12/04/2022 3
Ms.Rashi Jain
ARYA GROUP OF COLLEGES JAIPUR Distributed System CS VI Sem

OUTLINE

• Introduction
• A System Performance Model
• Static Process scheduling with communication
• Dynamic load sharing and balancing
• Distributed process implementation

• Distributed file systems (DFS):

-Transparencies and characteristics of DFS
- DFS design and implementation
- Transaction service and concurrrency control
- Data and file replication

• Case Studies:
- Sun network file systems
- General parallel system and window’s file systems
- Andrew and coda file systems

12/04/2022 4
Ms.Rashi Jain
ARYA GROUP OF COLLEGES JAIPUR Distributed System CS VI Sem

DISTRIBUTED PROCESS SCHEDULING

• The primary objective of scheduling is to enhance overall system performance metrics

such as process completion time and processor utilization.

• The existence of multiple processing nodes in distributed systems present a challenging

problem for scheduling processes onto processors and vice versa.

12/04/2022 5
Ms.Rashi Jain
 Distributed System CS VI Sem
ARYA GROUP OF COLLEGES JAIPUR

A SYSTEM PERFORMANCE MODEL

Partitioning a task into multiple processes for execution can result in a speedup of the total
task completion time.

The speedup factor S is a function.

S = F (Algorithm, System, Schedule)

Ms.Rashi Jain
12/04/2022 6
ARYA GROUP OF COLLEGES JAIPUR Distributed System CS VI Sem

SYSTEM MODEL CONT...

• C P T = concurrent processing time

• O C P T i d e a l = optimal concurrent processing time
• S i = the ideal speedup
• Sd = the degradation of the system due to actual implementation compared to an ideal system

S i can be rewritten as:

RC
Si = × n
RP
where m
Σ i=1 P i
R P = OSP T
and m
Σ Pi
i=1
R C = OCPT
i d e a l × n

Ms.Rashi Jain
12/04/2022
ARYA GROUP OF COLLEGES JAIPUR Distributed System CS VI Sem

SYSTEM MODEL CONT...

• RP is Relative Processing: how much loss of speedup is due to the substitution of the best sequential algorithm
by an algorithm better adapted for concurrent implementation.

• RC is the Relative Concurrency which measures how far from optimal the usage of the n-processor is. It reflects
how well adapted the given problem and its algorithm are to the ideal n-processor system. The final expression
for speedup S is:

S = (RC/RP) × (1/(1 + ρ)) × n

• The term ρ is called efficiency loss. It is a function of scheduling and the system architecture. It would be
decomposed into two independent terms: ρ = ρsched + ρsyst, but this is not easy to do since scheduling and the
architecture are interdependent. The best possible schedule on a given system hides the communication overhead
(overlapping with other computations).
Ms.Rashi Jain
12/04/2022 8
ARYA GROUP OF COLLEGES JAIPUR Distributed System CS VI Sem

SYSTEM MODEL CONT...

• The unified speedup model integrates three major components:

1. Algorithm development

2. System architecture

3. Scheduling policy
• With the objective of minimizing the total completion time (makespan) of a set of interacting processes.

• If processes are not constrained by precedence relations and are free to be redistributed or moved around
among processors in the system, performance can be further improved by sharing the workload.
 Statically - load sharing
 Dynamically - load balancing
Ms.Rashi Jain
12/04/2022 9
ARYA GROUP OF COLLEGES JAIPUR Distributed System CS VI Sem

WORKLOAD DISTRIBUTION

• Performance can be further improved by load distribution and

sharing.

• Load Sharing: Static Workload Distribution

I. Dispatch process to the idle processor statically on arrival.
II. Corresponding to processor- pool model.

• Load Balancing: Dynamic Workload Distribution

1. Migrate process dynamically from heavily loaded processor to lightly loaded processors.
2. Corresponding to migration workstation model.
12/04/2022 Ms.Rashi Jain 10
ARYA GROUP OF COLLEGES JAIPUR Distributed System CS VI Sem

FIRST WE MUST KNOW: WHAT IS LOAD BALANCING

AND LOAD SHARING??
Load Balancing:
The aim of load balancing products is to create
a distributed network where requests are evenly
split among various servers. 

Load Sharing:
Meanwhile, load sharing entails sending a
portion of the traffic to one server and another
portion elsewhere.

12/04/2022 Ms.Rashi Jain 11

 ARYA GROUP OF COLLEGES JAIPUR Distributed System CS VI Sem

STATIC LOAD BALANCING ALGORITHM

• In static algorithm the processes are assigned to the processors at
the compile time according to the performance of the nodes.

• Once the processes are assigned, no change or reassignment is

possible at the run time.

• Number of jobs in each node is fixed in static load balancing

algorithm. Static algorithms do not collect any information about
the nodes.

• The assignment of jobs is done to the processing nodes on the

basis of the following factors: incoming time, extent of resource
needed, mean execution time and inter-process
communications.

• Since these factors should be measured before the assignment, this

is why static load balance is also called probabilistic algorithm.
12/04/2022 Ms.Rashi Jain Fig: Model of Processing Node 12
 ARYA GROUP OF COLLEGES JAIPUR Distributed System CS VI Sem

SUB CLASSES OF SLBA

The static load balancing algorithms can be divided into two sub classes:
1. Optimal static load balancing.
2. Sub optimal static load balancing.

Optimal Static Load Balancing Algorithms:

• If all the information and resources related to a system are known Optimal static load
balancing can be done.
•It is possible to increase throughput of a system and to maximize the use of the resources by
optimal load balancing algorithm.
12/04/2022 13
Ms.Rashi Jain
 ARYA GROUP OF COLLEGES JAIPUR Distributed System CS VI Sem

SUB CLASSES OF SLBA CONT...

Sub-Optimal Static Load Balancing Algorithm:

•Sub-optimal load balancing algorithm will be mandatory for some applications when
optimal solution is not found.

•The thumb-rule and heuristics methods are important for sub-optimal algorithm.

12/04/2022 Ms.Rashi Jain 14

ARYA GROUP OF COLLEGES JAIPUR Distributed System CS VI Sem

DYNAMIC LOAD BALANCING (DLB)

• In dynamic load balancing algorithm assignment of jobs is done at the runtime.

• In DLB jobs are reassigned at the runtime depending upon the situation that is the load
will be transferred from heavily loaded nodes to the lightly loaded nodes.

• In this case communication over heads occur and becomes more when number of
processors increase.

• In dynamic load balancing no decision is taken until the process gets execution.

12/04/2022 Ms.Rashi Jain 15

 ARYA GROUP OF COLLEGES JAIPUR Distributed System CS VI Sem

DYNAMIC LOAD BALANCING (DLB) CONT...

• This strategy collects the information about
the system state and about the job
information.

• As more information is collected by an

algorithm in a short time, potentially the
algorithm can make better decision.

• Dynamic load balancing is mostly considered

in heterogeneous system because it consists of
nodes with different speeds, different
communication link speeds, different memory
sizes, and variable external loads due to the
multiple. Fig: Job Migration in Dynamic Load Balancing Strategy
12/04/2022 Ms.Rashi Jain 16
 ARYA GROUP OF COLLEGES JAIPUR Distributed System CS VI Sem

COMPARISON OF SLB AND DLB

.

12/04/2022 Ms.Rashi Jain 17

 ARYA GROUP OF COLLEGES JAIPUR Distributed System CS VI Sem

QUEUING THEORY
Queuing theory deals with problems which involve queuing (or waiting). Typical examples might be:

• banks/supermarkets - waiting for service

• computers - waiting for a response
• failure situations - waiting for a failure to occur e.g. in a piece of machinery
• public transport - waiting for a train or a bus

Performance of system described as queuing models can be computed using queuing theory. An x/y/z queue is one where:

• X: Arrival process
• Y: Service time distribution
• Z: Number of servers

•  λ: arrival rate μ: service rate

• γ : Migration rate which depend on the channel bandwidth, migration protocol, context and state information of the process being
transferred.
12/04/2022 Ms.Rashi Jain 18
ARYA GROUP OF COLLEGES JAIPUR Distributed System CS VI Sem

QUEUING CONT...

12/04/2022 Ms.Rashi Jain 19

ARYA GROUP OF COLLEGES JAIPUR Distributed System CS VI Sem

STATIC PROCESS SCHEDULING

• Scheduling a set of partially ordered tasks on a non-preemtive multiprocessor system of

identical processors to minimize the overall finishing time (makespan).
• Except for some very restricted cases scheduling to optimize makespan is NP-complete.

• A good heuristic distributed scheduling algorithm is one that can best balance and overlap
computation and communication.

• In static scheduling, the mapping of processes to processors is determined before the

execution of the processes. Once a process is started, it stays at the processor until
completion.
12/04/2022 Ms.Rashi Jain 20
ARYA GROUP OF COLLEGES JAIPUR Distributed System CS VI Sem

PRECEDENCE PROCESS MODEL

• Program is represented by a directed acyclic graph (DAG)

• Computational model
• Primary objective of task scheduling is to achieve maximal concurrency for task execution within a
program

12/04/2022 Ms.Rashi Jain 21

ARYA GROUP OF COLLEGES JAIPUR Distributed System CS VI Sem

PRECEDENCE PROCESS MODEL CONT...

• Finding the minimum makespan is NP-complete, so we will rely on heuristic algorithms for finding good mapping of
the process model to the system model.

• For precedence process graphs, the notion of critical path is useful - the longest execution path in the DAG, which
is the lower bound of the makespan. Simple heuristic: map all tasks in a critical path onto a single processor.

1.List Scheduling (LS) strategy: No processor remains idle if there are some tasks available that it could process
(without considering communication overhead).

2.Extended List Scheduling (ELS) strategy: Allocating tasks to processors according to LS and adding
communication delays, communication overhead.

3.Earliest Task First (ETF): The earliest schedulable task is scheduled first (calculation includes communication
overhead).
12/04/2022 Ms.Rashi Jain 22
ARYA GROUP OF COLLEGES JAIPUR Distributed System CS VI Sem

PRECEDENCE PROCESS MODEL CONT...

• Process scheduling for many system applications has a perspective very different
from precedence model – applications may be created independently, processes do not
have explicit completion time and precedence constraints.
• Primary objectives of process scheduling are to maximize resource utilization and to
minimize inter process communication.
• Communication process model is an undirected graph G with node and edge sets V
and E, where nodes represent processes and the weight on an edge is the amount of
interaction between two connected processes.
12/04/2022 Ms.Rashi Jain 23
ARYA GROUP OF COLLEGES JAIPUR Distributed System CS VI Sem

PRECEDENCE PROCESS MODEL CONT...

12/04/2022 Ms.Rashi Jain 24

ARYA GROUP OF COLLEGES JAIPUR Distributed System CS VI Sem

DYNAMIC LOAD SHARING AND LOAD BALANCING

Objective of scheduling: utilization of the system (has direct bearing on throughput and completion time)
and fairness to the user processes (difficult to define).

• If we can designate a controller process that maintains the information about the queue size of each
processor:

• Fairness in terms of equal workload on each processor (join the shortest queue) - migration workstation
model (use of load sharing and load balancing, perhaps load redistribution i.e. process migration)

• Fairness in terms of user’s share of computation resources (allocate processor to a waiting process at a user
site that has the least share of the processor pool) - processor pool model

• Solutions without a centralized controller: sender- and receiver-initiated algorithms.

12/04/2022 Ms.Rashi Jain 25
 ARYA GROUP OF COLLEGES JAIPUR Distributed System CS VI Sem

CONT...
Sender-initiated algorithms:
• push model
• includes probing strategy for finding a node with the smallest queue length (perhaps multicast)
• performs well on a lightly loaded system

Receiver-initiated algorithms:
• pull model
• probing strategy can also be used
• more stable
• perform on average better

• Combinations of both algorithms are possible: choice based on the estimated system load information or reaching threshold values of the
processing node’s queue.
12/04/2022 Ms.Rashi Jain 26
ARYA GROUP OF COLLEGES JAIPUR Distributed System CS VI Sem

PERFORMANCE COMPARISON

12/04/2022 Ms.Rashi Jain 27

ARYA GROUP OF COLLEGES JAIPUR Distributed System CS VI Sem

DISTRIBUTED PROCESS IMPLEMENTATION

Three significant application scenarios:

• Remote service: The message is interpreted as a
request for a known service at the remote site
(constrained only to services that are supported at the
remote host)
1. Remote procedure calls at the language level

2. Remote commands at the operating system

level

3. Interpretive messages at the application level

12/04/2022 Ms.Rashi Jain 28

ARYA GROUP OF COLLEGES JAIPUR Distributed System CS VI Sem

DISTRIBUTED PROCESS IMPLEMENTATION CONT...

• Remote execution: The messages contain a program to be executed at the remote site; implementation
issues:
– load sharing algorithms (sender-initiated, registered hosts, broker...)

– location independence of all IPC mechanisms including signals

– system heterogeneity (object code, data representation)

– protection and security

• Process migration: The messages represent a process being migrated to the remote site for continuing
execution (extension of load-sharing by allowing a remote execution to be preempted)

12/04/2022 Ms.Rashi Jain 29

ARYA GROUP OF COLLEGES JAIPUR Distributed System CS VI Sem

DISTRIBUTED PROCESS IMPLEMENTATION CONT...

State information of a process in distributed systems consists of two parts: computation

state (similar to conventional context switching) and communication state (status of the
process communication links and messages in transit). The transfer of the communication
state is performed by link redirection and message forwarding.

12/04/2022 Ms.Rashi Jain 30

ARYA GROUP OF COLLEGES JAIPUR Distributed System CS VI Sem

DISTRIBUTED FILE SYSTEMS (DFS)

 A Distributed File System (DFS) is a file system that supports sharing of files and resources in the form of persistent
storage over a network .

 First file servers were developed in the 1970s.

 First widely used distributed file system was Sun's Network File System (NFS) introduced in 1985.

 Examples of distributed file systems: Andrew File System (CMU), CODA (CMU), Google File System(Google).

12/04/2022 Ms.Rashi Jain 31

ARYA GROUP OF COLLEGES JAIPUR Distributed System CS VI Sem

FILE SERVICE ARCHITECTURE

 An architecture that offers Client computer Server computer

a clear separation of the
main concerns in Application Application Directory service
providing access to files is program program
obtained by structuring the
file service as three
components:
Flat file service
 A flat file service
 A directory service Client module

 A client module.

12/04/2022 Ms.Rashi Jain 32

ARYA GROUP OF COLLEGES JAIPUR Distributed System CS VI Sem

FILE SERVICE ARCHITECTURE

 The Client module implements exported interfaces by flat file and directory services on server side.
 Responsibilities of various modules can be defined as follows:

 Flat file service:

 Concerned with the implementation of operations on the contents of file. Unique File
Identifiers (UFIDs) are used to refer to files in all requests for flat file service operations.
UFIDs are long sequences of bits chosen so that each file has a unique among all of the
files in a distributed system.
 Directory service:
 Provides mapping between text names for the files and their UFIDs. Clients may obtain
the UFID of a file by quoting its text name to directory service. Directory service
supports functions needed generate directories, to add new files to directories.

12/04/2022 Ms.Rashi Jain 33

ARYA GROUP OF COLLEGES JAIPUR Distributed System CS VI Sem

FILE SERVICE ARCHITECTURE

Client module:
 It runs on each computer and provides integrated service (flat file and
directory) as a single API to application programs. For example, in
UNIX hosts, a client module emulates the full set of Unix file
operations.

 It holds information about the network locations of flat-file and

directory server processes; and achieve better performance through
implementation of a cache of recently used file blocks at the client.

12/04/2022 Ms.Rashi Jain 34

ARYA GROUP OF COLLEGES JAIPUR Distributed System CS VI Sem

FILE OPERATIONS

-Read(FileId, i, n) -> Data If 1≤i≤Length(File): Reads a sequence of up to n items

-throws Bad Position From a file starting at item i and returns it in Data.

-Write(FileId, i, Data) If 1≤i≤Length(File)+1: Write a sequence of Data to a

-throws BadPosition File, starting at item i, extending the file if necessary.

-Create() -> FileId Creates a new file of length0 and delivers a UFID for it.
-Delete(FileId) Removes the file from the file store.
-GetAttributes(FileId) -> Attr Returns the file attributes for the file.
-SetAttributes(FileId, Attr) Sets the file attributes (only those attributes that are not
shaded .)

12/04/2022 Ms.Rashi Jain 35

ARYA GROUP OF COLLEGES JAIPUR Distributed System CS VI Sem

CHARACTERISTICS OF FILE SYSTEMS

 File systems are responsible for the organization, storage, retrieval, naming, sharing and protection of fi
les.
 Files contains both data and attributes
 File systems are designed to store and manage large numbers of files, with facilities for creating, namin
g and deleting files

12/04/2022 Ms.Rashi Jain 36

ARYA GROUP OF COLLEGES JAIPUR Distributed System CS VI Sem

FILE ATTRIBUTE RECORD STRUCTURE

12/04/2022 Ms.Rashi Jain 37

ARYA GROUP OF COLLEGES JAIPUR Distributed System CS VI Sem

DESIGN & IMPLEMENTATION

• It is one of the two important components (process and file) in any distributed computation.
• It is a good example for illustrating the concept of transparency and client/server model.

• File sharing and data replication present many interesting research problems.

• File: sequential, direct, indexed, indexed-sequential

• File System: flat, hierarchical

12/04/2022 Ms.Rashi Jain 38

ARYA GROUP OF COLLEGES JAIPUR Distributed System CS VI Sem

12/04/2022 Ms.Rashi Jain 39

ARYA GROUP OF COLLEGES JAIPUR Distributed System CS VI Sem

TRANSACTION AND CONCURRENCY CONTROL

• The conventional definition of a transaction ACID properties:

• Atomicity – either all tasks in a transaction are performed, or none of them are;
• Consistency – data is in a consistent state when the transaction begins, and when it ends;
• Isolation – all operations in a transaction are isolated from operations outside the transaction;
• Durability – upon successful completion, the result of the transaction will persist.
• Transaction processing system (TPS)
• Transaction manager (TM)
• Scheduler (SCH)

• Object manager (OM)

• Atomicity
• All or none: TM, two-phase commit
• Indivisible (serializable): SCH, concurrency control protocols

• Atomic update: OM, replica management

12/04/2022 Ms.Rashi Jain 40
 ARYA GROUP OF COLLEGES JAIPUR Distributed System CS VI Sem

TRANSACTION AND CONCURRENCY CONTROL

Several concurrency control mechanisms are available for maintaining consistency of data items such as:
turn-taking, serialization , transactional locking mechanism , and operational transformation . Lock
mechanisms, as a widely used method for concurrency control in transaction models , provide enough isolation
on modified data items (Exclusive lock) to ensure there is no access to any of these data items before a
transaction that is accessing or updating them commits .
12/04/2022 Ms.Rashi Jain 41
 ARYA GROUP OF COLLEGES JAIPUR Distributed System CS VI Sem

DATA AND FILE REPLICATION

Data replication is a service responsible for
copying/duplicating files and folders from a primary
storage (production) to a secondary storage (replica).

File level replication is asynchronous, so files

schedule/time interval changes, are collected and
replicated to the secondary are first written to the
primary storage and then, based on a defined storage
unit.

Write operation
Because data is copied with latency, primary storage Read operations
• Write-one-primary
• Read-one-primary
failure can cause data loss if most recent modifications • Read-one
• Write-all

were not transferred to the secondary storage. • Read-quorum

• Write-all-available
• Write-quorum
Architecture of a replica manager: • Write-gossip
12/04/2022 Ms.Rashi Jain 42
 ARYA GROUP OF COLLEGES JAIPUR Distributed System CS VI Sem

DATA AND FILE REPLICATION

Data replication is a service responsible for
copying/duplicating files and folders from a primary
storage (production) to a secondary storage (replica).

File level replication is asynchronous, so files

schedule/time interval changes, are collected and
replicated to the secondary are first written to the
primary storage and then, based on a defined storage
unit.

Write operation
Because data is copied with latency, primary storage Read operations
• Write-one-primary
• Read-one-primary
failure can cause data loss if most recent modifications • Read-one
• Write-all

were not transferred to the secondary storage. • Read-quorum

• Write-all-available
• Write-quorum
Architecture of a replica manager: • Write-gossip
12/04/2022 Ms.Rashi Jain 43
ARYA GROUP OF COLLEGES JAIPUR Distributed System CS VI Sem

CASE STUDIES

Data replication is a service responsible for copying/duplicating files and folders from a

primary storage (production) to a secondary storage (replica).

File level replication is asynchronous, so files schedule/time interval changes, are collected
and replicated to the secondary are first written to the primary storage and then, based on a
defined storage unit.

Because data is copied with latency, primary storage failure can cause data loss if most
recent modifications were not transferred to the secondary storage.
Architecture of a replica manager:

12/04/2022 Ms.Rashi Jain 44

 ARYA GROUP OF COLLEGES JAIPUR Distributed System CS VI Sem

THANKS & REGARDS....

12/04/2022 45
Ms.Rashi Jain