6 Replication Nhom3

Principles of Distributed Database
Systems
TS. Phan Thị Hà
© 2020, M.T. Özsu & P. Valduriez TS.

Phan Thị Hà 1
Outline
◼ Introduction
◼ Distributed and parallel database design
◼ Distributed data control
◼ Distributed Transaction Processing
◼ Data Replication
◼ Database Integration – Multidatabase Systems
◼ Parallel Database Systems
◼ Peer-to-Peer Data Management
◼ Big Data Processing
◼ NoSQL, NewSQL and Polystores
◼ Web Data Management
2
Phan Thị Hà
Outline
❑ Consistency criteria
❑ Update Management Strategies
❑ Replication Protocols
❑ Replication and Failure Management

3
Phan Thị Hà
Replication
◼ Why replicate?
❑ System availability
◼ Avoid single points of failure
❑ Performance
◼ Localization
❑ Scalability
◼ Scalability in numbers and geographic area
❑ Application requirements
◼ Why not replicate?
❑ Replication transparency
❑ Consistency issues
◼ Updates are costly
◼ Availability may suffer if not careful

4
Phan Thị Hà
Execution Model
◼ There are physical copies of logical objects in the system.

◼ Operations are specified on logical objects, but translated to operate
on physical objects.
◼ One-copy equivalence
❑ Transaction effects on replicated objects should be the same as if they
had been performed on a single set of objects.
Write(x)
x Logical data item
Write(x1) Write(x2) Write(xn)
x1 x2 … xn
Physical data item (replicas, copies)

5
Phan Thị Hà
Replication Issues
◼ Consistency models - how do we reason about the

consistency of the “global execution state”?
❑ Mutual consistency
❑ Transactional consistency
◼ Where are updates allowed?
❑ Centralized
❑ Distributed
◼ Update propagation techniques – how do we propagate
updates to one copy to the other copies?
❑ Eager
❑ Lazy

6
Phan Thị Hà
Outline
❑ Consistency criteria
❑

7
Phan Thị Hà
Consistency
◼ Mutual Consistency
❑ How do we keep the values of physical copies of a logical data
item synchronized?
❑ Strong consistency
◼ All copies are updated within the context of the update transaction
◼ When the update transaction completes, all copies have the same
value
◼ Typically achieved through 2PC
❑ Weak consistency
◼ Eventual consistency: the copies are not identical when update
transaction completes, but they eventually converge to the same
value
◼ Many versions possible:
❑ Time-bounds
❑ Value-bounds
❑ Drifts
8
Phan Thị Hà
Transactional Consistency
◼ How can we guarantee that the global execution history

over replicated data is serializable?
◼ One-copy serializability (1SR)
❑ The effect of transactions performed by clients on replicated
objects should be the same as if they had been performed one
at-a-time on a single set of objects.
◼ Weaker forms are possible
❑ Snapshot isolation
❑ RC-serializability

9
Phan Thị Hà
Example 1
Site A Site B Site C

x x, y x, y, z
T1: x ← 20 T2: Read(x) T3: Read(x)
Write(x) x ← x+y Read(y)
Commit Write(y) z ← (x∗y)/100
Commit Write(z)
Commit
Consider the three histories:
HA={W1(xA), C1}
HB={W1(xB), C1, R2(xB), W2(yB), C2}
HC={W2(yC), C2, R3(xC), R3(yC),W3(zC), C3, W1(xC),C1}
Global history non-serializable: HB: T1→T2, HC: T2→T3→T1

Mutually consistent: Assume xA=xB=xC=10, yB=yC=15,yC=7 to begin; in the end
xA=xB=xC=20, yB=yC=35,yC=3.5
10
Phan Thị Hà
Example 2
Site A Site B
x x
T1: Read(x) T2: Read(x)

x ← x+5 x ← x∗10
Write(x) Write(x)
Commit Commit
Consider the two histories:
HA={R1(xA),W1(xA), C1, R2(xA), W2(xA), C2}

HB={R2(xB), W2(xB), C2, R1(xB), W1(xB), C1}
Global history non-serializable: HA: T1→ T2, HB: T2→ T1

Mutually inconsistent: Assume xA=xB=1 to begin; in the end xA=15, xB=60

11
Phan Thị Hà
Outline
❑
❑ Update Management Strategies

❑

12
Phan Thị Hà
Update Management Strategies
◼ Depending on when the updates are propagated

❑ Eager
❑ Lazy
◼ Depending on where the updates can take place
❑ Centralized
❑ Distributed Centralized Distributed
Eager
Lazy

13
Phan Thị Hà
Eager Replication
◼ Changes are propagated within the scope of the transaction making the
changes. The ACID properties apply to all copy updates.
❑ Synchronous
❑ Deferred
◼ ROWA protocol: Read-one/Write-all
Transaction
updates commit



Site 1 Site 2 Site 3 Site 4

14
Phan Thị Hà
Lazy Replication
● Lazy replication first executes the updating transaction on one copy. After
the transaction commits, the changes are propagated to all other copies
(refresh transactions)
● While the propagation takes place, the copies are mutually inconsistent.
● The time the copies are mutually inconsistent is an adjustable parameter
which is application dependent.
Transaction
updates commit
 


15
Phan Thị Hà
Centralized
● There is only one copy which can be updated (the master), all others
(slave copies) are updated reflecting the changes to the master.

16
Phan Thị Hà
Distributed
● Changes can be initiated at any of the copies. That is, any of the
sites which owns a copy can update the value of the data item.
Transaction
updates commit
Transaction
updates commit

17
Phan Thị Hà
Forms of Replication
Eager Centralized
+ No inconsistencies (identical copies) + No inter-site synchronization is
+ Reading the local copy yields the most necessary (it takes place at the
up to date value master)
+ Changes are atomic + There is always one site which has
− A transaction has to update all sites all the updates
− Longer execution time
− The load at the master can be high
− Lower availability
− Reading the local copy may not
Lazy yield the most up-to-date value
+ A transaction is always local (good
response time) Distributed
− Data inconsistencies + Any site can run a transaction
− A local read does not always return + Load is evenly distributed
the most up-to-date value − Copies need to be synchronized
− Changes to all copies are not
guaranteed
− Replication is not transparent
18
Phan Thị Hà
Outline
❑
❑ Replication Protocols
❑

19
Phan Thị Hà
Replication Protocols
The previous ideas can be combined into 4 different replication protocols:
Eager Eager centralized Eager distributed
Lazy Lazy centralized Lazy distributed
Centralized Distributed
20
Phan Thị Hà
Eager Centralized Protocols
◼ Design parameters:
❑ Distribution of master
◼ Single master: one master for all data items
◼ Primary copy: different masters for different (sets of) data items
❑ Level of transparency
◼ Limited: applications and users need to know who the master is
❑ Update transactions are submitted directly to the master
❑ Reads can occur on slaves
◼ Full: applications and users can submit anywhere, and the
operations will be forwarded to the master
❑ Operation-based forwarding
◼ Four alternative implementation architectures, only three
are meaningful:
❑ Single master, limited transparency
❑ Single master, full transparency
❑ Primary copy, full transparency
21
Phan Thị Hà
Eager Single Master/Limited
Transparency
◼ Applications submit update transactions directly to the master
◼ Master:
❑ Upon read: read locally and return to user
❑ Upon write: write locally, multicast write to other replicas (in FFO timestamps order)
❑ Upon commit request: run 2PC coordinator to ensure that all have really installed the
changes
❑ Upon abort: abort and inform other sites about abort
◼ Slaves install writes that arrive from the master

22
Phan Thị Hà
Eager Single Master/Limited
Transparency (cont’d)
◼ Applications submit read transactions directly to an appropriate slave
◼ Slave
❑ Upon read: read locally
❑ Upon write from master copy: execute conflicting writes in the proper order
(FIFO or timestamp)
❑ Upon write from client: refuse (abort transaction; there is error)
❑ Upon commit request from read-only: commit locally
❑ Participant of 2PC for update transaction running on primary

23
Phan Thị Hà
Eager Single Master/Full Transparency
Applications submit all transactions to the Transaction Manager at their
own sites (Coordinating TM)
Coordinating TM Master Site
1. Send op(x) to the master site 1. If op(x) = Read(x): set read lock on x
and send “lock granted” msg to the
coordinating TM
2. If op(x) = Write(x)
2. Send Read(x) to any site that has x
1. Set write lock on x
2. Update local copy of x
3. Inform coordinating TM
3. Send Write(x) to all the slaves
where a copy of x exists
4. When Commit arrives, act as
coordinator for 2PC
3. Act as participant in 2PC
2
Eager Primary Copy/Full Transparency
◼ Applications submit transactions directly to their local TMs

◼ Local TM:
❑ Forward each operation to the primary copy of the data item
❑ Upon granting of locks, submit Read to any slave, Write to all slaves
❑ Coordinate 2PC

25
Phan Thị Hà
Eager Primary Copy/Full Transparency
(cont’d)
◼ Primary copy site
❑ Read(x): lock xand reply to TM
❑ Write(x): lock x, perform update, inform TM
❑ Participate in 2PC
◼ Slaves: as before

26
Phan Thị Hà
Eager Distributed Protocol
◼ Updates originate at any copy

❑ Each sites uses 2 phase locking.
❑ Read operations are performed locally.
❑ Write operations are performed at all sites (using a distributed locking
protocol).
❑ Coordinate 2PC
◼ Slaves:
❑ As before

27
Phan Thị Hà
Eager Distributed Protocol (cont’d)
◼ Critical issue:
❑ Concurrent Writes initiated at different master sites are executed in the
same order at each slave site
❑ Local histories are serializable (this is easy)
◼ Advantages
❑ Simple and easy to implement
◼ Disadvantage
❑ Very high communication overhead
◼ n replicas; m update operations in each transaction: n*m messages (assume
no multicasting)
◼ For throughput of k tps: k* n*m messages
◼ Alternative
❑ Use group communication + deferred update to slaves to reduce
messages

28
Phan Thị Hà
Lazy Single Master/Limited
Transparency
◼ Update transactions submitted to master
◼ Master:
❑ Upon write: write locally and return to user
❑ Upon commit/abort: terminate locally
❑ Sometime after commit: multicast updates to slaves (in order)
◼ Slaves:
❑ Upon read: read locally
❑ Refresh transactions: install updates

29
Phan Thị Hà
Lazy Primary Copy/Limited
Transparency
◼ There are multiple masters; each master execution is similar to lazy
single master in the way it handles transactions
◼ Slave execution complicated: refresh transactions from multiple

masters and need to be ordered properly

30
Phan Thị Hà
Lazy Primary Copy/Limited
Transparency – Slaves
◼ Assign system-wide unique timestamps to refresh transactions and
execute them in timestamp order
❑ May cause too many aborts
◼ Replication graph
❑ Similar to serialization graph, but nodes are transactions (T) + sites (S);
edge 〈Ti,Sj〉exists iff Ti performs a Write(x) and x is stored in Sj
❑ For each operation (opk), enter the appropriate nodes (Tk) and edges; if
graph has no cycles, no problem
❑ If cycle exists and the transactions in the cycle have been committed at
their masters, but their refresh transactions have not yet committed at
slaves, abort Tk; if they have not yet committed at their masters, Tkwaits.
◼ Use group communication

31
Phan Thị Hà
Lazy Single Master/Full Transparency
◼ This is very tricky

❑ Forwarding operations to a master and then getting refresh
transactions cause difficulties
◼ Two problems:
❑ Violation of 1SR behavior
❑ A transaction may not see its own reads
◼ Problem arises in primary copy/full transparency as well

32
Phan Thị Hà
Example 3
Site M (Master) holds x, y; SiteB

holds slave copies of x, y
T1: Read(x), Write(y), Commit
T2: Read(x), Write(y), Commit

33
Phan Thị Hà
Example 4
◼ Master site M holds x, site C holds slave copy of x

◼ T3: Write(x), Read(x), Commit
◼ Sequence of execution
1. W3(x) submitted at C, forwarded to M for execution
2. W3(x) is executed at M, confirmation sent back to C
3. R3(x) submitted at C and executed on the local copy
4. T3 submits Commit at C, forwarded to M for execution
5. M executes Commit, sends notification to C, which also
commits T3
6. M sends refresh transaction for T3 to C (for W3(x) operation)
7. C executes the refresh transaction and commits it
◼ When C reads x at step 3, it does not see the effects of
Write at step 2
34
Phan Thị Hà
Lazy Single Master/
Full Transparency - Solution
◼ Assume T = Write(x)
◼ At commit time of transaction T, the master generates a
timestamp for it [ts(T)]
◼ Master sets last_modified(xM) ← ts(T)
◼ When a refresh transaction arrives at a slave site i, it
also sets last_modified(xi) ← last_modified(xM)
◼ Timestamp generation rule at the master:
❑ ts(T) should be greater than all previously issued timestamps
and should be less than the last_modified timestamps of the
data items it has accessed. If such a timestamp cannot be
generated, then T is aborted.

35
Phan Thị Hà
Lazy Distributed Replication
◼ Any site:
❑ Upon write: write locally and return to user
❑ Upon commit/abort: terminate locally
❑ Sometime after commit: send refresh transaction
❑ Upon message from other site
◼ Detect conflicts
◼ Install changes
◼ Reconciliation may be necessary

36
Phan Thị Hà
Reconciliation
◼ Such problems can be solved using pre-arranged

patterns:
❑ Latest update win (newer updates preferred over old ones)
❑ Site priority (preference to updates from headquarters)
❑ Largest value (the larger transaction is preferred)
◼ Or using ad-hoc decision making procedures:
❑ Identify the changes and try to combine them
❑ Analyze the transactions and eliminate the non-important ones
❑ Implement your own priority schemas

37
Phan Thị Hà
Replication Strategies
+ Updates do not need to be + No inconsistencies

coordinated + Elegant (symmetrical solution)
+ No inconsistencies - Long response times
Eager
- Longest response time - Updates need to be coordinated

- Only useful with few updates
- Local copies are can only be read
+ No coordination necessary + No centralized coordination

+ Short response times + Shortest response times
- Local copies are not up to date - Inconsistencies
- Inconsistencies - Updates can be lost
(reconciliation)
Centralized Distributed

38
Phan Thị Hà
Group Communication
◼ A node can multicast a message to all nodes of a group

with a delivery guarantee
◼ Multicast primitives
❑ There are a number of them
❑ Total ordered multicast: all messages sent by different nodes are
delivered in the same total order at all the nodes
◼ Used with deferred writes, can reduce communication
overhead
❑ Remember eager distributed requires k*m messages (with
multicast) for throughput of ktps when there are n replicas and m
update operations in each transaction
❑ With group communication and deferred writes: 2k messages

39
Phan Thị Hà
Outline
❑
❑ Replication and Failure Management

40
Phan Thị Hà
Failures
◼ So far we have considered replication protocols in the

absence of failures
◼ How to keep replica consistency when failures occur
❑ Site failures
◼ Read One Write All Available (ROWAA)
❑ Communication failures
◼ Quorums
❑ Network partitioning
◼ Quorums

41
Phan Thị Hà
ROWAA with Primary Site
◼ READ = read any copy, if time-out, read another copy.

◼ WRITE = send W(x) to all copies. If one site rejects the
operation, then abort. Otherwise, all sites not responding
are “missing writes”.
◼ VALIDATION = To commit a transaction
❑ Check that all sites in “missing writes” are still down. If not, then
abort the transaction.
◼ There might be a site recovering concurrent with transaction
updates and these may be lost
❑ Check that all sites that were available are still available. If some
do not respond, then abort.

42
Phan Thị Hà
Distributed ROWAA
◼ Each site has a copy of V
❑ V represents the set of sites a site believes is available
❑ V(A) is the “view” a site has of the system configuration.
◼ The view of a transaction T [V(T)] is the view of its coordinating site,
when the transaction starts.
❑ Read any copy within V; update all copies in V
❑ If at the end of the transaction the view has changed, the transaction is
aborted
◼ All sites must have the same view!
◼ To modify V, run a special atomic transaction at all sites.
❑ Take care that there are no concurrent views!
❑ Similar to commit protocol.
❑ Idea: Vs have version numbers; only accept new view if its version
number is higher than your current one
◼ Recovery: get missed updates from any active node
❑ Problem: no unique sequence of transactions
43
Phan Thị Hà
Quorum-Based Protocol
◼ Assign a vote to each copy of a replicated object (say

Vi) such that ∑iVi = V
◼ Each operation has to obtain a read quorum (Vr) to read
and a write quorum (Vw) to write an object
◼ Then the following rules have to be obeyed in
determining the quorums:
❑ Vr+ Vw>V an object is not read and written by two
transactions concurrently
❑ Vw>V/2 two write operations from two transactions cannot
occur concurrently on the same object

44
Phan Thị Hà

6 Replication Nhom3

Uploaded by

Copyright:

Available Formats

6 Replication Nhom3

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

6 Replication Nhom3

Uploaded by

Copyright:

Available Formats

Principles of Distributed Database

© 2020, M.T. Özsu & P. Valduriez TS.

© 2020, M.T. Özsu & P. Valduriez TS.

© 2020, M.T. Özsu & P. Valduriez TS.

◼ There are physical copies of logical objects in the system.

x Logical data item

Write(x1) Write(x2) Write(xn)

Physical data item (replicas, copies)

◼ Consistency models - how do we reason about the

© 2020, M.T. Özsu & P. Valduriez TS.

© 2020, M.T. Özsu & P. Valduriez TS.

◼ How can we guarantee that the global execution history

© 2020, M.T. Özsu & P. Valduriez TS.

Site A Site B Site C

Global history non-serializable: HB: T1→T2, HC: T2→T3→T1

T1: Read(x) T2: Read(x)

Consider the two histories:

HA={R1(xA),W1(xA), C1, R2(xA), W2(xA), C2}

Global history non-serializable: HA: T1→ T2, HB: T2→ T1

© 2020, M.T. Özsu & P. Valduriez TS.

❑ Update Management Strategies

© 2020, M.T. Özsu & P. Valduriez TS.

◼ Depending on when the updates are propagated

© 2020, M.T. Özsu & P. Valduriez TS.

Site 1 Site 2 Site 3 Site 4

© 2020, M.T. Özsu & P. Valduriez TS.

© 2020, M.T. Özsu & P. Valduriez TS.

Site 1 Site 2 Site 3 Site 4

Site 1 Site 2 Site 3 Site 4

© 2020, M.T. Özsu & P. Valduriez TS.

Site 1 Site 2 Site 3 Site 4

Site 1 Site 2 Site 3 Site 4

© 2020, M.T. Özsu & P. Valduriez TS.

© 2020, M.T. Özsu & P. Valduriez TS.

The previous ideas can be combined into 4 different replication protocols:

Eager Eager centralized Eager distributed

Lazy Lazy centralized Lazy distributed

© 2020, M.T. Özsu & P. Valduriez TS.

© 2020, M.T. Özsu & P. Valduriez TS.

◼ Applications submit transactions directly to their local TMs

© 2020, M.T. Özsu & P. Valduriez TS.

© 2020, M.T. Özsu & P. Valduriez TS.

◼ Updates originate at any copy

© 2020, M.T. Özsu & P. Valduriez TS.

© 2020, M.T. Özsu & P. Valduriez TS.

© 2020, M.T. Özsu & P. Valduriez TS.

◼ Slave execution complicated: refresh transactions from multiple

© 2020, M.T. Özsu & P. Valduriez TS.

© 2020, M.T. Özsu & P. Valduriez TS.

◼ This is very tricky

© 2020, M.T. Özsu & P. Valduriez TS.

Site M (Master) holds x, y; SiteB

T2: Read(x), Write(y), Commit

© 2020, M.T. Özsu & P. Valduriez TS.

◼ Master site M holds x, site C holds slave copy of x

© 2020, M.T. Özsu & P. Valduriez TS.

© 2020, M.T. Özsu & P. Valduriez TS.

◼ Such problems can be solved using pre-arranged

© 2020, M.T. Özsu & P. Valduriez TS.

+ Updates do not need to be + No inconsistencies