Introduction To Reliable and Secure Distributed Programming Slide
Introduction To Reliable and Secure Distributed Programming Slide
Programming
Christian Cachin*
Rachid Guerraoui
Luís Rodrigues
● C. Cachin, R. Guerraoui, L.
Rodrigues
● 2nd ed. of "Introduction to
Reliable Distributed Program-
ming" (Springer, 2011)
● The new content covers
Byzantine failures
Web: www.distributedprogramming.net
Distributed systems
● Basic abstractions
– Processes
– Links
– Timing models
– Cryptography
Prologue
● Sequential programming
– Array, record, list ...
● Concurrent programming
– Thread, semaphore, monitor ...
● Distributed programming
– Reliable broadcast
– Shared memory
– Consensus
– Atomic commit
– ...
Distributed programming
abstractions
p q r z
Processes
Communication abstraction
Component A
Events
Component B
Events
Request Indication
(send) (deliver)
Layer n
(invoke) (receive)
Request Indication
Layer n-1
Crash
Omission
Eavesdropping
Arbitrary
● Events
– Request <pl, Send | q, m>
● Sends a message m to process q
– Indication <pl, Deliver | p, m>
● Delivers a message m from sender p
● Properties
– PL1 (Reliability): If a correct sends message m to a
correct process q, then q eventually delivers m.
– PL2 (No duplication): No message is delivered
more than once.
– PL3 (No creation): If a process delivers a message
m with sender p, then s has sent m.
Time
● Confidentiality
– Encryption (not relevant here)
● Integrity
– Hash functions
– Digital signatures
Hash functions
● Cryptographic hash function H maps inputs of
arbitrary length to a short unique tag
Reliable broadcast
Broadcast
● Reliable broadcast
– Guarantees that messages are delivered to all
processes consistently
– Agreement on the delivered messages
– No ordering among delivered messages
Best-Effort Broadcast (beb)
● Events
– Request <beb, Broadcast | m>
● Broadcasts a message m to all processes
– Indication <beb, Deliver | p, m>
● Delivers a message m from sender p
● Properties
– BEB1 (Validity): If a correct process broadcasts m,
then every correct process eventually delivers m.
– BEB2 (No duplication): No message is delivered
more than once.
– BEB3 (No creation): If a process delivers a
message m with sender p, then p has broadcast m.
● Properties
– RB1 (Validity) = BEB1
– RB2 (No duplication) = BEB2
– RB3 (No creation) = BEB3
– RB4 (Uniform agreement): If some process*
delivers a message m, then every correct process
eventually delivers m.
● Events
– Request <bcb, Broadcast | m>
● Broadcasts a message m to all processes
– Indication <bcb, Deliver | p, m>
● Delivers a message m from sender p
● Properties
– BCB1 (Validity) = BEB1
– BCB2 (No duplication): Every correct process
delivers at most one message
– BCB3 (Integrity): If a correct process delivers m with
sender p, and p correct, then p has broadcast m.
(...)
Byzantine Consistent Bc. (bcb)
(cont.)
(... Properties)
– BCB4 (Consistency): If a correct process delivers
message m and another correct process delivers
message m', then m=m'.
● Faulty sender p
● Processes q and s bcb-deliver the message
● Process r does not deliver any message
●
O(n2) messages; O(n2 |m|) communication
Using Byzantine quorums
● Events
– Request <brb, Broadcast | m>
– Indication <brb, Deliver | p, m>
● Properties
– BRB1 (Validity) = BCB1
– BRB2 (No duplication) = BCB2
– BRB3 (Integrity) = BCB3
– BRB4 (Consistency) = BCB4
– BRB5 (Totality): If some correct process delivers a
message, then every correct process eventually
delivers a message
● Either all or none of the correct processes
deliver the message
Auth. Double-Echo Broadcast
Implements brb, uses abeb, with sender s (N>3f) [Bra87]
sentready := FALSE
● Two variants
– Consistent Channel
– Reliable Channel
Act II
Shared memory
Operations on shared memory
● Read(r) → v
– Reads from register instance r and returns value v
● Properties
– ONRR1 (Liveness): If a correct process invokes an
operation, then the operation eventually completes.
– ONRR1 (Validity): A read returns the last value
written or the* value written concurrently.
*Only one process can possibly write.
Implementations of registers
Communication abstraction
Client c1 c1
processes
● Specification of
– (1,N) safe Byzantine register (bonsr) and
– (1,N) regular Byzantine register (bonrr)
directly follows from (1,N) regular register
Implementations
● byz-highestval()
– eliminates all values occurring f or fewer times
– returns survivor value with highest timestamp
-- or -- special value if no such value exists
Comments
Consensus
Consensus
● Properties
– UC1 (Termination): Every correct process
eventually decides.
– UC2 (Validity): Any decided value has been
proposed by some process.
– UC3 (Integrity): No process decides twice.
– UC4 (Uniform Agreement): No two processes*
decide differently.
* whether correct or faulty
Weak Byzantine Cons. (wbc)
● Events
– Request <wbc, Propose | v>
● Proposes value v for consensus
– Indication <wbc, Decide | v>
● Outputs a decided value v of consensus
● Properties
– WBC1 (Termination) = UC1
– WBC2 (Weak Validity): Suppose all processes are
correct: if all propose v, then a process may only
decide v; if a process decides v, then v was
proposed by some process.
– WBC3 (Integrity): No correct process decides twice.
– WBC4 (Agreement): No two correct processes
decide differently.
Implementing consensus
● Introduced in
– Viewstamped replication [OL88]
– Paxos [L96]
– PBFT [CL02]
(these formulate it as total-order broadcast)
Leader-driven consensus
Epoch Cons.
Epoch Cons.
Epoch Cons.
Epoch-Change
(2,L2)
(3,L3)
(1,L1)
Eventual Leader
Detector
● Implement Epoch-Change
● Implement Epoch-Change
● Properties
– ELD1 (Eventual accuracy): Eventually every correct
process trusts some correct process.
– ELD2 (Eventual agreement): Eventually no two
correct processes trust a different process.
● Properties
– EC1 (Monotonicity): If a correct process starts epoch
(ts,L) and later starts epoch (ts',L'), then ts' > ts.
– EC2 (Consistency): If a correct process starts epoch
(ts,L) and another correct process starts epoch
(ts,L'), then L = L'.
– EC3 (Eventual Leadership): Eventually every correct
process starts no further epoch; moreover, every
correct process starts the same last epoch (ts,L),
where L is a correct process.
Epoch Consensus (ep)
● Associated with timestamp ts and leader L
(known to all processes)
● Events
– Request <ep, Propose | v>
● Proposes v for epoch consensus (executed by leader only)
– Request <ep, Abort>
● Aborts this epoch consensus
– Indication <ep, Decide | v>
● Outputs decided value v for epoch consensus
– Indication <ep, Aborted | s>
● Signals that this epoch consensus has completed the abort
and returns state s
Epoch Consensus (ep)
● Properties
– EP1 (Validity): If a correct process ep-decides v,
then v was proposed by the leader of some epoch
consensus (ts',L) with ts' ≤ ts.
– EP2 (Uniform Agreement): No two [correct*]
processes ep-decide differently.
– EP3 (Integrity): A correct process ep-decides at most
once.
– EP4 (Lock-in): If a process ep-decides v in epoch ts'
< ts, no process ep-decides a value different from v.
– EP5 (Termination): If the leader L is correct, has ep-
proposed a value and no process aborts, then every
correct process eventually ep-decides.
(...)
* for Byzantine epoch consensus
Epoch Consensus (ep)
● (... Properties)
– EP6 (Abort behavior): When a correct process
aborts, then it eventually completes the abort; plus,
a correct process completes an aborts only if it has
been aborted before.
(...)
Implementing Byzantine epoch
consensus (2)
● Processes choose value with highest timestamp
– If no value found, only then leader is free to take the
value from its ep-proposal
● Events
– Request <cc, Input | m>
● Inputs message m
– Indication <cc, Collected | M>
● Outputs vector M of collected messages or UNDEFINED
● Properties
– CC1 (Consistency): If L is correct, every correct pr.
collects the same M, which contains at least N-f
messages different from UNDEFINED.
– CC2 (Integrity): If a correct pr. collects M with M[p]
≠ UNDEFINED and p is correct, then p has input m.
(...)
Conditional Collect (cc)
● (... Properties)
– CC3 (Termination): If L is correct and all correct pr.
input messages such that they satisfy C, then every
correct process eventually collects M s.t. C(M).
● Note
– Every process inputs a message
– Output is vector of such messages, one per process
– If L correct, then output M satisfies the predicate
● Otherwise, may not terminate
Byz. read/write epoch cons. (1)
Implements ep, uses al, abeb, cc (N > 3f), with ts. ets and leader L
● Predicate binds(ts,v,S):
– Whether (ts,v) is confirmed by > (N+f)/2 entries in
S to be value associated to highest timestamp,
and
– Value v has not been invented out of thin air
● Hence, processes write this value again
● Predicate unbound(S):
– Evidence that no value can be bound by S
● Hence, processes write value of the leader
www.distributedprogramming.net