Efficiently repairing and measuring replica consistency in distributed databases

Abstract

In a distributed database, maintaining large table replicas with frequent asynchronous insertions is a challenging problem that requires carefully managing a tradeoff between consistency and availability. With that motivation in mind, we propose efficient algorithms to repair and measure replica consistency. Specifically, we adapt, extend and optimize distributed set reconciliation algorithms to efficiently compute the symmetric difference between replicated tables in a distributed relational database. Our novel algorithms enable fast synchronization of replicas being updated with small sets of new records, measuring obsolence of replicas having many insertions and deciding when to update a replica, as each table replica is being continuously updated in an asynchronous manner. We first present an algorithm to repair and measure distributed consistency on a large table continuously updated with new records at several sites when the number of insertions is small. We then present a complementary algorithm that enables fast synchronization of a summarization table based on foreign keys when the number of insertions is large, but happening on a few foreign key values. From a distributed systems perspective, in the first algorithm the large table with data is reconciled, whereas in the second case, its summarization table is reconciled. Both distributed database algorithms have linear communication complexity and cubic time complexity in the size of the symmetric difference between the respective table replicas they work on. That is, they are effective when the network speed is smaller than CPU speed at each site. A performance experimental evaluation with synthetic and real databases shows our algorithms are faster than a previous state-of-the art algorithm as well as more efficient than transferring complete tables, assuming large replicated tables and sporadic asynchronous insertions.

Appendix

Lemma 1

If ∀D _i , i=0,…,N−1, $\operatorname {cur}(D_{i}.T_{p}) = 0$ (i.e., if all replicas of table T _p are replica consistent), then $\operatorname {gcur}(T_{p}) = 0$ (that is, table T _p is global replica consistent).

Proof

If ∀D _i , i=0,…,N−1, $\operatorname {cur}(D_{i}.T_{p}) = 0$, then by Eqs. (1) and (2) that

(13)

Since $|\bigcup_{\substack{i=0}}^{N-1}D_{i}.T_{p}| = |D_{0}.T_{p} \cup ( \bigcup_{\substack{i=1}}^{N-1}(D_{i}.T_{p}- D_{0}.T_{p}) )|$ and $|\bigcap_{\substack{i=0}}^{N-1}D_{i}.T_{p}| = |D_{0}.T_{p} - ( \bigcup_{\substack{i=1}}^{N-1}(D_{0}.T_{p}- D_{i}.T_{p}) )|$, by virtue of Eq. (13) we have $|\bigcup_{\substack {i=0}}^{N-1}D_{i}.T_{p}| = |D_{0}.T_{p} |$ and $|\bigcap_{\substack{i=0}}^{N-1}D_{i}.T_{p}| = |D_{0}.T_{p} |$, and consequently, by definition of $\operatorname {gcur}()$ (Eq. (3)), it follows that

$$ \operatorname {gcur}(T_p) = 0 \\[15pt] $$

(14)

 □

Lemma 2

(15)

Proof

By definition $| \mathcal{T}_{r\cup} \Join_{K} \mathcal{T}_{s\cup} |$ is the number of tuples in $\mathcal{T}_{r\cup}$ having their foreign key K values match a primary key K value in $\mathcal{T}_{s\cup}$. $\pi_{K} (\mathcal{T}_{r\cup}) - \pi_{K} (\mathcal{T}_{s\cup})$ is a one attribute relation with values in $\pi_{K} (\mathcal{T}_{r\cup})$ that are not in $\pi_{K} (\mathcal{T}_{s\cup})$, so $(\pi_{K} (\mathcal{T}_{r\cup }) - \pi_{K} (\mathcal{T}_{s\cup})) \Join_{K} \mathcal{T}_{r\cup}$ are the tuples in $\mathcal{T}_{r\cup}$ that do not have a foreign key value matching a value in $\pi_{K} (\mathcal{T}_{s\cup})$. Observe that in these tuples, K is not null.

Consequently, $|\mathcal{T}_{r\cup}| - | (\pi_{K} (\mathcal{T}_{r\cup}) - \pi_{K} (\mathcal{T}_{s\cup})) \Join_{K} \mathcal{T}_{r\cup} |$ is the number of tuples in $\mathcal{T}_{r\cup}$ with their foreign key K values matching a primary key K value in $\mathcal{T}_{s\cup}$ plus the tuples with a null value in foreign key K. The last subtraction, $- | \sigma_{\text{isnull}(K)} (\mathcal{T}_{r\cup})|$, is to discount these tuples. □

To evaluate the above expression, among other computations we need the difference $\pi_{K} ( \mathcal{T}_{r\cup}) - \pi_{K} ( \mathcal {T}_{s\cup})$. This expression evaluates a table with the invalid foreign key values—that is, a table with the inclusion dependency violations. We note that, if the difference is low, this table difference would be relatively small compared to the size of $\mathcal{T}_{s\cup}$ or even $\pi_{K} ( \mathcal{T}_{s\cup})$.