Private Approximate Query over Horizontal Data Federation

Ala Eddine Laouir and Abdessamad Imine Université de Lorraine, CNRS and Inria, NancyFrance firstname.lastname@loria.fr

Abstract.

In many real-world scenarios, multiple data providers need to collaboratively perform analysis of their private data. The challenges of these applications, especially at the big data scale, are time and resource efficiency as well as end-to-end privacy with minimal loss of accuracy. Existing approaches rely primarily on cryptography, which improves privacy, but at the expense of query response time. However, current big data analytics frameworks require fast and accurate responses to large-scale queries, making cryptography-based solutions less suitable. In this work, we address the problem of combining Approximate Query Processing (AQP) and Differential Privacy (DP) in a private federated environment answering range queries on horizontally partitioned multidimensional data. We propose a new approach that considers a data distribution-aware online sampling technique to accelerate the execution of range queries and ensure end-to-end data privacy during and after analysis with minimal loss in accuracy. Through empirical evaluation, we show that our solution is able of providing up to $8$ times faster processing than the basic non-secure solution while maintaining accuracy, formal privacy guarantees and resilience to learning-based attacks.

Differential privacy, Big Data, Federated data, Online sampling.

1. Introduction

The extensive reliance of individuals on software solutions in daily and professional life has led to an exponential growth of data collected by companies, corporations, government organisations, and even hospitals. These vast mines of data, if carefully and efficiently analysed, can provide valuable insights that guide decision-making and business development. In large-scale studies and research, the analysis must be conducted on several data sources to obtain meaningful conclusions. An example of such a case is during a pandemic, where many hospitals jointly conduct studies to have a global view of the problem.

One of the most commonly used tools to analyse and explore these huge volumes of data is OLAP tasks, where various aggregation queries (SUM, COUNT, etc.) can be issued to learn existing patterns and trends within the data. These aggregation queries may seem simple, but they are very time-consuming in big databases. The analysis of data from multiple data providers comes with two main challenges: privacy and resource/time efficiency. The privacy issue arises from the fact that this data is personal and sensitive to individuals, and sharing it with other parties can be very harmful. Many regulations and restrictions like GDPR are imposed by governments on how to process and share such sensitive data. In the case of a federated environment, where a joint study requires the collaboration of many data providers, data sharing is highly restricted. Each data provider must ensure the security and privacy of the data collected from their users during and after the analysis.

To satisfy the requirement of end-to-end privacy, many solutions have been proposed in the literature, and most of them rely on cryptography to ensure there is no data leakage during the exchange and query evaluation. Secure multiparty computation (SMC) solutions(Bater et al., 2018, 2020) appear to be a prominent solution in federated environments. Others use oblivious operations(Bater et al., 2017) or secure hardware (Zheng et al., 2017; Eskandarian and Zaharia, 2017; Qiu et al., 2023) so that during query evaluation, each data provider can maintain the confidentiality of their data. Additionally for securing the end result of any OLAP query, Differential Privacy (DP) (Dwork et al., 2014) is generally considered the gold standard by government and private institutions (Team et al., 2017; Abowd, 2018; Bittau et al., 2017; Erlingsson et al., 2014). Due to its strong formal confidentiality guarantees, DP allows individuals to deny their participation in the database. These query evaluation solutions in a federated environment meet end-to-end security and privacy requirements. However, what they have in common is their reliance on encryption. This causes a huge processing time overhead, and for time-sensitive tasks, utility is measured by both accuracy and speed. They certainly address the privacy issue, but they are time and resource consuming.

The issue of reducing query response time has been widely addressed in the literature, through the need to obtain Approximate Query Processing (AQP). Existing AQP methods can be classified into two types, online approximation and offline synopsis creation. In online approximation, there is Online Aggregation based solutions (Hellerstein et al., 1997; Li et al., 2016; Qin and Rusu, 2014) that provide fast and reliable approximation of the query continuously, and other solutions based on applying online sampling to reduce the processed data and obtain an approximation from a sample (Zhang et al., 2016; Goiri et al., 2015; Song et al., 2018).In offline synopsis creation, views are generated offline using query workloads or/and data statistics (Acharya et al., 1999; Agarwal et al., 2013; Chaudhuri et al., 2007).
In this area of research, the main focus is on efficiency, but privacy has not been considered.

In our work, we address the challenge of answering OLAP aggregation range queries in a federated environment, while preserving end-to-end privacy and improving resource and time consumption for query processing. Our solution relies heavily on differential privacy to secure collaboration and end results, and ensure no information leaks. To speed up queries, we implement a cluster-based sampling method using a well-known statistical estimator that provides accurate estimates for range queries (such as SUM and COUNT) while processing minimal data portions. While existing systems ensure either privacy or speedup for query approximation, to the best of our knowledge, our solution is the first to offer speedup over plain-text execution with end-to-end privacy in a federated environment. Our main contributions can be listed as follows:

(1)

Definition of a lightweight collaboration method that determines optimal sampling decisions for data providers to maximize accuracy without needing access to their full datasets or information leakage.
(2)

Introduction of data distribution-aware cluster sampling method with DP guarantees for individual privacy.
(3)

Meticulous integration of DP at every step with minimal loss of precision.
(4)

Extensive experimentation to empirically validate the performance of our approach in terms of accuracy and time efficiency.
(5)

Extensive experimentation to ensure the resilience of our system against learning-based attacks.

Roadmap. The paper is structured as follows: Section 2 reviews some existing works. Section 3 introduces the notions used throughout our paper. Section 4 gives a detailed description of the problem solved by our approach. Section 5 presents our proposed solution in detail. The extensive evaluation of our approach is given in Section 6. In Section 7, we discuss the limitations/extensions of our solution and we conclude in Section 8 by giving some future works.

2. Related Works

Due to the increasing size and distribution of databases, querying and exploring such vast volumes for analytical purposes, quickly and without revealing sensitive information, has become a challenge. Here, we describe the state-of-the-art related to our work.

Approximate Query Processing (AQP). As the quality of a query is based on its accuracy and response time, especially for time-sensitive tasks like OLAP (Wang and Jajodia, 2008) and Business Intelligence (BI), approximating the query offers the best way to strike a balance between these two quality factors.

In the early $1990$ s, (Hellerstein et al., 1997) proposed a new interactive method for query processing that provides a quick initial answer with a certain error, refining it as processing continues. Other works followed in this direction (Xu et al., 2008; Li et al., 2016; Wu et al., 2010; Qin and Rusu, 2014), each enhancing specific aspects of the method by including support for group by or propose parallel and distributed versions. Another research direction focuses on processing a small subset of the original data, thereby reducing query run-time. In (Olken and Rotem, 1986, 1995; Piatetsky-Shapiro and Connell, 1984; Song et al., 2018), uniform row-level random sampling is applied online before query processing. Although row-level sampling may improve processing time for complex queries, it can introduce overhead and slow down queries that require a full table scan (Haas and König, 2004) (e.g. Bernoulli sampling). To avoid such overhead, the solutions from (Acharya et al., 1999; Agarwal et al., 2013; Chaudhuri et al., 2007) create the samples offline. Cluster sampling, also referred to as page-sampling (Haas and König, 2004), is utilized to speed-up aggregation queries in big databases. Methods in (Goiri et al., 2015; Zhang et al., 2016; Ahmadvand et al., 2019) use this sampling in the context of Hadoop Map-Reduce framework ¹¹1https://hadoop.apache.org, as it proves to be fast and I/O efficient compared to row-level sampling.

Federated query answering. Data is often distributed across multiple locations (e.g. data providers like hospitals and companies) and the collaboration among all parties is necessary to answer range aggregation queries. But for privacy and security reasons, each data provider cannot disclose their data to third parties.

Some solutions rely on secure hardware modules (i.e. enclaves), in which all sensitive code and data are processed. Methods in (Agrawal et al., 2006; Zheng et al., 2017; Eskandarian and Zaharia, 2017) focus on aggregation queries in this setting, and (Zheng et al., 2017; Eskandarian and Zaharia, 2017) use intel’s SGX for secure processing. These solutions are generally efficient, but their reliance on trusted hardware and weakness to side-channel attacks constitute a limitation. Recently in (Qiu et al., 2023), the notion of Differential Obliviousness was used to mitigate the risk of side channel attacks.‘

Other recent works presented Secure Multiparty Computation (SMC) query processing engines (Bater et al., 2017, 2018, 2020). These engines enable data providers to respond to OLAP queries securely by joining data with end-to-end privacy. Differential Privacy (DP) is used to perturb the final results, thereby mitigating any inference attacks based on the results. While these solutions incur computational overhead, (Bater et al., 2020) introduced online random sampling to improve secure computing performance by reducing the size of shared data for query processing. In (Cao et al., 2021), sampling is performed offline to create a synopsis to further improve performance. Another solution (Liagouris et al., 2021) focused on reducing the cost of SMC operation thus obtaining significant improvement in performances. All of these SMC (or enclaves)-based protocols are encryption-based, which prevents them from outperforming plain-text query execution. Even with significant improvements introduces in the past years, on real world big tables they still expensive for real-time queries(Liagouris et al., 2021).

To highlight the scale of this problem, we performed a simulation²²2https://github.com/AlaEddineLaouir/Federated-Range-Queries.git using a synthetic Adult(Becker and Kohavi, 1996) horizontally distributed on 4 data providers as a federated environment. We ran a set of random range queries, which are the type of queries we focus on. For the query processing, we considered two solutions using SMC: (i) data providers sharing the rows and collectively evaluating the query; and (ii) evaluating the query locally and only sharing the results and computing the final result.

Refer to caption — Figure 1. Runtime cost of data sharing in SMC.

We measured the time required to share the rows/results in SMC. The results in Figure 1 show that sharing only local results incurs an insignificant overhead of $0.04$ seconds. On average, this is less than $440$ times the time required for row sharing in SMC. Additionally, the cost of sharing only results remains constant and independent of the dataset, whereas the cost of sharing rows will increase with larger tables.

In our work, we propose a framework to approximate query processing in a federated environment, enabling accelerated query execution compared to plain text execution while ensuring end-to-end Differential Privacy guarantees.

3. Preliminaries

In this section, we give the notation and explain briefly notions used throughout the paper.

Data model. In a tabular database $T$ defined over a set of $n$ dimensions (or attributes) $D=\{d_{1},d_{2},...,d_{n}\}$ , each individual is a row with values on each dimension. We assume that each dimension $d$ is associated with a domain $|d|$ containing discrete and totally ordered values, the size of the domain is $||d||$ . For performance purposes during online analytics tasks, the table $T$ is transformed into a multidimensional data (or a count tensor) $T^{a}$ of dimensions $D^{a}\subset D$ , which has an attribute Measure storing the number of aggregated rows of $T$ . Figure 2 illustrates how to construct a count tensor $T^{a}$ from table $T$ by aggregating dimension Service. For simplicity, we use term “table” for “tabular data” and “count tensor”.

Queries. To analyze and extract insights from these tables, the analyst can issue aggregation queries, helping to explore the data and gain a general understanding of patterns and trends. In this work, we consider a range query $Q$ defined as:

SELECT Aggregation FROM Table WHERE Range, where:

•

Aggregation is COUNT(*) or SUM(Measure).
•

Range is a set of intervals $r_{d}=[l_{b}^{d},u_{b}^{d}]$ on each dimension $d\in D^{Q}\text{ where }D^{Q}\subseteq D$ in Table, such that $l_{b}^{d}\leq v\leq u_{b}^{d}$ for every value $v\in|d|$ .

In our work, we focus on COUNT and SUM queries because they are used in several analytics applications. For instance, in a big database aggregating per-stock order data for the NASDAQ exchange, these queries are typically used to analyze order data from past days. Additionally, aggregations, such as average, standard deviation, and variance, can be derived from COUNT and SUM.

Query Approximation and Sampling. The goal of query approximation is generally to speed up execution at the expense of answering the query exactly, while preserving answer accuracy as much as possible. Online sampling is employed for time-sensitive tasks to reduce the overhead of evaluating queries on large databases. Note that in this case, the sampling differs from one query to another. In statistical terms, random sampling is essentially the process of selecting a subpopulation $SP$ from the total population $P$ where a sampling rate $sr$ dictates the size of $SP$ . This subpopulation contains sufficiently representative individuals and properties, capturing various characteristics of $P$ such that the analysis conducted on $SP$ can be generalized to $P$ . All random sampling techniques can be categorized based on three main features:

•

Granularity: sampling elements are individuals or a bulk/cluster of individuals.
•

Uniformity: elements are sampled with equal/unequal probabilities.
•

Replacement: sampling elements can be chosen multiple times or only once.

Nowadays, all modern systems choose to split/store a big table $T$ into a set of smaller, manageable entities $T=\{C_{1},C_{2},...,C_{N}\}$ where each entity has a maximum size $S$ . The entity could be Table pages³³3https://www.postgresql.org/docs/current/storage-page-layout.html, HDFS file Blocks⁴⁴4https://hadoop.apache.org/docs/r1.2.1/hdfs_design.html, etc.

In this paper, we call these storage entities Clusters and we assume that our tables are already stored as a set of clusters. Given this storage format, sampling on databases can be done at two levels: Row/Cluster level (Haas and König, 2004).

In tabular databases with range queries, it is particularly challenging to find an online sampling algorithm that offers speed-up while maintaining accuracy.

Data providers. For many real-world use cases, multiple organizations or institutions, called data providers, publish access to their databases for joint analysis. Let $\mathbb{S}$ be the set of data providers. In this work, we assume that a large table $T$ is horizontally distributed over $\mathbb{S}$ such that all data providers share the same schema (i.e. a set of dimensions) of $T$ but each contains different rows. All data providers use clusters of the same size to store their local tables. More importantly, for privacy reasons, data providers collaborate on joint analyzes without revealing their data.

Differential Privacy (DP). A privacy model that provides formal guarantees of indistinguishability such that the query results do not yield much information about the presence or absence of any particular individual. Consequently, it hides information about which of the neighbouring tables (Dwork et al., 2014) was used to answer the query.

Definition 3.0 (Neighbouring Tables(Dwork et al., 2014)).

Two tables $T$ and $T^{\prime}$ are neighbouring if we can obtain one of them by inserting at most a row into the other.

We use $d(T,T^{\prime})$ to represent the distance between two tables $T$ and $T^{\prime}$ and we say that two tables are neighbouring if their distance is $1$ or less.

Definition 3.0 ( $(\epsilon,\delta)$ -Differential Privacy(Dwork et al., 2014)).

A mechanism $M$ satisfies $(\epsilon,\delta)$ -Differential Privacy (or $(\epsilon,\delta)$ -DP) if, for any two neighboring tables $T$ , $T^{\prime}$ and for any possible output $V$ of $M$ :

\mbox{Pr}\left[M\left(T\right)\in V\right]\leq exp(\epsilon)\times\mbox{Pr}% \left[M\left(T^{\prime}\right)\in V\right]+\delta

where $\delta$ represents the failure probability. We refer to $(\epsilon,\delta)$ as the privacy budget.

In practice, $M$ is a randomized algorithm, which has many possible outputs under the same input. It is well known that DP is used to answer specific queries on databases. Let $f$ be a query on a table $T$ whose its answer $f(\mathcal{T})$ returns a number. The global sensitivity of $f$ is the amount by which the output of $f$ changes for all neighboring tables.

Definition 3.0 (Global Sensitivity(Dwork et al., 2014)).

For any two neighboring tables $T$ and $T^{\prime}$ , the global sensitivity of function $f$ is:

GS_{f}=\max_{T,T^{\prime}:d(T,T^{\prime})\leq 1}\left\|f(T)-f(T^{\prime})% \right\|_{1}

where $\left\|\cdot\right\|_{1}$ is the $L_{1}$ norm.

For instance, if $f$ is a COUNT range query then $GS_{f}$ is $1$ .

The Laplace Mechanism is a randomized mechanism for enforcing $\epsilon$ -DP (or $(\epsilon,0)$ -DP referred to as pure DP), which adds calibrated noise to the output of a function $f$ based on its global sensitivity $GS_{f}$ .

Definition 3.0 (Laplace Mechanism (Dwork et al., 2014)).

The Laplace Mechanism adds noise to $f(T)$ as:

S=f(T)+\text{Lap}\left(\frac{GS_{f}}{\epsilon}\right)

where $GS_{f}$ is the global sensitivity of $f$ , and $Lap(\alpha)$ denotes sampling from the Laplace distribution with center $0$ and scale $\alpha$ .

Unlike the Laplace Mechanism, which is used to release noisy numerical values, the Exponential Mechanism can be used for biased selection of elements from a set based on a scoring function while preserving ( $\epsilon,0$ )-DP (Dwork et al., 2014).

Definition 3.0 (Exponential Mechanism (Dwork et al., 2014)).

Given a set of elements $SE$ and a scoring function $L$ , the Exponential Mechanism randomly selects $e\in SE$ with the probability of the element $e$ being proportional to:

\exp\left(\frac{\epsilon\times L(e)}{2\times\Delta_{L}}\right)

where $\Delta_{L}$ is the sensitivity of $L$ .

Local and Smooth Sensitivity. In many applications of DP, the global sensitivity $GS_{f}$ cannot bounded. In this case, there is an alternative definition of sensitivity called local sensitivity, where the maximum difference between the query’s results is based on a fixed database $T$ and any database $T^{\prime}$ neighbouring to it:

Definition 3.0 (Local Sensitivity(Nissim et al., 2007)).

Given a database $T$ and $T^{\prime}$ as any of its possible neighbouring tables, the local sensitivity of function $f$ is:

LS_{f}(T)=\max_{T^{\prime}:\,d(T,T^{\prime})\leq 1}\left\|f(T)-f(T^{\prime})% \right\|_{1}

where $\left\|\cdot\right\|_{1}$ is the $L_{1}$ norm.

The local sensitivity $LS_{f}(T)$ is often much less than the global sensitivity $GS_{f}$ because it is based on a specific instance of the data $T$ . This also makes it unsafe to use, as it can leak information about $T$ on which it is based. Nassim et al (Nissim et al., 2007). suggest the use of a smoothing function that finds a safe upper bound for $LS_{f}(T)$ and can be used to calibrate the randomness (noise) without any risk. These functions usually require that the local sensitivity be computed at any arbitrary distance $k$ from $T$ .

Definition 3.0 (Local Sensitivity at Distance $k$ (Nissim et al., 2007)).

Given a table $T$ , the local sensitivity of function $f$ is:

LS_{f}(T)^{k}=\max_{T^{\prime}:\,d(T,T^{\prime})\leq k}\left\|f(T)-f(T^{\prime% })\right\|_{1}

where $\left\|\cdot\right\|_{1}$ is the $L_{1}$ norm.

A safe approximate upper bound of $LS_{f}(T)$ , $S\_LS_{f}(T)$ , which is insensitive to small variations of data can be obtained by the smooth sensitivity framework (Nissim et al., 2007).

Definition 3.0 (Smooth Sensitivity Framework (Nissim et al., 2007)).

\begin{array}[]{ll}S\_LS_{f}(T)=max_{k=0,1,...n}\{exp(-\beta k)LS_{f}(T)^{k}\}% \end{array}

where $\beta=\frac{\epsilon}{2log(2/\delta)}$ .

After a number of $n$ iterations, this upper bound can be used to calibrate noise for the Laplace mechanism to ensure $(\epsilon,\delta)$ -DP.

DP Properties. Combining several DP mechanisms is possible, and the privacy accounting is managed using the sequential and the parallel composition properties of DP. Let $M_{1},\ldots,M_{n}$ be mechanisms satisfying $(\epsilon_{1},\delta_{1}),\ldots,(\epsilon_{n},\delta_{n})$ -DP.

Theorem 3.9 (Sequential Composition (Dwork et al., 2014)).

Applying sequentially $M_{1},\ldots,M_{n}$ satisfies $\left(\sum_{j=1}^{n}\epsilon_{j},\sum_{j=1}^{n}\delta_{j}\right)$ -DP.

Theorem 3.10 (Parallel Composition (Dwork et al., 2014)).

A mechanism that applies $M_{1},\ldots,M_{n}$ on disjoint parts of the data satisfies:
$\left(max_{i\in n}(\epsilon_{i}),max_{i\in n}(\delta_{i})\right)$ -DP

The post-processing property states that it is safe to execute any function on the output of a DP mechanism.

Theorem 3.11 (Post-Processing (Dwork et al., 2014)).

For any $(\epsilon,\delta)$ -DP mechanism $M$ and any function $f$ , $f(M)$ satisfies $(\epsilon,\delta)$ -DP.

In the context of online query answering, each query consumes $(\epsilon,\delta)$ to secure the results. In order to manage/limit the information released to the analyst, a total budget $(\xi,\psi)$ is given which will be consumed by $N$ queries such that $\xi=N\epsilon$ and $\psi=N\delta$ . The analyst can continue sending queries until their total budget is consumed.

Secure Multiparty Computation (SMC). it refers to cryptographic protocols that enable a set of independent parties to collaboratively evaluate a query without revealing their private inputs to each other. It also allows them to avoid trusting a third party with the union of their data for query evaluation. However, this safety assurance comes at the cost of resources and processing time. Using SMC is several times slower than insecure alternatives.

4. Problem Statement

Given a federated system in which $n$ data providers pool their private data for analysis querying. Consider a private table $T$ (as in Figure 2) which is horizontally partitioned among data providers as tables $T_{1}$ , $\ldots$ , $T_{n}$ . Each data provider wants to keep the individual tuples of their local table confidential and only the schema of $T$ is public. Suppose an end user sends the following range query $Q$ :

SELECT COUNT(*) FROM Table WHERE 20 <= Age <= 40

where $Q$ is performed on the union of tables stored at the data providers, $\cup^{n}_{i=1}T_{i}$ . However, even though $Q$ may seem very simple at first glance, the big data associated with $T_{i}$ makes $Q$ very complex and time-consuming.

To solve the problem of slow query response time, we can resort to Approximate Query Processing (AQP) to find a trade-off between accuracy and speed of results via approximation. One very straightforward technique of AQP is to perform random sampling, given a sampling rate $sr$ , to obtain a set of tuples from $T$ . For example, an end user can request an answer for $Q$ based only on $sr=20\%$ of the entire $T$ . Even for a single table $T$ , to obtain a good approximation of $Q$ , the sampled tuples must contain meaningful data in the ranges of $Q$ . Random sampling can be done at the row or cluster level. Although cluster-level sampling is faster than row-level sampling, both have linear performance with respect to sampling rate. The larger the sample, the more accurate and slower the result, and vice versa.

Consider $T$ is stored as a set of clusters. To get an accurate estimate of $Q$ when processing a few parts of the data, we use a statistical estimator (Lohr, 2009). To do this, we need to consider the distribution of rows between all clusters. It should be noted that the assumption of a uniform distribution of rows among all clusters is rarely valid in real databases. Indeed, the rows generally follow a skewed distribution. In contrast, unequal probability cluster sampling is more effective at providing better estimates, where the probability of a cluster being sampled is based on the data distribution for $Q$ .

Assume that each partition $T_{i}$ of $T$ is stored using clusters. How to apply the unequal probability cluster sampling in our federated context? Note that each cluster within each data provider should have a specific probability $p$ of being sampled to estimate $Q$ , taking into account all other clusters (even those from other data providers). As a result, capturing the inter/intra data distribution will bias the sampling toward clusters or data providers that hold most of the data related to $Q$ . We refer to this sampling as global sampling.

The other solution is local sampling, where each data provider computes the sampling probabilities for its clusters (without considering other data providers). In this sampling, the sample size is distributed uniformly on data providers, so it does not require a collaboration between data providers. This lack of global data distribution awareness makes this solution less appealing than global sampling.

To apply global data distribution-aware sampling and approximation, data providers must provide appropriate information about their data to quickly and accurately estimate $Q$ . The optimal solution to capture the data distribution in this context is achieved if data providers have access to each other’s data and sampling probabilities are computed collectively. This collaboration will lead to an overhead in processing time. The challenge is then to define the summarized and small pieces of information that data providers can share and be sufficient to capture the data distribution while producing negligible overhead. Once this global data distribution is captured, each data provider can locally sample clusters, estimate the query, and send its result. All results from data providers will be added together and the final result will be returned to the end user.

Another dimension of our problem concerns privacy and data protection. In the federated context, the end-to-end privacy property must be guaranteed. This essentially ensures that data is protected (i) during and after query execution, (ii) for intermediate results during collaboration, and (iii) for the final response. Differential Privacy (DP) is a widely accepted privacy model, typically applied to query results to prevent any inference about the presence or absence of individuals. As for the intermediate results produced during collaboration between data providers, they must also be protected, with each data provider seeking to prevent any leakage of information on its table. Even if the exchange is limited to summarized (aggregated) information, there will be no privacy guarantee. Thus, DP can also be used to publish intermediate results between data providers.

An alternative solution to DP is the use of Secure Multiparty Computation (SMC) to implement collaboration between data providers. This solution has two major drawbacks: If data providers use the summary information for sampling in SMC, query approximation (which includes running the query on each cluster) must also be done in SMC because the sampling is based on sensitive information and its results may disclose information to other data providers. Second, SMC relies heavily on cryptography, which will significantly reduce the utility of the query in terms of processing time, thereby diluting the purpose of approximations.

In this work, we aim to provide fast and accurate responses to range queries in a federated setup while preserving end-to-end privacy. The challenges we address are: defining a lightweight sampling algorithm considering data distribution for query approximation in a federated environment and carefully applying Differential Privacy to ensure end-to-end privacy with minimal loss of query accuracy.

5. Our solution

5.1. Overview

In our proposal, we combine DP with lightweight SMC to protect intermediate results when collaborating between data providers. This allows us to obtain significantly better performance in terms of speed-up and achieve end-to-end privacy, while maintaining high utility answers for online range queries. To achieve these goals, we propose an efficient and lightweight collaboration method, allowing data providers to decide how many samples to extract from each, guided by the summary information shared during this collaboration. To integrate knowledge of the data distribution into our sampling and approximation steps, we use the probability proportional to size (pps) method (Lohr, 2009). Here, the probability $p$ of including (or sampling) a cluster $C$ is determined by the proportion $R$ of rows in $C$ falling within the ranges of the query $Q$ . Computing $R$ is expensive and requires similar overhead as running the query. To minimize the processing time of $Q$ , we will approximate each $R$ of any cluster $C$ using lightweight metadata associated with $C$ .

Our solution has two main phases: offline data preprocessing and online query answering. In the offline data preprocessing phase, each data provider constructs global and individual metadata for its clusters. This metadata makes query approximation easier without imposing a significant overhead in terms of processing time. All data providers agree on the same maximum cluster size $S$ (more details are given in Section 7) before initiating the system. The size $S$ may not reflect the actual size of their clusters, but it would be used to calculate the $R$ of each cluster. The offline phase and metadata creation are detailed in Section 5.2, and Figure 3 (b) shows the general architecture of our system with each data provider as well as its metadata.

Once preprocessing is complete for all data providers, the system goes online. In the online query response phase, the end user interacts with an aggregator by sending their query $Q$ and desired sampling rate $sr$ and receives a secure response in return. The aggregator manages the rest of the exchanges with the data providers. The query lifecycle (see Figure 3 (a)) as well as the collaboration (exchange of summary data) are described as follows:

(1)

First, the aggregator sends the query $Q$ to the data providers. Each data provider performs two tasks: i) identify the set of clusters $C^{Q}$ covering $Q$ such that $N^{Q}=|C^{Q}|$ , ii) compute the proportion $R$ of rows for each $C\in C^{Q}$ . The data provider uses previously stored metadata to avoid overhead when performing these two tasks.
(2)

Each data provider securely (using DP) sends to the aggregator the summarized data needed for collaboration. The number of clusters $N^{Q}$ and average of proportions $Avg(\widehat{R})$ where $\widehat{R}=\{R_{1},...,R_{N^{Q}}\}$ .
(3)

The aggregator computes and sends the best allocation (sample size $s$ ) for each data provider while respecting the total sample size given by $sr$ .
(4)

Each data provider tests the condition $N^{Q}<N^{min}$ in order to compute $Q$ “regularly” without approximation. The $N^{min}$ is a threshold set by each data provider to trigger the approximation only if the query is significantly large (more details about $N^{min}$ are given in Section 5.2).
(5)

If the previous condition does not hold, each data provider randomly and securely with DP samples $C^{Q}_{S}$ , where $C^{Q}_{S}\subset C^{Q}$ .
(6)

After sampling, each data provider estimates $Q$ over $C^{Q}_{S}$ locally and then securely sends the result to the aggregator with DP guarantees.
(7)

Alternatively, data providers may use SMC to share their local estimations and ”sensitivities”. Then, the aggregator obliviously sums the estimations and applies DP using the maximum sensitivity before safely releasing the final result.

In Section 5.2, we will focus on the approximation via cluster sampling and the metadata created offline. Afterward, section 5.3 will be dedicated to the second phase of our solution. In Section 5.3.1, we will describe the allocation step and how it preserves the same semantics as the naive (sharing all data) method of collaboration by keeping the sampling data distribution aware without an overhead. In Section 5.3.2, we will present the privacy-preserving sampling used by each data provider locally to create $C^{Q}_{S}$ . In Section 5.3.3, we detail how to obtain a calibrated DP noise for the end result obtained by using a statistical estimator. Finally in section 5.4, we explain how the privacy budget for each query is managed and consumed.

5.2. Query Approximation and sampling

As previously mentioned in Section 5.1, our unequal probability sampling is based on the proportion $R$ of rows in cluster $C$ that corresponds to $Q$ . Computing the exact $R$ for each cluster is as costly as evaluating the query itself, rendering the approximation useless. Inspired by (Zhang et al., 2016), we will only approximate $R$ to avoid an overhead in response time. Given a query $Q$ defined by a set of ranges: $Q=\{\forall d\in D^{Q}\text{ | }r_{d}=[l_{b}^{d},u_{b}^{d}]\}$ on each dimension, we assume that the dimensions are not correlated (independent). We will compute the sub-proportions $R^{d}$ on each dimension as follows:

R^{d}=R^{d\geq}(l_{b}^{d})-R^{d\geq}(u_{b}^{d})

\text{ where : }R^{d\geq}(x)=\frac{|rows^{d}\geq x|}{S}\text{ and }S\text{ is % the cluster size}

The proportion $R^{d}$ is computed based on the proportions $R^{d\geq}(l_{b}^{d})$ and $R^{d\geq}(u_{b}^{d})$ of records whose dimension $d$ values are $\geq l_{b}^{d}$ and $\geq u_{b}^{d}$ , respectively. Based on the assumption of independence between dimensions, $R$ can be obtained as follows:

(1)

R=\prod^{d\in D^{Q}}R^{d}\textbf{ and }p_{j}=\frac{R_{j}}{\sum_{i=0}^{N^{Q}}R_% {i}}

where $N^{Q}$ is the number of clusters covering $Q$ . The approximated $R$ can then be used to obtain the sampling probabilities $p_{j}$ for the $jth$ cluster as shown in Equation 1. Even this approximation requires a lot of calculations, which may cause similar overhead as the exact $R$ . To bypass this limitation, we associate each cluster with a set of metadata that accelerates these computations for any given query (see Algorithm 1).

Algorithm 1 Cluster metadata

T=\{C_{1},C_{2},...,C_{N}\}

: Set of clusters

\textbf{Clusters\_metas}\leftarrow create\_global\_meta()

3:for

\text{each }C\in T

cluster\_meta\leftarrow[]

\textbf{datas\_meta}\leftarrow create\_datas\_meta\_(C)

6: for

\text{each }d\in D

7: for

\text{each }v\in|d|_{C}

R^{d\geq}(v)\leftarrow portions\_greater\_(C,d,v)

\textbf{datas\_metas}.add(d,v,R^{d\geq}(v))

10: end for

11:

v^{d}_{min},v^{d}_{max}\leftarrow min\_max(|d|_{C})

12:

cluster\_meta.add(v^{d}_{min},v^{d}_{max})

13: end for

14:

\textbf{Clusters\_metas}.add(cluster\_meta)

15:

save(\textbf{datas\_metas})

16:end for

17:

save(\textbf{Clusters\_metas})

For each cluster $C$ and for each distinct value $v$ of dimension $d\in D$ in $C\in T$ (Lines 5,6 Algorithm 1), $R^{d\geq}(v)$ is stored in the dedicated meta file for the cluster where the entry is in the form $\{d,v,R^{d\geq}(v)\}$ (Line 8 Algorithm 1). These metadata will be used by each data provider to quickly access precomputed proportions that correspond to the range of a given $Q$ . Thus significantly reducing the overhead in the online phase. To further improve the performances, Algorithm 1 stores additional global metadata about the clusters Clusters_metas, enabling the system to easily identify the clusters $C^{Q}$ that correspond to $Q$ before even computing the proportions. In a dedicated global file Clusters metas, for each dimension $d\in D$ in cluster $C$ , Algorithm 1 (Line 11,13) stores $v^{d}_{\min}$ ( $v^{d}_{\max}$ ), the minimum (maximum) value of $d$ in $C$ . Based on these metadata in Clusters metas, the system is able to focus only on a small subset of the database $C^{Q}$ that actually contains rows matching $Q$ instead of $T$ , thus reducing the processing time of $Q$ . The set $C^{Q}$ is defined as follows:

(2)

\begin{array}[]{ll}C^{Q}=\{\forall C\in T\text{ | }\forall d\in D^{Q}\text{ , % }[v^{d}_{min},v^{d}_{max}]\cap r_{d}\neq\emptyset\}&\\ \text{ where }r_{d}\text{ is the interval of $Q$ in dimension d.}\end{array}

Since we are able to identify the clusters $C^{Q}$ concerned by $Q$ , it only makes sense to approximate $Q$ only when $N^{Q}$ is bigger than a certain threshold $N^{min}$ . This threshold can be set independently by each data provider based on the size of the clusters, the processing time required for a single cluster, and the hardware and software infrastructure. The cost of saving these metadata is very negligible compared to the actual table and clusters. We used the same data structure like (Zhang et al., 2016) which is very efficient. In Section 6 we show the space needed for each database.

Once the sampling is applied according to the probability computed using Equation 1, the Hansen-Hurwitz estimator (Lohr, 2009) is used to obtain the final estimation of $Q$ . The estimation is done as follows:

(3)

\begin{array}[]{ll}E(Q,C^{Q}_{S})=\frac{1}{N_{S}}\sum_{i=1}^{N_{S}}\frac{Q(C_{% i})}{p_{i}}\\ \end{array}

where $p_{i}$ is the sampling probability of the $ith$ cluster and $Q(C_{i})$ is the query execution result on the $ith$ cluster

5.3. Federated protocol

In this section, we will review all the steps of online query approximation and how we were able to carefully integrate DP into each step.

5.3.1. Allocation phase

In this step, the data providers $\mathbb{S}$ need to jointly decide the number of clusters to be sampled from each one of them based on the distribution ( $R$ ’s) of data related to $Q$ . So upon receiving the query, each data provider identifies $C^{Q}$ and computes the $R$ for each $C\in C^{Q}$ using the metadata stored locally. Then each one sends to the $Aggregator$ its $N^{Q}$ and $\text{Avg}({\widehat{R}})$ , where $\widehat{R}$ is the set of $R$ ’s of the clusters in $C^{Q}$ and $Avg$ stands for Average. $N^{Q}$ indicates the number of clusters within that data provider that overlap with $Q$ , while $\text{Avg}({\widehat{R}})$ shows the average proportion of rows within those clusters that corresponds to $Q$ . Based on this information, we obtain an aggregated (summary) view of the data distribution of records corresponding to $Q$ in each data provider. Using these insights, the $Aggregator$ finds the best sample size $s_{i}$ for $ith$ data provider using an optimization problem given in Equation 4 that aims to assign a bigger allocation to the data provider with the most data related to $Q$ .

(4)

\begin{array}[]{ll}\mbox{ {\bf maximize}}&\sum_{i=0}^{|\mathbb{S}|}\text{Avg}(% {\widehat{R}})_{i}\times s_{i}\\ \mbox{ {\bf where}}&\sum_{i=0}^{|\mathbb{S}|}s_{i}=sr\times\sum_{i=0}^{|% \mathbb{S}|}{N^{Q}_{i}}\\ \mbox{ {\bf and }}&sr\in]0,1[\text{ is the sampling rate}\\ \mbox{ {\bf and }}&s_{i}\in]1,N^{Q}_{i}[\end{array}

In Equation 4, the data provider that holds the most data related to $Q$ (has the bigger $\text{Avg}({\widehat{R}})_{i}$ ) gets more allocation, thus sampling more clusters to approximate $Q$ locally. This reflects the same behaviour as the original collaboration method (described in Section 4): sampling probabilities are computed globally and the clusters of the data provider with the bigger $R^{\prime}s$ are more likely to be sampled than others (higher probabilities, Equation 3). So with our collaboration method, we are able to reproduce similar results and behaviour. It is important to highlight that comparing the $\text{Avg}({\widehat{R}})$ from each data provider is only possible because we imposed they use the same $S$ in order to compute the proportions during the metadata creation phase.

To solve the problem in Equation 4, each data provider shares the $N^{Q}$ and $\text{Avg}({\widehat{R}})$ . Both are sensitive pieces of information that may reveal insights about the individuals within the database. Even if the optimisation in Equation 4 is done over encrypted data, the released allocation $s_{i}$ might give a data provider insights about the other data providers. To secure the release of this information, each data provider uses Laplace mechanism to ensure formal guarantees of privacy. Given a privacy budget of $\epsilon^{O}$ , each data provider perturbs these two values as follows:

(5)

\begin{array}[]{ll}\widetilde{\text{Avg}}({\widehat{R}})=\text{Avg}({\widehat{% R})}+\text{Lap}(\frac{\Delta_{\text{Avg}(\widehat{R}})}{\epsilon^{O}/2})\\ \widetilde{N^{Q}}=N^{Q}+\text{Lap}(\frac{1}{\epsilon^{O}/2})\end{array}

where the sensitivity of $N^{Q}$ to the absence/presence of an individual is 1, and the sensitivity of $\text{Avg}(\widehat{R})$ , is $\Delta_{\text{Avg}(\widehat{R}}$ .

Theorem 5.1 (Sensitivity of estimator $\Delta_{\text{Avg}(\widehat{R}}$ ).

For any two neighbouring databases $T$ , $T^{\prime}$ the sensitivity of $\text{Avg}({\widehat{R})}$ is defined as:

\Delta_{Avg(\widehat{R})}=max(\frac{\Delta_{R}}{N^{min}},\frac{1}{N^{min}+1})% \\ \text{ where: }\Delta_{R}=1-(1-\frac{1}{S})^{\left|D\right|}

The proof is given in appendix A.

With this perturbation, the collaboration between data providers for deciding the allocation does not reveal any sensitive information. So the optimization problem is formulated as follows:

(6)

\begin{array}[]{ll}\mbox{ {\bf maximize}}&\sum_{i=0}^{|\mathbb{S}|}\widetilde{% \text{Avg}}({\widehat{R_{i}}})\times s_{i}\\ \mbox{ {\bf where}}&\sum_{i=0}^{|\mathbb{S}|}s_{i}=sr\times\sum_{i=0}^{|% \mathbb{S}|}\widetilde{N^{Q}_{i}}\\ \mbox{ {\bf and }}&sr\in]0,1[\text{ is the sampling rate}\\ \mbox{ {\bf and }}&s_{i}\in]1,\widetilde{N^{Q}_{i}}[\end{array}

The test of $N^{Q}<N^{min}$ comes after the allocation (collaboration) phase in order to encourage all data providers to participate. Otherwise, if a data provider does not participate in allocation because locally approximating $Q$ is not possible, this may reveal information about the size of its data to other data providers.

5.3.2. Sampling phase

After the allocation phase, each data provider receives an allocation $s$ : the number of clusters to process for the $Q$ approximation. Using the $\widehat{R}$ computed locally, the data provider computes the sampling probabilities for $C^{Q}$ and then performs unequal probability sampling to randomly select $s$ clusters. Since the sampling probabilities are computed based on the rows (individuals) in the database, the result of the sampling (choices) may leak information about the presence/absence of any individual. To guarantee DP, our system uses the Exponential Mechanism (EM) to select the $s$ clusters $C^{Q}_{S}\subset C^{Q}$ (Algorithm 2) while consuming $\epsilon^{S}$ privacy budget.

Algorithm 2

EM\_sampling

C^{Q}:\text{set of clusters}

\widehat{R}:\text{set of corresponding $R^{\prime}s$ to $C^{Q}$}

s:\text{sample size}

\epsilon^{S}:\text{total budget}

P\leftarrow\text{get\_sampling\_probabilities}(\widehat{R})

\triangleright

Equation 1

P^{EM}\leftarrow[]

\epsilon^{s}\leftarrow\epsilon^{S}/s

5:for

i\in[1,N^{Q}]

P^{EM}[i]\leftarrow\text{exp}\left(\frac{\epsilon^{s}\times P[i]}{2\times% \Delta p}\right)

7:end for

C^{Q}_{S}\leftarrow\text{random\_choice}(C^{Q},P^{EM},s)

\textbf{Return }C^{Q}_{S},P

The score of the $ith$ cluster $C_{i}\in C^{Q}$ is its own sampling probability $p_{i}$ (Algorithm 2 line 1), which means the scoring function $L$ of EM is defined by the computation in Equation 1. So to calibrate the noise (randomness) of EM, we must find the sensitivity of this function $L$ to the absence/presence of any individual in the database.

Consider two neighbouring databases $T$ and $T^{\prime}$ , where $T^{\prime}$ is obtained by adding any random record (which represents an individual) to $T$ at any possible cluster. Given a range query $Q$ , in order to measure $\Delta p_{i}$ (sensitivity of $p_{i}$ , which is the same as $L$ ) we assume the worst case scenario for $T$ and $T^{\prime}$ : all clusters of $C^{Q}(C^{Q}\subset T$ ) each have a record that corresponds to $Q$ . In this case, their probabilities are the same: $p=\frac{1}{N^{Q}}$ . In $T^{\prime}$ , one record is added to another cluster $C^{\prime}$ outside of $C^{Q}$ that matches $Q$ . Thus $C^{\prime Q}=C^{Q}\cup\{C^{\prime}\}$ and $N^{\prime Q}=N^{Q}+1$ , and for $Q$ all the clusters have the same sampling probability: $p^{\prime}=\frac{1}{N^{Q}+1}$ . So the $\Delta p$ can be computed as follows:

(7)

\begin{array}[]{ll}\Delta p\leq\left|\frac{1}{N^{Q}}-\frac{1}{N^{Q}+1}\right|% \implies\Delta p\leq\frac{1}{N^{Q}\times(N^{Q}+1)}\end{array}

We notice that $\Delta p$ is dependent on the query $Q$ . To find the global maximum value for $\Delta p$ , we replace the $N^{Q}$ by its minimum possible value $N^{min}$ .

Theorem 5.2 (Sensitivity of sampling probability).

For any two neighbouring databases $T$ , $T^{\prime}$ the sensitivity of the sampling probability of any cluster $C$ is bounded by :

\Delta p=\max_{T,T^{\prime}}\left\|p_{C}(T)-p_{C}(T^{\prime})\right\|_{1}=% \frac{1}{N^{min}\times(N^{min}+1)}

where $\left\|.\right\|_{1}$ is the $L_{1}$ norm.

In Algorithm 2 line 5, this sensitivity $\Delta p$ is used for sampling using $EM$ . To manage the total budget $\epsilon^{S}$ allocated for $EM$ in order to safely make $s$ selections (Algorithm 2 line 7), we set $\epsilon^{s}=\frac{\epsilon^{S}}{s}$ the budget of each random selection (Algorithm 2 line 3).

5.3.3. Approximation phase

To obtain the final result from $C^{Q}_{S}$ , each data provider uses the estimator $E$ defined in Equation 3. In order to release the final results securely and have DP privacy guarantees, a well-calibrated noise will be added to the final answer using Laplace Mechanism. To apply Laplace Mechanism, we need to find the sensitivity $\Delta_{E}$ of the estimator. Let us define $\mathbb{E}(C,Q,p)=\frac{Q(C)}{p}$ . We can re-write $E$ as follows :

(8)

\begin{array}[]{ll}E(Q,C^{Q}_{S})=\frac{1}{s}\sum_{i=1}^{s}\mathbb{E}(Q,C_{i},% p_{i})\\ \mbox{{\bf where }}\text{$s$ is the size of }C^{Q}_{S}\end{array}

Which implies that :

(9)

\Delta_{E}=\frac{1}{s}\sum_{i=1}^{s}\Delta_{\mathbb{E}}

To find $\Delta_{E}$ , we will focus on finding $\Delta_{\mathbb{E}}$ , and deduce $\Delta_{E}$ afterwards based on this implication. Given that $\mathbb{E}(C,Q,p)=\frac{Q(C)}{p}$ is a fraction of two real values, it gives a hint that its sensitivity might be unbounded similarly to $Average$ operator (Near and Abuah, 2021). Upon further analysis (see appendix B), we find that $\Delta_{\mathbb{E}}$ is unbounded, which implies $\Delta_{E}$ is also unbounded.

Theorem 5.3 (Sensitivity of estimator $\mathbb{E}$ ).

For any two neighbouring databases $T$ , $T^{\prime}$ the sensitivity of the estimator $\mathbb{E}$ for any cluster $C$ and query $Q$ is unbounded:

\Delta_{\mathbb{E}}=\max_{T,T^{\prime}}\left\|\mathbb{E}(Q,C)-\mathbb{E}(Q,C^{% \prime})\right\|_{1}\geq\frac{N\times S^{D}}{2}-1

where $\left\|.\right\|_{1}$ is the $L_{1}$ norm.

See appendix B for a proof of theorem 5.3.

Given that a global sensitivity does not exist, we resort to the Local Sensitivity (LS) which is measured based on the database instance $T$ . For any database $T^{\prime}$ neighbouring to $T$ obtained by adding 1 row (one individual) that matches the query $Q$ , we can distinguish four scenarios for a cluster $C\in C^{Q}$ (we focus on one cluster $C$ because we are looking for $\Delta_{\mathbb{E}}$ ) that might affect $\mathbb{E}$ :

•

Scenario 1: Cluster $C$ did not receive the new row, but another cluster did.
•

Scenario 2: Cluster $C$ did receive the new row.
•

Scenario 3: Cluster $C$ did not receive the new row but another cluster has been added to $C^{Q}$ , such that $N^{\prime Q}=N^{Q}+1$ .
•

Scenario 4: Cluster did receive the new individual, but only add $+1$ to the $Measure$ attribute of existing aggregate row.

Our aim is to find the upper bound of $LS_{\mathbb{E}}$ , thus we must consider the distance that provides the largest sensitivity. An analysis of each of these scenarios (see Appendix B.2) showed that under a certain condition, either scenario 1 or scenario 4 will yield the biggest distance. For a given cluster $C$ , we can choose the Dominant scenario (which will yield the biggest $LS_{\mathbb{E}}$ ) between scenarios 1 and 4 without needing to compute any of them.

Theorem 5.4 (Dominant distance LS).

the neighbouring scenario 1 will give bigger distance than scenario 4 iff:

Q(C)>\frac{\sum^{R\in\widehat{R}}R}{\Delta_{R}}

See Appendix B.2 for proof.
Since the $LS_{\mathbb{E}}$ is computed based on $T$ , it cannot be used directly to inject noise because the scale of the noise may reveal sensitive information about $T$ (Near and Abuah, 2021). To avoid such information leakage, we will use the smooth sensitivity framework (Nissim et al., 2007) for finding a safer upper bound $S\_LS_{\mathbb{E}}$ for the local sensitivity $LS_{\mathbb{E}}$ . So we redefine our $LS_{\mathbb{E}}$ in terms of a distance $k$ between $T$ and $T^{\prime}$ :

•

Scenario 1: $LS^{k}_{\mathbb{E}}=k\times\frac{Q(C)\times\Delta_{R}}{R}$
•

Scenario 4: $LS^{k}_{\mathbb{E}}=k\times\frac{1}{p}$

See Appendix B.2 for proof.

The safe smooth upper $S\_LS_{\mathbb{E}}$ is defined as follows:

(10)

\begin{array}[]{ll}S\_LS_{\mathbb{E}}=max_{k=0,1,...n}\{e^{-\beta k}\times LS^% {k}_{\mathbb{E}}\}\end{array}

$\text{ where }\beta=\frac{\epsilon^{E}}{2\times\ln(2/\delta)}$ and $(\epsilon^{E},\delta)$ is the privacy budget allocated for releasing the final result.

Algorithm 3

Estimate\_Q

Q:query,C^{Q}_{S}:clusters,(\epsilon^{E},\delta):budget,SMC:bool

result\leftarrow\text{approximate\_Q}(Q,C^{Q}_{S})

\triangleright

Equation 3

S\_LS\leftarrow[]

4:for

i\in[1,N^{Q}_{S}]

S\_LS[i]\leftarrow smooth\_LS(Q,C^{Q}_{S}[i],\epsilon^{E},\delta)

\triangleright

Equation 10

6:end for

LS\_smooth\leftarrow average(S\_LS)

\triangleright

Equation 9

8:if SMC then

send\_secure(result,LS\_smooth)

10:else

11:

dp\_result\leftarrow result+Lap(\frac{2\times LS\_smooth}{\epsilon^{E}})

12:

send(dp\_result)

13:end if

Based on the definitions we gave for $LS^{k}_{\mathbb{E}}$ , the computational overhead to compute the smooth sensitivity for each cluster $C\in C^{Q}_{S}$ is very negligible because: i) All the $R$ ’s and $p$ ’s are computed before this step, and will be reused for each iteration over $k$ ; ii) the maximum value of $k$ (steps) is also bounded by $k=\frac{1}{1-e^{\beta}}+1$ (see Appendix B.3 for proof), which guarantees that the process will terminate; iii) Theorem 5.4 allows to determine which scenario is dominant for any given cluster, thus only computing one $S\_LS_{\mathbb{E}}$ .

Algorithm 3 describes the process of estimating $Q$ over the subset of cluster $C^{Q}_{S}$ . It 3 starts in line 1 by estimating $Q$ according to Equation 3. Then it proceeds to compute the smooth sensitivity (Lines 2-6), where the function $smooth\_LS$ is responsible for computing the smooth sensitivity $S\_LS_{\mathbb{E}}$ for each cluster $C\in C^{Q}_{S}$ as described in Equation 10. Depending on the chosen setup by the data providers, either they compute and send a DP result to the aggregator (Algorithm 3, Lines 10–11) and the aggregator returns the sum to the user. The second option is that data providers share their estimations and computed sensitivities (Algorithm 3, Line 8) with the $Aggregator$ securely using SMC, and obliviously compute the sum of estimations and the max sensitivity to perturb the final result with Laplace Mechanism.

5.4. Privacy accounting

In the online query answering settings under DP, the end user is limited by a total privacy budget of $(\xi,\psi)$ . For each query $Q$ , a budget $(\epsilon,\delta)$ is consumed in order to publish the answer and the end user can interact with system as long as the total budget $(\xi,\psi)$ is not consumed. In this section, we will track the privacy budget $\epsilon$ consumption for each query.

In our proposed protocol the data providers do not share their data, and $Q$ is processed (data access and publishing) in parallel by each data provider. We can just track the consumption on one data provider, and based on the parallel composition property of DP we can deduce the budget consumption for $Q$ on the full system. A data provider starts by publishing the $N^{Q}$ and $\text{Avg}({\widehat{R}})$ using Laplace mechanism for the allocation phase, while consuming a total budget of $\epsilon^{O}$ . Based on the post-processing property of DP, obtaining the sample size $s$ is DP. Afterwards, each data provider uses Exponential Mechanism to sample a subset $C^{Q}_{S}\subset C^{Q}$ while consuming a budget of $\epsilon^{S}$ . To publish an estimation of $Q$ , each data provider uses Laplace mechanism once more, and consumes a budget of $\epsilon^{E}$ . The final step does not in fact guarantee pure DP, since the smooth sensitivity has a $\delta$ failure probability. Based on the sequential composition property of DP, the total budget is: $(\epsilon=\epsilon^{O}+\epsilon^{S}+\epsilon^{E},\delta)$ . Given the parallel composition property, the budget consumption for $Q$ is $(\epsilon,\delta)$ .

In case the data providers used SMC to inject a single noise, based on parallel composition property we deduce that data providers consumed $\epsilon^{O}+\epsilon^{S}$ for the local computation. Afterwards they collectively consumed (once) $\epsilon^{E}$ for publishing the result. By the sequential composition property of DP, the budget consumption for $Q$ is $(\epsilon=\epsilon^{O}+\epsilon^{S}+\epsilon^{E},\delta)$ .

Based on these results, a set of hyperparameters can be set in our system (by database admin for example) that regulates the $\epsilon$ budget distribution at each step of the query processing.
Let $hp_{1},hp_{2}\text{ and }hp_{3}$ be this set of hyperparameters (where $hp_{i}\in]0,1[$ and $hp_{1}+hp_{2}+hp_{3}=1$ ) such that : $\epsilon^{O}=hp_{1}\times\epsilon$ , $\epsilon^{S}=hp_{2}\times\epsilon$ and $\epsilon^{E}=hp_{3}\times\epsilon$ .

6. Evaluation

6.1. Setup

Datasets. We used two big datasets: (i) Adult (Becker and Kohavi, 1996) contains demographic and income information for individuals with $15$ dimensions and $48\times 10^{3}$ records, synthetically scaled up $4\times 10^{6}$ records. (ii) Amazon Review (Ni et al., 2019) is about reviews from Amazon clients across different product categories, with only three “range querable” dimensions and $231\times 10^{6}$ records ( $\sim 120$ Gb). We synthetically added three randomly populated dimensions and random records to reach $4\times 231\times 10^{6}$ records.

A count tensor with column Measure is created from each dataset, aggregating six dimensions of Adult and one dimension of Amazon Review.

Queries and Workloads. We generated random ranges for the queries and ran only those that lead to the approximation ( $N^{min}<N^{Q}$ ) on all data providers. A workload $(m,n)$ is a set of $m$ distinct queries with ranges over $n$ dimensions.

Metrics. An online query is useful if it has a low error rate and low processing time. To measure the query error, we used $\text{{Relative error}}=\frac{|\text{answer}-\text{estimation}|}{\text{answer}}$ . For performance in terms of response time, we used: Speed-UP= $\frac{\text{time of normal computation}}{\text{time of estimate computation}}$ .

Configuration. In our experiments, we assumed that there are one aggregator and four data providers and that each data provider has its own database. Datasets Adult and Amazon Review are horizontally partitioned equally across data providers.

Source code. Based on PostgreSQL⁵⁵5https://developers.google.com/optimization, our solution⁶⁶6https://github.com/AlaEddineLaouir/Federated-Range-Queries.git coded in Python uses the libraries: (i) OrTools⁷⁷7https://developers.google.com/optimization as solver; (ii) Pyro5⁸⁸8https://pyro5.readthedocs.io/en/latest/index.html as communication medium; and, (iii) MPyC⁹⁹9https://mpyc.readthedocs.io/en/latest/mpyc.html as SMC environment. Our implementation is a proof-of-concept in which the clusters of the original table are other smaller tables.

Hyperparameters. In our experiments, the total privacy budget $(\epsilon,\delta)$ for each query is set with $\delta=10^{-3}$ and $\epsilon=1$ (unless other values are indicated for $\epsilon$ ). The budget $\epsilon$ is shared between each step of our solution as follows: $\epsilon^{O}=0.1\times\epsilon$ , $\epsilon^{S}=0.1\times\epsilon$ and $\epsilon^{E}=0.8\times\epsilon$ . To get clusters of the same size, we set the cluster size $S$ to $1\%$ and $0.5\%$ of the total size $T^{a}$ of each data provider for Adult and Amazon Review, respectively.

Metadata space allocation. The metadata for Amazon Review dataset was about 11 MB (56 KB/cluster). As for Adult dataset, it occupied 6.4 MB (64 KB/cluster).

Hardware¹⁰¹⁰10Grid5000: Grisou cluster https://www.grid5000.fr/w/Nancy:Hardware. For each of the data providers and the aggregator, we allocated a dedicated server with the following configuration: $2$ X Intel Xeon E $5-2630$ v $3$ $8$ cores/CPU x $86\_64$ , RAM $128$ GB and $1.2$ TB HDD, and a network with $1$ Gbps + $4$ x $10$ Gbps (SR‑IOV).

6.2. Dimension-based analysis

In these experiments, we evaluated the impact of the number of dimensions in queries on accuracy. To this end, we generated random workloads $(m,n)$ with $m=100$ distinct queries (SUM and COUNT) and dimension $n\in[2,7]$ for Adult and $n\in[2,5]$ for Amazon Review. For the sampling rate, we set it to $5\%$ and $20\%$ for Amazon Review and Adult datasets, respectively.

The results presented in Figure 4 show that our solution achieves very high accuracy for COUNT and SUM queries. The relative error is less than $2.5\%$ (resp. $11\%$ ) on average for COUNT queries on Amazon Review (resp. Adult). As for SUM queries, the error is less than $5\%$ (resp. $17\%$ ) on Amazon Review (resp. Adult). This performance difference is due to the size difference between the databases. In big tables, query results are larger (contain more data), therefore less affected by Laplace Mechanism noise. Interestingly, the results also indicate that queries become more accurate as the number of dimensions decreases. Specifically, with workloads having only $2$ dimensions on both datasets, we reached an error close to $0\%$ . This observed behavior corresponds to our expectations. Because in Equation 1, we approximate $R$ of each cluster and the accuracy of this approximation improves as the number of dimensions decreases, bringing the approximation closer to the exact $R$ . Thus, we have more accurate sampling probabilities which affect the estimation of the final result. For the speedup, the results in Figure 7 show that the higher the number of dimensions, the less speedup is gained. From the results in Figure 7, the speedup drops from approximately $8x$ to $6x$ as the number of dimensions increases from $2$ to $5$ on Amazon Review dataset. This drop is attributed to the sampling probabilities approximation phase, where our algorithm looks up the preprocessed metadata. The higher the number of dimensions, the more metadata it needs to look up. However, this effect becomes negligible on larger databases. Because even in these results, the speedup remains very significant.

6.3. Sampling rate-based analysis

In this analysis, we examined the effect of sampling rate on query quality. For each database, we generated two random workloads for COUNT and SUM queries of $m=100$ and $n=4$ . We varied the sampling rate between $5\%$ and $20\%$ for each experiment and measured the quality obtained in terms of accuracy and speed-up. From the results in Figure 5, we observe that a higher sampling rate provides slightly better accuracy: reaching a relative error of less than $1\%$ with a $20\%$ sampling rate for COUNT queries on Amazon Review dataset.

Regarding the speed-up, we note that our solution reaches up to a $7x$ compared to a normal execution (without approximation) on Amazon Review (with $4$ dimensional queries). Additionally, the speed-up gains in Amazon Review are $4x$ more significant than those in Adult. This result indicates that our solution provides more speed for larger datasets. Also based on the results in Figure 5, the tradeoff between speed-up and accuracy is noticeable. We observe that the larger the sampling, the less the speed-up is gained. On the other hand, accuracy improves with higher sampling rates. We can say that, based on the results shown in this experiment, accuracy gains with higher sampling are very costly in terms of speed-up. But it is up to the users (data analysts) to define the sampling rate according to their needs.

6.4. Privacy budget-based analysis

In these experiments, we analyzed the effect of the privacy budget $\epsilon$ on query quality. We generated two random workloads of $m=100$ and $n=4$ for COUNT and SUM queries and set the sampling rate to $5\%$ and $10\%$ for Amazon Review and Adult, respectively. We varied $\epsilon$ between $0.1$ and $1.3$ and captured the performance on each workload. From the results in Figure 6, we can immediately observe the typical trend of any DP mechanism (larger $\epsilon$ leads to better accuracy).

Interestingly, SUM queries are able to provide better utility (lower relative error) than COUNT queries. This happens because SUM queries yield more substantial results (larger query responses) than COUNT queries, making them less affected by noise added to the response. A similar observation applies when comparing results between the two databases, with workloads on Amazon Review preserving more accuracy than those on Adult. This is attributed to the fact that the Amazon Review dataset is much larger than Adult, causing queries to be less affected by the added noise. Based on this observation, we can predict that as the database size increases, the accuracy of our solution will improve by using smaller values for $\epsilon$ . Regarding speed-up, the results in Figure 7 show that $\epsilon$ levels have no effect.

6.5. SMC vs DP in terms of sharing results

To examine the performance of our SMC-based solution to share final results, we conducted experiments using an Adult dataset split across four data providers. We generated five random two-dimensional COUNT queries. Each query was repeated five times (with and without SMC) and we measured the speed-up and the the range of noise added using the Laplace mechanism at each iteration.

The results in Figure 8 show, for each query, the range of noise sampled using the Laplace mechanism for both solutions at each iteration and speed-up. We notice in Figure 8 that using SMC to share only the sensitivity and the local result does not produce significant overhead, which corresponds to the simulation results in Figure 1. Concerning the injected noise, which affects the precision of the query result, the use of SMC allows a more restricted range of perturbation. Meanwhile, if each data provider perturbs its local data without SMC, there could be two cases: (i) the noises from the data providers cancel each other out, or (ii) the noise accumulates. In the first case, the sum of noises is close to zero because some are positive and others negative, which will help improve accuracy. In the second case, which represents the worst case where most of the noise is positive or negative, the accuracy of the results will be greatly affected.

Based on the experiment results, a user/data provider can choose the appropriate query execution process (with or without SMC) based on their needs, preferring accuracy over speed-up or vice versa.

6.6. Resilience to Learning-Based Attacks

DP prevents membership attacks revealing the presence/absence of an individual in the database. In (Cormode, 2010), the author introduced a simple attack that allows the disclosure of an individual’s sensitive $SA$ attribute based on anonymized data. This attack relies on training a Naive Bayes Classifier (NBC) using the results of COUNT queries from a noisy database, and this classifier will be used to predict the value of $SA$ based on a given set of $QI$ (quasi-identifiers) attribute values of an individual. In our data model, $SA$ corresponds to one of the dimensions $d_{SA}\in D$ , and $QI$ is the subset $D_{QI}\subseteq D\setminus\{d_{SA}\}$ . Given $V_{QI}=\{v_{1},...,v_{|D_{QI}|}\}$ for $D_{QI}$ , a NBC attaches a probability to each possible value $y$ of $d_{SA}$ ( $y\in|d_{SA}|$ ). The predicted value $\hat{y}$ is the one with the highest probability according to Bayes Theorem (Cormode, 2010):

\hat{y}=\underset{y\in|d_{SA}|}{\arg\max}\text{ }P(y)\prod^{|D_{QI}|}_{i=1}P(v% _{i}|y)/P(v_{i})

To make these predictions, the classifier goes through a training phase during which it learns the conditional probabilities using the queries COUNT(*) (or SUM(Measure)) issued by the attacker to the database. The learned probabilities are saved and later used to make predictions. The number of queries $nQueries$ needed is:

nQueries=1+||d_{SA}||+||d_{SA}||\times\sum_{d{QI}\in D_{QI}}||d{QI}||

which is used to compute the size of the database, $P(y)$ and $P(v|y)/P(v)$ for all values and dimensions. For instance, consider a table T with $10000$ rows and $|d_{SA}|=[20,..,60]$ is the dimension for Age attribute. To compute $P(Age=25)$ , we use the following COUNT query: SELECT COUNT(*) FROM T WHERE 25 <= Age <= 25 )/ 10000. This huge number of queries can be easily issued to a published database using a DP algorithm with a fixed privacy budget (e.g. PrivBayes(Zhang et al., 2017)), and from which the attacker can infer some knowledge (Cormode, 2010; Gkountouna et al., 2022).

However, the database is not published in our system. As we showed in Section 5.4 the attacker has a limited budget $(\xi>0,\psi>0)$ , from which each issued query consumes a privacy budget $(\epsilon>0,\delta>0)$ based on a sequential composition 3.9. Since $nQueries$ can be very large, $\epsilon$ must be very small $\epsilon=\xi/nQueries$ and $\delta=\psi/nQueries$ , thus losing the utility of query answers. An alternative to sequential composition is Advanced composition (Lohr, 2009; Kairouz et al., 2015), which allows the queries to have a greater budget $\epsilon$ without exceeding $\xi$ . With the advanced composition, the budget of each query is: $\epsilon=\xi/\left(2\times\sqrt{2\times nQueries\times log(\frac{1}{\delta})}% \right)\text{ and }\delta=\psi/nQuesries$ . We notice that $\xi/\left(2\times\sqrt{2\times nQueries\times log(\frac{1}{\delta})}\right)>% \xi/nQueries$ , which means queries have better utility.

To evaluate the resilience of our system against this learning-based attack, we tested both sequential compositions and the two allowed queries COUNT and SUM. We also considered parallel composition which allows multiple attackers to create a coalition, where each of them executes only one query (to maximize utility) and combines it with those of other attackers to train the classifier. The ingredients of our experiments are as follows:

Setup: We used Adult dataset with four data providers. We selected $3$ dimensions of our table to be $D_{QI}$ and $1$ dimension to be $d_{SA}$ where $||d_{SA}||=$ 100 (i.e. the number of classes for NBC). We also set $\psi=10^{-6}$ and we varied $\xi$ between $1$ and $100$ since there is no standard value (Lohr, 2009; Laud and Pankova, 2019).

Evaluation: To assess the quality of the learning attack, we measured the accuracy of the NBC in predicting the value of $SA$ for each row in the original table $accuracy=\frac{\text{number of correct predictions}}{\text{total number of % predictions}}$ .

	$\xi=1$	$\xi=20$	$\xi=50$	$\xi=100$
Sequential / COUNT	$<1\%$	$<1\%$	$<1\%$	$<1\%$
Sequential / SUM	$<1\%$	$<1\%$	$<1\%$	$<1\%$
Advanced / COUNT	$<1\%$	$<1\%$	$<1\%$	$<1\%$
Advanced / SUM	$<1\%$	$<1\%$	$<1\%$	$<1\%$
Coalition / COUNT	$<1\%$	$<1\%$	$<1\%$	$<1\%$
Coalition / SUM	$<1\%$	$<1\%$	$<1\%$	$<1\%$

Table 1. Inference accuracy based on

\xi

The results in Table 1 show that in all scenarios the accuracy is $<1\%$ . Since the $SA$ we used had $100$ possible values, this means that the trained classifier is given similar accuracy as randomly assigning a value for $SA$ in each row. Three reasons can be put forward to explain the failure of the learning-based attack: i) our system is interactive (the database is not released) and the budget is limited, thus it is difficult to have good accuracy for large numbers of queries by a single attacker; ii) query answers in our system are approximated with random sampling, which will introduce some error; iii) the smooth sensitivity has a considerable scale, and in the case of queries that collects small values, the accuracy can be lost even for large values of $\epsilon$ .

Similar results were obtained when fixing the $\xi=100$ and changing the number of dimensions in $D_{QI}$ from 1, 3, 5 to 8. This shows the resilience of our system in different settings.

7. Discussion

In this section, we discuss the constraints, limits and points of improvement that could be integrated into our solution. In order to approximate the sampling probabilities in Section 5.2, we assumed that the dimensions are independent and that there is no correlation between them. However, this assumption is not valid in some cases. For example, if an individual’s $Age$ is less than $25$ , this implies with a high probability that he/she is still studying ( $profession=student$ ). Likewise, if $Age>65$ , the attribute $profession=retired$ . When it comes to range queries, capturing and managing these dependencies is non-trivial; so we will leave it for future work.

We also restricted the data providers to using the same value of $S$ in order to approximate the $R$ . Otherwise, we cannot compare the $\text{Avg}({\widehat{R}})$ in the allocation phase (Section 5.3.1). To agree on the same $S$ , each data provider $\mathbb{S}_{i}$ can share their true $S_{i}$ with the others, and they will use then the maximum $S_{i}$ (which will guarantee that all the $R^{\prime}s$ computed are $\leq 1$ ). The value of $S_{i}$ itself is not sensitive since it is usually a constant in a database system. But if this is deemed sensitive in a particular case, then data providers can simply share a randomly chosen $S^{\prime}_{i}$ value such that: $S_{i}\leq S^{\prime}_{i}\leq S^{m}_{i}$ where $S^{m}_{i}$ is an upper bound chosen by each data provider (e.g. $S^{m}_{i}=2\times S_{i}$ ).

In our solution, we focused on protecting the intermediate (summary information) and final result from inference attacks with the use of Differential Privacy. However, we have not directly addressed the risks associated with side-channel attacks. It is easy to see that thanks to the collaboration method that we propose, we manage to avoid certain risks mentioned in (Qiu et al., 2023), such as: memory access models and communication volumes since all data-based computations are performed locally at each data provider and the communication cost is constant and independent of the query. But we have postponed further consideration of this aspect of the problem to dedicated work.

Our solution serves as the first building block towards a more comprehensive solution that handles more complex queries, such as GROUP-BY queries. Integrating such clauses in the SQL query is not so trivial, and adding noise to the final result will not be enough to guarantee privacy (Desfontaines et al., 2020). Other aggregations, such as average, standard deviation, and variance, can be derived from SUM and COUNT using the sequential composition of DP. However, to handle other aggregations (such as Min, Max and Mode), different estimators are required.

Finally, during our evaluation, we built a proof of concept of our solution on PostgreSQL. It would be interesting to incorporate it directly into any DBMS, which would further improve our results.

8. Conclusion

In our study, we introduced a lightweight collaborative approach for online range query approximation in a federated environment. Our experimental results demonstrated the performance improvements our solution is capable of delivering, with processing times improved by up to 8x compared to plain-text execution, while ensuring end-to-end privacy with minimal loss of accuracy. Our solution uses cluster sampling and query estimation techniques that take into account data distribution to preserve query utility in terms of speed and accuracy. This work lays a solid foundation for future work to handle more complex queries while maintaining the same level of performance.

References

(1)
Abowd (2018) John M Abowd. 2018. The US Census Bureau adopts differential privacy. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining. 2867–2867.
Acharya et al. (1999) Swarup Acharya, Phillip B Gibbons, Viswanath Poosala, and Sridhar Ramaswamy. 1999. The aqua approximate query answering system. In Proceedings of the 1999 ACM SIGMOD international conference on Management of data. 574–576.
Agarwal et al. (2013) Sameer Agarwal, Barzan Mozafari, Aurojit Panda, Henry Milner, Samuel Madden, and Ion Stoica. 2013. BlinkDB: queries with bounded errors and bounded response times on very large data. In Proceedings of the 8th ACM European conference on computer systems. 29–42.
Agrawal et al. (2006) Rakesh Agrawal, Dmitri Asonov, Murat Kantarcioglu, and Yaping Li. 2006. Sovereign joins. In 22nd International Conference on Data Engineering (ICDE’06). IEEE, 26–26.
Ahmadvand et al. (2019) Hossein Ahmadvand, Maziar Goudarzi, and Fouzhan Foroutan. 2019. Gapprox: using gallup approach for approximation in big data processing. Journal of Big Data 6 (2019), 1–24.
Bater et al. (2017) Johes Bater, Gregory Elliott, Craig Eggen, Satyender Goel, Abel N Kho, and Jennie Rogers. 2017. SMCQL: Secure Query Processing for Private Data Networks. Proc. VLDB Endow. 10, 6 (2017), 673–684.
Bater et al. (2018) Johes Bater, Xi He, William Ehrich, Ashwin Machanavajjhala, and Jennie Rogers. 2018. Shrinkwrap: efficient sql query processing in differentially private data federations. Proceedings of the VLDB Endowment 12, 3 (2018).
Bater et al. (2020) Johes Bater, Yongjoo Park, Xi He, Xiao Wang, and Jennie Rogers. 2020. Saqe: practical privacy-preserving approximate query processing for data federations. Proceedings of the VLDB Endowment 13, 12 (2020), 2691–2705.
Becker and Kohavi (1996) Barry Becker and Ronny Kohavi. 1996. Adult. UCI Machine Learning Repository. DOI: https://doi.org/10.24432/C5XW20.
Bittau et al. (2017) Andrea Bittau, Úlfar Erlingsson, Petros Maniatis, Ilya Mironov, Ananth Raghunathan, David Lie, Mitch Rudominer, Ushasree Kode, Julien Tinnes, and Bernhard Seefeld. 2017. Prochlo: Strong privacy for analytics in the crowd. In Proceedings of the 26th symposium on operating systems principles. 441–459.
Cao et al. (2021) Lei Cao, Dongqing Xiao, Yizhou Yan, Samuel Madden, and Guoliang Li. 2021. ATLANTIC: making database differentially private and faster with accuracy guarantee. (2021).
Chaudhuri et al. (2007) Surajit Chaudhuri, Gautam Das, and Vivek Narasayya. 2007. Optimized stratified sampling for approximate query processing. ACM Transactions on Database Systems (TODS) 32, 2 (2007), 9–es.
Cormode (2010) Graham Cormode. 2010. Individual privacy vs population privacy: Learning to attack anonymization. arXiv preprint arXiv:1011.2511 (2010).
Desfontaines et al. (2020) Damien Desfontaines, James Voss, Bryant Gipson, and Chinmoy Mandayam. 2020. Differentially private partition selection. arXiv preprint arXiv:2006.03684 (2020).
Dwork et al. (2014) Cynthia Dwork, Aaron Roth, et al. 2014. The algorithmic foundations of differential privacy. Foundations and Trends® in Theoretical Computer Science 9, 3–4 (2014), 211–407.
Erlingsson et al. (2014) Úlfar Erlingsson, Vasyl Pihur, and Aleksandra Korolova. 2014. Rappor: Randomized aggregatable privacy-preserving ordinal response. In Proceedings of the 2014 ACM SIGSAC conference on computer and communications security. 1054–1067.
Eskandarian and Zaharia (2017) Saba Eskandarian and Matei Zaharia. 2017. Oblidb: Oblivious query processing for secure databases. arXiv preprint arXiv:1710.00458 (2017).
Gkountouna et al. (2022) Olga Gkountouna, Katerina Doka, Mingqiang Xue, Jianneng Cao, and Panagiotis Karras. 2022. One-off disclosure control by heterogeneous generalization. In 31st USENIX Security Symposium (USENIX Security 22). 3363–3377.
Goiri et al. (2015) Inigo Goiri, Ricardo Bianchini, Santosh Nagarakatte, and Thu D Nguyen. 2015. Approxhadoop: Bringing approximations to mapreduce frameworks. In Proceedings of the Twentieth International Conference on Architectural Support for Programming Languages and Operating Systems. 383–397.
Haas and König (2004) Peter J Haas and Christian König. 2004. A bi-level bernoulli scheme for database sampling. In Proceedings of the 2004 ACM SIGMOD international conference on Management of data. 275–286.
Hellerstein et al. (1997) Joseph M. Hellerstein, Peter J. Haas, and Helen J. Wang. 1997. Online Aggregation. In SIGMOD 1997, Proceedings ACM SIGMOD International Conference on Management of Data, May 13-15, 1997, Tucson, Arizona, USA. 171–182.
Kairouz et al. (2015) Peter Kairouz, Sewoong Oh, and Pramod Viswanath. 2015. The composition theorem for differential privacy. In International conference on machine learning. PMLR, 1376–1385.
Laud and Pankova (2019) Peeter Laud and Alisa Pankova. 2019. Interpreting epsilon of differential privacy in terms of advantage in guessing or approximating sensitive attributes. arXiv preprint arXiv:1911.12777 (2019).
Li et al. (2016) Feifei Li, Bin Wu, Ke Yi, and Zhuoyue Zhao. 2016. Wander join: Online aggregation via random walks. In Proceedings of the 2016 International Conference on Management of Data. 615–629.
Liagouris et al. (2021) John Liagouris, Vasiliki Kalavri, Muhammad Faisal, and Mayank Varia. 2021. Secrecy: Secure collaborative analytics on secret-shared data. arXiv preprint arXiv:2102.01048 (2021).
Lohr (2009) Sharon L. Lohr. 2009. Sampling : Design and Analysis.
Near and Abuah (2021) Joseph P. Near and Chiké Abuah. 2021. Programming Differential Privacy. Vol. 1. https://programming-dp.com/
Ni et al. (2019) Jianmo Ni, Jiacheng Li, and Julian McAuley. 2019. Justifying recommendations using distantly-labeled reviews and fine-grained aspects. In Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP). 188–197.
Nissim et al. (2007) Kobbi Nissim, Sofya Raskhodnikova, and Adam Smith. 2007. Smooth sensitivity and sampling in private data analysis. In Proceedings of the thirty-ninth annual ACM symposium on Theory of computing. 75–84.
Olken and Rotem (1986) Frank Olken and Doron Rotem. 1986. Simple random sampling from relational databases. (1986).
Olken and Rotem (1995) Frank Olken and Doron Rotem. 1995. Random sampling from databases: a survey. Statistics and Computing 5 (1995), 25–42.
Piatetsky-Shapiro and Connell (1984) Gregory Piatetsky-Shapiro and Charles Connell. 1984. Accurate estimation of the number of tuples satisfying a condition. ACM Sigmod Record 14, 2 (1984), 256–276.
Qin and Rusu (2014) Chengjie Qin and Florin Rusu. 2014. PF-OLA: a high-performance framework for parallel online aggregation. Distributed and Parallel Databases 32 (2014), 337–375.
Qiu et al. (2023) Lina Qiu, Georgios Kellaris, Nikos Mamoulis, Kobbi Nissim, and George Kollios. 2023. Doquet: Differentially Oblivious Range and Join Queries with Private Data Structures. Proceedings of the VLDB Endowment 16, 13 (2023), 4160–4173.
Song et al. (2018) Guangxuan Song, Wenwen Qu, Xiaojie Liu, and Xiaoling Wang. 2018. Approximate calculation of window aggregate functions via global random sample. Data Science and Engineering 3 (2018), 40–51.
Team et al. (2017) ADP Team et al. 2017. Learning with privacy at scale. Apple Mach. Learn. J 1, 8 (2017), 1–25.
Wang and Jajodia (2008) Lingyu Wang and Sushil Jajodia. 2008. Security in Data Warehouses and OLAP Systems. In Handbook of Database Security - Applications and Trends, Michael Gertz and Sushil Jajodia (Eds.). Springer, 191–212.
Wu et al. (2010) Sai Wu, Beng Chin Ooi, and Kian-Lee Tan. 2010. Continuous sampling for online aggregation over multiple queries. In Proceedings of the 2010 ACM SIGMOD International Conference on Management of data. 651–662.
Xu et al. (2008) Fei Xu, Christopher M. Jermaine, and Alin Dobra. 2008. Confidence bounds for sampling-based group by estimates. ACM Trans. Database Syst. 33, 3 (2008), 16:1–16:44.
Zhang et al. (2017) Jun Zhang, Graham Cormode, Cecilia M Procopiuc, Divesh Srivastava, and Xiaokui Xiao. 2017. Privbayes: Private data release via bayesian networks. ACM Transactions on Database Systems (TODS) 42, 4 (2017), 1–41.
Zhang et al. (2016) Xuhong Zhang, Jun Wang, and Jiangling Yin. 2016. Sapprox: Enabling efficient and accurate approximations on sub-datasets with distribution-aware online sampling. Proceedings of the VLDB Endowment 10, 3 (2016), 109–120.
Zheng et al. (2017) Wenting Zheng, Ankur Dave, Jethro G Beekman, Raluca Ada Popa, Joseph E Gonzalez, and Ion Stoica. 2017. Opaque: An oblivious and encrypted distributed analytics platform. In 14th USENIX Symposium on Networked Systems Design and Implementation (NSDI 17). 283–298.

Appendix A Sensitivity summarised information

In order to obtain the allocation (sample size) $s$ based on the inter/intra data provider data distribution, each data provider communicate $Avg(\widehat{R_{i}})$ and $N^{Q}$ . The sensitivity of the $N^{Q}$ is straight forward, given a query $Q$ and any two neighbouring database $T$ and $T^{\prime}$ : $\Delta_{N^{Q}}=\left|N^{Q}-N^{\prime Q}\right|\leq 1$ . Adding/removing and individual at most add/remove a cluster $C$ from $C^{Q}$ . For the sensitivity of $Avg(\widehat{R_{i}})$ , we need to consider first the sensitivity of a single $R$ .

A.1. Sensitivity $R$

Given a range query $Q$ and two neighbouring databases $T$ (with cluster $C$ ) and $T^{\prime}$ (with cluster $C^{\prime}$ ), we consider the case where $C^{\prime}$ has and additional row for an extra individual. that implies:

(11)

\begin{array}[]{ll}R=\prod^{d\in D^{Q}}R^{d}\text{ and }R^{\prime}=\prod^{d\in D% ^{Q}}(R^{d}+\frac{1}{S})\\ \Delta_{R}=\left|R^{\prime}-R\right|=\prod^{d\in D^{Q}}(R^{d}+\frac{1}{S})-% \prod^{d\in D^{Q}}R^{d}\end{array}

where $D^{Q}$ is the set of dimensions defining $Q$ . In order to obtain the upper bound of $\Delta_{R}$ , we consider $R^{\prime}=1$ which implies $R=(1-\frac{1}{S})^{\left|D^{Q}\right|}$ :

(12)

\Delta_{R}=1-(1-\frac{1}{S})^{\left|D^{Q}\right|}

Since the values of $\left|D^{Q}\right|$ and $S$ are publicly known, there no information leak when using $\Delta_{R}$ based on these values. The other possible scenarios of neighbouring are : 1) $C^{\prime}$ has on row less, which will give the same result as the previous one. 2) $C^{\prime}$ has new/lost individual but only affected the column ”Measure” of a row by $\pm$ 1, then $\Delta_{R}=0$ . 3) Case where a cluster $C$ wasn’t in $C^{Q}$ in $C^{\prime}$ has an additional row and his in $C^{Q}$ (or vice versa), in this case $\Delta_{R}=\frac{1}{S^{\left|D^{Q}\right|}}$ . We can prove that $1-(1-\frac{1}{S})^{\left|D^{Q}\right|}\geq\frac{1}{S^{\left|D^{Q}\right|}}$ :

(13)

\begin{array}[]{ll}1-(1-\frac{1}{S})^{\left|D^{Q}\right|}=1-(1-\frac{\left|D^{% Q}\right|}{S}+\frac{\left|D^{Q}\right|\times(\left|D^{Q}\right|-1)}{2\times S^% {2}}-...)\\ 1-(1-\frac{1}{S})^{\left|D^{Q}\right|}=\frac{\left|D^{Q}\right|}{S}-\frac{% \left|D^{Q}\right|\times(\left|D^{Q}\right|-1)}{2\times S^{2}}+...\end{array}

Since $S\gg D$ , we can assume $1-(1-\frac{1}{S})^{\left|D^{Q}\right|}\approx\frac{\left|D^{Q}\right|}{S}$ . Then :

(14)

1-(1-\frac{1}{S})^{\left|D^{Q}\right|}>\frac{1}{S^{\left|D^{Q}\right|}}% \Leftrightarrow\frac{\left|D^{Q}\right|}{S}\geq\frac{1}{S^{\left|D^{Q}\right|}}

Which is always true ( $S\gg 0\text{ and }\left|D^{Q}\right|\geq 1$ )

A.2. Sensitivity $Avg(\widehat{R})$

The average of $R^{\prime}s$ , $\widehat{R}$ , of a data provider’s set of cluster $C^{Q}$ is computed as follows : $Avg(\widehat{R})=\frac{\sum^{R\in\widehat{R}}R}{N^{Q}}$ . For any two neighbouring databases $T$ and $T^{\prime}$ , there is two cases where the $Avg(\widehat{R})$ is effected: 1)One of the clusters in $C^{\prime Q}$ has additional row compared to his counter part in $C^{Q}$ 2) $C^{\prime}Q$ has new cluster due to the presence of an individual thus $N^{\prime Q}=N^{Q}+1$ . Which will give:

(15)

\Delta_{Avg(\widehat{R})}=\left\{\begin{array}[]{ll}&\left|\frac{\sum^{R\in% \widehat{R}}R}{N^{Q}}-\frac{\Delta_{R}+\sum^{R\in\widehat{R}}R}{N^{Q}}\right|=% \frac{\Delta_{R}}{N_{Q}}\\ &\left|\frac{\sum^{R\in\widehat{R}}R}{N^{Q}}-\frac{\frac{1}{S^{\left|D^{Q}% \right|}}+\sum^{R\in\widehat{R}}R}{N^{Q}+1}\right|\end{array}\right.

We can simplify the the second $\Delta_{Avg(\widehat{R})}$ as follows:

(16)

\begin{array}[]{ll}\left|\frac{\sum^{R\in\widehat{R}}R}{N^{Q}}-\frac{\frac{1}{% S^{\left|D^{Q}\right|}}+\sum^{R\in\widehat{R}}R}{N^{Q}+1}\right|=\left|\frac{% Avg(\widehat{R})}{N^{Q}+1}-\frac{\frac{1}{S^{\left|D^{Q}\right|}}}{N^{Q}+1}% \right|\\ \text{ which is :}\leq\frac{1-\frac{1}{S^{\left|D^{Q}\right|}}}{N^{Q}+1}\leq% \frac{1}{N^{Q}+1}\end{array}

$N^{Q}$ can be replaced by it’s smallest possible value $N^{min}$ to maximise $\Delta_{Avg(\widehat{(}R))}$ : We can simplify the the second $\Delta_{Avg(\widehat{R})}$ as follows:

(17)

\begin{array}[]{ll}\left|\frac{\sum^{R\in\widehat{R}}R}{N^{Q}}-\frac{\frac{1}{% S^{\left|D^{Q}\right|}}+\sum^{R\in\widehat{R}}R}{N^{Q}+1}\right|=\left|\frac{% Avg(\widehat{R})}{N^{Q}+1}-\frac{\frac{1}{S^{\left|D^{Q}\right|}}}{N^{Q}+1}% \right|\\ \text{ which is :}\leq\frac{1-\frac{1}{S^{\left|D^{Q}\right|}}}{N^{Q}+1}\leq% \frac{1}{N^{Q}+1}\end{array}

$N^{Q}$ can be replaced by it’s smallest possible value $N^{min}$ to maximise $\Delta_{Avg(\widehat{(}R))}$ :

(18)

\Delta_{Avg(\widehat{R})}\leq\left\{\begin{array}[]{cc}&\frac{\Delta_{R}}{N^{% min}}\\ &\frac{1}{N^{min}+1}\end{array}\right.\implies\Delta_{Avg(\widehat{R})}=max(% \frac{\Delta_{R}}{N^{min}},\frac{1}{N^{min}+1})

Since $\Delta_{R}$ and $N^{min}$ are not sensitives information, they can be used to express the $\Delta_{Avg(\widehat{(}R)}$ .

Appendix B Sensitivity estimator $\mathbb{E}$

For the estimator used to approximate the result of $Q$ , we first will give the bound for its global sensitivity, then we show how we bound his local sensitivity.

B.1. Global sensitivity of the estimator $\mathbb{E}$

Given a query $Q$ , two neighbouring databases $T$ and $T^{\prime}$ containing cluster $C\in C^{Q}$ and $C^{\prime}\in C^{\prime Q}$ where $C^{\prime}$ has an additional row that covers $Q$ . Thus both the sampling probability $p$ and $Q(C)$ are effected by this additional row, and we have $p^{\prime}$ and $Q(C^{\prime})$ , this implies :

(19)

\begin{array}[]{ll}\Delta_{\mathbb{E}}=\left|\frac{Q(C)}{p}-\frac{Q(C^{\prime}% )}{p^{\prime}}\right|\text{ where }Q(C^{\prime})=Q(C)+1\\ \Delta_{\mathbb{E}}=\left|\frac{Q(C)}{p}-\frac{Q(C)+1}{p^{\prime}}\right|=% \left|\frac{Q(C)}{p}-\frac{Q(C)+1}{p^{\prime}}\right|=\frac{1}{p\times p^{% \prime}}\times\left|Q(C)\times(p^{\prime}-p)-p\right|\end{array}

Since the $p\times p^{\prime}$ is in the denominator, we can minimise it to obtain the $\Delta_{\mathbb{E}}$ . Let’s consider the case where $C$ contains only one row $R_{C}=\frac{1}{S^{\left|D^{Q}\right|}}$ that covers $Q$ and the remain cluster in $C^{Q}$ fully covers $Q$ so their $R=1$ which implies :

(20)

p=\frac{1/S^{\left|D^{Q}\right|}}{N^{Q}-1+1/S^{\left|D^{Q}\right|}}=\frac{1}{(% N^{Q}-1)\times S^{\left|D^{Q}\right|}+1}

In $T^{\prime}$ , $C^{\prime}$ has an additional row so $p^{\prime}$ is:

(21)

p^{\prime}=\frac{2^{\left|D^{Q}\right|}/S^{\left|D^{Q}\right|}}{N^{Q}-1+2^{% \left|D^{Q}\right|}/S^{\left|D^{Q}\right|}}=\frac{2^{\left|D^{Q}\right|}}{(N^{% Q}-1)\times S^{\left|D^{Q}\right|}+2^{\left|D^{Q}\right|}}

From equations (20) and (21) :

(22)

\left\{\begin{array}[]{ll}&\frac{1}{p\times p^{\prime}}=\frac{((N^{Q}-1)\times S% ^{\left|D^{Q}\right|}+1)\times((N^{Q}-1)\times S^{\left|D^{Q}\right|}+2^{\left% |D^{Q}\right|})}{2^{\left|D^{Q}\right|}}\\ &p^{\prime}-p=\frac{(2^{\left|D^{Q}\right|}-1)\times(N^{Q}-1)\times S^{\left|D% ^{Q}\right|}}{((N^{Q}-1)\times S^{\left|D^{Q}\right|}+1)\times((N^{Q}-1)\times S% ^{\left|D^{Q}\right|}+2^{\left|D^{Q}\right|})}\end{array}\right.

Which implies:

(23)

\begin{array}[]{ll}\Delta_{\mathbb{E}}=\frac{1}{2^{\left|D^{Q}\right|}}\times% \left|2^{\left|D^{Q}\right|-1}\times(N^{Q}-1)\times S^{\left|D^{Q}\right|}-2^{% \left|D^{Q}\right|}\right|\\ \Delta_{\mathbb{E}}=\frac{(N^{Q}-1)\times S^{\left|D^{Q}\right|}}{2}-1\end{array}

This results shows that the sensitivity of this statistical estimator is very large and unbounded, if its result should be protected another alternative is mandatory.

B.2. Smooth sensitivity of the estimator $\mathbb{E}$

To compute the smooth upper bound of the $LS_{\mathbb{E}}$ , we considered four possible scenarios of neighbouring $T$ and $T^{\prime}$ :

(1)

$LS_{\mathbb{E}}^{1}=\left|\frac{Q(C)}{p}-\frac{Q(C)}{p^{\prime}}\right|$ and $p^{\prime}=\frac{R}{\Delta_{R}+\sum^{R\in\widehat{R}}R}$ : another cluster gained a row
(2)

$LS_{\mathbb{E}}^{2}=\left|\frac{Q(C)}{p}-\frac{Q(C)}{p^{\prime}}\right|$ and $p^{\prime}=\frac{R}{1/S^{\left|D^{Q}\right|}+\sum^{R\in\widehat{R}}R}$ : new cluster added to $C^{Q}$
(3)

$LS_{\mathbb{E}}^{3}=\left|\frac{Q(C)}{p}-\frac{Q(C^{\prime})}{p^{\prime}}\right|$ and $p^{\prime}=\frac{R+\Delta_{R}}{1/S^{\left|D^{Q}\right|}+\sum^{R\in\widehat{R}}% R},Q(C^{\prime})=Q(C)+1$ : the cluster gained a row.
(4)

$LS_{\mathbb{E}}^{4}=\left|\frac{Q(C)}{p}-\frac{Q(C^{\prime})}{p}\right|$ and $Q(C^{\prime})=Q(C)+1$ : the cluster gained an individual $\pm$ 1 in a measure and not a new row.

Our goal is to find the biggest one of these distances. we can quickly notice that $LS_{\mathbb{E}}^{1}>LS_{\mathbb{E}}^{2}$ since $\Delta_{R}>\frac{1}{S^{\left|D^{Q}\right|}}$ . And we have $LS_{\mathbb{E}}^{4}>LS_{\mathbb{E}}^{3}$ because this constraint is always true:

(24)

\begin{array}[]{ll}\frac{Q(C^{\prime})/p}{Q(C^{\prime})/p^{\prime}}=\frac{p^{% \prime}}{p}=\frac{R+\Delta_{R}}{R}>1\\ \implies LS_{\mathbb{E}}^{4}>LS_{\mathbb{E}}^{3}\end{array}

Between $LS_{\mathbb{E}}^{1}$ and $LS_{\mathbb{E}}^{4}$ , we need to find which is the bigger distance and under what conditions:

(25)

\begin{array}[]{ll}\frac{Q(C)/p^{\prime}}{Q(C^{\prime})/p}=\frac{Q(C)\times(% \Delta_{R}+\sum^{R\in\widehat{R}}R)}{R}\times\frac{R}{(Q(C)+1)\times\sum^{R\in% \widehat{R}}R}\\ \frac{Q(C)/p^{\prime}}{Q(C^{\prime})/p}=\frac{Q(C)}{Q(C)+1}\times\frac{\Delta_% {R}+\sum^{R\in\widehat{R}}R)}{\sum^{R\in\widehat{R}}R)}\\ \frac{Q(C)/p^{\prime}}{Q(C^{\prime})/p}>1\implies Q(C)>\frac{\sum^{R\in% \widehat{R}}R}{\Delta_{R}}\end{array}

In conclusion, only $LS_{\mathbb{E}}^{1}$ and $LS_{\mathbb{E}}^{4}$ need to be used in order to compute the smooth sensitivity, and for each cluster there is only one that dominated the other based if $Q(C)>\frac{\sum^{R\in\widehat{R}}R}{\Delta_{R}}$ . Both distances can be simplified as follows :

(26)

LS_{\mathbb{E}}^{=}\left\{\begin{array}[]{ll}&\frac{Q(C)\times\Delta_{R}}{R}% \text{ from }LS_{\mathbb{E}}^{1}\\ &\frac{1}{p}\text{ from }LS_{\mathbb{E}}^{4}\end{array}\right.

B.3. Bound the k for the smooth sensitivity

Since we have our distances $LS_{\mathbb{E}}^{k}$ are ascending function and $e^{-k\beta}$ is an exponential decay function, and since we are looking for the $max_{k=0,1,...}e^{-k\beta}\times LS_{\mathbb{E}}^{k}$ . To find the stopping point of $k$ , we need to find where the this product starts decaying. In other words, we find the $k$ such that :

(27)

e^{-k\beta}\times LS_{\mathbb{E}}^{k}<e^{-(k-1)\beta}\times LS_{\mathbb{E}}^{k% -1}\implies\frac{LS_{\mathbb{E}}^{k-1}}{LS_{\mathbb{E}}^{k}}>e^{-\beta}

For $LS_{\mathbb{E}}^{k}$ based on scenario 1:

(28)

\frac{LS_{\mathbb{E}}^{k-1}}{LS_{\mathbb{E}}^{k}}=\frac{(k-1)\times Q(C)\times% \Delta_{R}}{R}\times\frac{R}{k\times Q(C)\times\Delta_{R}}=\frac{k-1}{k}

For $LS_{\mathbb{E}}^{k}$ based on scenario 4:

(29)

\frac{LS_{\mathbb{E}}^{k-1}}{LS_{\mathbb{E}}^{k}}=\frac{k-1}{p}\times\frac{p}{% k}=\frac{k-1}{k}

So for both our distances, the smooth upper bound is reached when:

(30)

\frac{k-1}{k}>e^{-\beta}\implies k>\frac{1}{1-e^{-\beta}}

Where $\beta=\frac{\epsilon}{2\times\ln(2/\delta)}$ . Based on the $(\epsilon,\delta)$ budget we set for the estimator, we will obtain our exact upper bound for the $k$ . this shows that our process terminates and don’t run indefinitely.

Private Approximate Query over Horizontal Data Federation

Abstract.

1. Introduction

2. Related Works

3. Preliminaries

Definition 3.0 (Neighbouring Tables(Dwork et al., 2014)).

Definition 3.0 ((ϵ,δ)italic-ϵ𝛿(\epsilon,\delta)( italic_ϵ , italic_δ )-Differential Privacy(Dwork et al., 2014)).

Definition 3.0 (Global Sensitivity(Dwork et al., 2014)).

Definition 3.0 (Laplace Mechanism (Dwork et al., 2014)).

Definition 3.0 (Exponential Mechanism (Dwork et al., 2014)).

Definition 3.0 (Local Sensitivity(Nissim et al., 2007)).

Definition 3.0 (Local Sensitivity at Distance k𝑘kitalic_k (Nissim et al., 2007)).

Definition 3.0 (Smooth Sensitivity Framework (Nissim et al., 2007)).

Theorem 3.9 (Sequential Composition (Dwork et al., 2014)).

Theorem 3.10 (Parallel Composition (Dwork et al., 2014)).

Theorem 3.11 (Post-Processing (Dwork et al., 2014)).

4. Problem Statement

5. Our solution

5.1. Overview

5.2. Query Approximation and sampling

5.3. Federated protocol

5.3.1. Allocation phase

Theorem 5.1 (Sensitivity of estimator ΔAvg(R^\Delta_{\text{Avg}(\widehat{R}}roman_Δ start_POSTSUBSCRIPT Avg ( over^ start_ARG italic_R end_ARG end_POSTSUBSCRIPT).

5.3.2. Sampling phase

Theorem 5.2 (Sensitivity of sampling probability).

5.3.3. Approximation phase

Theorem 5.3 (Sensitivity of estimator 𝔼𝔼\mathbb{E}blackboard_E).

Theorem 5.4 (Dominant distance LS).

5.4. Privacy accounting

6. Evaluation

6.1. Setup

6.2. Dimension-based analysis

6.3. Sampling rate-based analysis

6.4. Privacy budget-based analysis

6.5. SMC vs DP in terms of sharing results

6.6. Resilience to Learning-Based Attacks

7. Discussion

8. Conclusion

References

Appendix A Sensitivity summarised information

A.1. Sensitivity R𝑅Ritalic_R

A.2. Sensitivity A⁢v⁢g⁢(R^)𝐴𝑣𝑔^𝑅Avg(\widehat{R})italic_A italic_v italic_g ( over^ start_ARG italic_R end_ARG )

Appendix B Sensitivity estimator 𝔼𝔼\mathbb{E}blackboard_E

B.1. Global sensitivity of the estimator 𝔼𝔼\mathbb{E}blackboard_E

B.2. Smooth sensitivity of the estimator 𝔼𝔼\mathbb{E}blackboard_E

B.3. Bound the k for the smooth sensitivity

Definition 3.0 ( $(\epsilon,\delta)$ -Differential Privacy(Dwork et al., 2014)).

Definition 3.0 (Local Sensitivity at Distance $k$ (Nissim et al., 2007)).

Theorem 5.1 (Sensitivity of estimator $\Delta_{\text{Avg}(\widehat{R}}$ ).

Theorem 5.3 (Sensitivity of estimator $\mathbb{E}$ ).

A.1. Sensitivity $R$

A.2. Sensitivity $Avg(\widehat{R})$

Appendix B Sensitivity estimator $\mathbb{E}$

B.1. Global sensitivity of the estimator $\mathbb{E}$

B.2. Smooth sensitivity of the estimator $\mathbb{E}$