Contextual Linear Types for Differential Privacy

Jazz follows Duet in being structured as two mutually embedded sublanguages, one for sensitivity and one for privacy. In a nutshell, this is because supporting scaling of both sensitivity and privacy in a uniform and tight fashion is sound only for ϵ-differential privacy, and not for (ϵ, δ)-differential privacy, which has nonlinear group privacy, as discussed in Section 2.

Let us elaborate on this. Recall that the framework of differential privacy builds on randomization to achieve privacy, and the randomization is typically calibrated according to the sensitivity of the function whose output one wants to protect. Therefore, in a language to describe differentially private computations, we can mostly distinguish two classes of functions: On the one hand, randomized (effectful) functions that are annotated with privacy information, and on the other hand, pure functions that are annotated with sensitivity information. However, when composing functions, the way their information is combined highly depends on the class of the composed functions.

For example, when composing two functions from the sensitivity fragment, their information is naturally combined via scaling. Consider, for instance, a 3-sensitive function f; it is not hard to see that the expressionis (2 · 3)-sensitive in x. The same scaling pattern remains valid when composing a function from the sensitivity fragment with a function from the privacy fragment, provided the latter satisfies pure differential privacy: If g is an, e.g., ϵ-differentially private function, then the computationis (2 · ϵ)-differentially private in x. This uniform scaling behavior, which besides being sound is also tight, lies at the heart of Fuzz/DFuzz design. In fact, in Fuzz/DFuzz, both classes of functions live within the same space, following the same typing rules. To enable this uniform treatment, the languages rely on the two fundamental ingredients: (i) the presence of a monad/modality to encode randomization and (ii) the association of a metric space to each type, which in the case of monadic types, is tailored to encode differential privacy. Said otherwise, in Fuzz/DFuzz, differential privacy is encoded as a sensitivity claim about functions (with a monadic return type).

Unfortunately, this linear scaling—pervasive in Fuzz/DFuzz—is no longer sound when composing a function from the sensitivity fragment with a function from the privacy fragment that satisfies approximate—rather than pure—differential privacy. Returning to the previous example, if g is instead (ϵ, δ)-differentially private, then g(x + x) does not satisfy (2ϵ, 2δ)-differential privacy in x, but only (2ϵ, 2δe^ϵ)-differential privacy as dictated by the group privacy property for approximate differential privacy (see Section 2 and Dwork and Roth [32, Section 2.3]). In effect, this is why approximate differential privacy lies out of the scope of Fuzz/DFuzz “uniform” design—although recently, de Amorim et al. [29] have shown that using different metrics allows approximate differential privacy functions to be linear, extending a terminating subset of Fuzz to support (ϵ, δ)-differentially privacy via a path metric construction.

In view of this, Duet introduced a two-language design, separating a sensitivity sublanguage in which scaling remains implicit (internalized by the typing rules) and pervasive (modeling function composition) and a privacy sublanguage in which scaling is explicit and restricted (ad hoc typing rules are needed, e.g., to model some privacy combinators). Jazz builds upon this approach and significantly improves both sublanguages, thanks to latent contextual effects, as we illustrate next.

Coloring convention. As noted in the introduction, this article uses colorblind-friendly colors in notation to convey information and is best consumed using an electronic device or color printer. Jazz consists of two mutually embedded sublanguages, and each language is given its own color. Furthermore, these two languages share the same language of types, so we use a third color for the shared fragment. Consequently, we use three color schemes throughout the article: (1) blue for general math notation and the type system shared between languages; (2) green for the sensitivity language; and (3) red for the privacy language. We have carefully chosen the schemes to be distinguishable (as much as possible) for persons with various forms of color blindness.³

In the sensitivity sublanguage of Jazz, the identity function is written \(\color{green}{ \lambda ^{\rm s }}\color{blue}{x.x}\). We write sensitivity lambdas as \(\color{green}{ \lambda ^{\rm s }}\) to more easily distinguish them from privacy lambdas, written \(\color{red}{ \lambda ^{\rm p }}\) (described later). The identity function is 1-sensitive in its argument x: If x changes by \(\color{green}{d}\), then the function’s output also changes by \(\color{green}{d}\). Similarly to the identity function, the doubling function \(\color{green}{ \lambda ^{\rm s }} \color{blue}{x.x+x}\) is 2-sensitive in x: If x changes by \(\color{green}{d}\), then the function’s output changes by \(\color{blue}{2}\color{green}{d}\).

Fuzz extends the notion of sensitivity to multi-argument functions by assigning a sensitivity to each argument. For example, the curried function \(\color{green}{\lambda^{\mathrm{s}}} \color{blue}{x.}\color{green}{\lambda^{\mathrm{s}} \color{blue}{y.x+x+y}}\) is 2-sensitive in x and 1-sensitive in y. If x changes by \(\color{green}{d_{x}}\) and y changes by \(\color{green}{d_{y}}\), then the function’s output changes by \(\color{blue}{ 2}\color {green}{d_{x}} +\color{blue}{1}\color {green}{d_{y}}\).

In general, the sensitivity of a function can be written as a linear combination of the changes in its inputs. In Jazz, we express function sensitivities as linear formulas over the function’s input variables, using the variable name itself as a placeholder for the change in that input. Jazz’s type system gives the following types for the three examples we have seen so far:The linear formulas written above function arrows represent the sensitivity effect of the corresponding function. The general form of sensitivity function types is \(\color{blue}{x:\tau_{1}}\ \color{green}{\xrightarrow\Sigma}\ \color{blue}{\tau_{2}}\), where Σ is the sensitivity effect of the function, expressed as a linear formula. Note that x is in scope for Σ and τ₂. Importantly, occurrences of x in Σ and τ₂ represent the sensitivity of the variable x, rather than its value, so Jazz supports sensitivity-dependent types. Also, we usually drop “null” effects over function arrows such as above and instead just write \(\color{green}{\rightarrow}\).

As usual, Jazz accommodates higher-order functions by scaling sensitivities. For example, applying a 2-sensitive function twice yields a 4-sensitive function:In addition to function types, other type connectives in Jazz such as sums and products also carry sensitivity effects, such as \(\color{blue}{\tau _{1}}\ {}^{\color {green}{\Sigma_{1}}}\color{blue}{\oplus} ^{{{{\color {green}{\Sigma_{2}}}}}} {\color {blue}{\tau _{2}}}\) for sums, \(\color{blue}{\tau _{1}}\ {}^{\color {green}{\Sigma_{1}}}\color{blue}{\otimes} ^{{{{\color {green}{\Sigma_{2}}}}}} {\color {blue}{\tau _{2}}}\) for multiplicative products, and \(\color{blue}{\tau _{1}}\ {}^{\color {green}{\Sigma_{1}}}\color{blue}{\&} ^{{{{\color {green}{\Sigma_{2}}}}}}\ {\color {blue}{\tau _{2}}}\) for additive products. These connectives are extensions of the linear type connectives \(\color{blue}{\tau _{1} \oplus \tau _{2}}\), \(\color{blue}{\tau _{1} \otimes \tau _{2}}\) and \(\color{blue}{\tau _{1} \& \tau _{2}}\) from Fuzz, augmented with latent sensitivity effects.

We say that sensitivity effects in Jazz are latent, because they only contribute to the sensitivity of an expression when the type connective is actually eliminated. For instance, in the third example above—a curried function—the sensitivity effect on the first argument is delayed until the second argument is received. If a second argument is never received, then the sensitivity effect on the first argument can be ignored. Likewise, the annotations Σ₁ and Σ₂ in the type \(\color{blue}{\tau _{1}}\ {}^{\color {green}{\Sigma_{1}}}\color{blue}{\oplus} ^{{{{\color {green}{\Sigma_{2}}}}}}\ {\color {blue}{\tau _{2}}}\) encode the latent sensitivity cost for each component of the connective: for τ₁ (the left) and τ₂ (the right), respectively. In contrast to Fuzz, creating a pair in Jazz can have no immediate sensitivity cost: Only projecting out of a pair has a cost in sensitivity, depending on which component is projected. Additionally, we say that sensitivity effects are contextual because Σ can refer to variables in scope.

To encode differential privacy, Jazz makes use of privacy functions (notated \(\color{red}{\twoheadrightarrow}\)) rather than of sensitivity functions (notated \(\color{green}{\rightarrow }\)) as in the previous examples. As such, privacy functions are annotated with privacy—rather than sensitivity—effects. The type of a function from τ₁ to τ₂ that is (ϵ,δ)-differentially private in its argument is as follows:Semantically, this type describes functions f, where if \(\color{blue}{ {\mathfrak {D}} _{\tau _{1}}(x,x^{\prime }) \le {{{\color {green}d,}}}}\) then f(x) and f(x′) yield distributions that are “(ϵ,δ)-close” according to the definition of (ϵ, δ)-differential privacy.

The annotation \(\color{green}{d}\) is necessary to support (and unique to) advanced variants of differential privacy like (ϵ, δ)-differential privacy. In the pure ϵ-differential privacy framework, it is common to first establish the property for \(\color{green}{d} \color{blue}{= 1}\), that is, if \(\color{blue}{ {\mathfrak {D}} _{\tau _{1}}(x,x^{\prime }) \le 1}\) then f(x) and f(x′) are ϵ-close. Once established, this property then implies that if \(\color{blue}{ {\mathfrak {D}} _{\tau _{1}}(x,x^{\prime }) \le {{{\color {green}d,}}}}\) then f(x) and f(x′) are \(\color {green}{d}\color{blue}{\epsilon}\)-close, for any \({ {{{\color {green}d}}}}\). However, this linear scaling does not carry over to advanced variants like (ϵ, δ)-differential privacy. As a consequence, \({ {{{\color {green}d}}}}\) must be specified directly as a parameter and cannot be recovered by scaling the property instantiated to \({ {{{\color {green}d}}} = 1}\). We refer to this distance—\({ {{{\color {green}d}}}}\)—as the relational distance, since it pertains to the (two-run) relational property of differential privacy, and specifically, the distance between inputs x and x′ for each of the two executions f(x) and f(x′). We also use this terminology in the context of sensitivity, e.g., an s-sensitive function is one that upon inputs within relational distance \({ {{{\color {green}d}}}}\) returns outputs within relational distance \(\color{blue}{ s{{{\color {green}d}}}}\).

As explained in Section 2, differential privacy is usually achieved by the use of mechanisms such as the Laplace (for ϵ-differential privacy) or the Gaussian mechanism (for (ϵ, δ)-differential privacy). In Jazz, the primitive function implementing the Laplace mechanism has the following type:There are three logical parameters to the laplace function: d is the relational distance (explained above) used in the statement of privacy satisfied by the function, ϵ is the privacy level we want to enforce, and x is the value we (want to protect and) are adding noise to. When executing laplace, the amount of noise added is \(\color{blue}{ {\text{Lap}}(\frac{d}{\epsilon })}\), which depends on both d and ϵ, so they must be runtime values. Also, when typechecking laplace, the amount of privacy obtained depends on ϵ, and the distance d must also be tracked to enforce that the computation feeding laplace with its argument x produces values within relational distance no larger than d. Because the values of d and ϵ are required for both runtime execution and typechecking, we require a form of dependent types.

To support dependent types, we use a singletons approach—a technique initially developed for dependently typed programming in Haskell [33, 39] and which we borrow directly from DFuzz in the context of supporting parameterized differentially private functions [34]. In this approach, each dependent argument has two representations—one for the type and term level, respectively. In the type of laplace, \(\color{blue}{ \hat{d}}\) is the type-level representation of term-level variable d, and likewise for \(\color{blue}{ \hat{\epsilon }}\) and ϵ. (We further discuss singletons and their implementation in Section 8.)

The final argument \(\color{blue}{ x \mathrel {:} {\mathbb {R}}\mathord {\cdotp }\hat{d}}\) will have Laplace noise added to it and then returned as the result of the laplace function. The annotation “\(\color{blue}{ \mathord {\cdotp }\hat{d}}\)” in the type of x places a precondition on the computation used to supply the value to protect: Its output must have relational distance no larger than \(\color{blue}{ \hat{{{\color {green}d}}}}\). After all, the noise added is only guaranteed to give ϵ-differential privacy for values that result from computations with relational distance \(\color{blue}{ \hat{{{\color {green}d}}}}\).

The final privacy effect for the function is \(\color{blue}{ \infty (d + \epsilon) + (\hat{\epsilon },0)x}\), indicating that no privacy promises are made for the values d and ϵ and that privacy is promised for input x with cost \(\color{blue}{ (\hat{\epsilon },0)}\); we write ∞(d + ϵ) as shorthand for ∞d + ∞ϵ.

Like Fuzz, DFuzz, and Duet, Jazz extends the notion of differential privacy from single-argument to multi-argument functions, assigning each argument a privacy cost (e.g., the privacy effect \(\color{blue}{ \infty (d + \epsilon) + (\hat{\epsilon },0)x}\) for the laplace function describes privacy costs for d, ϵ, and x). This approach is formalized in Section 7.3. By convention, most differentially private programs expect a single input to contain the sensitive data, and the privacy cost assigned to this argument is most important in ensuring privacy. The costs associated with the other arguments are typically infinite, indicating that the program does not preserve privacy for these inputs.

We can give a similar type to the gauss function, which provides (ϵ, δ)-differential privacy by adding Gaussian noise drawn from \(\color{blue}{ {\mathcal {N}}(\tfrac{2{{{\color {green}d}}} ^{2}\ln (1.25/\delta)}{\epsilon ^{2}})}\):In Jazz, privacy primitives are used in the privacy sublanguage. For example, the following privacy-sublanguage expression partially applies the Gaussian mechanism to values for \({ {{{\color {green}d}}}}\), ϵ and δ, resulting in a privacy function that satisfies (1.5, 10^{− 5})-differential privacy for any input at relational distance 4:Note that we omit the instantiation of forall-quantified type variables \(\color{blue}{ \hat{{{\color {green}d}}}}\), \(\color{blue}{ \hat{\epsilon }}\), and \(\color{blue}{ \hat{\delta }}\) to type-level constants 4, 1.5, and 10^{− 5}, as they can be inferred from the value-level arguments \({ {{{\color {green}d}}}}\), ϵ, and δ.

The privacy sublanguage also contains monadic bind (notated ←) and return constructs for composing differentially private computations. Privacy functions \({\color {red}{\lambda ^{{\mathrm{p}}}}}\color{blue}{(x}\mathord {\cdot }{\color {green}d}). {{\color {red}e}}\) are created in the sensitivity sublanguage (because function creation is “pure”), although the function body e lives in the privacy sublanguage. The annotation \({ {{{\color {green}d}}}}\) is the relational distance explained previously for privacy function types. For example, the following function computes two differentially private results and adds them together:The bind operator encodes the sequential composition property of differential privacy (Section 2), adding up the ϵ and δ values of subcomputations. The return operator encodes the post-processing property of differential privacy. The relational distance parameter of 1 is in general inferrable during typechecking; we include it as a visible term-level parameter for presentation purposes.

Beyond Duet. The privacy sublanguage of Jazz briefly introduced here lifts a number of important limitations of the privacy sublanguage of Duet. We sketch two of these here and postpone further comparison to later sections.

First, to avoid scaling in the privacy sublanguage, Duet requires the arguments to privacy functions to have a maximum sensitivity or relational distance of 1. This limitation makes it impossible to give general types to the gauss and laplace functions as we have just shown in Jazz. As a result, Duet includes a dedicated type rule for each basic differential privacy mechanism, where each rule is parametric in the sensitivity or relational distance of the argument. Jazz’s addition of relational distance annotation “\(\color{blue}{ \mathord {\cdotp }{{\color {blue}{{{\color {green}d}}}}}}\)” in the types of function arguments eliminates the need for special type rules, and mechanisms can instead be encoded as primitives with an axiomatized type. The primary benefit of this is that the metatheory need not be extended each time a new mechanism is considered.

Second, while pervasive scaling is generally undesirable for privacy costs, some constructs such as advanced composition rely on the ability to scale privacy costs in controlled ways that are supported by theorems specific to that privacy model. Because Duet’s privacy language disallows scaling entirely, these constructs are impossible to encode as functions and must also be given special typing rules. The latent privacy effects in Jazz allow constructs like advanced composition to be given regular function types. Overall, the Jazz design makes differential privacy by typing in the presence of higher-order programming possible for advanced differential privacy variants. The following sections dive into these benefits by focusing first on the sensitivity sublanguage (Section 4) and then the privacy sublanguage (Section 6). Sections 5 and 7 develop the metatheory of each respective sublanguage.

Existing approaches based on linear types [34, 46, 49] provide elementary datatype abstractions to programmers such as pairs (products) and tagged unions (sums). However, some of the sensitivity analysis they implement for these datatypes can lead to overly imprecise—or even unsound—approximations in some circumstances.

We now briefly overview these datatype abstractions; a summary is provided in Table 1.

Fuzz [49] is the first work to leverage linear (or affine) types for reasoning about program sensitivity. Since its introduction, other systems based on linear types were developed to address different limitations of Fuzz. These primarily comprise DFuzz [34], which allows value-dependent sensitivities and privacy costs, and Duet [46], which allows advanced variants of differential privacy.

Being based on the same underlying sensitivity analysis, all these systems suffer from common limitations related to the sensitivity tracking for products and sums. Through a series of minimal—yet instructive—examples, we now discuss the limitations we have identified.

Jazz adopts a novel approach to sensitivity tracking for product and sum types, which can address the previous limitations without the need to rely on scaling of types. The key insight is to delay the tracking of sensitivities whenever possible and to split it into two separate analyses: one for each side of the product or sum. Technically, the main idea is to encode latent sensitivity effects at the type-connective level. For instance, for multiplicative pairs Jazz has type \({ {{\color {blue}{\tau _{1}}}} \mathrel {^{{{{\color {green}{\Sigma _{1}}}}}}\color{blue}{\otimes} ^{{{{\color {green}{\Sigma _{2}}}}}}} {{\color {blue}{\tau _{2}}}}}\), where \(\color {green}{\Sigma _{1}}\) and \(\color {green}{\Sigma _{2}}\) denote the latent sensitivity effects of each of the pair components. This is in contrast to the Fuzz type \({ {{\color {blue}{\tau _{1}}}}\ {{\color {blue}{\otimes\ \tau _{2}}}}}\), which pays for all of its sensitivity effects upfront when the pair is created.

The Sax type system is technically a type-and-effect system [36]. It supports real numbers, functions, sums, and products. As Sax only deals with ambient effects, all metavariables and keywords are typeset in green. Syntax. Figure 1 presents the syntax of Sax.

Expressions \({ {{{\color {green}e}}}}\) are mostly standard and include: real number \({ {{{\color {green}r}}}}\), addition \({ {{{\color {green}e}}} \color{blue}+ {{{\color {green}e}}}}\), multiplication \({ {{{\color {green}e}}} \color{blue}{*} {{{\color {green}e}}}}\), comparison \(\color{blue}{ {{{\color {green}e}}} \le {{{\color {green}e}}}}\), variable x, sensitivity lambda \(\color{blue}{ {{\lambda ^{\!^{{}_{\mathrm{s}}}}}} (x\mathrel {:}\tau).{{{\color {green}e}}}}\), application \(\color{blue}{ {{{\color {green}e}}}\ {{{\color {green}e}}}}\), unit value \(\color {green}{\texttt {t t}}\), sum constructors \(\color {green}{\texttt {inl}}^{\color{blue}{\tau _{2}}}\ \color {green}{e}\) and \(\color {green}{\texttt {inr}}^{\color{blue}{\tau _{2}}}\ \color {green}{e}\), and the sum destructor .

Sax also supports two linear products types: additive and multiplicative. With additive products, the sensitivity of a pair may be approximated as the max of the sensitivities of each side; this sensitivity is paid for every projection. With multiplicative products, the sensitivity of a pair may be approximated as the sum of the sensitivities of each side; this sensitivity is paid for every tuple pattern match, scaled by the sensitivities of pattern variables in the body. We write additive product constructions \(\color{blue}{ {{\color {blue}{{({{{\color {green}e}}},{{{\color {green}e}}})}}}}}\) and destructions \(\color {green}{\texttt {fst}}\ \color {green}{e}\) and \(\color {green}{\texttt {snd}}\ \color {green}{e}\), and multiplicative product constructions \(\color{blue}{ {{\color {blue} {{\langle {{{\color {green}e}}},{{{\color {green}e}}}\rangle }}}}}\) and destructions \(\color {green}{\texttt {let}}\ \color{blue}{x,x=}\color {green}{e}\ \color {green}{\texttt {in}}\ \color {green}{e}\).

Finally, an expression \({ {{{\color {green}e}}}}\) can be an ascription \(\color{blue}{ {{{\color {green}e}}} \mathrel {:: } {{{\color {green}\tau }}}}\), or a derived expression such as a Boolean b, a conditional \(\color {green}{\texttt {if}\ e\ \texttt {then}\ e\ \texttt {else}\ e}\), or a let expression \(\color{green}{\texttt {let}}\ \color{blue}{x=}\color{green}{e\ \texttt{in}\ e}\). Booleans are encoded as \(\color{blue}{ \text{true} \triangleq} \color {green}{\mathtt {inl\ tt}}\), \(\color{blue}{\text{false} \triangleq} \color {green}{\mathtt {inr\ tt}}\), and \(\color{blue}{ {\mathbb {B}}}\) as \(\color {blue}{\texttt {unit}^{\varnothing }\oplus ^{\varnothing }\texttt {unit}}\), conditionals as , and let expressions as .

A sensitivity \({ {{{\color {green}s}}}}\) is either a non-negative real number or the symbol ∞, which represents an unbounded sensitivity; we notate this set \(\color{blue}{ {{{\color {blue} {{{\mathbb {R}}}}}}}^{\infty }_{\ge 0} \triangleq {{{\color {blue} {{{\mathbb {R}}}}}}}_{\ge 0} \uplus \lbrace \infty \rbrace}\). A sensitivity environment \({ {{{\color {green}\Sigma }}}}\) is a mapping from variables to their sensitivities. For convenience, we write sensitivity environments as first-order polynomials, e.g., \(\color{blue}{ {{{\color {green}\Sigma }}} = 1x + 2y}\) corresponds to an environment \({ {{{\color {green}\Sigma }}}}\) such that \(\color{blue}{ {{{\color {green}\Sigma }}}(x) = 1}\) and \(\color{blue}{ {{{\color {green}\Sigma }}}(y) = 2}\). A type τ is either the real number type \({ {{{\color {blue} {{{\mathbb {R}}}}}}}}\), the Boolean type \({ {{{\color {blue} {{{\mathbb {B}}}}}}}}\), the unit type \(\color {blue}{\texttt {unit}}\), a function type , a sum type \(\color{blue}{ \tau \mathrel {^{{{{\color {green}\Sigma }}}}\oplus ^{{{{\color {green}\Sigma }}}}} \tau}\), an additive product type \(\color{blue}{\tau}\ {}^{\color {green}{\Sigma}}\color{blue}{\&} ^{{{{\color {green}{\Sigma}}}}}\ {\color {blue}{\tau}}\), or a multiplicative product type \(\color{blue}{ \tau \mathrel {^{{{{\color {green}\Sigma }}}}\otimes ^{{{{\color {green}\Sigma }}}}} \tau}\). The sensitivity environment annotation \({ {{{\color {green}\Sigma }}}}\) is called the latent contextual sensitivity effect (also called latent effect when clear from the context) and represents a delayed effect that emerges when a term of said type is eliminated. The latent effect \({ {{{\color {green}\Sigma }}}}\) of a function of type corresponds to the effects of applying the function, i.e., a static approximation of the sensitivity of each variable used in its body. The sensitivity environment \(\color {green}{\Sigma _{1}}\) (respectively, \(\color {green}{\Sigma _{2}}\)) in \(\color{blue}{ \tau _{1} \mathrel {^{{{{\color {green}\Sigma _{1}}}}}\oplus ^{{{{\color {green}\Sigma _{2}}}}}} \tau _{2}}\) corresponds to the latent effect of the injected value using inl (respectively, inr). And similarly, \(\color {green}{\Sigma _{1}}\) and \(\color {green}{\Sigma _{2}}\) in \(\color{blue}{ \tau _{1} \mathrel {^{{{{\color {green}\Sigma _{1}}}}}\&^{{{{\color {green}\Sigma _{2}}}}}} \tau _{2}}\) or \(\color{blue}{ \tau _{1} \mathrel {^{{{{\color {green}\Sigma _{1}}}}}\otimes ^{{{{\color {green}\Sigma _{2}}}}}} \tau _{2}}\) correspond to the latent effect of accessing the first and second components of the pair, respectively. Finally, a type environment Γ is, as usual, a mapping from variables to types.

Type system. The Sax type system is presented in Figure 2. The judgment \({ {{{{\color {blue}\Gamma }}}\ {{{\color {green}\vdash }}}\ {{{\color {green}e}}}\ \color{blue}{:}\ {{{\color {blue}\tau }}} \mathrel {;} {{{\color {green}\Sigma }}}}}\) says that the term \({ {{{\color {green}e}}}}\) has type τ and ambient sensitivity effect \({ {{{\color {green}\Sigma }}}}\) (or ambient effect when clear from the context) under type environment Γ. The ambient effect \({ {{{\color {green}\Sigma }}}}\) represents an upper bound (conservative approximation) of the real sensitivity of \({ {{{\color {green}e}}}}\) after executing the program. The use of a sensitivity environment \({ {{{\color {green}\Sigma }}}}\) is different from Duet, where sensitivities are tracked in Γ and presented as a necessary condition to typecheck the expression. In other words, in Sax \({ {{{\color {green}\Sigma }}}}\) is used to infer sensitivities, whereas in Duet Γ is used to check sensitivities.

Subtyping for types and sensitivity environments is presented in Figure 6 and is mostly standard. We only allow subtyping for the sensitivity parts of types. A sensitivity environment is subtype of another if their sensitivities are less than or equal to the other for each variable. For instance, \(\color{blue}{ (x:{{{ {{{\mathbb {R}}}}}})} \mathrel {{{{\color {green}{\xrightarrow {{{{\color {blue}{x+y}}}}}}}}}} {{{\color {blue}{ {{{\mathbb {R}}}}}}} \lt : (x:{{{ {{{\mathbb {R}}}}}}})} \mathrel {{{{\color {green}{\xrightarrow {{{{\color {blue}{x+2y+3z}}}}}}}}}} {{{\color {blue}{{{{\mathbb {R}},}}}}}}}\) because x + y < : x + 2y + 3z (x ≤ x, y ≤ 2y, and 0z ≤ 3z).

Type safety is established relative to the runtime semantics of Sax. We adopt a big-step semantics with explicit substitutions. Concretely, we use \(\color{blue}{ \gamma \vdash }\ {{{\color {green}{e}}}\ \color{blue}{\Downarrow}\ {{{\color {green}{v}}}}}\) to represent that configuration \(\color{blue}{ \gamma \vdash}\ {{{\color {green}{e}}}}\) —formed by expression \({ {{{\color {green}e}}}}\) and value environment γ mapping variables to values— reduces to value \({ {{{\color {green}v}}}}\) after some number of steps. Reduction rules are rather standard and can be found in Appendix B, Figure 27.

To establish Sax type safety, we employ simple unary logical relations, called the type safety logical relations, that characterize well-typed, non-stuck execution. This relation is well defined, because it is defined by induction over the structure of types. The type safety result itself is derived as a corollary of the fundamental property of the type safety logical relations.

The type safety logical relations is defined in Figure 7. For simplicity, we only present the cases for real numbers, variables, functions, and sums. The other cases are similar and straightforward. The unary logical relations are split into mutually recursive value relations \(\color{blue}{ {\mathcal {V}}}\), computation relation \(\color{blue}{ {\mathcal {E}}}\), and environment relation \(\color{blue}{ {\mathcal {G}}}\), and defined as follows:

As usual, the fundamental property of the type safety logical relation states that well-typed open terms are in the relation closed by an adequate environment γ:

Type safety for closed terms follows immediately as a corollary:

This section establishes the type soundness of Sax, stated in terms of a metric preservation result. Loosely speaking, metric preservation captures the maximum variation of an open term when it is closed under two different (but related) environments.

Logical relations. To establish this soundness result, we make use of logical relations [4, 6]. In particular, we define (mutually recursive) logical relations for sensitivity values, computations, and environments; see Figure 8. The logical relations for values (\(\color{blue}{ {\mathcal {V}}}_{{{{\color {green}{d}}}}}\color{blue}{[\![ \sigma ]\!]}\)) and computations (\(\color{blue}{{ {\mathcal {E}}}_{{{{\color {green}{d}}}}}\color{blue}{[\![ \sigma ]\!]}}\)) are indexed by a relational distance \({ {{{\color {green}{d}}}} \color{blue}{\in} {{{\color {blue}{ {{{\mathbb {R}}}}}}}^{\infty }_{\ge 0}}}\) and a so-called relational distance type σ, which is a regular type where sensitivity environments are enriched with a constant \({ {{{\color {green}{d}}}} \color{blue}{\in} {{{\color {blue}{{{{\mathbb {R}}}}}}}^{\infty }_{\ge 0}}}\) denoting the distance induced by pair of substitutions. Formally, the syntax of relational distance types is defined as follows:Notice that the logical relations do not mention sensitivity environments \({ {{{\color {green}\Sigma }}}}\), because they are defined over closed terms and values. Nevertheless, relational distance types σ do mention sensitivity environments \({ {{{\color {green}\Sigma }}}}\). We use a combination of sensitivity environments and relational distances (\({ {{{\color {green}{\Sigma} }}} \color{blue}{+} {{{\color {green}{d}}}}}\)), because function types introduce binders that cannot be substituted until application. For instance, consider type \(\color{blue}{ (x\mathrel {:}{\mathbb {R}}) }\mathrel {{{{\color {green}{\xrightarrow {{{{\color {blue}{x + y}}}}}}}}}} \color{blue}{({\mathbb {R}} \mathrel {^{x + 3z}\oplus ^{2x + 2y}} {\mathbb {R}})}\) and two pair of substitutions for y and z, at distance 2 and 1, respectively, e.g., γ₁ = y↦1, z↦1 and γ₂ = y↦3, z↦2, where |γ₁(y) − γ₂(y)| ≤ 2 and |γ₁(z) − γ₂(z)| ≤ 1. The corresponding relational distance type after substitution is . For notation simplicity, in the rest of the section, we name relational distance types as types when the accompanying relational distances can be inferred from the context. Also, we omit the environment notations when they are empty. However, the logical relation for environments (\(\color{blue}{ {{\mathcal {G}}}_{{{{\color {green}{\Delta }}}}}\color{blue}{[\![ \Gamma ]\!]}}\)) is indexed by a relational distance environment \({ {{{\color {green}\Delta }}}}\), mapping variables to relational distances in \(\color {blue}{\mathbb {R}}^{\infty }_{\ge 0}\) and a type environment Γ. We use \(\color{blue}{ (}{{{\color {green}{v_{1}}}}}\color{blue}{,}{{{\color {green}{v_{2}}}}}\color{blue}{)}\ \color{blue}{\in {\mathcal {V}}}_{{{{\color {green}{d}}}}}\ \color{blue}{[\![ \sigma ]\!]}\) to denote that value \(\color {green}{v_{1}}\) is related to value \(\color {green}{v_{2}}\) at type σ and relational distance \({ {{{\color {green}d}}}}\) and likewise for expressions (i.e., computations) and environments.

To define logical relations, we also make use of relational distance instantiations, which have shape \({ {{{\color {green}{\Delta} }}} \color{blue}{\cdot }\color{blue}{(}{{{\color {green}{\Sigma} }}} \color{blue}{+} {{{\color {green}{d}}}}\color{blue}{)}}\) and act by replacing free variables in sensitivity environment \({ {{{\color {green}\Sigma }}}}\) with the distances provided by distance environment \({ {{{\color {green}\Delta }}}}\). Relational distance instantiations only close variables defined in \({ {{{\color {green}\Delta }}}}\) and are formally defined as:Furthermore, to close a type under a sensitivity environment, we use the relational distance type instantiation operator \({ {{{\color {green}{\Delta} }}}\color{blue}{(\sigma)}}\) (note that a τ is also an σ assuming that the “default” relational distance \({ {{{\color {green}d}}}}\) is 0) defined below:

Now that we have all the prerequisites, we briefly go through the definition of the logical relations (in Figure 8)⁵:

Sensitivity Metric Preservation. Armed with these logical relations, we can establish the notion of type soundness and prove the fundamental property—well-typed terms are related with themselves—which corresponds to metric preservation [49]. As usual, we state this property appealing to open terms, where free variables indicate input parameters, which are then closed by related value environments.

In other words, if a sensitivity term is well-typed, then for any valid distance environment \({ {{{\color {green}\Delta }}}}\) (that “fits” Γ) and any two value environments γ₁, γ₂ related at Γ and \({ {{{\color {green}\Delta }}}}\), configurations \(\color{blue}{ \gamma _{1}\vdash}\ \color {green}{e,}\ \color{blue}{\gamma _{2}\vdash}\ \color {green}{e}\) represent related computations at type \(\color {green}{\Delta }\color{blue}{(\tau)}\) (closing all free variables) and distance \(\color {green}{\Delta }\color{blue}{\mathord {\cdotp }}\color {green}{\Sigma }\). Note that, since \(\color{blue}{ dom(}\color {green}{\Sigma }\color{blue}{) \subseteq dom(\Gamma) \subseteq dom(}\color {green}{\Delta }\color{blue}{)}\), we have \(\color {green}{\Delta }\color{blue}{\mathord {\cdotp }}\color {green}{\Sigma } \color{blue}{\in \mathbb {R}^{\infty }_{\ge 0}}\).

From the above theorem it is easy to derive a corollary that only characterizes closed terms:

As a direct consequence of Theorem 5.1, we can also establish the sensitivity type soundness at base types:

Let us illustrate metric preservation by revisiting some examples. Consider Example 4.2:If we know that in two different executions x may differ in at most 1, and y in at most 3, i.e., Δ = 1x + 3y, then the result will differ in at most \(\color{blue}{ \Delta \mathord {\cdotp } (2x + 2y) = 1\mathord {\cdot }2 + 3\mathord {\cdot }2 = 8}\). For instance, if in one execution x is bound to 0 and y to 4, then the result will be 8. In a second execution, if x is bound to 1 and y to 6, then the result will be 14. Comparing both results, we get |8 − 14| = 6 ≤ 8. Finally, in a third execution, if x is bound to 1 and y to 7, then the result will be 16. Comparing with the first execution, we have |8 − 16| = 8 ≤ 8, and with the second |14 − 16| = 2 ≤ 8.

Now consider Example 4.3:In this case, if x varies in two different executions (Δ(x) > 0), then the outcome will differ in at most \(\color{blue}{ \Delta (x)\mathord {\cdot }\infty =\infty}\). For instance, if in one execution x is bound to 0 the result will be true, and if in a second execution x is bound to 1, then the result is also going to be true, and true is at distance zero with respect to itself, and 0 ≤ ∞. If in a third execution x is bound to 11, then the result will be false, and false is at distance infinity with respect to true. Now, if we now that x is constant across multiple executions (Δ(x) = 0), then we know from metric preservation that the result will differ in \(\color{blue}{ 0\mathord {\cdot }\infty = 0}\), i.e., the result will be constant.

Consider a family of looping combinators parameterized by the number of loop iterations n, e.g., where \(\color{blue}{ {\text{loop}}_{3}\ x\ f = f(f(f\ x))}\). In Fuzz, loop_n would have the type \(\color {blue}{\tau \rightarrow (\tau\rightarrow {\scriptstyle \bigcirc } \tau ) \multimap _{n} {\scriptstyle \bigcirc } \tau}\). In this type, regular arrows → mean no sensitivity is tracked for the argument. The linear arrow \(\color{blue}{ \multimap _{{{\color {blue}n}}}}\) means the result is n-sensitive (where n is the number of loop iterations) in the closure variables of the supplied function of type \(\color{blue}{(\tau \rightarrow {\scriptstyle \bigcirc } \tau )}\). This allows for instantiating loop with a closure capturing a sensitive variable, like db. So, \(\color{blue}{\text{loop}_{n}0(\lambda x.x+{\mathtt{laplace}_{\epsilon }\ db)}}\) will give nϵ differential privacy for db by scaling ϵ—the privacy cost of closure variable db—by the loop iteration n. When supporting advanced variants of differential privacy like (ϵ, δ), a different metric must be chosen to recover this kind of scaling; otherwise, this argument only holds for pure ϵ-differential privacy.

In Duet, to support (ϵ, δ)-differential privacy (and disallow problematic scaling), privacy closures immediately report unbounded privacy (∞) for any captured variables in privacy lambdas. The principle of loop’s type above is justified in Duet, but not via a scaling argument, and instead via a primitive type rule—it cannot be expressed as a type. This is problematic for two reasons: First, it is not possible to extend Duet’s implementation with new looping primitives by adding terms with axiomatically justified types, leading to a bloated set of core typing rules, and second, it is not possible to lambda abstract looping combinators, e.g., to chain or compose them in helper functions.

To see the root cause for the limitation in Duet, we show the type rules for looping (advanced composition) and function introduction (from Reference [46]):In the rule for advanced composition shown above (left), e₁ is the initial value for the looping state of type τ, and e₂ is the loop body that updates the looping state \(\color {blue}{\tau \rightarrow \tau }\) and may mention closure variables in Γ₂. Parameter δ′ is a meta-parameter for the advanced composition formula—this parameter is unique to looping in (ϵ, δ)-differential privacy. The notation means there must exist some privacy cost ϵ and δ which upper-bounds any individual cost for each of these closure variables. The privacy cost of the whole loop is calculated based on this upper bound for closure variables with the formula . An attempt to turn loop into a primitive (or abstract over loop, e.g., eta-expand via lambda abstraction) fails, because privacy types in Duet do not track privacy effects for closure variables; instead, they are just thrown away. In the rule for function introduction shown above (right), the function type τ₁@(ϵ, δ)⊸^*τ₂ is a probabilistic function from elements in τ₁ to elements in τ₂, which satisfies (ϵ, δ)-differential privacy in its argument. Notice the closure environment Γ above the line that is bumped to ∞ in below the line. This has the effect of tossing out privacy bounds for anything with non-zero privacy in Γ, i.e., any closure variables that are used in the function’s definition. Privacy is only tracked for the function parameter x (or possibly multiple parameters; privacy functions in Duet are n-ary).

A deeper limitation in Duet is that the iterated 1-ary function space does not generalize to support encoding n-ary functions (i.e., currification is not supported). For this reason, n-ary functions are primitive in Duet. Implementing n-ary from 1-ary functions is computationally possible in Duet, but results in discarding bounds on privacy effects. For example, the Duet term in a context where has type (τ₁@∞)⊸^*(τ₂@(ϵ₂, δ₂))⊸^*τ₃, i.e., the privacy bounds (ϵ₁, δ₁) for the first argument τ₁ get discarded due to the Duet: Privacy-Fun-I rule.

Privacy Closures in Jazz. In Jazz, both privacy and sensitivity effects are delayed and attached to type-level connectives, including for privacy functions. Whereas, in Duet, privacy functions are written , privacy functions in Jazz are written simply where Σ is a latent contextual effect that can mention x. A type can now be given to loop (a named constant, analogous to the loop primitive from Duet) in Jazz, and abstracting over loop is possible due to the function introduction rule, also shown below.N-ary functions are now recoverable from 1-ary ones using latent contextual effects in closures. The relational distance \(\color {green}{d}\) defaults to 1 when omitted. The encoding of lambda-abstracted gauss then follows the usual approach of nested lambda abstractions, but with sensitivity lambdas on the outside with a single privacy lambda on the inside. A 3-ary abstraction of the Gaussian mechanism applied to the sum of three arguments is as follows:

Notice here that the latent contextual effect is computed using a syntactic join operator (ϵ, δ)x⊔(ϵ, δ)y⊔(ϵ, δ)z, which computes the pointwise maximum, instead of the sum ((ϵ, δ)x + (ϵ, δ)y + (ϵ, δ)z). One of the novelties of Jazz is that we can reason about two executions where more than one input is at relational distance greater than 0. In particular, if x, y and z are at relational distance 1, i.e., the argument of \(\color{blue}{ {\text{gauss}}3\epsilon \delta}\) is at relational distance 3, then using addition would yield an over-approximated latent privacy of (3ϵ, 3δ), while using the join, we obtain a latent privacy of (ϵ, δ), as desired.

Abstracting Privacy Mechanisms. Even with support for privacy closures, there are still challenges in supporting lambda abstraction around privacy mechanisms in full generality. In Fuzz, the type assigned to the family of Laplace differential privacy mechanisms parameterized by ϵ is \(\color{blue}{\text{laplace}_{\epsilon } \mathrel {:} \mathbb {R} \multimap _{\epsilon } {\scriptstyle \bigcirc } \mathbb {R}}\) for achieved privacy ϵ. This mechanism does not need a dedicated type rule in the core calculus—it can be axiomatized as a primitive with the right type—and lambda-abstracting this primitive is natural via eta-expansion \(\color{blue}{ \lambda (x \mathrel {:}{\mathbb {R}}). \text{laplace}_{\epsilon }x}\) resulting in the same type and guarantee for privacy. However, this approach does not support (ϵ, δ)-differential privacy directly. Fuzz^ϵδ shows how to extend Fuzz to recover (ϵ, δ)-differential privacy by using graded comonadic liftings and path construction. In particular, the type assigned to the family of Gaussian differential privacy mechanisms parametrized by ε and δ is , where \(\color{blue}{ \lceil {\mathbb {R}}\rceil}\) is \(\color{blue}{ {\mathbb {R}}}\) but with the metric rounded up to the nearest integer. In Duet, to support (ϵ, δ)-differential privacy, the Gaussian mechanism requires its own typing rule, shown below. Furthermore, a use of the mechanism looks like where the argument e is a term in the sensitivity language with sensitivity bounded by s. Using privacy closures as described above, we can write , however, note that we have lost the ability to be parametric in s—it must be fixed to 1. This assumption that gauss will be called only with a 1-sensitive argument is enforced in the function application rule in Duet, also shown below:In the rule for gauss (left) it allows an argument of any sensitivity s, however, the privacy function application rule (right) restricts that arguments must have sensitivity equal to 1. Restricting gauss to only 1-distance arguments can be overly restrictive (e.g., \(\color {red}{\texttt {gauss}}\ \color{blue}{2\epsilon \delta (x + x)}\)), and relaxing the restriction on function application to an arbitrary s ≠ 1 in Duet would be unsound.

In Jazz, we extend function introduction to include an explicit bound on the sensitivity of the parameter and enforce this restriction in the application rule. Function introduction syntax introduces the bound and allows us to eta-expand the Gaussian mechanism with relational distance \({ {{{\color {green}d}}}}\) as a parameter, as shown below. The bound \({ {{{\color {green}d}}}}\) for the lambda argument is then enforced in function application as the upper bound of argument relational distance, instead of being fixed to 1 as in Duet. Now the use of a variable—like x in the body of eta-expanded gauss below—is not always considered 1-sensitive. To communicate non-zero sensitivities to variables in the type system, an environment of relational distances on lambda arguments must be threaded through the type system, which we notate \({ {{{\color {green}\Delta }}}}\). After extending this \({ {{{\color {green}\Delta }}}}\) to remember that x has relational distance \({ {{{\color {green}d}}}}\) in Jazz lambda abstraction, \(\color{blue}{ {\text{gauss}}\ s\ \epsilon\ \delta\ x}\) will see x as \({ {{{\color {green}d}}}}\) distant inside the lambda body. To do this, we allow lambda-abstracting gauss (including the distance parameter \({ {{{\color {green}d}}}}\) via singleton types) and extend the structure of typing for sensitivity and privacy terms respectively as follows:

Jazz improves on prior systems by supporting let-binding intermediate sensitivity computations within the privacy language while also supporting (ϵ, δ)-differential privacy. Fuzz and DFuzz encode let-binding through function application, which scales the sensitivity of the right-hand side of the let with the sensitivity of the let-variable in the body. So, \(\color{blue}{\texttt {let}\ y = 2 * x\ {\texttt {in}}\ 3 * y}\) is 6-sensitive in x, because the right-hand side is 2-sensitive, and this is scaled by 3, the sensitivity of y in the body. However, monadic return and bind in Fuzz can also be used to encode let-binding, e.g., \(\color{blue}{ x \leftarrow {\text{return}}\ e_{1} \mathrel {;} e_{2}}\) instead of \(\color{blue}{\texttt {let}\ x=e_{1}\texttt {in}\ e_{2}}\). Unfortunately, this encoding of let using return and monadic bind does not preserve typeability in Fuzz; instead, it destroys the sensitivity/privacy analysis of the right-hand side, bumping its privacy cost unnecessarily to ∞. For this reason, let statements are encoded exclusively through function application in Fuzz and not through monadic return/bind.

In Fuzz, let-binding a sensitivity computation (the pure fragment) inside a privacy computation (the monadic fragment)—via encoding through function application—is supported seamlessly without the addition of extra rules. This flexibility can be extended to advanced privacy variants as shown by de Amorim et al. [29]. In Duet, however, the privacy/monadic fragment of Fuzz is pulled out into its own language with explicit typing rules; the primary reason to do this is to place restrictions on function application to support advanced privacy variants, as described in the previous subsection. This leaves the need for either an explicit typing rule for let-binding inside the privacy language or an escape hatch so privacy analysis is not destroyed for let-binding in privacy contexts à la Fuzz. Duet solves this issue by introducing a boxed type that delays the payment of a sensitivity term at the point it is “boxed” and pays for it later when it is “unboxed.” This avoids the issue but is unfriendly to program with: Every let-binding requires an explicit box, and every use of a let-bound variable requires an explicit unbox. So, instead of writing the program below on the left, Duet programmers are forced to write the program on the right.In this program, it is essential to let-bind the expensive result, since inlining it would unnecessarily duplicate the computation, and many real programs in differential privacy require support for this pattern [46].

In Jazz, we recover the expressiveness that box types provide while eliminating the need for the programmer to explicitly introduce and eliminate them. In this way, our design can also be seen as a powerful box-inference capability, although we do not demonstrate explicit embeddings between a core language with box types. To recover the expressiveness of boxes without requiring the programmer to write them down, we add new information to typing judgments that has the effect of automatically boxing let-bound variables in privacy contexts and unboxing them at their use. The added information extends typing judgments with a new component Φ that tracks the sensitivities of all let-bound variables w.r.t. the sensitivities of all lambda-bound variables. All sensitivity contexts that mention both let-bound and lambda-bound variables are then reduced using Φ as needed to contexts that only mention lambda-bound variables. Φ can be seen as a matrix, and the reduction of contexts to only lambda-bound variables is then just matrix multiplication—a beautiful coincidence for a linear type system. The final form of type judgments for the sensitivity and privacy type systems are then:Although the prototype implementation adopts the typing rules with Φ, and because the manipulation of Φ is more tedious than insightful, we omit it in the following technical presentation.

\(\lambda _{\text{ {J}}}\) is divided in two mutually embedded sublanguages: the sensitivity sublanguage—an extension of Sax—used to reason about the sensitivity of computations, and the privacy sublanguage, used to reason about differential privacy. Thus, the type system of \(\lambda _{\text{ {J}}}\) contains two mutually embedded type systems, one for each of the sublanguages. Expressions of the sensitivity sublanguage remain typeset in green and expressions of the privacy sublanguage are typeset in red.

Syntax. Figure 9 presents the syntax of \(\lambda _{\text{ {J}}}\). Expressions of the language are divided into two mutually embedded expressions: sensitivity expressions \(\color {green}{e}\) and privacy expressions \(\color {red}{\mathsf {e}}\). Sensitivity expressions are defined the same way as in Sax, except that functions are split into sensitivity lambdas and privacy lambdas . Note that the only difference between a sensitivity lambda and a privacy lambda is that the body of a privacy lambda is a privacy expression \(\color {red}{\mathsf {e}}\). Also, both sensitivity lambdas and privacy lambdas are parametrized by a relational distance \({ {{{\color {green}d}}}}\) that represents an upper bound on distance between inputs pertained to the binary relational property of differential privacy: the maximum argument variation for each of two executions.

A privacy expression \(\color {red}{\mathsf {e}}\) can be a point distribution \(\color {red}{\texttt {return}}\ \color {green}{e}\), a sequential composition \(\color{blue}{ x:\tau \leftarrow} \color {red}{\mathsf {e}}\color{blue}{\mathrel {;}}\color {red}{\mathsf {e}}\), an application \(\color{red}{\mathsf {e\ e}}\), a conditional \(\color{red}{{\texttt{if}}\ \color {green}{e}\ \color{red}{\texttt {then}}\ {\mathsf{e}}\ {\texttt{else}}\ {\mathsf {e} }}\), a case expression , or a let .

A privacy cost \(\color{red}{p} \color{blue}{= (}\color {red}{\epsilon }, \color {red}{\delta }\color{blue}{)}\) is a pair of two (possibly infinite) real numbers, where the first component corresponds to the epsilon and the second to the delta in (ϵ, δ)-differential privacy. We use notation \(\color{red}{p}\color{blue}{.}\color {red}{\epsilon }\) and \(\color{red}{p}\color{blue}{.}\color {red}{\delta }\) to extract \({ {{{\color {red}\epsilon }}}}\) and \({ {{{\color {red}\delta }}}}\), respectively. A privacy environment \({ {{{\color {red}\Sigma }}}}\) is either an empty environment \(\color{blue}{ \varnothing}\), a pair \(\color {red}{p}\color{blue}{x}\) representing that variable x has privacy cost \({ {{{\color {red}p}}}}\), the addition \(\color {red}{\Sigma } \color{blue}{+} color {red}{\Sigma }\) of two privacy environments, the join \(\color {red}{\Sigma }\ \color{blue}{\sqcup}\ \color{red}{\Sigma }\) of two privacy environments, and the meet \(\color {red}{\Sigma }\ \color{blue}{\sqcap}\ \color{red}{\Sigma}\) of two privacy environments. Similarly to sensitivity environments, we also write privacy environments as first-order polynomials when possible. For instance, \(\color {red}{p_{1}}\color{blue}{x +} \color {red}{p_{2}}\color{blue}{x}\) can be written as \(\color{blue}{ (}\color{red}{p_{1}}\ \color{blue}{+}\ \color {red}{p_{2}}\color{blue}{)x}\), but cannot be rewritten as a polynomial without losing precision. Function types are now divided into sensitivity function types and privacy function types .

Sensitivity type system. The type system for the sensitivity sublanguage is presented in Figure 10. The judgment now includes a novel relational distance environment \(\color {green}{\Delta }\). The relational distance environment \(\color {green}{\Delta }\) stores how much each variable in Γ can vary in every two executions of a program. Most of the rules are straightforward extensions of the type system of Sax to include relational distance environments. We only present interesting cases.

Some of the rules use the sensitivity environment substitution operator \(\color{blue}{ [}\color {green}{\Sigma }\color{blue}{/x]\tau}\). We extend the definition of Sax to support privacy functions, as shown in Figure 11. Substitution on privacy function types depends on the definition of sensitivity environment substitution on privacy environments \(\color{blue}{ [}\color {green}{\Sigma }\color{blue}{/x]}\color {red}{\Sigma }\). \(\color{blue}{ [}\color {green}{\Sigma }\color{blue}{/x]}\color {red}{\Sigma }\) is defined inductively on the structure of \({ {{{\color {red}\Sigma }}}}\), where the only interesting case is when \(\color {red}{\Sigma } \color{blue}{=} \color {red}{p}\color{blue}{x}\). Substitution \(\color{blue}{ [}\color {green}{\Sigma }\color{blue}{/x]}\color {red}{p }\color{blue}{x}\) is defined using the lift operator: . Intuitively, if we wiggle⁶ x on \(\color {red}{p}\color{blue}{x}\), then the privacy obtained is at most \({ {{{\color {red}p}}}}\) (no scaling and zero if x does not change). After substitution, as x depends on all variables on \({ {{{\color {green}\Sigma }}}}\), if we wiggle all variables in \({ {{{\color {green}\Sigma }}}}\) at the same time, then the privacy obtained should still be \({ {{{\color {red}p}}}}\) (scaling \({ {{{\color {red}p}}}}\) would be an over-approximation). Because of this, is defined as the join , where \(\color{blue}{ x_{i} \in dom(}\color {green}{\Sigma }\color{blue}{)}\). If all x_i wiggle, then the privacy obtained would be at most \({ {{{\color {red}p}}}}\). But as any \(\color {green}{\Sigma }\color{blue}{(x_{i})}\) can be zero (it means that variable x_i is not used), we multiply each \({ {{{\color {red}p}}}}\) in \(\color {red}{p}\color{blue}{x_{i}}\), by to remove those variables from the resulting privacy environment. along other lift operators used by Near et al. [46] are defined in Figure 11. For instance, suppose that x depends on 2y + 0z, then \(\color{blue}{ [(2y+0z)/x]}\color{red}{p}\color{blue}{x}\) is computed as .

We now turn to describe the main changes of each type rule with respect to Sax.

Subtyping is extended accordingly and presented in Figure 12. Parameterized relational distances on function types are contravariant, and subtyping for privacy function types relies on the definition of subtyping for privacy environment also defined in Figure 12, where \(\color {red}\bullet\) is an operator to close privacy environments defined below:

Privacy type system. The type system of the privacy part of the language is presented in Figure 13. The judgment says that privacy term \(\color {red}{\mathsf {e}}\) has type τ and ambient privacy effect \({ {{{\color {red}\Sigma }}}}\) under type environment Γ and relational distance environment \(\color {green}{\Delta }\).

Type safety is defined in the same line of Section 5.2. To establish type safety of the privacy language, we define a non-deterministic sampling big-step semantics of privacy expressions; see Figure 14. We naturally extend the type safety logical relations of Figure 8 to support for both sensitivity and privacy lambdas and privacy expressions as shown in Figure 15. The fundamental property of the type safety logical relation is defined similarly to Proposition 5.1, but now accounting for relational distance environments and expressions:

Finally, type safety for closed terms is just a corollary of the fundamental property above:

This section establishes the soundness of \(\lambda _{\text{ {J}}}\), named metric preservation. Metric preservation for \(\lambda _{\text{{J}}}\) extends the notion of metric preservation of Sax. In addition to reasoning about sensitivity terms, given a privacy term with free variables, we can reason about the achieved privacy level when closing the privacy term under different (but related) environments.

Contrary to Sax, we establish soundness for \(\lambda _{\text{ {J}}}\) using a step-indexed logical relation [3]. Although \(\lambda _{\text{{J}}}\) is a strongly normalizing language, step indexing is still required to prove the bind case of the fundamental property of the logical relation.

We have implemented a prototype of the Jazz typechecker in Haskell and used it to verify the case studies from Table 3. The prototype implementation is available on GitHub.⁷ Table 3 lists the time needed to typecheck each of the case studies; our typechecker takes just a few milliseconds for each one.

Type inference & annotations. Our prototype implements type inference for both Sax and Jazz. Type annotations for sensitivity and privacy are required for inputs at top-level functions and lambda-expressions, but no additional annotations are required for function outputs or elsewhere in the program. The case studies described later in this section have been typeset for readability, but are otherwise identical to the input for our prototype; in particular, the actual examples typechecked by our prototype have the same annotations as the examples in this section, except for the input sensitivity annotations on inputs to top-level functions. The types given for primitives in this section are also drawn directly from our implementation.

Constraint solving. As described earlier, and detailed in Section 10, prior work has made extensive use of SMT solvers for the equations over real expressions that arise in type inference for sensitivity. Because SMT solvers are incomplete for non-linear operations (like the logarithms and square roots used in advanced composition), our implementation does not follow the same path.

Instead, we implement a custom solver for inequalities over symbolic real expressions, based on the solver from Duet [46]. Our custom solver is based on a simple decidable (but incomplete) theory; it supports logarithms, square roots, and polynomial formulas over real numbers. The solver is transparent to the programmer and produces readable output expressions for the privacy costs in our case studies.

The MWEM algorithm [38] generates differentially private synthetic data approximating the target sensitive data by iteratively optimizing the accuracy of a set of workload queries on the synthetic data. In each iteration, the algorithm uses the exponential mechanism to pick a query from the workload for which the synthetic data produces an inaccurate result, uses the Laplace mechanism to run that query on the real data, and uses the result to update the synthetic data via the multiplicative weights update rule. The Jazz program shown below implements the MWEM algorithm. Its inputs are a sensitive dataset X over a domain D, a workload Q of linear queries, the number of iterations to be performed k, the privacy parameter ϵ, initial synthetic data Y₀ (n times the uniform distribution over D), and the dimensions of the input data set (matrix) m rows by n columns. The algorithm performs k iterations, invoking laplace and exponential in each iteration. The privacy parameter for each invocation is \({\color {red}\epsilon} /{\color {blue}{2k}}\), yielding a total privacy cost of ϵ. We omit the sensitivity annotations on the function’s inputs for readability.The Jazz typechecker produces the following type for this implementation, indicating that the algorithm satisfies ϵ-differential privacy. Note the homogenous matrix type notation used here is \(\color{blue}{\mathbb {M} {\color {blue} {[m,n] \tau}}}\), where m denotes the number of rows, n the number of columns, and τ the type of each entry. < X > is the context schema argument for type-application of loop and indicates the program variable we want to preserve privacy for in this expression.On an average of 10 runs, it takes the Jazz typechecker 5.2 ms to produce this type for the MWEM algorithm.

Beyond Duet. This example demonstrates Jazz’s ability to define algorithms in terms of library functions and to parameterize algorithms by the choice of component pieces. In this case, we define MWEM in terms of a generic looping privacy combinator loop; the caller of MWEM can specify looping combinators based on sequential composition, advanced composition, or even a custom combinator. Jazz similarly allows functions like MWEM to be parameterized by the choice of basic mechanism (e.g., laplace vs. gauss) with the appropriate privacy function type. In both cases, the relevant functions can be pulled from libraries or defined by the programmer.

This kind of modularity is impossible in Duet. Functions cannot be parameterized by basic privacy mechanisms or looping combinators, because it is not possible to write their types in Duet.

Primitives used. This case study demonstrates the composition of a complex iterative algorithm from basic privacy mechanisms encoded as Jazz primitives (e.g., laplace and exponential) and privacy combinators (e.g., seqloop, which implements looping with sequential composition for privacy). These primitives with types shown above would require explicit typing rules in the core Duet language. In Jazz, they can be given regular types, as shown below:To typecheck MWEM, the privacy closure rule in Jazz creates a function type for the \(\color{red}{\color {cbsafeCBright}{\color {cbsafeCBright}\lambda ^{\!^{_{\mathrm{p}}}}}}\) that has a privacy effect for the body of \(\color{blue}{ \frac{\epsilon }{k}}\) because of the two uses of mechanisms that give \(\color{blue}{ \frac{\epsilon }{2k}}\) differential privacy. If we pass seqloop as the looping combinator loop, then this privacy effect is multiplied by the loop iteration number k as a result of the type of and the type rule for privacy function application. Finally, the new let rule for the Jazz privacy fragment that tracks latent contextual sensitivities is used in the let-binding for q_i to precompute an intermediate value that is used multiple times without the need for explicit boxing.

The current state-of-the-art in differentially private machine learning is noisy gradient descent [2]: At each iteration of training, compute the gradient, clip the gradient to have bounded L2 norm, and add noise in proportion to the clipping parameter. The clipping parameter is typically treated as a hyperparameter set by the analyst before training.Recent work by Thakkar et al. [53] proposed an algorithm for adaptively determining the clipping parameter during training by adaptively improving the clipping parameter based on a differentially private estimate of the percentage of gradients clipped in each iteration.

In each iteration, the implementation computes the gradients for a batch of examples gs, clips each gradient using the current parameter C^t, and uses the Gaussian mechanism to compute a differentially private average gradient g_p. Then, the algorithm updates the clipping parameter for the next iteration C^t′ using \({{\color {red}{\mathsf{clipUpdate}}}}\), which computes a noisy count β of the number of gradients in gs that are clipped under the clipping parameter C^t and uses the count to update the parameter. The inputs to the algorithm are the training data X, the training labels y, the number of iterations k, and the target percentage of gradients remaining un-clipped γ. ϵ, δ, and δ′ are the privacy cost parameters. We omit the sensitivity annotations on top-level function inputs for readability. The Jazz typechecker typechecks DPAL in 5.6 ms (averaged over 10 runs).

Beyond Duet. In this case study, we implement a library function for the differentially private average (\({{\color{red}{\mathsf{DPMean}}}}\)) and use it in two places. This refactoring is not possible in Duet, because Duet’s privacy functions require all sensitive arguments to have a sensitivity of 1. In this algorithm, one of the uses of \({{\color{red}{\mathsf{DPMean}}}}\) in fact has data-dependent sensitivity (the sensitivity of \({{\color {green}{\text{mclip}}}\ {{{\color {blue}C^{t}}}}\ {\color {blue}{gs}}}\) is \(\color{blue}{ {{{\color {blue}C^{t}}}}}\)).

Practical implementations of algorithms like this one often rely on libraries of differentially private functions like \({{\color {red}{{\mathsf{DPMean}}}}}\) (e.g., the Opacus library for differentially private deep learning [58], or the OpenDP library for differentially private analytics [35]). By lifting the limitations of Duet’s privacy functions, Jazz makes it possible to implement and use such libraries.

Primitives used. This algorithm demonstrates the use of privacy combinators (e.g., \({{\color{red}{\mathsf{aloop}}}}\)) that can be specified as primitives in Jazz but require special typing rules in Duet:This type for \({{\color{red}{\mathsf{aloop}}}}\) encodes the advanced composition theorem (introduced in Section 2). Our advanced composition combinator runs an (ϵ, δ)-differentially private function (\(\color{blue}{ {\mathcal {M}}}\)) representing the body of the loop k times, for a total privacy cost of \(\color{blue}{ (}\color {red}{2\epsilon (\sqrt {2k(\log (1/\delta ^{\prime }))}), k\delta + \delta ^{\prime }}\color{blue} {)}\) (a significant improvement over the sequential composition cost of \(\color{blue}{ (}\color {red}{k\epsilon , k\delta }\color{blue}{)}\)).

Our version of the adaptive clipping gradient descent algorithm uses advanced composition and (ϵ, δ)-differential privacy to demonstrate the encoding of \({\color {red}{\mathsf{aloop}}}\) as a regular function in Jazz. Our source code repository contains an alternative implementation that uses zero-concentrated differential privacy for improved composition and converts the privacy guarantee to (ϵ, δ)-differential privacy at the end of the algorithm.

There are two threads of prior work in type-system-based verification of differential privacy for high-level programs: those based on linear types and those based on relational refinement types. Reed and Pierce [49] proposed Fuzz, the first type system for differential privacy based on linear typing; its fundamental components are a linear type system with an indexed “scaling” modality !_s for tracking the sensitivity of programs and a monadic connective \(\color{blue}{ {\scriptstyle \bigcirc }}\) to model randomized computations. An s-sensitive function is encoded in Fuzz as a linear function with scaled domain !_sA⊸B and often notated A⊸_sB. An ϵ-differential privacy mechanism is represented as an ϵ-sensitive function with monadic return type as in \(\color{blue}{ A \multimap _{\epsilon } {\scriptstyle \bigcirc }B}\). DFuzz [34] extends Fuzz with dependent types to encode sensitivity and privacy bounds that depend on the values of function arguments. This allows, e.g., reasoning about the privacy of iterative algorithms whose privacy cost depend on the number of iterations. Fuzz and DFuzz can be characterized by strong support for higher-order programming and potential for automation via type inference. They support pure differential privacy but approximate, and any other recent variants of differential privacy fall out of their scope due to nonlinear scaling. Several recently proposed approaches allow a Fuzz-like analysis for (ϵ, δ)-differentially private programs: Azevedo de Amorim et al. [29] leverage a path construction and a Fuzz-like type system. Fuzzi [60] integrates a Fuzz-like type system with an expressive program logic. Fuzzi directly connects (automated) type-based proofs of composition for sensitivity and privacy properties with (manual) apRHL proofs for basic constructs such as sequential composition and the Laplace mechanism. In our approach, however, properties of basic mechanisms must be axiomatized. The Fuzzi system targets imperative programs and does not provide support for higher-order programming with privacy functions. Finally, Duet [46] proposes a two-language design with linear types for tracking sensitivity and privacy; crucial restrictions in the typing rules of the privacy language allow encoding advanced differential privacy variants such as approximate, Rényi, zero-concentrated, and truncated-concentrated differential privacy. In Duet privacy functions are n-ary and written \(\color{blue}{ ({{\color {blue}\tau _{1}}}@{{{\color {red}{p_1}}}},\ldots ,{{\color {blue}\tau _{n}}}@{{{\color {red}{p_n}}}}) \multimap ^{*} {{\color {blue}\tau }}}\) for privacy quantities p_i such as (ϵ_i, δ_i) in the case of approximate differential privacy.

Unlike previous works based on linear type systems, HOARe² [14] uses relational refinement types to encode arbitrary relational properties of programs, including differential privacy. In HOARe², an \({ {{{\color {green}s}}}}\)-sensitive function type is written \(\color{blue}{ \Pi s^{\prime }. \lbrace x \mathrel {:: } \tau _{1} \mathrel {|} {\mathfrak {D}} _{\tau _{1}}(x_{\vartriangleleft },x_{\vartriangleright }) \le s^{\prime }\rbrace \rightarrow \lbrace y \mathrel {:: } \tau _{2} \mathrel {|} {\mathcal {D}}_{\tau _{2}}(y_{\vartriangleleft },y_{\vartriangleright }) \le {{{\color {green}s}}} s^{\prime }\rbrace}\) where \(\color{blue}{ {\mathfrak {D}} _{\tau }}\) is a type-indexed distance metric, and x_⊲ and x_⊳ are explicit symbolic representations of the “left” and “right” executions of the program in support of encoding relational properties. To account for probabilistic private computations, HOARe² uses an indexed monad \(\color{blue}{ {\mathfrak {M}} _{\epsilon ,\delta }[\tau ]}\): The type of an (ϵ, δ)-differentially private function is written \(\color{blue}{ \lbrace x \mathrel {:: } \tau _{1} \mathrel {|} {\mathfrak {D}} _{\tau _{1}}(x_{\vartriangleleft },x_{\vartriangleright }) \le 1\rbrace \rightarrow {\mathfrak {M}} _{\epsilon ,\delta }[\lbrace y \mathrel {:: } \tau _{2} \mathrel {|} y_{\vartriangleleft } = y_{\vartriangleright }\rbrace ]}\). A limitation of this encoding is that a function of two arguments that provides different privacy bounds for each argument (as described in Section 3.3) will report a summed, global privacy bound, because the tracking of privacy occurs in a single global index to the privacy monad \(\color{blue}{ {\mathfrak {M}} _{\epsilon ,\delta }}\). Because privacy is proved as a relational property, rather than as a sensitivity/Lipschitz continuity property, HOARe² is also capable of placing relational distance bounds on arguments to functions. Regarding the implementation, both HOARe² and Jazz use dependent types to capture sizes. Regarding typechecking, even though some automation has been achieved [24], the automation relies heavily on what is achievable with SMT solvers and has limited application to programs that make generous use of compositional or higher-order programming techniques or metric-distance relationships between values at non-base types. SMT solvers are not complete for non-linear operations, which are common in complex differential privacy mechanisms. Consider, for example, the expression for privacy cost under advanced composition: \(\color{blue}{ \epsilon ^{\prime } = k\epsilon (e^\epsilon -1) + \epsilon \sqrt {2 k \ln (1/\delta ^{\prime })}}\); SMT solvers are not capable of proving universally quantified qualities between equations like these, which limits their ability to automate reasoning about privacy cost.

It is important to note that all of these type systems support some form of recursion. Adding any form of recursion in the formalism of Jazz would make the technical development even more complicated. We just focused on a small core that could illustrate the main novelties of the latent or contextual approach.

To conclude the overview about type-system-based verification techniques, we refer the reader to Table 4, comparing different aspects of the reviewed type systems.

Approximate couplings [12] are a probabilistic abstraction that witnesses differential privacy properties of programs and have been successfully exploited for verification purposes. The relational Hoare logic apRHL [16] and its successors apRHL⁺ [15] and span-apRHL [50] internalize the compositional construction of such couplings and capture from pure and approximate differential privacy to more recent variants such as Rényi, zero-concentrated, and truncated-concentrated differential privacy. While compared to other methods, these program logics are rather expressive going beyond the composition of (a set of predefined) basic mechanisms, derivations in the logics involve complex quantitative reasoning, not always amenable to automation. Even though there has been a successful report on partial automation [11], e.g., the synthesis of (quantitative relational) loop invariants for iterative algorithms remains challenging. To synthesize couplings, Albarghouthi and Hsu [5] use an alternative approach based on constraint solving, which is highly amenable to automation; the approach is, however, confined to ϵ-DP. Finally, to verify programs that achieve (ϵ, δ)-DP composing basic mechanisms, Barthe et al. [13] use a customized program product construction and traditional (non-relational and non-probabilistic) Hoare logic augmented with mechanism-specific rules. In recent work, Barthe et al. [9] show that checking differential privacy for imperative programs is undecidable in general but present a reduction to a decidable fragment of first-order logic for a restricted class of programs. Barthe et al. [10] also show that checking accuracy bounds for differentially private programs is also undecidable (in general). A common limitation of all these approaches is that they are restricted to first-order imperative programs.

LightDP [59] and ShadowDP [55] take a third approach to verifying differential privacy based on randomness alignments. A randomness alignment is an injective function relating the randomness from one execution of a differentially private mechanism to a second execution of the same mechanism (i.e., \(\color{blue}{ {\mathcal {M}}(x)}\) outputs the same result with noise H as \(\color{blue}{ {\mathcal {M}}(x^{\prime })}\) outputs with noise f(H), where f is the randomness alignment). Both LightDP and ShadowDP are capable of verifying complex low-level mechanisms like the sparse vector technique in just a few seconds. However, both tools target a first-order imperative programming language and have limited support for higher-order programming.

Since differential privacy mechanisms are randomized, traditional methods of software testing do not apply. Two recent works by Bichsel et al. [19] and Ding et al. [30] address this challenge by automatically generating neighboring inputs for the mechanism being tested and sampling from their outputs many times to approximate their output distributions. For privacy mechanisms with major bugs, these tools are able to show that the approximated distributions do not satisfy the claimed differential privacy guarantee. Wilson et al. [56] have implemented a testing tool based on this approach in their open-source library. DPCheck [61] combines static analysis with instrumented concrete execution of the target program to detect bugs in even more complex algorithms.

The technical device of contextual latent effects used in Sax and Jazz is related to prior approaches to expose information about captured variables in function types. Leroy [40] and Hannan et al. [37] use function types augmented with the set of captured variables, the former for tracking dangerous type variables for polymorphic generalization and the latter for lifetime analysis. Scherer and Hoffmann [51] introduce open closure types to track additional information about closed-over variables in first-class functions. An open closure type augments the traditional arrow type with a lexical environment of closed over variables, further decorated by a mapping characterizing the use of each variable. For instance, they formalize a system where the mapping marks each variable with a Boolean indicating whether the evaluation of the body of the function depends on the variable or not. An open closure type also includes the name of the function argument similarly decorated. For the considered system, they prove a non-interference property, which states that closing an open term with two valuations that coincide on used variables yield the same result, up to used variables. Function types in Sax and Jazz can be seen as specific cases of open closure types, in which the information attached to captured variables (and arguments) is not a Boolean value, but sensitivity and privacy information. Consequently, the type soundness result we establish is more general than noninterference; for instance, in Sax, for two valuations that are at a given distance apart, the distance between the results is bounded by the sensitivities of each variable. The technical development of contextual linear types shares many concerns with that of open closure types, notably regarding the proper handling of the typing environment—in which order matters—and of scoping. However, our soundness results rely on logical relations, while they adopt a more restricted technique, sufficient for the purpose of the simple type system tracking Boolean information.

As observed by Scherer and Hoffmann, using open closure types allows delaying the accounting of information flow into closures from abstraction time to application time. Likewise, contextual sensitivity in Sax delays sensitivity accounting to application time. We extend this principle to positive type constructors such as products and sums, generally deferring accounting to elimination forms; we believe this would likewise apply to the simpler information-flow control setting they study.

Recently, Bao et al. [8] use a similar context-annotation technique to track reachability information in types. For instance, they augment the type of reference cell with the variables in scope that alias the cell. This information is tracked on function types as well to track the reachability information from closures. In all these approaches, type information depends on variable names; this is quite different from dependent types, however, where types depend on arbitrary terms. The telescope nature of the typing environment is similar, but many of the deep challenges of dependent types do not manifest in this restricted setting.

Hoare Type Theory [45] supports specifying effectful, heap-manipulating computations by introducing Hoare-style pre/postconditions in types. Computation types include contexts of variables and heap locations to track footprints of logical assertions. In a similar vein as Leroy [40], the context of variables is not decorated with any information, so it is unclear whether one could handle contextual sensitivity and privacy in this approach.

Finally, it would be interesting to study if generic approaches such as coeffects [48] and graded modal types [47] can express delayed sensitivity and privacy tracking as developed here. These generic approaches use resource algebras (such as the semiring of natural numbers) for capturing modalities in types. To the best of our knowledge, these are not contextual: For instance, the arrow type is annotated with a scalar, label, and so on, not with a decorated environment as in open closure types and this work. Extending these generic approaches to support contextual information on all type constructors could bring the benefits of lazy accounting to a wide range of type-based quantitative program reasoning.