Computer Science > Data Structures and Algorithms
[Submitted on 9 Jul 2015 (v1), revised 25 Nov 2015 (this version, v3), latest version 21 Aug 2017 (v4)]
Title:Differentially Private Ordinary Least Squares: $t$-Values, Confidence Intervals and Rejecting Null-Hypotheses
View PDFAbstract:Linear regression is one of the most prevalent techniques in data analysis. Given a large collection of samples composed of features $\vec x$ and a label $y$, linear regression is used to find the best prediction of the label as a linear combination of the features. However, it is also common to use linear regression for its \emph{explanatory} capabilities rather than label prediction. Ordinary Least Squares (OLS) is often used in statistics to establish a correlation between an attribute (e.g. gender) and a label (e.g. income) in the presence of other features. Under the assumption of a certain random generative model for the data, OLS derives \emph{$t$-values} --- representing the likelihood of each real value to be the true correlation in the underlying distribution. Using $t$-values, OLS can release a \emph{confidence interval} that is likely to contain the true correlation. When this interval does not intersect the origin, we can \emph{reject the null hypothesis} as it is likely that $x_j$ indeed has a non-zero correlation with $y$.
Our work aims at achieving similar guarantees on data under differentially private estimators. We use the Gaussian Johnson-Lindenstrauss transform, which has been shown to satisfy differential privacy if the given data has large singular values. We analyze the result of projecting the data using JLT under the OLS model and derive approximated $t$-values, confidence intervals and bound the number of samples needed to reject the null hypothesis when the data is drawn i.i.d from a multivariate Gaussian. When not all singular values of the data are sufficiently large, we increase the singular values, thus our projected data yields an approximation for the Ridge Regression problem. We derive, under certain conditions, confidence intervals in this case as well. We also derive confidence intervals for the "Analyze Gauss" algorithm of Dwork et al.
Submission history
From: Or Sheffet [view email][v1] Thu, 9 Jul 2015 12:32:19 UTC (41 KB)
[v2] Mon, 2 Nov 2015 02:19:03 UTC (46 KB)
[v3] Wed, 25 Nov 2015 00:24:42 UTC (46 KB)
[v4] Mon, 21 Aug 2017 21:30:27 UTC (605 KB)
Current browse context:
cs.DS
References & Citations
Bibliographic and Citation Tools
Bibliographic Explorer (What is the Explorer?)
Litmaps (What is Litmaps?)
scite Smart Citations (What are Smart Citations?)
Code, Data and Media Associated with this Article
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub (What is DagsHub?)
Gotit.pub (What is GotitPub?)
Papers with Code (What is Papers with Code?)
ScienceCast (What is ScienceCast?)
Demos
Recommenders and Search Tools
Influence Flower (What are Influence Flowers?)
Connected Papers (What is Connected Papers?)
CORE Recommender (What is CORE?)
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.