Computer Science > Data Structures and Algorithms
[Submitted on 9 Jul 2015 (this version), latest version 21 Aug 2017 (v4)]
Title:Differentially Private Least Squares: Estimation, Confidence and Rejecting the Null Hypothesis
View PDFAbstract:Linear regression is one of the most prevalent techniques in data analysis. Given a collection of samples composed of features $x$ and a label $y$, linear regression is used to find the best prediction of the label as a linear combination of the features. However, it is also common to use linear regression for its explanatory capabilities rather than label prediction. Ordinary Least Squares (OLS) is often used in statistics to establish a correlation between an attribute (e.g. gender) and a label (e.g. income) in the presence of other features. OLS uses linear regression in order to estimate the correlation between the label and a feature $x_j$ on a given dataset; and then, under the assumption of a certain generative model for the data, OLS outputs an interval that is likely to contain the correlation between $y$ and $x_j$ in the underlying distribution (a confidence interval). When this interval does not intersect the origin, we can reject the null hypothesis as it is likely that $x_j$ has a non-zero correlation with $y$.
Our work aims at achieving similar guarantees on data under differential privacy. We use the Gaussian Johnson-Lindenstrauss transform, which has been shown to satisfy differential privacy if the data has large singular values. We analyze the result of JL projection in the OLS model and show how to approximate confidence intervals using only the projected data, and bound the number of samples needed to reject the null hypothesis with i.i.d draws from a multivariate Gaussian. When not all singular values of the data are sufficiently large, we increase the input's singular values and then use the JL transform. Thus our projected data yields an approximation for the Ridge Regression problem - a variant of the linear regression that uses a $l_2$-regularization term. We give conditions under which the regularized problem is still helpful in establishing correlations.
Submission history
From: Or Sheffet [view email][v1] Thu, 9 Jul 2015 12:32:19 UTC (41 KB)
[v2] Mon, 2 Nov 2015 02:19:03 UTC (46 KB)
[v3] Wed, 25 Nov 2015 00:24:42 UTC (46 KB)
[v4] Mon, 21 Aug 2017 21:30:27 UTC (605 KB)
Current browse context:
cs.DS
References & Citations
Bibliographic and Citation Tools
Bibliographic Explorer (What is the Explorer?)
Litmaps (What is Litmaps?)
scite Smart Citations (What are Smart Citations?)
Code, Data and Media Associated with this Article
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub (What is DagsHub?)
Gotit.pub (What is GotitPub?)
Papers with Code (What is Papers with Code?)
ScienceCast (What is ScienceCast?)
Demos
Recommenders and Search Tools
Influence Flower (What are Influence Flowers?)
Connected Papers (What is Connected Papers?)
CORE Recommender (What is CORE?)
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.