research-article

Tight bounds for l_p oblivious subspace embeddings

Authors:

Ruosong Wang,

David P. WoodruffAuthors Info & Claims

SODA '19: Proceedings of the Thirtieth Annual ACM-SIAM Symposium on Discrete Algorithms

Pages 1825 - 1843

Published: 06 January 2019 Publication History

Get Access

Abstract

An l_p oblivious subspace embedding is a distribution over r x n matrices Π such that for any fixed n x d matrix A,

Pr[for all x, ||Ax||_p ≤||ΠAx||_p≤k||Ax||_p] ≥ 9/10,Π

where r is the dimension of the embedding, k is the distortion of the embedding, and for an n-dimensional vector y, ||y||_p = (Σⁿ_i=1 |yi|)^1/p is the l_p-norm. Another important property is the sparsity of Π, that is, the maximum number of non-zero entries per column, as this determines the running time of computing Π · A. While for p = 2 there are nearly optimal tradeoffs in terms of the dimension, distortion, and sparsity, for the important case of 1 ≤ p < 2, much less was known. In this paper we obtain nearly optimal tradeoffs for l_p oblivious subspace embeddings for every 1 ≤ p < 2. Our main results are as follows:

1. We show for every 1 ≤ p < 2, any oblivious subspace embedding with dimension r has distortion k = [MATH HERE]. When r = poly(d) ⩽ n in applications, this gives a k = Ω(d^1/p log^−2/p d) lower bound, and shows the oblivious subspace embedding of Sohler and Woodruff (STOC, 2011) for p = 1 and the oblivious subspace embedding of Meng and Mahoney (STOC, 2013) for 1 < p < 2 are optimal up to poly(log(d)) factors.

2. We give sparse oblivious subspace embeddings for every 1 ≤ p < 2 which are optimal in dimension and distortion, up to poly(log d) factors. Importantly for p = 1, we achieve r = O(d log d), k = O(d log d) and s = O(log d) non-zero entries per column. The best previous construction with s ≤ poly(log d) is due to Woodruff and Zhang (COLT, 2013), giving k = Ω(d²poly(log d)) or [MATH HERE] and r ≥ d · poly(log d); in contrast our r = O(d log d) and k = O(d log d) are optimal up to poly(log(d)) factors even for dense matrices.

We also give (1) nearly-optimal l_p oblivious subspace embeddings with an expected 1 + ε number of non-zero entries per column for arbitrarily small ε > 0, and (2) the first oblivious subspace embeddings for 1 ≤ p < 2 with O(1)-distortion and dimension independent of n. Oblivious subspace embeddings are crucial for distributed and streaming environments, as well as entrywise l_p low rank approximation. Our results give improved algorithms for these applications.

References

[1]

Nir Ailon and Bernard Chazelle. Approximate nearest neighbors and the fast johnson-lindenstrauss transform. In Proceedings of the thirty-eighth annual ACM symposium on Theory of computing, pages 557--563. ACM, 2006.

Abstract

References

Cited By

Recommendations

Tight Bounds for ℓ1 Oblivious Subspace Embeddings

Tight Bounds for Adopt-Commit Objects

Tight bounds for depth-two superconcentrators

Comments

Information

Published In

Sponsors

In-Cooperation

Publisher

Publication History

Check for updates

Qualifiers

Conference

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Share

Share this Publication link

Share on social media

Affiliations

Tight Bounds for ℓ₁ Oblivious Subspace Embeddings