Abstract
Random projection methods give distributions over k×d matrices such that if a matrix Ψ (chosen according to the distribution) is applied to a finite set of vectors x i ∈ℝd the resulting vectors Ψx i ∈ℝk approximately preserve the original metric with constant probability. First, we show that any matrix (composed with a random ±1 diagonal matrix) is a good random projector for a subset of vectors in ℝd. Second, we describe a family of tensor product matrices which we term Lean Walsh. We show that using Lean Walsh matrices as random projections outperforms, in terms of running time, the best known current result (due to Matousek) under comparable assumptions.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Johnson, W.B., Lindenstrauss, J.: Extensions of Lipschitz mappings into a Hilbert space. Contemp. Math. 26, 189–206 (1984)
Frankl, P., Maehara, H.: The Johnson–Lindenstrauss lemma and the sphericity of some graphs. J. Comb. Theory Ser. A 44, 355–362 (1987)
Indyk, P., Motwani, R.: Approximate nearest neighbors: Towards removing the curse of dimensionality. In: Proceedings of the 30th Annual ACM Symposium on Theory of Computing (STOC), pp. 604–613 (1998)
DasGupta, S., Gupta, A.: An elementary proof of the Johnson–Lindenstrauss lemma. Technical Report, UC Berkeley, 99-006 (1999)
Achlioptas, D.: Database-friendly random projections: Johnson–Lindenstrauss with binary coins. J. Comput. Syst. Sci. 66(4), 671–687 (2003)
Matousek, J.: On variants of the Johnson–Lindenstrauss lemma. Preprint (2006)
Drineas, P., Mahoney, M.W., Muthukrishnan, S.M.: Sampling algorithms for ℓ 2 regression and applications. In: Proceedings of the 17th Annual ACM-SIAM Symposium on Discrete Algorithms (SODA), Miami, Florida, United States (2006)
Sarlós, T.: Improved approximation algorithms for large matrices via random projections. In: Proceedings of the 47th Annual IEEE Symposium on Foundations of Computer Science (FOCS), Berkeley, CA (2006)
Frieze, A.M., Kannan, R., Vempala, S.: Fast Monte-Carlo algorithms for finding low-rank approximations. In: IEEE Symposium on Foundations of Computer Science, pp. 370–378 (1998)
Har-Peled, S.: A replacement for Voronoi diagrams of near linear size. In: Proceedings of the 42nd Annual IEEE Symposium on Foundations of Computer Science (FOCS), Las Vegas, Nevada, USA, pp. 94–103 (2001)
Achlioptas, D., McSherry, F.: Fast computation of low rank matrix approximations. In: STOC: ACM Symposium on Theory of Computing (STOC) (2001)
Drineas, P., Kannan, R.: Fast Monte-Carlo algorithms for approximate matrix multiplication. In: IEEE Symposium on Foundations of Computer Science, pp. 452–459 (2001)
Martinsson, P.G., Rokhlin, V., Tygert, M.: A randomized algorithm for the approximation of matrices. Appl. Comput. Harmon. Anal. 30(1), 47–68 (2011)
Liberty, E., Woolfe, F., Martinsson, P.G., Rokhlin, V., Tygert, M.: Randomized algorithms for the low-rank approximation of matrices. In: Proceedings of the National Academy of Sciences (PNAS), Dec. (2007)
Dasgupta, A., Drineas, P., Harb, B., Kumar, R., Mahoney, M.W.: Sampling algorithms and coresets for ℓ p regression. In: Proc. of the 19th Annual ACM-SIAM Symposium on Discrete Algorithms (SODA) (2008)
Drineas, P., Mahoney, M.W., Muthukrishnan, S., Sarlos, T.: Faster least squares approximation. TR arXiv:0710.1435, submitted for publication (2007)
Drineas, P., Mahoney, M.W., Muthukrishnan, S.: Relative-error CUR matrix decompositions. SIAM J. Matrix Anal. Appl. 30(2), 844–881 (2008)
Candes, E.J., Tao, T.: Near-optimal signal recovery from random projections: Universal encoding strategies? IEEE Trans. Inf. Theory 52(12), 5406–5425 (2006)
Donoho, D.L.: Compressed sensing. IEEE Trans. Inf. Theory 52(4), 1289–1306 (2006)
Elad, M.: Optimized projections for compressed sensing. IEEE Trans. Signal Process. 55(12), 5695–5702 (2007)
Paschou, P., Ziv, E., Burchard, E., Choudhry, S., Rodriguez-Cintron, W., Mahoney, M.W., Drineas, P.: PCA-correlated SNPs for structure identification in worldwide human populations. PLOS Gen. 3, 1672–1686 (2007)
Paschou, P., Mahoney, M.W., Kidd, J., Pakstis, A., Kidd, K., Gu, S., Drineas, P.: Intra- and inter-population genotype reconstruction from tagging SNPs. Genome Res. 17(1), 96–107 (2007)
Liberty, E., Zucker, S.W.: The Mailman algorithm: A note on matrix–vector multiplication. Inf. Process. Lett. 109(3), 179–182 (2009)
Ailon, N., Chazelle, B.: Approximate nearest neighbors and the fast Johnson–Lindenstrauss transform. In: Proceedings of the 38th Annual Symposium on the Theory of Computing (STOC), Seattle, WA, pp. 557–563 (2006)
Ailon, N., Liberty, E.: Fast dimension reduction using Rademacher series on dual BCH codes. Discrete Comput. Geom. 42(4), 615–630 (2000)
Ledoux, M., Talagrand, M.: Probability in Banach Spaces: Isoperimetry and Processes. Springer, Berlin (1991)
Dasgupta, A., Kumar, R., Sarlos, T.: A Sparse Johnson–Lindenstrauss Transform (2010)
Weinberger, K.Q., Dasgupta, A., Langford, J., Smola, A.J., Attenberg, J.: Feature hashing for large scale multitask learning. In: ICML, p. 140 (2009)
Author information
Authors and Affiliations
Corresponding author
Additional information
Edo Liberty was supported by NGA and AFOSR.
Edo Liberty and Amit Singer thank the Institute for Pure and Applied Mathematics (IPAM) and its director Mark Green for their warm hospitality during the fall semester of 2007.
Rights and permissions
About this article
Cite this article
Liberty, E., Ailon, N. & Singer, A. Dense Fast Random Projections and Lean Walsh Transforms. Discrete Comput Geom 45, 34–44 (2011). https://doi.org/10.1007/s00454-010-9309-5
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00454-010-9309-5