Abstract

Recently several results appeared that show significant reduction in time for matrix multiplication, singular value decomposition as well as linear (ℓ2) regression, all based on data dependent random sampling. Our key idea is that low dimensional embeddings can be used to eliminate data dependence and provide more versatile, linear time pass efficient matrix computation. Our main contribution is summarized as follows. • Independent of the recent results of Har-Peled and of Deshpande and Vempala, one of the first – and to the best of our knowledge the most efficient – relative-error (1 + ɛ) ‖A − Ak‖F approximation algorithms for the singular value decomposition of an m × n matrix A with M non-zero entries that requires 2 passes over the data and runs in time O M k + (n + m)k2 ɛ ɛ2) log 1 δ • The first o(nd 2) time (1+ɛ) relative-error approximation algorithm for n×d linear (ℓ2) regression. • A matrix multiplication algorithm that easily applies to implicitly given matrices. 1

Citations

2221	Authoritative Sources in a Hyperlinked Environment - Kleinberg - 1998
693	Matrix multiplication via arithmetic progressions - Coppersmith, Winograd - 1990
570	The space complexity of approximating the frequency moments - Alon, Matias, et al. - 1999
533	Approximate nearest neighbors: Towards removing the curse of dimensionality - Indyk, Motwani - 1998
450	Using Linear Algebra for Intelligent Information Retrieval - Berry, Dumais, et al. - 1995
291	Extensions of Lipschitz mapping into Hilbert space - Johnson, Lindenstrauss - 1984
210	Latent semantic indexing: A probabilistic analysis - Papadimitriou, Tamaki, et al. - 1998
205	Stable distributions, pseudorandom generators, embeddings, and data stream computation - Indyk
197	Finding frequent items in data streams - Charikar, Chen, et al. - 2002
141	Computing on data streams - Hezinger, Raghavan, et al. - 1998
137	Fast Monte-Carlo Algorithms for finding low-rank approximations - Frieze, Kannan, et al.
130	Spectral analysis of data - Azar, Fiat, et al. - 2001
99	M (2006) Fast Monte Carlo algorithms for matrices III: Computing a compressed approximate matrix decomposition - Drineas, Kannan, et al.
91	Software reliability via run-time result-checking - Wasserman, Blum - 1997
89	Tracking join and self-join sizes in limited storage - ALON, GIBBONS, et al. - 2002
80	Database-friendly random projections: Johnson-Lindenstrauss with binary coins - Achlioptas - 2003
74	The Johnson-Lindenstrauss Lemma and the sphericity of some graphs - Frankl, Maehara - 1987
73	Approximate nearest neighbors and the fast Johnson-Lindenstrauss transform - Ailon, Chazelle - 2006
67	An algorithmic theory of learning: Robust concepts and random projection - Arriaga, Vempala - 1999
64	Spectral partitioning of random graphs - McSherry - 2001
53	Clustering large graphs via the singular value decomposition - Drineas, Frieze, et al. - 2004
49	Fast Monte Carlo algorithms for matrices I: Approximating matrix multiplication - DRINEAS, KANNAN, et al. - 2006
46	Matrix approximation and projective clustering via volume sampling - Deshpande, Rademacher, et al. - 2006
43	P.: Competitive recommendation systems - Drineas, Kerenidis, et al. - 2002
43	Sampling from large matrices: An approach through geometric functional analysis - Rudelson, Vershynin - 2007
36	On spectral learning of mixtures of distributions - Achlioptas, McSherry
33	Estimating the largest eigenvalue by the power and Lanczos algorithms with a random start - Kuczyński, Wo´zniakowski - 1992
32	Grouptheoretic algorithms for matrix multiplication - Cohn, Kleinberg, et al.
32	The spectral method for general mixture models - Kannan, Salmasian, et al. - 2005
28	Web search via hub synthesis - Achlioptas, Fiat, et al. - 2001
27	Approximating matrix multiplication for pattern recognition tasks - COHEN, LEWIS - 1999
26	Spectral techniques applied to sparse random graphs, Random Struct - Feige, Ofek
25	The johnson-lindenstrauss lemma meets compressed sensing,” http://www.dsp.rice.edu/cs/jlcs-v03.pdf - Baraniuk, Davenport, et al. - 2006
25	A.: An elementary proof of a theorem of Johnson and Lindenstrauss. Random Struct - Dasgupta, Gupta - 2003
25	Dimensionality Reductions that Preserve Volumes and Distance to Affine Spaces, and their Algorithmic Applications - Magen - 2002
23	Adaptive Sampling and Fast Low-rank Matrix Approximation - Deshpande, Vempala - 2006
22	A Randomized Algorithm for a Tensor-Based Generalization of the Singular Value Decomposition - Drineas, Mahoney - 2005
20	Sampling algorithms for ℓ2 regression and applications - Drineas, Mahoney, et al. - 2006
17	Fast Computation of Low Rank Approximations - Achlioptas, McSherry - 2001
17	Probabilistic machines can use less running time - Freivalds - 1977
14	A fast random sampling algorithm for sparsifying matrices - Arora, Hazan, et al. - 2006
14	Subspace sampling and relativeerror matrix approximation: Column-based methods - Drineas, Mahoney, et al. - 2006
14	To randomize or not to randomize: space optimal summaries for hyperlink analysis - Sarlós, Benczúr, et al. - 2006
13	Sampling lower bounds via information theory - Bar-Yossef - 2003
12	A randomized algorithm for the approximation of matrices - Martinsson, Rokhlin, et al. - 2006
10	Low rank matrix approximation in linear time,” note - Har-Peled - 2006
8	Approximating a gram matrix for improved kernel-based learning - Drineas, Mahoney
7	Variable latent semantic indexing - Dasgupta, Kumar, et al. - 2005
7	Spaces with large distance to ℓ n ∞ and random matrices - Szarek - 1990
1	Spaces with large distance to ℓn ∞ and random matrices - Szarek - 1990