Results 1  10
of
33
A Probabilistic and RIPless Theory of Compressed Sensing
, 2010
"... This paper introduces a simple and very general theory of compressive sensing. In this theory, the sensing mechanism simply selects sensing vectors independently at random from a probability distribution F; it includes all models — e.g. Gaussian, frequency measurements — discussed in the literature, ..."
Abstract

Cited by 95 (3 self)
 Add to MetaCart
(Show Context)
This paper introduces a simple and very general theory of compressive sensing. In this theory, the sensing mechanism simply selects sensing vectors independently at random from a probability distribution F; it includes all models — e.g. Gaussian, frequency measurements — discussed in the literature, but also provides a framework for new measurement strategies as well. We prove that if the probability distribution F obeys a simple incoherence property and an isotropy property, one can faithfully recover approximately sparse signals from a minimal number of noisy measurements. The novelty is that our recovery results do not require the restricted isometry property (RIP) — they make use of a much weaker notion — or a random model for the signal. As an example, the paper shows that a signal with s nonzero entries can be faithfully recovered from about s log n Fourier coefficients that are contaminated with noise.
Simple and deterministic matrix sketching
 CoRR
"... We adapt a well known streaming algorithm for approximating item frequencies to the matrix sketching setting. The algorithm receives the rows of a large matrix A ∈ R n×m one after the other in a streaming fashion. For ℓ = ⌈1/ε ⌉ it maintains a sketch matrix B ∈ R ℓ×m such that for any unit vector x ..."
Abstract

Cited by 23 (2 self)
 Add to MetaCart
(Show Context)
We adapt a well known streaming algorithm for approximating item frequencies to the matrix sketching setting. The algorithm receives the rows of a large matrix A ∈ R n×m one after the other in a streaming fashion. For ℓ = ⌈1/ε ⌉ it maintains a sketch matrix B ∈ R ℓ×m such that for any unit vector x ‖Ax ‖ 2 ≥ ‖Bx ‖ 2 ≥ ‖Ax ‖ 2 − ε‖A ‖ 2 f. Sketch updates per row in A require amortized O(mℓ) operations. This gives the first algorithm whose error guaranty decreases proportional to 1/ℓ using O(mℓ) space. Prior art algorithms produce bounds proportional to 1 / √ ℓ. Our experiments corroborate that the faster convergence rate is observed in practice. The presented algorithm also stands out in that it is: deterministic, simple to implement, and elementary to prove. Regardless of streaming aspects, the algorithm can be used to compute a 1+ε ′ approximation to the best rank k approximation of any matrix A ∈ R n×m. This requires O(mnℓ ′ ) operations and O(mℓ ′ ) space where ℓ ′ =
von Neumann entropy penalization and low rank matrix approximation.
, 2010
"... Abstract We study a problem of estimation of a Hermitian nonnegatively definite matrix ρ of unit trace (for instance, a density matrix of a quantum system) based on n i.i.d. measurements (X 1 , Y 1 ), . . . , (X n , Y n ), where {X j } being random i.i.d. Hermitian matrices and {ξ j } being i.i.d. ..."
Abstract

Cited by 19 (2 self)
 Add to MetaCart
Abstract We study a problem of estimation of a Hermitian nonnegatively definite matrix ρ of unit trace (for instance, a density matrix of a quantum system) based on n i.i.d. measurements (X 1 , Y 1 ), . . . , (X n , Y n ), where {X j } being random i.i.d. Hermitian matrices and {ξ j } being i.i.d. random variables with E(ξ j X j ) = 0. The estimator is considered, where S is the set of all nonnegatively definite Hermitian m × m matrices of trace 1. The goal is to derive oracle inequalities showing how the estimation error depends on the accuracy of approximation of the unknown state ρ by lowrank matrices.
MATRIX CONCENTRATION INEQUALITIES VIA THE METHOD OF EXCHANGEABLE PAIRS
 SUBMITTED TO THE ANNALS OF PROBABILITY
, 2013
"... This paper derives exponential concentration inequalities and polynomial moment inequalities for the spectral norm of a random matrix. The analysis requires a matrix extension of the scalar concentration theory developed by Sourav Chatterjee using Stein’s method of exchangeable pairs. Whenapplied to ..."
Abstract

Cited by 18 (4 self)
 Add to MetaCart
This paper derives exponential concentration inequalities and polynomial moment inequalities for the spectral norm of a random matrix. The analysis requires a matrix extension of the scalar concentration theory developed by Sourav Chatterjee using Stein’s method of exchangeable pairs. Whenapplied toasum of independentrandom matrices, this approach yields matrix generalizations of the classical inequalities due to Hoeffding, Bernstein, Khintchine, and Rosenthal. The same technique delivers bounds for sums of dependent random matrices and more general matrixvalued functions of dependent random variables. This paper is based on two independent manuscripts from mid2011 that both applied the method of exchangeable pairs to establish matrix concentration inequalities. One manuscript is by Mackey and Jordan; the other is by Chen, Farrell, and Tropp. The authors have combined this research into a single unified presentation, with equal contributions from both groups.
Learning functions of few arbitrary linear parameters in high dimensions
 CoRR
"... Let us assume that f is a continuous function defined on the unit ball of R d, of the form f(x) = g(Ax), where A is a k ×d matrix and g is a function of k variables for k ≪ d. We are given a budget m ∈ N of possible point evaluations f(xi), i = 1,...,m, of f, which we are allowed to query in order ..."
Abstract

Cited by 11 (1 self)
 Add to MetaCart
(Show Context)
Let us assume that f is a continuous function defined on the unit ball of R d, of the form f(x) = g(Ax), where A is a k ×d matrix and g is a function of k variables for k ≪ d. We are given a budget m ∈ N of possible point evaluations f(xi), i = 1,...,m, of f, which we are allowed to query in order to construct a uniform approximating function. Under certain smoothness and variation assumptions on the function g, and an arbitrary choice of the matrix A, we present in this paper 1. a sampling choice of the points {xi} drawn at random for each function approximation; 2. algorithms (Algorithm 1 and Algorithm 2) for computing the approximating function, whose complexity is at most polynomial in the dimension d and in the number m of points. Due to the arbitrariness of A, the choice of the sampling points will be according to suitable random distributions and our results hold with overwhelming probability. Our approach uses tools taken from the compressed sensing framework, recent Chernoff bounds for sums of positivesemidefinite matrices, and classical stability bounds for invariant subspaces of singular value decompositions. AMS subject classification (MSC 2010): 65D15, 03D32, 68Q30, 60B20, 60G50
Sharp nonasymptotic bounds on the norm of random matrices with independent entries. Annals of Probability, to appear
, 2015
"... ar ..."
(Show Context)
Subadditivity of matrix ϕentropy and concentration of random matrices
 arXiv:1308.2952v1
, 2013
"... ar ..."
Heavytailed regression with a generalized medianofmeans
 In Proceedings of the 31st International Conference on Machine Learning
, 2014
"... Abstract This work proposes a simple and computationally efficient estimator for linear regression, and other smooth and strongly convex loss minimization problems. We prove loss approximation guarantees that hold for general distributions, including those with heavy tails. All prior results only h ..."
Abstract

Cited by 3 (1 self)
 Add to MetaCart
(Show Context)
Abstract This work proposes a simple and computationally efficient estimator for linear regression, and other smooth and strongly convex loss minimization problems. We prove loss approximation guarantees that hold for general distributions, including those with heavy tails. All prior results only hold for estimators which either assume bounded or subgaussian distributions, require prior knowledge of distributional properties, or are not known to be computationally tractable. In the special case of linear regression with possibly heavytailed responses and with bounded and wellconditioned covariates in ddimensions, we show that a random sample of sizeÕ(d log(1/δ)) suffices to obtain a constant factor approximation to the optimal loss with probability 1−δ, a minimax optimal sample complexity up to log factors. The core technique used in the proposed estimator is a new generalization of the medianofmeans estimator to arbitrary metric spaces.