Results 1 - 10
of
33
A Probabilistic and RIPless Theory of Compressed Sensing
, 2010
"... This paper introduces a simple and very general theory of compressive sensing. In this theory, the sensing mechanism simply selects sensing vectors independently at random from a probability distribution F; it includes all models — e.g. Gaussian, frequency measurements — discussed in the literature, ..."
Abstract
-
Cited by 95 (3 self)
- Add to MetaCart
(Show Context)
This paper introduces a simple and very general theory of compressive sensing. In this theory, the sensing mechanism simply selects sensing vectors independently at random from a probability distribution F; it includes all models — e.g. Gaussian, frequency measurements — discussed in the literature, but also provides a framework for new measurement strategies as well. We prove that if the probability distribution F obeys a simple incoherence property and an isotropy property, one can faithfully recover approximately sparse signals from a minimal number of noisy measurements. The novelty is that our recovery results do not require the restricted isometry property (RIP) — they make use of a much weaker notion — or a random model for the signal. As an example, the paper shows that a signal with s nonzero entries can be faithfully recovered from about s log n Fourier coefficients that are contaminated with noise.
Simple and deterministic matrix sketching
- CoRR
"... We adapt a well known streaming algorithm for approximating item frequencies to the matrix sketching setting. The algorithm receives the rows of a large matrix A ∈ R n×m one after the other in a streaming fashion. For ℓ = ⌈1/ε ⌉ it maintains a sketch matrix B ∈ R ℓ×m such that for any unit vector x ..."
Abstract
-
Cited by 23 (2 self)
- Add to MetaCart
(Show Context)
We adapt a well known streaming algorithm for approximating item frequencies to the matrix sketching setting. The algorithm receives the rows of a large matrix A ∈ R n×m one after the other in a streaming fashion. For ℓ = ⌈1/ε ⌉ it maintains a sketch matrix B ∈ R ℓ×m such that for any unit vector x ‖Ax ‖ 2 ≥ ‖Bx ‖ 2 ≥ ‖Ax ‖ 2 − ε‖A ‖ 2 f. Sketch updates per row in A require amortized O(mℓ) operations. This gives the first algorithm whose error guaranty decreases proportional to 1/ℓ using O(mℓ) space. Prior art algorithms produce bounds proportional to 1 / √ ℓ. Our experiments corroborate that the faster convergence rate is observed in practice. The presented algorithm also stands out in that it is: deterministic, simple to implement, and elementary to prove. Regardless of streaming aspects, the algorithm can be used to compute a 1+ε ′ approximation to the best rank k approximation of any matrix A ∈ R n×m. This requires O(mnℓ ′ ) operations and O(mℓ ′ ) space where ℓ ′ =
von Neumann entropy penalization and low rank matrix approximation.
, 2010
"... Abstract We study a problem of estimation of a Hermitian nonnegatively definite matrix ρ of unit trace (for instance, a density matrix of a quantum system) based on n i.i.d. measurements (X 1 , Y 1 ), . . . , (X n , Y n ), where {X j } being random i.i.d. Hermitian matrices and {ξ j } being i.i.d. ..."
Abstract
-
Cited by 19 (2 self)
- Add to MetaCart
Abstract We study a problem of estimation of a Hermitian nonnegatively definite matrix ρ of unit trace (for instance, a density matrix of a quantum system) based on n i.i.d. measurements (X 1 , Y 1 ), . . . , (X n , Y n ), where {X j } being random i.i.d. Hermitian matrices and {ξ j } being i.i.d. random variables with E(ξ j |X j ) = 0. The estimator is considered, where S is the set of all nonnegatively definite Hermitian m × m matrices of trace 1. The goal is to derive oracle inequalities showing how the estimation error depends on the accuracy of approximation of the unknown state ρ by low-rank matrices.
MATRIX CONCENTRATION INEQUALITIES VIA THE METHOD OF EXCHANGEABLE PAIRS
- SUBMITTED TO THE ANNALS OF PROBABILITY
, 2013
"... This paper derives exponential concentration inequalities and polynomial moment inequalities for the spectral norm of a random matrix. The analysis requires a matrix extension of the scalar concentration theory developed by Sourav Chatterjee using Stein’s method of exchangeable pairs. Whenapplied to ..."
Abstract
-
Cited by 18 (4 self)
- Add to MetaCart
This paper derives exponential concentration inequalities and polynomial moment inequalities for the spectral norm of a random matrix. The analysis requires a matrix extension of the scalar concentration theory developed by Sourav Chatterjee using Stein’s method of exchangeable pairs. Whenapplied toasum of independentrandom matrices, this approach yields matrix generalizations of the classical inequalities due to Hoeffding, Bernstein, Khintchine, and Rosenthal. The same technique delivers bounds for sums of dependent random matrices and more general matrix-valued functions of dependent random variables. This paper is based on two independent manuscripts from mid-2011 that both applied the method of exchangeable pairs to establish matrix concentration inequalities. One manuscript is by Mackey and Jordan; the other is by Chen, Farrell, and Tropp. The authors have combined this research into a single unified presentation, with equal contributions from both groups.
Learning functions of few arbitrary linear parameters in high dimensions
- CoRR
"... Let us assume that f is a continuous function defined on the unit ball of R d, of the form f(x) = g(Ax), where A is a k ×d matrix and g is a function of k variables for k ≪ d. We are given a budget m ∈ N of possible point evaluations f(xi), i = 1,...,m, of f, which we are allowed to query in order ..."
Abstract
-
Cited by 11 (1 self)
- Add to MetaCart
(Show Context)
Let us assume that f is a continuous function defined on the unit ball of R d, of the form f(x) = g(Ax), where A is a k ×d matrix and g is a function of k variables for k ≪ d. We are given a budget m ∈ N of possible point evaluations f(xi), i = 1,...,m, of f, which we are allowed to query in order to construct a uniform approximating function. Under certain smoothness and variation assumptions on the function g, and an arbitrary choice of the matrix A, we present in this paper 1. a sampling choice of the points {xi} drawn at random for each function approximation; 2. algorithms (Algorithm 1 and Algorithm 2) for computing the approximating function, whose complexity is at most polynomial in the dimension d and in the number m of points. Due to the arbitrariness of A, the choice of the sampling points will be according to suitable random distributions and our results hold with overwhelming probability. Our approach uses tools taken from the compressed sensing framework, recent Chernoff bounds for sums of positive-semidefinite matrices, and classical stability bounds for invariant subspaces of singular value decompositions. AMS subject classification (MSC 2010): 65D15, 03D32, 68Q30, 60B20, 60G50
Sharp nonasymptotic bounds on the norm of random matrices with independent entries. Annals of Probability, to appear
, 2015
"... ar ..."
(Show Context)
Subadditivity of matrix ϕ-entropy and concentration of random matrices
- arXiv:1308.2952v1
, 2013
"... ar ..."
Heavy-tailed regression with a generalized median-of-means
- In Proceedings of the 31st International Conference on Machine Learning
, 2014
"... Abstract This work proposes a simple and computationally efficient estimator for linear regression, and other smooth and strongly convex loss minimization problems. We prove loss approximation guarantees that hold for general distributions, including those with heavy tails. All prior results only h ..."
Abstract
-
Cited by 3 (1 self)
- Add to MetaCart
(Show Context)
Abstract This work proposes a simple and computationally efficient estimator for linear regression, and other smooth and strongly convex loss minimization problems. We prove loss approximation guarantees that hold for general distributions, including those with heavy tails. All prior results only hold for estimators which either assume bounded or subgaussian distributions, require prior knowledge of distributional properties, or are not known to be computationally tractable. In the special case of linear regression with possibly heavy-tailed responses and with bounded and well-conditioned covariates in d-dimensions, we show that a random sample of sizeÕ(d log(1/δ)) suffices to obtain a constant factor approximation to the optimal loss with probability 1−δ, a minimax optimal sample complexity up to log factors. The core technique used in the proposed estimator is a new generalization of the median-of-means estimator to arbitrary metric spaces.
