Results 1  10
of
11
Sensing by Random Convolution
 IEEE Int. Work. on Comp. Adv. MultiSensor Adaptive Proc., CAMPSAP
, 2007
"... Abstract. This paper outlines a new framework for compressive sensing: convolution with a random waveform followed by random time domain subsampling. We show that sensing by random convolution is a universally efficient data acquisition strategy in that an ndimensional signal which is S sparse in a ..."
Abstract

Cited by 64 (4 self)
 Add to MetaCart
Abstract. This paper outlines a new framework for compressive sensing: convolution with a random waveform followed by random time domain subsampling. We show that sensing by random convolution is a universally efficient data acquisition strategy in that an ndimensional signal which is S sparse in any fixed representation can be recovered from m � S log n measurements. We discuss two imaging scenarios — radar and Fourier optics — where convolution with a random pulse allows us to seemingly superresolve finescale features, allowing us to recover highresolution signals from lowresolution measurements. 1. Introduction. The new field of compressive sensing (CS) has given us a fresh look at data acquisition, one of the fundamental tasks in signal processing. The message of this theory can be summarized succinctly [7, 8, 10, 15, 32]: the number of measurements we need to reconstruct a signal depends on its sparsity rather than its bandwidth. These measurements, however, are different than the samples that
Circulant and Toeplitz Matrices in Compressed Sensing
"... Compressed sensing seeks to recover a sparse vector from a small number of linear and nonadaptive measurements. While most work so far focuses on Gaussian or Bernoulli random measurements we investigate the use of partial random circulant and Toeplitz matrices in connection with recovery by ℓ1mini ..."
Abstract

Cited by 32 (9 self)
 Add to MetaCart
Compressed sensing seeks to recover a sparse vector from a small number of linear and nonadaptive measurements. While most work so far focuses on Gaussian or Bernoulli random measurements we investigate the use of partial random circulant and Toeplitz matrices in connection with recovery by ℓ1minization. In contrast to recent work in this direction we allow the use of an arbitrary subset of rows of a circulant and Toeplitz matrix. Our recovery result predicts that the necessary number of measurements to ensure sparse reconstruction by ℓ1minimization with random partial circulant or Toeplitz matrices scales linearly in the sparsity up to a logfactor in the ambient dimension. This represents a significant improvement over previous recovery results for such matrices. As a main tool for the proofs we use a new version of the noncommutative Khintchine inequality.
Statistical properties of kernel principal component analysis
 Machine Learning
, 2004
"... The main goal of this paper is to prove inequalities on the reconstruction error for Kernel Principal Component Analysis. With respect to previous work on this topic, our contribution is twofold: (1) we give bounds that explicitly take into account the empirical centering step in this algorithm, and ..."
Abstract

Cited by 25 (2 self)
 Add to MetaCart
The main goal of this paper is to prove inequalities on the reconstruction error for Kernel Principal Component Analysis. With respect to previous work on this topic, our contribution is twofold: (1) we give bounds that explicitly take into account the empirical centering step in this algorithm, and (2) we show that a “localized” approach allows to obtain more accurate bounds. In particular, we show faster rates of convergence towards the minimum reconstruction error; more precisely, we prove that the convergence rate can typically be faster than n −1/2. We also obtain a new relative bound on the error. A secondary goal, for which we present similar contributions, is to obtain convergence bounds for the partial sums of the biggest or smallest eigenvalues of the kernel Gram matrix towards eigenvalues of the corresponding kernel operator. These quantities are naturally linked to the KPCA procedure; furthermore these results can have applications to the study of various other kernel algorithms. The results are presented in a functional analytic framework, which is suited to deal rigorously with reproducing kernel Hilbert spaces of infinite dimension. 1
Restricted isometries for partial random circulant matrices
 Appl. Comput. Harmon. Anal
"... In the theory of compressed sensing, restricted isometry analysis has become a standard tool for studying how efficiently a measurement matrix acquires information about sparse and compressible signals. Many recovery algorithms are known to succeed when the restricted isometry constants of the sampl ..."
Abstract

Cited by 19 (5 self)
 Add to MetaCart
In the theory of compressed sensing, restricted isometry analysis has become a standard tool for studying how efficiently a measurement matrix acquires information about sparse and compressible signals. Many recovery algorithms are known to succeed when the restricted isometry constants of the sampling matrix are small. Many potential applications of compressed sensing involve a dataacquisition process that proceeds by convolution with a random pulse followed by (nonrandom) subsampling. At present, the theoretical analysis of this measurement technique is lacking. This paper demonstrates that the sth order restricted isometry constant is small when the number m of samples satisfies m � (s log n) 3/2, where n is the length of the pulse. This bound improves on previous estimates, which exhibit quadratic scaling. 1
Sparse channel separation using random probes
, 2010
"... This paper considers the problem of estimating the channel response (or Green’s function) between multiple sourcereceiver pairs. Typically, the channel responses are estimated oneatatime: a single source sends out a known probe signal, the receiver measures the probe signal convolved with the ch ..."
Abstract

Cited by 13 (4 self)
 Add to MetaCart
This paper considers the problem of estimating the channel response (or Green’s function) between multiple sourcereceiver pairs. Typically, the channel responses are estimated oneatatime: a single source sends out a known probe signal, the receiver measures the probe signal convolved with the channel response, and the responses are recovered using deconvolution. In this paper, we show that if the channel responses are sparse and the probe signals are random, then we can significantly reduce the total amount of time required to probe the channels by activating all of the sources simultaneously. With all sources activated simultaneously, the receiver measures a superposition of all the channel responses convolved with the respective probe signals. Separating this cumulative response into individual channel responses can be posed as a linear inverse problem. We show that channel response separation is possible (and stable) even when the probing signals are relatively short in spite of the corresponding linear system of equations becoming severely underdetermined. We derive a theoretical lower bound on the length of the source signals that guarantees that this separation is possible with high probability. The bound is derived by putting the problem in the context of finding a sparse solution to an underdetermined system of equations, and then using mathematical tools from the theory of compressive sensing. Finally, we discuss some practical applications of these results, which include forward modeling for seismic imaging, channel equalization in multipleinput multipleoutput communication, and increasing the fieldofview in an imaging system by using coded apertures.
Kernel Choice and Classifiability for RKHS Embeddings of Probability Distributions
"... Embeddings of probability measures into reproducing kernel Hilbert spaces have been proposed as a straightforward and practical means of representing and comparing probabilities. In particular, the distance between embeddings (the maximum mean discrepancy, or MMD) has several key advantages over man ..."
Abstract

Cited by 10 (7 self)
 Add to MetaCart
Embeddings of probability measures into reproducing kernel Hilbert spaces have been proposed as a straightforward and practical means of representing and comparing probabilities. In particular, the distance between embeddings (the maximum mean discrepancy, or MMD) has several key advantages over many classical metrics on distributions, namely easy computability, fast convergence and low bias of finite sample estimates. An important requirement of the embedding RKHS is that it be characteristic: in this case, the MMD between two distributions is zero if and only if the distributions coincide. Three new results on the MMD are introduced in the present study. First, it is established that MMD corresponds to the optimal risk of a kernel classifier, thus forming a natural link between the distance between distributions and their ease of classification. An important consequence is that a kernel must be characteristic to guarantee classifiability between distributions in the RKHS. Second, the class of characteristic kernels is broadened to incorporate all strictly positive definite kernels: these include nontranslation invariant kernels and kernels on noncompact domains. Third, a generalization of the MMD is proposed for families of kernels, as the supremum over MMDs on a class of kernels (for instance the Gaussian kernels with different bandwidths). This extension is necessary to obtain a single distance measure if a large selection or class of characteristic kernels is potentially appropriate. This generalization is reasonable, given that it corresponds to the problem of learning the kernel by minimizing the risk of the corresponding kernel classifier. The generalized MMD is shown to have consistent finite sample estimates, and its performance is demonstrated on a homogeneity testing example. 1
Improved Approximation Bound for Quadratic Optimization Problems with Orthogonality Constraints
 In Proceedings of the ACMSIAM Symposium on Discrete Algorithms (SODA
, 2009
"... In this paper we consider the problem of approximating a class of quadratic optimization problems that contain orthogonality constraints, i.e. constraints of the form X T X = I, where X ∈ R m×n is the optimization variable. This class of problems, which we denote by (Qp–Oc), is quite general and cap ..."
Abstract

Cited by 3 (0 self)
 Add to MetaCart
In this paper we consider the problem of approximating a class of quadratic optimization problems that contain orthogonality constraints, i.e. constraints of the form X T X = I, where X ∈ R m×n is the optimization variable. This class of problems, which we denote by (Qp–Oc), is quite general and captures several well–studied problems in the literature as special cases. In a recent work, Nemirovski [17] gave the first non–trivial approximation algorithm for (Qp– Oc). His algorithm is based on semidefinite programming and has an approximation guarantee of O (m + n) 1/3. We improve upon this result by providing the first logarithmic approximation guarantee for (Qp–Oc). Specifically, we show that (Qp–Oc) can be approximated to within a factor of O (ln (max{m, n})). The main technical tool used in the analysis is the so–called non–commutative Khintchine inequality, which allows us to prove a concentration inequality for the spectral norm of a Rademacher sum of matrices. As a by–product, we resolve in the affirmative a conjecture of Nemirovski concerning the typical spectral norm of a sum of certain random matrices. The aforementioned concentration inequality also has ramifications in the design of so– called safe tractable approximations of chance constrained optimization problems. In particular, we use it to simplify and improve a recent result of Ben–Tal and Nemirovski [4] concerning certain chance constrained linear matrix inequality systems. 1
SOME REMARKS ON TANGENT MARTINGALE DIFFERENCE SEQUENCES IN L 1SPACES
, 801
"... Abstract. Let X be a Banach space. Suppose that for all p ∈ (1, ∞) a constant Cp,X depending only on X and p exists such that for any two Xvalued martingales f and g with tangent martingale difference sequences one has E‖f ‖ p ≤ Cp,XE‖g ‖ p This property is equivalent to the UMD condition. In fact, ..."
Abstract

Cited by 2 (1 self)
 Add to MetaCart
Abstract. Let X be a Banach space. Suppose that for all p ∈ (1, ∞) a constant Cp,X depending only on X and p exists such that for any two Xvalued martingales f and g with tangent martingale difference sequences one has E‖f ‖ p ≤ Cp,XE‖g ‖ p This property is equivalent to the UMD condition. In fact, it is still equivalent to the UMD condition if in addition one demands that either f or g satisfy the socalled (CI) condition. However, for some applications it suffices to assume that (∗) holds whenever g satisfies the (CI) condition. We show that the class of Banach spaces for which (∗) holds whenever only g satisfies the (CI) condition is more general than the class of UMD spaces, in particular it includes the space L1. We state several problems related to (∗) and other decoupling inequalities. 1.
On ranking and generalization bounds. The
 Journal of Machine Learning Research
"... The problem of ranking is to predict or to guess the ordering between objects on the basis of their observed features. In this paper we consider ranking estimators that minimize the empirical convex risk. We prove generalization bounds for the excess risk of such estimators with rates that are faste ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
The problem of ranking is to predict or to guess the ordering between objects on the basis of their observed features. In this paper we consider ranking estimators that minimize the empirical convex risk. We prove generalization bounds for the excess risk of such estimators with rates that are faster than 1 √ n. We apply our results to commonly used ranking algorithms, for instance boosting or support vector machines. Moreover, we study the performance of considered estimators on real data sets. Keywords: Uprocess
1 Circulant and Toeplitz Matrices in Compressed Sensing
"... Abstract—Compressed sensing seeks to recover a sparse vector from a small number of linear and nonadaptive measurements. While most work so far focuses on Gaussian or Bernoulli random measurements we investigate the use of partial random circulant and Toeplitz matrices in connection with recovery b ..."
Abstract
 Add to MetaCart
Abstract—Compressed sensing seeks to recover a sparse vector from a small number of linear and nonadaptive measurements. While most work so far focuses on Gaussian or Bernoulli random measurements we investigate the use of partial random circulant and Toeplitz matrices in connection with recovery by ℓ1minization. In contrast to recent work in this direction we allow the use of an arbitrary subset of rows of a circulant and Toeplitz matrix. Our recovery result predicts that the necessary number of measurements to ensure sparse reconstruction by ℓ1minimization with random partial circulant or Toeplitz matrices scales linearly in the sparsity up to a logfactor in the ambient dimension. This represents a significant improvement over previous recovery results for such matrices. As a main tool for the proofs we use a new version of the noncommutative Khintchine inequality. I.