Results 1 
9 of
9
FAST COMPRESSIVE SAMPLING WITH STRUCTURALLY RANDOM MATRICES
"... This paper presents a novel framework of fast and efficient compressive sampling based on the new concept of structurally random matrices. The proposed framework provides four important features. (i) It is universal with a variety of sparse signals. (ii) The number of measurements required for exact ..."
Abstract

Cited by 19 (6 self)
 Add to MetaCart
This paper presents a novel framework of fast and efficient compressive sampling based on the new concept of structurally random matrices. The proposed framework provides four important features. (i) It is universal with a variety of sparse signals. (ii) The number of measurements required for exact reconstruction is nearly optimal. (iii) It has very low complexity and fast computation based on block processing and linear filtering. (iv) It is developed on the provable mathematical model from which we are able to quantify tradeoffs among streaming capability, computation/memory requirement and quality of reconstruction. All currently existing methods only have at most three out of these four highly desired features. Simulation results with several interesting structurally random matrices under various practical settings are also presented to verify the validity of the theory as well as to illustrate the promising potential of the proposed framework. Index Terms — Fast compressive sampling, random projections, nonlinear reconstruction, structurally random matrices 1.
Concentration of Measure Inequalities for Compressive Toeplitz Matrices with Applications to Detection and System Identification
"... Abstract — In this paper, we derive concentration of measure inequalities for compressive Toeplitz matrices (having fewer rows than columns) with entries drawn from an independent and identically distributed (i.i.d.) Gaussian random sequence. These inequalities show that the norm of a vector mapped ..."
Abstract

Cited by 9 (8 self)
 Add to MetaCart
Abstract — In this paper, we derive concentration of measure inequalities for compressive Toeplitz matrices (having fewer rows than columns) with entries drawn from an independent and identically distributed (i.i.d.) Gaussian random sequence. These inequalities show that the norm of a vector mapped by a Toeplitz matrix to a lower dimensional space concentrates around its mean with a tail probability bound that decays exponentially in the dimension of the range space divided by a factor that is a function of the sample covariance of the vector. Motivated by the emerging field of Compressive Sensing (CS), we apply these inequalities to problems involving the analysis of highdimensional systems from convolutionbased compressive measurements. We discuss applications such as system identification, namely the estimation of the impulse response of a system, in cases where one can assume that the impulse response is highdimensional, but sparse. We also consider the problem of detecting a change in the dynamic behavior of a system, where the change itself can be modeled by a system with a sparse impulse response. I.
VC Theory of Large Margin MultiCategory Classifiers
"... In the context of discriminant analysis, Vapnik’s statistical learning theory has mainly been developed in three directions: the computation of dichotomies with binaryvalued functions, the computation of dichotomies with realvalued functions, and the computation of polytomies with functions taking ..."
Abstract

Cited by 7 (4 self)
 Add to MetaCart
In the context of discriminant analysis, Vapnik’s statistical learning theory has mainly been developed in three directions: the computation of dichotomies with binaryvalued functions, the computation of dichotomies with realvalued functions, and the computation of polytomies with functions taking their values in finite sets, typically the set of categories itself. The case of classes of vectorvalued functions used to compute polytomies has seldom been considered independently, which is unsatisfactory, for three main reasons. First, this case encompasses the other ones. Second, it cannot be treated appropriately through a naïve extension of the results devoted to the computation of dichotomies. Third, most of the classification problems met in practice involve multiple categories. In this paper, a VC theory of large margin multicategory classifiers is introduced. Central in this theory are generalized VC dimensions called the γΨdimensions. First, a uniform convergence bound on the risk of the classifiers of interest is derived. The capacity measure involved in this bound is a covering number. This covering number can be upper bounded in terms of the γΨdimensions thanks to generalizations of Sauer’s lemma, as is illustrated in the specific case of the scalesensitive Natarajan dimension. A bound on this latter dimension is then computed for the class of functions on which multiclass SVMs are based. This makes it possible to apply the structural risk minimization inductive principle to those machines.
FAST AND EFFICIENT DIMENSIONALITY REDUCTION USING STRUCTURALLY RANDOM MATRICES
"... Structurally Random Matrices (SRM) are first proposed in [1] as fast and highly efficient measurement operators for large scale compressed sensing applications. Motivated by the bridge between compressed sensing and the JohnsonLindenstrauss lemma [2] , this paper introduces a related application of ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
Structurally Random Matrices (SRM) are first proposed in [1] as fast and highly efficient measurement operators for large scale compressed sensing applications. Motivated by the bridge between compressed sensing and the JohnsonLindenstrauss lemma [2] , this paper introduces a related application of SRMs regarding to realizing a fast and highly efficient embedding. In particular, it shows that a SRM is also a promising dimensionality reduction transform that preserves all pairwise distances of high dimensional vectors within an arbitrarily small factor ɛ, provided that the projection dimension is on the order of O(ɛ −2 log 3 N), where N denotes the number of ddimensional vectors. In other words, SRM can be viewed as the suboptimal JohnsonLindenstrauss embedding that, however, owns very low computational complexity O(d log d) and highly efficient implementation that uses only O(d) random bits, making it a promising candidate for practical, large scale applications where efficiency and speed of computation are highly critical. Index Terms — Lowdistortion embedding, JohnsonLindenstrauss, dimensionality reduction, compressed sensing, machine learning.
ARXIV Version manuscript No. (will be inserted by the editor) The Optimal Unbiased Value Estimator and its Relation
, 908
"... Abstract In this analytical study we derive the optimal unbiased value estimator (MVU) and compare its statistical risk to three well known value estimators: Temporal Difference learning (TD), Monte Carlo estimation (MC) and LeastSquares Temporal Difference Learning (LSTD). We demonstrate that LSTD ..."
Abstract
 Add to MetaCart
Abstract In this analytical study we derive the optimal unbiased value estimator (MVU) and compare its statistical risk to three well known value estimators: Temporal Difference learning (TD), Monte Carlo estimation (MC) and LeastSquares Temporal Difference Learning (LSTD). We demonstrate that LSTD is equivalent to the MVU if the Markov Reward Process (MRP) is acyclic and show that both differ for most cyclic MRPs as LSTD is then typically biased. More generally, we show that estimators that fulfill the Bellman equation can only be unbiased for special cyclic MRPs. The main reason being the probability measures with which the expectations are taken. These measure vary from state to state and due to the strong coupling by the Bellman equation it is typically not possible for a set of value estimators to be unbiased with respect to each of these measures. Furthermore, we derive relations of the MVU to MC and TD. The most important one being the equivalence of MC to the MVU and to LSTD for undiscounted MRPs in which MC has the same amount of information. In the discounted case this equivalence does not hold anymore. For TD we show that it is essentially unbiased for acyclic MRPs and biased for cyclic MRPs. We also order estimators according to their risk and present counterexamples to show that no general ordering exists between the MVU and LSTD, between MC and LSTD and between TD and MC. Theoretical results are supported by examples and an empirical evaluation.
A note on concentration of submodular functions
, 2010
"... We survey a few concentration inequalities for submodular and fractionally subadditive functions of independent random variables, implied by the entropy method for selfbounding functions. The power of these concentration bounds is that they are dimensionfree, in particular implying standard deviat ..."
Abstract
 Add to MetaCart
We survey a few concentration inequalities for submodular and fractionally subadditive functions of independent random variables, implied by the entropy method for selfbounding functions. The power of these concentration bounds is that they are dimensionfree, in particular implying standard deviation O ( p E[f]) rather than O ( √ n) which can be obtained for any 1Lipschitz function of n variables. 1
UNCERTAINTY QUANTIFICATION FOR MODULAR AND HIERARCHICAL MODELS ∗
"... Abstract. We propose a modular/hierarchical uncertainty quantification framework based on a recently developed methodology using concentrationofmeasure inequalities for probabilityoffailure upper bound calculations. In this framework, the relations between the variables of the underlying inputo ..."
Abstract
 Add to MetaCart
Abstract. We propose a modular/hierarchical uncertainty quantification framework based on a recently developed methodology using concentrationofmeasure inequalities for probabilityoffailure upper bound calculations. In this framework, the relations between the variables of the underlying inputoutput model are represented by directed, acyclic graphs and the bounded uncertainty in the input variables is propagated to the output variable (performance measure) in an inductive manner respecting this underlying graph structure. The proposed methodology offers reductions in computational complexity, especially when there is a modularity in the relations between the inputs and the output where relatively smaller modules with strong withinmodule dependencies have weak intermodule coupling, by assembling a measure of uncertainty based on the measures for smaller subsystems. It also provides practical advantages by enabling independent analysis of the modules as well as repeated usage of the uncertainty quantification results for the modules in different context/systems. We demonstrate the methodology on the analysis of a LC electrical circuit. Key words. AMS subject classifications.