Results 1  10
of
18
Bounds on the sample complexity of Bayesian learning using information theory
 and the VC dimension,” in Proc. Conf. Comp. Learning Theory
, 1991
"... ..."
Compressive Sensing and Structured Random Matrices
 RADON SERIES COMP. APPL. MATH XX, 1–95 © DE GRUYTER 20YY
"... These notes give a mathematical introduction to compressive sensing focusing on recovery using ℓ1minimization and structured random matrices. An emphasis is put on techniques for proving probabilistic estimates for condition numbers of structured random matrices. Estimates of this type are key to ..."
Abstract

Cited by 94 (15 self)
 Add to MetaCart
These notes give a mathematical introduction to compressive sensing focusing on recovery using ℓ1minimization and structured random matrices. An emphasis is put on techniques for proving probabilistic estimates for condition numbers of structured random matrices. Estimates of this type are key to providing conditions that ensure exact or approximate recovery of sparse vectors using ℓ1minimization.
Uniform Ratio Limit Theorems for Empirical Processes
, 1995
"... . Analysis of nonparametric estimators is easier if one makes use of refined empirical process inequalities. These inequalities have not been widely appreciated or applied, perhaps because they lie buried quite deep in the empirical process literature. This paper seeks to explain a novel empirical p ..."
Abstract

Cited by 11 (0 self)
 Add to MetaCart
. Analysis of nonparametric estimators is easier if one makes use of refined empirical process inequalities. These inequalities have not been widely appreciated or applied, perhaps because they lie buried quite deep in the empirical process literature. This paper seeks to explain a novel empirical process technique that might make the results more accessible. A simple method is presented for proving maximal inequalities for empirical processes indexed by unbounded classes of functions. The main result is stated in the form of a tail bound for the ratio of a centered empirical measure empirical to a sum of various L 1 norms. The method avoids the usual chaining argument. The bound delivers the correct rates of convergence for the sorts of quantities encountered in nonparametric smoothing applications. Research partially supported by NSF Grant DMS9102286 Keywords and Phrases: Maximal inequalities for empirical process; ratio limit theorems; nonparametric bounds. Running head: Empirical...
DistributionDependent VapnikChervonenkis Bounds
, 1999
"... . VapnikChervonenkis (VC) bounds play an important role in statistical learning theory as they are the fundamental result which explains the generalization ability of learning machines. There have been consequent mathematical works on the improvement of VC rates of convergence of empirical mean ..."
Abstract

Cited by 6 (1 self)
 Add to MetaCart
. VapnikChervonenkis (VC) bounds play an important role in statistical learning theory as they are the fundamental result which explains the generalization ability of learning machines. There have been consequent mathematical works on the improvement of VC rates of convergence of empirical means to their expectations over the years. The result obtained by Talagrand in 1994 seems to provide more or less the final word to this issue as far as universal bounds are concerned. Though for fixed distributions, this bound can be practically outperformed. We show indeed that it is possible to replace the 2ffl 2 under the exponential of the deviation term by the corresponding Cram'er transform as shown by large deviations theorems. Then, we formulate rigorous distributionsensitive VC bounds and we also explain why these theoretical results on such bounds can lead to practical estimates of the effective VC dimension of learning structures. 1 Introduction and motivations One of t...
ON THE LIMITING DISTRIBUTIONS OF MULTIVARIATE DEPTHBASED RANK SUM STATISTICS AND RELATED TESTS
, 708
"... A depthbased rank sum statistic for multivariate data introduced ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
(Show Context)
A depthbased rank sum statistic for multivariate data introduced
Maximal Inequalities via Bracketing with Adaptive Truncation, Annales de l’Institut Henri Poincaré 38
, 2002
"... Abstract. The paper provides a recursive interpretation for the technique known as bracketing with adaptive truncation. By way of illustration, a simple bound is derived for the expected value of the supremum of an empirical process, thereby leading to a simpler derivation of a functional central l ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
Abstract. The paper provides a recursive interpretation for the technique known as bracketing with adaptive truncation. By way of illustration, a simple bound is derived for the expected value of the supremum of an empirical process, thereby leading to a simpler derivation of a functional central limit limit due to Ossiander. The recursive method is also abstracted into a framework that consists of only a small number of assumptions about processes and functionals indexed by sets of functions. In particular, the details of the underlying probability model are condensed into a single inequality involving finite sets of functions. A functional central limit theorem of Doukhan, Massart and Rio, for empirical processes defined by absolutely regular sequences, motivates the generalization. 1.
Learning Using Information Theory and the VC Dimension
"... Abstract. In this paper we study a Bayesian or averagecase model of concept learning with a twofold goal: to provide more precise characterizations of learning curve (sample complexity) behavior that depend on properties of both the prior distribution over concepts and the sequence of instances see ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
Abstract. In this paper we study a Bayesian or averagecase model of concept learning with a twofold goal: to provide more precise characterizations of learning curve (sample complexity) behavior that depend on properties of both the prior distribution over concepts and the sequence of instances seen by the learner, and to smoothly unite in a common framework the popular statistical physics and VC dimension theories of learning curves. To achieve this, we undertake a systematic investigation and comparison of two fundamental quantities in learning and information theory: the probability of an incorrect prediction for an optimal learning algorithm, and the Shannon information gain. This study leads to a new understanding of the sample complexity of learning in several existing models.
Large Deviations Bounds for Empirical Processes
, 1999
"... VapnikChervonenkis bounds on speeds of convergence of empirical means to their expectations have been continuously improved over the years. The result obtained by M. Talagrand in 1994 [11] seems to provide the final word as far as universal bounds are concerned. However, for fixed families of under ..."
Abstract
 Add to MetaCart
VapnikChervonenkis bounds on speeds of convergence of empirical means to their expectations have been continuously improved over the years. The result obtained by M. Talagrand in 1994 [11] seems to provide the final word as far as universal bounds are concerned. However, for fixed families of underlying probability distributions, the exponential rate in the deviation term can be fairly improved by the more adequate Cramer transform, as shown by large deviations theorems.
Exact Rates In VapnikChervonenkis Bounds
"... VapnikChervonenkis bounds on speeds of uniform convergence of empirical means to their expectations have been continuously improved over the years since the precursory work in [26]. The result obtained by Talagrand in 1994 [21] seems to provide the final word as far as universal bounds are concerne ..."
Abstract
 Add to MetaCart
VapnikChervonenkis bounds on speeds of uniform convergence of empirical means to their expectations have been continuously improved over the years since the precursory work in [26]. The result obtained by Talagrand in 1994 [21] seems to provide the final word as far as universal bounds are concerned. However, in the case where there are some additional assumptions on the underlying probability distribution, the exponential rate of convergence can be fairly improved. Alexander [1] and Massart [15] have found better exponential rates (similar to those in BennettBernstein inequalities) under the assumption of a control on the variance of the empirical process. In this paper, the case of a particular distribution is considered for the empirical process indexed by a family of sets, and we provide the exact exponential rate based on large deviations theorems, as predicted by Azencott [2].
1 ON CONSISTENCY OF KERNEL DENSITY ESTIMATORS FOR RANDOMLY CENSORED DATA: RATES HOLDING UNIFORMLY OVER ADAPTIVE INTERVALS
, 1999
"... In the usual rightcensored data situation, let fn, n∈N, denote the convolution of the KaplanMeier product limit estimator with the kernels a −1 n K(·/an), where K is a smooth probability density with bounded support and an→0. That is, fn is the usual kernel density estimator based on KaplanMeier. ..."
Abstract
 Add to MetaCart
In the usual rightcensored data situation, let fn, n∈N, denote the convolution of the KaplanMeier product limit estimator with the kernels a −1 n K(·/an), where K is a smooth probability density with bounded support and an→0. That is, fn is the usual kernel density estimator based on KaplanMeier. Let ¯ fn denote the convolution of the distribution of the uncensored data, which is assumed to have a bounded density, with the same kernels. For each n, let Jn denote the half line with right end point Zn,n(1−εn)−an, where εn→0 and, for each m, Zn,m is the mth order statistic of the censored data. It is shown that, under some mild conditions on an and εn, supJn fn(t) − ¯ fn(t)  converges a.s. to zero as n→ ∞ at least as fast as √  log(an∧εn)/(nanεn). For εn=constant, this rate compares, up to constants, with the exact rate for fixed intervals.