Results 1  10
of
136
On the mathematical foundations of learning
 Bulletin of the American Mathematical Society
, 2002
"... The problem of learning is arguably at the very core of the problem of intelligence, both biological and arti cial. T. Poggio and C.R. Shelton ..."
Abstract

Cited by 329 (12 self)
 Add to MetaCart
(Show Context)
The problem of learning is arguably at the very core of the problem of intelligence, both biological and arti cial. T. Poggio and C.R. Shelton
Data compression and harmonic analysis
 IEEE Trans. Inform. Theory
, 1998
"... In this paper we review some recent interactions between harmonic analysis and data compression. The story goes back of course to Shannon’s R(D) theory... ..."
Abstract

Cited by 177 (23 self)
 Add to MetaCart
In this paper we review some recent interactions between harmonic analysis and data compression. The story goes back of course to Shannon’s R(D) theory...
Sickel: Optimal approximation of elliptic problems by linear and nonlinear mappings III
 Triebel, Function Spaces, Entropy Numbers, Differential Operators
, 1996
"... We study the optimal approximation of the solution of an operator equation A(u) = f by four types of mappings: a) linear mappings of rank n; b) nterm approximation with respect to a Riesz basis; c) approximation based on linear information about the right hand side f; d) continuous mappings. We co ..."
Abstract

Cited by 135 (28 self)
 Add to MetaCart
(Show Context)
We study the optimal approximation of the solution of an operator equation A(u) = f by four types of mappings: a) linear mappings of rank n; b) nterm approximation with respect to a Riesz basis; c) approximation based on linear information about the right hand side f; d) continuous mappings. We consider worst case errors, where f is an element of the unit ball of a Sobolev or Besov space Br q(Lp(Ω)) and Ω ⊂ Rd is a bounded Lipschitz domain; the error is always measured in the Hsnorm. The respective widths are the linear widths (or approximation numbers), the nonlinear widths, the Gelfand widths, and the manifold widths. As a technical tool, we also study the Bernstein numbers. Our main results are the following. If p ≥ 2 then the order of convergence is the same for all four classes of approximations. In particular, the best linear approximations are of the same order as the best nonlinear ones. The best linear approximation can be quite difficult to realize as a numerical algorithm since the optimal Galerkin space usually depends on the operator and of the shape of the domain Ω. For p < 2 there is a difference, nonlinear approximations are better than linear ones. However, in this case, it turns out that linear information about the right hand side f is again optimal. Our main theoretical tool is the best nterm approximation with respect to an optimal Riesz basis and related nonlinear widths. These general results are used to study the Poisson equation in a polygonal domain. It turns out that best nterm wavelet approximation is (almost) optimal. The main results of
Approximation, Metric Entropy and Small Ball Estimates for Gaussian Measures
 Ann. Probab
, 1999
"... A precise link proved by J. Kuelbs and W. V. Li relates the small ball behavior of a Gaussian measure on a Banach space E with the metric entropy behavior of K , the unit ball of the RKHS of in E. We remove the main regularity assumption imposed on the unknown function in the link. This enables t ..."
Abstract

Cited by 63 (27 self)
 Add to MetaCart
A precise link proved by J. Kuelbs and W. V. Li relates the small ball behavior of a Gaussian measure on a Banach space E with the metric entropy behavior of K , the unit ball of the RKHS of in E. We remove the main regularity assumption imposed on the unknown function in the link. This enables the application of tools and results from functional analysis to small ball problems and leads to small ball estimates of general algebraic type as well as to new estimates for concrete Gaussian processes. Moreover, we show that the small ball behavior of a Gaussian process is also tightly connected with the speed of approximation by "nite rank" processes. Abbreviated title: Metric Entropy and Small Ball Estimates Keywords: Gaussian process, small deviation, metric entropy, approximation number. AMS 1991 Subject Classications: Primary: 60G15 ; Secondary: 60F99, 47D50, 47G10 . 1 Supported in part by NSF 1 1 Introduction Let denote a centered Gaussian measure on a real separable B...
Statistical performance of support vector machines
 ANN. STATIST
, 2008
"... The support vector machine (SVM) algorithm is well known to the computer learning community for its very good practical results. The goal of the present paper is to study this algorithm from a statistical perspective, using tools of concentration theory and empirical processes. Our main result build ..."
Abstract

Cited by 62 (10 self)
 Add to MetaCart
(Show Context)
The support vector machine (SVM) algorithm is well known to the computer learning community for its very good practical results. The goal of the present paper is to study this algorithm from a statistical perspective, using tools of concentration theory and empirical processes. Our main result builds on the observation made by other authors that the SVM can be viewed as a statistical regularization procedure. From this point of view, it can also be interpreted as a model selection principle using a penalized criterion. It is then possible to adapt general methods related to model selection in this framework to study two important points: (1) what is the minimum penalty and how does it compare to the penalty actually used in the SVM algorithm; (2) is it possible to obtain “oracle inequalities ” in that setting, for the specific loss function used in the SVM algorithm? We show that the answer to the latter question is positive and provides relevant insight to the former. Our result shows that it is possible to obtain fast rates of convergence for SVMs.
Metric entropy of convex hulls in Banach spaces
 J. London Math. Soc
, 2000
"... The paper presents diverse methods for estimating the covering number of a precompact subset of a Banach space when the entropy of the set of its extremal points is already known. In the case of a Hilbert space, the Gelfand diameters of the subset are also estimated. The Krein–Milman theorem is a po ..."
Abstract

Cited by 30 (1 self)
 Add to MetaCart
(Show Context)
The paper presents diverse methods for estimating the covering number of a precompact subset of a Banach space when the entropy of the set of its extremal points is already known. In the case of a Hilbert space, the Gelfand diameters of the subset are also estimated. The Krein–Milman theorem is a powerful tool in analysis. The aim of this paper is to quantify this theorem in terms of entropy numbers. More precisely, if we have information about the entropy of a precompact set in a Banach or a Hilbert space, then what can be said about the entropy of its convex hull? We give sharp estimates
Smolyak’s algorithm, sampling on sparse grids and Sobolev spaces of dominating mixed smoothness
 East J. Approx., 14:1
"... We investigate the rate of convergence in ‖ · Lp‖, 1 ≤ p ≤ ∞, of the ddimensional Smolyak algorithm, associated to a sequence of sampling operators in the framework of periodic Sobolev and Besov spaces with dominating mixed smoothness. Key Words. Trigonometric interpolation, sampling operators, b ..."
Abstract

Cited by 21 (3 self)
 Add to MetaCart
(Show Context)
We investigate the rate of convergence in ‖ · Lp‖, 1 ≤ p ≤ ∞, of the ddimensional Smolyak algorithm, associated to a sequence of sampling operators in the framework of periodic Sobolev and Besov spaces with dominating mixed smoothness. Key Words. Trigonometric interpolation, sampling operators, blending operators, Smolyak algorithm, rate of convergence, function spaces of dominating mixed smoothness, approximate recovery. AMS subject classifications. 41A25, 41A63, 42B99, 46E35. 1
Entropy and approximation numbers of embeddings of function spaces with Muckenhoupt weights
 I. Rev. Mat. Complut
"... We study compact embeddings for weighted spaces of Besov and TriebelLizorkin type where the weight belongs to some Muckenhoupt Ap class. For weights of purely polynomial growth, both near some singular point and at infinity, we obtain sharp asymptotic estimates for the entropy numbers and approxima ..."
Abstract

Cited by 21 (4 self)
 Add to MetaCart
(Show Context)
We study compact embeddings for weighted spaces of Besov and TriebelLizorkin type where the weight belongs to some Muckenhoupt Ap class. For weights of purely polynomial growth, both near some singular point and at infinity, we obtain sharp asymptotic estimates for the entropy numbers and approximation numbers of this embedding. The main tool is a discretization in terms of wavelet bases. Key words: wavelet bases, Muckenhoupt weighted function spaces, compact embeddings, entropy numbers, approximation numbers.
Optimal Rates for Regularized Least Squares Regression
, 2009
"... We establish a new oracle inequality for kernelbased, regularized least squares regression methods, which uses the eigenvalues of the associated integral operator as a complexity measure. We then use this oracle inequality to derive learning rates for these methods. Here, it turns out that these rat ..."
Abstract

Cited by 19 (3 self)
 Add to MetaCart
(Show Context)
We establish a new oracle inequality for kernelbased, regularized least squares regression methods, which uses the eigenvalues of the associated integral operator as a complexity measure. We then use this oracle inequality to derive learning rates for these methods. Here, it turns out that these rates are independent of the exponent of the regularization term. Finally, we show that our learning rates are asymptotically optimal whenever, e.g., the kernel is continuous and the input space is a compact metric space.
Widths of embeddings in function spaces
, 2007
"... We study the approximation, Gelfand and Kolmogorov numbers of embeddings in function spaces of Besov and TriebelLizorkin type. Our aim here is to provide sharp estimates in several cases left open in the literature and give a complete overview of the known results. We also add some historical remar ..."
Abstract

Cited by 16 (0 self)
 Add to MetaCart
We study the approximation, Gelfand and Kolmogorov numbers of embeddings in function spaces of Besov and TriebelLizorkin type. Our aim here is to provide sharp estimates in several cases left open in the literature and give a complete overview of the known results. We also add some historical remarks.