This thesis attempts to quantify the amount of information needed to learn certain tasks. The tasks chosen vary from learning functions in a Sobolev space using radial basis function networks to learning grammars in the principles and parameters framework of modern linguistic theory. These problems are analyzed from the perspective of computational learning theory and certain unifying perspectives emerge. Copyright c fl Massachusetts Institute of Technology, 1996 This report describes research done within the Center for Biological and Computational Learning in the Department of Brain and Cognitive Sciences and at the Artificial Intelligence Laboratory at the Massachusetts Institute of Technology. This research is sponsored by a grant from the National Science Foundation under contract ASC-9217041 (this award includes funds from ARPA provided under the HPCC program); and by a grant from ARPA/ONR under contract N00014-92-J-1879. Additional support has been provided by Siemens Co...

. Radial basis functions are used in the recovery step of finite volume methods for the numerical solution of conservation laws. Being conditionally positive definite such functions generate optimal recovery splines in the sense of Micchelli and Rivlin in associated native spaces. We analyse the solvability to the recovery problem of point functionals from cell average values with radial basis functions. Furthermore, we characterise the corresponding native function spaces and provide error estimates of the recovery scheme. Finally, we explicitly list the native spaces to a selection of radial basis functions, thin plate splines included, before we provide some numerical examples of our method. Contents 1. Introduction 2 2. Finite volume approximations 4 2.1. The governing equations 2.2. Finite volume approximations on triangulations 2.3. Node sets and ENO methods 3. Recovery splines 10 3.1. Radial recovery 3.2. Well-posedness of the recovery problem 4. Error estimates a...

We discuss a method for curve estimation based on n noisy data; one translates the empirical wavelet coefficients towards the origin by an amount p 2 log(n) \Delta oe= p n. The method is nearly minimax for a wide variety of loss functions -- e.g. pointwise error, global error measured in L p norms, pointwise and global error in estimation of derivatives -- and for a wide range of smoothness classes, including standard Holder classes, Sobolev classes, and Bounded Variation. This is a broader near-optimality than anything previously proposed in the minimax literature. The theory underlying the method exploits a correspondence between statistical questions and questions of optimal recovery and information-based complexity. This paper contains a detailed proof of the result announced in Donoho, Johnstone, Kerkyacharian & Picard (1995).

We consider a class of interpolation algorithms, including the least-squares optimal Yen interpolator, and we derive a closed-form expression for the interpolation error for interpolators of this type. The error depends on the eigenvalue distribution of a matrix which is specified for each set of sampling points. The error expression can be used to prove that the Yen interpolator is optimal. The implementation of the Yen algorithm suffers from numerical ill-conditioning, forcing the use of a regularized, approximate solution. We suggest a new, approximate solution, consisting of a sinc-kernel interpolator with specially chosen weighting coefficients. The newly designed sinc-kernel interpolator is compared with the usual sinc interpolator using Jacobian (area) weighting, through numerical simulations. We show that the sinc interpolator with Jacobian weighting works well only when the sampling is nearly uniform. The newly designed sinc-kernel interpolator is shown to perform better than ...

The field of nonparametric function estimation has broadened its appeal in recent years with an array of new tools for statistical analysis. In particular, theoretical and applied research on the field of wavelets has had noticeable influence on statistical topics such as nonparametric regression, nonpararametric density estimation, nonparametric discrimination and many other related topics. This is a survey article that attempts to synthetize a broad variety of work on wavelets in statistics and includes some recent developments in nonparametric curve estimation that have been omitted from review articles and books on the subject. After a short introduction to wavelet theory, wavelets are treated in the familiar context of estimation of "smooth" functions. Both "linear" and "nonlinear" wavelet estimation methods are discussed and cross-validation methods for choosing the smoothing parameters are addressed. Finally, some areas of related research are mentioned, such as hypothesis testi...

Introduction Let H be a real Hilbert space of real--valued functions on some domain\Omega\Gamma and assume that point--evaluation functionals are continuous, which is a very reasonable assumption for applications. Then for any x 2\Omega there is a Riesz representer x 2 H for the point evaluation functional ffi x at x, i.e. ffi x f = f(x) = ( x ; f) H (1.1) for all f 2 H; x 2 \Omega\Gamma Now the function K(x; y) := ( x ; y ) H = y (x) = K(y; x) (1.2) R. Schaback / Translation--Invariant spaces of functions 2 on\Omega<F3

Abstract not found

Abstract. When deriving rates of convergence for the approximations generated by the application of Tikhonov regularization to ill–posed operator equations, assumptions must be made about the nature of the stabilization (i.e., the choice of the seminorm in the Tikhonov regularization) and the regularity of the least squares solutions which one looks for. In fact, it is clear from works of Hegland, Engl and Neubauer and Natterer that, in terms of the rate of convergence, there is a trade–off between stabilization and regularity. It is this matter which is examined in this paper by means of the best–possible worst–error estimates. The results of this paper provide better estimates than those of Engl and Neubauer, and also include and extend the best possible rate derived by Natterer. The paper concludes with an application of these results to first–kind integral equations with smooth kernels. 1.

: This paper formulates the nonlinear system modeling problem as a scattered data interpolation problem, and develops a new method that computes interpolants that minimize a wavelet-based norm subject to interpolatory constraints. The norm is that of a Reproducing Kernel Hilbert Space (RKHS) for which the wavelet functions that form an orthonormal basis for L 2 are orthogonal. In contrast to radial basis function kernels, these kernels are not translation invariant and they may be designed to provide spatially varying resolution useful for interpolating from unevenly distributed data samples. Furthermore, the Author to whom all the correspondences should be addressed. This work was partially supported by the EU grant KIT 124 SYSIDENT and the Wavelets Strategic Research Programme in the National University of Singapore. 2 discrete wavelet transform can be exploited to efficiently compute the values of the interpolant on a uniform grid. Modeling of systems in Sobolev spaces using...

In this paper, the problem of asymptotic identification for fading memory systems in the presence of bounded noise is studied. For any experiment, the worst-case error is characterized in terms of the diameter of the worst-case uncertainty set. Optimal inputs that minimize the radius of uncertainty are studied and characterized. Finally, a convergent algorithm that does not require knowledge of the noise upper bound is furnished. The algorithm is based on interpolating data with spline functions, which are shown to be well suited for identification in the presence of bounded noise; more so than other basis functions such as polynomials.