Abstract not found

We suggest two nonparametric approaches, based on kernel methods and orthogonal series to estimating regression functions in the presence of instrumental variables. For the first time in this class of problems, we derive optimal convergence rates, and show that they are attained by particular estimators. In the presence of instrumental variables the relation that identifies the regression function also defines an ill-posed inverse problem, the “difficulty ” of which depends on eigenvalues of a certain integral operator which is determined by the joint density of endogenous and instrumental variables. We delineate the role played by problem difficulty in determining both the optimal convergence rate and the appropriate choice of smoothing parameter. 1. Introduction. Data (Xi,Yi

The last years have witnessed an increasing interest in Support Vector (SV) machines, which use Mercer kernels for efficiently performing computations in high-dimensional spaces. In pattern recognition, the SV algorithm constructs nonlinear decision functions by training a classifier to perform a linear separation in some high-dimensional space which is nonlinearly related to input space. Recently, we have developed a technique for Nonlinear Principal Component Analysis (Kernel PCA) based on the same types of kernels. This way, we can for instance efficiently extract polynomial features of arbitrary order by computing projections onto principal components in the space of all products of n pixels of images. We explain the idea of Mercer kernels and associated feature spaces, and describe connections to the theory of reproducing kernels and to regularization theory, followed by an overview of the above algorithms employing these kernels. 1. Introduction For the case of two-class pattern...

Contents Preface vii 1 Introduction to Large Margin Classifiers 1 Alex J. Smola, Peter Bartlett, Bernhard Scholkopf, and Dale Schuurmans 2 Large Margin Rank Boundaries for Ordinal Regression 29 Ralf Herbrich, Thore Graepel, and Klaus Obermayer References 46 Smola, Bartlett, Scholkopf, and Schuurmans: Advances in Large Margin Classifiers 1999/07/12 09:57 Preface Some good quote who knows some clever stuff ... and some more visionary comments Alexander J. Smola, Peter Bartlett, Bernhard Scholkopf, Dale Schuurmans Berlin, Canberra, Waterloo, July 1999 Smola, Bartlett, Scholkopf, and Schuurmans: Advances in Large Margin Classifiers 1999/07/12 09:57 1 Introduction to Large Margin Classifiers The aim of this chapter is to provide a brief introduction to the basic concepts of large margin classifiers for readers unfamiliar with the topic. Moreover it is aimed at establishing a common ba

. In this paper we prove convergence rates for the problem of approximating functions f by neural networks and similar constructions. We show that the rates are the better the smoother the activation functions are, provided that f satisfies an integral representation. We give error bounds not only in Hilbert spaces but in general Sobolev spaces W m;r Finally, we apply our results to a class of perceptrons and present a sufficient smoothness condition on f guaranteeing the integral representation. Key Words: Neural networks, error bounds, nonlinear function approximation. AMS Subject Classifications: 41A30, 41A25, 92B20, 68T05 1.

This paper is devoted to the analysis of network approximation in the framework of approximation and regularization theory. It is shown that training neural networks and similar network approximation techniques are equivalent to least-squares collocation for a corresponding integral equation with mollified data. Results about convergence and convergence rates for exact data are derived based upon well-known convergence results about least-squares collocation. Finally, the stability properties with respect to errors in the data are examined and stability bounds are obtained, which yield rules for the choice of the number of network elements. Keywords: ill-posed problems, least-squares collocation, neural networks, network training, regularization. AMS Subject Classification: 41A15, 41A30, 45L10, 65J20, 92B20. Short Title: Training Neural Networks with Noisy Data. 1