Results 1  10
of
10
Nonparametric Methods for Inference in the Presence of Instrumental Variables
 Annals of Statistics
, 2005
"... We suggest two nonparametric approaches, based on kernel methods and orthogonal series to estimating regression functions in the presence of instrumental variables. For the first time in this class of problems, we derive optimal convergence rates, and show that they are attained by particular estima ..."
Abstract

Cited by 28 (7 self)
 Add to MetaCart
We suggest two nonparametric approaches, based on kernel methods and orthogonal series to estimating regression functions in the presence of instrumental variables. For the first time in this class of problems, we derive optimal convergence rates, and show that they are attained by particular estimators. In the presence of instrumental variables the relation that identifies the regression function also defines an illposed inverse problem, the “difficulty ” of which depends on eigenvalues of a certain integral operator which is determined by the joint density of endogenous and instrumental variables. We delineate the role played by problem difficulty in determining both the optimal convergence rate and the appropriate choice of smoothing parameter. 1. Introduction. Data (Xi,Yi
Support Vector Methods in Learning and Feature Extraction
, 1998
"... The last years have witnessed an increasing interest in Support Vector (SV) machines, which use Mercer kernels for efficiently performing computations in highdimensional spaces. In pattern recognition, the SV algorithm constructs nonlinear decision functions by training a classifier to perform a li ..."
Abstract

Cited by 10 (1 self)
 Add to MetaCart
The last years have witnessed an increasing interest in Support Vector (SV) machines, which use Mercer kernels for efficiently performing computations in highdimensional spaces. In pattern recognition, the SV algorithm constructs nonlinear decision functions by training a classifier to perform a linear separation in some highdimensional space which is nonlinearly related to input space. Recently, we have developed a technique for Nonlinear Principal Component Analysis (Kernel PCA) based on the same types of kernels. This way, we can for instance efficiently extract polynomial features of arbitrary order by computing projections onto principal components in the space of all products of n pixels of images. We explain the idea of Mercer kernels and associated feature spaces, and describe connections to the theory of reproducing kernels and to regularization theory, followed by an overview of the above algorithms employing these kernels. 1. Introduction For the case of twoclass pattern...
Error Bounds for Approximation with Neural Networks
 J. Approx. Theory
, 2000
"... . In this paper we prove convergence rates for the problem of approximating functions f by neural networks and similar constructions. We show that the rates are the better the smoother the activation functions are, provided that f satisfies an integral representation. We give error bounds not only i ..."
Abstract

Cited by 6 (1 self)
 Add to MetaCart
. In this paper we prove convergence rates for the problem of approximating functions f by neural networks and similar constructions. We show that the rates are the better the smoother the activation functions are, provided that f satisfies an integral representation. We give error bounds not only in Hilbert spaces but in general Sobolev spaces W m;r Finally, we apply our results to a class of perceptrons and present a sufficient smoothness condition on f guaranteeing the integral representation. Key Words: Neural networks, error bounds, nonlinear function approximation. AMS Subject Classifications: 41A30, 41A25, 92B20, 68T05 1.
Training Neural Networks with Noisy Data as an IllPosed Problem
 Adv. Comp. Math
, 2000
"... This paper is devoted to the analysis of network approximation in the framework of approximation and regularization theory. It is shown that training neural networks and similar network approximation techniques are equivalent to leastsquares collocation for a corresponding integral equation with mo ..."
Abstract

Cited by 5 (5 self)
 Add to MetaCart
This paper is devoted to the analysis of network approximation in the framework of approximation and regularization theory. It is shown that training neural networks and similar network approximation techniques are equivalent to leastsquares collocation for a corresponding integral equation with mollified data. Results about convergence and convergence rates for exact data are derived based upon wellknown convergence results about leastsquares collocation. Finally, the stability properties with respect to errors in the data are examined and stability bounds are obtained, which yield rules for the choice of the number of network elements. Keywords: illposed problems, leastsquares collocation, neural networks, network training, regularization. AMS Subject Classification: 41A15, 41A30, 45L10, 65J20, 92B20. Short Title: Training Neural Networks with Noisy Data. 1
A class of approximate solutions to linear operator equations, J. Approximation Theory 9
 30332 DEPARTMENT OF STATISTICS
, 1973
"... A certain class of approximate solutions to linear operator equations is studied, in which the domain and range of the operator are both Hilbert spaces possessing continuous reproducing kernels. The broad class of operators considered here includes integral, differential, and integrodifferential ope ..."
Abstract

Cited by 4 (1 self)
 Add to MetaCart
A certain class of approximate solutions to linear operator equations is studied, in which the domain and range of the operator are both Hilbert spaces possessing continuous reproducing kernels. The broad class of operators considered here includes integral, differential, and integrodifferential operators. The results are applied to obtain approximate solutions and related (favorable) convergence rates for twopoint boundaryvalue problems and associated integrodifferential equations. I. SUMMARY We consider a class of approximate solutions to linear operator equations where the domain and range of the operator are both Hilbert spaces possessing continuous reproducing kernels. The (broad) class of operators considered here includes integral, differential, and integrodifferential operators. The specialization to Fredholm integral equations of the first kind has been considered in detail in [5]. The main convergence theorem has been proved there. The purpose of this article is to reformulate the approximate solutions and convergence results of [5] in a more general framework. Then these results are applied to obtain approximate solutions and related convergence rates for twopoint boundary value problems and associated integrodifferential equations. We note that there is an interesting history of the use of reproducing kernel Hilbert spaces to solve problems in approximation theory. See, for example Golomb and Weinberger [2], and, especially, Ciarlet and Varga [l] who consider approximate solutions to differential equations. However, it is
Multivariate Function and Operator Estimation, Based on Smoothing Splines and Reproducing Kernels
"... We review a general approach to multivariate function estimation based on optimal approximation and smoothing in reproducing kernel spaces. Learning with radial basis functions (including thin plate splines) is an important special case. In this context we describe a test for linearity of a function ..."
Abstract
 Add to MetaCart
We review a general approach to multivariate function estimation based on optimal approximation and smoothing in reproducing kernel spaces. Learning with radial basis functions (including thin plate splines) is an important special case. In this context we describe a test for linearity of a functional iteration, and a general class of modelbuilding methods based on sums and products of reproducing kernels. We describe the use of (radial) basis functions in the context of regularization, for use with very large data sets. We generalize some of the results from the estimation of realvalued functions to the estimation of vectorvalued functions. Finally, we generalize from the estimation of vectorvalued functions to the estimation of functionvalued functions on arbitrary index sets, thereby proposing a theory of regularized estimates of nonlinear operators.
Generalization of GMM to a continuum of moment conditions Marine Carrasco
, 1999
"... This paper proposes a version of the Generalized Method of Moments procedure that handles both the case where the number of moment conditions is nite and the case where there is a continuum of moment conditions. Typically, the moment conditions are indexed by an index parameter that takes its values ..."
Abstract
 Add to MetaCart
This paper proposes a version of the Generalized Method of Moments procedure that handles both the case where the number of moment conditions is nite and the case where there is a continuum of moment conditions. Typically, the moment conditions are indexed by an index parameter that takes its values in an interval. The objective function to minimize is then the norm of the moment conditions in a Hilbert space. The estimator is shown to be consistent and asymptotically normal. The optimal estimator is obtained by minimizing the norm of the moment conditions in the reproducing kernel Hilbert space associated with the covariance. We show an easy way to calculate this estimator. Finally, we study properties of a speci cation test using overidentifying restrictions. Results of this paper are useful in many instances where a continuum of moment conditions arise. Examples include e cient estimation of continuous time regression models, cross sectional models that satisfy conditional moment restrictions, as well as scalar di usion processes.
Support Vector Learning Promotionsausschuss: vorgelegt von DiplomPhysiker, M.Sc. (Mathematics)
, 1997
"...  Zum Studium dieser Frage wurde eine neue Form von nichtlinearer Hauptkomponentenanalyse (\Kernel PCA") entwickelt. Durch die Benutzung von Integraloperatorkernen kann in Merkmalsraumen sehr hoher Dimensionalitat (z.B. im 10 10dimensionalen Raum aller Produkte von 5Pixelnin16 16dimensionalen Bild ..."
Abstract
 Add to MetaCart
 Zum Studium dieser Frage wurde eine neue Form von nichtlinearer Hauptkomponentenanalyse (\Kernel PCA") entwickelt. Durch die Benutzung von Integraloperatorkernen kann in Merkmalsraumen sehr hoher Dimensionalitat (z.B. im 10 10dimensionalen Raum aller Produkte von 5Pixelnin16 16dimensionalen Bildern) eine lineare Hauptkomponentenanalyse durchgefuhrt werden. Im Ursprungsraum betrachtet, fuhrt dies zu nichtlinearen Merkmalsextraktoren. Der Algorithmus besteht in der Losung eines Eigenwertproblemes, in dem die Wahl verschiedener Kerne die Verwendung einer gro en Klasse verschiedener Nichtlinearitaten gestattet. Welche der Trainingsmuster enthalten am meisten Information uber die zu konstruierende Entscheidungsfunktion?  Diese Frage, wie auch die folgende, wurde anhand des vor wenigen Jahren von Vapnik vorgeschlagenen \SupportVektorAlgorithmus " innerhalb des von Vapnik und Chervonenkis entwickelten statistischen Paradigmas des Lernens aus Beispielen untersucht. Durch die Wahl verschiedener Integraloperatorkerne ermoglicht dieser Algorithmus die Konstruktion