Abstract not found

We consider the solution of binary classification problems via Tikhonov regularization in a Reproducing Kernel Hilbert Space using the square loss, and denote the resulting algorithm Regularized Least-Squares Classification (RLSC). We sketch

We present a non-linear kernel-based version of the Recursive Least Squares (RLS) algorithm. Our Kernel-RLS (KRLS) algorithm performs linear regression in the feature space induced by a Mercer kernel, and can therefore be used to recursively construct the minimum mean squared -error regressor. Sparsity of the solution is achieved by a sequential sparsification process that admits into the kernel representation a new input sample only if its feature space image cannot be suffciently well approximated by combining the images of previously admitted samples. This sparsification procedure is crucial to the operation of KRLS, as it allows it to operate on-line, and by effectively regularizing its solutions. A theoretical analysis of the sparsification method reveals its close affinity to kernel PCA, and a data-dependent loss bound is presented, quantifying the generalization performance of the KRLS algorithm. We demonstrate the performance and scaling properties of KRLS and compare it to a stateof -the-art Support Vector Regression algorithm, using both synthetic and real data. We additionally test KRLS on two signal processing problems in which the use of traditional least-squares methods is commonplace: Time series prediction and channel equalization.

We introduce a new notion of algorithmic stability, which we call training stability.

Abstract. We investigate the problem of model selection for learning algorithms depending on a continuous parameter. We propose a model selection procedure based on a worst case analysis and data-independent choice of the parameter. For regularized least-squares algorithm we bound the generalization error of the solution by a quantity depending on few known constants and we show that the corresponding model selection procedure reduces to solving a bias-variance problem. Under suitable smoothness condition on the regression function, we estimate the optimal parameter as function of the number of data and we prove that this choice ensures consistency of the algorithm. 1.

This report on learning theory is written in the spirit of: The best understanding of what one can see comes from theories of what one can’t see. This thought has been expressed in a number of ways by different scientists, and is supported everywhere. Obvious choices vary from gravity to economic equilibrium. For

Using a recently introduced proximal support vector machine classifier [4] a very fast and simple incremental support vector machine (SVM) classifier is proposed which is capable of modifying an existing linear classifier by both retiring old data and adding new data. A very important feature of the proposed single-pass algorithm which allows it to handle massive datasets is that huge blocks of data say of the order of millions of points can be stored in blocks of size (n 1)2 where n is the usually small (typically less than 100) dimensional input space in which the data resides. To demonstrate the effectiveness of the algorithm we classify a dataset of I billion points in 10-dimensional input space into two classes in less than 2.5 hours on a 400 MHz Pentium II processor.

We develop a theoretical analysis of generalization performances of regularized leastsquares on reproducing kernel Hilbert spaces for supervised learning. We show that the concept of effective dimension of an integral operator plays a central role in the definition of a criterion for the choice of the regularization parameter as a function of the number of samples. In fact a minimax analysis is performed which shows asymptotic optimality of the above mentioned criterion.

In this paper, we develop an adaptive RBF fitting procedure for a high quality approximation of a set of points scattered over a piecewise smooth surface. We use compactly supported RBFs whose centers are randomly chosen from the points. The randomness is controlled by the point density and surface geometry. For each RBF, its support size is chosen adaptively according to surface geometry at a vicinity of the RBF center. All these lead to a noise-robust high quality approximation of the set. We also adapt our basic technique for shape reconstruction from registered range scans by taking into account measurement confidences. Finally, an interesting link between our RBF fitting procedure and partition of unity approximations is established and discussed.

“If we knew what it was we were doing, it would not be called research, would it?”