Results 1  10
of
45
The Kernel Recursive Least Squares Algorithm
 IEEE Transactions on Signal Processing
, 2003
"... We present a nonlinear kernelbased version of the Recursive Least Squares (RLS) algorithm. Our KernelRLS (KRLS) algorithm performs linear regression in the feature space induced by a Mercer kernel, and can therefore be used to recursively construct the minimum mean squared error regressor. Spars ..."
Abstract

Cited by 141 (2 self)
 Add to MetaCart
(Show Context)
We present a nonlinear kernelbased version of the Recursive Least Squares (RLS) algorithm. Our KernelRLS (KRLS) algorithm performs linear regression in the feature space induced by a Mercer kernel, and can therefore be used to recursively construct the minimum mean squared error regressor. Sparsity of the solution is achieved by a sequential sparsification process that admits into the kernel representation a new input sample only if its feature space image cannot be suffciently well approximated by combining the images of previously admitted samples. This sparsification procedure is crucial to the operation of KRLS, as it allows it to operate online, and by effectively regularizing its solutions. A theoretical analysis of the sparsification method reveals its close affinity to kernel PCA, and a datadependent loss bound is presented, quantifying the generalization performance of the KRLS algorithm. We demonstrate the performance and scaling properties of KRLS and compare it to a stateof theart Support Vector Regression algorithm, using both synthetic and real data. We additionally test KRLS on two signal processing problems in which the use of traditional leastsquares methods is commonplace: Time series prediction and channel equalization.
Bayes Meets Bellman: The Gaussian Process Approach to Temporal Difference Learning
 Proc. of the 20th International Conference on Machine Learning
, 2003
"... We present a novel Bayesian approach to the problem of value function estimation in continuous state spaces. We de ne a probabilistic generative model for the value function by imposing a Gaussian prior over value functions and assuming a Gaussian noise model. ..."
Abstract

Cited by 76 (8 self)
 Add to MetaCart
We present a novel Bayesian approach to the problem of value function estimation in continuous state spaces. We de ne a probabilistic generative model for the value function by imposing a Gaussian prior over value functions and assuming a Gaussian noise model.
Learning interpretable SVMs for biological sequence classification
 BMC BIOINFORMATICS
, 2005
"... We propose novel algorithms for solving the socalled Support Vector Multiple Kernel Learning problem and show how they can be used to understand the resulting support vector decision function. While classical kernelbased algorithms (such as SVMs) are based on a single kernel, in Multiple Kernel Le ..."
Abstract

Cited by 52 (16 self)
 Add to MetaCart
(Show Context)
We propose novel algorithms for solving the socalled Support Vector Multiple Kernel Learning problem and show how they can be used to understand the resulting support vector decision function. While classical kernelbased algorithms (such as SVMs) are based on a single kernel, in Multiple Kernel Learning a quadraticallyconstraint quadratic program is solved in order to find a sparse convex combination of a set of support vector kernels. We show how this problem can be cast into a semiinfinite linear optimization problem which can in turn be solved efficiently using a boostinglike iterative method in combination with standard SVM optimization algorithms. The proposed method is able to deal with thousands of examples while combining hundreds of kernels within reasonable time. In the second part we show how this technique can be used to understand the obtained decision function in order to extract biologically relevant knowledge about the sequence analysis problem at hand. We consider the problem of splice site identification and combine string kernels at different sequence positions and with various substring (oligomer) lengths. The proposed algorithm computes a sparse weighting over the length and the substring, highlighting which substrings are important for discrimination. Finally, we propose a bootstrap scheme in order to reliably identify a few statistically significant positions, which can then be used for further analysis such as consensus finding.
Kernelbased least squares policy iteration for reinforcement learning
 IEEE Transactions on Neural Networks
, 2007
"... Abstract—In this paper, we present a kernelbased least squares policy iteration (KLSPI) algorithm for reinforcement learning (RL) in large or continuous state spaces, which can be used to realize adaptive feedback control of uncertain dynamic systems. By using KLSPI, nearoptimal control policies c ..."
Abstract

Cited by 33 (0 self)
 Add to MetaCart
(Show Context)
Abstract—In this paper, we present a kernelbased least squares policy iteration (KLSPI) algorithm for reinforcement learning (RL) in large or continuous state spaces, which can be used to realize adaptive feedback control of uncertain dynamic systems. By using KLSPI, nearoptimal control policies can be obtained without much a priori knowledge on dynamic models of control plants. In KLSPI, Mercer kernels are used in the policy evaluation of a policy iteration process, where a new kernelbased least squares temporaldifference algorithm called KLSTDQ is proposed for efficient policy evaluation. To keep the sparsity and improve the generalization ability of KLSTDQ solutions, a kernel sparsification procedure based on approximate linear dependency (ALD) is performed. Compared to the previous works on approximate RL methods, KLSPI makes two progresses to eliminate the main difficulties of existing results. One is the better convergence and (near) optimality guarantee by using the KLSTDQ algorithm for policy evaluation with high precision. The other is the automatic feature selection using the ALDbased kernel sparsification. Therefore, the KLSPI algorithm provides a general RL method with generalization performance and convergence guarantee for largescale Markov decision problems (MDPs). Experimental results on a typical RL task for a stochastic chain problem demonstrate that KLSPI can consistently achieve better learning efficiency and policy quality than the previous least squares policy iteration (LSPI) algorithm. Furthermore, the KLSPI method was also evaluated on two nonlinear feedback control problems, including a ship heading control problem and the swing up control of a doublelink underactuated pendulum called acrobot. Simulation results illustrate that the proposed method can optimize controller performance using little a priori information of uncertain dynamic systems. It is also demonstrated that KLSPI can be applied to online learning control by incorporating an initial controller to ensure online performance. Index Terms—Approximate dynamic programming, kernel methods, least squares, Markov decision problems (MDPs), reinforcement learning (RL).
The Projectron: a Bounded KernelBased Perceptron
"... We present a discriminative online algorithm with a bounded memory growth, which is based on the kernelbased Perceptron. Generally, the required memory of the kernelbased Perceptron for storing the online hypothesis is not bounded. Previous work has been focused on discarding part of the instances ..."
Abstract

Cited by 29 (3 self)
 Add to MetaCart
(Show Context)
We present a discriminative online algorithm with a bounded memory growth, which is based on the kernelbased Perceptron. Generally, the required memory of the kernelbased Perceptron for storing the online hypothesis is not bounded. Previous work has been focused on discarding part of the instances in order to keep the memory bounded. In the proposed algorithm the instances are not discarded, but projected onto the space spanned by the previous online hypothesis. We derive a relative mistake bound and compare our algorithm both analytically and empirically to the stateoftheart Forgetron algorithm (Dekel et al, 2007). The first variant of our algorithm, called Projectron, outperforms the Forgetron. The second variant, called Projectron++, outperforms even the Perceptron. 1.
Indoor place recognition using online independent support vector machines
 In BMVC ’07
, 2007
"... In the framework of indoor mobile robotics, place recognition is a challenging task, where it is crucial that selflocalization be enforced precisely, notwithstanding the changing conditions of illumination, objects being shifted around and/or people affecting the appearance of the scene. In this sc ..."
Abstract

Cited by 21 (14 self)
 Add to MetaCart
(Show Context)
In the framework of indoor mobile robotics, place recognition is a challenging task, where it is crucial that selflocalization be enforced precisely, notwithstanding the changing conditions of illumination, objects being shifted around and/or people affecting the appearance of the scene. In this scenario online learning seems the main way out, thanks to the possibility of adapting to changes in a smart and flexible way. Nevertheless, standard machine learning approaches usually suffer when confronted with massive amounts of data and when asked to work online. Online learning requires a high training and testing speed, all the more in place recognition, where a continuous flow of data comes from one or more cameras. In this paper we follow the Support Vector Machinesbased approach of Pronobis et al. [26], proposing an improvement that we call Online Independent Support Vector Machines. This technique exploits linear independence in the image feature space to incrementally keep the size of the learning machine remarkably small while retaining the accuracy of a standard machine. Since the training and testing time crucially depend on the size of the machine, this solves the above stated problems. Our experimental results prove the effectiveness of the approach. 1
Extended Kernel Recursive Least Squares Algorithm
"... This paper presents a kernelized version of the extended recursive least squares (EXKRLS) algorithm which implements for the first time a general linear state model in reproducing kernel Hilbert spaces (RKHS), or equivalently a general nonlinear state model in the input space. The center piece of t ..."
Abstract

Cited by 17 (3 self)
 Add to MetaCart
(Show Context)
This paper presents a kernelized version of the extended recursive least squares (EXKRLS) algorithm which implements for the first time a general linear state model in reproducing kernel Hilbert spaces (RKHS), or equivalently a general nonlinear state model in the input space. The center piece of this development is a reformulation of the well known extended recursive least squares (EXRLS) algorithm in RKHS which only requires inner product operations between input vectors, thus enabling the application of the kernel property (commonly known as the kernel trick). The first part of the paper presents a set of theorems that shows the generality of the approach. The EXKRLS is preferable to: (1) a standard kernel recursive least squares (KRLS) in applications that require tracking the statevector of general linear statespace models in the kernel space, or (2) an EXRLS when the application requires a nonlinear observation and state models. The second part of the paper compares the EXKRLS in nonlinear Rayleigh multipath channel tracking and in Lorenz system modeling problem. We show that the proposed algorithm is able to outperform the standard EXRLS and KRLS in both simulations.
Model Learning for Robot Control: A Survey
 COGNITIVE SCIENCE
"... Models are among the most essential tools in robotics, such as kinematics and dynamics models of the robot’s own body and controllable external objects. It is widely believed that intelligent mammals also rely on internal models in order to generate their actions. However, while classical robotics ..."
Abstract

Cited by 13 (1 self)
 Add to MetaCart
Models are among the most essential tools in robotics, such as kinematics and dynamics models of the robot’s own body and controllable external objects. It is widely believed that intelligent mammals also rely on internal models in order to generate their actions. However, while classical robotics relies on manually generated models that are based on human insights into physics, future autonomous, cognitive robots need to be able to automatically generate models that are based on information which is extracted from the data streams accessible to the robot. In this paper, we survey the progress in model learning with a strong focus on robot control on a kinematic as well as dynamical level. Here, a model describes essential information about the behavior of the environment and the influence of an agent on this environment. In the context of model based learning control, we view the model from three different perspectives. First, we need to study the different possible model learning architectures for robotics. Second, we discuss what kind of problems these architecture and the domain of robotics imply for the applicable learning methods. From this discussion, we deduce future directions of realtime learning algorithms. Third, we show where these scenarios have been used successfully in several case studies.
Kernel Recursive Least Squares
 IEEE Transactions on Signal Processing
, 2004
"... We present a nonlinear kernelbased version of the Recursive Least Squares (RLS) algorithm. Our KernelRLS algorithm performs linear regression in the feature space induced by a Mercer kernel, and can therefore be used to recursively construct the minimum meansquared error regressor. Sparsity (and ..."
Abstract

Cited by 13 (0 self)
 Add to MetaCart
(Show Context)
We present a nonlinear kernelbased version of the Recursive Least Squares (RLS) algorithm. Our KernelRLS algorithm performs linear regression in the feature space induced by a Mercer kernel, and can therefore be used to recursively construct the minimum meansquared error regressor. Sparsity (and therefore regularization) of the solution is achieved by an explicit greedy sparsification process that admits into the kernel representation a new input sample only if its feature space image is linearly independent of the images of previously admitted samples. Most importantly, this sparsification procedure allows the algorithm to operate online. We demonstrate the performance and scaling properties of the KernelRLS algorithm as compared to a stateoftheart Support Vector Regression algorithm, on both synthetic and real data.