Results 1  10
of
29
Reinforcement learning for humanoid robotics
 Autonomous Robot
, 2003
"... Abstract. The complexity of the kinematic and dynamic structure of humanoid robots make conventional analytical approaches to control increasingly unsuitable for such systems. Learning techniques offer a possible way to aid controller design if insufficient analytical knowledge is available, and lea ..."
Abstract

Cited by 132 (21 self)
 Add to MetaCart
(Show Context)
Abstract. The complexity of the kinematic and dynamic structure of humanoid robots make conventional analytical approaches to control increasingly unsuitable for such systems. Learning techniques offer a possible way to aid controller design if insufficient analytical knowledge is available, and learning approaches seem mandatory when humanoid systems are supposed to become completely autonomous. While recent research in neural networks and statistical learning has focused mostly on learning from finite data sets without stringent constraints on computational efficiency, learning for humanoid robots requires a different setting, characterized by the need for realtime learning performance from an essentially infinite stream of incrementally arriving data. This paper demonstrates how even highdimensional learning problems of this kind can successfully be dealt with by techniques from nonparametric regression and locally weighted learning. As an example, we describe the application of one of the most advanced of such algorithms, Locally Weighted Projection Regression (LWPR), to the online learning of three problems in humanoid motor control: the learning of inverse dynamics models for modelbased control, the learning of inverse kinematics of redundant manipulators, and the learning of oculomotor reflexes. All these examples demonstrate fast, i.e., within seconds or minutes, learning convergence with highly accurate final peformance. We conclude that realtime learning for complex motor system like humanoid robots is possible with appropriately tailored algorithms, such that increasingly autonomous robots with massive learning abilities should be achievable in the near future. 1.
Locally Weighted Projection Regression: An O(n) Algorithm for Incremental Real Time Learning in High Dimensional Space
 in Proceedings of the Seventeenth International Conference on Machine Learning (ICML 2000
"... Locally weighted projection regression is a new algorithm that achieves nonlinear function approximation in high dimensional spaces with redundant and irrelevant input dimensions. At its core, it uses locally linear models, spanned by a small number of univariate regressions in selected direct ..."
Abstract

Cited by 103 (15 self)
 Add to MetaCart
Locally weighted projection regression is a new algorithm that achieves nonlinear function approximation in high dimensional spaces with redundant and irrelevant input dimensions. At its core, it uses locally linear models, spanned by a small number of univariate regressions in selected directions in input space. This paper evaluates different methods of projection regression and derives a nonlinear function approximator based on them. This nonparametric local learning system i) learns rapidly with second order learning methods based on incremental training, ii) uses statistically sound stochastic cross validation to learn iii) adjusts its weighting kernels based on local information only, iv) has a computational complexity that is linear in the number of inputs, and v) can deal with a large number of  possibly redundant  inputs, as shown in evaluations with up to 50 dimensional data sets. To our knowledge, this is the first truly incremental spatially localized l...
Scalable Techniques from Nonparametric Statistics for Real Time Robot Learning
, 2000
"... Locally weighted learning (LWL) is a class of techniques from nonparametric statistics that provides useful representations and training algorithms for learning about complex phenomena during autonomous adaptive control of robotic systems. This paper introduces several LWL algorithms that have been ..."
Abstract

Cited by 50 (3 self)
 Add to MetaCart
Locally weighted learning (LWL) is a class of techniques from nonparametric statistics that provides useful representations and training algorithms for learning about complex phenomena during autonomous adaptive control of robotic systems. This paper introduces several LWL algorithms that have been tested successfully in realtime learning of complex robot tasks. We discuss two major classes of LWL, memorybased LWL and purely incremental LWL that does not need to remember any data explicitly. In contrast to the traditional belief that LWL methods cannot work well in highdimensional spaces, we provide new algorithms that have been tested on up to 90 dimensional learning problems. The applicability of our LWL algorithms is demonstrated in various robot learning examples, including the learning of devilsticking, polebalancing by a humanoid robot arm, and inversedynamics learning for a seven and a 30 degreeoffreedom robot. In all these examples, the application of our statistical n...
The Bayesian Backfitting Relevance Vector Machine
 IN PROCEEDINGS OF THE 21ST INTERNATIONAL CONFERENCE ON MACHINE LEARNING
, 2004
"... Traditional nonparametric statistical learning techniques are often computationally attractive, but lack the same generalization and model selection abilities as stateoftheart Bayesian algorithms which, however, are usually computationally prohibitive. This paper makes several important co ..."
Abstract

Cited by 22 (8 self)
 Add to MetaCart
Traditional nonparametric statistical learning techniques are often computationally attractive, but lack the same generalization and model selection abilities as stateoftheart Bayesian algorithms which, however, are usually computationally prohibitive. This paper makes several important contributions that allow Bayesian learning to scale to more complex, realworld learning scenarios. Firstly, we show that backfitting  a traditional nonparametric, yet highly e#cient regression tool  can be derived in a novel formulation within an expectation maximization (EM) framework and thus can finally be given a probabilistic interpretation. Secondly, we show that the general framework of sparse Bayesian learning and in particular the relevance vector machine (RVM), can be derived as a highly e#cient algorithm using a Bayesian version of backfitting at its core. As we demonstrate on several regression and classification benchmarks, Bayesian backfitting o#ers a compelling alternative to current regression methods, especially when the size and dimensionality of the data challenge computational resources.
Locally adaptable nonparametric methods for estimating stand characteristics for wood procurement planning
 Silva Fenn. 37 ( 1 ), 109 – 120
, 2003
"... Malinen, J. 2003. Locally adaptable nonparametric methods for estimating stand characteristics for wood procurement planning. Silva Fennica 37(1): 109–120. The purpose of this study was to examine the use of the local adaptation of the nonparametric Most Similar Neighbour (MSN) method in estimatin ..."
Abstract

Cited by 10 (3 self)
 Add to MetaCart
Malinen, J. 2003. Locally adaptable nonparametric methods for estimating stand characteristics for wood procurement planning. Silva Fennica 37(1): 109–120. The purpose of this study was to examine the use of the local adaptation of the nonparametric Most Similar Neighbour (MSN) method in estimating stand characteristics for wood procurement planning purposes. Local adaptation was performed in two different ways: 1) by selecting local data from a database with the MSN method and using that data as a database in the basic knearest neighbour (knn) MSN method, 2) by selecting a combination of neighbours from the neighbourhood where the average of the predictor variables was closest to the target stand predictor variables (Locally Adaptable Neighbourhood (LAN) MSN method). The study data used comprised 209 spruce dominated stands located in central Finland and was collected with harvesters. The accuracy of the methods was analysed by estimating the tree stock characteristics and the log length/diameter distribution produced by a bucking simulation. The local knn MSN method was not notably better than the knn MSN method, although it produced less biased estimates on the edges of the input space. The LAN MSN method was found to be a more accurate method than the knn methods. Keywords local nonparametric estimation, MSN method, stand characteristics, wood procurement planning
Predicting EMG Data from M1 Neurons with Variational Bayesian Least Squares
"... An increasing number of projects in neuroscience requires the statistical analysis of high dimensional data sets, as, for instance, in predicting behavior from neural firing or in operating artificial devices from brain recordings in brainmachine interfaces. Linear analysis techniques remain preval ..."
Abstract

Cited by 7 (3 self)
 Add to MetaCart
(Show Context)
An increasing number of projects in neuroscience requires the statistical analysis of high dimensional data sets, as, for instance, in predicting behavior from neural firing or in operating artificial devices from brain recordings in brainmachine interfaces. Linear analysis techniques remain prevalent in such cases, but classical linear regression approaches are often numerically too fragile in high dimensions. In this paper, we address the question of whether EMG data collected from arm movements of monkeys can be faithfully reconstructed with linear approaches from neural activity in primary motor cortex (M1). To achieve robust data analysis, we develop a full Bayesian approach to linear regression that automatically detects and excludes irrelevant features in the data, regularizing against overfitting. In comparison with ordinary least squares, stepwise regression, partial least squares, LASSO regression and a brute force combinatorial search for the most predictive input features in the data, we demonstrate that the new Bayesian method offers a superior mixture of characteristics in terms of regularization against overfitting, computational efficiency and ease of use, demonstrating its potential as a dropin replacement for other linear regression techniques. As neuroscientific results, our analyses demonstrate that EMG data can be well predicted from M1 neurons, further opening the path for possible realtime interfaces between brains and machines. 1
Real Time Learning in Humanoids: A Challenge for Scalability of Online Algorithms
, 2000
"... . While recent research in neural networks and statistical learning has focused mostly on learning from finite data sets without stringent constraints on computational efficiency, there is an increasing number of learning problems that require realtime performance from an essentially infinite str ..."
Abstract

Cited by 5 (1 self)
 Add to MetaCart
(Show Context)
. While recent research in neural networks and statistical learning has focused mostly on learning from finite data sets without stringent constraints on computational efficiency, there is an increasing number of learning problems that require realtime performance from an essentially infinite stream of incrementally arriving data. This paper demonstrates how even highdimensional learning problems of this kind can successfully be dealt with by techniques from nonparametric regression and locally weighted learning. As an example, we describe the application of one of the most advanced of such algorithms, Locally Weighted Projection Regression (LWPR), to the online learning of the inverse dynamics model of an actual seven degreeoffreedom anthropomorphic robot arm. LWPR's linear computational complexity in the number of input dimensions, its inherent mechanisms of local dimensionality reduction, and its sound learning rule basedon incremental stochastic leaveone out cross va...
Localized Classification
, 2004
"... The main problem with localized discriminant techniques is the curse of dimensionality, which seems to restrict their use to the case of few variables. This restriction does not hold if localization is combined with a reduction of dimension. In particular it is shown that localization yields powerfu ..."
Abstract

Cited by 5 (0 self)
 Add to MetaCart
The main problem with localized discriminant techniques is the curse of dimensionality, which seems to restrict their use to the case of few variables. This restriction does not hold if localization is combined with a reduction of dimension. In particular it is shown that localization yields powerful classifiers even in higher dimensions if localization is combined with locally adaptive selection of predictors. A robust localized logistic regression (LLR) method is developed for which all tuning parameters are chosen dataadaptively. In an extended simulation study we evaluate the potential of the proposed procedure for various types of data and compare it to other classification procedures. In addition we demonstrate that automatic choice of localization, predictor selection and penalty parameters based on cross validation is working well. Finally the method is applied to real data sets and its real world performance is compared to alternative procedures.
A Bayesian approach to empirical local linearizations for robotics
 in Proc. Int. Conf. Robotics and Automation (ICRA2008
, 2008
"... AbstractLocal linearizations are ubiquitous in the control of robotic systems. Analytical methods, if available, can be used to obtain the linearization, but in complex robotics systems where the dynamics and kinematics are often not faithfully obtainable, empirical linearization may be preferable ..."
Abstract

Cited by 4 (1 self)
 Add to MetaCart
(Show Context)
AbstractLocal linearizations are ubiquitous in the control of robotic systems. Analytical methods, if available, can be used to obtain the linearization, but in complex robotics systems where the dynamics and kinematics are often not faithfully obtainable, empirical linearization may be preferable. In this case, it is important to only use data for the local linearization that lies within a "reasonable" linear regime of the system, which can be defined from the Hessian at the point of the linearizationa quantity that is not available without an analytical model. We introduce a Bayesian approach to solve statistically what constitutes a "reasonable" local regime. We approach this problem in the context local linear regression. In contrast to previous locally linear methods, we avoid crossvalidation or complex statistical hypothesis testing techniques to find the appropriate local regime. Instead, we treat the parameters of the local regime probabilistically and use approximate Bayesian inference for their estimation. The approach results in an analytical set of iterative update equations that are easily implemented on real robotics systems for realtime applications. As in other locally weighted regressions, our algorithm also lends itself to complete nonlinear function approximation for learning empirical internal models. We sketch the derivation of our Bayesian method and provide evaluations on synthetic data and actual robot data where the analytical linearization was known.
Local Dimensionality Reduction for NonParametric Regression
, 2009
"... Abstract Locallyweighted regression is a computationallyefficient technique for nonlinear regression. However, for highdimensional data, this technique becomes numerically brittle and computationally too expensive if many local models need to be maintained simultaneously. Thus, local linear dime ..."
Abstract

Cited by 4 (1 self)
 Add to MetaCart
(Show Context)
Abstract Locallyweighted regression is a computationallyefficient technique for nonlinear regression. However, for highdimensional data, this technique becomes numerically brittle and computationally too expensive if many local models need to be maintained simultaneously. Thus, local linear dimensionality reduction combined with locallyweighted regression seems to be a promising solution. In this context, we review linear dimensionalityreduction methods, compare their performance on nonparametric locallylinear regression, and discuss their ability to extend to incremental learning. The considered methods belong to the following three groups: (1) reducing dimensionality only on the input data, (2) modeling the joint inputoutput data distribution, and (3) optimizing the correlation between projection directions and output data. Group 1 contains principal component regression (PCR); group 2 contains principal component analysis (PCA) in joint input and output space, factor analysis, and probabilistic PCA; and group 3 contains reduced rank regression (RRR) and partial least squares (PLS) regression. Among the tested methods, only group 3 managed to achieve robust performance even for a nonoptimal number of components (factors or projection directions). In contrast, group 1 and 2 failed for fewer components since these methods rely on the correct estimate of the true intrinsic dimensionality. In group 3, PLS is the only method for which a computationallyefficient incremental implementation exists.