Results 1  10
of
46
On graph kernels: Hardness results and efficient alternatives
 IN: CONFERENCE ON LEARNING THEORY
, 2003
"... As most ‘realworld’ data is structured, research in kernel methods has begun investigating kernels for various kinds of structured data. One of the most widely used tools for modeling structured data are graphs. An interesting and important challenge is thus to investigate kernels on instances tha ..."
Abstract

Cited by 185 (5 self)
 Add to MetaCart
(Show Context)
As most ‘realworld’ data is structured, research in kernel methods has begun investigating kernels for various kinds of structured data. One of the most widely used tools for modeling structured data are graphs. An interesting and important challenge is thus to investigate kernels on instances that are represented by graphs. So far, only very specific graphs such as trees and strings have been considered. This paper investigates kernels on labeled directed graphs with general structure. It is shown that computing a strictly positive definite graph kernel is at least as hard as solving the graph isomorphism problem. It is also shown that computing an inner product in a feature space indexed by all possible graphs, where each feature counts the number of subgraphs isomorphic to that graph, is NPhard. On the other hand, inner products in an alternative feature space, based on walks in the graph, can be computed in polynomial time. Such kernels are defined in this paper.
A Survey of Kernels for Structured Data
, 2003
"... Kernel methods in general and support vector machines in particular have been successful in various learning tasks on data represented in a single table. Much ‘realworld’ data, however, is structured – it has no natural representation in a single table. Usually, to apply kernel methods to ‘realwor ..."
Abstract

Cited by 146 (2 self)
 Add to MetaCart
Kernel methods in general and support vector machines in particular have been successful in various learning tasks on data represented in a single table. Much ‘realworld’ data, however, is structured – it has no natural representation in a single table. Usually, to apply kernel methods to ‘realworld’ data, extensive preprocessing is performed to embed the data into a real vector space and thus in a single table. This survey describes several approaches of defining positive definite kernels on structured instances directly.
Bellman goes Relational
 In ICML
, 2004
"... Motivated by the interest in relational reinforcement learning, we introduce a novel relational Bellman update operator called ReBel. It employs a constraint logic programming language to compactly represent Markov decision processes over relational domains. ..."
Abstract

Cited by 46 (3 self)
 Add to MetaCart
(Show Context)
Motivated by the interest in relational reinforcement learning, we introduce a novel relational Bellman update operator called ReBel. It employs a constraint logic programming language to compactly represent Markov decision processes over relational domains.
Relational Reinforcement Learning
 MultiAgent Systems and Applications, 9th ECCAI Advanced Course ACAI 2001 and Agent Link’s 3rd European Agent Systems Summer School (EASSS 2001), volume 2086 of Lecture Notes in Computer Science
, 2001
"... This paper presents an introduction to reinforcement learning and relational reinforcement learning at a level to be understood by students and researchers with different backgrounds. ..."
Abstract

Cited by 28 (3 self)
 Add to MetaCart
(Show Context)
This paper presents an introduction to reinforcement learning and relational reinforcement learning at a level to be understood by students and researchers with different backgrounds.
NonParametric Policy Gradients: A Unified Treatment of Propositional and Relational Domains
"... Policy gradient approaches are a powerful instrument for learning how to interact with the environment. Existing approaches have focused on propositional and continuous domains only. Without extensive feature engineering, it is difficult – if not impossible – to apply them within structured domains, ..."
Abstract

Cited by 28 (10 self)
 Add to MetaCart
(Show Context)
Policy gradient approaches are a powerful instrument for learning how to interact with the environment. Existing approaches have focused on propositional and continuous domains only. Without extensive feature engineering, it is difficult – if not impossible – to apply them within structured domains, in which e.g. there is a varying number of objects and relations among them. In this paper, we describe a nonparametric policy gradient approach – called NPPG – that overcomes this limitation. The key idea is to apply Friedmann’s gradient boosting: policies are represented as a weighted sum of regression models grown in an stagewise optimization. Employing offtheshelf regression learners, NPPG can deal with propositional, continuous, and relational domains in a unified way. Our experimental results show that it can even improve on established results. 1.
Planning with Noisy Probabilistic Relational Rules
"... Noisy probabilistic relational rules are a promising world model representation for several reasons. They are compact and generalize over world instantiations. They are usually interpretable and they can be learned effectively from the action experiences in complex worlds. We investigate reasoning w ..."
Abstract

Cited by 26 (6 self)
 Add to MetaCart
(Show Context)
Noisy probabilistic relational rules are a promising world model representation for several reasons. They are compact and generalize over world instantiations. They are usually interpretable and they can be learned effectively from the action experiences in complex worlds. We investigate reasoning with such rules in grounded relational domains. Our algorithms exploit the compactness of rules for efficient and flexible decisiontheoretic planning. As a first approach, we combine these rules with the Upper Confidence Bounds applied to Trees (UCT) algorithm based on lookahead trees. Our second approach converts these rules into a structured dynamic Bayesian network representation and predicts the effects of action sequences using approximate inference and beliefs over world states. We evaluate the effectiveness of our approaches for planning in a simulated complex 3D robot manipulation scenario with an articulated manipulator and realistic physics and in domains of the probabilistic planning competition. Empirical results show that our methods can solve problems where existing methods fail. 1.
Practical solution techniques for firstorder mdps
 Artificial Intelligence
"... Many traditional solution approaches to relationally specified decisiontheoretic planning problems (e.g., those stated in the probabilistic planning domain description language, or PPDDL) ground the specification with respect to a specific instantiation of domain objects and apply a solution approa ..."
Abstract

Cited by 25 (1 self)
 Add to MetaCart
(Show Context)
Many traditional solution approaches to relationally specified decisiontheoretic planning problems (e.g., those stated in the probabilistic planning domain description language, or PPDDL) ground the specification with respect to a specific instantiation of domain objects and apply a solution approach directly to the resulting ground Markov decision process (MDP). Unfortunately, the space and time complexity of these grounded solution approaches are polynomial in the number of domain objects and exponential in the predicate arity and the number of nested quantifiers in the relational problem specification. An alternative to grounding a relational planning problem is to tackle the problem directly at the relational level. In this article, we propose one such approach that translates an expressive subset of the PPDDL representation to a firstorder MDP (FOMDP) specification and then derives a domainindependent policy without grounding at any intermediate step. However, such generality does not come without its own set of challenges—the purpose of this article is to explore practical solution techniques for solving FOMDPs. To demonstrate the applicability of our techniques, we present proofofconcept results of our firstorder approximate linear programming (FOALP) planner on problems from the probabilistic track
On the numeric stability of gaussian processes regression for relational reinforcement learning
 In ICML2004 Workshop on Relational Reinforcement Learning
, 2004
"... In this work we investigate the behavior of Gaussian processes as a regression technique for reinforcement learning. When confronted with too many mutually dependant learning examples, the matrix inversion needed for prediction of a new target value becomes numerically unstable. By paying attention ..."
Abstract

Cited by 13 (0 self)
 Add to MetaCart
(Show Context)
In this work we investigate the behavior of Gaussian processes as a regression technique for reinforcement learning. When confronted with too many mutually dependant learning examples, the matrix inversion needed for prediction of a new target value becomes numerically unstable. By paying attention to using suitable numerical techniques and employing QRfactorization these instabilities can be avoided. This leads to better and more stable performance of the attached reinforcement learner. 1.
Exploration in Relational Domains for Modelbased Reinforcement Learning
"... A fundamental problem in reinforcement learning is balancing exploration and exploitation. We address this problem in the context of modelbased reinforcement learning in large stochastic relational domains by developing relational extensions of the concepts of the E 3 and RMAX algorithms. Efficien ..."
Abstract

Cited by 10 (1 self)
 Add to MetaCart
(Show Context)
A fundamental problem in reinforcement learning is balancing exploration and exploitation. We address this problem in the context of modelbased reinforcement learning in large stochastic relational domains by developing relational extensions of the concepts of the E 3 and RMAX algorithms. Efficient exploration in exponentially large state spaces needs to exploit the generalization of the learned model: what in a propositional setting would be considered a novel situation and worth exploration may in the relational setting be a wellknown context in which exploitation is promising. To address this we introduce relational count functions which generalize the classical notion of state and action visitation counts. We provide guarantees on the exploration efficiency of our framework using count functions under the assumption that we had a relational KWIK learner and a nearoptimal planner. We propose a concrete exploration algorithm which integrates a practically efficient probabilistic rule learner and a relational planner (for which there are no guarantees, however) and employs the contexts of learned relational rules as features to model the novelty of states and actions. Our results in noisy 3D simulated robot manipulation problems and in domains of the international planning competition demonstrate that our approach is more effective than existing propositional and factored exploration techniques.
Transfer learning for reinforcement learning through goal and policy parametrization
 In ICML Workshop on Structural Knowledge Transfer for Machine Learning
, 2006
"... Relational reinforcement learning has allowed results from reinforcement learning tasks to be reused in other, closely related, tasks. This transfer of knowledge is made possible by the use of parameters in the representations of the taskdescription and the learned policy. In this paper, we will ..."
Abstract

Cited by 9 (2 self)
 Add to MetaCart
(Show Context)
Relational reinforcement learning has allowed results from reinforcement learning tasks to be reused in other, closely related, tasks. This transfer of knowledge is made possible by the use of parameters in the representations of the taskdescription and the learned policy. In this paper, we will give a description of the current state of the art of transfer learning with relational reinforcement learning, make some observations about the usefulness and limitations of this current state and discuss some directions for future research. We also present a first small step along one of those directions. 1.