We describe and evaluate a system for learning domainspecific control knowledge. In particular, given a planning domain, the goal is to output a control policy that performs well on “long random walk ” problem distributions. The system is based on viewing planning domains as very large Markov decision processes and then applying a recent variant of approximate policy iteration that is bootstrapped with a new technique based on random walks. We evaluate the system on the AIPS-2000 planning domains (among others) and show that often the learned policies perform well on problems drawn from the long–random-walk distribution. In addition, we show that these policies often perform well on the original problem distributions from the domains involved. Our evaluation also uncovers limitations of our current system that point to future challenges.

In this paper, we propose a new representation for physical control -- teleoreactive logic programs -- along with an interpreter that uses them to achieve goals. In addition, we present a new learning method that acquires recursive forms of these structures from traces of successful problem solving. We report

This paper addresses weighting and partitioning in complex reinforcement learning tasks, with the aim of facilitating learning. The paper presents some ideas regarding weighting of multiple agents and extends them into partitioning an input/state space into multiple regions with differential weighting in these regions, to exploit differential characteristics of regions and differential characteristics of agents to reduce the learning complexity of agents (and their function approximators) and thus to facilitate the learning overall. It analyzes, in reinforcement learning tasks, different ways of partitioning a task and using agents selectively based on partitioning. Based on the analysis, some heuristic methods are described and experimentally tested. We find that some off-line heuristic methods performed the best, significantly better than single-agent models. Keywords: weighting, averaging, neural networks, partitioning, gating, reinforcement learning, 1 Introduction Multiple ag...

Abstract. In this paper, we focus on the problem of learning reactive skills for use by physical agents. We propose a new representation for such procedures, teleoreactive logic programs, along with an interpreter that utilizes them to achieve goals. After this, we describe a learning method that acquires these structures in a cumulative manner through problem solving. We report experiments in three domains that involve multiple levels of skilled behavior. We also review related work and discuss directions for future research. 1

We describe HTN-MAKER, an algorithm for learning hierarchical planning knowledge in the form of decomposition methods for Hierarchical Task Networks (HTNs). HTN-MAKER takes as input the initial states from a set of classical planning problems in a planning domain and solutions to those problems, as well as a set of semantically-annotated tasks to be accomplished. The algorithm analyzes this semantic information in order to determine which portions of the input plans accomplish a particular task and constructs HTN methods based on those analyses. Our theoretical results show that HTN-MAKER is sound and complete. We also present a formalism for a class of planning problems that are more expressive than classical planning. These planning problems can be represented as HTN planning problems. We show that the methods learned by HTN-MAKER enable an HTN planner to solve those problems. Our experiments confirm the theoretical results and demonstrate convergence in three well-known planning domains toward a set of HTN methods that can be used to solve nearly any problem expressible as a classical planning problem in that domain, relative to a set of goals.

A Horn definition is a set of Horn clauses with the same head literal. In this paper, we consider learning non-recursive, function-free first-order Horn definitions. We show that this class is exactly learnable from equivalence and membership queries. It follows then that this class is PAC learnable using examples and membership queries. Our results have been shown to be applicable to learning efficient goal-decomposition rules in planning domains.

A significant challenge in developing planning systems for practical applications is the difficulty of acquiring the domain knowledge needed by such systems. One method for acquiring this knowledge is to learn it from plan traces, but this method typically requires a huge number of plan traces to converge. In this paper, we show that the problem with slow convergence can be circumvented by having the learner generate solution plans even before the planning domain is completely learned. Our empirical results show that these improvements reduce the size of the training set that is needed to find correct answers to a large percentage of planning problems in the test set. 1.

A Horn definition is a set of Horn clauses with the same head literal. In this paper, we consider learning non-recursive, first-order Horn definitions from entailment. We show that this class is exactly learnable from equivalence and membership queries. It follows then that this class is PAC learnable using examples and membership queries. Finally, we apply our results to learning control knowledge for efficient planning in the form of goal-decomposition rules. Keywords Horn Definition, Horn Program, Horn Sentence, Horn Clause, Learning from Entailment, Planning, Control Knowledge, Queries, PAC-learning. x1 Introduction Horn clauses is one of the popular ways of representing first order knowledge. In this paper, we consider learning Horn definitions---multiple Horn clauses with the same predicate in the heads of all clauses---in the learning from entailment setting. 15, 8) In this setting, the target concept is a Horn definition. A positive (negative) example is a Horn clause (not)...

. In this paper, we consider learning first-order Horn programs from entailment. In particular, we show that any subclass of first-order acyclic Horn programs with constant arity is exactly learnable from equivalence and entailment membership queries provided it allows a polynomial-time subsumption procedure and satisfies some closure conditions. One consequence of this is that first-order acyclic determinate Horn programs with constant arity are exactly learnable from equivalence and entailment membership queries. 1 Introduction Learning first-order Horn programs---sets of first-order Horn clauses---is an important problem in inductive logic programming with applications ranging from speedup learning to grammatical inference. We are interested in speedup learning, which concerns learning domainspecific control knowledge to alleviate the computational hardness of planning. One kind of control knowledge, which is particularly useful in many domains, is represented as goal-decomposition...

Agreat challenge in developing planning systems for practical applications is the difficulty of acquiring the domain information needed to guide such systems. This paper describes a way to learn some of that knowledge. More specifically, the following points are discussed. (1) We introduce a theoretical basis for formally defining algorithms that learn preconditions for Hierarchical Task Network (HTN) methods. (2) We describe Candidate Elimination Method Learner (CaMeL), a supervised, eager, and incremental learning process for preconditions of HTN methods. We state and prove theorems about CaMeL’s soundness, completeness, and convergence properties. (3) We present empirical results about CaMeL’s convergence under various conditions. Among other things, CaMeL converges the fastest on the preconditions of the HTN methods that are needed the most often. Thus CaMeL’s output can be useful even before it has fully converged.