Results 1  10
of
23
Comparing RealTime and Incremental Heuristic Search for RealTime Situated Agents
, 2009
"... Realtime situated agents, such as characters in realtime computer games, often do not know the terrain in advance but automatically observe it within a certain range around themselves. They have to interleave searches with action executions to make the searches tractable when moving autonomously t ..."
Abstract

Cited by 36 (2 self)
 Add to MetaCart
(Show Context)
Realtime situated agents, such as characters in realtime computer games, often do not know the terrain in advance but automatically observe it within a certain range around themselves. They have to interleave searches with action executions to make the searches tractable when moving autonomously to userspecified coordinates. The searches face realtime requirements since it is important that the agents be responsive to the commands of the users and move smoothly. In this article, we compare two classes of fast heuristic search methods for these navigation tasks that speed up A * searches in different ways, namely realtime heuristic search and incremental heuristic search, to understand their advantages and disadvantages and make recommendations about when each one should be used. We first develop a competitive realtime heuristic search method. LSSLRTA * is a version of Learning RealTime A* that uses A * to determine its local search spaces and learns quickly. We analyze the properties of LSSLRTA * and then compare it experimentally against the stateoftheart incremental heuristic search method D * Lite [21] on our navigation tasks, for which D * Lite was specifically developed, resulting in the first comparison of realtime and incremental heuristic search in the literature. We characterize when to choose each one of the two heuristic search methods, depending on the search objective and the kind of terrain. Our experimental results show that LSSLRTA * can outperform D * Lite under the right conditions, namely when there is time pressure or the usersupplied hvalues are generally not misleading.
Efficient Incremental Search for Moving Target Search
, 2009
"... Incremental search algorithms reuse information from previous searches to speed up the current search and are thus often able to find shortest paths for series of similar search problems faster than by solving each search problem independently from scratch. However, they do poorly on moving target s ..."
Abstract

Cited by 12 (5 self)
 Add to MetaCart
Incremental search algorithms reuse information from previous searches to speed up the current search and are thus often able to find shortest paths for series of similar search problems faster than by solving each search problem independently from scratch. However, they do poorly on moving target search problems, where both the start and goal cells change over time. In this paper, we thus develop FringeRetrieving A * (FRA*), an incremental version of A * that repeatedly finds shortest paths for moving target search in known gridworlds. We demonstrate experimentally that it runs up to one order of magnitude faster than a variety of stateoftheart incremental search algorithms applied to moving target search in known gridworlds.
Adaptive Resource Control: Machine Learning Approaches to Resource Allocation in Uncertain and Changing Environments
, 2008
"... [...] we must not look for equal exactness in all departments of study, but only such as belongs to the subject matter of each, and in such a degree as is appropriate to the particular line of enquiry. A carpenter and a geometrician both try to find a right angle, but in different ways; the former i ..."
Abstract

Cited by 3 (1 self)
 Add to MetaCart
(Show Context)
[...] we must not look for equal exactness in all departments of study, but only such as belongs to the subject matter of each, and in such a degree as is appropriate to the particular line of enquiry. A carpenter and a geometrician both try to find a right angle, but in different ways; the former is content with that approximation to it which satisfies the purpose of his work; the latter, being a student of truth, seeks to find its essence or essential attributes. We should therefore proceed in the same manner in other subjects also, and not allow side issues to outbalance the main task in hand. (Aristotle in 23 Volumes, Vol. 19, translated by Harris Rackham, Harvard University Press, 1934) Declaration Herewith I confirm that all of the research described in this dissertation is my own original work and expressed in my own words. Any use made within it of works of other authors in any form, e.g., ideas, figures, text, tables, are properly indicated through the application of citations and references. I also declare that no part of the dissertation has been submitted for any other degree — either from the Eötvös Loránd University or another institution.
Online detection of dead states in realtime agentcentered search
 In Proceedings of the Sixth Annual Symposium on Combinatorial Search (SoCS
, 2013
"... In this paper we introduce techniques for state pruning at runtime in a priori unknown domains. We describe how to identify states that can be deleted from the statespace when looking for both optimal and suboptimal solutions. We discuss general graphs and special cases like 8connected grids. Exp ..."
Abstract

Cited by 3 (0 self)
 Add to MetaCart
In this paper we introduce techniques for state pruning at runtime in a priori unknown domains. We describe how to identify states that can be deleted from the statespace when looking for both optimal and suboptimal solutions. We discuss general graphs and special cases like 8connected grids. Experimental results show a speed up of up to an order of magnitude when applying our techniques on realtime agentcentered search problems.
MonteCarlo Planning for Pathfinding in RealTime Strategy Games
"... In this work, we explore two MonteCarlo planning approaches: Upper Confidence Tree (UCT) and Rapidlyexploring Random Tree (RRT). These MonteCarlo planning approaches are applied in a realtime strategy game for solving the path finding problem. The planners are evaluated using a gridbased represe ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
In this work, we explore two MonteCarlo planning approaches: Upper Confidence Tree (UCT) and Rapidlyexploring Random Tree (RRT). These MonteCarlo planning approaches are applied in a realtime strategy game for solving the path finding problem. The planners are evaluated using a gridbased representation of our game world. The results show that the UCT planner solves the path planning problem with significantly less search effort than the RRT planner. The game playing performance of each planner is evaluated using the mean, maximum and minimum scores in the test games. With respect to the mean scores, the RRT planner shows better performance than the UCT planner. The RRT planner achieves more maximum scores than the UCT planner in the test games.
Adaptive Stochastic Resource Control: A Machine Learning Approach
"... The paper investigates stochastic resource allocation problems with scarce, reusable resources and nonpreemtive, timedependent, interconnected tasks. This approach is a natural generalization of several standard resource management problems, such as scheduling and transportation problems. First, r ..."
Abstract

Cited by 2 (1 self)
 Add to MetaCart
(Show Context)
The paper investigates stochastic resource allocation problems with scarce, reusable resources and nonpreemtive, timedependent, interconnected tasks. This approach is a natural generalization of several standard resource management problems, such as scheduling and transportation problems. First, reactive solutions are considered and defined as control policies of suitably reformulated Markov decision processes (MDPs). We argue that this reformulation has several favorable properties, such as it has finite state and action spaces, it is aperiodic, hence all policies are proper and the space of control policies can be safely restricted. Next, approximate dynamic programming (ADP) methods, such as fitted Qlearning, are suggested for computing an efficient control policy. In order to compactly maintain the costtogo function, two representations are studied: hash tables and support vector regression (SVR), particularly, νSVRs. Several additional improvements, such as the application of limitedlookahead rollout algorithms in the initial phases, action space decomposition, task clustering and distributed sampling are investigated, too. Finally, experimental results on both benchmark and industryrelated data are presented. 1.
A Review of Machine Learning for Automated Planning
, 2009
"... Recent discoveries in automated planning are broadening the scope of planners, from toy problems to real applications. However, applying automated planners to realworld problems is far from simple. On the one hand, the definition of accurate action models for planning is still a bottleneck. On the ..."
Abstract

Cited by 2 (1 self)
 Add to MetaCart
Recent discoveries in automated planning are broadening the scope of planners, from toy problems to real applications. However, applying automated planners to realworld problems is far from simple. On the one hand, the definition of accurate action models for planning is still a bottleneck. On the other hand, offtheshelf planners fail to scale up and to provide good solutions in many domains. In these problematic domains, planners can exploit domainspecific control knowledge to improve their performance in terms of both speed and quality of the solutions. However, manual definition of control knowledge is quite difficult. This paper reviews recent techniques in machine learning for the automatic definition of planning knowledge. It has been organized according to the target of the learning process: automatic definition of planning action models and automatic definition of planning control knowledge. In addition, the paper reviews the advances in the related field of reinforcement learning.
A Review of Machine Learning for Automated Planning
, 2009
"... Recent discoveries in automated planning are broadening the scope of planners, from toy problems to real applications. However, applying automated planners to realworld problems is far from simple. On the one hand, the definition of accurate action models for planning is still a bottleneck. On the ..."
Abstract
 Add to MetaCart
Recent discoveries in automated planning are broadening the scope of planners, from toy problems to real applications. However, applying automated planners to realworld problems is far from simple. On the one hand, the definition of accurate action models for planning is still a bottleneck. On the other hand, offtheshelf planners fail to scale up and to provide good solutions in many domains. In these problematic domains, planners can exploit domainspecific control knowledge to improve their performance in terms of both speed and quality of the solutions. However, manual definition of control knowledge is quite difficult. This paper reviews recent techniques in machine learning for the automatic definition of planning knowledge. It has been organized according to the target of the learning process: automatic definition of planning action models and automatic definition of planning control knowledge. In addition, the paper reviews the advances in the related field of reinforcement learning.
unknown title
"... Characters in realtime computer games need to move smoothly and thus need to search in real time. In this paper, we describe a simple but powerful way of speeding up repeated A * searches with the same goal states, namely by updating the heuristics between A* searches. We then use this technique to ..."
Abstract
 Add to MetaCart
(Show Context)
Characters in realtime computer games need to move smoothly and thus need to search in real time. In this paper, we describe a simple but powerful way of speeding up repeated A * searches with the same goal states, namely by updating the heuristics between A* searches. We then use this technique to develop a novel realtime heuristic search method, called RealTime Adaptive A*, which is able to choose its local search spaces in a finegrained way. It updates the values of all states in its local search spaces and can do so very quickly. Our experimental results for characters in realtime computer games that need to move to given goal coordinates in unknown terrain demonstrate that this property allows RealTime Adaptive A * to follow trajectories of smaller cost for given time limits per search episode than a recently proposed realtime heuristic search method [5] that is more difficult to implement.
unknown title
"... Characters in realtime computer games need to move smoothly and thus need to search in real time. In this paper, we describe a simple but powerful way of speeding up repeated A * searches with the same goal states, namely by updating the heuristics between A * searches. We then use this technique t ..."
Abstract
 Add to MetaCart
(Show Context)
Characters in realtime computer games need to move smoothly and thus need to search in real time. In this paper, we describe a simple but powerful way of speeding up repeated A * searches with the same goal states, namely by updating the heuristics between A * searches. We then use this technique to develop a novel realtime heuristic search method, called RealTime Adaptive A*, which is able to choose its local search spaces in a finegrained way. It updates the values of all states in its local search spaces and can do so very quickly. Our experimental results for characters in realtime computer games that need to move to given goal coordinates in unknown terrain demonstrate that this property allows RealTime Adaptive A * to follow trajectories of smaller cost for given time limits per search episode than a recently proposed realtime heuristic search method (Koenig 2004) that is more difficult to implement.