Results 1  10
of
18
State abstraction in realtime heuristic search
 Journal of Artificial Intelligence Research
, 2006
"... Realtime heuristic search methods are used by situated agents in applications that require the amount of planning per move to be constantbounded regardless of the problem size. Such agents plan only a few actions in a local search space and avoid getting trapped in heuristic local minima by improv ..."
Abstract

Cited by 33 (10 self)
 Add to MetaCart
(Show Context)
Realtime heuristic search methods are used by situated agents in applications that require the amount of planning per move to be constantbounded regardless of the problem size. Such agents plan only a few actions in a local search space and avoid getting trapped in heuristic local minima by improving their heuristic function over time. We extend a wide class of realtime search algorithms with automatically built graph abstraction. Extensive empirical evaluation in the domain of goaldirected navigation demonstrates that the use of abstraction accelerates learning of the heuristic function while maintaining realtime performance. The resulting algorithm outperforms virtually all tested algorithms simultaneously along negatively correlated performance measures.
TBA*: TimeBounded A * ∗
"... Realtime heuristic search algorithms are used for planning by agents in situations where a constantbounded amount of deliberation time is required for each action regardless of the problem size. Such algorithms interleave their planning and execution to ensure realtime response. Furthermore, to gu ..."
Abstract

Cited by 19 (4 self)
 Add to MetaCart
Realtime heuristic search algorithms are used for planning by agents in situations where a constantbounded amount of deliberation time is required for each action regardless of the problem size. Such algorithms interleave their planning and execution to ensure realtime response. Furthermore, to guarantee completeness, they typically store improved heuristic estimates for previously expanded states. Although subsequent planning steps can benefit from updated heuristic estimates, many of the same states are expanded over and over again. Here we propose a variant of the A * algorithm, TimeBounded A * (TBA*), that guarantees realtime response. In the domain of pathfinding on videogame maps TBA * expands an order of magnitude fewer states than traditional realtime search algorithms, while finding paths of comparable quality. It reaches the same level of performance as recent stateoftheart realtime search algorithms but, unlike these, requires neither statespace abstractions nor precomputed pattern databases. 1
Casebased subgoaling in realtime heuristic search for video game pathfinding
 J. Artif. Intell. Res
, 2010
"... Realtime heuristic search algorithms satisfy a constant bound on the amount of planning per action, independent of problem size. As a result, they scale up well as problems become larger. This property would make them well suited for video games where Artificial Intelligence controlled agents must ..."
Abstract

Cited by 17 (4 self)
 Add to MetaCart
(Show Context)
Realtime heuristic search algorithms satisfy a constant bound on the amount of planning per action, independent of problem size. As a result, they scale up well as problems become larger. This property would make them well suited for video games where Artificial Intelligence controlled agents must react quickly to user commands and to other agents ’ actions. On the downside, realtime search algorithms employ learning methods that frequently lead to poor solution quality and cause the agent to appear irrational by revisiting the same problem states repeatedly. The situation changed recently with a new algorithm, D LRTA*, which attempted to eliminate learning by automatically selecting subgoals. D LRTA * is well poised for video games, except it has a complex and memorydemanding precomputation phase during which it builds a database of subgoals. In this paper, we propose a simpler and more memoryefficient way of precomputing subgoals thereby eliminating the main obstacle to applying stateoftheart realtime search methods in video games. The new algorithm solves a number of randomly chosen problems offline, compresses the solutions into a series of subgoals and stores them in a database. When presented with a novel problem online, it queries the database for the most similar previously solved case and uses its subgoals to solve the problem. In the domain of pathfinding on four large video game maps, the new algorithm delivers solutions eight times better while using 57 times less memory and requiring 14 % less precomputation time. 1.
Dynamic control in realtime heuristic search
, 2008
"... Realtime heuristic search is a challenging type of agentcentered search because the agent’s planning time per action is bounded by a constant independent of problem size. A common problem that imposes such restrictions is pathfinding in modern computer games where a large number of units must plan ..."
Abstract

Cited by 16 (11 self)
 Add to MetaCart
Realtime heuristic search is a challenging type of agentcentered search because the agent’s planning time per action is bounded by a constant independent of problem size. A common problem that imposes such restrictions is pathfinding in modern computer games where a large number of units must plan their paths simultaneously over large maps. Common search algorithms (e.g., A*, IDA*, D*, ARA*, AD*) are inherently not realtime and may lose completeness when a constant bound is imposed on peraction planning time. Realtime search algorithms retain completeness but frequently produce unacceptably suboptimal solutions. In this paper, we extend classic and modern realtime search algorithms with an automated mechanism for dynamic depth and subgoal selection. The new algorithms remain realtime and complete. On large computer game maps, they find paths within 7 % of optimal while on average expanding roughly a single state per action. This is nearly a threefold improvement in suboptimality over the existing stateoftheart algorithms and, at the same time, a 15fold improvement in the amount of planning per action.
Dynamic Control in PathPlanning with RealTime Heuristic Search
"... Realtime heuristic search methods, such as LRTA*, are used by situated agents in applications that require the amount of planning per action to be constantbounded regardless of the problem size. LRTA * interleaves planning and execution, with a fixed search depth being used to achieve progress tow ..."
Abstract

Cited by 15 (4 self)
 Add to MetaCart
(Show Context)
Realtime heuristic search methods, such as LRTA*, are used by situated agents in applications that require the amount of planning per action to be constantbounded regardless of the problem size. LRTA * interleaves planning and execution, with a fixed search depth being used to achieve progress towards a fixed goal. Here we generalize the algorithm to allow for a dynamically changing search depth and a dynamically changing (sub)goal. Evaluation in pathplanning on videogame maps shows that the new algorithm significantly outperforms fixeddepth, fixedgoal LRTA*. The new algorithm can achieve the same quality solutions as LRTA*, but with nine times less computation, or use the same amount of computation, but produce four times better quality solutions. These extensions make realtime heuristic search a practical choice for pathplanning in computer videogames.
Avoiding and Escaping Depressions in RealTime Heuristic Search
"... Heuristics used for solving hard realtime search problems have regions with depressions. Such regions are bounded areas of the search space in which the heuristic function is inaccurate compared to the actual cost to reach a solution. Early realtime search algorithms, like LRTA ∗ , easily become t ..."
Abstract

Cited by 11 (3 self)
 Add to MetaCart
(Show Context)
Heuristics used for solving hard realtime search problems have regions with depressions. Such regions are bounded areas of the search space in which the heuristic function is inaccurate compared to the actual cost to reach a solution. Early realtime search algorithms, like LRTA ∗ , easily become trapped in those regions since the heuristic values of their states may need to be updated multiple times, which results in costly solutions. Stateoftheart realtime search algorithms, like LSSLRTA ∗ or LRTA ∗ (k), improve LRTA ∗ ’s mechanism to update the heuristic, resulting in improved performance. Those algorithms, however, do not guide search towards avoiding depressed regions. This paper presents depression avoidance, a simple realtime search principle to guide search towards avoiding states that have been marked as part of a heuristic depression. We propose two ways in which depression avoidance can be implemented: markandavoid and movetoborder. We implement these strategies on top of LSSLRTA ∗ and RTAA ∗ , producing 4 new realtime heuristic search algorithms: aLSSLRTA ∗ , daLSSLRTA ∗ , aRTAA ∗ , and daRTAA ∗. When the objective is to find a single solution by running the realtime search algorithm once, we show that daLSSLRTA ∗ and daRTAA ∗ outperform their predecessors sometimes by one order of magnitude. Of the four new algorithms, daRTAA ∗ produces the best solutions given a fixed deadline on the average time allowed per planning episode. We prove all our algorithms have good theoretical properties: in finite search spaces, they find a solution if one exists, and converge to an optimal after a number of trials. 1.
Prioritizing Bellman backups without a priority queue
 In Proc. of the 17th International Conference on Automated Planning and Scheduling (ICAPS07), this volumn
"... Several researchers have shown that the efficiency of value iteration, a dynamic programming algorithm for Markov decision processes, can be improved by prioritizing the order of Bellman backups to focus computation on states where the value function can be improved the most. In previous work, a pri ..."
Abstract

Cited by 10 (1 self)
 Add to MetaCart
(Show Context)
Several researchers have shown that the efficiency of value iteration, a dynamic programming algorithm for Markov decision processes, can be improved by prioritizing the order of Bellman backups to focus computation on states where the value function can be improved the most. In previous work, a priority queue has been used to order backups. Although this incurs overhead for maintaining the priority queue, previous work has argued that the overhead is usually much less than the benefit from prioritization. However this conclusion is usually based on a comparison to a nonprioritized approach that performs Bellman backups on states in an arbitrary order. In this paper, we show that the overhead for maintaining the priority queue can be greater than the benefit, when it is compared to very simple heuristics for prioritizing backups that do not require a priority queue. Although the order of backups induced by our simple approach is often suboptimal, we show that its smaller overhead allows it to converge faster than other stateoftheart prioritybased solvers.
On Learning In AgentCentered Search
"... Since the introduction of the LRTA * algorithm, realtime heuristic search algorithms have generally followed the same planactlearn cycle: an agent plans one or several actions based on locally available information, executes them and then updates (i.e., learns) its heuristic function. Algorithm ev ..."
Abstract

Cited by 5 (2 self)
 Add to MetaCart
(Show Context)
Since the introduction of the LRTA * algorithm, realtime heuristic search algorithms have generally followed the same planactlearn cycle: an agent plans one or several actions based on locally available information, executes them and then updates (i.e., learns) its heuristic function. Algorithm evaluation has almost exclusively been empirical with the results often being domainspecific and incomparable across papers. Even when unification and crossalgorithm comparisons have been carried out in a single paper, there was no understanding of how efficient the learning process was with respect to a theoretical optimum. This paper addresses the problem with two primary contributions. First, we formally define a lower bound on the amount of learning any heuristiclearning algorithm needs to do. This bound is based on the notion of heuristic depressions and allows us to have a domainindependent measure of learning efficiency across different algorithms. Second, using this measure we propose to learn “costssofar ” (gcosts) instead of “coststogo ” (hcosts). This allows us to quickly identify redundant paths and deadend states, thereby leading to asymptotic performance improvement as well as 12 orders of magnitude convergence speedups in practice.
RealTime A* Search With Depthk Lookahead
, 2009
"... We consider realtime planning problems in which some states are unsolvable, i.e., have no path to a goal. Such problems are difficult for realtime planning algorithms such as RTA* in which all states must be solvable. We identify a property called ksafeness, in which the consequences of a bad cho ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
(Show Context)
We consider realtime planning problems in which some states are unsolvable, i.e., have no path to a goal. Such problems are difficult for realtime planning algorithms such as RTA* in which all states must be solvable. We identify a property called ksafeness, in which the consequences of a bad choice become apparent within k moves after the choice is made. When k is not too large, this makes it possible to identify unsolvable states in real time. We provide a modified version of RTA* that is provably complete on all ksafe problems, and we show that realtime deterministic versions of the wellknown Tireworld and Racetrack domains are ksafe.
Acknowledgements
, 2009
"... on climate change and the environmentTowards a Global Green Recovery ..."