Computer Go is one of the biggest challenges faced by game programmers. This survey describes the typical components of a Go program, and discusses knowledge representation, search methods and techniques for solving specific subproblems in this domain. Along with a summary of the

. Macro search is used to derive solutions quickly for large search spaces at the expense of optimality. We present a novel way of building macro tables. Our contribution is twofold: (1) for the first time, we use automatically generated heuristics to find optimal macros, (2) due to the speed-up achieved by (1), we merge consecutive subgoals to reduce the solution lengths. We use the Rubik's Cube to demonstrate our techniques. For this puzzle, a 44% improvement of the average solution length was achieved over macro tables built with previous techniques. 1

In this paper we present a production system which acts on fixed length vectors of labels. Our goal is to automatically generate heuristics to search the state space for shortest paths between states efficiently. The heuristic values which guide search in the state space are obtained by searching for the shortest path in an abstract space derived from the definition of the original space. In PSVN, a state is a fixed length vector of labels and abstractions are generated by simply mapping the set of labels to another smaller set of labels (domain abstraction). A domain abstraction on labels induces a state space abstraction and this abstract space preserves important properties of the original space while usually being significantly smaller in size. It is guaranteed that the shortest path between two states in the original space is at least as long as the shortest path between their images in the abstract space. Hence, such abstractions provide admissible heuristics for search algorith...

Abstract. CPU bound client puzzles have been suggested as a defense mechanism against connection depletion attacks. However, the wide disparity in CPU speeds prevents such puzzles from being globally deployed. Recently, Abadi et. al. [1] and Dwork et. al. [2] addressed this limitation by showing that memory access times vary much less than CPU speeds, and hence offer a viable alternative. In this paper, we further investigate the applicability of memory bound puzzles from a new perspective and propose constructions based on heuristic search methods. Our constructions are derived from a more algorithmic foundation, and as a result, allow us to easily tune parameters that impact puzzle creation and verification costs. Moreover, unlike prior approaches, we address client-side cost and present an extension that allows memory constrained clients (e.g., PDAs) to implement our construction in a secure fashion. 1

This thesis demonstrates how the power of symbolic processing can be exploited in the learning of low level control functions. It proposes a novel hybrid architecture with a tight coupling between a variant of symbolic planning and reinforcement learning. This architecture combines the strengths of the function approximation of subsymbolic learning with the more abstract compositional nature of symbolic learning. The former is able to represent mappings of world states to actions in an accurate way. The latter allows a more rapid solution to problems by exploiting structure within the domain. A control function is learnt over time through interaction with the world. Symbols are attached to features in the functions. The symbolic attachments act as anchor points used to transform the function of a previously learnt task to that of a new task. The solution of more complex tasks is achieved through composing simpler functions, using the symbolic attachments to determine the composition. T...

Knowledge Fundamental to artificial intelligence, as well as to the theory of systems and control, is the problem of representing knowledge about the system and about possible courses of action at a multiplicity of interrelated temporal scales. For example, a human traveler must decide 6 which cities to go to, whether to fly, drive, or walk, and the individual muscle contractions involved in each step. We propose to develop further an approach to this problem based on the theory of options [49, 29]. Options are a generic concept of "courses of action" which includes both primitive actions such as muscle contractions and temporally extended actions such as traveling to a distant city. The theory of options is based on the theories of Markov and semi-Markov decision processes (SMDPs), but extends these in significant ways. Options can be used in place of actions in all of the planning and learning methods conventionally used in RL. Options and models of options can be learned for a wide variety of di#erent subtasks, and then rapidly combined to solve new tasks. Options provide a bridge between the two most important existing theoretical frameworks used in reinforcement learning---MDPs and SMDPs. Options permit planning and learning simultaneously at a wide variety of times scales, and toward a wide variety of subtasks, which substantially increases the e#ciency and abilities of RL methods.

Given a set of problems, a set of heuristics, and assuming that everything else is equal, it is never worse and usually better to solve each problem with the heuristic that is best for that problem than to use, for all problems, the heuristic that is best, on average, for the set of problems. Unfortunately, everything else is seldom equal. In particular, the cost of determining which heuristic is best, on average, is usually cheaper than determining for each problem, which is best. But, does this mean that when you include the costs of determining the best heuristic that the former approach (i.e. determining for each problem which is best for it) is always worse, on average, than the latter approach? This is not the case if the average cost of determining the best heuristic, for each problem, is lower than the average cost advantage of using it. However, how low can we bring this cost? This paper introduces two new data structures, minTrees and culprit counter lattices, that lower this cost and then we compare using this approach with these data structures against simply using a known effective compact pattern database variation. The experiments show that the key to success is finding cost-effective predictions of which heuristic will be best for a problem rather than precisely determining which is best. While our experiments do not provide conclusive proof that it is better on average to determine, for each problem, the best heuristic, it does provide evidence that suggests this can be the case.

In this paper, we consider how to improve the scalability and efficiency of target-value path search on directed acyclic graphs. To this end, we introduce a depth-first heuristic search algorithm and a dynamic-programming method to compute the heuristic’s pattern database in linear (in the number of edges) time. We show the benefits of the new approach over previous work on this problem (Kuhn et al. 2008a).