Artificial intelligence; algorithmic probability; sequential decision theory; rational

Abstract. Finding an equilibrium of an extensive form game of imperfect information is a fundamental problem in computational game theory, but current techniques do not scale to large games. To address this, we introduce the ordered game isomorphism and the related ordered game isomorphic abstraction transformation. For a multi-player sequential game of imperfect information with observable actions and an ordered signal space, we prove that any Nash equilibrium in an abstracted smaller game, obtained by one or more applications of the transformation, can be easily converted into a Nash equilibrium in the original game. We present an algorithm, GameShrink, for abstracting the game using our isomorphism exhaustively. Its complexity is Õ(n2), where n is the number of nodes in a structure we call the signal tree. It is no larger than the game tree, and on nontrivial games it is drastically smaller, so GameShrink has time and space complexity sublinear in the size of the game tree. Using GameShrink, we find an equilibrium to a poker game with 3.1 billion nodes—over four orders of magnitude more than in the largest poker game solved previously. To address even larger games, we introduce approximation methods that do not preserve equilibrium, but nevertheless yield (ex post) provably close-to-optimal strategies.

There is a growing interest in producing story based game experiences that do not follow fixed scripts predefined by the author, but change the experience based on actions performed by the player during his interaction. In order to achieve this objective, previous approaches have employed a drama management component that produces a narratively pleasing arc based on an author specified aesthetic value of a story, ignoring a player’s personal preference for that story path. Furthermore, previous approaches have used a simulated player model to assess their approach, ignoring real human players interacting with the story based game. This paper presents an approach that uses a case based player preference modeling component that predicts an interestingness value for a particular plot point within the story. These interestingness values are based on real human players ’ interactions with the story. We also present a drama manager that uses a search process (based on the expectimax algorithm) and combines the author specified aesthetic values with the player model.

While much of the research done in heuristic search has concentrated on deterministic domains, not much work has been done to investigate search techniques in stochastic domains other than statistical sampling methods. When full search is required, Expectimax is often the algorithm of choice. However, Expectimax is a full-width search algorithm. A class of algorithms called *-Minimax were developed by Bruce Ballard to improve on Expectimax's runtime. They allow for cutoffs in trees with chance nodes similar to how Alpha-Beta allows for cutoffs in Minimax trees. This thesis presents new performance results for Expectimax, as well as Star1 and Star2 (the two main *-Minimax algorithms), in real-world domains. Ballard's work is verified and new insights into move ordering and probe successor selection are presented.

Abstract. This paper presents the first performance results for Ballard’s *-Minimax algorithms applied to a real–world domain: backgammon. It is shown that with effective move ordering and probing the Star2 algorithm considerably outperforms Expectimax. Star2 allows strong backgammon programs to conduct depth 5 full-width searches (up from 3) under tournament conditions on regular hardware without using risky forward pruning techniques. We also present empirical evidence that with today’s sophisticated evaluation functions good checker play in backgammon does not require deep searches. 1

www.hutter1.net Specialized intelligent systems can be found everywhere: finger print, handwriting, speech, and face recognition, spam filtering, chess and other game programs, robots, et al. This decade the first presumably complete mathematical theory of artificial intelligence based on universal induction-predictiondecision-action has been proposed. This information-theoretic approach solidifies the foundations of inductive inference and artificial intelligence. Getting the foundations right usually marks a significant progress and maturing of a field. The theory provides a gold standard and guidance for researchers working on intelligent algorithms. The roots of universal induction have been laid exactly half-a-century ago and the roots of universal intelligence exactly one decade ago. So it is timely to take stock of what has been achieved and what remains to be done. Since there are already good recent surveys, I describe the state-of-the-art only in passing and refer the reader to the literature.

Abstract. The article describes a gradient search based reinforcement learning algorithm for two-player zerosum games with imperfect information. Simple gradient search may result in oscillation around solution points, a problem similar to the “Crawford puzzle”. To dampen oscillations, the algorithm uses lagging anchors, drawing the strategy state of the players toward a weighted average of earlier strategy states. The algorithm is applicable to games represented in extensive form. We develop methods for sampling the parameter gradient of a player’s performance against an opponent, using temporal-difference learning. The algorithm is used successfully for a simplified poker game with infinite sets of pure strategies, and for the air combat game Campaign, using neural nets. We prove exponential convergence of the algorithm for a subset of matrix games.

Decision theory formally solves the problem of rational agents in uncertain worlds if the true environmental prior probability distribution is known. Solomonoff's theory of universal induction formally solves the problem of sequence prediction for unknown prior distribution. We combine both ideas and get a parameterless theory of universal Artificial Intelligence. We give strong arguments that the resulting AIXI model is the most intelligent unbiased agent possible. We outline for a number of problem classes, including sequence prediction, strategic games, function minimization, reinforcement and supervised learning, how the AIXI model can formally solve them. The major drawback of the AIXI model is that it is uncomputable. To overcome this problem, we construct a modified algorithm AIXItl, which is still effectively more intelligent than any other time t and space l bounded agent. The computation time of AIXItl is of the order t·2^l. Other discussed topics are formal definitions of intelligence order relations, the horizon problem and relations of the AIXI theory to other AI approaches.

A growing research community is working towards employing drama management components in story-based games that guide the story towards specific narrative arcs depending on a particular player’s playing patterns. Intuitively, player modeling should be a key component for Drama Manager (DM) based approaches to succeed with human players. In this paper, we report a particular implementation of the DM component connected to an interactive story game, Anchorhead, while specifically focusing on the player modeling component. We analyze results from our evaluation study and show that similarity in the trace of DM decisions in previous games can be used to predict interestingness of game events for the current player. Results from our current analysis indicate that the average time spent in performing player actions provides a strong distinction between players with varying degrees of gaming experience, thereby helping the DM to adapt its strategy based on this information.

A growing research community is working towards employing drama management components in interactive story-based games. These components gently guide the story towards a narrative arc that improves the player’s experience. However, the success of drama management approaches has not been evaluated using human players in a real game implementation. In this paper, we evaluate our drama management approach deployed in our own implementation of an interactive fiction game Anchorhead. Our approach uses player’s feedback as a basis for guiding the personalization of the interaction. The results indicate that our Drama Manager (DM) aids in providing a better overall experience for the players while guiding them through their interaction. Based on this work, we suggest that the strategies employed by the DM should take into account the player’s previous playing experience with the current game as well as his general game-playing experience.