Results 1  10
of
10
Abstraction Pathologies in Extensive Games
"... Extensive games can be used to model many scenarios in which multiple agents interact with an environment. There has been considerable recent research on finding strong strategies in very large, zerosum extensive games. The standard approach in such work is to employ abstraction techniques to deriv ..."
Abstract

Cited by 28 (19 self)
 Add to MetaCart
Extensive games can be used to model many scenarios in which multiple agents interact with an environment. There has been considerable recent research on finding strong strategies in very large, zerosum extensive games. The standard approach in such work is to employ abstraction techniques to derive a more tractably sized game. An extensive game solver is then employed to compute an equilibrium in that abstract game, and the resulting strategy is presumed to be strong in the full game. Progress in this line of research has focused on solving larger abstract games, which more closely resemble the full game. However, there is an underlying assumption that by abstracting less, and solving a larger game, an agent will have a stronger strategy in the full game. In this work we show that this assumption is not true in general. Refining an abstraction can actually lead to a weaker strategy. We show examples of these abstraction pathologies in a small game of poker that can be analyzed exactly. These examples show that pathologies arise when abstracting both chance nodes as well as a player’s actions. In summary, this paper shows that the standard approach to finding strong strategies for large extensive games rests on shaky ground.
Using Counterfactual Regret Minimization to Create Competitive Multiplayer Poker Agents
, 2010
"... Games are used to evaluate and advance Multiagent and Artificial Intelligence techniques. Most of these games are deterministic with perfect information (e.g. Chess and Checkers). A deterministic game has no chance element and in a perfect information game, all information is visible to all players. ..."
Abstract

Cited by 14 (4 self)
 Add to MetaCart
Games are used to evaluate and advance Multiagent and Artificial Intelligence techniques. Most of these games are deterministic with perfect information (e.g. Chess and Checkers). A deterministic game has no chance element and in a perfect information game, all information is visible to all players. However, many realworld scenarios with competing agents are stochastic (nondeterministic) with imperfect information. For twoplayer zerosum perfect recall games, a recent technique called Counterfactual Regret Minimization (CFR) computes strategies that are provably convergent to an εNash equilibrium. A Nash equilibrium strategy is useful in twoplayer games since it maximizes its utility against a worstcase opponent. However, for multiplayer (three or more player) games, we lose all theoretical guarantees for CFR. However, we believe that CFRgenerated
The State of Solving Large IncompleteInformation Games, and Application to Poker
, 2010
"... Gametheoretic solution concepts prescribe how rational parties should act, but to become operational the concepts need to be accompanied by algorithms. I will review the state of solving incompleteinformation games. They encompass many practical problems such as auctions, negotiations, and securi ..."
Abstract

Cited by 10 (3 self)
 Add to MetaCart
Gametheoretic solution concepts prescribe how rational parties should act, but to become operational the concepts need to be accompanied by algorithms. I will review the state of solving incompleteinformation games. They encompass many practical problems such as auctions, negotiations, and security applications. I will discuss them in the context of how they have transformed computer poker. In short, gametheoretic reasoning now scales to many large problems, outperforms the alternatives on those problems, and in some games beats the best humans.
Computing Equilibria in Multiplayer Stochastic Games of Imperfect Information
"... Computing a Nash equilibrium in multiplayer stochastic games is a notoriously difficult problem. Prior algorithms have been proven to converge in extremely limited settings and have only been tested on small problems. In contrast, we recently presented an algorithm for computing approximate jam/fold ..."
Abstract

Cited by 7 (3 self)
 Add to MetaCart
Computing a Nash equilibrium in multiplayer stochastic games is a notoriously difficult problem. Prior algorithms have been proven to converge in extremely limited settings and have only been tested on small problems. In contrast, we recently presented an algorithm for computing approximate jam/fold equilibrium strategies in a threeplayer nolimit Texas hold’em tournament—a very large realworld stochastic game of imperfect information [5]. In this paper we show that it is possible for that algorithm to converge to a nonequilibrium strategy profile. However, we develop an ex post procedure that determines exactly how much each player can gain by deviating from his strategy and confirm that the strategies computed in that paper actually do constitute an ɛequilibrium for a very small ɛ (0.5% of the tournament entry fee). Next, we develop several new algorithms for computing a Nash equilibrium in multiplayer stochastic games (with perfect or imperfect information) which can provably never converge to a nonequilibrium. Experiments show that one of these algorithms outperforms the original algorithm on the same poker tournament. In short, we present the first algorithms for provably computing an ɛequilibrium of a large stochastic game for small ɛ. Finally, we present an efficient algorithm that minimizes external regret in both the perfect and imperfect information cases.
Approximation guarantees for fictitious play
 In Proceedings of the 47th Annual Allerton Conference on Communication, Control, and Computing
, 2009
"... Abstract—Fictitious play is a simple, wellknown, and oftenused algorithm for playing (and, especially, learning to play) games. However, in general it does not converge to equilibrium; even when it does, we may not be able to run it to convergence. Still, we may obtain an approximate equilibrium. I ..."
Abstract

Cited by 3 (0 self)
 Add to MetaCart
Abstract—Fictitious play is a simple, wellknown, and oftenused algorithm for playing (and, especially, learning to play) games. However, in general it does not converge to equilibrium; even when it does, we may not be able to run it to convergence. Still, we may obtain an approximate equilibrium. In this paper, we study the approximation properties that fictitious play obtains when it is run for a limited number of rounds. We show that if both players randomize uniformly over their actions in the first r rounds of fictitious play, then the result is an ǫequilibrium, where ǫ = (r + 1)/(2r). (Since we are examining only a constant number of pure strategies, we know that ǫ < 1/2 is impossible, due to a result of Feder et al.) We show that this bound is tight in the worst case; however, with an experiment on random games, we illustrate that fictitious play usually obtains a much better approximation. We then consider the possibility that the players fail to choose the same r. We show how to obtain the optimal approximation guarantee when both the opponent’s r and the game are adversarially chosen (but there is an upper bound R on the opponent’s r), using a linear program formulation. We show that if the action played in the ith round of fictitious play is chosen with probability proportional to: 1 for i = 1 and 1/(i −1) for all 2 ≤ i ≤ R+1, this gives an approximation guarantee of 1 − 1/(2 + ln R). We also obtain a lower bound of 1 − 4/ln R. This provides an actionable prescription for how long to run fictitious play. I.
On Strategy Stitching in Large Extensive Form Multiplayer Games
, 2011
"... Computing a good strategy in a large extensive form game often demands an extraordinary amount of computer memory, necessitating the use of abstraction to reduce the game size. Typically, strategies from abstract games perform better in the real game as the granularity of abstraction is increased. T ..."
Abstract

Cited by 3 (1 self)
 Add to MetaCart
Computing a good strategy in a large extensive form game often demands an extraordinary amount of computer memory, necessitating the use of abstraction to reduce the game size. Typically, strategies from abstract games perform better in the real game as the granularity of abstraction is increased. This paper investigates two techniques for stitching a base strategy in a coarse abstraction of the full game tree, to expert strategies in fine abstractions of smaller subtrees. We provide a general framework for creating static experts, an approach that generalizes some previous strategy stitching efforts. In addition, we show that static experts can create strong agents for both 2player and 3player Leduc and Limit Texas Hold’em poker, and that a specific class of static experts can be preferred among a number of alternatives. Furthermore, we describe a poker agent that used static experts and won the 3player events of the 2010 Annual Computer Poker Competition.
Algorithms for abstracting and solving imperfect information games
, 2007
"... Game theory is the mathematical study of rational behavior in strategic environments. In many settings, most notably twoperson zerosum games, game theory provides particularly strong and appealing solution concepts. Furthermore, these solutions are efficiently computable in the complexitytheory s ..."
Abstract

Cited by 2 (1 self)
 Add to MetaCart
Game theory is the mathematical study of rational behavior in strategic environments. In many settings, most notably twoperson zerosum games, game theory provides particularly strong and appealing solution concepts. Furthermore, these solutions are efficiently computable in the complexitytheory sense. However, in most interesting potential applications in artificial intelligence, the solutions are difficult to compute using current techniques due primarily to the extremely large statespaces of the environments. In this thesis, we propose new algorithms for tackling these computational difficulties. In one stream of research, we introduce automated abstraction algorithms for sequential games of imperfect information. These algorithms take as input a description of a game and produce a description of a strategically similar, but smaller, game as output. We present algorithms that are lossless (i.e., equilibriumpreserving), as well as algorithms that are lossy, but which can yield much smaller games while still retaining the most important features of the original game. In a second stream of research, we develop specialized optimization algorithms for finding ɛequilibria in sequential games of imperfect information. The algorithms are based on recent advances in nonsmooth convex optimization (namely the excessive gap technique) and provide significant improvements
Tartanian5: A HeadsUp NoLimit Texas Hold’em PokerPlaying Program
, 2012
"... We present an overview of Tartanian5, a nolimit Texas Hold’em agent which we submitted to the 2012 Annual Computer Poker Competition. The agent plays a gametheoretic approximate Nash equilibrium strategy. First, it applies a potentialaware, perfectrecall, automated abstraction algorithm to group ..."
Abstract

Cited by 2 (2 self)
 Add to MetaCart
We present an overview of Tartanian5, a nolimit Texas Hold’em agent which we submitted to the 2012 Annual Computer Poker Competition. The agent plays a gametheoretic approximate Nash equilibrium strategy. First, it applies a potentialaware, perfectrecall, automated abstraction algorithm to group similar game states together and construct a smaller game that is strategically similar to the full game. In order to maintain a tractable number of possible betting sequences, it employs a discretized betting model, where only a small number of bet sizes are allowed at each game state. The strategies for both players are then computed using an improved version of Nesterov’s excessive gap technique specialized for poker. To mitigate the effect of overfitting, we employ an expost purification procedure to remove actions that are played with small probability. One final feature of our agent is a novel algorithm for interpreting bet sizes of the opponent that fall outside our model. We describe our new approach in detail, and present theoretical and empirical advantages over prior approaches. Finally, we briefly describe ongoing research in our group involving realtime computation and opponent exploitation, which will hopefully be incorporated into our agents in future years.
JID:ARTINT AID:2578 /REV [m3G; v 1.51; Prn:28/01/2011; 9:50] P.1 (130) Artificial Intelligence •• • (••••) •••–•••
"... Contents lists available at ScienceDirect Artificial Intelligence www.elsevier.com/locate/artint ..."
Abstract
 Add to MetaCart
Contents lists available at ScienceDirect Artificial Intelligence www.elsevier.com/locate/artint