Results 1  10
of
35
Regret minimization in games with incomplete information
, 2007
"... Extensive games are a powerful model of multiagent decisionmaking scenarios with incomplete information. Finding a Nash equilibrium for very large instances of these games has received a great deal of recent attention. In this paper, we describe a new technique for solving large games based on regr ..."
Abstract

Cited by 58 (18 self)
 Add to MetaCart
Extensive games are a powerful model of multiagent decisionmaking scenarios with incomplete information. Finding a Nash equilibrium for very large instances of these games has received a great deal of recent attention. In this paper, we describe a new technique for solving large games based on regret minimization. In particular, we introduce the notion of counterfactual regret, which exploits the degree of incomplete information in an extensive game. We show how minimizing counterfactual regret minimizes overall regret, and therefore in selfplay can be used to compute a Nash equilibrium. We demonstrate this technique in the domain of poker, showing we can solve abstractions of limit Texas Hold’em with as many as 10 12 states, two orders of magnitude larger than previous methods. 1
Potentialaware automated abstraction of sequential games, and holistic equilibrium analysis of Texas Hold’em poker
 IN AAAI’07
, 2007
"... We present a new abstraction algorithm for sequential imperfect information games. While most prior abstraction algorithms employ a myopic expectedvalue computation as a similarity metric, our algorithm considers a higherdimensional space consisting of histograms over abstracted classes of states f ..."
Abstract

Cited by 37 (11 self)
 Add to MetaCart
We present a new abstraction algorithm for sequential imperfect information games. While most prior abstraction algorithms employ a myopic expectedvalue computation as a similarity metric, our algorithm considers a higherdimensional space consisting of histograms over abstracted classes of states from later stages of the game. This enables our bottomup abstraction algorithm to automatically take into account potential: a hand can become relatively better (or worse) over time and the strength of different hands can get resolved earlier or later in the game. We further improve the abstraction quality by making multiple passes over the abstraction, enabling the algorithm to narrow the scope of analysis to information that is relevant given abstraction decisions made for earlier parts of the game. We also present a custom indexing scheme based on suit isomorphisms that enables one to work on significantly larger models than before. We apply the techniques to headsup limit Texas Hold’em poker. Whereas all prior game theorybased work for Texas Hold’em poker used generic offtheshelf linear program solvers for the equilibrium analysis of the abstracted game, we make use of a recently developed algorithm based on the excessive gap technique from convex optimization. This paper is, to our knowledge, the first to abstract and gametheoretically analyze all four betting rounds in one run (rather than splitting the game into phases). The resulting player, GS3, beats BluffBot, GS2, Hyperborean, MonashBPP, Sparbot, Teddy, and Vexbot, each with statistical significance. To our knowledge, those competitors are the best prior programs for the game.
Gradientbased algorithms for finding nash equilibria in extensive form games
 In Proceedings of the Eighteenth International Conference on Game Theory
, 2007
"... We present a computational approach to the saddlepoint formulation for the Nash equilibria of twoperson, zerosum sequential games of imperfect information. The algorithm is a firstorder gradient method based on modern smoothing techniques for nonsmooth convex optimization. The algorithm requires ..."
Abstract

Cited by 32 (13 self)
 Add to MetaCart
We present a computational approach to the saddlepoint formulation for the Nash equilibria of twoperson, zerosum sequential games of imperfect information. The algorithm is a firstorder gradient method based on modern smoothing techniques for nonsmooth convex optimization. The algorithm requires O(1/ɛ) iterations to compute an ɛequilibrium, and the work per iteration is extremely low. These features enable us to find approximate Nash equilibria for sequential games with a tree representation of about 10 10 nodes. This is three orders of magnitude larger than what previous algorithms can handle. We present two heuristic improvements to the basic algorithm and demonstrate their efficacy on a range of realworld games. Furthermore, we demonstrate how the algorithm can be customized to a specific class of problems with enormous memory savings. 1
Computing robust counterstrategies
 In Proceedings of the Annual Conference on Neural Information Processing Systems (NIPS
, 2007
"... Adaptation to other initially unknown agents often requires computing an effective counterstrategy. In the Bayesian paradigm, one must find a good counterstrategy to the inferred posterior of the other agents ’ behavior. In the experts paradigm, one may want to choose experts that are good counter ..."
Abstract

Cited by 25 (6 self)
 Add to MetaCart
Adaptation to other initially unknown agents often requires computing an effective counterstrategy. In the Bayesian paradigm, one must find a good counterstrategy to the inferred posterior of the other agents ’ behavior. In the experts paradigm, one may want to choose experts that are good counterstrategies to the other agents ’ expected behavior. In this paper we introduce a technique for computing robust counterstrategies for adaptation in multiagent scenarios under a variety of paradigms. The strategies can take advantage of a suspected tendency in the decisions of the other agents, while bounding the worstcase performance when the tendency is not observed. The technique involves solving a modified game, and therefore can make use of recently developed algorithms for solving very large extensive games. We demonstrate the effectiveness of the technique in twoplayer Texas Hold’em. We show that the computed poker strategies are substantially more robust than best response counterstrategies, while still exploiting a suspected tendency. We also compose the generated strategies in an experts algorithm showing a dramatic improvement in performance over using simple best responses. 1
Better automated abstraction techniques for imperfect information games, with application to Texas Hold’em poker
 In International Conference on Autonomous Agents and MultiAgent Systems (AAMAS
, 2007
"... We present new approximation methods for computing gametheoretic strategies for sequential games of imperfect information. At a high level, we contribute two new ideas. First, we introduce a new statespace abstraction algorithm. In each round of the game, there is a limit to the number of strategic ..."
Abstract

Cited by 24 (8 self)
 Add to MetaCart
We present new approximation methods for computing gametheoretic strategies for sequential games of imperfect information. At a high level, we contribute two new ideas. First, we introduce a new statespace abstraction algorithm. In each round of the game, there is a limit to the number of strategically different situations that an equilibriumfinding algorithm can handle. Given this constraint, we use clustering to discover similar positions, and we compute the abstraction via an integer program that minimizes the expected error at each stage of the game. Second, we present a method for computing the leaf payoffs for a truncated version of the game by simulating the actions in the remaining portion of the game. This allows the equilibriumfinding algorithm to take into account the entire game tree while having to explicitly solve only a truncated version. Experiments show that each of our two new techniques improves performance dramatically in Texas Hold’em poker. The techniques lead to a drastic improvement over prior approaches for automatically generating agents, and our agent plays competitively even against the best agents overall.
Lossless abstraction of imperfect information games
 Journal of the ACM
, 2007
"... Abstract. Finding an equilibrium of an extensive form game of imperfect information is a fundamental problem in computational game theory, but current techniques do not scale to large games. To address this, we introduce the ordered game isomorphism and the related ordered game isomorphic abstractio ..."
Abstract

Cited by 21 (9 self)
 Add to MetaCart
Abstract. Finding an equilibrium of an extensive form game of imperfect information is a fundamental problem in computational game theory, but current techniques do not scale to large games. To address this, we introduce the ordered game isomorphism and the related ordered game isomorphic abstraction transformation. For a multiplayer sequential game of imperfect information with observable actions and an ordered signal space, we prove that any Nash equilibrium in an abstracted smaller game, obtained by one or more applications of the transformation, can be easily converted into a Nash equilibrium in the original game. We present an algorithm, GameShrink, for abstracting the game using our isomorphism exhaustively. Its complexity is Õ(n2), where n is the number of nodes in a structure we call the signal tree. It is no larger than the game tree, and on nontrivial games it is drastically smaller, so GameShrink has time and space complexity sublinear in the size of the game tree. Using GameShrink, we find an equilibrium to a poker game with 3.1 billion nodes—over four orders of magnitude more than in the largest poker game solved previously. To address even larger games, we introduce approximation methods that do not preserve equilibrium, but nevertheless yield (ex post) provably closetooptimal strategies.
Effective ShortTerm Opponent Exploitation in Simplified Poker
 In Proceedings of the National Conference on Artificial Intelligence (AAAI
, 2005
"... Uncertainty in poker stems from two key sources, the shuffled deck and an adversary whose strategy is unknown. One approach to playing poker is to find a pessimistic gametheoretic solution (i.e., a Nash equilibrium), but human players have idiosyncratic weaknesses that can be exploited if some mode ..."
Abstract

Cited by 19 (0 self)
 Add to MetaCart
Uncertainty in poker stems from two key sources, the shuffled deck and an adversary whose strategy is unknown. One approach to playing poker is to find a pessimistic gametheoretic solution (i.e., a Nash equilibrium), but human players have idiosyncratic weaknesses that can be exploited if some model or counterstrategy can be learned by observing their play. However, games against humans last for at most a few hundred hands, so learning must be very fast to be useful. We explore two approaches to opponent modelling in the context of Kuhn poker, a small game for which gametheoretic solutions are known. Parameter estimation and expert algorithms are both studied. Experiments demonstrate that, even in this small game, convergence to maximally exploitive solutions in a small number of hands is impractical, but that good (e.g., better than Nash) performance can be achieved in as few as 50 hands. Finally, we show that amongst a set of strategies with equal gametheoretic value, in particular the set of Nash equilibrium strategies, some are preferable because they speed learning of the opponent’s strategy by exploring it more effectively. 1
Computing an Approximate Jam/Fold Equilibrium for 3player NoLimit Texas Hold’em Tournaments
, 2008
"... A recent paper computes nearoptimal strategies for twoplayer nolimit Texas hold’em tournaments; however, the techniques used are unable to compute equilibrium strategies for tournaments with more than two players. Motivated by the widespread popularity of multiplayer tournaments and the observatio ..."
Abstract

Cited by 11 (3 self)
 Add to MetaCart
A recent paper computes nearoptimal strategies for twoplayer nolimit Texas hold’em tournaments; however, the techniques used are unable to compute equilibrium strategies for tournaments with more than two players. Motivated by the widespread popularity of multiplayer tournaments and the observation that jam/fold strategies are nearoptimal in the two player case, we develop an algorithm that computes approximate jam/fold equilibrium strategies in tournaments with three — and potentially even more — players. Our algorithm combines an extension of fictitious play to imperfect information games, an algorithm similar to value iteration for solving stochastic games, and a heuristic from the poker community known as the Independent Chip Model which we use as an initialization. Several ways of exploiting suit symmetries and the use of custom indexing schemes made the approach computationally feasible. Aside from the initialization and the restriction to jam/fold strategies, our high level algorithm makes no pokerspecific assumptions and thus also applies to other multiplayer stochastic games of imperfect information.
A New Algorithm for Generating Equilibria in Massive ZeroSum Games
, 2007
"... In normal scenarios, computer scientists often consider the number of states in a game to capture the difficulty of learning an equilibrium. However, players do not see games in the same light: most consider Go or Chess to be more complex than Monopoly. In this paper, we discuss a new measure of gam ..."
Abstract

Cited by 11 (2 self)
 Add to MetaCart
In normal scenarios, computer scientists often consider the number of states in a game to capture the difficulty of learning an equilibrium. However, players do not see games in the same light: most consider Go or Chess to be more complex than Monopoly. In this paper, we discuss a new measure of game complexity that links existing stateoftheart algorithms for computing approximate equilibria to a more human measure. In particular, we consider the range of skill in a game, i.e.how many different skill levels exist. We then modify existing techniques to design a new algorithm to compute approximate equilibria whose performance can be captured by this new measure. We use it to develop the first near Nash equilibrium for a four round abstraction of poker, and show that it would have been able to win handily the bankroll competition from last year’s AAAI poker competition.