Results 1  10
of
155
Regret minimization in games with incomplete information
, 2007
"... Extensive games are a powerful model of multiagent decisionmaking scenarios with incomplete information. Finding a Nash equilibrium for very large instances of these games has received a great deal of recent attention. In this paper, we describe a new technique for solving large games based on regr ..."
Abstract

Cited by 86 (22 self)
 Add to MetaCart
(Show Context)
Extensive games are a powerful model of multiagent decisionmaking scenarios with incomplete information. Finding a Nash equilibrium for very large instances of these games has received a great deal of recent attention. In this paper, we describe a new technique for solving large games based on regret minimization. In particular, we introduce the notion of counterfactual regret, which exploits the degree of incomplete information in an extensive game. We show how minimizing counterfactual regret minimizes overall regret, and therefore in selfplay can be used to compute a Nash equilibrium. We demonstrate this technique in the domain of poker, showing we can solve abstractions of limit Texas Hold’em with as many as 10 12 states, two orders of magnitude larger than previous methods. 1
A competitive Texas Hold’em poker player via automated abstraction and realtime equilibrium computation
 IN PROCEEDINGS OF THE NATIONAL CONFERENCE ON ARTIFICIAL INTELLIGENCE (AAAI
, 2006
"... We present a game theorybased headsup Texas Hold’em poker player, GS1. To overcome the computational obstacles stemming from Texas Hold’em’s gigantic game tree, the player employs our automated abstraction techniques to reduce the complexity of the strategy computations. Texas Hold’em consists of ..."
Abstract

Cited by 62 (21 self)
 Add to MetaCart
We present a game theorybased headsup Texas Hold’em poker player, GS1. To overcome the computational obstacles stemming from Texas Hold’em’s gigantic game tree, the player employs our automated abstraction techniques to reduce the complexity of the strategy computations. Texas Hold’em consists of four betting rounds. Our player solves a large linear program (offline) to compute strategies for the abstracted first and second rounds. After the second betting round, our player updates the probability of each possible hand based on the observed betting actions in the first two rounds as well as the revealed cards. Using these updated probabilities, our player computes in realtime an equilibrium approximation for the last two abstracted rounds. We demonstrate that our player, which incorporates very little pokerspecific knowledge, is competitive with leading pokerplaying programs which incorporate extensive domain knowledge, as well as with advanced human players.
Potentialaware automated abstraction of sequential games, and holistic equilibrium analysis of Texas Hold’em poker
 IN AAAI’07
, 2007
"... We present a new abstraction algorithm for sequential imperfect information games. While most prior abstraction algorithms employ a myopic expectedvalue computation as a similarity metric, our algorithm considers a higherdimensional space consisting of histograms over abstracted classes of states f ..."
Abstract

Cited by 47 (14 self)
 Add to MetaCart
(Show Context)
We present a new abstraction algorithm for sequential imperfect information games. While most prior abstraction algorithms employ a myopic expectedvalue computation as a similarity metric, our algorithm considers a higherdimensional space consisting of histograms over abstracted classes of states from later stages of the game. This enables our bottomup abstraction algorithm to automatically take into account potential: a hand can become relatively better (or worse) over time and the strength of different hands can get resolved earlier or later in the game. We further improve the abstraction quality by making multiple passes over the abstraction, enabling the algorithm to narrow the scope of analysis to information that is relevant given abstraction decisions made for earlier parts of the game. We also present a custom indexing scheme based on suit isomorphisms that enables one to work on significantly larger models than before. We apply the techniques to headsup limit Texas Hold’em poker. Whereas all prior game theorybased work for Texas Hold’em poker used generic offtheshelf linear program solvers for the equilibrium analysis of the abstracted game, we make use of a recently developed algorithm based on the excessive gap technique from convex optimization. This paper is, to our knowledge, the first to abstract and gametheoretically analyze all four betting rounds in one run (rather than splitting the game into phases). The resulting player, GS3, beats BluffBot, GS2, Hyperborean, MonashBPP, Sparbot, Teddy, and Vexbot, each with statistical significance. To our knowledge, those competitors are the best prior programs for the game.
Gradientbased algorithms for finding nash equilibria in extensive form games
 In Proceedings of the Eighteenth International Conference on Game Theory
, 2007
"... We present a computational approach to the saddlepoint formulation for the Nash equilibria of twoperson, zerosum sequential games of imperfect information. The algorithm is a firstorder gradient method based on modern smoothing techniques for nonsmooth convex optimization. The algorithm requires ..."
Abstract

Cited by 44 (15 self)
 Add to MetaCart
(Show Context)
We present a computational approach to the saddlepoint formulation for the Nash equilibria of twoperson, zerosum sequential games of imperfect information. The algorithm is a firstorder gradient method based on modern smoothing techniques for nonsmooth convex optimization. The algorithm requires O(1/ɛ) iterations to compute an ɛequilibrium, and the work per iteration is extremely low. These features enable us to find approximate Nash equilibria for sequential games with a tree representation of about 10 10 nodes. This is three orders of magnitude larger than what previous algorithms can handle. We present two heuristic improvements to the basic algorithm and demonstrate their efficacy on a range of realworld games. Furthermore, we demonstrate how the algorithm can be customized to a specific class of problems with enormous memory savings. 1
Abstraction Pathologies in Extensive Games
"... Extensive games can be used to model many scenarios in which multiple agents interact with an environment. There has been considerable recent research on finding strong strategies in very large, zerosum extensive games. The standard approach in such work is to employ abstraction techniques to deriv ..."
Abstract

Cited by 43 (25 self)
 Add to MetaCart
(Show Context)
Extensive games can be used to model many scenarios in which multiple agents interact with an environment. There has been considerable recent research on finding strong strategies in very large, zerosum extensive games. The standard approach in such work is to employ abstraction techniques to derive a more tractably sized game. An extensive game solver is then employed to compute an equilibrium in that abstract game, and the resulting strategy is presumed to be strong in the full game. Progress in this line of research has focused on solving larger abstract games, which more closely resemble the full game. However, there is an underlying assumption that by abstracting less, and solving a larger game, an agent will have a stronger strategy in the full game. In this work we show that this assumption is not true in general. Refining an abstraction can actually lead to a weaker strategy. We show examples of these abstraction pathologies in a small game of poker that can be analyzed exactly. These examples show that pathologies arise when abstracting both chance nodes as well as a player’s actions. In summary, this paper shows that the standard approach to finding strong strategies for large extensive games rests on shaky ground.
Bayes’ bluff: Opponent modelling in poker
 In Proceedings of the 21st Annual Conference on Uncertainty in Artificial Intelligence (UAI
, 2005
"... Poker is a challenging problem for artificial intelligence, with nondeterministic dynamics, partial observability, and the added difficulty of unknown adversaries. Modelling all of the uncertainties in this domain is not an easy task. In this paper we present a Bayesian probabilistic model for a br ..."
Abstract

Cited by 40 (3 self)
 Add to MetaCart
(Show Context)
Poker is a challenging problem for artificial intelligence, with nondeterministic dynamics, partial observability, and the added difficulty of unknown adversaries. Modelling all of the uncertainties in this domain is not an easy task. In this paper we present a Bayesian probabilistic model for a broad class of poker games, separating the uncertainty in the game dynamics from the uncertainty of the opponent’s strategy. We then describe approaches to two key subproblems: (i) inferring a posterior over opponent strategies given a prior distribution and observations of their play, and (ii) playing an appropriate response to that distribution. We demonstrate the overall approach on a reduced version of poker using Dirichlet priors and then on the full game of Texas hold’em using a more informed prior. We demonstrate methods for playing effective responses to the opponent, based on the posterior. 1
Smoothing Techniques for Computing Nash Equilibria of Sequential Games
, 2008
"... We develop firstorder smoothing techniques for saddlepoint problems that arise in the Nash equilibria computation of sequential games. The crux of our work is a construction of suitable proxfunctions for a certain class of polytopes that encode the sequential nature of the games. An implementatio ..."
Abstract

Cited by 39 (9 self)
 Add to MetaCart
(Show Context)
We develop firstorder smoothing techniques for saddlepoint problems that arise in the Nash equilibria computation of sequential games. The crux of our work is a construction of suitable proxfunctions for a certain class of polytopes that encode the sequential nature of the games. An implementation based on our smoothing techniques computes approximate Nash equilibria for games that are four orders of magnitude larger than what conventional computational approaches can handle.
Computing robust counterstrategies
 In Proceedings of the Annual Conference on Neural Information Processing Systems (NIPS
, 2007
"... Adaptation to other initially unknown agents often requires computing an effective counterstrategy. In the Bayesian paradigm, one must find a good counterstrategy to the inferred posterior of the other agents ’ behavior. In the experts paradigm, one may want to choose experts that are good counter ..."
Abstract

Cited by 38 (7 self)
 Add to MetaCart
Adaptation to other initially unknown agents often requires computing an effective counterstrategy. In the Bayesian paradigm, one must find a good counterstrategy to the inferred posterior of the other agents ’ behavior. In the experts paradigm, one may want to choose experts that are good counterstrategies to the other agents ’ expected behavior. In this paper we introduce a technique for computing robust counterstrategies for adaptation in multiagent scenarios under a variety of paradigms. The strategies can take advantage of a suspected tendency in the decisions of the other agents, while bounding the worstcase performance when the tendency is not observed. The technique involves solving a modified game, and therefore can make use of recently developed algorithms for solving very large extensive games. We demonstrate the effectiveness of the technique in twoplayer Texas Hold’em. We show that the computed poker strategies are substantially more robust than best response counterstrategies, while still exploiting a suspected tendency. We also compose the generated strategies in an experts algorithm showing a dramatic improvement in performance over using simple best responses. 1
Finding equilibria in large sequential games of imperfect information
 In ACM Conference on Electronic Commerce
, 2006
"... Information ∗ ..."