Results 1  10
of
22
SMOOTHING TECHNIQUES FOR COMPUTING NASH EQUILIBRIA OF SEQUENTIAL GAMES
"... We develop firstorder smoothing techniques for saddlepoint problems that arise in the Nash equilibria computation of sequential games. The crux of our work is a construction of suitable proxfunctions for a certain class of polytopes that encode the sequential nature of the games. An implementatio ..."
Abstract

Cited by 25 (7 self)
 Add to MetaCart
We develop firstorder smoothing techniques for saddlepoint problems that arise in the Nash equilibria computation of sequential games. The crux of our work is a construction of suitable proxfunctions for a certain class of polytopes that encode the sequential nature of the games. An implementation based on our smoothing techniques computes approximate Nash equilibria for games that are four orders of magnitude larger than what conventional computational approaches can handle.
Effective ShortTerm Opponent Exploitation in Simplified Poker
 In Proceedings of the National Conference on Artificial Intelligence (AAAI
, 2005
"... Uncertainty in poker stems from two key sources, the shuffled deck and an adversary whose strategy is unknown. One approach to playing poker is to find a pessimistic gametheoretic solution (i.e., a Nash equilibrium), but human players have idiosyncratic weaknesses that can be exploited if some mode ..."
Abstract

Cited by 20 (0 self)
 Add to MetaCart
Uncertainty in poker stems from two key sources, the shuffled deck and an adversary whose strategy is unknown. One approach to playing poker is to find a pessimistic gametheoretic solution (i.e., a Nash equilibrium), but human players have idiosyncratic weaknesses that can be exploited if some model or counterstrategy can be learned by observing their play. However, games against humans last for at most a few hundred hands, so learning must be very fast to be useful. We explore two approaches to opponent modelling in the context of Kuhn poker, a small game for which gametheoretic solutions are known. Parameter estimation and expert algorithms are both studied. Experiments demonstrate that, even in this small game, convergence to maximally exploitive solutions in a small number of hands is impractical, but that good (e.g., better than Nash) performance can be achieved in as few as 50 hands. Finally, we show that amongst a set of strategies with equal gametheoretic value, in particular the set of Nash equilibrium strategies, some are preferable because they speed learning of the opponent’s strategy by exploring it more effectively. 1
Computing an Approximate Jam/Fold Equilibrium for 3player NoLimit Texas Hold’em Tournaments
, 2008
"... A recent paper computes nearoptimal strategies for twoplayer nolimit Texas hold’em tournaments; however, the techniques used are unable to compute equilibrium strategies for tournaments with more than two players. Motivated by the widespread popularity of multiplayer tournaments and the observatio ..."
Abstract

Cited by 11 (3 self)
 Add to MetaCart
A recent paper computes nearoptimal strategies for twoplayer nolimit Texas hold’em tournaments; however, the techniques used are unable to compute equilibrium strategies for tournaments with more than two players. Motivated by the widespread popularity of multiplayer tournaments and the observation that jam/fold strategies are nearoptimal in the two player case, we develop an algorithm that computes approximate jam/fold equilibrium strategies in tournaments with three — and potentially even more — players. Our algorithm combines an extension of fictitious play to imperfect information games, an algorithm similar to value iteration for solving stochastic games, and a heuristic from the poker community known as the Independent Chip Model which we use as an initialization. Several ways of exploiting suit symmetries and the use of custom indexing schemes made the approach computationally feasible. Aside from the initialization and the restriction to jam/fold strategies, our high level algorithm makes no pokerspecific assumptions and thus also applies to other multiplayer stochastic games of imperfect information.
The State of Solving Large IncompleteInformation Games, and Application to Poker
, 2010
"... Gametheoretic solution concepts prescribe how rational parties should act, but to become operational the concepts need to be accompanied by algorithms. I will review the state of solving incompleteinformation games. They encompass many practical problems such as auctions, negotiations, and securi ..."
Abstract

Cited by 10 (3 self)
 Add to MetaCart
Gametheoretic solution concepts prescribe how rational parties should act, but to become operational the concepts need to be accompanied by algorithms. I will review the state of solving incompleteinformation games. They encompass many practical problems such as auctions, negotiations, and security applications. I will discuss them in the context of how they have transformed computer poker. In short, gametheoretic reasoning now scales to many large problems, outperforms the alternatives on those problems, and in some games beats the best humans.
ExpectationBased Versus PotentialAware Automated Abstraction in Imperfect Information Games: An Experimental Comparison Using Poker
, 2008
"... Automated abstraction algorithms for sequential imperfect information games have recently emerged as a key component in developing competitive game theorybased agents. The existing literature has not investigated the relative performance of different abstraction algorithms. Instead, agents whose co ..."
Abstract

Cited by 10 (4 self)
 Add to MetaCart
Automated abstraction algorithms for sequential imperfect information games have recently emerged as a key component in developing competitive game theorybased agents. The existing literature has not investigated the relative performance of different abstraction algorithms. Instead, agents whose construction has used automated abstraction have only been compared under confounding effects: different granularities of abstraction and equilibriumfinding algorithms that yield different accuracies when solving the abstracted game. This paper provides the first systematic evaluation of abstraction algorithms. Two families of algorithms have been proposed. The distinguishing feature is the measure used to evaluate the strategic similarity between game states. One algorithm uses the probability of winning as the similarity measure. The other uses a potentialaware similarity measure based on probability distributions over future states. We conduct experiments on Rhode Island Hold’em poker. We compare the algorithms against each other, against optimal play, and against each agent’s nemesis. We also compare them based on the resulting game’s value. Interestingly, for very coarse abstractions the expectationbased algorithm is better, but for moderately coarse and fine abstractions the potentialaware approach is superior. Furthermore, agents constructed using the expectationbased approach are highly exploitable beyond what their performance against the game’s optimal strategy would suggest.
Strategy purification and thresholding: Effective nonequilibrium approaches for playing large games
 INTERNATIONAL CONFERENCE ON AUTONOMOUS AGENTS AND MULTIAGENT SYSTEMS (AAMAS
, 2012
"... There has been significant recent interest in computing effective strategies for playing large imperfectinformation games. Much prior work involves computing an approximate equilibrium strategy in a smaller abstract game, then playing this strategy in the full game (with the hope that it also well ..."
Abstract

Cited by 8 (3 self)
 Add to MetaCart
There has been significant recent interest in computing effective strategies for playing large imperfectinformation games. Much prior work involves computing an approximate equilibrium strategy in a smaller abstract game, then playing this strategy in the full game (with the hope that it also well approximates an equilibrium in the full game). In this paper, we present a family of modifications to this approach that work by constructing nonequilibrium strategies in the abstract game, which are then played in the full game. Our new procedures, called purification and thresholding, modify the action probabilities of an abstract equilibrium by preferring the higherprobability actions. Using a variety of domains, we show that these approaches lead to significantly stronger play than the standard equilibrium approach. As one example, our program that uses purification came in first place in the twoplayer nolimit Texas Hold’em total bankroll division of the 2010 Annual Computer Poker Competition. Surprisingly, we also show that purification significantly improves performance (against the full equilibrium strategy) in random 4 × 4 matrix games using random 3 × 3 abstractions. We present several additional results (both theoretical and empirical). Overall, one can view these approaches as ways of achieving robustness against overfitting one’s strategy to one’s lossy abstraction. Perhaps surprisingly, the performance gains do not necessarily come at the expense of worstcase exploitability.
ExtensiveForm Correlated Equilibrium: Definition and Computational Complexity
, 2008
"... This paper defines the extensiveform correlated equilibrium (EFCE) for extensive games with perfect recall. The EFCE concept extends Aumann’s strategicform correlated equilibrium (CE). Before the game starts, a correlation device generates a move for each information set. This move is recommended ..."
Abstract

Cited by 4 (0 self)
 Add to MetaCart
This paper defines the extensiveform correlated equilibrium (EFCE) for extensive games with perfect recall. The EFCE concept extends Aumann’s strategicform correlated equilibrium (CE). Before the game starts, a correlation device generates a move for each information set. This move is recommended to the player only when the player reaches the information set. In twoplayer perfectrecall extensive games without chance moves, the set of EFCE can be described by a polynomial number of consistency and incentive constraints. Assuming P is not equal to NP, this is not possible for the set of CE, or if the game has chance moves.
Tartanian5: A HeadsUp NoLimit Texas Hold’em PokerPlaying Program
, 2012
"... We present an overview of Tartanian5, a nolimit Texas Hold’em agent which we submitted to the 2012 Annual Computer Poker Competition. The agent plays a gametheoretic approximate Nash equilibrium strategy. First, it applies a potentialaware, perfectrecall, automated abstraction algorithm to group ..."
Abstract

Cited by 2 (2 self)
 Add to MetaCart
We present an overview of Tartanian5, a nolimit Texas Hold’em agent which we submitted to the 2012 Annual Computer Poker Competition. The agent plays a gametheoretic approximate Nash equilibrium strategy. First, it applies a potentialaware, perfectrecall, automated abstraction algorithm to group similar game states together and construct a smaller game that is strategically similar to the full game. In order to maintain a tractable number of possible betting sequences, it employs a discretized betting model, where only a small number of bet sizes are allowed at each game state. The strategies for both players are then computed using an improved version of Nesterov’s excessive gap technique specialized for poker. To mitigate the effect of overfitting, we employ an expost purification procedure to remove actions that are played with small probability. One final feature of our agent is a novel algorithm for interpreting bet sizes of the opponent that fall outside our model. We describe our new approach in detail, and present theoretical and empirical advantages over prior approaches. Finally, we briefly describe ongoing research in our group involving realtime computation and opponent exploitation, which will hopefully be incorporated into our agents in future years.