Results 1  10
of
19
SMOOTHING TECHNIQUES FOR COMPUTING NASH EQUILIBRIA OF SEQUENTIAL GAMES
"... We develop firstorder smoothing techniques for saddlepoint problems that arise in the Nash equilibria computation of sequential games. The crux of our work is a construction of suitable proxfunctions for a certain class of polytopes that encode the sequential nature of the games. An implementatio ..."
Abstract

Cited by 25 (7 self)
 Add to MetaCart
We develop firstorder smoothing techniques for saddlepoint problems that arise in the Nash equilibria computation of sequential games. The crux of our work is a construction of suitable proxfunctions for a certain class of polytopes that encode the sequential nature of the games. An implementation based on our smoothing techniques computes approximate Nash equilibria for games that are four orders of magnitude larger than what conventional computational approaches can handle.
Effective ShortTerm Opponent Exploitation in Simplified Poker
 In Proceedings of the National Conference on Artificial Intelligence (AAAI
, 2005
"... Uncertainty in poker stems from two key sources, the shuffled deck and an adversary whose strategy is unknown. One approach to playing poker is to find a pessimistic gametheoretic solution (i.e., a Nash equilibrium), but human players have idiosyncratic weaknesses that can be exploited if some mode ..."
Abstract

Cited by 19 (0 self)
 Add to MetaCart
Uncertainty in poker stems from two key sources, the shuffled deck and an adversary whose strategy is unknown. One approach to playing poker is to find a pessimistic gametheoretic solution (i.e., a Nash equilibrium), but human players have idiosyncratic weaknesses that can be exploited if some model or counterstrategy can be learned by observing their play. However, games against humans last for at most a few hundred hands, so learning must be very fast to be useful. We explore two approaches to opponent modelling in the context of Kuhn poker, a small game for which gametheoretic solutions are known. Parameter estimation and expert algorithms are both studied. Experiments demonstrate that, even in this small game, convergence to maximally exploitive solutions in a small number of hands is impractical, but that good (e.g., better than Nash) performance can be achieved in as few as 50 hands. Finally, we show that amongst a set of strategies with equal gametheoretic value, in particular the set of Nash equilibrium strategies, some are preferable because they speed learning of the opponent’s strategy by exploring it more effectively. 1
Computing an Approximate Jam/Fold Equilibrium for 3player NoLimit Texas Hold’em Tournaments
, 2008
"... A recent paper computes nearoptimal strategies for twoplayer nolimit Texas hold’em tournaments; however, the techniques used are unable to compute equilibrium strategies for tournaments with more than two players. Motivated by the widespread popularity of multiplayer tournaments and the observatio ..."
Abstract

Cited by 11 (3 self)
 Add to MetaCart
A recent paper computes nearoptimal strategies for twoplayer nolimit Texas hold’em tournaments; however, the techniques used are unable to compute equilibrium strategies for tournaments with more than two players. Motivated by the widespread popularity of multiplayer tournaments and the observation that jam/fold strategies are nearoptimal in the two player case, we develop an algorithm that computes approximate jam/fold equilibrium strategies in tournaments with three — and potentially even more — players. Our algorithm combines an extension of fictitious play to imperfect information games, an algorithm similar to value iteration for solving stochastic games, and a heuristic from the poker community known as the Independent Chip Model which we use as an initialization. Several ways of exploiting suit symmetries and the use of custom indexing schemes made the approach computationally feasible. Aside from the initialization and the restriction to jam/fold strategies, our high level algorithm makes no pokerspecific assumptions and thus also applies to other multiplayer stochastic games of imperfect information.
ExpectationBased Versus PotentialAware Automated Abstraction in Imperfect Information Games: An Experimental Comparison Using Poker
, 2008
"... Automated abstraction algorithms for sequential imperfect information games have recently emerged as a key component in developing competitive game theorybased agents. The existing literature has not investigated the relative performance of different abstraction algorithms. Instead, agents whose co ..."
Abstract

Cited by 10 (4 self)
 Add to MetaCart
Automated abstraction algorithms for sequential imperfect information games have recently emerged as a key component in developing competitive game theorybased agents. The existing literature has not investigated the relative performance of different abstraction algorithms. Instead, agents whose construction has used automated abstraction have only been compared under confounding effects: different granularities of abstraction and equilibriumfinding algorithms that yield different accuracies when solving the abstracted game. This paper provides the first systematic evaluation of abstraction algorithms. Two families of algorithms have been proposed. The distinguishing feature is the measure used to evaluate the strategic similarity between game states. One algorithm uses the probability of winning as the similarity measure. The other uses a potentialaware similarity measure based on probability distributions over future states. We conduct experiments on Rhode Island Hold’em poker. We compare the algorithms against each other, against optimal play, and against each agent’s nemesis. We also compare them based on the resulting game’s value. Interestingly, for very coarse abstractions the expectationbased algorithm is better, but for moderately coarse and fine abstractions the potentialaware approach is superior. Furthermore, agents constructed using the expectationbased approach are highly exploitable beyond what their performance against the game’s optimal strategy would suggest.
The State of Solving Large IncompleteInformation Games, and Application to Poker
, 2010
"... Gametheoretic solution concepts prescribe how rational parties should act, but to become operational the concepts need to be accompanied by algorithms. I will review the state of solving incompleteinformation games. They encompass many practical problems such as auctions, negotiations, and securi ..."
Abstract

Cited by 10 (3 self)
 Add to MetaCart
Gametheoretic solution concepts prescribe how rational parties should act, but to become operational the concepts need to be accompanied by algorithms. I will review the state of solving incompleteinformation games. They encompass many practical problems such as auctions, negotiations, and security applications. I will discuss them in the context of how they have transformed computer poker. In short, gametheoretic reasoning now scales to many large problems, outperforms the alternatives on those problems, and in some games beats the best humans.
Strategy purification and thresholding: Effective nonequilibrium approaches for playing large games
 INTERNATIONAL CONFERENCE ON AUTONOMOUS AGENTS AND MULTIAGENT SYSTEMS (AAMAS
, 2012
"... There has been significant recent interest in computing effective strategies for playing large imperfectinformation games. Much prior work involves computing an approximate equilibrium strategy in a smaller abstract game, then playing this strategy in the full game (with the hope that it also well ..."
Abstract

Cited by 8 (3 self)
 Add to MetaCart
There has been significant recent interest in computing effective strategies for playing large imperfectinformation games. Much prior work involves computing an approximate equilibrium strategy in a smaller abstract game, then playing this strategy in the full game (with the hope that it also well approximates an equilibrium in the full game). In this paper, we present a family of modifications to this approach that work by constructing nonequilibrium strategies in the abstract game, which are then played in the full game. Our new procedures, called purification and thresholding, modify the action probabilities of an abstract equilibrium by preferring the higherprobability actions. Using a variety of domains, we show that these approaches lead to significantly stronger play than the standard equilibrium approach. As one example, our program that uses purification came in first place in the twoplayer nolimit Texas Hold’em total bankroll division of the 2010 Annual Computer Poker Competition. Surprisingly, we also show that purification significantly improves performance (against the full equilibrium strategy) in random 4 × 4 matrix games using random 3 × 3 abstractions. We present several additional results (both theoretical and empirical). Overall, one can view these approaches as ways of achieving robustness against overfitting one’s strategy to one’s lossy abstraction. Perhaps surprisingly, the performance gains do not necessarily come at the expense of worstcase exploitability.
ExtensiveForm Correlated Equilibrium: Definition and Computational Complexity
, 2008
"... This paper defines the extensiveform correlated equilibrium (EFCE) for extensive games with perfect recall. The EFCE concept extends Aumann’s strategicform correlated equilibrium (CE). Before the game starts, a correlation device generates a move for each information set. This move is recommended ..."
Abstract

Cited by 4 (0 self)
 Add to MetaCart
This paper defines the extensiveform correlated equilibrium (EFCE) for extensive games with perfect recall. The EFCE concept extends Aumann’s strategicform correlated equilibrium (CE). Before the game starts, a correlation device generates a move for each information set. This move is recommended to the player only when the player reaches the information set. In twoplayer perfectrecall extensive games without chance moves, the set of EFCE can be described by a polynomial number of consistency and incentive constraints. Assuming P is not equal to NP, this is not possible for the set of CE, or if the game has chance moves.
Algorithms for abstracting and solving imperfect information games
, 2007
"... Game theory is the mathematical study of rational behavior in strategic environments. In many settings, most notably twoperson zerosum games, game theory provides particularly strong and appealing solution concepts. Furthermore, these solutions are efficiently computable in the complexitytheory s ..."
Abstract

Cited by 2 (1 self)
 Add to MetaCart
Game theory is the mathematical study of rational behavior in strategic environments. In many settings, most notably twoperson zerosum games, game theory provides particularly strong and appealing solution concepts. Furthermore, these solutions are efficiently computable in the complexitytheory sense. However, in most interesting potential applications in artificial intelligence, the solutions are difficult to compute using current techniques due primarily to the extremely large statespaces of the environments. In this thesis, we propose new algorithms for tackling these computational difficulties. In one stream of research, we introduce automated abstraction algorithms for sequential games of imperfect information. These algorithms take as input a description of a game and produce a description of a strategically similar, but smaller, game as output. We present algorithms that are lossless (i.e., equilibriumpreserving), as well as algorithms that are lossy, but which can yield much smaller games while still retaining the most important features of the original game. In a second stream of research, we develop specialized optimization algorithms for finding ɛequilibria in sequential games of imperfect information. The algorithms are based on recent advances in nonsmooth convex optimization (namely the excessive gap technique) and provide significant improvements