Results 1  10
of
32
SMOOTHING TECHNIQUES FOR COMPUTING NASH EQUILIBRIA OF SEQUENTIAL GAMES
"... We develop firstorder smoothing techniques for saddlepoint problems that arise in the Nash equilibria computation of sequential games. The crux of our work is a construction of suitable proxfunctions for a certain class of polytopes that encode the sequential nature of the games. An implementatio ..."
Abstract

Cited by 40 (10 self)
 Add to MetaCart
(Show Context)
We develop firstorder smoothing techniques for saddlepoint problems that arise in the Nash equilibria computation of sequential games. The crux of our work is a construction of suitable proxfunctions for a certain class of polytopes that encode the sequential nature of the games. An implementation based on our smoothing techniques computes approximate Nash equilibria for games that are four orders of magnitude larger than what conventional computational approaches can handle.
The State of Solving Large IncompleteInformation Games, and Application to Poker
, 2010
"... Gametheoretic solution concepts prescribe how rational parties should act, but to become operational the concepts need to be accompanied by algorithms. I will review the state of solving incompleteinformation games. They encompass many practical problems such as auctions, negotiations, and securi ..."
Abstract

Cited by 27 (8 self)
 Add to MetaCart
Gametheoretic solution concepts prescribe how rational parties should act, but to become operational the concepts need to be accompanied by algorithms. I will review the state of solving incompleteinformation games. They encompass many practical problems such as auctions, negotiations, and security applications. I will discuss them in the context of how they have transformed computer poker. In short, gametheoretic reasoning now scales to many large problems, outperforms the alternatives on those problems, and in some games beats the best humans.
Effective ShortTerm Opponent Exploitation in Simplified Poker
 In Proceedings of the National Conference on Artificial Intelligence (AAAI
, 2005
"... Uncertainty in poker stems from two key sources, the shuffled deck and an adversary whose strategy is unknown. One approach to playing poker is to find a pessimistic gametheoretic solution (i.e., a Nash equilibrium), but human players have idiosyncratic weaknesses that can be exploited if some mode ..."
Abstract

Cited by 24 (0 self)
 Add to MetaCart
Uncertainty in poker stems from two key sources, the shuffled deck and an adversary whose strategy is unknown. One approach to playing poker is to find a pessimistic gametheoretic solution (i.e., a Nash equilibrium), but human players have idiosyncratic weaknesses that can be exploited if some model or counterstrategy can be learned by observing their play. However, games against humans last for at most a few hundred hands, so learning must be very fast to be useful. We explore two approaches to opponent modelling in the context of Kuhn poker, a small game for which gametheoretic solutions are known. Parameter estimation and expert algorithms are both studied. Experiments demonstrate that, even in this small game, convergence to maximally exploitive solutions in a small number of hands is impractical, but that good (e.g., better than Nash) performance can be achieved in as few as 50 hands. Finally, we show that amongst a set of strategies with equal gametheoretic value, in particular the set of Nash equilibrium strategies, some are preferable because they speed learning of the opponent’s strategy by exploring it more effectively. 1
Evaluating StateSpace Abstractions in Extensiveform Games
, 2013
"... Efficient algorithms exist for finding optimal policies in extensiveform games. However, humanscale problems are typically so large that this computation remains infeasible with modern computing resources. Statespace abstraction techniques allow for the derivation of a smaller and strategically s ..."
Abstract

Cited by 16 (4 self)
 Add to MetaCart
Efficient algorithms exist for finding optimal policies in extensiveform games. However, humanscale problems are typically so large that this computation remains infeasible with modern computing resources. Statespace abstraction techniques allow for the derivation of a smaller and strategically similar abstract domain, in which an optimal strategy can be computed and then used as a suboptimal strategy in the real domain. In this paper, we consider the task of evaluating the quality of an abstraction, independent of a specific abstract strategy. In particular, we use a recent metric for abstraction quality and examine imperfect recall abstractions, in which agents “forget ” previously observed information to focus the abstraction effort on more recent and relevant state information. We present experimental results in the domain of Texas hold’em poker that validate the use of distributionaware abstractions over expectationbased approaches, demonstrate that the new metric better predicts tournament performance, and show that abstractions built using imperfect recall outperform those built using perfect recall in terms of both exploitability and oneonone play.
N.: Automated abstractions for patrolling security games
 In: AAAI. (2011
"... Recently, there has been a significant interest in studying security games to provide tools for addressing resource allocation problems in security applications. Patrolling security games (PSGs) constitute a special class of security games wherein the resources are mobile. One of the most relevant ..."
Abstract

Cited by 14 (0 self)
 Add to MetaCart
(Show Context)
Recently, there has been a significant interest in studying security games to provide tools for addressing resource allocation problems in security applications. Patrolling security games (PSGs) constitute a special class of security games wherein the resources are mobile. One of the most relevant open problems in security games is the design of scalable algorithms to tackle realistic scenarios. While the literature mainly focuses on heuristics and decomposition techniques (e.g., double oracle), in this paper we provide, to the best of our knowledge, the first study on the use of abstractions in security games (specifically for PSGs) to design scalable algorithms. We define some classes of abstractions and we provide parametric algorithms to automatically generate abstractions. We show that abstractions allow one to relax the constraint of patrolling strategies ’ Markovianity (customary in PSGs) and to solve large game instances. We additionally pose the problem to search for the optimal abstraction and we develop an anytime algorithm to find it.
Computing an Approximate Jam/Fold Equilibrium for 3player NoLimit Texas Hold’em Tournaments
, 2008
"... A recent paper computes nearoptimal strategies for twoplayer nolimit Texas hold’em tournaments; however, the techniques used are unable to compute equilibrium strategies for tournaments with more than two players. Motivated by the widespread popularity of multiplayer tournaments and the observatio ..."
Abstract

Cited by 14 (3 self)
 Add to MetaCart
A recent paper computes nearoptimal strategies for twoplayer nolimit Texas hold’em tournaments; however, the techniques used are unable to compute equilibrium strategies for tournaments with more than two players. Motivated by the widespread popularity of multiplayer tournaments and the observation that jam/fold strategies are nearoptimal in the two player case, we develop an algorithm that computes approximate jam/fold equilibrium strategies in tournaments with three — and potentially even more — players. Our algorithm combines an extension of fictitious play to imperfect information games, an algorithm similar to value iteration for solving stochastic games, and a heuristic from the poker community known as the Independent Chip Model which we use as an initialization. Several ways of exploiting suit symmetries and the use of custom indexing schemes made the approach computationally feasible. Aside from the initialization and the restriction to jam/fold strategies, our high level algorithm makes no pokerspecific assumptions and thus also applies to other multiplayer stochastic games of imperfect information.
Strategy purification and thresholding: Effective nonequilibrium approaches for playing large games
 INTERNATIONAL CONFERENCE ON AUTONOMOUS AGENTS AND MULTIAGENT SYSTEMS (AAMAS
, 2012
"... There has been significant recent interest in computing effective strategies for playing large imperfectinformation games. Much prior work involves computing an approximate equilibrium strategy in a smaller abstract game, then playing this strategy in the full game (with the hope that it also well ..."
Abstract

Cited by 13 (6 self)
 Add to MetaCart
There has been significant recent interest in computing effective strategies for playing large imperfectinformation games. Much prior work involves computing an approximate equilibrium strategy in a smaller abstract game, then playing this strategy in the full game (with the hope that it also well approximates an equilibrium in the full game). In this paper, we present a family of modifications to this approach that work by constructing nonequilibrium strategies in the abstract game, which are then played in the full game. Our new procedures, called purification and thresholding, modify the action probabilities of an abstract equilibrium by preferring the higherprobability actions. Using a variety of domains, we show that these approaches lead to significantly stronger play than the standard equilibrium approach. As one example, our program that uses purification came in first place in the twoplayer nolimit Texas Hold’em total bankroll division of the 2010 Annual Computer Poker Competition. Surprisingly, we also show that purification significantly improves performance (against the full equilibrium strategy) in random 4 × 4 matrix games using random 3 × 3 abstractions. We present several additional results (both theoretical and empirical). Overall, one can view these approaches as ways of achieving robustness against overfitting one’s strategy to one’s lossy abstraction. Perhaps surprisingly, the performance gains do not necessarily come at the expense of worstcase exploitability.
ExpectationBased Versus PotentialAware Automated Abstraction in Imperfect Information Games: An Experimental Comparison Using Poker
, 2008
"... Automated abstraction algorithms for sequential imperfect information games have recently emerged as a key component in developing competitive game theorybased agents. The existing literature has not investigated the relative performance of different abstraction algorithms. Instead, agents whose co ..."
Abstract

Cited by 12 (5 self)
 Add to MetaCart
Automated abstraction algorithms for sequential imperfect information games have recently emerged as a key component in developing competitive game theorybased agents. The existing literature has not investigated the relative performance of different abstraction algorithms. Instead, agents whose construction has used automated abstraction have only been compared under confounding effects: different granularities of abstraction and equilibriumfinding algorithms that yield different accuracies when solving the abstracted game. This paper provides the first systematic evaluation of abstraction algorithms. Two families of algorithms have been proposed. The distinguishing feature is the measure used to evaluate the strategic similarity between game states. One algorithm uses the probability of winning as the similarity measure. The other uses a potentialaware similarity measure based on probability distributions over future states. We conduct experiments on Rhode Island Hold’em poker. We compare the algorithms against each other, against optimal play, and against each agent’s nemesis. We also compare them based on the resulting game’s value. Interestingly, for very coarse abstractions the expectationbased algorithm is better, but for moderately coarse and fine abstractions the potentialaware approach is superior. Furthermore, agents constructed using the expectationbased approach are highly exploitable beyond what their performance against the game’s optimal strategy would suggest.
Lossy Stochastic Game Abstraction with Bounds
, 2012
"... Abstraction followed by equilibrium finding has emerged as the leading approach to solving games. Lossless abstraction typically yields games that are still too large to solve, so lossy abstraction is needed. Unfortunately, prior lossy game abstraction algorithms have no guarantees on solution quali ..."
Abstract

Cited by 11 (5 self)
 Add to MetaCart
Abstraction followed by equilibrium finding has emerged as the leading approach to solving games. Lossless abstraction typically yields games that are still too large to solve, so lossy abstraction is needed. Unfortunately, prior lossy game abstraction algorithms have no guarantees on solution quality. We developed a framework that enables the design of lossy game abstraction algorithms with guarantees on solution quality. It simultaneously handles state and action abstraction. We define a measure of reward approximation error and transition probability error achieved by state and action abstraction in stochastic games such that the regret of the equilibrium found in the abstract game when implemented in the original, unabstracted game is upperbounded by a function of those measures. We then develop the first lossy game abstraction algorithms with bounds on solution quality. Both of them work levelbylevel up from the end of the game. One of the algorithms is greedy and the other is an integer linear program. We also prove that the abstraction problem is NPcomplete (even with just action abstraction, 2 agents, and a 1step game), but point out that this does not mean that the game abstraction problems that occur in practice cannot be solved quickly.