Results 1  10
of
13
Lossy Stochastic Game Abstraction with Bounds
, 2012
"... Abstraction followed by equilibrium finding has emerged as the leading approach to solving games. Lossless abstraction typically yields games that are still too large to solve, so lossy abstraction is needed. Unfortunately, prior lossy game abstraction algorithms have no guarantees on solution quali ..."
Abstract

Cited by 11 (5 self)
 Add to MetaCart
Abstraction followed by equilibrium finding has emerged as the leading approach to solving games. Lossless abstraction typically yields games that are still too large to solve, so lossy abstraction is needed. Unfortunately, prior lossy game abstraction algorithms have no guarantees on solution quality. We developed a framework that enables the design of lossy game abstraction algorithms with guarantees on solution quality. It simultaneously handles state and action abstraction. We define a measure of reward approximation error and transition probability error achieved by state and action abstraction in stochastic games such that the regret of the equilibrium found in the abstract game when implemented in the original, unabstracted game is upperbounded by a function of those measures. We then develop the first lossy game abstraction algorithms with bounds on solution quality. Both of them work levelbylevel up from the end of the game. One of the algorithms is greedy and the other is an integer linear program. We also prove that the abstraction problem is NPcomplete (even with just action abstraction, 2 agents, and a 1step game), but point out that this does not mean that the game abstraction problems that occur in practice cannot be solved quickly.
Action Translation in ExtensiveForm Games with Large Action Spaces: Axioms, Paradoxes, and the PseudoHarmonic Mapping
"... When solving extensiveform games with large action spaces, typically significant abstraction is needed to make the problem manageable from a modeling or computational perspective. When this occurs, a procedure is needed to interpret actions of the opponent that fall outside of our abstraction (by m ..."
Abstract

Cited by 8 (7 self)
 Add to MetaCart
When solving extensiveform games with large action spaces, typically significant abstraction is needed to make the problem manageable from a modeling or computational perspective. When this occurs, a procedure is needed to interpret actions of the opponent that fall outside of our abstraction (by mapping them to actions in our abstraction). This is called an action translation mapping. Prior action translation mappings have been based on heuristics without theoretical justification. We show that the prior mappings are highly exploitable and that most of them violate certain natural desiderata. We present a new mapping that satisfies these desiderata and has significantly lower exploitability than the prior mappings. Furthermore, we observe that the cost of this worstcase performance benefit (low exploitability) is not high in practice; our mapping performs competitively with the prior mappings against nolimit Texas Hold’em agents submitted to the 2012 Annual Computer Poker Competition. We also observe several paradoxes that can arise when performing action abstraction and translation; for example, we show that it is possible to improve performance by including suboptimal actions in our abstraction and excluding optimal actions.
Tartanian5: A HeadsUp NoLimit Texas Hold’em PokerPlaying Program
, 2012
"... We present an overview of Tartanian5, a nolimit Texas Hold’em agent which we submitted to the 2012 Annual Computer Poker Competition. The agent plays a gametheoretic approximate Nash equilibrium strategy. First, it applies a potentialaware, perfectrecall, automated abstraction algorithm to group ..."
Abstract

Cited by 4 (4 self)
 Add to MetaCart
We present an overview of Tartanian5, a nolimit Texas Hold’em agent which we submitted to the 2012 Annual Computer Poker Competition. The agent plays a gametheoretic approximate Nash equilibrium strategy. First, it applies a potentialaware, perfectrecall, automated abstraction algorithm to group similar game states together and construct a smaller game that is strategically similar to the full game. In order to maintain a tractable number of possible betting sequences, it employs a discretized betting model, where only a small number of bet sizes are allowed at each game state. The strategies for both players are then computed using an improved version of Nesterov’s excessive gap technique specialized for poker. To mitigate the effect of overfitting, we employ an expost purification procedure to remove actions that are played with small probability. One final feature of our agent is a novel algorithm for interpreting bet sizes of the opponent that fall outside our model. We describe our new approach in detail, and present theoretical and empirical advantages over prior approaches. Finally, we briefly describe ongoing research in our group involving realtime computation and opponent exploitation, which will hopefully be incorporated into our agents in future years.
Improving Performance in ImperfectInformation Games with Large State and Action Spaces by Solving Endgames
, 2013
"... Sequential games of perfect information can be solved by backward induction, where solutions to endgames are propagated up the game tree. However, this does not work in imperfectinformation games because different endgames can contain states that belong to the same information set and cannot be tre ..."
Abstract

Cited by 4 (0 self)
 Add to MetaCart
Sequential games of perfect information can be solved by backward induction, where solutions to endgames are propagated up the game tree. However, this does not work in imperfectinformation games because different endgames can contain states that belong to the same information set and cannot be treated independently. In fact, we show that this approach can fail even in a simple game with a unique equilibrium and a single endgame. Nonetheless, we show that endgame solving can have significant benefits in imperfectinformation games with large state and action spaces: computation of exact (rather than approximate) equilibrium strategies, computation of relevant equilibrium refinements, significantly finergrained action and information abstraction, new information abstraction algorithms that take into account the relevant distribution of players ’ types entering the endgame, being able to select the coarseness of the action abstraction dynamically, additional abstraction techniques for speeding up endgame solving, a solution to the “offtree ” problem, and using different degrees of probability thresholding in modeling versus playing. We discuss each of these topics in detail, and introduce techniques that enable one to conduct endgame solving in a scalable way even when the number of states and actions in the game is large. Our experiments on twoplayer nolimit Texas Hold’em poker show that our approach leads to significant performance improvements in practice.
Abstraction for solving large incompleteinformation games
 In AAAI Conference on Artificial Intelligence (AAAI). Senior Member Track
"... Most realworld games and many recreational games are games of incomplete information. Over the last dozen years, abstraction has emerged as a key enabler for solving large incompleteinformation games. First, game that is strategically similar to the original game. Second, an approximate equilibriu ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
Most realworld games and many recreational games are games of incomplete information. Over the last dozen years, abstraction has emerged as a key enabler for solving large incompleteinformation games. First, game that is strategically similar to the original game. Second, an approximate equilibrium is computed in game is mapped back to the original game. In this paper, I will review key developments in the field. I present reasons for abstracting games, and point out the issue of abstraction pathology. I then review the practical algorithms for information abstraction and action abstraction. I then cover recent theoretical breakthroughs that beget bounds on the quality of the strategy from the abstract game, when measured in the original game. I then discuss how to reverse map the opponent’s action into the abstraction if the opponent makes a move that is not in the abstraction. Finally, I discuss other topics of current and future research.
Automating Collusion Detection in Sequential Games
"... Collusion is the practice of two or more parties deliberately cooperating to the detriment of others. While such behavior may be desirable in certain circumstances, in many it is considered dishonest and unfair. If agents otherwise hold strictly to the established rules, though, collusion can be cha ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
Collusion is the practice of two or more parties deliberately cooperating to the detriment of others. While such behavior may be desirable in certain circumstances, in many it is considered dishonest and unfair. If agents otherwise hold strictly to the established rules, though, collusion can be challenging to police. In this paper, we introduce an automatic method for collusion detection in sequential games. We achieve this through a novel object, called a collusion table, that captures the effects of collusive behavior, i.e., advantage to the colluding parties, without assuming any particular pattern of behavior. We show the effectiveness of this method in the domain of poker, a popular game where collusion is prohibited. 1
An Introduction to Counterfactual Regret Minimization
, 2013
"... In 2000, Hart and MasColell introduced the important gametheoretic algorithm of regret matching. ..."
Abstract
 Add to MetaCart
(Show Context)
In 2000, Hart and MasColell introduced the important gametheoretic algorithm of regret matching.
nomics
"... The leading approach for solving large imperfectinformation games is automated abstraction followed by running an equilibriumfinding algorithm. We introduce a distributed version of the most commonly used equilibriumfinding algorithm, counterfactual regret minimization (CFR), which enables CF ..."
Abstract
 Add to MetaCart
(Show Context)
The leading approach for solving large imperfectinformation games is automated abstraction followed by running an equilibriumfinding algorithm. We introduce a distributed version of the most commonly used equilibriumfinding algorithm, counterfactual regret minimization (CFR), which enables CFR to scale to dramatically larger abstractions and numbers of cores. The new algorithm begets constraints on the abstraction so as to make the pieces running on different computers disjoint. We introduce an algorithm for generating such abstractions while capitalizing on stateoftheart abstraction ideas such as imperfect recall and earthmover’s distance. Our techniques enabled an equilibrium computation of unprecedented size on a supercomputer with a high interblade memory latency. Prior approaches run slowly on this architecture. Our approach also leads to a significant improvement over using the prior best approach on a large sharedmemory server with low memory latency. Finally, we introduce a family of postprocessing techniques that outperform prior ones. We applied these techniques to generate an agent for twoplayer nolimit Texas Hold’em, called Tarta
Endgame Solving in Large ImperfectInformation Games∗
"... The leading approach for computing strong gametheoretic strategies in large imperfectinformation games is to first solve an abstracted version of the game offline, then perform a table lookup during game play. We consider a modification to this approach where we solve the portion of the game tha ..."
Abstract
 Add to MetaCart
(Show Context)
The leading approach for computing strong gametheoretic strategies in large imperfectinformation games is to first solve an abstracted version of the game offline, then perform a table lookup during game play. We consider a modification to this approach where we solve the portion of the game that we have actually reached in real time to a greater degree of accuracy than in the initial computation. We call this approach endgame solving. Theoretically, we show that endgame solving can produce highly exploitable strategies in some games; however, we show that it can guarantee a low exploitability in certain games where the opponent is given sufficient exploitative power within the endgame. Furthermore, despite the lack of a general worstcase guarantee, we describe many benefits of endgame solving. We present an efficient algorithm for performing endgame solving in large imperfectinformation games, and present a new variancereduction technique for evaluating the performance of an agent that uses endgame solving. Experiments on nolimit Texas Hold’em show that our algorithm leads to significantly stronger performance against the strongest agents from the
University of Alberta COLLUSION DETECTION IN SEQUENTIAL GAMES
"... copies of this thesis and to lend or sell such copies for private, scholarly or scientific research purposes only. Where the thesis is converted to, or otherwise made available in digital form, the University of Alberta will advise potential users of the thesis of these terms. The author reserves al ..."
Abstract
 Add to MetaCart
(Show Context)
copies of this thesis and to lend or sell such copies for private, scholarly or scientific research purposes only. Where the thesis is converted to, or otherwise made available in digital form, the University of Alberta will advise potential users of the thesis of these terms. The author reserves all other publication and other rights in association with the copyright in the thesis, and except as herein before provided, neither the thesis nor any substantial portion thereof may be printed or otherwise reproduced in any material form whatever without the author’s prior written permission. To my dear family, Collusion is the deliberate cooperation of two or more parties to the detriment of others. While this behaviour can be highly profitable for colluders (for example, in auctions and online games), it is considered illegal and unfair in many sequential decisionmaking domains and presents many challenging problems in these systems. In this thesis we present an automatic collusion detection method for extensive form games. This method uses a novel object, called a collusion table, that aims