Results 1  10
of
45
Multiagent influence diagrams for representing and solving games
 GAMES AND ECONOMIC BEHAVIOR
, 2001
"... The traditional representations of games using the extensive form or the strategic (normal) form obscure much of the structure that is present in realworld games. In this paper, we propose a new representation language for general multiplayer games — multiagent influence diagrams (MAIDs). This rep ..."
Abstract

Cited by 190 (2 self)
 Add to MetaCart
The traditional representations of games using the extensive form or the strategic (normal) form obscure much of the structure that is present in realworld games. In this paper, we propose a new representation language for general multiplayer games — multiagent influence diagrams (MAIDs). This representation extends graphical models for probability distributions to a multiagent decisionmaking context. MAIDs explicitly encode structure involving the dependence relationships among variables. As a consequence, we can define a notion of strategic relevance of one decision variable to another: ¢¡ is strategically relevant to if, to optimize the decision rule at, the decision maker needs to take into consideration the decision rule at ¡. We provide a sound and complete graphical criterion for determining strategic relevance. We then show how strategic relevance can be used to detect structure in games, allowing a large game to be broken up into a set of interacting smaller games, which can be solved in sequence. We show that this decomposition can lead to substantial savings in the computational cost of finding Nash equilibria in these games.
A competitive Texas Hold’em poker player via automated abstraction and realtime equilibrium computation
 IN PROCEEDINGS OF THE NATIONAL CONFERENCE ON ARTIFICIAL INTELLIGENCE (AAAI
, 2006
"... We present a game theorybased headsup Texas Hold’em poker player, GS1. To overcome the computational obstacles stemming from Texas Hold’em’s gigantic game tree, the player employs our automated abstraction techniques to reduce the complexity of the strategy computations. Texas Hold’em consists of ..."
Abstract

Cited by 63 (21 self)
 Add to MetaCart
We present a game theorybased headsup Texas Hold’em poker player, GS1. To overcome the computational obstacles stemming from Texas Hold’em’s gigantic game tree, the player employs our automated abstraction techniques to reduce the complexity of the strategy computations. Texas Hold’em consists of four betting rounds. Our player solves a large linear program (offline) to compute strategies for the abstracted first and second rounds. After the second betting round, our player updates the probability of each possible hand based on the observed betting actions in the first two rounds as well as the revealed cards. Using these updated probabilities, our player computes in realtime an equilibrium approximation for the last two abstracted rounds. We demonstrate that our player, which incorporates very little pokerspecific knowledge, is competitive with leading pokerplaying programs which incorporate extensive domain knowledge, as well as with advanced human players.
Optimal and approximate Qvalue functions for decentralized POMDPs
 J. Artificial Intelligence Research
"... Decisiontheoretic planning is a popular approach to sequential decision making problems, because it treats uncertainty in sensing and acting in a principled way. In singleagent frameworks like MDPs and POMDPs, planning can be carried out by resorting to Qvalue functions: an optimal Qvalue functi ..."
Abstract

Cited by 62 (26 self)
 Add to MetaCart
(Show Context)
Decisiontheoretic planning is a popular approach to sequential decision making problems, because it treats uncertainty in sensing and acting in a principled way. In singleagent frameworks like MDPs and POMDPs, planning can be carried out by resorting to Qvalue functions: an optimal Qvalue function Q ∗ is computed in a recursive manner by dynamic programming, and then an optimal policy is extracted from Q ∗. In this paper we study whether similar Qvalue functions can be defined for decentralized POMDP models (DecPOMDPs), and how policies can be extracted from such value functions. We define two forms of the optimal Qvalue function for DecPOMDPs: one that gives a normative description as the Qvalue function of an optimal pure joint policy and another one that is sequentially rational and thus gives a recipe for computation. This computation, however, is infeasible for all but the smallest problems. Therefore, we analyze various approximate Qvalue functions that allow for efficient computation. We describe how they relate, and we prove that they all provide an upper bound to the optimal Qvalue function Q ∗. Finally, unifying some previous approaches for solving DecPOMDPs, we describe a family of algorithms for extracting policies from such Qvalue functions, and perform an experimental evaluation on existing test problems, including a new firefighting benchmark problem. 1.
Potentialaware automated abstraction of sequential games, and holistic equilibrium analysis of Texas Hold’em poker
 IN AAAI’07
, 2007
"... We present a new abstraction algorithm for sequential imperfect information games. While most prior abstraction algorithms employ a myopic expectedvalue computation as a similarity metric, our algorithm considers a higherdimensional space consisting of histograms over abstracted classes of states f ..."
Abstract

Cited by 49 (16 self)
 Add to MetaCart
(Show Context)
We present a new abstraction algorithm for sequential imperfect information games. While most prior abstraction algorithms employ a myopic expectedvalue computation as a similarity metric, our algorithm considers a higherdimensional space consisting of histograms over abstracted classes of states from later stages of the game. This enables our bottomup abstraction algorithm to automatically take into account potential: a hand can become relatively better (or worse) over time and the strength of different hands can get resolved earlier or later in the game. We further improve the abstraction quality by making multiple passes over the abstraction, enabling the algorithm to narrow the scope of analysis to information that is relevant given abstraction decisions made for earlier parts of the game. We also present a custom indexing scheme based on suit isomorphisms that enables one to work on significantly larger models than before. We apply the techniques to headsup limit Texas Hold’em poker. Whereas all prior game theorybased work for Texas Hold’em poker used generic offtheshelf linear program solvers for the equilibrium analysis of the abstracted game, we make use of a recently developed algorithm based on the excessive gap technique from convex optimization. This paper is, to our knowledge, the first to abstract and gametheoretically analyze all four betting rounds in one run (rather than splitting the game into phases). The resulting player, GS3, beats BluffBot, GS2, Hyperborean, MonashBPP, Sparbot, Teddy, and Vexbot, each with statistical significance. To our knowledge, those competitors are the best prior programs for the game.
A continuation method for Nash equilibria in structured games
 In Proceedings of the 18th International Joint Conference on Artificial Intelligence (IJCAI
, 2003
"... We describe algorithms for computing Nash equilibria in structured game representations, including both graphical games and multiagent influence diagrams (MAIDs). The algorithms are derived from a continuation method for normalform and extensiveform games due to Govindan and Wilson; they follow a ..."
Abstract

Cited by 47 (0 self)
 Add to MetaCart
(Show Context)
We describe algorithms for computing Nash equilibria in structured game representations, including both graphical games and multiagent influence diagrams (MAIDs). The algorithms are derived from a continuation method for normalform and extensiveform games due to Govindan and Wilson; they follow a trajectory through the space of perturbed games and their equilibria. Our algorithms exploit game structure through fast computation of the Jacobian of the game's payoff function. They are guaranteed to find at least one equilibrium of the game and may find more. Our approach provides the first exact algorithm for computing an exact equilibrium in graphical games with arbitrary topology, and the first algorithm to exploit finegrain structural properties of MAIDs. We present experimental results for our algorithms. The running time for our graphical game algorithm is similar to, and often better than, the running time of previous approximate algorithms. Our algorithm for MAIDs can effectively solve games that arc much larger than those that could be solved using previous methods. 1
Gradientbased algorithms for finding nash equilibria in extensive form games
 In Proceedings of the Eighteenth International Conference on Game Theory
, 2007
"... We present a computational approach to the saddlepoint formulation for the Nash equilibria of twoperson, zerosum sequential games of imperfect information. The algorithm is a firstorder gradient method based on modern smoothing techniques for nonsmooth convex optimization. The algorithm requires ..."
Abstract

Cited by 45 (16 self)
 Add to MetaCart
(Show Context)
We present a computational approach to the saddlepoint formulation for the Nash equilibria of twoperson, zerosum sequential games of imperfect information. The algorithm is a firstorder gradient method based on modern smoothing techniques for nonsmooth convex optimization. The algorithm requires O(1/ɛ) iterations to compute an ɛequilibrium, and the work per iteration is extremely low. These features enable us to find approximate Nash equilibria for sequential games with a tree representation of about 10 10 nodes. This is three orders of magnitude larger than what previous algorithms can handle. We present two heuristic improvements to the basic algorithm and demonstrate their efficacy on a range of realworld games. Furthermore, we demonstrate how the algorithm can be customized to a specific class of problems with enormous memory savings. 1
SMOOTHING TECHNIQUES FOR COMPUTING NASH EQUILIBRIA OF SEQUENTIAL GAMES
"... We develop firstorder smoothing techniques for saddlepoint problems that arise in the Nash equilibria computation of sequential games. The crux of our work is a construction of suitable proxfunctions for a certain class of polytopes that encode the sequential nature of the games. An implementatio ..."
Abstract

Cited by 40 (10 self)
 Add to MetaCart
(Show Context)
We develop firstorder smoothing techniques for saddlepoint problems that arise in the Nash equilibria computation of sequential games. The crux of our work is a construction of suitable proxfunctions for a certain class of polytopes that encode the sequential nature of the games. An implementation based on our smoothing techniques computes approximate Nash equilibria for games that are four orders of magnitude larger than what conventional computational approaches can handle.
Finding equilibria in large sequential games of imperfect information
 In ACM Conference on Electronic Commerce
, 2006
"... Information ∗ ..."