## SMOOTHING TECHNIQUES FOR COMPUTING NASH EQUILIBRIA OF SEQUENTIAL GAMES

### Cached

### Download Links

Citations: | 27 - 7 self |

### BibTeX

@MISC{Hoda_smoothingtechniques,

author = {Samid Hoda and Andrew Gilpin and Javier Peña},

title = {SMOOTHING TECHNIQUES FOR COMPUTING NASH EQUILIBRIA OF SEQUENTIAL GAMES},

year = {}

}

### Years of Citing Articles

### OpenURL

### Abstract

We develop first-order smoothing techniques for saddle-point problems that arise in the Nash equilibria computation of sequential games. The crux of our work is a construction of suitable prox-functions for a certain class of polytopes that encode the sequential nature of the games. An implementation based on our smoothing techniques computes approximate Nash equilibria for games that are four orders of magnitude larger than what conventional computational approaches can handle.

### Citations

1915 | A Course in Game Theory
- Osborne, Rubinstein
- 1994
(Show Context)
Citation Context ...game, branches that correspond to players’ moves, payoffs at the tree’s leaves, and information sets. For a detailed exposition on the extensive form representation, see, e.g., Osborne and Rubinstein =-=[18]-=-. We shall refer to the number of leaves in the game tree as the size of the game tree. We show next that for games with uniform treeplexes, the total number of basic arithmetic operations in each EGT... |

279 | 2005b. Smooth minimization of non-smooth functions
- Nesterov
(Show Context)
Citation Context ...1) when X and Y are polytopes of this type. Complexes generalize simplexes and include as a special case the strategy sets of sequential games. In this paper, we adapt Nesterov’s smoothing techniques =-=[11, 12]-=- for approximating (1). In particular, we develop first-order algorithms that take O(1/ɛ) iterations to compute x ∈ X and y ∈ Y such that 0 ≤ max〈v, Ax〉 − min〈y, Au〉 ≤ ɛ. v∈Y u∈X Such a pair of strate... |

222 |
Introductory lectures on convex optimization: Basic course
- Nesterov
- 2003
(Show Context)
Citation Context ...exity modulus of d.SMOOTHING TECHNIQUES FOR COMPUTING NASH EQUILIBRIA 3 • min{d(x) : x ∈ Q} = 0. When d : Q → R is differentiable, (4) can be equivalently stated in either of the following two forms =-=[10]-=-: (5) d(y) ≥ d(x) + 〈∇d(x), y − x〉 + 1 2 σ‖x − y‖2 for all x, y ∈ Q. (6) 〈∇d(x) + ∇d(y), x − y〉 ≥ σ‖x − y‖ 2 for all x, y ∈ Q. Assume dX and dY are prox-functions for the sets X and Y respectively. Th... |

134 | Approximating game-theoretic optimal strategies for full-scale poker
- Billings, Burch, et al.
- 2003
(Show Context)
Citation Context ...n five abstractions of poker games ranging from relatively small to very large. An abstraction of a game is a smaller game that captures some of the mainfeatures of the original game (Billings et al. =-=[2]-=-; Gilpin and Sandholm [4, 5]; Shi and Littman [20]). The approach of abstracting a game and then solving for the equilibrium of the abstracted game is a practical way of constructing good strategies f... |

128 |
Fundamentals of Convex Analysis
- Hirriart-Urruty, Lemaréchal
- 2001
(Show Context)
Citation Context ...tor e with entries in {0, 1}. We now present our general procedure for constructing nice prox-functions for complexes. The construction relies on the following dilation operation from convex analysis =-=[7]-=-. Given a compact set K ⊆ R d and a function Φ: K → R, define the set ¯ K ⊆ R d+1 as ¯K := { (x, y) ∈ R d+1 : x ∈ [0, 1], y ∈ x · K } ,SMOOTHING TECHNIQUES FOR COMPUTING NASH EQUILIBRIA 5 and define ... |

119 | The challenge of poker
- Billings, Davidson, et al.
- 2002
(Show Context)
Citation Context ...R n is a convex compact set. A function d : Q → R is a prox-function if it satisfies the following properties • d is strongly convex in Q, i.e., there exists σ > 0 such that for all x, y ∈ Q, and α ∈ =-=[0, 1]-=- (4) d(αx + (1 − α)y) ≤ αd(x) + (1 − α)d(y) − 1 2 σα(1 − α)‖x − y‖2 . The largest value of the constant σ that satisfies (4) is the strong convexity modulus of d.SMOOTHING TECHNIQUES FOR COMPUTING NA... |

106 | Robust stochastic approximation approach to stochastic programming
- Nemirovski, Juditsky, et al.
(Show Context)
Citation Context ...plexes and include as a special case the strategy sets of sequential games. Our approach follows a current trend of applying first-order algorithms to nonsmooth optimization problems (Juditsky et al. =-=[10]-=-; Lan et al. [12]; Nemirovski [13]; Nesterov [16, 17]). A key feature of these algorithms is their low computational cost per iteration, which makes them particularly attractive for large problems. We... |

90 | Efficient Computation of Equilibria for Extensive Two-Person Games
- Koller, Megiddo, et al.
- 1996
(Show Context)
Citation Context ...zero-sum sequential games are the solutions to min x∈� max y∈� �y�Ax�=max y∈� min�y�Ax�� (1) x∈� where � and � are polytopes defining the players’ strategies and A is the payoff matrix (Koller et al. =-=[11]-=-; Romanovskii [19]; vonStengel [21, 22]). Whenthe minimizer plays a strategy x ∈ � and the maximizer plays y ∈ �, the expected utility to the maximizer is �y�Ax�, and since the game is zero sum, the m... |

75 | Prox-method with rate of convergence O(1/t) for variational inequalities with Lipschitz continuous monotone operators and smooth convex-concave saddle point problems
- Nemirovski
(Show Context)
Citation Context ...se the strategy sets of sequential games. Our approach follows a current trend of applying first-order algorithms to nonsmooth optimization problems (Juditsky et al. [10]; Lan et al. [12]; Nemirovski =-=[13]-=-; Nesterov [16, 17]). A key feature of these algorithms is their low computational cost per iteration, which makes them particularly attractive for large problems. We adapt Nesterov’s [16, 17] smoothi... |

65 |
A method for unconstrained convex minimization problem with the rate of convergence O(1/k2
- Nesterov
- 1983
(Show Context)
Citation Context ...ion of h, such as its gradient or subgradient. When h is smooth with Lipschitz gradient, there is a first-order algorithm for finding a point x ∈ X such that h(x) ≤ ¯ h + ɛ after O(1/ √ ɛ) iterations =-=[9]-=-. When h is non-smooth, subgradient algorithms can be applied, but they have a worst-case complexity of O(1/ɛ 2 ) iterations [6]. However, that pessimistic result is based on treating h as a black-box... |

61 | Regret minimization in games with incomplete information
- Zinkevich, Bowling, et al.
- 2007
(Show Context)
Citation Context ...ce (WINE-07), an algorithm based on a totally different paradigm (regret minimization at each information set of the game), but comparable performance, was published in a conference (Zinkevich et al. =-=[23]-=-). It would be interesting to conduct direct scalability comparisons of the two algorithms in the future. For one, we expect that our algorithm exhibits better parallelization; no convergence guarante... |

52 | Excessive gap technique in nonsmooth convex minimization
- Nesterov
(Show Context)
Citation Context ...1) when X and Y are polytopes of this type. Complexes generalize simplexes and include as a special case the strategy sets of sequential games. In this paper, we adapt Nesterov’s smoothing techniques =-=[11, 12]-=- for approximating (1). In particular, we develop first-order algorithms that take O(1/ɛ) iterations to compute x ∈ X and y ∈ Y such that 0 ≤ max〈v, Ax〉 − min〈y, Au〉 ≤ ɛ. v∈Y u∈X Such a pair of strate... |

50 | Efficient computation of behavior strategies
- Stengel
- 1996
(Show Context)
Citation Context ... two-person, zero-sum sequential games are the solutions to (1) min x∈X max〈y, Ax〉 = max y∈Y y∈Y min〈y, Ax〉 x∈X where X and Y are polytopes defining the players’ strategies and A is the payoff matrix =-=[13, 14]-=-. When the minimizer plays a strategy x ∈ X and the maximizer plays y ∈ Y, the expected utility to the maximizer is 〈y, Ax〉 and, since the game is zerosum, the minimizer’s expected utility is 〈y, −Ax〉... |

39 | Potential-aware automated abstraction of sequential games, and holistic equilibrium analysis of Texas Hold’em poker
- Gilpin, Sandholm, et al.
- 2007
(Show Context)
Citation Context ...s more than 1012 entries [2]. (This is four orders of magnitude larger than what was previously possible.) That implementation is a key component of several successful poker-playing computer programs =-=[4, 5]-=-. The paper is organized as follows. Section 2 summarizes Nesterov’s smoothing technique as it applies to problem (1). We highlight that technique’s crucial ingredient, a pair of suitable prox-functio... |

37 |
Abstraction methods for game theoretic poker
- Shi, Littman
(Show Context)
Citation Context ...elatively small to very large. An abstraction of a game is a smaller game that captures some of the mainfeatures of the original game (Billings et al. [2]; Gilpin and Sandholm [4, 5]; Shi and Littman =-=[20]-=-). The approach of abstracting a game and then solving for the equilibrium of the abstracted game is a practical way of constructing good strategies for the original game (Billings et al. [2], Gilpin ... |

35 | Gradient-based algorithms for finding Nash equilibria in extensive form games
- Gilpin, Hoda, et al.
- 2007
(Show Context)
Citation Context ...+1 = (1 − τ)xk + τ ˆx (vi) yk+1 = (1 − τ)yk + τ ˜y (vii) µ k+1 Y = (1 − τ)µk Y 5.2. Computational experiments. Here we summarize our computational experiments which were published in a separate paper =-=[2]-=-. We first conducted experiments comparing the performance of the EGT algorithm for the prox-functions induced by the entropy and Euclidean prox-functions on simplexes. Based on our experience with a ... |

34 |
Reduction of a game with complete memory to a matrix game
- Romanovskii
- 1962
(Show Context)
Citation Context ...l games are the solutions to min x∈� max y∈� �y�Ax�=max y∈� min�y�Ax�� (1) x∈� where � and � are polytopes defining the players’ strategies and A is the payoff matrix (Koller et al. [11]; Romanovskii =-=[19]-=-; vonStengel [21, 22]). Whenthe minimizer plays a strategy x ∈ � and the maximizer plays y ∈ �, the expected utility to the maximizer is �y�Ax�, and since the game is zero sum, the minimizer’s expecte... |

27 |
On the convergence rate of subgradient optimization methods
- Goffin
- 1977
(Show Context)
Citation Context ...nding a point x ∈ X such that h(x) ≤ ¯ h + ɛ after O(1/ √ ɛ) iterations [9]. When h is non-smooth, subgradient algorithms can be applied, but they have a worst-case complexity of O(1/ɛ 2 ) iterations =-=[6]-=-. However, that pessimistic result is based on treating h as a black-box where the value and subgradient are accessed via an oracle. For non-smooth functions with a suitable max structure, Nesterov de... |

25 |
A heads-up no-limit Texas Hold’em poker player: Discretized betting models and automatically generated equilibriumfinding programs
- Gilpin, Sandholm, et al.
- 2008
(Show Context)
Citation Context ...s more than 1012 entries [2]. (This is four orders of magnitude larger than what was previously possible.) That implementation is a key component of several successful poker-playing computer programs =-=[4, 5]-=-. The paper is organized as follows. Section 2 summarizes Nesterov’s smoothing technique as it applies to problem (1). We highlight that technique’s crucial ingredient, a pair of suitable prox-functio... |

22 | Lossless abstraction of imperfect information games
- Gilpin, Sandholm
(Show Context)
Citation Context ... a substantially smaller game with a 106 × 106 payoff matrix containing 50 million non-zeros with conventional linear programming solvers is computationally demanding both in terms of time and memory =-=[3]-=-. We present a novel algorithmic approach for finding approximate solutions to (1). To this end, we define polytopes called complexes and concentrate on solving (1) when X and Y are polytopes of this ... |

16 |
Primal-dual first-order methods with o(1/ɛ) iteration-complexity for cone programming
- Lan, Lu, et al.
- 2009
(Show Context)
Citation Context ...f magnitude larger than what conventional computational approaches can handle. In contrast to a direct first-order approach to solve the linear programming formulation of (1) such as that proposed in =-=[8]-=-, our approach automatically yields algorithms that generate feasible strategies x ∈ X , y ∈ Y throughout execution. This is of crucial importance because points that violate the constraints defining ... |

5 |
Primal-dual First-order Methods with O(1/) Iterationcomplexity for Cone Programming
- Lan, Lu, et al.
(Show Context)
Citation Context ...e as a special case the strategy sets of sequential games. Our approach follows a current trend of applying first-order algorithms to nonsmooth optimization problems (Juditsky et al. [10]; Lan et al. =-=[12]-=-; Nemirovski [13]; Nesterov [16, 17]). A key feature of these algorithms is their low computational cost per iteration, which makes them particularly attractive for large problems. We adapt Nesterov’s... |

3 |
Lossless abstraction method for sequential games of imperfect information
- Gilpin, Sandholm
(Show Context)
Citation Context ...er game with a 10 6 × 10 6 payoff matrix containing 50 million nonzeros with conventional linear programming solvers is computationally demanding both in terms of time and memory (Gilpin and Sandholm =-=[5]-=-). We present a novel algorithmic approach for finding approximate solutions to (1). To this end, we define polytopes called treeplexes and concentrate on solving (1) when � and � are polytopes of thi... |

2 |
Equilibrium computation for games in strategic and extensive form
- Stengel
- 2007
(Show Context)
Citation Context ... two-person, zero-sum sequential games are the solutions to (1) min x∈X max〈y, Ax〉 = max y∈Y y∈Y min〈y, Ax〉 x∈X where X and Y are polytopes defining the players’ strategies and A is the payoff matrix =-=[13, 14]-=-. When the minimizer plays a strategy x ∈ X and the maximizer plays y ∈ Y, the expected utility to the maximizer is 〈y, Ax〉 and, since the game is zerosum, the minimizer’s expected utility is 〈y, −Ax〉... |

2 | Algorithms for abstracting and solving imperfect information games
- Gilpin
- 2009
(Show Context)
Citation Context ... instances are lossy (nonequilibrium preserving) abstractions, while the RI instance is a lossless abstraction. The Texas and GS4 instances are lossy abstractions of limit Texas Hold’em poker (Gilpin =-=[3]-=-, Gilpin et al. [6]). Table 2 provides the average time per EGT iterationof our implementationfor each of the test problems both with and without the heuristics. Table 1. Problem sizes (whenformulated... |