## Quantitative Solution of Omega-Regular Games

### Cached

### Download Links

Citations: | 39 - 13 self |

### BibTeX

@MISC{Alfaro_quantitativesolution,

author = {Luca de Alfaro and Rupak Majumdar},

title = {Quantitative Solution of Omega-Regular Games},

year = {}

}

### Years of Citing Articles

### OpenURL

### Abstract

We consider two-player games played for an infinite number of rounds, with ω-regular winning conditions. The games may be concurrent, in that the players choose their moves simultaneously and independently, and probabilistic, in that the moves determine a probability distribution for the successor state. We introduce quantitative game µ-calculus, and we show that the maximal probability of winning such games can be expressed as the fixpoint formulas in this calculus. We develop the arguments both for deterministic and for probabilistic concurrent games; as a special case, we solve probabilistic turn-based games with ω-regular winning conditions, which was also open. We also characterize the optimality, and the memory requirements, of the winning strategies. In particular, we show that while memoryless strategies suffice for winning games with safety and reachability conditions, Büchi conditions require the use of strategies with infinite memory. The existence of optimal strategies, as opposed to ε-optimal, is only guaranteed in games with safety winning conditions.

### Citations

1545 |
The Temporal Logic of Reactive and Concurrent Systems: Specification
- Manna, Pnueli
- 1991
(Show Context)
Citation Context ...ame structure G = hS; Moves ; 1 ; 2 ; pi, we consider winning conditions expressed by linear-time temporal logic (LTL) formulas, whose atomic propositions correspond to subsets of the set S of states =-=[1-=-6]. We focus on winning conditions that correspond to safety or reachability properties, as well as winning conditions that correspond to the accepting criteria of Buchi, co-Buchi, and Rabin-chain aut... |

1104 |
Theory of Games and Economic Behavior
- Neumann, Morgenstern
- 1947
(Show Context)
Citation Context ...ls[0; 1], the function g = Ppre(f) associates with each state the maximal expected value of f that player 1 can ensure in one step. The operator Ppre can be evaluated using results about matrix games =-=[29, 23]-=-. Related quantitative predecessor operators for one-player structures were considered in [13, 20, 12, 18]. We show that the values of concurrent games with !-regular conditions can be obtained simply... |

572 | A decision method for elementary algebra and geometry - Tarski - 1951 |

480 |
Dynamic programming and optimal control, Athena Scientific
- Bertsekas
- 1995
(Show Context)
Citation Context ...iform framework for understanding and solving concurrent games with !-regular winning conditions. In quantitative game -calculus, sets of states are replaced by functions from states to the interval [=-=0; 1]-=-, and the controllable predecessor operator Cpre is replaced by a quantitative version Ppre. Given a function f from states to the intervals[0; 1], the function g = Ppre(f) associates with each state ... |

336 | Dynamic Programming and Optimal Control - Bertsekas - 1995 |

302 | Results on the Propositional µ-Calculus - Kozen - 1983 |

274 |
Probability with martingales
- Williams
- 1991
(Show Context)
Citation Context ...matrix game corresponding to Ppre 1 (w)(s), and at all s 2 S n U plays arbitrarily. Fix a state s0 2 S and an arbitrary strategys2 2 2 . The process fHng n0 dened by Hn = w(n) is a submartingale [30]: in fact, from w(s) = Ppre 1 (w)(s) for s 2 U and from the choice of 1 follows that E 1 ; 2 s 0 fHn+1 j H0 ; H1 ; : : : ; Hng Hn for all n 0. Hence, we have E 1 ; 2 s 0 fHng H0 = w(s0 ).... |

262 |
Results on the propositional -calculus
- Kozen
- 1983
(Show Context)
Citation Context ...ions can be computed with the algorithms of [2, 11, 8, 26]. The algorithms of [8] are based 1 on the use of game -calculus, obtained by replacing the predecessor operator Pre of classical -calculus [1=-=4-=-] by the controllable predecessor operator Cpre: for a set of states U , the set Cpre(U) consists of the states from which player 1 can force the game into U in one step. A richer version of game -cal... |

261 |
Game Theory
- Owen
- 1995
(Show Context)
Citation Context ...ls[0; 1], the function g = Ppre(f) associates with each state the maximal expected value of f that player 1 can ensure in one step. The operator Ppre can be evaluated using results about matrix games =-=[29, 23]-=-. Related quantitative predecessor operators for one-player structures were considered in [13, 20, 12, 18]. We show that the values of concurrent games with !-regular conditions can be obtained simply... |

228 |
Competitive Markov Decision Processes
- Filar, Vrieze
- 1997
(Show Context)
Citation Context ...ves, and by studying the maximal discounted, total, or average reward that player 1 can obtain in such a game; a survey of algorithms for solving games with respect to such winning conditions is e.g. =-=[24, 10-=-]. Here, we consider win This research was supported in part by the AFOSR MURI grant F49620-00-1-0327 and the NSF Theory grant CCR9988172. Permission to make digital or hard copies of all or part of t... |

223 | Automatic verification of probabilistic concurrent finite-state programs - Vardi - 1985 |

216 | Tree automata, mu-calculus and determinacy (extended abstract
- Emerson, Jutla
- 1991
(Show Context)
Citation Context ...gardless of the strategy of player 2; indeed, the value of the game at s0 is 1/2. The value of deterministic turn-based games with !- regular winning conditions can be computed with the algorithms of =-=[2, 11, 8, 2-=-6]. The algorithms of [8] are based 1 on the use of game -calculus, obtained by replacing the predecessor operator Pre of classical -calculus [14] by the controllable predecessor operator Cpre: for a ... |

213 | Model checking of probabilistic and nondeterministic systems - Bianco, Alfaro - 1995 |

179 | Sequential circuit verification using symbolic model checking - Burch, Clarke, et al. - 1990 |

155 | On the synthesis of strategies in infinite games - Thomas - 1995 |

137 | Solving sequential conditions by finite-state strategies - Büchi, Landweber - 1969 |

123 | Borel determinacy - Martin - 1975 |

110 | Probabilistic Predicate Transformers
- Morgan, McIver, et al.
- 1996
(Show Context)
Citation Context ... player 1 can ensure in one step. The operator Ppre can be evaluated using results about matrix games [29, 23]. Related quantitative predecessor operators for one-player structures were considered in =-=[13, 20, 12, 18-=-]. We show that the values of concurrent games with !-regular conditions can be obtained simply by replacing Cpre by Ppre in the solutions of [8]. The result is surprising because concurrent games die... |

106 | On the complexity of ω-automata - Safra - 1988 |

98 |
A Probabilistic PDL
- Kozen
- 1985
(Show Context)
Citation Context ... player 1 can ensure in one step. The operator Ppre can be evaluated using results about matrix games [29, 23]. Related quantitative predecessor operators for one-player structures were considered in =-=[13, 20, 12, 18-=-]. We show that the values of concurrent games with !-regular conditions can be obtained simply by replacing Cpre by Ppre in the solutions of [8]. The result is surprising because concurrent games die... |

87 | It usually works: the temporal logic of stochastic systems - Aziz, Singhal, et al. - 1995 |

85 | Infinite games played on finite graphs - McNaughton - 1993 |

67 |
The determinacy of Blackwell games
- Martin
- 1998
(Show Context)
Citation Context ...s, we study the maximal probability with which player 1 can ensure that the condition holds from s; we call this maximal probability the value of the game at s for player 1. The determinacy result of =-=[17]-=- ensures that, at all states and for all !-regular winning conditions, the value of the game for player 1 is equal to one minus the value of the game with complementary condition for player 2. We dist... |

63 | Quantitative Analysis and Model Checking
- Huth, Kwiatkowska
- 1997
(Show Context)
Citation Context ... player 1 can ensure in one step. The operator Ppre can be evaluated using results about matrix games [29, 23]. Related quantitative predecessor operators for one-player structures were considered in =-=[13, 20, 12, 18-=-]. We show that the values of concurrent games with !-regular conditions can be obtained simply by replacing Cpre by Ppre in the solutions of [8]. The result is surprising because concurrent games die... |

61 |
M.: Markov decision processes and regular events (extended abstract
- Courcoubetis, Yannakakis
- 1990
(Show Context)
Citation Context ...decision processes, where monotonically-converging approximation schemes are available, and where the maximal winning probability can be computed in polynomial time by reduction to linear programming =-=[5]-=-. We show that this discrepancy is no accident, since the basic device for solving Markov decision processes with !-regular conditions, viz., a reduction to reachability, fails for games. 2. CONCURREN... |

45 | Regular Expressions for Infinite Trees and a Standard Form of Automata - Mostowski - 1984 |

44 | O.: Concurrent reachability games - Alfaro, Henzinger, et al. - 2007 |

35 | Symbolic model checking of concurrent probabilistic processes using MTBDDs and the Kronecker representation
- Alfaro, Kwiatkowska, et al.
- 2000
(Show Context)
Citation Context ... algorithms for the computation of approximate solutions. By representing value functions symbolically, these algorithms may be used for the approximate analysis of games with very large state spaces =-=[3, 7]-=-. Unfortunately, except for safety and reachability conditions, the alternance of least and greatestsxpoint operators in the solution formulas leads to approximation schemes that do not converge monot... |

30 | T.A.: Concurrent omega-regular games
- Alfaro, Henzinger
(Show Context)
Citation Context ...le predecessor operator Cpre: for a set of states U , the set Cpre(U) consists of the states from which player 1 can force the game into U in one step. A richer version of game -calculus was used in [=-=6]-=- to provide qualitative solutions for concurrent probabilistic games with !-regular conditions. There, multi-argument predecessor operators are used to compute the set of states from which player 1 ca... |

26 |
Multi-Terminal Binary Decision Diagrams and Hybrid Decision Diagrams
- Clarke, Fujita, et al.
- 1996
(Show Context)
Citation Context ...r n 0. A quantitative game -calculus formula suggests a way to implement approximation algorithms for large state spaces, using a subset F 0 F of base functions that have compact representations [1,=-= 4, 7]-=-. We note that the solution algorithms presented in this paper apply also to games with countable (rather thansnite) state space andsnite set of moves (see Theorem 4); in this case, however, the itera... |

26 |
Recursive games
- EVERETT
- 1957
(Show Context)
Citation Context ... Let w = [[x:(U _ Ppre 1 (x))]]. For all " > 0 player 1 has a strategy " 1 such that Pr " 1 ; 2 s (3U) > w(s) " for all 2 2 2 and all s 2 S. The proof follows a classical argumen=-=t (see, e.g., [9, 1-=-0]). For n 0, consider the n-step version of the game, whose winning condition 3nU requires reaching U in at most n steps. Let also x0 = 0 and xn+1 = U _ Ppre 1 (xn) for n 0. By induction on n, we c... |

18 |
On the synthesis of strategies in in games
- Thomas
- 1995
(Show Context)
Citation Context ...-8, 2001, Hersonissos, Crete, Greece. Copyright 2001 ACM 1-58113-349-9/01/0007 ...$5.00. ning conditions consisting in !-regular automata acceptance conditions dened over the state space of the game [=-=2, 11, 26]-=-. Given a game with an !-regular winning condition and a starting state s, we study the maximal probability with which player 1 can ensure that the condition holds from s; we call this maximal probabi... |

16 |
Algorithms for stochastic games — a survey
- Raghavan, Filar
- 1991
(Show Context)
Citation Context ...ves, and by studying the maximal discounted, total, or average reward that player 1 can obtain in such a game; a survey of algorithms for solving games with respect to such winning conditions is e.g. =-=[24, 10-=-]. Here, we consider win This research was supported in part by the AFOSR MURI grant F49620-00-1-0327 and the NSF Theory grant CCR9988172. Permission to make digital or hard copies of all or part of t... |

16 | Exponential Determinization for ω-Automata with Strong-Fairnes Acceptance Condition - Safra - 1992 |

15 | angelic and unbounded probabilistic choices in sequential programs - McIver, Morgan - 2001 |

14 |
Existence of value and randomized strategies in zero-sum discrete-time stochastic dynamic games
- Kumar, Shiau
- 1981
(Show Context)
Citation Context ...trategies for concurrent reachability games follows from [22]. The existence of deterministic concurrent reachability games without optimal strategies is demonstrated by Example 2 below, adapted from =-=[9, 15]-=-. The existence of optimal strategies for concurrent safety games is classical; it also follows from the proof of Lemma 2. The existence of deterministic concurrent safety games without optimal determ... |

12 |
Sequential circuit veri using symbolic model checking
- Burch, Clarke, et al.
- 1990
(Show Context)
Citation Context ... algorithms for the computation of approximate solutions. By representing value functions symbolically, these algorithms may be used for the approximate analysis of games with very large state spaces =-=[3, 7]-=-. Unfortunately, except for safety and reachability conditions, the alternance of least and greatestsxpoint operators in the solution formulas leads to approximation schemes that do not converge monot... |

11 |
In games played on graphs
- McNaughton
- 1993
(Show Context)
Citation Context ... games both "-optimal strategies, and optimal strategies if they exist, may need an innite amount of memory [6]. Fourth, the standard recursive structure of proofs for deterministic turn-based ga=-=mes [19, 26]-=- breaks down, as both players can choose a distribution over moves at each state. We develop the arguments both for deterministic and for probabilistic concurrent games. Hence, as a special case we so... |

9 | Reasoning about efficiency within a probabilistic µ-calculus - McIver - 1998 |

8 |
Automatic veri of probabilistic concurrent state programs
- Vardi
- 1985
(Show Context)
Citation Context ...omaton [21, 26]. Given an LTL winning conditions, by abuse of notation we denote equally bysthe set of paths s 2 that satisfys; this set is measurable for any choice of strategies for the two players [28]. Hence, the probability that a path satisessstarting from state s 2 S under strategies 1 ; 2 for the two players is Pr 1 ; 2 s ( ). Given a state s 2 S and a winning conditions, we are interest... |

7 | Solving sequential conditions by strategies - Buchi, Landweber - 1969 |

6 |
Regular expressions for in trees and a standard form of automata
- Mostowski
- 1984
(Show Context)
Citation Context ...ocus on winning conditions that correspond to safety or reachability properties, as well as winning conditions that correspond to the accepting criteria of Buchi, co-Buchi, and Rabin-chain automata [2=-=1,-=- 8]. We call games with such winning conditions safety, reachability, Buchi, co-Buchi, and Rabin-chain games, respectively. The 1 To be precise, we should dene events as measurable sets of paths shari... |

6 |
The bad match, a total reward stochastic game, Operations Research Spektrum 9
- Thuijsman, Vrieze
- 1987
(Show Context)
Citation Context ...s usually necessary to consider discounted versions of the games. 3. REACHABILITY AND SAFETY GAMES Concurrent reachability and safety games can be solved by reducing them to positive stochastic games =-=[27, 1-=-0]. We present the solution algorithms, reformulating them in quantitative game -calculus. As mentioned above, by relying on 4 the complementation of quantitative game -calculus, we are able to prove ... |

5 |
On the existence of stationary optimal strategies
- Ornstein
- 1969
(Show Context)
Citation Context ...sical [9, 10], except for the notation; the result also follows from the combination of Lemmas 1 and 2. The existence of memoryless "-optimal strategies for concurrent reachability games follows =-=from [22]-=-. The existence of deterministic concurrent reachability games without optimal strategies is demonstrated by Example 2 below, adapted from [9, 15]. The existence of optimal strategies for concurrent s... |

5 | Filar, “Algorithms for stochastic games — a survey - Raghavan, A - 1991 |

3 |
Reasoning about eciency within a probabilistic -calculus
- McIver
- 1998
(Show Context)
Citation Context |