Results 1  10
of
30
Recursive Markov chains, stochastic grammars, and monotone systems of nonlinear equations
 IN STACS
, 2005
"... We define Recursive Markov Chains (RMCs), a class of finitely presented denumerable Markov chains, and we study algorithms for their analysis. Informally, an RMC consists of a collection of finitestate Markov chains with the ability to invoke each other in a potentially recursive manner. RMCs offer ..."
Abstract

Cited by 95 (13 self)
 Add to MetaCart
We define Recursive Markov Chains (RMCs), a class of finitely presented denumerable Markov chains, and we study algorithms for their analysis. Informally, an RMC consists of a collection of finitestate Markov chains with the ability to invoke each other in a potentially recursive manner. RMCs offer a natural abstract model for probabilistic programs with procedures. They generalize, in a precise sense, a number of well studied stochastic models, including Stochastic ContextFree Grammars (SCFG) and MultiType Branching Processes (MTBP). We focus on algorithms for reachability and termination analysis for RMCs: what is the probability that an RMC started from a given state reaches another target state, or that it terminates? These probabilities are in general irrational, and they arise as (least) fixed point solutions to certain (monotone) systems of nonlinear equations associated with RMCs. We address both the qualitative problem of determining whether the probabilities are 0, 1 or inbetween, and
On the Complexity of Nash Equilibria and Other Fixed Points (Extended Abstract)
 IN PROC. FOCS
, 2007
"... We reexamine what it means to compute Nash equilibria and, more generally, what it means to compute a fixed point of a given Brouwer function, and we investigate the complexity of the associated problems. Specifically, we study the complexity of the following problem: given a finite game, Γ, with 3 ..."
Abstract

Cited by 68 (8 self)
 Add to MetaCart
We reexamine what it means to compute Nash equilibria and, more generally, what it means to compute a fixed point of a given Brouwer function, and we investigate the complexity of the associated problems. Specifically, we study the complexity of the following problem: given a finite game, Γ, with 3 or more players, and given ɛ> 0, compute an approximation within ɛ of some (actual) Nash equilibrium. We show that approximation of an actual Nash Equilibrium, even to within any nontrivial constant additive factor ɛ < 1/2 in just one desired coordinate, is at least as hard as the long standing squareroot sum problem, as well as a more general arithmetic circuit decision problem that characterizes Ptime in a unitcost model of computation with arbitrary precision rational arithmetic; thus placing the approximation problem in P, or even NP, would resolve major open problems in the complexity of numerical computation. We show similar results for market equilibria: it is hard to estimate with any nontrivial accuracy the equilibrium prices in an exchange economy with a unique equilibrium, where the economy is given by explicit algebraic formulas for the excess demand functions. We define a class, FIXP, which captures search problems that can be cast as fixed point
Concurrent Reachability Games
, 2008
"... We consider concurrent twoplayer games with reachability objectives. In such games, at each round, player 1 and player 2 independently and simultaneously choose moves, and the two choices determine the next state of the game. The objective of player 1 is to reach a set of target states; the objecti ..."
Abstract

Cited by 68 (22 self)
 Add to MetaCart
We consider concurrent twoplayer games with reachability objectives. In such games, at each round, player 1 and player 2 independently and simultaneously choose moves, and the two choices determine the next state of the game. The objective of player 1 is to reach a set of target states; the objective of player 2 is to prevent this. These are zerosum games, and the reachability objective is one of the most basic objectives: determining the set of states from which player 1 can win the game is a fundamental problem in control theory and system verification. There are three types of winning states, according to the degree of certainty with which player 1 can reach the target. From type1 states, player 1 has a deterministic strategy to always reach the target. From type2 states, player 1 has a randomized strategy to reach the target with probability 1. From type3 states, player 1 has for every real ε> 0 a randomized strategy to reach the target with probability greater than 1 − ε. We show that for finite state spaces, all three sets of winning states can be computed in polynomial time: type1 states in linear time, and type2 and type3 states in quadratic time. The algorithms to compute the three sets of winning states also enable the construction of the winning and spoiling strategies.
Recursive Markov decision processes and recursive stochastic games
 In Proc. of 32nd Int. Coll. on Automata, Languages, and Programming (ICALP’05
, 2005
"... Abstract. We introduce Recursive Markov Decision Processes (RMDPs) and Recursive Simple Stochastic Games (RSSGs), and study the decidability and complexity of algorithms for their analysis and verification. These models extend Recursive Markov Chains (RMCs), introduced in [EY05a,EY05b] as a natural ..."
Abstract

Cited by 52 (11 self)
 Add to MetaCart
Abstract. We introduce Recursive Markov Decision Processes (RMDPs) and Recursive Simple Stochastic Games (RSSGs), and study the decidability and complexity of algorithms for their analysis and verification. These models extend Recursive Markov Chains (RMCs), introduced in [EY05a,EY05b] as a natural model for verification of probabilistic procedural programs and related systems involving both recursion and probabilistic behavior. RMCs define a class of denumerable Markov chains with a rich theory generalizing that of stochastic contextfree grammars and multitype branching processes, and they are also intimately related to probabilistic pushdown systems. RMDPs & RSSGs extend RMCs with one controller or two adversarial players, respectively. Such extensions are useful for modeling nondeterministic and concurrent behavior, as well as modeling a system’s interactions with an environment. We provide a number of upper and lower bounds for deciding, given an RMDP (or RSSG) A and probability p, whether player 1 has a strategy to force termination at a desired exit with probability at least p. We also address “qualitative ” termination questions, where p = 1, and model checking questions. 1
Strategy improvement for concurrent reachability games
 In Proceedings of the Third Annual Conference on Quantitative Evaluation of Systems. IEEE Computer
, 2006
"... A concurrent reachability game is a twoplayer game played on a graph: at each state, the players simultaneously and independently select moves; the two moves determine jointly a probability distribution over the successor states. The objective for player 1 consists in reaching a set of target state ..."
Abstract

Cited by 17 (6 self)
 Add to MetaCart
(Show Context)
A concurrent reachability game is a twoplayer game played on a graph: at each state, the players simultaneously and independently select moves; the two moves determine jointly a probability distribution over the successor states. The objective for player 1 consists in reaching a set of target states; the objective for player 2 is to prevent this, so that the game is zerosum. Our contributions are twofold. First, we present a simple proof of the fact that in concurrent reachability games, for all ε>0, memoryless εoptimal strategies exist. A memoryless strategy is independent of the history of plays, and an εoptimal strategy achieves the objective with probability within ε of the value of the game. In contrast to previous proofs of this fact, which rely on the limit behavior of discounted games using advanced Puisieux series analysis, our proof is elementary and combinatorial. Second, we present a strategyimprovement (a.k.a. policyiteration) algorithm for concurrent games with reachability objectives. 1.
Winning concurrent reachability games requires doublyexponential patience
"... We exhibit a deterministic concurrent reachability game PURGATORYn with n nonterminal positions and a binary choice for both players in every position so that any positional strategy for Player 1 achieving the value of the game within given ɛ < 1/2 must use nonzero behavior probabilities that a ..."
Abstract

Cited by 13 (6 self)
 Add to MetaCart
(Show Context)
We exhibit a deterministic concurrent reachability game PURGATORYn with n nonterminal positions and a binary choice for both players in every position so that any positional strategy for Player 1 achieving the value of the game within given ɛ < 1/2 must use nonzero behavior probabilities that are less than (ɛ2/(1 − ɛ)) 2n−2. Also, even to achieve the value within say 1 − 2−n/2, doubly exponentially small behavior probabilities in the number of positions must be used. This behavior is close to worst case: We show that for any such game and 0 < ɛ < 1/2, there is an ɛoptimal strategy with all nonzero behavior probabilities being at least ɛ2O(n). As a corollary to our results, we conclude that any (deterministic or nondeterministic) algorithm that given a concurrent reachability game explicitly manipulates ɛoptimal strategies for Player 1 represented in several standard ways (e.g., with binary representation of probabilities or as the uniform distribution over a multiset) must use at least exponential space in the worst case.
PartialObservation Stochastic Games: How to Win when Belief Fails
, 2011
"... Abstract. In twoplayer finitestate stochastic games of partial observation on graphs, in every state of the graph, the players simultaneously choose an action, and their joint actions determine a probability distribution over the successor states. The game is played for infinitely many rounds and ..."
Abstract

Cited by 12 (6 self)
 Add to MetaCart
Abstract. In twoplayer finitestate stochastic games of partial observation on graphs, in every state of the graph, the players simultaneously choose an action, and their joint actions determine a probability distribution over the successor states. The game is played for infinitely many rounds and thus the players construct an infinite path in the graph. We consider reachability objectives where the first player tries to ensure a target state to be visited almostsurely (i.e., with probability 1) or positively (i.e., with positive probability), no matter the strategy of the second player. We classify such games according to the information and to the power of randomization available to the players. On the basis of information, the game can be onesided with either (a) player 1, or (b) player 2 having partial observation (andtheotherplayerhasperfect observation),or twosided with (c) both players having partial observation. On the basis of randomization, (a) the players may not be allowed to use randomization
QUALITATIVE REACHABILITY IN STOCHASTIC BPA GAMES
, 2009
"... We consider a class of infinitestate stochastic games generated by stateless pushdown automata (or, equivalently, 1exit recursive state machines), where the winning objective is specified by a regular set of target configurations and a qualitative probability constraint ‘>0’ or ‘=1’. The goal ..."
Abstract

Cited by 10 (1 self)
 Add to MetaCart
We consider a class of infinitestate stochastic games generated by stateless pushdown automata (or, equivalently, 1exit recursive state machines), where the winning objective is specified by a regular set of target configurations and a qualitative probability constraint ‘>0’ or ‘=1’. The goal of one player is to maximize the probability of reaching the target set so that the constraint is satisfied, while the other player aims at the opposite. We show that the winner in such games can be determined in NP∩coNP. Further, we prove that the winning regions for both players are regular, and we design algorithms which compute the associated finitestate automata. Finally, we show that winning strategies can be synthesized effectively.