Results 1  10
of
24
Recursive Markov chains, stochastic grammars, and monotone systems of nonlinear equations
 IN STACS
, 2005
"... We define Recursive Markov Chains (RMCs), a class of finitely presented denumerable Markov chains, and we study algorithms for their analysis. Informally, an RMC consists of a collection of finitestate Markov chains with the ability to invoke each other in a potentially recursive manner. RMCs offer ..."
Abstract

Cited by 67 (11 self)
 Add to MetaCart
We define Recursive Markov Chains (RMCs), a class of finitely presented denumerable Markov chains, and we study algorithms for their analysis. Informally, an RMC consists of a collection of finitestate Markov chains with the ability to invoke each other in a potentially recursive manner. RMCs offer a natural abstract model for probabilistic programs with procedures. They generalize, in a precise sense, a number of well studied stochastic models, including Stochastic ContextFree Grammars (SCFG) and MultiType Branching Processes (MTBP). We focus on algorithms for reachability and termination analysis for RMCs: what is the probability that an RMC started from a given state reaches another target state, or that it terminates? These probabilities are in general irrational, and they arise as (least) fixed point solutions to certain (monotone) systems of nonlinear equations associated with RMCs. We address both the qualitative problem of determining whether the probabilities are 0, 1 or inbetween, and
Algorithmic verification of recursive probabilistic state machines
 In Proc. 11th TACAS
, 2005
"... Abstract. Recursive Markov Chains (RMCs) ([EY04]) are a natural abstract model of procedural probabilistic programs and related systems involving recursion and probability. They succinctly define a class of denumerable Markov chains that generalize multitype branching (stochastic) processes. In thi ..."
Abstract

Cited by 37 (7 self)
 Add to MetaCart
Abstract. Recursive Markov Chains (RMCs) ([EY04]) are a natural abstract model of procedural probabilistic programs and related systems involving recursion and probability. They succinctly define a class of denumerable Markov chains that generalize multitype branching (stochastic) processes. In this paper, we study the problem of model checking an RMC against a given ωregular specification. Namely, given an RMC A and a Büchi automaton B, we wish to know the probability that an execution of A is accepted by B. We establish a number of strong upper bounds, as well as lower bounds, both for qualitative problems (is the probability = 1, or = 0?), and for quantitative problems (is the probability ≥ p?, or, approximate the probability to within a desired precision). Among these, we show that qualitative model checking for general RMCs can be decided in PSPACE in A  and EXPTIME in B, and when A is either a singleexit RMC or when the total number of entries and exits in A is bounded, it can be decided in polynomial time in A. We then show that quantitative model checking can also be done in PSPACE in A, and in EXPSPACE in B. When B is deterministic, all our complexities in B  come down by one exponential. For lower bounds, we show that the qualitative model checking problem, even for a fixed RMC, is already EXPTIMEcomplete. On the other hand, even for simple reachability analysis, we showed in [EY04] that our PSPACE upper bounds in A can not be improved upon without a breakthrough on a wellknown open problem in the complexity of numerical computation. 1
Recursive Markov decision processes and recursive stochastic games
 In Proc. of 32nd Int. Coll. on Automata, Languages, and Programming (ICALP’05
, 2005
"... Abstract. We introduce Recursive Markov Decision Processes (RMDPs) and Recursive Simple Stochastic Games (RSSGs), and study the decidability and complexity of algorithms for their analysis and verification. These models extend Recursive Markov Chains (RMCs), introduced in [EY05a,EY05b] as a natural ..."
Abstract

Cited by 37 (9 self)
 Add to MetaCart
Abstract. We introduce Recursive Markov Decision Processes (RMDPs) and Recursive Simple Stochastic Games (RSSGs), and study the decidability and complexity of algorithms for their analysis and verification. These models extend Recursive Markov Chains (RMCs), introduced in [EY05a,EY05b] as a natural model for verification of probabilistic procedural programs and related systems involving both recursion and probabilistic behavior. RMCs define a class of denumerable Markov chains with a rich theory generalizing that of stochastic contextfree grammars and multitype branching processes, and they are also intimately related to probabilistic pushdown systems. RMDPs & RSSGs extend RMCs with one controller or two adversarial players, respectively. Such extensions are useful for modeling nondeterministic and concurrent behavior, as well as modeling a system’s interactions with an environment. We provide a number of upper and lower bounds for deciding, given an RMDP (or RSSG) A and probability p, whether player 1 has a strategy to force termination at a desired exit with probability at least p. We also address “qualitative ” termination questions, where p = 1, and model checking questions. 1
Recursive concurrent stochastic games
 In Proc. of 33rd Int. Coll. on Automata, Languages, and Programming (ICALP’06
, 2006
"... Abstract. We study Recursive Concurrent Stochastic Games (RCSGs), extending our recent analysis of recursive simple stochastic games [16, 17] to a concurrent setting where the two players choose moves simultaneously and independently at each state. For multiexit games, our earlier work already show ..."
Abstract

Cited by 22 (4 self)
 Add to MetaCart
Abstract. We study Recursive Concurrent Stochastic Games (RCSGs), extending our recent analysis of recursive simple stochastic games [16, 17] to a concurrent setting where the two players choose moves simultaneously and independently at each state. For multiexit games, our earlier work already showed undecidability for basic questions like termination, thus we focus on the important case of singleexit RCSGs (1RCSGs). We first characterize the value of a 1RCSG termination game as the least fixed point solution of a system of nonlinear minimax functional equations, and use it to show PSPACE decidability for the quantitative termination problem. We then give a strategy improvement technique, which we use to show that player 1 (maximizer) has ǫoptimal randomized Stackless & Memoryless (rSM) strategies for all ǫ> 0, while player 2 (minimizer) has optimal rSM strategies. Thus, such games are rSMdetermined. These results mirror and generalize in a strong sense the randomized memoryless determinacy results for finite stochastic games, and extend the classic HoffmanKarp [22] strategy improvement approach from the finite to an infinite state setting. The proofs in our infinitestate setting are very different however, relying on subtle analytic properties of certain power series that arise from studying 1RCSGs. We show that our upper bounds, even for qualitative (probability 1) termination, can not be improved, even to NP, without a major breakthrough, by giving two reductions: first a Ptime reduction from the longstanding squareroot sum problem to the quantitative termination decision problem for finite concurrent stochastic games, and then a Ptime reduction from the latter problem to the qualitative termination problem for 1RCSGs. 1.
On the convergence of Newton’s method for monotone systems of polynomial equations
 In Proceedings of STOC
, 2007
"... kiefersn, luttenml, esparza o ..."
Efficient qualitative analysis of classes of recursive markov decision processes and simple stochastic games
 In Proc. STACS’06
, 2006
"... Abstract. Recursive Markov Decision Processes (RMDPs) and Recursive Simple Stochastic Games (RSSGs) are natural models for recursive systems involving both probabilistic and nonprobabilistic actions. As shown recently [10], fundamental problems about such models, e.g., termination, are undecidable ..."
Abstract

Cited by 13 (7 self)
 Add to MetaCart
Abstract. Recursive Markov Decision Processes (RMDPs) and Recursive Simple Stochastic Games (RSSGs) are natural models for recursive systems involving both probabilistic and nonprobabilistic actions. As shown recently [10], fundamental problems about such models, e.g., termination, are undecidable in general, but decidable for the important class of 1exit RMDPs and RSSGs. These capture controlled and game versions of multitype Branching Processes, an important and wellstudied class of stochastic processes. In this paper we provide efficient algorithms for the qualitative termination problem for these models: does the process terminate almost surely when the players use their optimal strategies? Polynomial time algorithms are given for both maximizing and minimizing 1exit RMDPs (the two cases are not symmetric). For 1exit RSSGs the problem is in NP∩coNP, and furthermore, it is at least as hard as other wellknown NP∩coNP problems on games, e.g., Condon’s quantitative termination problem for finite SSGs ([3]). For the class of linearlyrecursive 1exit RSSGs, we show that the problem can be solved in polynomial time.
Verifying Probabilistic Procedural Programs
, 2004
"... Monolithic nitestate probabilistic programs have been abstractly modeled by nite Markov chains, and the algorithmic veri  cation problems for them have been investigated very extensively. In this paper we survey recent work conducted by the authors together with colleagues on the algorithmi ..."
Abstract

Cited by 12 (3 self)
 Add to MetaCart
Monolithic nitestate probabilistic programs have been abstractly modeled by nite Markov chains, and the algorithmic veri  cation problems for them have been investigated very extensively. In this paper we survey recent work conducted by the authors together with colleagues on the algorithmic veri cation of probabilistic procedural programs ([BKS,EKM04,EY04]). Probabilistic procedural programs can more naturally be modeled by recursive Markov chains ([EY04]), or equivalently, probabilistic pushdown automata ([EKM04]). A very rich theory emerges for these models. While our recent work solves a number of veri cation problems for these models, many intriguing questions remain open.
CONVERGENCE THRESHOLDS OF NEWTON’S METHOD FOR MONOTONE POLYNOMIAL EQUATIONS
, 2008
"... Monotone systems of polynomial equations (MSPEs) are systems of fixedpoint equations X1 = f1(X1,..., Xn),..., Xn = fn(X1,..., Xn) where each fi is a polynomial with positive real coefficients. The question of computing the least nonnegative solution of a given MSPE X = f(X) arises naturally in the ..."
Abstract

Cited by 12 (8 self)
 Add to MetaCart
Monotone systems of polynomial equations (MSPEs) are systems of fixedpoint equations X1 = f1(X1,..., Xn),..., Xn = fn(X1,..., Xn) where each fi is a polynomial with positive real coefficients. The question of computing the least nonnegative solution of a given MSPE X = f(X) arises naturally in the analysis of stochastic models such as stochastic contextfree grammars, probabilistic pushdown automata, and backbutton processes. Etessami and Yannakakis have recently adapted Newton’s iterative method to MSPEs. In a previous paper we have proved the existence of a threshold kf for strongly connected MSPEs, such that after kf iterations of Newton’s method each new iteration computes at least 1 new bit of the solution. However, the proof was purely existential. In this paper we give an upper bound for kf as a function of the minimal component of the least fixedpoint µf of f(X). Using this result we show that kf is at most single exponential resp. linear for strongly connected MSPEs derived from probabilistic pushdown automata resp. from backbutton processes. Further, we prove the existence of a threshold for arbitrary MSPEs after which each new iteration computes at least 1/w2 h new bits of the solution, where w and h are the width and height of the DAG of strongly connected components.
Quasibirthdeath processes, TreeLike QBDs, probabilistic 1counter automata, and pushdown systems
, 2008
"... We begin by observing that (discretetime) QuasiBirthDeath Processes (QBDs) are equivalent, in a precise sense, to (discretetime) probabilistic 1Counter Automata (p1CAs), and both TreeLike QBDs (TLQBDs) and TreeStructured QBDs (TSQBDs) are equivalent to both probabilistic Pushdown Systems ..."
Abstract

Cited by 9 (3 self)
 Add to MetaCart
We begin by observing that (discretetime) QuasiBirthDeath Processes (QBDs) are equivalent, in a precise sense, to (discretetime) probabilistic 1Counter Automata (p1CAs), and both TreeLike QBDs (TLQBDs) and TreeStructured QBDs (TSQBDs) are equivalent to both probabilistic Pushdown Systems
TOPK Projection Queries for Probabilistic Business Processes
"... A Business Process (BP) consists of some business activities undertaken by one or more organizations in pursuit of some business goal. Tools for querying and analyzing BP specifications are extremely valuable for companies. In particular, given a BP specification, identifying the topk flows that ar ..."
Abstract

Cited by 7 (5 self)
 Add to MetaCart
A Business Process (BP) consists of some business activities undertaken by one or more organizations in pursuit of some business goal. Tools for querying and analyzing BP specifications are extremely valuable for companies. In particular, given a BP specification, identifying the topk flows that are most likely to occur in practice, out of those satisfying the criteria of a given query, is crucial for various applications such as personalized advertisements and BP website design. This paper studies, for the first time, topk query evaluation for queries with projection in this context. We analyze the complexity of the problem for different classes of distribution functions for the flows likelihood, and provide efficient (PTIME) algorithms whenever possible. Furthermore, we show an interesting application of our algorithms to the analysis of BP execution traces (logs), for recovering missing information about the runtime process behavior, that has not been recorded in the logs. 1