## Recursive Markov chains, stochastic grammars, and monotone systems of non-linear equations (2005)

Venue: | IN STACS |

Citations: | 67 - 11 self |

### BibTeX

@INPROCEEDINGS{Etessami05recursivemarkov,

author = {Kousha Etessami and Mihalis Yannakakis},

title = {Recursive Markov chains, stochastic grammars, and monotone systems of non-linear equations},

booktitle = {IN STACS},

year = {2005},

pages = {340--352},

publisher = {Springer}

}

### Years of Citing Articles

### OpenURL

### Abstract

We define Recursive Markov Chains (RMCs), a class of finitely presented denumerable Markov chains, and we study algorithms for their analysis. Informally, an RMC consists of a collection of finite-state Markov chains with the ability to invoke each other in a potentially recursive manner. RMCs offer a natural abstract model for probabilistic programs with procedures. They generalize, in a precise sense, a number of well studied stochastic models, including Stochastic Context-Free Grammars (SCFG) and Multi-Type Branching Processes (MT-BP). We focus on algorithms for reachability and termination analysis for RMCs: what is the probability that an RMC started from a given state reaches another target state, or that it terminates? These probabilities are in general irrational, and they arise as (least) fixed point solutions to certain (monotone) systems of nonlinear equations associated with RMCs. We address both the qualitative problem of determining whether the probabilities are 0, 1 or in-between, and