## Abstract interpretation of programs as Markov decision processes (2005)

Venue: | Science of Computer Programming 58 |

Citations: | 23 - 0 self |

### BibTeX

@INPROCEEDINGS{Monniaux05abstractinterpretation,

author = {David Monniaux and École Normale Supérieure},

title = {Abstract interpretation of programs as Markov decision processes},

booktitle = {Science of Computer Programming 58},

year = {2005},

pages = {179--205},

publisher = {Springer Verlag}

}

### OpenURL

### Abstract

Abstract. We propose a formal language for the specification of trace properties of probabilistic, nondeterministic transition systems, encompassing the properties expressible in Linear Time Logic. Those formulas are in general undecidable on infinite deterministic transition systems and thus on infinite Markov decision processes. This language has both a semantics in terms of sets of traces, as well as another semantics in terms of measurable functions; we give and prove theorems linking the two semantics. We then apply abstract interpretation-based techniques to give upper bounds on the worst-case probability of the studied property. We propose an enhancement of this technique when the state space is partitioned — for instance along the program points —, allowing the use of faster iteration methods. 1

### Citations

403 |
Cousot and Radhia Cousot. Abstract interpretation: a unified lattice model for static analysis of programs by construction or approximation of fixpoints
- Patrick
- 1977
(Show Context)
Citation Context ...a pessimistic point of view when it cannot solve the problemsexactly). In this paper, we take the latter approach and build our analysis methods upon the existing framework of abstract interpretation =-=[9]-=-, a general theory of approximation between semantics. We have earlier proposed two classes of automatic methods to analyze such system: some forward [16,17], some backward [19,20]. In this paper, we ... |

234 | L.: Model checking of probabilistic and nondeterministic systems
- Bianco, Alfaro
- 1995
(Show Context)
Citation Context ... theorem (Appendix B) then constructs from the (Tn) a transition probability G(T, (Un)n∈N) from Ω (the initial state space) to Ω N . We note ST (f, (Un)n∈N) = t0 ↦→ 〈t ↦→ f(t0, t), G(T, (Un)n∈N)(t0)〉 =-=(3)-=- (S short for ST if there is no ambiguity about T ) and R(f) the set of all functions S(T, (Un)n∈N) when (Un)n∈N is a sequence of transition probabilities, Un being a transition probability from Ω n t... |

136 |
L.: Formal Verification of Probabilistic Systems
- Alfaro
- 1997
(Show Context)
Citation Context ...he natural extension of the notion of deterministic state is the notion of probability distribution on the set of states. Definition 1 Let Ω be a finite or countable set of states. A function f : Ω → =-=[0, 1]-=- is called a probability distribution if � ω∈Ω f(ω) = 1. We shall note D(Ω) the set of probabilistic distributions on Ω. Now that we have the probabilistic counterpart of the notion of state, we need ... |

89 |
Méthodes itératives de construction et d’approximation de points fixes d’opérateurs monotones sur un treillis, analyse sémantique des programmes. State doctorate thesis, Université scientifique et médicale de Grenoble and Institut National Polytechnique d
- Cousot
- 1978
(Show Context)
Citation Context ...nstant (5) �f1 +S f2� t .env = λt.χS(t0).(�f1� t env) + χ S C (t0).(�f2� t env) (6) �lfp(name ↦→ f)� t .env = lfp(λφ.�f� t .env[name ↦→ φ]) (7) �gfp(name ↦→ f)� t .env = gfp(λφ.�f� t .env[name ↦→ φ]) =-=(8)-=- �shift(f)� t .env = (�f� t .env) ◦ shift (9) �let name = f1 in f2� t .env = �f2� t .env[name ↦→ �f1� t .env] (10) χS is the characteristic function of S and S C the complement of S. t0 ∈ Ω is the fir... |

63 |
M.: Markov decision processes and regular events
- Courcoubetis, Yannakakis
- 1998
(Show Context)
Citation Context ... finance mathematics [23]. More recently, they have been studied from the angle of probabilistic computing systems [1–4,15,24]. Effective resolution techniques include linear programming [23, §7.2.7] =-=[7]-=- and newer data structures such as MTBDDs [2]. However, the problem of large- or infinite-state systems has not been so well studied. In the case of deterministic or nondeterministic systems without a... |

51 |
Testing preorders for probabilistic processes (extended abstract
- Cleaveland, Smolka, et al.
- 1992
(Show Context)
Citation Context ...is leads to the study of partially observable Markov decision processes (POMDP); however, their effective analysis is much more complex than that of fully observable processes [14]. Cleaveland’s work =-=[6]-=- focuses on the model where the nondeterministic choices are taken after the probabilistic ones. This simplifies the theory to some extent, since taking the product of the analyzed process with an non... |

44 | Quantitative solution of omega-regular games
- Alfaro, Majumdar
- 2004
(Show Context)
Citation Context ...is granted the first time the process leaves the set of states to consider. Formal languages similar to the one we consider have been introduced by other authors, such as quantitative game µ-calculus =-=[10]-=-. The differences between our approach and this game calculus approach are threefold: – We give a semantics in terms of traces, then prove its link with a semantics in terms of expectation functions; ... |

38 | Verifying quantitative properties of continuous probabilistic timed automata
- Kwiatkowska, Norman, et al.
(Show Context)
Citation Context ... lfp(f ↦→ 1 +A shiftf) (11) – Let A be a set of states. The liveness property associated with A defines the set of traces that always remain in A. It corresponds to the formula: gfp(f ↦→ shiftf +A 0) =-=(12)-=- – Let A be a (measurable) set of states. The Büchi acceptance property associated with A defines the set of traces that pass through A infinitely often; it is written as: gfp(C ↦→ lfp(R ↦→ shift(C) +... |

32 | Abstract interpretation of probabilistic semantics
- Monniaux
- 2000
(Show Context)
Citation Context ... existing framework of abstract interpretation [9], a general theory of approximation between semantics. We have earlier proposed two classes of automatic methods to analyze such system: some forward =-=[16,17]-=-, some backward [19,20]. In this paper, we focus on the backward approach and extend it to a larger class of properties (including those specified by LTL formulas). We also prove that chaotic iteratio... |

26 | Efficient dynamic-programming updates in partially observable markov decision processes
- Littman, Cassandra, et al.
- 1996
(Show Context)
Citation Context ...e taken into account. This leads to the study of partially observable Markov decision processes (POMDP); however, their effective analysis is much more complex than that of fully observable processes =-=[14]-=-. Cleaveland’s work [6] focuses on the model where the nondeterministic choices are taken after the probabilistic ones. This simplifies the theory to some extent, since taking the product of the analy... |

10 | Reasoning about efficiency within a probabilistic mu-calculus
- McIver
- 1998
(Show Context)
Citation Context ...perty [10, sect. 5]: k−1 � R = (�♦U2i ∧ ¬�♦U2i+1) (14) i=0 It corresponds to the following formula: gfp(x2k−1 ↦→ lfp(x2k ↦→ · · · gfp(x1 ↦→ lfp(x0 ↦→ ((· · · (x2k−1 + U2k−1x2k−2) · · · +U1 x0) +U0 0) =-=(15)-=- Summation valuator A related family of trace valuators are the summing valuators. The summation valuator associated with a (measurable) function f : Ω ↦→ [0, +∞] is the function �ΣA� t : � ��� Ω N → ... |

7 | Computing probability bounds for linear time formulas over concurrent probabilistic systems
- Baier, Kwiatkowska, et al.
- 1999
(Show Context)
Citation Context ...k+1. Let envt be the set of environments of valuators, mapping each name to a valuator, ordered point-wise. �formula� t : envt → (Ω N → I) is defined inductively as follows: �name� t .env = env(name) =-=(4)-=- �constant� t .env = constant (5) �f1 +S f2� t .env = λt.χS(t0).(�f1� t env) + χ S C (t0).(�f2� t env) (6) �lfp(name ↦→ f)� t .env = lfp(λφ.�f� t .env[name ↦→ φ]) (7) �gfp(name ↦→ f)� t .env = gfp(λφ.... |

5 | An abstract analysis of the probabilistic termination of programs
- Monniaux
- 2001
(Show Context)
Citation Context ... existing framework of abstract interpretation [9], a general theory of approximation between semantics. We have earlier proposed two classes of automatic methods to analyze such system: some forward =-=[16,17]-=-, some backward [19,20]. In this paper, we focus on the backward approach and extend it to a larger class of properties (including those specified by LTL formulas). We also prove that chaotic iteratio... |

2 |
Vasiliki Hartonas-Garmhausen, and Marta Kwiatkowska. Symbolic model checking for probabilistic processes
- Baier, Clarke
- 1997
(Show Context)
Citation Context ... Tn = T ◦ � � Id n , which is a transition probability between Ω Un and Ω. By this notation, we mean that, using the notation of Def. 2, Tn(x0, . . . , xn−1; xn) = T (xn−1, Un(x0, . . . , xn−1); xn). =-=(2)-=- Ionescu Tulcea’s theorem (Appendix B) then constructs from the (Tn) a transition probability G(T, (Un)n∈N) from Ω (the initial state space) to Ω N . We note ST (f, (Un)n∈N) = t0 ↦→ 〈t ↦→ f(t0, t), G(... |

2 |
A logic for reasoning about time and reability
- Hansson, Jonsson
- 1990
(Show Context)
Citation Context ...lysis of nondeterministic (albeit non probabilistic) systems. It was therefore quite natural to extend this notion to probabilistic systems. Proposed extensions to the probabilistic case include pCTL =-=[11]-=- and pCTL*. We shall see here briefly how we deal with some pCTL* formulas. CTL* formulas define sets of states as the starting states of sets of traces defined by LTL path formulas (in which state fo... |