In this paper we present a novel abstraction technique for Markov decision processes (MDPs), which are widely used for modelling systems that exhibit both probabilistic and nondeterministic behaviour. In the field of model checking, abstraction has proved an extremely successful tool to combat the state-space explosion problem. In the probabilistic setting, however, little practical progress has been made in this area. We propose an abstraction method for MDPs based on stochastic two-player games. The key idea behind this approach is to maintain a separation between nondeterminism present in the original MDP and nondeterminism introduced through abstraction, each type being represented by a different player in the game. Crucially, this allows us to obtain distinct lower and upper bounds for both the best and worst-case performance (minimum or maximum probabilities) of the MDP. We have implemented our techniques and illustrate their practical utility by applying them to a quantitative analysis of the Zeroconf dynamic network configuration protocol. 1

In the past, partial order reduction has been used successfully to combat the state explosion problem in the context of model checking for non-probabilistic systems. For both linear time and branching time specifications, methods have been developed to apply partial order reduction in the context of model checking. Only recently, results were published that give criteria on applying partial order reduction for verifying quantitative linear time properties for probabilistic systems. This paper presents partial order reduction criteria for Markov decision processes and branching time properties, such as formulas of probabilistic computation tree logic. Moreover, we provide a comparison of the results established so far about reduction conditions for Markov decision processes.

A wide range of coordination protocols for distributed systems, internet protocols or systems with unreliable components can formally be modelled by Markov decision processes (MDP). MDPs can be viewed as a variant of state-transition diagrams with discrete probabilities and nondeterminism. While traditional model checking techniques for non-probabilistic systems aim to establish properties stating that all (or some) computations fulfill a certain condition, the verification problem for randomized systems requires reasoning about the quantitative behavior by means of properties that refer to the probabilities for certain computations, for instance, the probability to find a leader within 5 rounds or the probability for not reaching an error state.

We consider a variant of probabilistic timed automata called parametric determinate probabilistic timed automata. Such automata are fully probabilistic: there is a single distribution of outgoing transitions from each of the automaton’s nodes, and it is possible to remain at a node only for a given amount of time. The residence time within a node may be given in terms of a parameter, and hence we do not assume that its concrete value is known. We claim that, often in practice, the maximal expected time to reach a given absorbing node of a probabilistic timed automaton can be captured using a parametric determinate probabilistic timed automaton. We give a method for computing the expected time for a parametric determinate probabilistic timed automaton to reach an absorbing node. The method consists in constructing a variant of a Markov chain with costs (where the costs correspond to durations), and is parametric in the sense that the expected absorption time is computed as a function of the model’s parameters. The complexity of the analysis is independent from the maximal constant bounding the values of the clocks, and is polynomial in the number of edges of the original parametric determinate probabilistic timed automaton. 1

We present a technique to tackle the parameterised probabilistic model checking problem for a particular class of randomised distributed systems, which we model as Markov Decision Processes. These systems, termed degenerative, have the property that a model of a system with some communication graph will eventually behave like a model of a system with a reduced graph. We describe an induction schema for reasoning about models of a degenerative system over arbitrary graphs. We thereby show that a certain class of quantitative LTL properties will hold for a model of a system with any communication graph if it holds for all models of a system with some base graph. We demonstrate our technique via a case study (a randomised leader election protocol) specified using the PRISM modelling language. Keywords: Probabilistic model checking, parameterised model checking, degenerative systems, PRISM.

Abstract not found

Abstract. This paper presents a novel technique for state space reduction of probabilistic specifications, based on a newly developed notion of confluence for probabilistic automata. We prove that this reduction preserves branching probabilistic bisimulation and can be applied on-the-fly. To support the technique, we introduce a method for detecting confluent transitions in the context of a probabilistic process algebra with data, facilitated by an earlier defined linear format. A case study demonstrates that significant reductions can be obtained. 1