DMCA
Game-based abstraction for markov decision processes (2006)
Cached
Download Links
| Venue: | In Proc. of QEST: Quantitative Evaluation of Systems |
| Citations: | 52 - 15 self |
BibTeX
@INPROCEEDINGS{Kwiatkowska06game-basedabstraction,
author = {Marta Kwiatkowska and Gethin Norman and David Parker},
title = {Game-based abstraction for markov decision processes},
booktitle = {In Proc. of QEST: Quantitative Evaluation of Systems},
year = {2006},
pages = {157--166},
publisher = {IEEE Computer Society}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
In this paper we present a novel abstraction technique for Markov decision processes (MDPs), which are widely used for modelling systems that exhibit both probabilistic and nondeterministic behaviour. In the field of model checking, abstraction has proved an extremely successful tool to combat the state-space explosion problem. In the probabilistic setting, however, little practical progress has been made in this area. We propose an abstraction method for MDPs based on stochastic two-player games. The key idea behind this approach is to maintain a separation between nondeterminism present in the original MDP and nondeterminism introduced through abstraction, each type being represented by a different player in the game. Crucially, this allows us to obtain distinct lower and upper bounds for both the best and worst-case performance (minimum or maximum probabilities) of the MDP. We have implemented our techniques and illustrate their practical utility by applying them to a quantitative analysis of the Zeroconf dynamic network configuration protocol. 1
Keyphrases
markov decision process game-based abstraction probabilistic setting original mdp successful tool model checking different player zeroconf dynamic network configuration protocol quantitative analysis little practical progress key idea practical utility abstraction method novel abstraction technique nondeterministic behaviour upper bound state-space explosion problem worst-case performance stochastic two-player game maximum probability nondeterminism present
