BibTeX
@MISC{Altman95constrainedmarkov,
author = {Eitan Altman},
title = {Constrained Markov Decision Processes},
year = {1995}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
This report presents a unified approach for the study of constrained Markov decision processes with a countable state space and unbounded costs. We consider a single controller having several objectives; it is desirable to design a controller that minimize one of cost objective, subject to inequality constraints on other cost objectives. The objectives that we study are both the expected average cost, as well as the expected total cost (of which the discounted cost is a special case). We provide two frameworks: the case were costs are bounded below, as well as the contracting framework. We characterize the set of achievable expected occupation measures as well as performance vectors. This allows us to reduce the original control dynamic problem into an infinite Linear Programming. We present a Lagrangian approach that enables us to obtain sensitivity analysis. In particular, we obtain asymptotical results for the constrained control problem: convergence of both the value and the pol...
Keyphrases
markov decision process cost objective performance vector average cost contracting framework asymptotical result markov decision unified approach sensitivity analysis single controller inequality constraint infinite linear programming several objective countable state space expected total cost lagrangian approach unbounded cost special case original control dynamic problem control problem occupation measure
