We consider decentralized control of Markov decision processes and give complexity bounds on the worst-case running time for algorithms that find optimal solutions. Generalizations of both the fullyobservable case and the partially-observable case that allow for decentralized control are described. For even two agents, the finite-horizon problems corresponding to both of these models are hard for nondeterministic exponential time. These complexity results illustrate a fundamental difference between centralized and decentralized control of Markov decision processes. In contrast to the problems involving centralized control, the problems we consider provably do not admit polynomial-time algorithms. Furthermore, assuming EXP NEXP, the problems require super-exponential time to solve in the worst case.

We investigate the computability of problems in probabilistic planning and partially observable infinite-horizon Markov decision processes. The undecidability of the string-existence problem for probabilistic finite automata is adapted to show that the following problem of plan existence in probabilistic planning is undecidable: given a probabilistic planning problem, determine whether there exists a plan with success probability exceeding a desirable threshold. Analogous policy-existence problems for partially observable infinite-horizon Markov decision processes under discounted and undiscounted total reward models, average-reward models, and state-avoidance models are all shown to be undecidable. The results apply to corresponding approximation problems as well. 1 Introduction We show that problems in probabilistic planning (Kushmerick, Hanks, & Weld 1995; Boutilier, Dean, & Hanks 1999) and infinite-horizon partially observable Markov decision processes (POMDPs) (L...

We show that the boundedness of the set of all products of a given pair Sigma of rational matrices is undecidable. Furthermore, we show that the joint (or generalized) spectral radius #(#) is not computable because testing whether #(#)61 is an undecidable problem. As a consequence, the robust stability of linear systems under time-varying perturbations is undecidable, and the same is true for the stability of a simple class of hybrid systems. We also discuss some connections with the so-called "finiteness conjecture". Our results are based on a simple reduction from the emptiness problem for probabilistic finite automata, which is known to be undecidable.

Abstract not found

Automated planning, the problem of how an agent achieves a goal given a repertoire of actions, is one of the foundational and most widely studied problems in the AI literature. The original formulation of the problem makes strong assumptions regarding the agent's knowledge and control over the world, namely that its information is complete and correct, and that the results of its actions are deterministic and known.

We show that termination of a class of linear loop programs is decidable. Linear loop programs are discrete-time linear systems with a loop condition governing termination, that is, a while loop with linear assignments. We relate the termination of such a simple loop, on all initial values, to the eigenvectors corresponding to only the positive real eigenvalues of the matrix defining the loop assignments. This characterization of termination is reminiscent of the famous stability theorems in control theory that characterize stability in terms of eigenvalues.

this paper, weconsider simple classes of nonlinear systems and provethatbasic questions related to their stabilityandcontrollabilityare either undecidable or computationally intractable (NP-hard). As a special case, weconsider a class of hybrid systems in which the state space is partitioned into two halfspaces, and the dynamics in eachhalfspace correspond to a differentlinear system

We introduce a new notion of decentralized observability for discrete-event systems, which we call joint observability. We prove that checking joint observability of a regular language w.r.t. one observer is decidable, whereas for two (or more) observers the problem becomes undecidable. Based on this result, we show that a related decentralized control problem is also undecidable. We finally provide an extensive study relating our work to existing work in the literature.

The study of the stability properties of switched and hybrid systems gives rise to a number of interesting and challenging mathematical problems. The objective of this paper is to outline some of these problems, to review progress made in solving these problems in a number of diverse communities, and to review some problems that remain open. An important contribution of our work is to bring together material from several areas of research and to present results in a unified manner. We begin our review by relating the stability problem for switched linear systems and a class of linear differential inclusions. Closely related to the concept of stability are the notions of exponential growth rates and converse Lyapunov theorems, both of which are discussed in detail. In particular, results on common quadratic Lyapunov functions and piecewise linear Lyapunov functions are presented, as they represent constructive methods for proving stability, and also represent problems in which significant progress has been made. We also comment on the inherent difficulty of determining stability of switched systems in general which is exemplified by NP-hardness and undecidability results. We then proceed by considering the stability of switched systems in which there are constraints on the switching rules, through both dwell time requirements and state dependent switching laws. Also in this case the theory of Lyapunov functions and the existence of converse theorems is reviewed. We briefly comment on the classical Lur’e problem and on the theory of stability radii, both of which contain many of the features of switched systems and are rich sources of practical results on the topic. Finally we present a list of questions and open problems which provide motivation for continued research in this area.

Abstract not found