Planning under uncertainty is a central problem in the study of automated sequential decision making, and has been addressed by researchers in many different fields, including AI planning, decision analysis, operations research, control theory and economics. While the assumptions and perspectives adopted in these areas often differ in substantial ways, many planning problems of interest to researchers in these fields can be modeled as Markov decision processes (MDPs) and analyzed using the techniques of decision theory. This paper presents an overview and synthesis of MDP-related methods, showing how they provide a unifying framework for modeling many classes of planning problems studied in AI. It also describes structural properties of MDPs that, when exhibited by particular classes of problems, can be exploited in the construction of optimal or approximately optimal policies or plans. Planning problems commonly possess structure in the reward and value functions used to de...

A fundamental assumption made by classical AI planners is that there is no uncertainty in the world: the planner has full knowledge of the conditions under which the plan will be executed and the outcome of every action is fully predictable. These planners cannot therefore construct contingency plans, i.e., plans in which different actions are performed in different circumstances. In this paper we discuss some issues that arise in the representation and construction of contingency plans and describe Cassandra, a partial-order contingency planner. Cassandra uses explicit decision-steps that enable the agent executing the plan to decide which plan branch to follow. The decision-steps in a plan result in subgoals to acquire knowledge, which are planned for in the same way as any other subgoals. Cassandra thus distinguishes the process of gathering information from the process of making decisions. The explicit representation of decisions in Cassandra allows a coherent approach to...

Abstract not found

We describe a framework for planning in dynamic environments. A central question is how to focus the sensing performed by such a system, so that it responds appropriately to relevant changes, but does not attempt to monitor all the changes that could possibly occur in the world. To achieve the required balance, we introduce rationale-based monitors, which represent the features of the world state that are included in the plan rationale, i.e., the reasons for the planning decisions so far made. Rationale-based monitors capture information both about the plan currently under development and the alternative choices that were found but not pursued. We discuss

A key question in conditional planning is: how many, and which of the possible execution failures should be planned for? One cannot, in general, plan for all the failures that can be anticipated: there are simply too many. But neither can one ignore all the possible failures, or one will fail to produce sufficiently flexible plans. We describe a planning system that attempts to identify the contingencies that contribute the most to a plan's overall value. Plan generation proceeds by extending the plan to include actions that will be taken in case the identified contingencies fail, iterating until either a given expected value threshold is reached or planning time is exhausted. Introduction Classical AI plan generation systems assume static environments and omniscient agents, and thus ignore the possibility that events may occur in unexpected ways---that contingencies might arise---during plan execution. A problem with classical planners is, of course, that things do not always go "acc...

The recent advances in computer speed and algorithms for probabilistic inference have led to a resurgence of work on planning under uncertainty. The aim is to design AI planners for environments where there may be incomplete or faulty information, where actions may not always have the same results and where there may be tradeoffs between the different possible outcomes of a plan. Addressing uncertainty in AI planning algorithms will greatly increase the range of potential applications but there is plenty of work to be done before we see practical decision-theoretic planning systems. This article outlines some of the challenges that need to be overcome and surveys some of the recent work in the area. Introduction AI planning algorithms are concerned with finding a course of action to be carried out by some agent to achieve its goals. In problems where actions can lead to a number of different possible outcomes, or where the benefits of executing a plan must be weighed against t...

ing Probabilistic Actions Peter Haddawy AnHai Doan Department of Electrical Engineering and Computer Science University of Wisconsin-Milwaukee PO Box 784 Milwaukee, WI 53201 fhaddawy, anhaig@cs.uwm.edu Abstract This paper discusses the problem of abstracting conditional probabilistic actions. We identify two distinct types of abstraction: intra-action abstraction and inter-action abstraction. We define what it means for the abstraction of an action to be correct and then derive two methods of intra-action abstraction and two methods of inter-action abstraction which are correct according to this criterion. We illustrate the developed techniques by applying them to actions described with the temporal action representation used in the drips decision-theoretic planner and we describe how the planner uses abstraction to reduce the complexity of planning. 1 Introduction Optimal planning in a decision-theoretic framework requires finding the plan or set of plans that maximizes expected ...

In dynamic environments, optimal deliberation in the decision-theoretic sense is impossible. Instead, it is sometimes necessary to trade potential decision quality for decision timeliness. One approach to achieving this trade-off is to endow intelligent agents with meta-level strategies that provide them guidance about when to reason--- and what to reason about---and when to act instead. In this paper, we describe our investigations of a particular meta-level reasoning strategy, filtering, in which an agent commits to the goals it has already adopted, and then tends to filter from consideration new options that would conflict with the successful completion of existing goals [Bratman et al. 1988]. To investigate the utility of filtering, we conducted a series of experiments using the Tileworld testbed [Pollack and Ringuette 1990]. Previous experiments [Kinny and Georgeff 1991] provided preliminary evidence of the feasibility of filtering; our results generalize and refine those earlier ...

ion-Based Probabilistic Planning Vu Ha Peter Haddawy Department of EE & CS University of Wisconsin-Milwaukee fvu, haddawyg@cs.uwm.edu Abstract Modeling worlds and actions under uncertainty is one of the central problems in the framework of decision-theoretic planning. The representation must be general enough to capture real-world problems but at the same time it must provide a basis upon which theoretical results can be derived. The central notion in the framework we propose here is that of the affine-operator, which serves as a tool for constructing (convex) sets of probability distributions, and which can be considered as a generalization of belief functions and interval mass assignments. Uncertainty in the state of the worlds is modeled with sets of probability distributions, represented by affine-trees, while actions are defined as tree-manipulators. A small set of key properties of the affine-operator is presented, forming the basis for most existing operator-based definitio...

Based on the assumption that a rational agent will adopt a plan that maximizes the expected utility, we present a utility-based approach to plan recognition problem in this paper. The approach explicitly takes the observed agent’s preferences into consideration, and computes the estimated expected utilities of plans to disambiguate competing hypotheses. Online plan recognition is realized by incrementally using plan knowledge and observations to change state probabilities. We also discuss the work and compare it with other probabilistic models in the paper.