## Contingent Planning Under Uncertainty via Stochastic Satisfiability (1999)

### Cached

### Download Links

- [www.cs.duke.edu]
- [www.cs.rutgers.edu]
- [athos.rutgers.edu]
- DBLP

### Other Repositories/Bibliography

Venue: | Artificial Intelligence |

Citations: | 61 - 9 self |

### BibTeX

@INPROCEEDINGS{Majercik99contingentplanning,

author = {Stephen M. Majercik and Michael L. Littman},

title = {Contingent Planning Under Uncertainty via Stochastic Satisfiability},

booktitle = {Artificial Intelligence},

year = {1999},

pages = {549--556}

}

### Years of Citing Articles

### OpenURL

### Abstract

We describe two new probabilistic planning techniques ---c-maxplan and zander---that generate contingent plans in probabilistic propositional domains. Both operate by transforming the planning problem into a stochastic satisfiability problem and solving that problem instead. c-maxplan encodes the problem as an E-Majsat instance, while zander encodes the problem as an S-Sat instance. Although S-Sat problems are in a higher complexity class than E-Majsat problems, the problem encodings produced by zander are substantially more compact and appear to be easier to solve than the corresponding E-Majsat encodings. Preliminary results for zander indicate that it is competitive with existing planners on a variety of problems. Introduction When planning under uncertainty, any information about the state of the world is precious. A contingent plan is one that can make action choices contingent on such information. In this paper, we present an implemented framework for contingent pl...

### Citations

11412 |
Computers and Intractability. A Guide to the Theory of NP-Completeness. edited by W.H
- Garey, Johnson
- 1997
(Show Context)
Citation Context ...complete problem and many important, practical problems in areas such as planning and scheduling, network design, and data storage and retrieval (to name just a few) can be expressed as Sat problems (=-=Garey and Johnson 1979-=-). As such, Sat is a very well-studied problem, both from a theoretical point of view (e.g. how does the solution difficulty of random Sat problems vary as one varies the parameters of the problem?) a... |

2856 |
Dynamic Programming
- Bellman
- 1957
(Show Context)
Citation Context ...eng (Hansen and Feng 2000), gpt (Bonet and Geffner 2001), spudd (Hoey et al. 1999). zander, the contingent planner we developed (Section 6), falls into this category, as do traditional OR approaches (=-=Bellman 1957-=-; Howard 1960; Puterman and Shin 1978; Denardo 1982). Note that Cases 2 and 3 subsume Case 1, and Case 4 subsumes all the other cases; thus, a planner for addressing Case 4 can be used in all four sce... |

2418 | Computational complexity - Papadimitriou - 1994 |

1864 |
Strips: a new approach to the application of theorem proving to problem solving
- Fikes, Nilsson
- 1971
(Show Context)
Citation Context ...sults in a deterministic transition to a new state s ′ . The objective is to choose actions, one after another, to move from the initial state s0 to one of the goal states. The strips representation (=-=Fikes and Nilsson 1971-=-) of M, which we will describe informally, uses a propositional state representation; a state is described by an assignment to a set of Boolean variables. Actions are specified by three sets of propos... |

1113 |
A computing procedure for quantification theory
- Davis, Putnam
- 1960
(Show Context)
Citation Context ... that although satplan uses stochastic local search, other satisfiability testing algorithms have been used in the context of planning. The original Davis-Putnam procedure for satisfiability testing (=-=Davis and Putnam 1960-=-) uses resolution as a key algorithmic component. Resolution was later replaced by variable splitting (Davis et al. 1962), and this latter procedure has completely overshadowed the earlier version. Ot... |

891 | Planning and acting in partially observable stochastic domains
- Kaelbling, Littman, et al.
- 1998
(Show Context)
Citation Context ...te in a fixed number of steps, if one exists. An alternate formulation is to maximize the probability of reaching a goal state. 6.2 Example Domain Consider a simple domain based on the Tiger problem (=-=Kaelbling et al. 1998-=-). The domain consists of four propositions: tiger-behind-left-door, dead, rewarded and hear-tigerbehind-left-door, the last of which is observable. In the initial state, tiger-behind-leftdoor is True... |

769 |
A machine program for theorem proving
- Davis, Longemann, et al.
- 1962
(Show Context)
Citation Context ...f planning. The original Davis-Putnam procedure for satisfiability testing (Davis and Putnam 1960) uses resolution as a key algorithmic component. Resolution was later replaced by variable splitting (=-=Davis et al. 1962-=-), and this latter procedure has completely overshadowed the earlier version. Other systematic solvers that incorporate efficient data structures (sato, Zhang and Stickel 2000), better heuristics (sat... |

587 |
Principles of Artificial Intelligence
- Nilsson
- 1980
(Show Context)
Citation Context ...en-for-tiger-2, open-left-door-2, open-right-door-2) will be different for different values of this variable. This representation of the planning problem is similar to AND/OR trees and MINIMAX trees (=-=Nilsson 1980-=-). Choice variable nodes are analogous to OR, or MAX, nodes, and chance variable nodes are analogous to AND, or MIN, nodes. However, the probabilities associated with chance variables (our opponent is... |

562 |
Dynamic Programming and Markov Processes
- Howard
- 1960
(Show Context)
Citation Context ...d Feng 2000), gpt (Bonet and Geffner 2001), spudd (Hoey et al. 1999). zander, the contingent planner we developed (Section 6), falls into this category, as do traditional OR approaches (Bellman 1957; =-=Howard 1960-=-; Puterman and Shin 1978; Denardo 1982). Note that Cases 2 and 3 subsume Case 1, and Case 4 subsumes all the other cases; thus, a planner for addressing Case 4 can be used in all four scenarios. Our r... |

558 | Learning to act using real-time dynamic programming
- Barto, Bradtke, et al.
- 1995
(Show Context)
Citation Context ...solving planning problems that involve uncertainty and partial information (Bonet and Geffner 2001). It uses optimal heuristic search for conformant planning and real-time dynamic programming (RTDP) (=-=Barto et al. 1995-=-) for nondeterministic (non-probabilistic), probabilistic, and contingent planning. RTDP is a version of dynamic programming that finds a policy by running a number of trials, each one starting in the... |

547 | Pushing the envelope: Planning, propositional logic, and stochastic search
- Kautz, Selman
- 1996
(Show Context)
Citation Context ... solution of the resulting satisfiability problem produces the required plan. This work is inspired in large measure by the success of satplan, a similar planning technique for deterministic domains (=-=Kautz and Selman 1996-=-) that encodes the planning problem as a Boolean satisfiability problem and uses stochastic local search to solve the resulting satisfiability problem. There are significant problems in developing a p... |

456 | Planning as satisfiability
- Kautz, Selman
- 1992
(Show Context)
Citation Context ...art planner called UCPOP on several natural and artificial planning problems (Blum and Furst 1997); it remains one of the best current planners and research on this paradigm is quite active. satplan (=-=Kautz and Selman 1992-=-; Kautz and Selman 1996) works by first converting the bounded-horizon planning problem to a propositional satisfiability problem and then using stochastic local search to solve the resulting satisfia... |

441 | Decisiontheoretic planning: structural assumptions and computational leverage
- Boutilier, Dean, et al.
- 1999
(Show Context)
Citation Context ...tion called the sequential-effects-tree representation (st) (Littman 1997), which is a syntactic variant of two-time-slice Bayes nets (2TBNs) with conditional probability tables represented as trees (=-=Boutilier et al. 1999-=-; Boutilier et al. 1995). (This representation is also equivalent to the strips-like probabilistic state-space operators, or PSOs, Hanks and McDermott 1993; Littman 1997). The st representation of a p... |

301 | The computational complexity of propositional STRIPS planning
- Bylander
- 1994
(Show Context)
Citation Context ...te). The computational complexity of propositional planning varies with bounds on the plan size and plan horizon. In the deterministic case, for example, unbounded strips planning is PSPACE-complete (=-=Bylander 1994-=-). If we put a polynomial bound on the plan horizon (Kautz and Selman 1996), however, strips planning becomes an NP-complete problem. The complexity of probabilistic propositional planning varies in a... |

273 | Unifying sat-based and graphbased planning
- Kautz, Selman
- 1999
(Show Context)
Citation Context ...better heuristics (satz, Li and Anbulagan 1997), and constraint satisfaction solution techniques (relsat, Bayardo and Schrag 1997) have been developed more recently. blackbox (Kautz and Selman 1998a; =-=Kautz and Selman 1999-=-) integrates several of these approaches—walksat (stochastic local search), satz, andrelsat—in a planning system that allows the user to try different solvers on the Sat encoding of a planning problem... |

273 | An algorithm for probabilistic planning
- Kushmerick, Hanks, et al.
- 1995
(Show Context)
Citation Context ...babilities attached to action outcomes allow the planner to specify the straight-line plan that has the highest probability of succeeding, even if that probability is less than 1.0. Example: buridan (=-=Kushmerick et al. 1995-=-) 3 Note that the term conditional has been used in different ways in the literature. Plans in which the execution of actions depends on the outcomes of earlier actions were originally called “conditi... |

229 | Exploiting structure in policy construction
- Boutilier, Dearden, et al.
- 1995
(Show Context)
Citation Context ...ial-effects-tree representation (st) (Littman 1997), which is a syntactic variant of two-time-slice Bayes nets (2TBNs) with conditional probability tables represented as trees (Boutilier et al. 1999; =-=Boutilier et al. 1995-=-). (This representation is also equivalent to the strips-like probabilistic state-space operators, or PSOs, Hanks and McDermott 1993; Littman 1997). The st representation of a planning domain can be d... |

224 | Conditional Nonlinear Planning
- Peot, Smith
- 1992
(Show Context)
Citation Context ...lity 0.05. A non-probabilistic version of the same action might specify that if the 2 Such actions have historically been called conditional actions (Goldman and Boddy 1994a; Goldman and Boddy 1994b; =-=Peot and Smith 1992-=-). In our taxonomy of planning under uncertainty, however, we wish to make a distinction between the type of planning and the type of actions used, so we will use the term non-probabilistic action to ... |

218 | Using csp look-back techniques to solve real-world sat instances - Bayardo, Schrag - 1997 |

209 | Probabilistic planning with information gathering and contingent execution
- Draper, Hanks, et al.
- 1994
(Show Context)
Citation Context ...sed in different ways in the literature. Plans in which the execution of actions depends on the outcomes of earlier actions were originally called “conditional plans” (Warren 1976). Some researchers (=-=Draper et al. 1994-=-) suggested calling such plans “contingent plans”, reserving the term “conditional” for plans in which only the effects of actions are contingent on the outcomes of earlier actions, and this terminolo... |

191 | SPUDD: Stochastic planning using decision diagrams
- Hoey, St-Aubin, et al.
- 1999
(Show Context)
Citation Context ...ck 1997; Onder and Pollack 1999), and pgraphplan/tgraphplan (Blum and Langford 1999), pomdp:inc prune (Cassandra et al. 1997), Hansen-Feng (Hansen and Feng 2000), gpt (Bonet and Geffner 2001), spudd (=-=Hoey et al. 1999-=-). zander, the contingent planner we developed (Section 6), falls into this category, as do traditional OR approaches (Bellman 1957; Howard 1960; Puterman and Shin 1978; Denardo 1982). Note that Cases... |

186 | A Sparse Sampling Algorithm for Near-Optimal Planning
- Kearns, Mansour, et al.
- 2002
(Show Context)
Citation Context ...te a “pretty good” first action (or action sequence), execute that action or action sequence, and then continue this planning/execution cycle from the new initial state (see, for example, the work of =-=Kearns et al. 1999-=-). This approach could improve efficiency greatly (at the expense of optimality) by focusing the planner’s efforts only on those contingencies that actually materialize. 10 Summary Probabilistic plann... |

166 | Extending Graphplan to Handle Uncertainty and Sensing Actions
- Weld, Anderson, et al.
- 1998
(Show Context)
Citation Context ...probabilistic actions means that the planner must look for a plan that will succeed under all circumstances. Examples: cnlp (Peot and Smith 1992), plinth (Goldman and Boddy 1994b), sensory graphplan (=-=Weld et al. 1998-=-), cassandra (Pryor and Collins 1996). 3. Conditional planning with probabilistic actions: As in Case 1, these planners engage in conformant planning, but the probabilities attached to action outcomes... |

165 | Incremental Pruning: A simple, fast, exact method for partially observable Markov decision processes
- Cassandra, Littman, et al.
- 1997
(Show Context)
Citation Context ...succeeding. Examples: c-buridan (Draper et al. 1994), dtpop (Peot 1998), mahinur (Onder and Pollack 1997; Onder and Pollack 1999), and pgraphplan/tgraphplan (Blum and Langford 1999), pomdp:inc prune (=-=Cassandra et al. 1997-=-), Hansen-Feng (Hansen and Feng 2000), gpt (Bonet and Geffner 2001), spudd (Hoey et al. 1999). zander, the contingent planner we developed (Section 6), falls into this category, as do traditional OR a... |

161 |
Using csp look-back techniques to solve real-world sat instances
- Schrag
- 1997
(Show Context)
Citation Context ...olvers that incorporate efficient data structures (sato, Zhang and Stickel 2000), better heuristics (satz, Li and Anbulagan 1997), and constraint satisfaction solution techniques (relsat, Bayardo and =-=Schrag 1997-=-) have been developed more recently. blackbox (Kautz and Selman 1998a; Kautz and Selman 1999) integrates several of these approaches—walksat (stochastic local search), satz, andrelsat—in a planning sy... |

160 | The complexity of stochastic games
- Condon
- 1992
(Show Context)
Citation Context ...1995; Boutilier and Dearden 1996; Koller and Parr 1999; Koller and Parr 2000). A planning problem with this objective can be transformed to an equivalent goal-oriented probabilistic planning problem (=-=Condon 1992-=-; Zwick and Paterson 1996). See Appendix A for a proof.sThis definition ensures a well-defined probability distribution over s ′ for each a and s. Using decision trees for next-state distributions cap... |

157 | Encoding plans in propositional logic
- Kautz, McAllester, et al.
- 1996
(Show Context)
Citation Context ...anning as satisfiability has been an active area of research. Researchers have looked at the issues that arise in connection with efficient conversion of planning problems to satisfiability problems (=-=Kautz et al. 1996-=-; Ernst et al. 1997), improving systematic satisfiability testers (Bayardo and Schrag 1997; Li and Anbulagan 1997), understanding and improving stochastic local search (Selman et al. 1996; McAllester ... |

146 | Monotonic solution of the frame problem in the situation calculus: an ecient method for worlds with fully speci actions - Schubert - 1990 |

133 |
Conformant graphplan
- Smith, Weld
- 1998
(Show Context)
Citation Context ... actions: These types of planners engage in conformant planning: producing a straight-line plan that is guaranteed to succeed no matter what conditions are encountered. Example: conformant graphplan (=-=Smith and Weld 1998-=-). 2. Contingent planning with non-probabilistic actions: Sensing allows this type of planner to produce a contingent plan, but the lack of probabilistic actions means that the planner must look for a... |

131 | A probabilistic algorithm for k-SAT and constraint satisfaction problems
- Schöning
- 1999
(Show Context)
Citation Context ...stic local search. walksat is not complete; it may not find a satisfying assignment when one exists. In addition, it cannot report that a satisfying assignment does not exist (although recent work by =-=Schöning 1999-=- provides probability bounds on the likelihood of missing a satisfying assignment if one exists). walksat, however, can solve satisfiability problems that are orders of magnitude larger than those the... |

129 | Automatic SATcompilation of planning problems
- Ernst, Millstein, et al.
- 1997
(Show Context)
Citation Context ...ility has been an active area of research. Researchers have looked at the issues that arise in connection with efficient conversion of planning problems to satisfiability problems (Kautz et al. 1996; =-=Ernst et al. 1997-=-), improving systematic satisfiability testers (Bayardo and Schrag 1997; Li and Anbulagan 1997), understanding and improving stochastic local search (Selman et al. 1996; McAllester et al. 1997; Kautz ... |

128 | BLACKBOX: A new Approach to the application of theorem proving to problem solving
- Kautz, Selman
- 1998
(Show Context)
Citation Context ...hang and Stickel 2000), better heuristics (satz, Li and Anbulagan 1997), and constraint satisfaction solution techniques (relsat, Bayardo and Schrag 1997) have been developed more recently. blackbox (=-=Kautz and Selman 1998-=-a; Kautz and Selman 1999) integrates several of these approaches—walksat (stochastic local search), satz, andrelsat—in a planning system that allows the user to try different solvers on the Sat encodi... |

117 | Computing optimal policies for partially observable decision processes using compact representations - Boutilier, Poole - 1996 |

107 | Games against nature - Papadimitriou - 1985 |

97 | Computing factored value functions for policies in structured MDPs
- Koller, Parr
- 1999
(Show Context)
Citation Context ..., if pi ∈ s ′ , 1 − ρi, otherwise. 5 This is just one possibility. Another commonly used objective is that of maximizing expected discounted reward (Boutilier et al. 1995; Boutilier and Dearden 1996; =-=Koller and Parr 1999-=-; Koller and Parr 2000). A planning problem with this objective can be transformed to an equivalent goal-oriented probabilistic planning problem (Condon 1992; Zwick and Paterson 1996). See Appendix A ... |

84 | Probabilistic planning in the graphplan framework
- Blum, Langford
- 1999
(Show Context)
Citation Context ...describe the effects of an action that cannot be taken at that time step (or will have no impact if executed). The first author is currently working on an approach that is analogous to the graphplan (=-=Blum and Langford 1998-=-) approach of incrementally extending the depth of the planning graph in the search for a successful plan. The idea is to build the SSat encoding incrementally, attempting to find a satisfactory plan ... |

84 | Probabilistic propositional planning: Representations and complexity
- Littman
- 1997
(Show Context)
Citation Context ...y of probabilistic propositional planning varies in a similar fashion. If the plan size is unbounded and the plan horizon is infinite, the problem is EXP-complete if states are completely observable (=-=Littman 1997-=-), or, in the more general case, undecidable (Madani et al. 1999). If plan size or plan horizon alone is bounded by a polynomial in the size of the representation of the problem, the problem is PSPACE... |

82 | On the undecidability of probabilistic planning and infinite-horizon partially observable Markov decision problems
- Madani, Hanks, et al.
- 1999
(Show Context)
Citation Context ...anning that is still capable of expressing interesting problems) is PSPACE-complete (Bylander 1994); unrestricted probabilistic propositional planning in partially observable domains is undecidable (=-=Madani et al. 1999-=-). Traditionally, the decision-making models that have been studied in AI planning admit no uncertainty: every aspect of the world that is relevant to the generation and execution of a plan is known t... |

81 | The role of domain-specific knowledge in the planning as satisfiability framework
- Kautz, Selman
- 1998
(Show Context)
Citation Context ...hang and Stickel 2000), better heuristics (satz, Li and Anbulagan 1997), and constraint satisfaction solution techniques (relsat, Bayardo and Schrag 1997) have been developed more recently. blackbox (=-=Kautz and Selman 1998-=-a; Kautz and Selman 1999) integrates several of these approaches—walksat (stochastic local search), satz, andrelsat—in a planning system that allows the user to try different solvers on the Sat encodi... |

78 | Policy iteration for factored mdps
- Koller, Parr
(Show Context)
Citation Context ..., otherwise. 5 This is just one possibility. Another commonly used objective is that of maximizing expected discounted reward (Boutilier et al. 1995; Boutilier and Dearden 1996; Koller and Parr 1999; =-=Koller and Parr 2000-=-). A planning problem with this objective can be transformed to an equivalent goal-oriented probabilistic planning problem (Condon 1992; Zwick and Paterson 1996). See Appendix A for a proof.sThis defi... |

74 | Representing action: Indeterminacy and ramifications
- Giunchiglia, Kartha, et al.
- 1997
(Show Context)
Citation Context ...language underlying zander, for convenience we write down this representation using the Probabilistic Planning Language (PPL). PPL is a high-level action language that extends the action language AR (=-=Giunchiglia et al. 1997-=-) to support probabilistic domains. An st representation can be easily expressed by PPL. Each path through each decision tree is replaced by a PPL statement. The general form of a PPL statement for a ... |

72 | MAXPLAN: A New Approach to Probabilistic Planning
- Majercik, Littman
- 1998
(Show Context)
Citation Context ... which only the effects of actions are contingent on the outcomes of earlier actions, and this terminology has been generally adopted.sand udtpop (Peot 1998). The first planner we developed, maxplan (=-=Majercik and Littman 1998-=-a), falls into this category. 4. Contingent planning with probabilistic actions: As in Case 2, sensing allows planners in this category to produce contingent plans. As in Case 3, probabilistic actions... |

68 |
Generating conditional plans and programs
- Warren
- 1976
(Show Context)
Citation Context ... the term conditional has been used in different ways in the literature. Plans in which the execution of actions depends on the outcomes of earlier actions were originally called “conditional plans” (=-=Warren 1976-=-). Some researchers (Draper et al. 1994) suggested calling such plans “contingent plans”, reserving the term “conditional” for plans in which only the effects of actions are contingent on the outcomes... |

63 |
Dynamic Programming: Models and Applications
- Denardo
- 2003
(Show Context)
Citation Context ...2001), spudd (Hoey et al. 1999). zander, the contingent planner we developed (Section 6), falls into this category, as do traditional OR approaches (Bellman 1957; Howard 1960; Puterman and Shin 1978; =-=Denardo 1982-=-). Note that Cases 2 and 3 subsume Case 1, and Case 4 subsumes all the other cases; thus, a planner for addressing Case 4 can be used in all four scenarios. Our research has established a novel framew... |

63 | probabilistic planning: A unifying algorithm and effective search control mechanisms
- Onder, Pollack, et al.
- 1999
(Show Context)
Citation Context ...obabilistic actions allow the planner to specify the plan that has the highest probability of succeeding. Examples: c-buridan (Draper et al. 1994), dtpop (Peot 1998), mahinur (Onder and Pollack 1997; =-=Onder and Pollack 1999-=-), and pgraphplan/tgraphplan (Blum and Langford 1999), pomdp:inc prune (Cassandra et al. 1997), Hansen-Feng (Hansen and Feng 2000), gpt (Bonet and Geffner 2001), spudd (Hoey et al. 1999). zander, the ... |

63 | Stochastic constraint programming
- Walsh
- 2002
(Show Context)
Citation Context ...ping and learning (deriving no-goods) (Bayardo and Schrag 1997). Perhaps a more direct way of exploiting the connection to CSPs is to model planning problems using stochastic constraint satisfaction (=-=Walsh 2002-=-), as this provides a more direct way of expressing multivalued domain variables. 9.2 Extending ZANDER The improvements discussed in the sections above focus on accelerating zander’s performance. An e... |

54 | Dynamic programming for POMDPs using a factored state representation
- Hansen, Feng
- 2000
(Show Context)
Citation Context ...er et al. 1994), dtpop (Peot 1998), mahinur (Onder and Pollack 1997; Onder and Pollack 1999), and pgraphplan/tgraphplan (Blum and Langford 1999), pomdp:inc prune (Cassandra et al. 1997), Hansen-Feng (=-=Hansen and Feng 2000-=-), gpt (Bonet and Geffner 2001), spudd (Hoey et al. 1999). zander, the contingent planner we developed (Section 6), falls into this category, as do traditional OR approaches (Bellman 1957; Howard 1960... |

54 | Implementing the davis-putnam method - Zhang, Stickel |

44 | Modeling a dynamic and uncertain world i: Symbolic and probabilistic reasoning about change
- Hanks, McDermott
- 1994
(Show Context)
Citation Context ...tional probability tables represented as trees (Boutilier et al. 1999; Boutilier et al. 1995). (This representation is also equivalent to the strips-like probabilistic state-space operators, or PSOs, =-=Hanks and McDermott 1993-=-; Littman 1997). The st representation of a planning domain can be defined formally as M = 〈P, I, A, T, G T , G F , O〉. Here, P is a finite set of n distinct propositions. The set of states is the pow... |

41 | Conditional linear planning
- Goldman, Boddy
- 1994
(Show Context)
Citation Context ...bility 0.10, and that nothing happens with probability 0.05. A non-probabilistic version of the same action might specify that if the 2 Such actions have historically been called conditional actions (=-=Goldman and Boddy 1994-=-a; Goldman and Boddy 1994b; Peot and Smith 1992). In our taxonomy of planning under uncertainty, however, we wish to make a distinction between the type of planning and the type of actions used, so we... |