Results 1  10
of
12
The Stochastic Motion Roadmap: A sampling framework for planning with Markov motion uncertainty
 in Robotics: Science and Systems III (Proc. RSS 2007
, 2008
"... Abstract — We present a new motion planning framework that explicitly considers uncertainty in robot motion to maximize the probability of avoiding collisions and successfully reaching a goal. In many motion planning applications ranging from maneuvering vehicles over unfamiliar terrain to steering ..."
Abstract

Cited by 95 (20 self)
 Add to MetaCart
(Show Context)
Abstract — We present a new motion planning framework that explicitly considers uncertainty in robot motion to maximize the probability of avoiding collisions and successfully reaching a goal. In many motion planning applications ranging from maneuvering vehicles over unfamiliar terrain to steering flexible medical needles through human tissue, the response of a robot to commanded actions cannot be precisely predicted. We propose to build a roadmap by sampling collisionfree states in the configuration space and then locally sampling motions at each state to estimate state transition probabilities for each possible action. Given a query specifying initial and goal configurations, we use the roadmap to formulate a Markov Decision Process (MDP), which we solve using Infinite Horizon Dynamic Programming in polynomial time to compute stochastically optimal plans. The Stochastic Motion Roadmap (SMR) thus combines a samplingbased roadmap representation of the configuration space, as in PRM’s, with the wellestablished theory of MDP’s. Generating both states and transition probabilities by sampling is far more flexible than previous Markov motion planning approaches based on problemspecific or gridbased discretizations. We demonstrate the SMR framework by applying it to nonholonomic steerable needles, a new class of medical needles that follow curved paths through soft tissue, and confirm that SMR’s generate motion plans with significantly higher probabilities of success compared to traditional shortestpath plans. I.
Bounded realtime dynamic programming: RTDP with monotone upper bounds and performance guarantees
 In ICML’05
, 2005
"... MDPs are an attractive formalization for planning, but realistic problems often have intractably large state spaces. When we only need a partial policy to get from a fixed start state to a goal, restricting computation to states relevant to this task can make much larger problems tractable. We intro ..."
Abstract

Cited by 39 (1 self)
 Add to MetaCart
(Show Context)
MDPs are an attractive formalization for planning, but realistic problems often have intractably large state spaces. When we only need a partial policy to get from a fixed start state to a goal, restricting computation to states relevant to this task can make much larger problems tractable. We introduce a new algorithm, Bounded RTDP, which can produce partial policies with strong performance guarantees while only touching a fraction of the state space, even on problems where other algorithms would have to visit the full state space. To do so, Bounded RTDP maintains both upper and lower bounds on the optimal value function. The performance of Bounded RTDP is greatly aided by the introduction of a new technique to efficiently find suitable upper bounds; this technique can also be used to provide informed initialization to a wide range of other planning algorithms. 1.
Prioritizing Bellman backups without a priority queue
 In Proc. of the 17th International Conference on Automated Planning and Scheduling (ICAPS07), this volumn
"... Several researchers have shown that the efficiency of value iteration, a dynamic programming algorithm for Markov decision processes, can be improved by prioritizing the order of Bellman backups to focus computation on states where the value function can be improved the most. In previous work, a pri ..."
Abstract

Cited by 10 (1 self)
 Add to MetaCart
Several researchers have shown that the efficiency of value iteration, a dynamic programming algorithm for Markov decision processes, can be improved by prioritizing the order of Bellman backups to focus computation on states where the value function can be improved the most. In previous work, a priority queue has been used to order backups. Although this incurs overhead for maintaining the priority queue, previous work has argued that the overhead is usually much less than the benefit from prioritization. However this conclusion is usually based on a comparison to a nonprioritized approach that performs Bellman backups on states in an arbitrary order. In this paper, we show that the overhead for maintaining the priority queue can be greater than the benefit, when it is compared to very simple heuristics for prioritizing backups that do not require a priority queue. Although the order of backups induced by our simple approach is often suboptimal, we show that its smaller overhead allows it to converge faster than other stateoftheart prioritybased solvers.
Topological Value Iteration Algorithms
"... Value iteration is a powerful yet inefficient algorithm for Markov decision processes (MDPs) because it puts the majority of its effort into backing up the entire state space, which turns out to be unnecessary in many cases. In order to overcome this problem, many approaches have been proposed. Amon ..."
Abstract

Cited by 4 (0 self)
 Add to MetaCart
(Show Context)
Value iteration is a powerful yet inefficient algorithm for Markov decision processes (MDPs) because it puts the majority of its effort into backing up the entire state space, which turns out to be unnecessary in many cases. In order to overcome this problem, many approaches have been proposed. Among them, ILAO * and variants of RTDP are stateoftheart ones. These methods use reachability analysis and heuristic search to avoid some unnecessary backups. However, none of these approaches build the graphical structure of the state transitions in a preprocessing step or use the structural information to systematically decompose a problem, whereby generating an intelligent backup sequence of the state space. In this paper, we present two optimal MDP algorithms. The first algorithm, topological value iteration (TVI), detects the structure of MDPs and backs up states based on topological sequences. It (1) divides an MDP into stronglyconnected components (SCCs), and (2) solves these components sequentially. TVI outperforms VI and other stateoftheart algorithms vastly when an MDP has multiple, closetoequalsized SCCs. The second algorithm, focused topological value iteration (FTVI), is an extension of TVI. FTVI restricts its attention to connected components that are relevant for solving the MDP. Specifically, it uses a small amount of heuristic search to eliminate provably suboptimal actions; this pruning allows FTVI to find smaller connected components, thus running faster. We demonstrate that FTVI outperforms TVI by an order of magnitude, averaged across several domains. Surprisingly, FTVI also significantly outperforms popular ‘heuristicallyinformed ’ MDP algorithms such as ILAO*, LRTDP, BRTDP and BayesianRTDP in many domains, sometimes by as much as two orders of magnitude. Finally, we characterize the type of domains where FTVI excels — suggesting a way to an informed choice of solver. 1.
Generalizing Dijkstra's Algorithm and Gaussian Elimination for Solving MDPs
, 2005
"... We study the problem of computing the optimal value function for a Markov decision process with positive costs. Computing this function quickly and accurately is a basic step in many schemes for deciding how to act in stochastic environments. There are efficient algorithms which compute value functi ..."
Abstract

Cited by 3 (1 self)
 Add to MetaCart
We study the problem of computing the optimal value function for a Markov decision process with positive costs. Computing this function quickly and accurately is a basic step in many schemes for deciding how to act in stochastic environments. There are efficient algorithms which compute value functions for special types of MDPs: for deterministic MDPs with S states and A actions, Dijkstra's algorithm runs in time O(AS log S). And, in singleaction MDPs (Markov chains), standard linearalgebraic algorithms find the value function in time O(S ), or faster by taking advantage of sparsity or good conditioning. Algorithms for solving general MDPs can take much longer: we are not aware of any speed guarantees better than those for comparablysized linear programs. We present a family of algorithms which reduce to Dijkstra's algorithm when applied to deterministic MDPs, and to standard techniques for solving linear equations when applied to Markov chains. More importantly, we demonstrate experimentally that these algorithms perform well when applied to MDPs which "almost" have the required special structure.
Faster dynamic programming for Markov decision processes
, 2007
"... Markov decision processes (MDPs) are a general framework used by Artificial Intelligence (AI) researchers to model decision theoretic planning problems. Solving real world MDPs has been a major and challenging research topic in the AI literature. This paper discusses two main groups of approaches in ..."
Abstract
 Add to MetaCart
(Show Context)
Markov decision processes (MDPs) are a general framework used by Artificial Intelligence (AI) researchers to model decision theoretic planning problems. Solving real world MDPs has been a major and challenging research topic in the AI literature. This paper discusses two main groups of approaches in solving MDPs. The first group of approaches combines the strategies of heuristic search and dynamic programming to expedite the convergence process. The second makes use of graphical structures in MDPs to decrease the effort of classic dynamic programming algorithms. Two new algorithms proposed by the author, MBLAO* and TVI, are described here.
Actuator Networks for Navigating an Unmonitored Mobile Robot ∗
"... Abstract—Building on recent work in sensoractuator networks and distributed manipulation, we consider the use of pure actuator networks for localizationfree robotic navigation. We show how an actuator network can be used to guide an unobserved robot to a desired location in space and introduce an ..."
Abstract
 Add to MetaCart
(Show Context)
Abstract—Building on recent work in sensoractuator networks and distributed manipulation, we consider the use of pure actuator networks for localizationfree robotic navigation. We show how an actuator network can be used to guide an unobserved robot to a desired location in space and introduce an algorithm to calculate optimal actuation patterns for such a network. Sets of actuators are sequentially activated to induce a series of static potential fields that robustly drive the robot from a start to an end location under movement uncertainty. Our algorithm constructs a roadmap with probabilityweighted edges based on motion uncertainty models and identifies an actuation pattern that maximizes the probability of successfully guiding the robot to its goal. Simulations of the algorithm show that an actuator network can robustly guide robots with various uncertainty models through a twodimensional space. We experiment with additive Gaussian Cartesian motion uncertainty models and additive Gaussian polar models. Motion randomly chosen destinations within the convex hull of a 10actuator network succeeds with with up to 93.4 % probability. For n actuators, and m samples per transition edge in our roadmap, our runtime is O(mn 6).
Focussed Processing of MDPs for Path Planning
"... We present a heuristicbased algorithm for solving restricted Markov decision processes (MDPs). Our approach, which combines ideas from deterministic search and recent dynamic programming methods, focusses computation towards promising areas of the state space. It is thus able to significantly reduc ..."
Abstract
 Add to MetaCart
(Show Context)
We present a heuristicbased algorithm for solving restricted Markov decision processes (MDPs). Our approach, which combines ideas from deterministic search and recent dynamic programming methods, focusses computation towards promising areas of the state space. It is thus able to significantly reduce the amount of processing required to produce a solution. We demonstrate this improvement by comparing the performance of our approach to the performance of several existing algorithms on a robotic path planning domain. 1.