## The Stochastic Motion Roadmap: A sampling framework for planning with Markov motion uncertainty (2008)

### Cached

### Download Links

- [www.cs.unc.edu]
- [robotics.cs.unc.edu]
- [www.ieor.berkeley.edu]
- [goldberg.berkeley.edu]
- [www.ieor.berkeley.edu]
- [queue.ieor.berkeley.edu]
- [ieor.berkeley.edu]
- [queue.berkeley.edu]
- [www.ieor.berkeley.edu]
- [www.roboticsproceedings.org]
- [roboticsproceedings.org]
- [www.phriends.eu]
- [homepages.laas.fr]
- [www.laas.fr]
- DBLP

### Other Repositories/Bibliography

Venue: | in Robotics: Science and Systems III (Proc. RSS 2007 |

Citations: | 54 - 16 self |

### BibTeX

@INPROCEEDINGS{Alterovitz08thestochastic,

author = {Ron Alterovitz and Thierry Siméon and Ken Goldberg},

title = {The Stochastic Motion Roadmap: A sampling framework for planning with Markov motion uncertainty},

booktitle = {in Robotics: Science and Systems III (Proc. RSS 2007},

year = {2008},

pages = {246--253},

publisher = {MIT Press}

}

### OpenURL

### Abstract

Abstract — We present a new motion planning framework that explicitly considers uncertainty in robot motion to maximize the probability of avoiding collisions and successfully reaching a goal. In many motion planning applications ranging from maneuvering vehicles over unfamiliar terrain to steering flexible medical needles through human tissue, the response of a robot to commanded actions cannot be precisely predicted. We propose to build a roadmap by sampling collision-free states in the configuration space and then locally sampling motions at each state to estimate state transition probabilities for each possible action. Given a query specifying initial and goal configurations, we use the roadmap to formulate a Markov Decision Process (MDP), which we solve using Infinite Horizon Dynamic Programming in polynomial time to compute stochastically optimal plans. The Stochastic Motion Roadmap (SMR) thus combines a sampling-based roadmap representation of the configuration space, as in PRM’s, with the well-established theory of MDP’s. Generating both states and transition probabilities by sampling is far more flexible than previous Markov motion planning approaches based on problem-specific or grid-based discretizations. We demonstrate the SMR framework by applying it to nonholonomic steerable needles, a new class of medical needles that follow curved paths through soft tissue, and confirm that SMR’s generate motion plans with significantly higher probabilities of success compared to traditional shortest-path plans. I.

### Citations

1005 | Probablistic Roadmaps for path planning in high-dimensional configuration space
- Kavraki, Svestka, et al.
- 1996
(Show Context)
Citation Context ...is to compute plans that maximize the probability of success. Our framework builds on the highly successful approach used in Probabilistic Roadmaps (PRM’s): a learning phase followed by a query phase =-=[20]-=-. During the learning phase, a random (or quasi-random) sample of discrete states is selected in the configuration space, and a roadmap is built that represents their collision-free connectivity. Duri... |

883 |
Probabilistic Robotics
- Thrun, Burgard, et al.
- 2005
(Show Context)
Citation Context ...n the current state and future actions are known) [22]. Extensive work has explored uncertainty associated with robot sensing, including SLAM and POMDP’s to represent uncertainty in the current state =-=[32, 12]-=-. In this paper, we assume the current state is known (or can be precisely determined from sensors), and we focus on the latter type of uncertainty, predictability. Predictability can be affected by u... |

714 | Planning Algorithms
- LaValle
- 2006
(Show Context)
Citation Context ...state to the goal, possibly optimizing some criteria such as minimum length. PRM’s have successfully solved many path planning problems for applications such as robotic manipulators andsmobile robots =-=[12, 22]-=-. The term “probabilistic” in PRM comes from the random sampling of states. An underlying assumption is that the collision-free connectivity of states is specified using boolean values rather than dis... |

322 |
On curves of minimal length with a constraint on average curvature, and with prescribed initial and terminal positions and tangents
- Dubins
- 1957
(Show Context)
Citation Context ...robotics and related fields [22, 13, 24]. Past work has addressed deterministic curvature-constrained path planning where a mobile robot’s path is, like a car, constrained by a minimum turning radius =-=[17, 20, 23, 9, 31, 1]-=-. For steerable needles, Park et al. applied a numeric diffusion-based method but did not consider obstacle avoidance or motion uncertainty [30]. Alterovitz et al. proposed an MDP formulation to find ... |

270 |
Principles of Robot Motion: Theory, Algorithms, and Implementations
- Choset, Lynch, et al.
- 2005
(Show Context)
Citation Context ...state to the goal, possibly optimizing some criteria such as minimum length. PRM’s have successfully solved many path planning problems for applications such as robotic manipulators andsmobile robots =-=[12, 22]-=-. The term “probabilistic” in PRM comes from the random sampling of states. An underlying assumption is that the collision-free connectivity of states is specified using boolean values rather than dis... |

237 | The Parti-game Algorithm for Variable Resolution Reinforcement Learning
- Moore
- 1993
(Show Context)
Citation Context ...any existing planners for deterministic motion specialize in finding feasible paths through narrow passageways in complex configuration spaces using specialized sampling [3, 9] or learning approaches =-=[29]-=-. Since a narrow passageway is unlikely to be robust to motion uncertainty, finding these passageways is not the ultimate goal of our method. Our method builds a roadmap that samples the configuration... |

173 | Planning under time constraints in stochastic domains
- Dean, Kaelbling, et al.
- 1995
(Show Context)
Citation Context ...ition probabilities between states, have also been applied to motion planning by subdividing the workspace using regular grids and defining transition probabilities for motions between the grid cells =-=[14, 16, 22]-=-. These methods differ from SMR’s since they use grids or problem-specific discretization. Many existing motion planners specialize in finding feasible paths through narrow passageways in complex conf... |

170 |
der Stappen. The gaussian sampling strategy for probabilistic roadmap planners
- Boor, Overmars, et al.
- 1999
(Show Context)
Citation Context ...ds or problem-specific discretization. Many existing motion planners specialize in finding feasible paths through narrow passageways in complex configuration spaces using specialized sampling methods =-=[3, 9]-=-. Since a narrow passageway is unlikely to be robust to motion uncertainty, finding these passageways is not the ultimate goal of our method. Our method builds a roadmap that samples the configuration... |

147 |
A Motion Planner for Nonholonomic Mobile Robots
- Laumond, Jacobs, et al.
- 1994
(Show Context)
Citation Context ...d related fields [12, 22]. Past work has addressed deterministic curvature-constrained path planning with obstacles where a mobile robot’s path is, like a car, constrained by a minimum turning radius =-=[19, 21, 8, 30, 28]-=-. For steerable needles, Park et al. applied a numeric diffusion-based method but did not consider obstacles or motion uncertainty [29]. Alterovitz et al. proposed an MDP formulation to find a stochas... |

110 | On the relationship between classical grid search and probabilistic roadmaps
- Lavalle, Branicky, et al.
- 2004
(Show Context)
Citation Context ...rm random sampling of the configuration space inside the bounds defined by (Bmin i , Bmax i ) for i = 1, . . . , d, although other random distributions or quasi- random sampling methods could be used =-=[12, 23]-=-. For each state s ∈ V and an action u ∈ U, buildSMR calls the function getTransitions, defined in Procedure 2, to obtain a set of possible next states in V and probabilities of entering those states ... |

100 |
J.Canny, "Planning smooth paths for mobile robots
- Jacobs
- 1989
(Show Context)
Citation Context ...d related fields [12, 22]. Past work has addressed deterministic curvature-constrained path planning with obstacles where a mobile robot’s path is, like a car, constrained by a minimum turning radius =-=[19, 21, 8, 30, 28]-=-. For steerable needles, Park et al. applied a numeric diffusion-based method but did not consider obstacles or motion uncertainty [29]. Alterovitz et al. proposed an MDP formulation to find a stochas... |

68 | Finding an Unpredictable Target in a Workspace with Obstacles
- LaValle, Lin, et al.
- 1997
(Show Context)
Citation Context ...nd by uncertainty in the robot’s motion. Previous and ongoing work addresses many aspects of uncertainty in the workspace, including uncertainty in the goal location, such as in pursuit-evasion games =-=[22, 24]-=-, and in dynamic environments with moving obstacles [34, 33, 27]. A recently +sdeveloped method for grasp planning uses POMDP’s to consider uncertainty in the configuration of the robot and the state ... |

65 | Motion planning: A journey of robots, molecules, digital actors, and other artifacts,” Int
- Latombe
- 1999
(Show Context)
Citation Context ...mpute optimal actions. We apply SMRM’s to needle steering, a type of nonholonomic control-based motion planning problem. Nonholonomic motion planning has a long history in robotics and related fields =-=[22, 13, 24]-=-. Past work has addressed deterministic curvature-constrained path planning where a mobile robot’s path is, like a car, constrained by a minimum turning radius [17, 20, 23, 9, 31, 1]. For steerable ne... |

62 | Stochastic roadmap simulation: an efficient representation and algorithm for analyzing molecular motion
- Apaydin, Brutlag, et al.
- 2002
(Show Context)
Citation Context ...hortest path plans. In SMR, “stochastic” refers to the motion of the robot, not to the sampling of states. PRM’s were previously extended to explore stochastic motion in molecular conformation spaces =-=[4, 5]-=-, but without a planning component to optimize actions. Our SMR formulation is applicable to a variety of decision-based robotics problems. It is particularly suited for nonholonomic robots, for which... |

49 |
Dynamic Programming and Optimal Control, 2nd ed
- Bertsekas
(Show Context)
Citation Context ...ome p then Remove (t, p) from R R ← R ∪ {(t, p + 1/m)} else R ← R ∪ {(t, 1/m)} return R We observe that Eq. 2 is an MDP and has the form of the Bellman equation for a stochastic shortest path problem =-=[7]-=-: J ∗ � (i) = max Pij(ui) (g(i, ui, j) + J ui ∗ (j)). (3) j∈V where g(i, ui, j) is a “reward” for transitioning from state i to j after action ui. In our case, g(i, ui, j) = 0 for all i, ui, and j, an... |

47 |
Motion planning with uncertainty: A landmark approach
- Lazanas, Latombe
- 1995
(Show Context)
Citation Context ...rid-based numerical methods and geometric analysis have been applied to robots with motion uncertainty (sometimes combined with sensing uncertainty) using costbased objectives and worst-case analysis =-=[26, 10, 25]-=-. MDP’s, a general approach that requires explicitly defining transition probabilities between states, have also been applied to motion planning by subdividing the workspace using regular grids and de... |

42 |
Toward optimal configuration space sampling
- Burns, Brock
- 2005
(Show Context)
Citation Context ...nd current state, which encodes all necessary past history. As in PRM’s, the method first samples the configuration space, where the sampling can be random [20], pseudo-random [23], or utility-guided =-=[11]-=-. We then sample the robot’s motion model to build a Stochastic Motion Roadmap (SMR), a set of weighted directed graphs with vertices as sampled states and edges encoding feasible state transitions an... |

41 |
OBPRM: An obstacle-based PRM for 3D workspaces,” in Robotics: The Algorithmic Perspective
- Amato, Bayazit, et al.
- 1998
(Show Context)
Citation Context ...ds or problem-specific discretization. Many existing motion planners specialize in finding feasible paths through narrow passageways in complex configuration spaces using specialized sampling methods =-=[3, 9]-=-. Since a narrow passageway is unlikely to be robust to motion uncertainty, finding these passageways is not the ultimate goal of our method. Our method builds a roadmap that samples the configuration... |

41 | Grasping POMDPs
- Hsiao, Kaelbling, et al.
- 2007
(Show Context)
Citation Context ...s with moving obstacles [34, 33, 27]. A recently +sdeveloped method for grasp planning uses POMDP’s to consider uncertainty in the configuration of the robot and the state of the objects in the world =-=[18]-=-. In this paper we focus explicitly on the case of uncertainty in the robot’s motion rather than in goal or obstacle locations. Apaydin et al. previously explored the connection between probabilistic ... |

33 | Efficient nearest-neighbor searching for motion planning
- Atramentov, LaValle
(Show Context)
Citation Context ...ation. Hence, each call requires O(mdn) time using brute-force nearest neighbor checking. For certain low-dimensional configuration spaces, this can be reduced to O (m exp (d) log (n)) using kd-trees =-=[6]-=-. Hence, the total time complexity of building an SMR is O � wmdn 2� or O (wm exp (d) n log (n)). This does not include the cost of n state collision checks and nm checks of collision free paths, whic... |

32 | A numerical technique for planning motion strategies of a mobile robot in presence of uncertainty
- Bouily, Simeon, et al.
- 1995
(Show Context)
Citation Context ...rid-based numerical methods and geometric analysis have been applied to robots with motion uncertainty (sometimes combined with sensing uncertainty) using costbased objectives and worst-case analysis =-=[26, 10, 25]-=-. MDP’s, a general approach that requires explicitly defining transition probabilities between states, have also been applied to motion planning by subdividing the workspace using regular grids and de... |

27 | An objective-based framework for motion planning under sensing and control uncertainties
- LaValle, Hutchinson
- 1998
(Show Context)
Citation Context ...rid-based numerical methods and geometric analysis have been applied to robots with motion uncertainty (sometimes combined with sensing uncertainty) using costbased objectives and worst-case analysis =-=[26, 10, 25]-=-. MDP’s, a general approach that requires explicitly defining transition probabilities between states, have also been applied to motion planning by subdividing the workspace using regular grids and de... |

25 |
Diffusion-based motion planning for a nonholonomic flexible needle model
- Park, Kim, et al.
- 2005
(Show Context)
Citation Context ...ke a car, constrained by a minimum turning radius [19, 21, 8, 30, 28]. For steerable needles, Park et al. applied a numeric diffusion-based method but did not consider obstacles or motion uncertainty =-=[29]-=-. Alterovitz et al. proposed an MDP formulation to find a stochastic shortest path for a steerable needle to a goal configuration, subject to userspecified “cost” parameters for direction changes, ins... |

25 | Nonholonomic modeling of needle steering
- Kim, Cowan, et al.
- 2006
(Show Context)
Citation Context ...lans for steerable needles, a new class of flexible bevel-tip medical needles that clinicians can steer through soft tissue around obstacles to reach targets inaccessible to traditional stiff needles =-=[2, 35]-=-. As in many medical applications, considering uncertainty is crucial to the success of medical needle insertion procedures: the needle tip may deflect from the expected path due to tissue inhomogenei... |

19 | Steering Flexible Needles under Markov Motion Uncertainty - Alterovitz, Lim, et al. - 2005 |

19 | Planning Shortest Bounded-Curvature Paths for a Class of NonHolonomic Vehicles Among Obstacles
- Bicchi, Casalino, et al.
- 1996
(Show Context)
Citation Context ...d related fields [12, 22]. Past work has addressed deterministic curvature-constrained path planning with obstacles where a mobile robot’s path is, like a car, constrained by a minimum turning radius =-=[19, 21, 8, 30, 28]-=-. For steerable needles, Park et al. applied a numeric diffusion-based method but did not consider obstacles or motion uncertainty [29]. Alterovitz et al. proposed an MDP formulation to find a stochas... |

19 | Curvatureconstrained shortest paths in a convex polygon
- Agarwal, Biedl, et al.
- 2002
(Show Context)
Citation Context ...robotics and related fields [22, 13, 24]. Past work has addressed deterministic curvature-constrained path planning where a mobile robot’s path is, like a car, constrained by a minimum turning radius =-=[17, 20, 23, 9, 31, 1]-=-. For steerable needles, Park et al. applied a numeric diffusion-based method but did not consider obstacle avoidance or motion uncertainty [30]. Alterovitz et al. proposed an MDP formulation to find ... |

18 |
Design considerations for robotic needle steering
- Memisevic, Okamura
- 2005
(Show Context)
Citation Context ...e. A new class of needles, steerable needles, are composed of a flexible material and with a bevel-tip that can be steered to targets in soft tissue that are inaccessible to traditional stiff needles =-=[35, 36, 2]-=-. Steerable needles are controlled by two degrees of freedom actuated at the needle base: insertion distance and bevel direction. When the needle is inserted, the asymmetric force applied by the bevel... |

11 | Constant-curvature motion planning under uncertainty with applications in image-guided medical needle steering
- Alterovitz, Branicky, et al.
- 2006
(Show Context)
Citation Context ... for direction changes, insertion distance, and collisions [2]. Because these costs are difficult to quantify, Alterovitz et al. introduced the objective function of maximizing probability of success =-=[1]-=-. These methods use a regular grid of states and an ad-hoc, identical discretization of the motion uncertainty distribution at all states. The methods do not consider the advantages of sampling states... |

11 | Focussed dynamic programming: Extensive comparative results,” Robotics
- Ferguson, Stentz
- 2004
(Show Context)
Citation Context ...ition probabilities between states, have also been applied to motion planning by subdividing the workspace using regular grids and defining transition probabilities for motions between the grid cells =-=[14, 16, 22]-=-. These methods differ from SMR’s since they use grids or problem-specific discretization. Many existing motion planners specialize in finding feasible paths through narrow passageways in complex conf... |

11 |
Approximation and decision algorithms for curvature constrained path planning: A state-space approach
- Sellen
- 1998
(Show Context)
Citation Context |

11 | Planning the shortest safe path amidst unpredictably moving obstacles
- Berg, Overmars
- 2008
(Show Context)
Citation Context ...g work addresses many aspects of uncertainty in the workspace, including uncertainty in the goal location, such as in pursuit-evasion games [22, 24], and in dynamic environments with moving obstacles =-=[34, 33, 27]-=-. A recently +sdeveloped method for grasp planning uses POMDP’s to consider uncertainty in the configuration of the robot and the state of the objects in the world [18]. In this paper we focus explici... |

10 |
Computer Graphics with OpenGL” 3rd Ed
- Hearn, Baker
- 2004
(Show Context)
Citation Context ...hen uncertainty is high. We define the workspace as a rectangle of width xmax and height ymax and define obstacles as polygons in the plane. To detect obstacle collisions, we use the zerowinding rule =-=[17]-=-. We define the distance between two states s1 and s2 to be the weighted Euclidean distance between the poses plus an indicator variable to ensure the turning directions match: distance(s1, s2) = � (x... |

8 | Stochastic conformational roadmaps for computing ensemble properties of molecular motion
- Apaydin, Brutlag, et al.
- 2004
(Show Context)
Citation Context ...hortest path plans. In SMR, “stochastic” refers to the motion of the robot, not to the sampling of states. PRM’s were previously extended to explore stochastic motion in molecular conformation spaces =-=[4, 5]-=-, but without a planning component to optimize actions. Our SMR formulation is applicable to a variety of decision-based robotics problems. It is particularly suited for nonholonomic robots, for which... |

7 | Real time feedback control for nonholonomic mobile robots with obstacles
- Lindemann, Hussein, et al.
- 2006
(Show Context)
Citation Context ...g work addresses many aspects of uncertainty in the workspace, including uncertainty in the goal location, such as in pursuit-evasion games [22, 24], and in dynamic environments with moving obstacles =-=[34, 33, 27]-=-. A recently +sdeveloped method for grasp planning uses POMDP’s to consider uncertainty in the configuration of the robot and the state of the objects in the world [18]. In this paper we focus explici... |

7 | High-speed autonomous navigation with motion prediction for unknown moving obstacles
- Vasquez, Large, et al.
- 2004
(Show Context)
Citation Context ...g work addresses many aspects of uncertainty in the workspace, including uncertainty in the goal location, such as in pursuit-evasion games [22, 24], and in dynamic environments with moving obstacles =-=[34, 33, 27]-=-. A recently +sdeveloped method for grasp planning uses POMDP’s to consider uncertainty in the configuration of the robot and the state of the objects in the world [18]. In this paper we focus explici... |

5 |
Segmentation of surgical needles for fluoroscopy servoing using the Insight Software Toolkit (ITK
- Cleary, Ibanez, et al.
- 2003
(Show Context)
Citation Context ...ease uncertainty due to stiffness along the needle shaft. Medical imaging in the operating room can be used to measure the needle’s current position and orientation to provide feedback to the planner =-=[13, 15]-=-, but this measurement by itself provides no information about the effect of future deflections during insertion due to motion uncertainty. Stochastic motion roadmaps offer features particularly benef... |

3 |
Needle artifact localization
- DiMaio, Kacher, et al.
- 2006
(Show Context)
Citation Context ...ease uncertainty due to stiffness along the needle shaft. Medical imaging in the operating room can be used to measure the needle’s current position and orientation to provide feedback to the planner =-=[13, 15]-=-, but this measurement by itself provides no information about the effect of future deflections during insertion due to motion uncertainty. Stochastic motion roadmaps offer features particularly benef... |

3 | Trajectory Planning Using Reachable-State Density Functions
- Mason, Burdick
- 2002
(Show Context)
Citation Context |

1 |
dD spatial searching
- Tangelder, Fabri
(Show Context)
Citation Context ...tch: distance(s1, s2) = � (x1 − x2) 2 + (y1 − y2) 2 + α(θ1 − θ2) 2 + M, where M → ∞ if b1 �= b2, and M = 0 otherwise. For fast nearest-neighbor computation, we use the CGAL implementation of kd-trees =-=[31]-=-. Since the distance function is non-Euclidean, we use the formulation developed by Atramentov and LaValle to build the kd-tree [6]. We define the goal T ∗ as all configuration states within a ball of... |