Autonomous mobile robots need very reliable navigation capabilities in order to operate unattended for long periods of time. We present a technique for achieving this goal that uses partially observable Markov decision process models (POMDPs) to explicitly model navigation uncertainty, including actuator and sensor uncertainty and approximate knowledge of the environment. This allows the robot to maintain a probability distribution over its current pose. Thus, while the robot rarely knows exactly where it is, it always has some belief as to what its true pose is, and is never completely lost. We present a navigation architecture based on POMDPs that provides a uniform framework with an established theoretical foundation for pose estimation, path planning, robot control during navigation, and learning. Our experiments show that this architecture indeed leads to robust corridor navigation for an actual indoor mobile robot. 1

Autonomous mobile robots need good models of their environment, sensors and actuators to navigate reliably and efficiently. While this information can be supplied by humans, or learned from scratch through active exploration, such approaches are tedious and time-consuming. Our approach is to provide the robot with the topological and geometrical constraints that are easily obtainable by humans, and have the robot learn the rest while in the course of performing its tasks. We present GROW-BW, an unsupervised and passive distance learning algorithm that overcomes the problem that the robot can never be sure about its location if it is not allowed to reduce its uncertainty by asking a teacher or executing localization actions. Advantages of GROW-BW include that the robot can be used immediately to perform navigation tasks and improves its performance over time, focusing its attention to routes that are more relevant for its tasks. We demonstrate that GROW-BW can learn good distance, sensor, and actuator models with only a small amount of experience.

A new approach to skill acquisition in assembly is proposed. An assembly skill is represented by a hybrid dynamic system where a discrete event controller models the skill at the task level. The output of the discrete event controller provides the reference commands for the underlying robot controller. This structure is naturally encoded by a hidden Markov model (HMM). The HMM parameters are obtained by training on sensory data from human demonstrations of the skill. Currently, assembly tasks have to be performed by human operators or by robots using expensive fixtures. Our approach transfers the assembly skill from an expert human operator to the robot, thus making it possible for a robot to perform assembly tasks without the use of expensive fixtures. 1 Introduction Manipulation tasks such as assembly are easily performed by human operators. However, these tasks are still difficult for robots and require the use of precise and expensive fixtures. Furthermore, human operators are ab...

The many degrees-of-freedom and distributed sensing capability of dextrous robot hands permits the use of control programs that rely on qualitative changes in sensor feedback rather than precise positioning and force information. One way of designing such a control program is to have the robot learn the qualitative control characteristics from examples. A convenient way of providing these examples is via teleoperation. To this end, this paper presents results for recognizing and segmenting manipulation primitives from a teleoperated task by analysis of features in sensor feedback. k-nearest quantized pattern vectors determine potential classifications. A hidden Markov model provides task context for the final segmentation. The illustrative task is picking up a plastic egg with a spatula. 1 Introduction Dextrous hands can be controlled qualitatively using strategies that servo on significant changes in sensor feedback. Using the many degrees of freedom and distributed sensing capabili...

Laparoscopic surgical skills evaluation of surgery residents is usually a subjective process, carried out in the operating room by senior surgeons. By its nature, this process is performed using fuzzy criteria. The objective of the current study was to develop and assess an objective laparoscopic surgical skill scale using Hidden Markov Models (HMM) based on haptic information, tool/tissue interactions and visual task decomposition. Methods: Eight subjects (six surgical trainees: first year surgical residents 2xR1, third year surgical residents 2xR3 fifth year surgical residents 2xR5; and two expert laparoscopic surgeons: 2xES) performed laparoscopic cholecystectomy following a specific 7 steps protocol on a pig. An instrumented laparoscopic grasper equipped with a three-axis force/torque sensor located at the proximal end with an additional force sensor located on the handle, was used to measure the forces and torques. The hand/tool interface force/torque data was synchronized with a video of the tool operative maneuvers. A synthesis of frame-by-frame video analysis was used to define 14 different types of tool/tissue interactions, each one associated with unique force/torque (F/T) signatures. HMMs were developed for each subject representing the surgical skills by defining the various tool/tissue interactions as states and the associated F/T signatures as observations. The statistical distance between the HMMs representing residents at different levels of their training and the HMMs of expert surgeons were calculated in order to generate a learning curve of selected steps during laparoscopic cholecystectomy. Results: Comparison of HMM's between groups showed significant differences between all skill levels, supporting the objective definition of a l...

A process monitor for robotic assembly based on Hidden Markov Models (HMMs) is presented. The HMM process monitor is based on the dynamic force/torque signals arising from interaction between the workpiece and the environment. The HMMs represent a stochastic, knowledge-based system where the models are trained off-line with the Baum-Welch re-estimation algorithm. The assembly task is modeled as a discrete event dynamic system, where a discrete event is defined as a change in contact state between the workpiece and the environment. Our method 1) allows for dynamic motions of the workpiece, 2) accounts for sensor noise and friction and 3) exploits the fact that the amount of force information is large when there is a sudden change of discrete state in robotic assembly. After the HMMs have been trained, we use them on-line in a 2D experimental setup to recognise discrete events as they occur. Successful event recognition with an accuracy as high as 97% was achieved in 0.5-0.6 seconds with...

The difficulty associated with programing existing robots is one of the main impediments to them finding application in domestic environments such as the home. A promising method for simplifying robot programing is Programing by Demonstration (PbD). Here, an end user can provide a demonstration of the task to be programed, with a PbD "interface" interpreting the demonstration in order to determine low-level control details for the robot. A key aspect of the interpretation process is to make it robust to the noise typically included in a demonstration by the human. In this paper we present a method to help identify and eliminate any noise present in the demonstration. Our method involves two steps. The first step uses the demonstration to build up a partial knowledge of the geometry present in the task. Statistical regression analysis is used on demonstrated trajectories to determine equations describing curved surfaces in configuration space. The second step in our method uses the geometric information obtained in the first step to determine if there are more optimal paths than those demonstrated for completing the task. If there are, our method proposes these as the appropriate control commands for the robot. We show the validity of our approach by presenting successful experiments on a realistic household-type task---changing rolls on a paper roll holder.

Modeling dynamic human control strategy (HCS), or human skill in response to real-time sensing is becoming an increasingly popular paradigm in many different research areas, such as intelligent vehicle systems, virtual reality, and space robotics. Such models are often learned from experimental data, and as such can be characterized despite the lack of a good physical model. Unfortunately, learned models presently offer few, if any, guarantees in terms of model fidelity to the training data. This is especially true for dynamic reaction skills, where errors can feed back on themselves to generate state and command trajectories uncharacteristic of the source process. Thus, we propose a stochastic similarity measure---based on hidden Markov model (HMM) analysis---capable of comparing and contrasting stochastic, dynamic, multidimensional trajectories. This similarity measure is the first step in validating a learned model's fidelity to its training data by comparing the model's dynamic trajectories in the feedback loop to the human's dynamic trajectories. In this paper, we first derive and demonstrate properties of the similarity measure for stochastic systems. We then apply the similarity measure to real-time human driving data by comparing different control strategies among different individuals. We show that the proposed similarity measure out performs the more traditional Bayes classifier in correctly grouping driving data from the same individual. Finally, we illustrate how the similarity measure can be used in the validation of models which are learned from experimental data, and how we can connect model validation and model learning to iteratively improve our models of human control strategy.

this article, we describe and develop methodologies for mod- eling and transferring human control strategy (HCS). This research has potential application in a variety of areas such as the Intelligent Vehicle Highway System (IVHS), human-machine interfacing, real-time training, space telerobotics, and agile manufacturing. We specifically address the following issues: (1) how to efficiently model human control strategy through learning cascade neural networks, (2) how to select state inputs in order to generate reliable models, (3) how to validate the computed models through an independent, Hidden Markov Model-based procedure, and (4) how to effectively transfer human control strategy. We have implemented this approach experimentally in the real-time control of a human driving simulator, and are working to transfer these methodologies for the control of an autonomous vehicle and a mobile robot. In providing a framework for abstracting computational models of human skill, we expect to facilitate analysis of human control, the development of humanlike intelligent machines, improved human-robot coordination, and the transfer of skill from one human to another

The best method of training for laparoscopic surgical skills is controversial. Some advocate observation in the operating room, while others promote animal and simulated models or a combination of surgery-related tasks. A crucial process in surgical education is to evaluate the level of surgical skills. For laparoscopic surgery, skill evaluation is traditionally performed subjectively by experts grading a video of a procedure performed by a student. By its nature, this process uses fuzzy criteria. The objective of the current study was to develop and assess a skill scale using Markov models (MMs). Ten surgeons [five novice surgeons (NS); five expert surgeons (ES)] performed a cholecystectomy and Nissen fundoplication in a porcine model. An instrumented laparoscopic grasper equipped with a three-axis force/torque ( ) sensor was used to measure the forces/torques at the hand/tool interface synchronized with a video of the tool operative maneuvers. A synthesis of frame-by-frame video analysis and a vector quantization algorithm, allowed to define signatures associated with 14 different types of tool/tissue interactions. The magnitude of applied by NS and ES were significantly different ( 0 05) and varied based on the task being performed. High magnitudes were applied by NS compared with ES while performing tissue manipulation and vise versa in tasks involved tissue dissection. From each step of the surgical procedures, two MMs were developed representing the performance of three surgeons out of the five in the ES and NS groups. The data obtained by the remaining two surgeons in each group were used for evaluating the performance scale. The final result was a surgical performance index which represented a ratio of statistical similarity between the examined surgeon 's MM ...