Abstract — This paper considers the problem of learning to recognize different terrains from color imagery in a fully automatic fashion, using the robot’s mechanical sensors as supervision. We present a probabilistic framework in which the visual information and the mechanical supervision interact to learn the available terrain types. Within this framework, a novel supervised dimensionality reduction method is proposed, in which the automatic supervision provided by the robot helps select better lower dimensional representations, more suitable for the discrimination task at hand. Incorporating supervision into the dimensionality reduction process is important, as some terrains might be visually similar but induce very different robot mobility. Therefore, choosing a lower dimensional visual representation adequately is expected to improve the vision-based terrain learning and the final classification performance. This is the first work that proposes automatically supervised dimensionality reduction in a probabilistic framework using the supervision coming from the robot’s sensors. The proposed method stands in between methods for reasoning under uncertainty using probabilistic models and methods for learning the underlying structure of the data. The proposed approach has been tested on field test data collected by an autonomous robot while driving on soil, gravel and asphalt. Although the supervision might be ambiguous or noisy, our experiments show that it helps build a more appropriate lower dimensional visual representation and achieves improved terrain recognition performance compared to unsupervised learning methods. I.

A critical challenge in robot learning from demonstration is the ability to map the behavior of the trainer onto a robot’s existing repertoire of basic/primitive capabilities. In part, this problem is due to the fact that the observed behavior of the teacher may consist of a combination (or superposition) of the robot’s individual primitives. The problem becomes more complex when the task involves temporal sequences of goals. We introduce an autonomous control architecture that allows for learning of hierarchical task representations, in which: 1) every goal is achieved through a linear superposition (or fusion) of robot primitives and 2) sequencing across goals is achieved through arbitration. We treat learning of the appropriate superposition as a state estimation problem over the space of possible linear fusion weights, inferred through a particle filter. We validate our approach in both simulated and real world environments with a Pioneer 3DX mobile robot.

This paper proposes a systematic approach of representing abstract features in terms of low-level, subjective state representations. We demonstrate that a mapping between the agent’s predictive state representation and abstract features can be derived automatically from high-level training data supplied by the designer. Our empirical evaluation demonstrates that an experience-oriented state representation built around a single-bit sensor can represent useful abstract features such as “back against a wall”, “in a corner”, or “in a room”. As a result, the agent gains virtual sensors that could be used by its control policy. 1 1

We consider autonomous robots as having associated control policies that determine their actions in response to perceptions of the environment. Often, these controllers are explicitly transferred from a human via programmatic description or physical instantiation. Alternatively, Robot Learning from Demonstration (RLfD) can enable a robot to learn a policy from observing only demonstrations of the task itself. We focus on interactive, teleoperative teaching, where the user manually controls the robot and provides demonstrations while receiving learner feedback. With regression, the collected perception-actuation pairs are used to directly estimate the underlying policy mapping. This dissertation contributes an RLfD methodology for interactive, mixed-initiative learning of unknown tasks. The goal of the technique is to enable users to implicitly instantiate autonomous robot controllers that perform desired tasks as well as the demonstrator, as measured by task-specific metrics. With standard regression techniques, we show that such “on-par” learning is restricted to policies typified by a many-to-one mapping (a unimap) from perception to actuation. Thus, controllers representable as multi-state Finite State Machines (FSMs) and that exhibit a one-tomany mapping (a multimap) cannot be learnt. To be able to do so we must address the three issues of model selection (how many subtasks or FSM states), policy learning (for each subtask),

We present an implementation and analysis of a real-time, on-line, supervised learning system for non-parametrically learning behaviors from a human trainer on a mobile robot in outdoor environments. This approach enables a human operator to train and tune robot behaviors simply by driving the robot with a remote control. Hand-designed behaviors for outdoor environments often require many parameters, and complicated behaviors can be difficult or impossible to specify with a manageable number of parameters. Furthermore, their design requires knowledge of the robot’s internal models, and knowledge of the environment in which the behaviors will be used. In real-world scenarios, we can design new behaviors using our learning system much more quickly than we can write hand-crafted behaviors. We present the results of training the robot to execute several specialized and general-purpose behaviors, including traversing a slalom, staying near “cover”, navigating on paths, navigating in an obstacle field, and general-purpose navigation. Our system learns and executes most of these behaviors well after 1-4 hours of operator training time. In quantitative tests, the learned behavior is not as robust as a hand-crafted behavior, but often completes obstacle courses more quickly. Additionally, we identify the factors that influence the effectiveness of this approach and investigate the properties of the training data provided by the human trainer. Based on our analyses, we suggest future work to ensure sufficient training, handle conflicting training examples, model robot dynamics, and further investigate dimensionality reduction of perception features. 1