s prior to execution. It thus offers several potential advantages. Natural language can concisely express rules and command sequences. Also, because it uses symbols and syntactic rules, it is well suited to interact with robots that represent knowledge at the symbolic level. Such symbolic communication can help robots learn faster 1 than those that learn at the sensory -motor association level. Here we describe our initial steps toward realizing an IBL system. Along with an overview of how IBL works, we discuss its key steps in detail, our progress on each, and the challenges we're encountering along the way. How IBL works In IBL, we convert the user's verbal instructions into new internal program code that represents new procedures. Such procedures become part of a procedure pool that robots reuse to learn increasingly complex procedures. Hence, the robot should be capable of executing increasingly complex

rnal stimuli. That is, an agent must perceive something about its environment and be able to perform appropriate actions as conditions change. Let's call this the animate criterion. This does not mean an agent must have a camera or a robotic arm, merely that it has some means of input and output. This might be as simple as a text-based system. The point is that an agent should not always emit the exact same response; at the least, some changes in its input should lead to changes in its output. Certainly we would not consider intelligent a robot that just sat on a table or drove forward. Given that perception and action are definitely required, determining such a system 's intelligence comes down to judging its actions' appropriateness in light of the prevailing conditions. In fact, robotics is often defined as "the intelligent connection of perception to action." 2 The question now becomes, how do we make this connection? The 1950s and 1960s saw the creation of the first robotic dev

Future domestic robots will need to adapt to the special needs of their users and to their environment. Programming by natural language will be a key method enabling computer language-naïve users to instruct their robots. Its main advantages over other learning methods are speed of acquisition and ability to build high level symbolic rules into the robot. This paper describes the design of a practical system that uses unconstrained speech to teach a vision-based robot how to navigate in a miniature town. The robot knows a set of primitive navigation procedures that the user can refer to when giving route instructions. A particularity of this project is that the primitive procedures are determined by analysing a corpus of route instructions. It is found that primitives natural to the user, such as “turn left after the church ” are very complex procedures for the robot, involving visual scene analysis and local route planning. Thus, to enable natural user-robot interaction, a high-level of intelligence needs to be built into “primitive ” robot procedures. Another finding is that the set of primitive procedures is likely not to be closed. Thus, on time to time, a user is likely to refer to a procedure that is not preprogrammed in the robot. How best to handle this is currently investigated. In general, the use of Instruction-Based Learning (IBL) imposes a number of constraints on the design of robotics systems and knowledge representation. These issues and proposed solutions are described in the paper. 1.

Future robots are expected to communicate with humans using natural language. The nave human user will expect a robot to easily understand what he/she is meaning by instructions concerning robot's tasks. This implies that the robot will need to have a means of grounding, in its own sensors, the natural language terms and constructions used by the human user. This paper presents an approach to solve this problem that is based on the integration of a "learning server" in the software architecture of the robot. Such server should be capable of on-line, incremental learning from examples

Speech interfaces should have a capability of dealing with inexplicit utterances including such as ellipsis and deixis since they are common phenomena in our daily conversation. Their resolution using context and a priori knowledge has been investigated in the fields of natural language and speech understanding. However, there are utterances that cannot be understood by such symbol processing alone. In this paper, we consider inexplicit utterances caused from the fact that humans have vision. If we are certain that the listeners share some visual information, we often omit or mention ambiguously things about it in our utterances. We propose a method of understanding speech with such ambiguities using computer vision. It tracks the human’s gaze direction, detecting objects in the direction. It also recognizes the human’s actions. Based on these bits of visual information, it understands the human’s inexplicit utterances. Experimental results show that the method helps to realize human-friendly speech interfaces. 1.

www.elsevier.com/locate/robot Robots are rapidly evolving from factory work-horses to robot-companions. The future of robots, as our companions, is highly dependent on their abilities to understand, interpret and represent the environment in an efficient and consistent fashion, in a way that is comprehensible to humans. The work presented here is oriented in this direction. It suggests a hierarchical probabilistic representation of space that is based on objects. A global topological representation of places with object graphs serving as local maps is proposed. The work also details the first efforts towards conceptualizing space on the basis of the human compatible representation so formed. Such a representation and the resulting conceptualization would be useful for enabling robots to be cognizant of their surroundings. Experiments on place classification and place recognition are reported in order to demonstrate the applicability of such a representation towards understanding space and thereby performing spatial cognition. Further, relevant results from user studies validating the proposed representation are also reported. Thus, the theme of the work is — representation for spatial cognition. c ○ 2007 Elsevier B.V. All rights reserved.

We introduce an approach for physically-grounded natural language interpretation by robots which reacts appropriately to unanticipated physical changes in the environment and dynamically assimilates new information pertinent to ongoing tasks. At the core of the approach is a model of object schemas that enables a robot to encode beliefs about physical objects in its environment using collections of coupled processes responsible for sensorimotor interaction. These interaction processes run concurrently in order to ensure responsiveness to the environment, while coordinating sensorimotor expectations, action planning, and language use. The model has been implemented on a robot that manipulates objects on a tabletop in response to verbal input. The implementation responds to verbal requests such as “Group the green block and the red apple, ” while adapting in real-time to unexpected physical collisions and taking opportunistic advantage of any new information it may receive through perceptual and linguistic channels.

Abstract not found

Carl is a prototype of an intelligent service robot, designed having in mind such tasks as serving food in a reception or acting as a host in an organization.

We describe a speech system for commanding robots in human-occupied outdoor military supply depots. To operate in such environments, the robots must be as easy to interact with as are humans, i.e. they must reliably understand ordinary spoken instructions, such as orders to move supplies, as well as commands and warnings, spoken or shouted from distances of tens of meters. These design goals preclude close-talking microphones and “push-to-talk ” buttons that are typically used to isolate commands from the sounds of vehicles, machinery and non-relevant speech. We used multiple microphones to provide omnidirectional coverage. A novel voice activity detector was developed to detect speech and select the appropriate microphone to listen to. Finally, we developed a recognizer model that could successfully recognize commands when heard amidst other speech within a noisy environment. When evaluated on speech data in the field, this system performed significantly better than a more computationally intensive baseline system, reducing the effective false alarm rate by a factor of 40, while maintaining the same level of precision. Index Terms — Human-robot interaction, real-time speech recognition, voice activity detection, modulation frequency.