The Robot World Cup Initiative (RoboCup) is an attempt to foster AI and intelligent robotics research by providing a standard problem where wide range of technologies can be integrated and examined. In order for a robot team to actually perform a soccer game, various technologies must be incorporated including: design principles of autonomous agents, multiagent collaboration, strategy acquisition, realtime reasoning, robotics, and sensor-fusion. Unlike AAAI robot competition, which is tuned for a single heavy-duty slow-moving robot, RoboCup is a task for a team of multiple fastmoving robots under a dynamic environment. Although RoboCup's final target is a world cup with real robots, RoboCup offers a software platform for research on the software aspects of RoboCup. This paper describes technical challenges involved in RoboCup, rules, and simulation environment. 1 Introduction: RoboCup as a Standard AI Problem We propose a Robot World Cup (RoboCup), as a new standard problem for AI an...

In this paper, we rst discuss the meaning of physical embodiment and the complexity of the environment in the context of multiagent learning. We then propose a vision-based reinforcement learning method that acquires cooperative behaviors in a dynamic environment. We use the robot soccer game initiated by RoboCup [12] to illustrate the eectiveness of our method. Each agent works with other team members to achieve a common goal against opponents. Our method estimates the relationships between a learner's behaviors and those of other agents in the environment through interactions (observations and actions) using a technique from system identication. In order to identify the model of each agent, Akaike's Information Criterion is applied to the results of Canonical Variate Analysis to clarify the relationship between the observed data in terms of actions and future observations. Next, reinforcement learning based on the estimated state vectors is performed to obtain the optimal behavior...

Sensor and motor systems are not separable for autonomous agents to accomplish tasks in a dynamic environment. This paper proposes a method to represent the interaction between a vision-based learning agent and its environment. The method is called “motion sketch ” by which a one-eyed mobile robot can learn several behaviors such as obstacle avoidance and target pursuit. A motion sketch is a collection of visual motion cues detected by a group of visual tracking routines of which visual behaviors are determined by individual tasks, and is tightly coupled with motor behaviors which are obtained by Q-learning, a most widely used reinforcement learning method, based on the visual motion cues. In order for the motion sketch to work, first the fundamental relationship between visual motions and motor commands is obtained, and then the Q-learning is applied to obtain the set of motor commands tightly coupled with the motion cues. Finally, the experimental results of real robot implementation with real-time motion tracker are shown. 1

This paper proposes a method which estimates the relationships between learner's behaviors and other agents' ones in the environment through interactions (observation and action) using the method of system identification. In order to identify the model of each agent, Akaike's Information Criterion is applied to the results of Canonical Variate Analysis for the relationship between the observed data in terms of action and future observation. Next, reinforcement learning based on the estimated state vectors is performed to obtain the optimal behavior. The proposed method is applied to a soccer playing situation, where a rolling ball and other moving agents are well modeled and the learner's behaviors are successfully acquired by the method. Computer simulations and real experiments are shown and a discussion is given.

In this paper, we proposed a method by which a stereo vision-based mobile robot learns to reach a target by detecting and avoiding occlusions. We call the internal representation that describes the learned behavior \stereo sketch." First, an input scene is segmented into homogeneous regions by the enhanced ISODATA algorithm with MDL principle in terms of image coordinates and disparity information obtained from the fast stereo matcher based on the coarse-to- ne control method. Then, in terms of the segmented regions including the target area and their occlusion status identied during the stereo and motion disparity estimation process, we construct a state space for a reinforcement learning method to obtain target reaching behavior. As a result, the robot can avoid obstacles without explicitly describing them. We give the computer simulation results and real robot implementation to show the validity of our method. 1 Introduction Realization o# a#tono#o## a#ent# t#at o##anize t#ei# o#...

In this paper we describe a real time object tracking system consisting of three modules (motion detection, object tracking, robot control), each working with a moderate accuracy, implemented in parallel on a workstation cluster, and therefore operating fast without any specialized hardware. The robustness and quality of the system is achieved by a combination of these vision modules with an additional attention module which recognizes errors during the tracking. For object tracking in image sequences we apply the method of active contour models (snakes) which can be used for contour description and extraction as well. We show how the snake is initialized automatically by the motion detection module, explain the tracking module, and demonstrate the detection of errors during the tracking by the attention module. Experiments show that this approach allows a robust real--time object tracking over long image sequences. Using a formal error measurement presented in this paper it will be sh...

This paper describes a purposive computer vision system for visually guided tasks and a Multi-Agent architecture used to model it. In this architecture, the vision system's purpose is decomposed into a set of behaviors, which are translated into specific tasks. Purpose, behaviors and tasks, as well as the relationship among them, are modeled using a Multi-Agent approach: purpose is modeled by a society of autonomous agents, which communicate through a common language, each one responsible for a specific visually guided behavior; tasks are represented by basic agents, organized in a hierarchical structure. A description of a system that is being implemented as a testbed for the architecture is given, with some details on the implementation of the agents and its communication methods. Finally, a brief discussion about the use of the basic agents is done and research directions are proposed. 1. Introduction The purposive paradigm for computer vision [1], [2] is a field of computer visio...

An agent must acquire internal representation appropriate for its task, environment, sensors. As a learning algorithm, reinforcement learning is often utilized to acquire the relation between sensory input and action. Learning agents in the real world using visual sensors is often confronted with critical problem how to build necessary and sufficient state space for the agent to execute the task. In this paper, we propose acquisition of relation between vision and action using Visual State-Action Map (VSAM). VSAM is the application of Self-Organizing Map (SOM). Input image data is mapped on the node of learned VSAM. Then VSAM outputs the appropriate action for the state. We applied VSAM to real robot. The experimental result shows that a real robot avoids the wall while moving around the environment. 1 Introduction A sensor based agent performing a task in an environment must acquire internal representation about its environment. In order to adapt uncertain and unforgiving environmen...

We introduce our approach that makes a robot learn to behave adequately to accomplish a given task at hand through the interactions with its environment with less a priori knowledge about the environment or the robot itself. We briey present three research topics of vision-based robot learning in each of which visual perception is tightly coupled with actuator eects so as to learn an adequate behavior. First, a method of vision-based reinforcement learning by which a robot learns to shoot a ball into a goal is presented. Next, \motion sketch" for a one-eyed mobile robot to learn several behaviors such as obstacle avoidance and target pursuit is introduced. Finally, we show a method of purposive visual control consisting of an on-line estimator and a feedback/feedforward controller for uncalibrated camera-manipulator systems. All topics include the real robot experiments. 1 Introduction Realization of autonomous agents that organize their own internal structure in order to take actio...

An active vision system has to enable the implementation of reactive visual processes and of elementary visual behaviors in real time. Therefore the control architecture is extremely important. In this paper we discuss a number of issues related with the implementation of a real-time control architecture and describe the architecture we are using with camera heads. Another important issue of the operation of active vision binocular heads is their integration into more complex robotic systems. The design of the control architecture has to be suited to the integration of the system in other robotic systems. We claim that higher levels of autonomy and integration can be obtained by designing the system architecture based on the concept of purposive behavior. At the lower levels we consider vision as a sensor and integrate it in control systems (both feed-forward and servo loops) and several visual processes are implemented in parallel, computing relevant measures for the control process....