This paper describes Soccer Server, a simulator of the game of soccer designed as a test-bench for evaluating multi-agent systems and cooperative algorithms. In real life, successful soccer teams require many qualities, such as basic ball control skills, the ability to carry out plans, and teamwork. We believe that simulating such behaviors is a significant challenge for Computer Science, Artificial Intelligence and Robotics technologies. It is to promote the development of such technologies, and to help define a new standard problem for research, that we have developed Soccer Server. We demonstrate the potential of Soccer Server by reporting an experiment that uses the system to compare the performance of a neural network architecture and a decision tree algorithm at learning the selection of soccer play-plans. Other researchers using Soccer Server to investigate the nature of cooperative behavior in a multi-agent environment will have the chance to assess their progress at RoboCup-97...

In the past few years, Multiagent Systems (MAS) has emerged as an active subfield of Artificial Intelligence (AI). Because of the inherent complexity of MAS, there is much interest in using Machine Learning (ML) techniques to help build multiagent systems. Robotic soccer is a particularly good domain for studying MAS and Multiagent Learning. Our approach to using ML as a tool for building Soccer Server clients involves layering increasingly complex learned behaviors. In this article, we describe two levels of learned behaviors. First, the clients learn a low-level individual skill that allows them to control the ball effectively. Then, using this learned skill, they learn a higher-level skill that involves multiple players. For both skills, we describe the learning method in detail and report on our extensive empirical testing. We also verify empirically that the learned skills are applicable to game situations.

Soccer is a rich domain for the study of multiagent learning issues. Not only must the players learn low-level skills, but they must also learn to work together and to adapt to the behaviors of different opponents. We are using a robotic soccer system to study these different types of multiagent learning: low-level skills, collaborative, and adversarial. Here we describe in detail our experimental framework. We present a learned, robust, low-level behavior that is necessitated by the multiagent nature of the domain, namely shooting a moving ball. We then discuss the issues that arise as we extend the learning scenario to require collaborative and adversarial learning.

. We use simulated soccer to study multiagent learning. Each team's players (agents) share action set and policy, but may behave differently due to position-dependent inputs. All agents making up a team are rewarded or punished collectively in case of goals. We conduct simulations with varying team sizes, and compare several learning algorithms: TD-Q learning with linear neural networks (TD-Q), Probabilistic Incremental Program Evolution (PIPE), and a PIPE version that learns by coevolution (CO-PIPE). TD-Q is based on learning evaluation functions (EFs) mapping input/action pairs to expected reward. PIPE and CO-PIPE search policy space directly. They use adaptive probability distributions to synthesize programs that calculate action probabilities from current inputs. Our results show that linear TD-Q encounters several difficulties in learning appropriate shared EFs. PIPE and CO-PIPE, however, do not depend on EFs and find good policies faster and more reliably. This suggests that in s...

Robotic soccer is a challenging research domain which involves multiple agents that need to collaborate in an adversarial environment to achieve specific objectives. In this paper, we describe CMUnited, the team of small robotic agents that we developed to enter the RoboCup-97 competition. We designed and built the robotic agents, devised the appropriate vision algorithm, and developed and implemented algorithms for strategic collaboration between the robots in an uncertain and dynamic environment. The robots can organize themselves in formations, hold specific roles, and pursue their goals. In game situations, they have demonstrated their collaborative behaviors on multiple occasions.

Learning how to adjust to an opponent's position is critical to the success of having intelligent agents collaborating towards the achievement of specific tasks in unfriendly environments. This paper describes our work on developing methods to learn to choose an action based on a continuous-valued state attribute indicating the position of an opponent. We use a framework in which teams of agents compete in a simulator of a game of robotic soccer. We introduce a memory-based supervised learning strategy which enables an agent to choose to pass or shoot in the presence of a defender. In our memory model, training examples affect neighboring generalized learned instances with different weights. We conduct experiments in which the agent incrementally learns to approximate a function with a continuous domain. Then we investigate the question of how the agent performs in nondeterministic variations of the training situations. Our experiments indicate that when the random variations fall within some bound of the initial training, the agent performs better with some initial training rather than from a tabula-rasa.

Robotic soccer involves multiple agents that need to collaborate in an adversarial environment to achieve specific objectives. In this paper, we describe the robotic agents and architecture that we have developed to enter RoboCup-97. The framework integrates high-level and low-level reasoning as an overall wireless communication system of small-size robots, an overhead vision camera for perception, a centralized interface computer, and several clients as the minds of the robot players. We present the mobile robot platform specifications, the different communication servers and links, the vision processing algorithm, and the control code that enables strategic collaboration between teammates.

Reactive deliberation is a novel robot architecture that has been designed to overcome some of the problems posed by dynamic robot environments. It is argued that the problem of action selection in nontrivial domains cannot be intelligently resolved without attention to detailed planning. Experimental evidence is provided that the goals and actions of a robot must be evaluated at a rate commensurate with changes in the environment. The goal-oriented behaviours of reactive deliberation are a useful abstraction that allow sharing of scarce computational resources and effective goal-arbitration through inter-behaviour bidding. The effectiveness of reactive deliberation has been demonstrated through a tournament of one-on-one soccer games between real-world robots. Soccer is a dynamic environment; the locations of the ball and the robots are constantly changing. The results suggest that the architectural elements in reactive deliberation are sufficient for real-time intelligent control in ...

Robotic soccer is a new challenging multi-agent domain, in which agents need to collaborate in an adversarial environment to achieve specific objectives. In this paper, we describe CMUnited-97, our team of robotic agents that we developed to enter the RoboCup-97 competition. We first discuss the challenges underlying the robotic soccer domain as a multi-agent system. We then introduce our team architecture, briefly describe the system's perception, and present the robots' actions ranging from low-level individual behaviors to coordinated, strategic team behaviors. The robots can organize themselves in formations, hold specific roles, and pursue their goals. In game situations, they extensively used their role-based behaviors, and demonstrated collaboration on multiple occasions. As homogeneous agents, the robots can also switch roles to maximize the overall performance of the team. CMUnited-97 won the RoboCup-97 small-robot competition at IJCAI in Nagoya, Japan.

The goal of creating an integrated cognitive robot is still only a tantalizing dream. Current artificial intelligence and robotics research is highly divergent with little or no commonality among specialized subfields. New rich task domains are needed to pose the right challenges to extant theories and promote convergence. We propose soccerplaying as such a task since it requires situated robotics, perception, real-time decision making, planning, plan recognition, learning and multirobot coordination and control. The technology to perform real-time vision and build autonomous robots is available; the Dynamite testbed has been built to perform experiments with multiple robots.