Agent-oriented techniques represent an exciting new means of analysing, designing and building complex software systems. They have the potential to significantly improve current practice in software engineering and to extend the range of applications that can feasibly be tackled. Yet, to date, there have been few serious attempts to cast agent systems as a software engineering paradigm. This paper seeks to rectify this omission. Specifically, it will be argued that: (i) the conceptual apparatus of agent-oriented systems is well-suited to building software solutions for complex systems and (ii) agent-oriented approaches represent a genuine advance over the current state of the art for engineering complex systems. Following on from this view, the major issues raised by adopting an agent-oriented approach to software engineering are highlighted and discussed. 1.

Two minimal requirements for a satisfactory multiagent learning algorithm are that it 1. learns to play optimally against stationary opponents and 2. converges to a Nash equilibrium in self-play. The previous algorithm that has come closest, WoLF-IGA, has been proven to have these two properties in 2-player 2-action (repeated) games -- assuming that the opponent's mixed strategy is observable. Another algorithm, ReDVaLeR (which was introduced after the algorithm described in this paper), achieves the two properties in games with arbitrary numbers of actions and players, but still requires that the opponents' mixed strategies are observable. In this paper we present AWESOME, the first algorithm that is guaranteed to have the two properties in games with arbitrary numbers of actions and players. It is still the only algorithm that does so while only relying on observing the other players' actual actions (not their mixed strategies). It also learns to play optimally against opponents that eventually become stationary. The basic idea behind AWESOME (Adapt When Everybody is Stationary, Otherwise Move to Equilibrium) is to try to adapt to the others' strategies when they appear stationary, but otherwise to retreat to a precomputed equilibrium strategy. We provide experimental results that suggest that AWESOME converges fast in practice. The techniques used to prove the properties of AWESOME are fundamentally different from those used for previous algorithms, and may help in analyzing future multiagent learning algorithms as well.

become a new fashioned managerial practice. Though KM theories seem to benefit from a "contamination" with cognitive and social sciences, which emphasize a subjective, contextual, and distributed approach to knowledge representation and integration, current technologies support what we may call a "god's eye" paradigm, in which knowledge is viewed as an objective resource. In this paper we discuss artificial intelligence theories and technologies that can support a shift to a new paradigm, called the "distributed intelligence" paradigm, in designing KM systems. Using the evolution of KM systems within Arthur Andersen Consulting as a motivating case study, we propose the framework of MultiContext Systems as a specification language for distributed intelligence KM systems, and sketch an agent-based architecture as an example of a KM system which embodies the assumptions of the distributed intelligence paradigm.

Agent orientation is currently pursued primarily as a software paradigm. Software with characteristics such as autonomy, sociality, reactivity and proactivity, and communicative and cooperative abilities are expected to offer greater functionality and higher quality, in comparison to earlier paradigms such as object orientation. Agent models and languages are thus intended as abstractions of computational behaviour, eventually to be realized in software programs. However, for the successful application of any software technology, the software system must be understood and analyzed in the context of its environment in the world. This paper argues for a notion of agent suitable for modelling the strategic relationships among agents in the world, so that users and stakeholders can reason about the implications of alternate technology solutions and social structures, thus to better decide on solutions that address their strategic interests and needs. The discussion draws on recent work in requirements engineering and agent-oriented methodologies. A small example from telemedicine is used to illustrate.

In this paper we focus on the problem of designing a collective of autonomous agents that individually learn sequences of actions such that the resultant sequence of joint actions achieves a predetermined global objective. We are particularly interested in instances of this problem where centralized control is either impossible or impractical. For single agent systems in similar domains, machine learning methods (e.g., reinforcement learners [18]) have been successfully used [1, 2, 3, 31]. However, applying such solutions directly to multi-agent systems often proves problematic, as agents may work at cross-purposes, or have difficulty in evaluating their contribution to achievement of the global objective, or both. Accordingly, the crucial design step in multiagent systems centers on determining the private objectives of each agent so that as the agents strive for those objectives, the system reaches a good global solution. In this work we consider a version of this problem involving multiple autonomous agents in a grid world. We use concepts from collective intelligence [19, 27, 30] to design goals for the agents that are "aligned" with the global goal, and are "learnable" in that agents can readily see how their behavior affects their utility. We show that reinforcement learning agents using those goals outperform both "natural" extensions of single agent algorithms and global reinforcement learning solutions based on "team games".

. Much of agent system development to date has been done on an ad hoc basis. These problems limit the extent to which "industrial applications" can be built using agent technology, as the building blocks, reusable techniques, approaches and architectures have either not been exposed or have not yet been fully elaborated. In the mid 80's, supporters of object -oriented technology had similar problems. However, with the aid of software patterns, objects have provided an important shift in the way developers successfully build applications today. In this paper, after describing an agent pattern's generic format, we identify a set of software patterns for agent coordination. 1.

Abstract. This paper provides a survey of existing research on agent-based approaches to transportation and traffic management. A framework for describing and assessing this work will be presented and systematically applied. We are mainly adopting a logistical perspective, thus focusing on freight transportation. However, when relevant, work of traffic and transport of people will be considered. A general conclusion from our study is that agent-based approaches seem very suitable for this domain, but that this still needs to be verified by more deployed system. 1

In this paper the notion of collective intention in teams of agents involved in cooperative problem solving (CPS) in multiagent systems (MAS) is investigated. Starting from individual intentions, goals,andbeliefs defining agents' local asocial motivational and informational attitudes, we arrive at an understanding of a collective intention in a team of agents. The presented definitions are rather strong, in particular a collective intention implies that all members intend for all others to share that intention. Based on this, we assume that a team of agents is created on the basis of a collective intention, and exists as long as this attitude between team members exists, after which the group may disintegrate. For this reason it is crucial that a collective intention within a group lasts long enough.

This paper investigates a relatively new direction in Multiagent Reinforcement Learning. Most multiagent learning techniques focus on Nash equilibria as elements of both the learning algorithm and its evaluation criteria. In contrast, we propose a multiagent learning algorithm that is optimal in the sense of finding a best-response policy, rather than in reaching an equilibrium. We present the first learning algorithm that is provably optimal against restricted classes of non-stationary opponents. The algorithm infers an accurate model of the opponent's non-stationary strategy, and simultaneously creates a best-response policy against that strategy. Our learning algorithm works within the very general framework of #-player, general-sum stochastic games, and learns both the game structure and its associated optimal policy.

When designing agent systems, it is often infeasible to foresee all the potential situations an agent may encounter and specify an agent behavior optimally in advance. In order to overcome these design problems, agents have to learn from and adapt to their environment.