This paper describes a new multi-player computer game, Colored Trails (CT), which may be played by people, computers and heterogeneous groups. CT was designed to enable investigation of properties of decision-making strategies in multi-agent situations of varying complexity. The paper presents the results of an initial series of experiments of CT games in which agents ’ choices affected not only their own outcomes but also the outcomes of other agents. It compares the behavior of people with that of computer agents deploying a variety of decision-making strategies. The results align with behavioral economics studies in showing that people cooperate when they play and that factors of social dependency influence their levels of cooperation. Preliminary results indicate that people design agents to play strategies closer to game-theory predictions, yielding lower utility. Additional experiments show that such agents perform worse than agents designed to make choices that resemble human cooperative behavior. The paper describes challenges raised by these results for designers of agents, especially agents that need to operate in heterogeneous groups that include people. 1.

This paper presents protocols and strategies for coalition formation with incomplete information under time constraints. It focuses on strategies for coalition members to distribute revenues amongst themselves. Such strategies should preferably be stable, lead to a fair distribution, and maximize the social welfare of the agents. These properties are only partially supported by existing coalition formation mechanisms. In particular, stability and the maximization of social welfare are supported only in the case of complete information, and only at a high computational complexity. Recent studies on coalition formation with incomplete and uncertain information address revenue distribution in a naïve manner. In this study we specifically refer to environments with limited computational resources and incomplete information. We propose a variety of strategies for revenue distribution, including the strategy in which the agents attempt to distribute the estimated net value of a coalition equally. A variation of the equal distribution strategy in which agents compromise and agree to a payoff lower than their estimated equal share, was specifically examined. Our experimental results show that, under time constraints, the compromise strategy is stable and increases the social welfare compared to non-compromise strategies. 1.

In this paper, we present ARF, our initial effort at solving taskallocation problems where cooperative agents need to perform tasks simultaneously. An example is multi-agent routing problems where several agents need to visit targets simultaneously, for example, to move obstacles out of the way cooperatively. First, we propose reaction functions as a novel way of characterizing the costs of agents in a distributed way. Second, we show how to approximate reaction functions so that their computation and communication times are polynomial. Third, we show how reaction functions can be used by a central planner to allocate tasks to agents. Finally, we show experimentally that the resulting task allocations are better than those of other greedy methods that do not use reaction functions.

Abstract. Networked multi-agent systems are comprised of many autonomous yet interdependent agents situated in a virtual social network. Two examples of such systems are supply chain networks and sensor networks. A common challenge in many networked multiagent systems is decentralized team formation among the spatially and logically extended agents. Even in cooperative multi-agent systems, efficient team formation is made difficult by the limited local information available to the individual agents. We present a model of distributed multi-agent team formation in networked multi-agent systems, describe a policy learning framework for joining teams based on local information, and give empirical results on improving team formation performance. In particular, we show that local policy learning from limited information leads to a significant increase in organizational team formation performance compared to a random policy.

Abstract. The continuous growth in ubiquitous and mobile network connectivity, together with the increasing number of networked devices populating our everyday environments, call for a deep rethinking of traditional communication and service architectures. The emerging area of autonomic communication addresses such challenging issues by trying to identify novel flexible network architectures, and by conceiving novel conceptual and practical tools for the design, development, and execution of “autonomic ” (i.e., self-organizing, self-adaptive and context-aware) communication services. In this paper, after having introduced the general concepts behind autonomic communication and autonomic communication services, we analyze the key issues of defining suitable “component ” models for autonomic communication services, and discuss the strict relation between such models and agent models. On this basis, we survey and compare different approaches, and eventually try to synthesize the key desirable characteristics that one should expect from a general-purpose component model for autonomic communication services. The key message we will try to deliver is that current research in software agents and multi-agent systems have the potential for playing a major role in inspiring and driving the identification of such a model, and more in general for influencing and advancing the whole area of autonomic communication.

The problem of coalition formation when agents are uncertain about the types or capabilities of their potential partners is a critical one. In [3] a Bayesian reinforcement learning framework is developed for this problem when coalitions are formed (and tasks undertaken) repeatedly: not only does the model allow agents to refine their beliefs about the types of others, but uses value of information to define optimal exploration policies. However, computational approximations in that work are purely myopic. We present novel, non-myopic learning algorithms to approximate the optimal Bayesian solution, providing tractable means to ensure good sequential performance. We evaluate our algorithms in a variety of settings, and show that one, in particular, exhibits consistently good sequential performance. Further, it enables the Bayesian agents to transfer acquired knowledge among different dynamic tasks.

Networked multiagent systems are comprised of many autonomous yet interdependent agents situated in a virtual social network. Two examples of such systems are supply chain networks and sensor networks. A common challenge in many networked multiagent systems is decentralized team formation among the spatially and logically extended agents. Even in cooperative multiagent systems, efficient team formation is made difficult by the limited local information available to the individual agents. We present a model of distributed multiagent team formation in networked multi-agent systems, describe a policy learning framework for joining teams based on local information, and give empirical results on improving team formation performance. In particular, we show that local policy learning from limited information leads to a significant increase in organizational team formation performance compared to a naive heuristic.

Sequential decision making under uncertainty is always a challenge for autonomous agents populating a multiagent environment, since their behaviour is inevitably influenced by the be-haviour of others. Further, agents have to constantly struggle to find the right balance between exploiting current information regarding the environment and the rest of its inhabitants, and ex-ploring so that they acquire additional information. Moreover, they need to profitably trade off short-term rewards with anticipated long-term ones, while learning through interaction about the environment and others—employing techniques from reinforcement learning (RL), a fun-damental area of study within artificial intelligence (AI). Coalition formation is a problem of great interest within game theory and AI, allowing autonomous individually rational agents to form stable or transient teams (or coalitions) to tackle an underlying task. Agents participating in realistic scenarios of repeated coalition formation under uncertainty face the issues identified above, and need to bargain to succesfully negotiate the terms of their participation in coalitions—often having to compromise individual with team welfare effectively. In this thesis, we provide theoretical and algorithmic tools to accommodate sequential de-

Coalition formation in multi-agent systems is a process where agents form coalitions and work together to solve a joint problem via cooperating or coordinating their actions within each coalition. It is important for distributed applications ranging from electronic business to mobile and ubiquitous computing where adaptation to changing resources and environments is crucial. Coalition formation is useful as it may increase the ability of agents to accomplish tasks and achieve their goals. However, in complex real-world environments that agents operate in, the available resources are generally constrained. Agents only have incomplete even inaccurate information about the dynamically changing world. The occurrence of events may require the agents to react in a real-time manner. Agents’ actions may result in uncertain outcomes. These factors inevitably influence the formation process and formation outcome of a coalition. We employ a learning-based two-phased coalition formation approach to help agents form coalitions in complex environments. The approach consists of (1) a two-phase (planning and instantiation) coalition formation model, (2) a two-level (strategic and tactical) learning mechanism, (3) an adaptive, confidence-based negotiation strategy, and

We consider a dynamic market-place of selfinterested agents with di#ering capabilities.