Choosing when to communicate is a fundamental problem in multi-agent systems. This problem becomes particularly challenging when communication is constrained and each agent has different partial information about the overall situation. We take a decision-theoretic approach to this problem that balances the benefits of communication against the costs. Although computing the exact value of communication is intractable, it can be estimated using a standard myopic assumption—that communication is only possible at the present time. We examine specific situations in which this assumption leads to poor performance and demonstrate an alternative approach that relaxes the assumption and improves performance. The results provide an effective method for value-driven communication policies in multi-agent systems. Key words: multi-agent systems, decentralized MDPs, communication, decision-theoretic planning. 1.

Abstract not found

Information

Traditional game-theoretic analysis for decision-making takes a normative approach, in which agents derive rational decisions from the game description. This approach cannot naturally and compactly capture agents that are uncertain about the structure of the game, the strategies of other agents or whether agents may deviate from their optimal strategy. This paper presents Networks of Influence Diagrams (NID), a compact, natural and highly expressive language for reasoning about agents ’ beliefs and decision-making processes. NIDs are graphical structures in which agents ’ mental models are represented as nodes in a network; a mental model for an agent may itself use descriptions of the mental models of other agents. NIDs are demonstrated by examples, showing how they can be used to describe conflicting and cyclic belief structures, and certain forms of bounded rationality. In an opponent modeling domain, NIDs were able to outperform other computational agents whose strategies were not known in advance. A novel equilibrium concept is defined that makes a distinction between agents ’ optimal strategies, and how they actually behave in reality. It is also shown that NIDs are more compact and structured than Bayesian games, the traditional formalism used to model uncertainty in multi-agent decision-making problems.

Abstract. In a collaborative environment, knowledge about collaborators ’ skills is an important factor when determining which team members should perform a task. However, this knowledge may be incomplete or uncertain. In this paper, we extend our ETAPP (Environment-Task-Agents-Policy-Protocol) collaboration framework by modeling team members that exhibit non-deterministic performance, and comparing two alternative ways of using these models to assign agents to tasks. Our simulation-based evaluation shows that performance variability has a large impact on task performance, and that task performance is improved by consulting agent models built from a small number of observations of agents’ recent performance. 1

Traditional game-theoretic analysis for decision-making takes a normative approach, in which agents derive rational decisions from the game description. This approach cannot naturally and compactly capture agents that are uncertain about the structure of the game, the strategies of other agents or whether agents may deviate from their optimal strategy. This paper presents Networks of Influence Diagrams (NID), a compact, natural and highly expressive language for reasoning about agents ’ beliefs and decision-making processes. NIDs are graphical structures in which agents ’ mental models are represented as nodes in a network; a mental model for an agent may itself use descriptions of the mental models of other agents. NIDs are demonstrated by examples, showing how they can be used to describe conflicting and cyclic belief structures, and certain forms of bounded rationality. In an opponent modeling domain, NIDs were able to outperform other computational agents whose strategies were not known in advance. A novel equilibrium concept is defined that makes a distinction between agents ’ optimal strategies, and how they actually behave in reality. It is also shown that NIDs are more compact and structured than Bayesian games, the traditional formalism used to model uncertainty in multi-agent decision-making problems. 1

Abstract. Collaboration plays a critical role when a group is striving for goals which are difficult or impossible to achieve by an individual. Knowledge about collaborators ’ contributions to a task is important when solving problems as a team. However, a problem in many collaboration scenarios is the uncertainty and incompleteness of such knowledge. To investigate this problem, we present a collaboration framework where team members use models of collaborators ’ performance to estimate contributions to a task, and propose agents for tasks based on these estimations. We conducted a simulation-based study to assess the impact of modeling limitations on task performance. The main results of our simulation are that maintaining models of agents improves task performance, but exhaustive model maintenance is not essential. Additionally, we found that the ability of agents to update their models has a large impact on task performance. We then extended our framework to support more refined agent models, and performed additional simulated studies. Our results indicated that task performance is improved by the availability of additional reasoning resources and the use of probabilistic models that represent variable agent performance. 1

The ability for a given agent to adapt on-line to better interact with another agent is a difficult and important problem. This problem becomes even more difficult when the agent to interact with is a human, since humans learn quickly and behave non-deterministically. In this paper we present a novel method whereby an agent can incrementally learn to predict the actions of another agent (even a human), and thereby can learn to better interact with that agent. We take a case-based approach, where the behavior of the other agent is learned in the form of state-action pairs. We generalize these cases either through continuous k-nearest neighbor, or a modified bounded minimax search. Through our case studies, our technique is empirically shown to require little storage, learn very quickly, and be fast and robust in practice. It can accurately predict actions several steps into the future. Our case studies include interactive virtual environments involving mixtures of synthetic agents and humans, with cooperative and/or competitive relationships. Key words: autonomous agents, user modeling, agent modeling, action prediction, plan recognition. 2

This paper presents a theory for role playing simulation games intended to support analysts (and trainees) with generating and testing alternative competing hypotheses on how to influence world conflict situations. Simulated leaders and followers capable of playing these games are implemented in a cognitive modeling framework, called PMFserv, which covers value systems, personality and cultural factors, emotions, relationships, perception, stress/coping style and decision making. Of direct interest, as Section 1.1 explains, is codification and synthesis of best-of-breed social science models within PMFserv to improve the internal validity of the agent implementations. Sections 2 and 3 present this for leader profiling instruments and group membership decision-making, respectively. Section 4 then offers two real world case studies (The Third Crusade and SE Asia today) where the agent models are subjected to Turing and correspondence tests under each case study. The agent models are then used in a number of sensitivity and parameter elasticity studies. We observe the emergence of a ‘civil rights ’ demand curve that correlates with real world data about when followers will shift from phases of peaceful to vigorous protest to insurgency against a leader. In sum, substantial effort on game realism, best-of-breed social science models, and agent validation efforts is essential if analysis and training tools are to help explore cultural issues and alternative ways to influence outcomes. Such exercises, in turn, are likely to improve the state of the science as well.

Decentralised coordination in multi-agent systems is typically achieved using communication. However, in many cases, communication is expensive to utilise because there is limited bandwidth, it may be dangerous to communicate, or communication may simply be unavailable at times. In this context, we argue for a rational approach to communication — if it has a cost, the agents should be able to calculate a value of communicating. By doing this, the agents can balance the need to communicate with the cost of doing so. In this research, we present a novel model of rational communication, that uses reward shaping to value communications, and employ this valuation in decentralised POMDP policy generation. In this context, reward shaping is the process by which expectations over joint actions are adjusted based on how coordinated the agent team is. An empirical evaluation of the benefits of this approach is presented in two domains. First, in the context of an idealised benchmark problem, the multiagent Tiger problem, our method is shown to require significantly less communication (up to 30 % fewer messages) and still achieves a 30 % performance improvement over the current state of the art. Second, in the context of a larger-scale problem, RoboCupRescue, our method is shown to scale well, and operate without recourse to significant amounts of domain knowledge.