Coordination, as the act of managing interdependencies between activities, is one of the central research issues in Distributed Artificial Intelligence. Many researchers have shown that there is no single best organization or coordination mechanism for all environments. Problems in coordinating the activities of distributed intelligent agents appear in many domains: the control of distributed sensor networks; multi-agent scheduling of people and/or machines; distributed diagnosis of errors in local-area or telephone networks; concurrent engineering; `software agents' for information gathering. The design of coordination mechanisms for group...

This paper extends the framework of partially observable Markov decision processes (POMDPs) to multi-agent settings by incorporating the notion of agent models into the state space. Agents maintain beliefs over physical states of the environment and over models of other agents, and they use Bayesian update to maintain their beliefs over time. The solutions map belief states to actions. Models of other agents may include their belief states and are related to agent types considered in games of incomplete information. We express the agents ’ autonomy by postulating that their models are not directly manipulable or observable by other agents. We show that important properties of POMDPs, such as convergence of value iteration, the rate of convergence, and piece-wise linearity and convexity of the value functions carry over to our framework. Our approach complements a more traditional approach to interactive settings which uses Nash equilibria as a solution paradigm. We seek to avoid some of the drawbacks of equilibria which may be non-unique and are not able to capture off-equilibrium behaviors. We do so at the cost of having to represent, process and continually revise models of other agents. Since the agent’s beliefs may be arbitrarily nested the optimal solutions to decision making problems are only asymptotically computable. However, approximate belief updates and approximately optimal plans are computable. We illustrate our framework using a simple application domain, and we show examples of belief updates and value functions. 1.

A new concept of mutually expected rationality in noncooperative games is proposed: joint coherence. This is an extension of the “no arbitrage opportunities” axiom that underlies subjective probability theory and a variety of economic models. It sheds light on the controversy over the strategies that can reasonably be recommended to or expected to arise among Bayesian rational players. Joint coherence is shown to support Aumann’s position in favor of objective correlated equilibrium, although the common prior assumption is weakened and viewed as a theorem rather than an axiom. An elementary proof of the existence of correlated equilibria is given, and relationships with other solution concepts (Nash equilibrium, independent and correlated rationalizability) are also discussed.

We address the issue of rational communicative behavior among autonomous intelligent agents that have to make decisions as to what, to whom, and how to communicate. We treat communicative actions as aimed at increasing the efficiency of interaction among agents. We postulate that a rational speaker design a speech act so as to maximally increase the benefit obtained as the result of the interaction. We quantify the gain in the quality of interaction as the expected utility, and we present a framework that allows an agent to compute the expected utility of various communicative actions. Our framework uses the Recursive Modeling Method as the representation of the agent's state of knowledge, including the agent's preferences, abilities and beliefs about the world, as well as the beliefs the agent has about the other agents, the beliefs it has about the other agents ' beliefs, and so on. A decision-theoretic pragmatics of a communicative act can be then defined as the transformation it induces on the agent's state of knowledge about its decision-making situation. This transformation leads to a change in the quality of the interaction, expressed in terms of the benefit to the agent. We analyze decision-theoretic pragmatics of a number of important communicative acts, and investigate their expected utility using examples.

De Finetti's treatise on the theory of probability begins with the provocative statement PROBABILITY DOES NOT EXIST, meaning that probability does not exist in an objective sense. Rather, probability exists only subjectively within the minds of individuals. De Finetti defined subjective probabilities in terms of the rates at which individuals are willing to bet money on events, even though, in principle, such betting rates could depend on statedependent marginal utility for money as well as on beliefs. Most later authors, from Savage onward, have attempted to disentangle beliefs from values by introducing hypothetical bets whose payoffs are abstract consequences that are assumed to have state-independent utility. In this paper, I argue that de Finetti was right all along: PROBABILITY, considered as a numerical measure of pure belief uncontaminated by attitudes toward money, does not exist. Rather, what exist are de Finetti's "previsions," or betting rates for money, otherwise known in the literature as "risk neutral probabilities." But the fact that previsions are not measures of pure belief turns out not to be problematic for statistical inference, decision analysis, or economic modeling.

We adopt the decision-theoretic principle of expected utility maximization as a paradigm for designing autonomous rational agents, and present a framework that uses this paradigm to determine the choice of coordinated action. We endow an agent with a specialized representation that captures the agent's knowledge about the environment and about the other agents, including its knowledge about their states of knowledge, which can include what they know about the other agents, and so on. This reciprocity leads to a recursive nesting of models. Our framework puts forth a representation for the recursive models and, under the assumption that the nesting of models is finite, uses dynamic programming to solve this representation for the agent's rational choice of action. Using a decision-theoretic approach, our work addresses concerns of agent decision-making about coordinated action in unpredictable situations, without imposing upon agents pre-designed prescriptions, or protocols, ...

. Drawing on our work in the area of distributed artificial intelligence, we put forth a framework for modeling a human user interacting with a knowledge-based system. We assume that the human user is situated in some decision making setting, and view the computer system as taking an active role in supporting the user's decision making and problem solving activities. The model the system has of the decision making situation and of the user can be applied to determine what the system should do, both in terms of the system's physical action, if such is possible, as well as in terms of the information that should be transmitted to the user. An important part of the user's model is the model that the user may have of the system itself, and, further, how the user may think it is being modeled by the system. Our framework, the Recursive Modeling Method (RMM), explicitly represents this nesting of models, and lets the system to coordinate with the expected actions of the human user, and to r...

We analyze the asymptotic behavior of agents engaged in an infinite horizon partially observable stochastic game as formalized by the interactive POMDP framework. We show that when agents' initial beliefs satisfy a truth compatibility condition, their behavior converges to a subjective #-equilibrium in a finite time, and subjective equilibrium in the limit. This result is a generalization of a similar result in repeated games, to partially observable stochastic games. However, it turns out that the equilibrating process is difficult to demonstrate computationally because of the difficulty in coming up with initial beliefs that are both natural and satisfy the truth compatibility condition. Our results, therefore, shed some negative light on using equilibria as a solution concept for decision making in partially observable stochastic games.

Conventional noncooperative game theory hypothesizes that the joint (mixed) strategy of a set of reasoning players in a game will necessarily satisfy an “equilibrium concept”. All other joint strategies are considered impossible. Moroever, often the number of joint strategies satisfying that equilibrium concept has measure zero. (Indeed, this is often considered a desirable property of an equilibrium concept.) Under this hypothesis the only issue is what equilibrium concept is “correct”. This hypothesis violates the first-principles arguments underlying probability theory. Indeed, probability theory renders moot the controversy over what equilibrium concept is correct — while in general there are joint (mixed) strategies with zero probability, in general the set {strategies with non-zero probability} has measure greater than zero. Rather than a firstprinciples derivation of an equilibrium concept, game theory requires a first-principles derivation of a distribution over joint strategies.

This paper focuses on the foundational role of interactive epistemology in the problem of generating plans for rational agents in multiagent settings. Interactive epistemology deals with the logic of knowledge and belief when there is more than one agent. In multiagent settings, we are interested in not only the agent’s knowledge of the state of the world, but also its belief over the other agents ’ beliefs and their beliefs over others’. We adopt a probabilistic approach for formalizing the epistemology. This paper attempts to answer the question of why we should study the interactive epistemology of agents within the context of multiagent planning. In doing so, it motivates the need for a more detailed examination of the epistemological foundations of multiagent planning. We conclude this paper with a framework for multiagent planning that explicitly constructs and reasons with nested belief structures.