This paper describes a new multi-player computer game, Colored Trails (CT), which may be played by people, computers and heterogeneous groups. CT was designed to enable investigation of properties of decision-making strategies in multi-agent situations of varying complexity. The paper presents the results of an initial series of experiments of CT games in which agents ’ choices affected not only their own outcomes but also the outcomes of other agents. It compares the behavior of people with that of computer agents deploying a variety of decision-making strategies. The results align with behavioral economics studies in showing that people cooperate when they play and that factors of social dependency influence their levels of cooperation. Preliminary results indicate that people design agents to play strategies closer to game-theory predictions, yielding lower utility. Additional experiments show that such agents perform worse than agents designed to make choices that resemble human cooperative behavior. The paper describes challenges raised by these results for designers of agents, especially agents that need to operate in heterogeneous groups that include people. 1.

Automated negotiation agents capable of negotiating efficiently with people must deal with the fact that people are diverse in their behavior and each individual might negotiate in a different manner. Thus, automated agents must rely on a good opponent modeling component to model their counterpart and adapt their behavior to their partner. In this paper we present the KBAgent. TheKBAgent is an automated negotiator that negotiates with each person only once, and uses past negotiation sessions of others as a knowledge base for general opponent modeling. The database is used to extract the likelihood of acceptance and proposals that may be offered by the opposite side. Experiments conducted with people show that the KBAgent negotiates efficiently with people and even achieves better utility values than another automated negotiator, shown to be efficient in negotiations with people. Moreover, the KBAgent achieves significantly better agreements, in terms of individual utility, than the human counterparts playing the same role.

Information

This paper proposes an efficient agent for competing in Cliff Edge (CE) environments, such as sealed-bid auctions, dynamic pricing and the ultimatum game. The agent competes in one-shot CE interactions repeatedly, each time against a different human opponent, and its performance is evaluated based on all the interactions in which it participates. The agent, which learns the general pattern of the population’s behavior, does not apply any examples of previous interactions in the environment, neither of other competitors nor its own. We propose a generic approach which competes in different CE environments under the same configuration, with no knowledge about the specific rules of each environment. The underlying mechanism of the proposed agent is a new meta-algorithm, Deviated Virtual Learning (DVL), which extends existing methods to efficiently cope with environments comprising a large number of optional decisions at each decision point. Experiments comparing the performance of the proposed algorithm with algorithms taken from the literature, as well as another intuitive meta-algorithm, reveal a significant superiority of the former in average payoff and stability. In addition, the agent performed better than human competitors executing the same task.

In strategic multiagent decision making, it is often the case that a strategic reasoner must hold beliefs about other agents and use these beliefs to inform its decision making. The behavior thus produced by the reasoner involves an interaction between the reasoner’s beliefs about other agents and the reasoner’s own preferences. A significant challenge faced by model designers, therefore, is how to model such a reasoner’s behavior so that the reasoner’s preferences and beliefs can each be identified and distinguished from each other. In this paper, we introduce a model of strategic reasoning that allows us to distinguish between the reasoner’s utility function and the reasoner’s beliefs about another agent’s utility function as well as the reasoner’s beliefs about how that agent might interact with yet other agents. We show that our model is uniquely identifiable. That is, no two different parameter settings will cause the model to give the same behavior over all possible inputs. We then illustrate the performance of our model in a multiagent negotiation game played by human subjects. We find that our subjects have slightly incorrect beliefs about other agents in the game.

This paper investigates the problem of determining when a computer agent should interrupt a person with whom it is working collaboratively as part of a distributed, multi-agent team, which is operating in environments in which conditions may be rapidly changing, actions occur at a fast pace, and decisions must be made within tightly constrained time frames. An interruption would enable the agent to obtain information useful for performing its role in the team task, but the person will incur a cost in responding. The paper presents a formalization of interruptions as multi-agent decision making. It defines a novel, efficient approximation method that decouples the multi-agent decision model into separate MDPs, thereby overcoming the complexity of finding optimal solutions of the Dec-POMDP model. For singleshot situations, the separate outcomes can be combined to give an exact value for the interruption. In more general settings, the closeness of the approximation to the optimal solution depends on the structure of the problem. The paper describes domain specific heuristic functions that improve the efficiency of the approximation further for a specific application. 1.

Computer systems increasingly carry out tasks in mixed networks, that is in group settings in which they interact both with other computer systems and with people. Participants in these heterogeneous human-computer groups vary in their capabilities, goals, and strategies; they may cooperate, collaborate, or compete. The presence of people in mixed networks raises challenges for the design and the evaluation of decision-making strategies for computer agents. This paper describes several new decision-making models that represent, learn and adapt to various social attributes that influence people’s decision-making and presents a novel approach to evaluating such models. It identifies a range of social attributes in an open-network setting that influence people’s decision-making and thus affect the performance of computeragent strategies, and establishes the importance of learning and adaptation to the success of such strategies. The settings vary in the capabilities, goals, and strategies that people bring into their interactions. The studies deploy a configurable system called Colored Trails (CT) that generates a family of games. CT is an abstract, conceptually simple but highly versatile game in which players negotiate and exchange resources to enable them to achieve their individual or group goals. It provides a realistic analogue to multi-agent task than payoff matrices, and people exhibit less strategic and more helpful behavior in CT than in the identical payoff matrix decision-making context. By not requiring extensive domain modeling, CT enables agent researchers to focus their attention on strategy design, and it provides an environment in which the influence of social factors can be better isolated and studied.

Abstract. We propose a machine learning approach to action prediction in oneshot games. In contrast to the huge literature on learning in games where an agent’s model is deduced from its previous actions in a multi-stage game, we propose the idea of inferring correlations between agents ’ actions in different one-shot games in order to predict an agent’s action in a game which she did not play yet. We define the approach and show, using real data obtained in experiments with human subjects, the feasibility of this approach. Furthermore, we demonstrate that this method can be used to increase payoffs of an adequately informed agent. This is, to the best of our knowledge, the first proposed and tested approach for learning in one-shot games, which is the most basic form of multiagent interaction. 1

Traditional game-theoretic analysis for decision-making takes a normative approach, in which agents derive rational decisions from the game description. This approach cannot naturally and compactly capture agents that are uncertain about the structure of the game, the strategies of other agents or whether agents may deviate from their optimal strategy. This paper presents Networks of Influence Diagrams (NID), a compact, natural and highly expressive language for reasoning about agents ’ beliefs and decision-making processes. NIDs are graphical structures in which agents ’ mental models are represented as nodes in a network; a mental model for an agent may itself use descriptions of the mental models of other agents. NIDs are demonstrated by examples, showing how they can be used to describe conflicting and cyclic belief structures, and certain forms of bounded rationality. In an opponent modeling domain, NIDs were able to outperform other computational agents whose strategies were not known in advance. A novel equilibrium concept is defined that makes a distinction between agents ’ optimal strategies, and how they actually behave in reality. It is also shown that NIDs are more compact and structured than Bayesian games, the traditional formalism used to model uncertainty in multi-agent decision-making problems.

This paper reports the results of an empirical investigation of the ways in which task-dependencies and inter-personal relationships influence the social preferences and outcomes of two-party negotiations. The investigation used a game, Colored Trails, configured for two-players in an ultimatum-game-like arrangement, but with more task context. It varied the player(s) who needed assistance and a friend-stranger relationship between the two players. The results indicate that friends play the game differently from strangers; player-dependence status affects some outcomes, but not all; and, therefore there is a need to explore additional potential influencers of behavior in negotiation. 1