Abstract

Coordination of agent activities is a key problem in multiagent systems. Set in a larger decision theoretic context, the existence of coordination problems leads to difficulty in evaluating the utility of a situation. This in turn makes defining optimal policies for sequential decision processes problematic. We propose a method for solving sequential multiagent decision problems by allowing agents to reason explicitly about specific coordination mechanisms. We define an extension of value iteration in which the system’s state space is augmented with the state of the coordination mechanism adopted, allowing agents to reason about the short and long term prospects for coordination, the long term consequences of (mis)coordination, and make decisions to engage or avoid coordination problems based on expected value. We also illustrate the benefits of mechanism generalization. 1

Citations

878	Markov Decision Processes: Discrete Stochastic Dynamic Programming - Puterman - 1994
561	The Theory of Learning in Games - Fudenberg, Levine - 1998
417	Markov games as a framework for multi-agent reinforcement learning - Littman - 1994
278	Convention: A Philosophical Study - Lewis - 1969
170	Rational Learning Leads to Nash Equilibrium - Kalai, Lehrer - 1993
159	Stochastic games - Shapley - 1953
158	On the synthesis of useful social laws for artificial agent societies - Shoham, Tennenholtz - 1992
133	Decision theoretic planning: Structural assumptions and computational leverage - Boutilier, Dean, et al. - 1999
107	Iterative Solution of Games by Fictitious Play - Brown - 1951
64	Fictitious play property for games with identical interests - Monderer, Shapley - 1996
50	Decentralized supervisory control of discrete event systems - Lin, Wonham - 1988
39	Steady State Learning and Nash Equilibrium - FUDENBERG, LEVINE - 1993
19	Learning to coordinate actions in multi-agent systems - Wei - 1993
18	Learning Conventions in Multiagent Stochastic Domains using Likelihood Estimates - Boutilier - 1996
14	Learning to coordinate actions in multi-agent systems - Weiß - 1993
3	A General Equilibrium Selection in Games - Harsanyi, Reinhard - 1988