Results 11  20
of
1,376
Multiagent Learning Using a Variable Learning Rate
 Artificial Intelligence
, 2002
"... Learning to act in a multiagent environment is a difficult problem since the normal definition of an optimal policy no longer applies. The optimal policy at any moment depends on the policies of the other agents and so creates a situation of learning a moving target. Previous learning algorithms hav ..."
Abstract

Cited by 180 (8 self)
 Add to MetaCart
Learning to act in a multiagent environment is a difficult problem since the normal definition of an optimal policy no longer applies. The optimal policy at any moment depends on the policies of the other agents and so creates a situation of learning a moving target. Previous learning algorithms have one of two shortcomings depending on their approach. They either converge to a policy that may not be optimal against the specific opponents' policies, or they may not converge at all. In this article we examine this learning problem in the framework of stochastic games. We look at a number of previous learning algorithms showing how they fail at one of the above criteria. We then contribute a new reinforcement learning technique using a variable learning rate to overcome these shortcomings. Specifically, we introduce the WoLF principle, "Win or Learn Fast", for varying the learning rate. We examine this technique theoretically, proving convergence in selfplay on a restricted class of iterated matrix games. We also present empirical results on a variety of more general stochastic games, in situations of selfplay and otherwise, demonstrating the wide applicability of this method.
Distributed Rational Decision Making
, 1999
"... Introduction Automated negotiation systems with selfinterested agents are becoming increasingly important. One reason for this is the technology push of a growing standardized communication infrastructureInternet, WWW, NII, EDI, KQML, FIPA, Concordia, Voyager, Odyssey, Telescript, Java, etco ..."
Abstract

Cited by 167 (0 self)
 Add to MetaCart
Introduction Automated negotiation systems with selfinterested agents are becoming increasingly important. One reason for this is the technology push of a growing standardized communication infrastructureInternet, WWW, NII, EDI, KQML, FIPA, Concordia, Voyager, Odyssey, Telescript, Java, etcover which separately designed agents belonging to different organizations can interact in an open environment in realtime and safely carry out transactions. The second reason is strong application pull for computer support for negotiation at the operative decision making level. For example, we are witnessing the advent of small transaction electronic commerce on the Internet for purchasing goods, information, and communication bandwidth [29]. There is also an industrial trend toward virtual enterprises: dynamic alliances of small, agile enterprises which together can take advantage of economies of scale when available (e.g., respond to mor
A Modal Logic for Coalitional Power in Games
, 2002
"... We present a modal logic for reasoning about what groups of agents can bring about by collective action. Given a set of states, we introduce game frames which associate with every state a strategic game among the agents. Game frames are essentially extensive games of perfect information with simulta ..."
Abstract

Cited by 136 (5 self)
 Add to MetaCart
We present a modal logic for reasoning about what groups of agents can bring about by collective action. Given a set of states, we introduce game frames which associate with every state a strategic game among the agents. Game frames are essentially extensive games of perfect information with simultaneous actions, where every action profile is associated with a new state, the outcome of the game. A coalition of players is effective for a set of states # in a game if the coalition can guarantee the outcome of the game to lie in # . We propose a modal logic (Coalition Logic) to formalize reasoning about effectivity in game frames, where #### expresses that coalition # is effective for #. An axiomatization is presented and completeness proved. Coalition Logic provides a unifying gametheoretic view of modal logic: Since nondeterministic processes and extensive games without parallel moves emerge as particular instances of game frames, normal and nonnormal modal logics correspond to 1 and 2player versions of Coalition Logic. The satisfiability problem for Coalition Logic is shown to be PSPACEcomplete.
Algorithms, Games, and the Internet
 In STOC
, 2001
"... If the Internet is the next great subject for Theoretical Computer Science to model and illuminate mathematically, then Game Theory, and Mathematical Economics more generally, are likely to prove useful tools. In this talk I survey some opportunities and challenges in this important frontier. 1. ..."
Abstract

Cited by 135 (0 self)
 Add to MetaCart
If the Internet is the next great subject for Theoretical Computer Science to model and illuminate mathematically, then Game Theory, and Mathematical Economics more generally, are likely to prove useful tools. In this talk I survey some opportunities and challenges in this important frontier. 1.
Learning to Cooperate via Policy Search
, 2000
"... Cooperative games are those in which both agents share the same payoff structure. Valuebased reinforcementlearning algorithms, such as variants of Qlearning, have been applied to learning cooperative games, but they only apply when the game state is completely observable to both agents. Poli ..."
Abstract

Cited by 131 (4 self)
 Add to MetaCart
Cooperative games are those in which both agents share the same payoff structure. Valuebased reinforcementlearning algorithms, such as variants of Qlearning, have been applied to learning cooperative games, but they only apply when the game state is completely observable to both agents. Policy search methods are a reasonable alternative to valuebased methods for partially observable environments. In this paper, we provide a gradientbased distributed policysearch method for cooperative games and compare the notion of local optimum to that of Nash equilibrium. We demonstrate the effectiveness of this method experimentally in a small, partially observable simulated soccer domain. 1 INTRODUCTION The interaction of decision makers who share an environment is traditionally studied in game theory and economics. The game theoretic formalism is very general, and analyzes the problem in terms of solution concepts such as Nash equilibrium [12], but usually works under the assu...
Competitive auctions and digital goods
 In Proc. 12th Symp. on Discrete Alg
, 2001
"... Abstract We study a class of single round, sealed bid auctions for items in unlimited supply such as digital goods. We focus on auctions that are truthful and competitive. Truthful auctions encourage bidders to bid their utility; competitive auctions yield revenue within a constant factor of the rev ..."
Abstract

Cited by 126 (27 self)
 Add to MetaCart
Abstract We study a class of single round, sealed bid auctions for items in unlimited supply such as digital goods. We focus on auctions that are truthful and competitive. Truthful auctions encourage bidders to bid their utility; competitive auctions yield revenue within a constant factor of the revenue for optimal fixed pricing. We show that for any truthful auction, even a multiprice auction, the expected revenue does not exceed that for optimal fixed pricing. We also give a bound on how far the revenue for optimal fixed pricing can be from the total market utility. We show that several randomized auctions are truthful and competitive under certain assumptions, and that no truthful deterministic auction is competitive. We present simulation results which confirm that our auctions compare favorably to fixed pricing. Some of our results extend to bounded supply markets, for which we also get truthful and competitive auctions.
Negotiation and cooperation in multiagent environments
 Artificial Intelligence
, 1997
"... Automated intelligent agents inhabiting a shared environmentmust coordinate their activities. Cooperation { not merely coordination { may improve the performance of the individual agents or the overall behavior of the system they form. Research in Distributed Arti cial Intelligence (DAI) addresses t ..."
Abstract

Cited by 123 (5 self)
 Add to MetaCart
Automated intelligent agents inhabiting a shared environmentmust coordinate their activities. Cooperation { not merely coordination { may improve the performance of the individual agents or the overall behavior of the system they form. Research in Distributed Arti cial Intelligence (DAI) addresses the problem of designing automated intelligent systems which interact e ectively. DAI is not the only eld to take on the challenge of understanding cooperation and coordination. There are a variety of other multientity environments in which the entities coordinate their activity and cooperate. Among them are groups of people, animals, particles, and computers. We argue that in order to address the challenge of building coordinated and collaborated intelligent agents, it is bene cial to combine AI techniques with methods and techniques from a range of multientity elds, such as game theory, operations research, physics and philosophy. To support this claim, we describe some of our projects, where we have successfully taken an interdisciplinary approach. We demonstrate the bene ts in applying multientity methodologies and show the adaptations, modi cations and extensions necessary for solving the DAI problems.
The Price of Anarchy of Finite Congestion Games
 In Proceedings of the 37th Annual ACM Symposium on Theory of Computing (STOC
, 2005
"... Abstract We consider the price of anarchy of pure Nash equilibria in congestion games with linearlatency functions. For asymmetric games, the price of anarchy of maximum social cost is \Theta (p N),where N is the number of players. For all other cases of symmetric or asymmetric games andfor both max ..."
Abstract

Cited by 122 (7 self)
 Add to MetaCart
Abstract We consider the price of anarchy of pure Nash equilibria in congestion games with linearlatency functions. For asymmetric games, the price of anarchy of maximum social cost is \Theta (p N),where N is the number of players. For all other cases of symmetric or asymmetric games andfor both maximum and average social cost, the price of anarchy is 5 /2. We extend the results tolatency functions that are polynomials of bounded degree. We also extend some of the results to mixed Nash equilibria.
Frugal path mechanisms
, 2002
"... We consider the problem of selecting a low cost s − t path in a graph, where the edge costs are a secret known only to the various economic agents who own them. To solve this problem, Nisan and Ronen applied the celebrated VickreyClarkeGroves (VCG) mechanism, which pays a premium to induce the edg ..."
Abstract

Cited by 115 (2 self)
 Add to MetaCart
We consider the problem of selecting a low cost s − t path in a graph, where the edge costs are a secret known only to the various economic agents who own them. To solve this problem, Nisan and Ronen applied the celebrated VickreyClarkeGroves (VCG) mechanism, which pays a premium to induce the edges to reveal their costs truthfully. We observe that this premium can be unacceptably high. There are simple instances where the mechanism pays Θ(k) times the actual cost of the path, even if there is an alternate path available that costs only (1 + ɛ) times as much. This inspires the frugal path problem, which is to design a mechanism that selects a path and induces truthful cost revelation without paying such a high premium. This paper contributes negative results on the frugal path problem. On two large classes of graphs, including ones having three nodedisjoint s − t paths, we prove that no reasonable mechanism can always avoid paying a high premium to induce truthtelling. In particular, we introduce a general class of min function mechanisms, and show that all min function mechanisms can be forced to overpay just as badly as VCG. On the other hand, we prove that (on two large classes of graphs) every truthful mechanism satisfying some reasonable properties is a min function mechanism. 1