In this paper we consider dynamic processes, in repeated games, that are subject to the natural informational restriction of uncoupledness. We study the almost sure convergence of play (the period-byperiod behavior as well as the long-run frequency) to Nash equilibria of the one-shot stage game, and present a number of possibility and impossibility results. Basically, we show that if in addition to random experimentation some recall, or memory, is introduced, then successful search procedures that are uncoupled can be devised. In particular, to get almost sure convergence to pure Nash equilibria when these exist, it suffices to recall the last two periods of play.

We construct an uncoupled randomized strategy of repeated play such that, if every player plays according to it, mixed action profiles converge almost surely to a Nash equilibrium of the stage game. The strategy requires very little in terms of information about the game, as players ’ actions are based only on their own past payoffs. Moreover, in a variant of the procedure, players need not know that there are other players in the game and that payoffs are determined through other players ’ actions. The procedure works for finite generic games and is based on appropriate modifications of a simple stochastic learning rule introduced by Foster and Young [12]. Keywords Regret testing; Regret-based learning; Random search; Stochastic dynamics; Uncoupled dynamics; Global convergence to

A general class of adaptive processes in games is developed, which significantly generalises weakened fictitious play [Van der Genugten, B., 2000. A weakened form of fictitious play in two-person zero-sum games. Int. Game Theory Rev. 2, 307–328] and includes several interesting fictitious-play-like processes as special cases. The general model is rigorously analysed using the best response differential inclusion, and shown to converge in games with the fictitious play property. Furthermore, a new actor–critic process is introduced, in which the only information given to a player is the reward received as a result of selecting an action—a player need not even know they are playing a game. It is shown that this results in a generalised weakened fictitious play process, and can therefore be considered as a first step towards explaining how players might learn to play Nash equilibrium strategies without having any knowledge of the game, or even that they are playing a game.

Interactive learning is inherently more complex than single-agent learning, because the act of learning changes the object to be learned. If agent A is trying to learn about agent B, A’s behavior will naturally depend on what she has learned so far, and also on what she hopes to learn next. But A’s behavior can be observed by B, hence B’s behavior may change as a result of A’s attempts to learn it. The same holds for B’s attempts to learn about A. This feedback loop is a central and inescapable feature of multi-agent learning situations. It suggests that methods which work for single-agent learning problems may fail in multi-agent settings. It even suggests that learning could fail in general, that is, there may exist situations in which no rules allow players to learn one another’s behavior in a completely satisfactory sense. This turns out to be the case: in the next section I formulate an uncertainty principle for strategic interactions which states that if there is enough ex ante uncertainty about the other players ’ payoffs (and therefore their potential behaviors), there is no way that rational players can learn to predict one another’s behavior, even over an

Young's regret testing

Learning by reinforcement ➟ ➠ Other learning models with low rationality ➟ ➠ Fictitious play and best-response dynamics ➟ ➠ The experience-weighted attraction learning model ➟ ➠