Abstract There has been substantial work developing simple, efficient no-regret algorithms for a wideclass of repeated decision-making problems including online routing. These are adaptive strategies an individual can use that give strong guarantees on performance even in adversarially-changing environments. There has also been substantial work on analyzing properties of Nash equilibria in routing games. In this paper, we consider the question: if each player in a rout-ing game uses a no-regret strategy, will behavior converge to a Nash equilibrium? In general games the answer to this question is known to be no in a strong sense, but routing games havesubstantially more structure. In this paper we show that in the Wardrop setting of multicommodity flow and infinitesimalagents, behavior will approach Nash equilibrium (formally, on most days, the cost of the flow will be close to the cost of the cheapest paths possible given that flow) at a rate that dependspolynomially on the players ' regret bounds and the maximum slope of any latency function. We also show that price-of-anarchy results may be applied to these approximate equilibria, and alsoconsider the finite-size (non-infinitesimal) load-balancing model of Azar [2].

Abstract—The problem of distributed learning and channel access is considered in a cognitive network with multiple secondary users. The availability statistics of the channels are initially unknown to the secondary users and are estimated using sensing decisions. There is no explicit information exchange or prior agreement among the secondary users. We propose policies for distributed learning and access which achieve order-optimal cognitive system throughput (number of successful secondary transmissions) under self play, i.e., when implemented at all the secondary users. Equivalently, our policies minimize the regret in distributed learning and access. We first consider the scenario when the number of secondary users is known to the policy, and prove that the total regret is logarithmic in the number of transmission slots. Our distributed learning and access policy achieves order-optimal regret by comparing to an asymptotic lower bound for regret under any uniformly-good learning and access policy. We then consider the case when the number of secondary users is fixed but unknown, and is estimated through feedback. We propose a policy in this scenario whose asymptotic sum regret which grows slightly faster than logarithmic in the number of transmission slots. Index Terms—Cognitive medium access control, multi-armed bandits, distributed algorithms, logarithmic regret. I.

Abstract: Many natural games can have a dramatic difference between the quality of their best and worst Nash equilibria, even in pure strategies. Yet, nearly all work to date on dynamics shows only convergence to some equilibrium, especially within a polynomial number of steps. In this work we study how agents with some knowledge of the game might be able to quickly (within a polynomial number of steps) find their way to states of quality close to the best equilibrium. We consider two natural learning models in which players choose between greedy behavior and following a proposed good but untrusted strategy and analyze two important classes of games in this context, fair cost-sharing and consensus games. Both games have extremely high Price of Anarchy and yet we show that behavior in these models can efficiently reach low-cost states. Keywords: Dynamics in Games, Price of Anarchy, Price of Stability, Cost-sharing games, Consensus games, Learning from untrusted experts

We analyze the performance of protocols for load balancing in distributed systems based on no-regret algorithms from online learning theory. These protocols treat load balancing as a repeated game and apply algorithms whose average performance over time is guaranteed to match or exceed the average performance of the best strategy in hindsight. Our approach captures two major aspects of distributed systems. First, in our setting of atomic load balancing, every single process can have a significant impact on the performance and behavior of the system. Furthermore, although in distributed systems participants can query the current state of the system they cannot reliably predict the effect of their actions on it. We address this issue by considering load balancing games in the bulletin board model, where players can find out the delay on all machines, but do not have information on what their experienced delay would have been if they had selected another machine. We show that under these more realistic assumptions, if all players use the wellknown multiplicative weights algorithm, then the quality of the resulting solution is exponentially better than the worst correlated equilibrium, and almost as good as that of the worst Nash. These tighter bounds are derived from analyzing the dynamics of a multi-agent learning system.

Price of anarchy and price of stability are the primary notions for measuring the efficiency (i.e. the social welfare) of the outcome of a game. Both of these notions focus on extreme cases: one is defined as the inefficiency ratio of the worst-case equilibrium and the other as the best one. Therefore, studying these notions often results in discovering equilibria that are not necessarily the most likely outcomes of the dynamics of selfish and non-coordinating agents. The current paper studies the inefficiency of the equilibria that are most stable in the presence of noise. In particular, we study two variations of non-cooperative games: atomic congestion games and selfish load balancing. The noisy best-response dynamics in these games keeps the joint action profile around a particular set of equilibria that minimize the potential function. The inefficiency ratio in the neighborhood of these “stable ” equilibria is much better than the price of anarchy. Furthermore, the dynamics reaches these equilibria in polynomial time. Our observations show that in the game environments where a small noise is present, the system as a whole works better than what a pessimist may predict. They also suggest that in congestion games, introducing a small noise in the payoff of the agents may improve the social welfare.

The central theme of my research is to explore the impact of combining learning algorithms and game theory. Game theory attempts to mathematically capture behavior in strategic situations, in which an individual’s success depends on the choices of others. In practice, the interacting entities may be numerous and entangled via complex networks of interdependencies. Over the last decade, the prevalence of these issues has risen dramatically following a number of paradigm-shifting events such as the cataclysmic rise of the Internet as a social networking tool, the painful realization of the extent of inter-connectivity of the global economy as well as the necessity of international cooperation for addressing global sustainability concerns. As a result, there has been recorded a swift increase of the interest for a more detailed, realistic and quantitative understanding of such networked interactions. Algorithmic game theory (AGT) employs analytic tools from computer science, such as worst case analysis and complexity theory, to characterize behavioral solutions to strategic situations prescribed by (classical) game theory. Research in this area tends to focus on one of the following challenges: • Price of Anarchy: Characterize the inefficiency of equilibria vs the global optimum • Algorithmic Mechanism Design: Design games with desirable properties that are efficiently implementable. • Computational Complexity of Equilibria Many strategic interactions are in their nature recurrent (e.g. financial markets) with the agents participating in them repeatedly. These agents learn over time to adapt to their environment as is defined by the game and the dynamic behavior of the other agents. Understanding how agents can learn in the presence of other agents that are simultaneously learning constitutes a research problem that is as expansive as it is challenging. Such questions have fueled research endeavors both in economics as well as within computer science (e.g. multi-agent learning). My research interests concentrate on the intersection of algorithmic game theory and learning. Incorporating the rather natural assumption that agents learn to adapt to their environment can lead to exciting new insights to long standing questions. For each subarea of AGT, I will present current results and directions for future research. My plan is to examine how far these implications reach.

Abstract—The problem of cooperative allocation among multiple secondary users to maximize cognitive system throughput is considered. The channel availability statistics are initially unknown to the secondary users and are learnt via sensing samples. Two distributed learning and allocation schemes which maximize the cognitive system throughput or equivalently minimize the total regret in distributed learning and allocation are proposed. The first scheme assumes minimal prior information in terms of pre-allocated ranks for secondary users while the second scheme is fully distributed and assumes no such prior information. The two schemes have sum regret which is provably logarithmic in the number of sensing time slots. A lower bound is derived for any learning scheme which is asymptotically logarithmic in the number of slots. Hence, our schemes achieve asymptotic order optimality in terms of regret in distributed learning and allocation. Index Terms—Cognitive medium access, learning, multi-armed bandits, logarithmic regret, distributed algorithms. I.

Recent developments in many domains have highlighted the complexity and interconnectedness of our economic systems. Over the past decade, the growth of global computing and information networks has enabled new kinds of markets — for on-line commerce, search advertising, and other applications — that are growing rapidly in societal importance, but in which there are

1.1. Motivation. Many modern computer science applications involve autonomous, self-interested agents. It is therefore important for us to consider agents ' strategic behavior in modelling the problems, where non-cooperative game theory can be very helpful. Unfortunately, as one can expect, strategic behavior of the agents often