#### DMCA

## The agencies method for coalition formation in experimental games

### BibTeX

@MISC{Jr_theagencies,

author = {John F Nash Jr and Rosemarie Nagel and Axel Ockenfels and Reinhard Selten},

title = {The agencies method for coalition formation in experimental games},

year = {}

}

### OpenURL

### Abstract

In society, power is often transferred to another person or group. A previous work studied the evolution of cooperation among robot players through a coalition formation game with a noncooperative procedure of acceptance of an agency of another player. Motivated by this previous work, we conduct a laboratory experiment on finitely repeated three-person coalition formation games. Human players with different strength according to the coalition payoffs can accept a transfer of power to another player, the agent, who then distributes the coalition payoffs. We find that the agencies method for coalition formation is quite successful in promoting efficiency. However, the agent faces a tension between short-term incentives of not equally distributing the coalition payoff and the long-term concern to keep cooperation going. In a given round, the strong player in our experiment often resolves this tension approximately in line with the Shapley value and the nucleolus. Yet aggregated over all rounds, the payoff differences between players are rather small, and the equal division of payoffs predicts about 80% of all groups best. One reason is that the voting procedure appears to induce a balance of power, independent of the individual player's strength: Selfish subjects tend to be voted out of their agency and are further disciplined by reciprocal behaviors. leader | rules | reciprocity | fairness T he evolution of human altruism and cooperation is a puzzle. Unlike other animals, people frequently cooperate even absent of any material or reputational incentive to do so. In this paper we show how a voting procedure to transfer power to another person successfully promotes cooperation by balancing the tension between short-term incentives to defect and long-term incentives to keep cooperation going. Our work is inspired by John Nash (1), who theoretically studied the evolution of cooperation among robot players through acceptance of an agency of another player. Beyond Nash's (1) work, there is virtually no work on the agencies method in (experimental) economics as we apply it in our paper. The underlying idea is simple and important: Human subjects can transfer the power to an agency, who determines the final payoff distribution within the group.* Our game reflects that, often, efficiency requires people's willingness to accept the agency of others, such as political, social, or economic leaders (for voting of an expert, see ref. 7). In Nash's (1) work, the robots employed optimal strategies, being the computational result of complex systems of equations. Motivated by Nash's paper, we study laboratory three-person coalition formation games with a non-cooperative procedure of acceptance of an agency of another player. The base games are finitely repeated for 40 rounds with the same three subjects, allowing cooperation and coordination to evolve. In our games non-cooperative game theory cannot organize behavior because it is basically consistent with any outcome. Thus, even the strategies of fully rational agents cannot be predicted by the theory. † We show, however, that the solution concepts of cooperative game theory together with the equal split solution provide some structure on the emergence of cooperation in our experiment. Yet understanding how cooperation is affected by decisions to transfer power to others requires theories that go beyond these approaches. More specifically, our model specifies the coalition formation process in extensive form (for more details see Methods and For a long time the focus of attention in the analysis of coalition formation has been cooperative game theory. One underlying idea of cooperative game theory is that there are no restrictions on how agreements can be reached among players. The coalition formation process, making offers and counteroffers, can thus remain largely unspecified. In our cooperative games it is to the joint benefit of the group to form the grand coalition. Yet, which Author contributions: J.F.N., R.N., A.O., and R.S. designed research; R.N. and A.O. performed research; R.N., A.O., and R.S. analyzed data; J.F.N., R.N., A.O., and R.S. contributed to the theory and the supporting material; and J.F.N., R.N., A.O., and R.S. wrote the paper. The authors declare no conflict of interest. Freely available online through the PNAS open access option. Data deposition: The data reported in this paper are available in Dataset S1. 1 To whom correspondence should be addressed. E-mail: xkjfnj@princeton.edu. This article contains supporting information online at www.pnas.org/lookup/suppl/doi:10. 1073/pnas.1216361109/-/DCSupplemental. *Unlike in our case, where there is no precommitment to a particular policy prior to voting, most models in the theoretical voting literature assume that candidates are fully committed to their campaign policy proposals. Thus, by assumption, when a candidate is elected, he implements the policies that he promised to his constituency during the campaign. There are a few papers that drop the assumption of full commitment and analyze the strategic policy choice of candidates after they are elected. In one-shot elections the only possible outcome is the implementation of the most preferred policy of the winning candidate (2). In repeated elections, the value of reputation allows candidates to make credible policy proposals in equilibrium (3). In the experimental literature an important exception is ref. Loosely speaking, divisions of the total return are called points of the core if they are stable in the sense that no coalition should have the desire and power to upset the agreement. If, e.g., a coalition of two members is assigned a smaller total payoff in the grand coalition proposal than what the two members coalition can achieve alone, this proposal is inherently instable and thus not in the core. However, one problem is that the core can consist of many points without distinguishing a preferred point, or it may even be empty. The Shapley value (22), on the other hand, assigns to each player a unique payoff ("value"), which may be interpreted as a measure of power of the respective player in the game. One way of arriving at the Shapley value is to suppose that the grand coalition is formed by each player entering into this coalition one by one. As each player enters, he receives a payoff equal to his marginal contribution to the grand coalition payoff. This contribution generally depends on the entering order. The Shapley value is the average payoff to the players if they enter in a random order. ‡ The nucleolus We discuss the performance of these solution concepts for our games in the next section (see Built on Nash's seminal papers in 1950 (24), since the 1980s there have been many attempts to understand coalition formation and the distributions of payoffs as equilibria of non-cooperative games (25-27 and references therein). Starting in the field of industrial organization, extensive game models of oligopolistic competition and the analysis of their subgame perfect equilibria (28, 29) turned out to be a fruitful approach. Harsanyi's theory of incomplete information (30) opened further opportunities for non-cooperative game models. Somewhat later non-cooperative game modeling spread to many other fields of economic theory and much less attention was paid to cooperative games. There are exceptions. One is the "Gale-Shapley algorithm" (31) that rather recently turned out to be useful in practical market design; Roth (32) surveys the literature. However, the earliest attempt to develop non-cooperative modeling of cooperation and bargaining was by Nash (33) on two-person bargaining. There he not only presented his axiomatic theory but also offered a non-cooperative interpretation. Both Nash's model (1) and our agencies method ultimately build on this non-cooperative approach. A description of a strategic situation as a non-cooperative game is much more detailed than for a cooperative game. For experimental purposes an extensive game procedure for coalition bargaining, as we have devised it, has the advantage that the players interact in a formal and anonymous way. Thereby one isolates the strategic situation from social influences like personal sympathies and easily protocols every decision. However, any coalition (including no coalition with zero payoffs for all) can be supported in a pure equilibrium of our base game. The final agent who can be any of the three players takes the entire coalition value. When repeating the base game, as in our experiment, non-cooperative game theory imposes even less structure on behavior and outcomes: In the supergame almost any payoff division can be chosen in equilibrium, supported by a threat to convert to a one-shot base game equilibrium with no acceptances of any player in case of a deviation from the equilibrium path (SI Text A). Thus, it is not possible to derive predictions from noncooperative game theory. [The complexity involved in analyzing supergames was first emphasized by Nash in the context of the theoretically much less demanding repeated prisoner's dilemma (34) and has been confirmed in the laboratory (35-37).] We find that the agencies method for coalition formation is an effective mechanism to promote efficient cooperation and balanced payoffs. In particular, we observe that even though the players' strengths differ, long-run payoffs aggregated over all rounds tend to converge to the equal division. This is consistent with Nash's (1) simulations with robots in a similar context, as well as with parts of the behavioral economics literature, indicating a general attraction for payoff equality in bargaining and cooperation games, especially when payoff comparisons between players are possible. § However, equality is not generally the leading principle in each round separately. Here, many agencies seem to succumb to short-term incentives and allocate a significantly larger portion of the payoffs to themselves. Yet, they generally resist taking the whole surplus, as would have been predicted by non-cooperative game theory for the one-shot version of the game. Rather, in the short run, many agents appear to be guided by strength comparisons as captured by the Shapley value and the nucleolus. So, although there is inequality at many snapshots, some of which is organized by cooperative game theory, the repeated and symmetric voting procedure makes sure that, ultimately, In columns 2, 3, and 4 we state for each of the 10 games the values for the twoperson coalitions, V(XY) = V(AB), V(AC), and V(BC), respectively, of the characteristic function; the three-person coalition (grand coalition) is always 120, and the one-person coalition is normalized to 0. Columns 5, 6, and 7 present the theoretical payoffs for players A, B, and C, respectively, according to nucleolus, and columns 8, 9, and 10 present those for the Shapley value in a one-shot cooperative game. These theoretical payoffs are always distributions of the grand coalition value. Player A is the strong player in all games, because with him the highest twoperson coalition payoffs can be achieved compared to a coalition without him. For a similar reason B is the second strongest player. In all theoretical cooperative solutions player A receives always the highest payoff, followed by player B. *Games where no core exists. † Core has unique solution equal to nucleolus. ‡ Core is multivalued with equal split outside the core. § Core is multivalued with equal split inside the core; for core area, see Nash et al. PNAS | December 11, 2012 | vol. 109 | no. 50 | 20359 ECONOMIC SCIENCES everybody is taken care of equally-even to the extent that it mitigates strong differences in the subjects' strengths. Experimental Results Each of the 10 games in Coalition Building. Overall, cooperation is very successful. Only 1% of all 4,000 rounds end in no agreement, 7.5% in two-person coalitions, and 91.5% in the grand coalition. { The high level of efficiency appears to be only to a small extent the result of the low exogenously given probability of conflict (10%) in case of an impasse, because the random mechanism (see ║ However, although player A has some advantage,** in no group is the same player always the final agent; in 14/100 groups, the same player (A in 8 of these groups) is the final agent 80% or more of the time. In 50 groups the same player (A in 27 of these groups) becomes the final agent between 50% and 80% of the time. We conclude that the voting procedure does not strongly and consistently discriminate according to the players' strengths. Rather, the symmetric, procedural aspect of coalition formation in our game gives all players a chance in creating and leading the grand coalition, whether weak or strong. Payoff Distribution. Equal splits. The equal split in the grand coalition has considerable attraction. We observe it in 54% of the 4,000 rounds. (As a contrast, in 3% we observe that a player takes all.) † † For the remaining 71 groups Payoff distributions in nonequal-split groups. Whereas non-cooperative game theory is consistent with basically any payoff distribution in our games, cooperative game theory predictions are not. Our results demonstrate that some solution concepts from cooperative game theory can structure the data of the 71 nonhomogeneous groups in some (qualitative) ways. We concentrate on three concepts, the Shapley value, the nucleolus, and the core. We also consider the equal split. The average payoff distributions across all rounds are typically closest to the equal split (72%, 51 groups), whereas 25% (18 groups) distribute near the Shapley value and 3% (2 groups) near the nucleolus ‡ ‡ The core distribution has predictive power only in game 10 with all theoretical solutions, the equal split, and all data points lying in the core. No actual payoff distribution of the other games lies, however, in the core. None of the payoff splits are close to the corners of the triangles, the selfish splits. On a more qualitative level, we find that in 37% of all groups the payoffs are ordered as suggested by the cooperative solutions concepts, with player A earning the highest payoff and C the lowest (a random order would produce 17% of this particular order). In 24% of all groups B receives the highest payoff and in 7%, C. Further, in 32% A receives the highest payoff, but the order is reversed between B and C or one of these gets the same as A. Thus there is a significant difference between the payoff ordering between the three players, using the Friedman test (P < 0.001), with each group as an independent observation. So far we have compared aggregate payoff distributions over all rounds with cooperative concepts and the equal split. However, it is also interesting to compare the theoretical concepts with the average proposal of the final agent. In all rounds the final agent either splits the coalition payoff in his favor or uses an equal split. Thus, the proposals of players B and C are inconsistent with the strengths of the players. Therefore it makes sense to compare only the proposal of the strong player, A, with the theoretical concepts. Summing up, overall payoff differences mirror differences in the players' strengths and are thus qualitatively captured by cooperative concepts. In particular, focusing on the divisions by the strong player, outcomes are better organized by the Shapley For each group we count how many times across all 40 rounds the final agreement is according to the equal split. We observe that about 30% of all groups use equal split in more than 35 rounds (righthand column), whereas almost 15% use this rule in less than 4 rounds (lefthand column). { There is no significant difference between the frequency of two-person coalitions in games 1-4 (146 of the 301 rounds with two-person coalitions of all games), with the two-person coalition implying full efficiency (v(A, B) = 120), and the frequency in games 5-10 (151 of 301), when full efficiency is reached only in the grand coalition (using the Mann-Whitney U test, using each individual group as an independent observation, P > 0.12). In 32% of all 100 groups there are never two-person coalitions, and in 18% it happens only once per group. ║ Recall, however, that non-cooperative game theory predicts that, in every pure equilibrium of the base game where one player is voted for, this player will be voted for by either one or two players but the voted player will never vote for another player. In our supergame, we find that players A do not vote in 40% of the cases when they are voted for by either or both of the other players, whereas players B and C do not vote in 33% and 30% of the cases, respectively, with no significant differences between the players (sign tests based on group level with P = 0.38, 0.31, and 0.18 for the three comparisons). **Over all groups, player A is most often the final agent in 42% of all rounds [16.7 rounds (SD = 10.4)], player B in 32% [12.8 rounds (SD = 9.1)], and player C in 25% [10.1 rounds (SD = 8.6)], and in 1% no coalition is formed. Rank ordering A, B, and C players in a group by number of being representative, there is a significant difference between the three players (Friedman's test, two-way analysis on ranks, P < 0.01). † † Twenty-three of these 29 groups choose the equal split from the very beginning, whereas only 6 groups manage to converge to the equal-split norm when the first three rounds are not equal-split proposals (see ref. 40 for how norms may emerge in competitive environments). However, even when the equal split is the start-off norm, 20 groups fail to maintain it throughout. ‡ ‡ Our measure of success for a particular prediction is the mean squared error (MSE) between the payoff vector of the coalition formation solution and the actual average payoff vector of a group. The theoretical concept having the smallest MSE to the actual average data best describes a particular group on average. 20360 | www.pnas.org/cgi/doi/10.1073/pnas.1216361109 Nash et al. value than by equality and selfishness. [Kahan and Rapoport (41) and references therein summarize many of the experiments that competitively test several solution concepts; interestingly, in this literature, the Shapley value is generally not supported.] Nucleolus also organizes a significant share of observations well. The core does not have predictive value. However, over all rounds the equal split outperforms other principles of behavior. Reciprocity. How can average payoff vectors chosen by A be successfully organized by the Shapley value and, partly, the nucleolus, whereas equality is the dominant principle across all rounds and players? Before starting the experiment, our hypotheses were guided by the simulation in Nash (1) that cooperation can emerge only when no demand is selfish or when a selfish demand is matched with forgiving play. As mentioned above, 29 groups of the 100 groups showed equal splits across most rounds. Yet, in the remaining 71 groups our voting procedure produces just the opposite pattern: The more aggressive the demand of one player is, the more aggressive are those of the others. This kind of reciprocity is possible, because bargaining strength is offset by the voting procedure as shown below. The reciprocal relationship between gifts and demands as revealed by the correlations shows that payoff mitigation is made possible through a "fair" voting mechanism that disciplines too selfish demands. In particular, Summing up, whereas the strong player's behavior is better organized by the Shapley value and, partly, by the nucleolus, reciprocity explains the strong prominence of the equal split in the aggregate. The three players mimic each other, so that both gifts and demands are highly correlated between players. Summary and Conclusion The agencies method by Nash (1) is very effective in promoting human cooperation and fair outcomes: Full efficiency is almost always reached in our laboratory coalition formation game, and the divisions of payoffs across rounds are much less extreme than one might expect from a non-cooperative analysis of the base game. The tension between short-term incentives of not sharing the coalition value with others and the long-term concern to keep cooperation going is, by the strong player, often solved approximately in line with the Shapley value and the nucleolus. Also, the players' average payoff differences reflect the different strengths of players as measured by these concepts. However, over all rounds the payoff differences are rather small, and the equal division is the concept best describing 80% of all average payoff vectors. One reason is that the symmetry of the voting procedure induces a balance of power: Selfish agents tend to be voted out of their agency and are disciplined by reciprocal behavior. In fact, all players have a good chance to become the final agent. As a result, even if the short-run round payoffs are dispersed, long-run average payoffs tend to converge. Simplex for each game: Average payoff distribution for each group and game (stars), when A is agent and in non-equal-split groups (equal split in less than 36 rounds), and theoretical cooperative solutions (nucleolus, triangle; Shapley value, square; core area, in yellow; and equal split, circle). For all non-equal-split groups, we compute the average payoff distribution when player A is the agent for each group to compare his proposal with the theoretical concepts. The Shapley value and nucleolus assign the highest payoff to the strong player A. Furthermore, we plot the core, which is an area theory, allowing for several different splits (not in games 1-5, when the core does not exist, and in game 6 where the core is identical to the Shapley value). The equal split is the center of the triangle as it gives the same payoff to all members of a group. Because every agent within a group assigns himself at least the equal-split payoff, it makes no sense to compare the theoretical outcomes to proposals of B and C as these would be closest to equal split. PNAS | December 11, 2012 | vol. 109 | no. 50 | 20361 ECONOMIC SCIENCES We use the non-cooperative approach to clearly define and control the coalition formation process. Yet non-cooperative theory does not structure the behavior as the base game solutions are inconsistent regarding final payoffs and voting behavior. This complements earlier research in one-shot characteristic function games, where a great number of different extensive game procedures have been employed [see, in particular, the work on demand commitment models (41-44)]. For instance, the noncooperative theoretical analysis of these procedures suggests that the results depend strongly on procedural details. In fact, however, human behavior depends less on such details than predicted. Humans often seem to analyze the situation more in the flavor of cooperative game theory, ignoring the strategic consequences of the specific procedures used (38). Similarly, in earlier work on repeated asymmetric cooperation games, behavior could not be explained by optimizing behavior but rather by fairness criteria and cooperative goals (37). The cooperative solution concepts, on the other hand, can help us organize the payoff division data, but they do not capture the effect of the underlying institutions and procedures. Whereas the strength of the players captures some of the average payoff differences when the strong player is in charge, voting and longrun distribution behavior was essentially independent of the characteristic function. Here, the repeated voting procedure, which gives all an equal weight when transferring power to an agency, leads to rather equal total payoffs. This mitigating effect of the voting procedure is not captured by theory. (The distribution of power across subjects in our experiments-as is generally the case in experimental economics-was random, which may also contribute to the attractiveness of the equal split.) We conclude that other approaches to modeling human cooperation and coalition formation are needed, models that take people's cognitive and motivational limits in dealing with institutions and other players seriously. In this connection, an interesting related experimental study is the "three-person cooperative game with no side payments" by Kalisch, Milnor, Nash, and Nering (4). This study is one of the first experimental economics studies of negotiation and characteristic function games. In one treatment (section IV of their paper), two players could vote for another player; yet a player attracting two votes could not choose the distribution but was automatically awarded 40 monetary units, whereas the other two lost 20 each (otherwise, all payoffs were zero). They observed, like we do, that in the long run players typically equalized payoffs. Sometimes this was accomplished by randomization and sometimes by sequential reciprocity ("if you vote for me, I'll vote for you"). In the same paper the authors suggested to investigate these two mitigating mechanisms in an asymmetric setting as a robustness check for their findings. Although our experiment differs in some other ways too, we implement asymmetric characteristic functions-and observe the same two basic mechanisms at work in the following sense. Randomizing can be interpreted as a fair procedure, because it equalizes expected payoffs in the (base) game, where a deterministic equal outcome is not feasible Given the repeated structure both in our study and in ref. 4, reciprocity comes in as an additional, dynamic balancing mechanism, further reinforcing the convergence of power and payoffs in the groups. This suggests that the interaction of "fair institutions," such as voting, randomization, and reciprocity might be a key ingredient of the evolution of cooperation. However, it is captured neither by cooperative nor by non-cooperative theory, and it has been rarely studied outside these two papers. (An exception is ref. S ta rt 1 E v e ry p la y e r a c c e p ts a t m o st o n e o th e r p la y e r. 2 Is th e re a n e lig ib le p a ir?