## Rank Aggregation Methods for the Web (2001)

### Cached

### Download Links

- [www.www10.org]
- [www.eecs.harvard.edu]
- [www.cs.brown.edu]
- [cs.brown.edu]
- [www.inf.unibz.it]
- [cs.wellesley.edu]
- DBLP

### Other Repositories/Bibliography

Citations: | 344 - 6 self |

### BibTeX

@MISC{Dwork01rankaggregation,

author = {Cynthia Dwork and Ravi Kumar and Moni Naor and D. Sivakumar},

title = {Rank Aggregation Methods for the Web},

year = {2001}

}

### Years of Citing Articles

### OpenURL

### Abstract

We consider the problem of combining ranking results from various sources. In the context of the Web, the main applications include building meta-search engines, combining ranking functions, selecting documents based on multiple criteria, and improving search precision through word associations. Wedevelop a set of techniques for the rank aggregation problem and compare their performance to that of well-known methods. A primary goal of our work is to design rank aggregation techniques that can effectively combat "spam," a serious problem in Web searches. Experiments show that our methods are simple, efficient, and effective. Keywords: rank aggregation, ranking functions, metasearch, multi-word queries, spam 1.

### Citations

3573 | The anatomy of a large-scale hypertextual Web search engine
- Brin, Page
- 1998
(Show Context)
Citation Context ...can be trusted to perform well for all queries. A few years ago, query term frequency was the single main heuristic in ranking Web pages� since the in uential work of Kleinberg [16] and Brin and Page =-=[7]-=-, link analysis has come to be identied as a very powerful technique in ranking Web pages and other hyperlinked documents. Several other heuristics have been added, including anchor-text analysis [8],... |

2953 | Authoritative sources in a hyperlinked environment
- Kleinberg
- 1999
(Show Context)
Citation Context ...ingle ranking function can be trusted to perform well for all queries. A few years ago, query term frequency was the single main heuristic in ranking Web pages� since the in uential work of Kleinberg =-=[16]-=- and Brin and Page [7], link analysis has come to be identied as a very powerful technique in ranking Web pages and other hyperlinked documents. Several other heuristics have been added, including anc... |

430 | Improved algorithms for topic distillation in a hyperlinked environment
- Bharat, Henzinger
- 1998
(Show Context)
Citation Context ...ch engines. For measuring the performance of our methods ( rst experiment), we selected the following 38 general queries (these queries are a superset of the 28 queries used in several earlier papers =-=[4, 8]-=-). For the second experiment, we pick some queries that were spammed in popular search engines. For the third experiment, we pickmulti-word queries that perform poorly with existing search engines. Ou... |

356 | Combining Fuzzy Information from Multiple Systems
- Fagin
- 1996
(Show Context)
Citation Context ...bout multimedia documents, e.g. \ nd a document that has information about Greek islands with pictures of beautiful blue beaches." This is a problem conventionally studied in database middleware (see =-=[15]-=-). Several novel approaches have been invented for this purpose, but this problem cannot be considered well-solved by any measure. Naturally, these problems fall under the realm of rank aggregation. T... |

301 |
Group representations in probability and statistics
- Diaconis
- 1988
(Show Context)
Citation Context ...n all the elements in T , then jT is a full list with respect to T . 2.1.1 Distance measures How dowemeasure distance between two full lists with respect to a set S? Two popular distance measures are =-=[12]-=-: (1) The Spearman footrule distance is the sum, over all elements i 2 S, of the absolute di erence between the rank of i according to the two lists. Formally, given two full lists and , the distance ... |

111 |
Spearman’s Footrule as a Measure of Disarray
- Diaconis, Graham
- 1977
(Show Context)
Citation Context ...gregation is useful, but computationally hard, how dowe compute it? The following relation shows that Kendall distance can be approximated very well via the Spearman footrule distance. Proposition 1. =-=[13]-=- For any two full lists � , K( � ) F ( � ) 2K( � ). This leads us to the problem of footrule optimal aggregation. This is the same as before, except that the optimizing criterion is the footrule dista... |

100 | Approximating minimum feedback sets and multi-cuts in directed graphs - Even, Naor, et al. - 1998 |

97 |
Analyzing and Modeling Rank Data
- Marden
- 1995
(Show Context)
Citation Context ...res seem natural, why these measures are good is moot. We do not delve into such discussions here� the interested reader can nd such arguments in the books by Diaconis [12], Critchlow [11], or Marden =-=[17]-=-. 2.1.2 Optimal rank aggregation In the generic context of rank aggregation, the notion of `better' depends on what distance measure we strive tooptimize. Suppose we wish to optimize Kendall distance,... |

75 |
Condorcet’s Theory of Voting
- Young
- 1988
(Show Context)
Citation Context ...ts with some probability less than 1=2. Thus, the 's are \noisy" versions of . A Kemeny optimal aggregation of 1�:::� k is one that is maximally likely to have produced the 's (it need not be unique) =-=[24]-=-. Viewed di erently, Kemeny optimal aggregation has the property of eliminating noise from various di erent ranking schemes. Furthermore, Kemeny optimal aggregations are essentially the only ones that... |

74 |
M.emoire sur les elections au scrutin. Histoire de l'Academie Royale des Sciences
- Borda
(Show Context)
Citation Context ...� k, use your favorite aggregation method to obtain a full list . Output the (unique) local Kemenization of with respect to 1�:::� k. 6174. RANK AGGREGATION METHODS 4.1 Borda’s method Borda's method =-=[6]-=- is a \positional" method, in that it assigns a score corresponding to the positions in which acandidate appears within each voter's ranked list of preferences, and the candidates are sorted by their ... |

63 |
Metric methods for analyzing partially ranked data
- Critchlow
- 1985
(Show Context)
Citation Context ...e distance measures seem natural, why these measures are good is moot. We do not delve into such discussions here� the interested reader can nd such arguments in the books by Diaconis [12], Critchlow =-=[11]-=-, or Marden [17]. 2.1.2 Optimal rank aggregation In the generic context of rank aggregation, the notion of `better' depends on what distance measure we strive tooptimize. Suppose we wish to optimize K... |

49 |
A consistent extension of Condorcet’s election principle
- Young, Levenglick
- 1978
(Show Context)
Citation Context ...nes that simultaneously satisfy natural and important properties of rank aggregation functions, called neutrality and consistency in the social choice literature, and the so-called Condorcet property =-=[25]-=-. Indeed, Kemeny optimal 615aggregations satisfy the extended Condorcet criterion. In Section 3 we establish a strong connection between satisfaction of the extended Condorcet criterion and ghting se... |

37 |
Aggregation of Preferences with Variable Electorates, Econometrica 41
- Smith
- 1973
(Show Context)
Citation Context ...nowknown as the Condorcet alternative, that defeats every other in pairwise simple majority voting, then that this element should be ranked rst [9]. A natural extension, due to Truchon [22] (see also =-=[21]-=-), mandates that if there is a partition (C� C) of S such thatforanyx2Cand y 2 C the majority prefers x to y, then x must be ranked above y. This is called the extended Condorcet criterion (ECC). We w... |

37 |
An axiomatization of Borda’s rule
- Young
- 1974
(Show Context)
Citation Context ...ethods is that they are computationally very easy: they can be implemented in linear time. They also enjoy the properties called anonymity, neutrality, and consistency in the social choice literature =-=[23]-=-. However, they cannot satisfy the Condorcet criterion. In fact, it is possible to show that no method that assigns a weights to each position and then sorts the results by applying a function to the ... |

25 |
Automatic text processing—the transformation, analysis and retrieval of information by computer
- Salton
- 1989
(Show Context)
Citation Context ...nner that given a search term, those most relevant to the search term can be retrieved easily. This is a classic information retrieval problem and reasonably well-understood for static documents (see =-=[20]-=-). When the documents are hypertext documents, however, indexing algorithms could exploit the latent relationship between documents implied by thehyperlinks. On the Web, such an approach has already p... |

16 | An extension of the Condorcet Criterion and Kemeny Orders
- Truchon
- 1998
(Show Context)
Citation Context ...e element ofS, nowknown as the Condorcet alternative, that defeats every other in pairwise simple majority voting, then that this element should be ranked rst [9]. A natural extension, due to Truchon =-=[22]-=- (see also [21]), mandates that if there is a partition (C� C) of S such thatforanyx2Cand y 2 C the majority prefers x to y, then x must be ranked above y. This is called the extended Condorcet criter... |

15 |
A ‘reasonable’ social welfare function. mimeographed, University of Michigan Seminar on Applications of Mathematics to the social sciences
- Copeland
- 1951
(Show Context)
Citation Context ...y of the lists that ranked both P and Q, then go to Q, else stay inP . This chain generalizes Copeland's suggestion of sorting the candidates by the number of pairwise majority contests they have won =-=[10]-=-. There are examples that di erentiate the behavior of these chains. One can also show that the Markov ordering implied by these chains need not satisfy the extended Condorcet principle. 5. APPLICATIO... |

7 | Voting schemes for which it can be di cult to tell who won the election - Bartholdi, Tovey, et al. - 1989 |

7 |
Essai sur l’Application de l’Analyse a‘la Probabilite des Decisions Rendues a‘ la Pluralite des Voix, 1785 22 Duchacek I. D., Power Maps: comparative politics of constitutions, ABC-Clio
- Condorcet
- 1973
(Show Context)
Citation Context ... Condorcet, proposed that if there is some element ofS, nowknown as the Condorcet alternative, that defeats every other in pairwise simple majority voting, then that this element should be ranked rst =-=[9]-=-. A natural extension, due to Truchon [22] (see also [21]), mandates that if there is a partition (C� C) of S such thatforanyx2Cand y 2 C the majority prefers x to y, then x must be ranked above y. Th... |

5 |
The mathematics of voting: Democratic symmetry. The Economist
- Saari
- 2000
(Show Context)
Citation Context ... values, etc. This intuition leads us to several Markov chain based approaches, described in Section 4.3. Partial lists. It has been proposed (e.g., in a recent article that appeared in The Economist =-=[19]-=-) that the right way to extend Borda to partial lists is by apportioning all the excess scores equally among all unranked candidates. This idea stems from the goal of being unbiased� however, it is ea... |

1 |
Engine Watch article, www.searchenginewatch.com /sereport/00/11-inclusion.html
- Search
(Show Context)
Citation Context ...nd alliances, it is not clear that users have anyform of protection against the biases/interests of individual search engines. As a case in point, note that \paid placement" and \paid inclusion" (see =-=[2]-=-) appear to be gaining popularity among search engines. In some cases, individual ranking functions are inadequate 613for a more fundamental reason: the data being ranked are simply not amenable to s... |

1 |
Metrix search engine ratings. www.searchenginewatch.com/reports/mediametrix.html
- Media
(Show Context)
Citation Context ...ex, on a given query, if it ranks the page \too highly" with respect to other pages in the index, in the view of a \typical" user. Indeed, in accord with this intuition, search engines are both rated =-=[18]-=- and trained byhuman evaluators. This approach to de ning spam: (1) permits an author to raise the rank of her page by improving the content� (2) puts ground truth about the relative value of pages in... |