Results 1  10
of
36
An Efficient Boosting Algorithm for Combining Preferences
, 1999
"... The problem of combining preferences arises in several applications, such as combining the results of different search engines. This work describes an efficient algorithm for combining multiple preferences. We first give a formal framework for the problem. We then describe and analyze a new boosting ..."
Abstract

Cited by 515 (18 self)
 Add to MetaCart
The problem of combining preferences arises in several applications, such as combining the results of different search engines. This work describes an efficient algorithm for combining multiple preferences. We first give a formal framework for the problem. We then describe and analyze a new boosting algorithm for combining preferences called RankBoost. We also describe an efficient implementation of the algorithm for certain natural cases. We discuss two experiments we carried out to assess the performance of RankBoost. In the first experiment, we used the algorithm to combine different WWW search strategies, each of which is a query expansion for a given domain. For this task, we compare the performance of RankBoost to the individual search strategies. The second experiment is a collaborativefiltering task for making movie recommendations. Here, we present results comparing RankBoost to nearestneighbor and regression algorithms.
Learning to rank: from pairwise approach to listwise approach
 In Proc. ICML’07
, 2007
"... The paper is concerned with learning to rank, which is to construct a model or a function for ranking objects. Learning to rank is useful for document retrieval, collaborative filtering, and many other applications. Several methods for learning to rank have been proposed, which take object pairs as ..."
Abstract

Cited by 132 (23 self)
 Add to MetaCart
The paper is concerned with learning to rank, which is to construct a model or a function for ranking objects. Learning to rank is useful for document retrieval, collaborative filtering, and many other applications. Several methods for learning to rank have been proposed, which take object pairs as ‘instances ’ in learning. We refer to them as the pairwise approach in this paper. Although the pairwise approach offers advantages, it ignores the fact that ranking is a prediction task on list of objects. The paper postulates that learning to rank should adopt the listwise approach in which lists of objects are used as ‘instances ’ in learning. The paper proposes a new probabilistic method for the approach. Specifically it introduces two probability models, respectively referred to as permutation probability and top one probability, to define a listwise loss function for learning. Neural Network and Gradient Descent are then employed as model and algorithm in the learning method. Experimental results on information retrieval show that the proposed listwise approach performs better than the pairwise approach. Microsoft technique report. A short version of this work is published
Unifying collaborative and contentbased filtering
 In ICML
, 2004
"... Collaborative and contentbased filtering are two paradigms that have been applied in the context of recommender systems and user preference prediction. This paper proposes a novel, unified approach that systematically integrates all available training information such as past useritem ratings as w ..."
Abstract

Cited by 70 (2 self)
 Add to MetaCart
Collaborative and contentbased filtering are two paradigms that have been applied in the context of recommender systems and user preference prediction. This paper proposes a novel, unified approach that systematically integrates all available training information such as past useritem ratings as well as attributes of items or users to learn a prediction function. The key ingredient of our method is the design of a suitable kernel or similarity function between useritem pairs that allows simultaneous generalization across the user and item dimensions. We propose an online algorithm (JRank) that generalizes perceptron learning. Experimental results on the EachMovie data set show significant improvements over standard approaches. 1.
Consensus ranking under the exponential model
 In Conf. on Uncertainty in Artificial Intelligence (UAI
, 2007
"... Assume that we are given a set of N rankings, a.k.a linear orderings on n objects. For instance, the rankings represent the individual preferences of a panel of N judges, each presented with the same set of n candidate objects. The problem of rank aggregation or of finding a consensus ranking, is fo ..."
Abstract

Cited by 26 (4 self)
 Add to MetaCart
Assume that we are given a set of N rankings, a.k.a linear orderings on n objects. For instance, the rankings represent the individual preferences of a panel of N judges, each presented with the same set of n candidate objects. The problem of rank aggregation or of finding a consensus ranking, is formulated as finding a single ranking π0 that best agrees with all the N rankings. 1 Kendall’s correlation [Fligner and Verducci, 1986] is a widely used models of agreement [Ailon et al., 2005, Lebanon and Lafferty, 2003, Cohen et al., 1999]. The Kendall distance is defined as dK(π, π0) = ∑ l≺πj 1 [j≺π0 l] (1) In the above, π, π0 represent permutations and i ≺π j (i ≺π0 j) mean that l precedes j (i.e is preferred to j) in permutation π (π0). Hence dK is the total number of pairwise disagreements between π and π0. This distance was further generalized to a family of parametrized distances [Fligner and Verducci, 1986] by dθ(π, π0) = ∑n−1 −1 j=1 θjVj(ππ0 are given functions on the ranking poset. A
Cluster Analysis of Heterogeneous Rank Data
"... This revision of the ICML 2007 proceedings article corrects an error in Sec. 3. Cluster analysis of ranking data, which occurs in consumer questionnaires, voting forms or other inquiries of preferences, attempts to identify typical groups of rank choices. Empirically measured rankings are often inco ..."
Abstract

Cited by 26 (0 self)
 Add to MetaCart
This revision of the ICML 2007 proceedings article corrects an error in Sec. 3. Cluster analysis of ranking data, which occurs in consumer questionnaires, voting forms or other inquiries of preferences, attempts to identify typical groups of rank choices. Empirically measured rankings are often incomplete, i.e. different numbers of filled rank positions cause heterogeneity in the data. We propose a mixture approach for clustering of heterogeneous rank data. Rankings of different lengths can be described and compared by means of a single probabilistic model. A maximum entropy approach avoids hidden assumptions about missing rank positions. Parameter estimators and an efficient EM algorithm for unsupervised inference are derived for the ranking mixture model. Experiments on both synthetic data and realworld data demonstrate significantly improved parameter estimates on heterogeneous data when the incomplete rankings are included in the inference process. 1.
Global models of document structure using latent permutations
 In NAACL’09
, 2009
"... We present a novel Bayesian topic model for learning discourselevel document structure. Our model leverages insights from discourse theory to constrain latent topic assignments in a way that reflects the underlying organization of document topics. We propose a global model in which both topic selec ..."
Abstract

Cited by 18 (4 self)
 Add to MetaCart
We present a novel Bayesian topic model for learning discourselevel document structure. Our model leverages insights from discourse theory to constrain latent topic assignments in a way that reflects the underlying organization of document topics. We propose a global model in which both topic selection and ordering are biased to be similar across a collection of related documents. We show that this space of orderings can be elegantly represented using a distribution over permutations called the generalized Mallows model. Our structureaware approach substantially outperforms alternative approaches for crossdocument comparison and singledocument segmentation. 1 1
Comparing partial rankings
 SIAM Journal on Discrete Mathematics
, 2004
"... Abstract. We provide a comprehensive picture of how to compare partial rankings, that is, rankings that allow ties. We propose several metrics to compare partial rankings and prove that they are within constant multiples of each other. Key words. partial ranking, bucket order, permutation, metric AM ..."
Abstract

Cited by 18 (1 self)
 Add to MetaCart
Abstract. We provide a comprehensive picture of how to compare partial rankings, that is, rankings that allow ties. We propose several metrics to compare partial rankings and prove that they are within constant multiples of each other. Key words. partial ranking, bucket order, permutation, metric AMS subject classifications. 06A06, 68R99 DOI. 10.1137/05063088X
Unsupervised Rank Aggregation with DistanceBased Models
"... The need to meaningfully combine sets of rankings often comes up when one deals with ranked data. Although a number of heuristic and supervised learning approaches to rank aggregation exist, they require domain knowledge or supervised ranked data, both of which are expensive to acquire. In order to ..."
Abstract

Cited by 16 (6 self)
 Add to MetaCart
The need to meaningfully combine sets of rankings often comes up when one deals with ranked data. Although a number of heuristic and supervised learning approaches to rank aggregation exist, they require domain knowledge or supervised ranked data, both of which are expensive to acquire. In order to address these limitations, we propose a mathematical and algorithmic framework for learning to aggregate (partial) rankings without supervision. We instantiate the framework for the cases of combining permutations and combining topk lists, and propose a novel metric for the latter. Experiments in both scenarios demonstrate the effectiveness of the proposed formalism. 1.
An Experimental Comparison of Performance Measures for Classification
, 2007
"... Performance metrics in classification are fundamental to assess the quality of learning methods and learned models. However, many different measures have been defined in the literature with the aim of making better choices in general or for a specific application area. Choices made by one metric are ..."
Abstract

Cited by 13 (5 self)
 Add to MetaCart
Performance metrics in classification are fundamental to assess the quality of learning methods and learned models. However, many different measures have been defined in the literature with the aim of making better choices in general or for a specific application area. Choices made by one metric are claimed to be different from choices made by other metrics. In this work we analyse experimentally the behaviour of 18 different performance metrics in several scenarios, identifying clusters and relationships between measures. We also perform a sensitivity analysis for all of them in terms of several traits: class threshold choice, separability/ranking quality, calibration performance and sensitivity to changes in prior class distribution. From the definitions and the experiments, we give a comprehensive analysis on the relationships between metrics, and a taxonomy and arrangement of them according to the previous traits. This can be useful to choose the most adequate measure (or set of measures) for a specific application. Additionally, the study also highlights some niches in which new measures might be defined and also shows that some supposedly innovative measures make the same choices (or almost) than existing ones. Finally, this work can also be used as a reference for comparing experimental results in the pattern recognition and machine learning literature, when using different measures.
Content Modeling Using Latent Permutations
"... We present a novel Bayesian topic model for learning discourselevel document structure. Our model leverages insights from discourse theory to constrain latent topic assignments in a way that reflects the underlying organization of document topics. We propose a global model in which both topic selec ..."
Abstract

Cited by 8 (5 self)
 Add to MetaCart
We present a novel Bayesian topic model for learning discourselevel document structure. Our model leverages insights from discourse theory to constrain latent topic assignments in a way that reflects the underlying organization of document topics. We propose a global model in which both topic selection and ordering are biased to be similar across a collection of related documents. We show that this space of orderings can be effectively represented using a distribution over permutations called the Generalized Mallows Model. We apply our method to three complementary discourselevel tasks: crossdocument alignment, document segmentation, and information ordering. Our experiments show that incorporating our permutationbased model in these applications yields substantial improvements in performance over previously proposed methods. 1 1.