Results 1  10
of
11
Machine Learning Techniques—Reductions Between Prediction Quality Metrics
"... Abstract Machine learning involves optimizing a loss function on unlabeled data points given examples of labeled data points, where the loss function measures the performance of a learning algorithm. We give an overview of techniques, called reductions, for converting a problem of minimizing one los ..."
Abstract

Cited by 3 (0 self)
 Add to MetaCart
Abstract Machine learning involves optimizing a loss function on unlabeled data points given examples of labeled data points, where the loss function measures the performance of a learning algorithm. We give an overview of techniques, called reductions, for converting a problem of minimizing one loss function into a problem of minimizing another, simpler loss function. This tutorial discusses how to create robust reductions that perform well in practice. The reductions discussed here can be used to solve any supervised learning problem with a standard binary classification or regression algorithm available in any machine learning toolkit. We also discuss common design flaws in folklore reductions. 1
M.: Preferencebased learning to rank
 Machine Learning
, 2010
"... Abstract. This paper presents an efficient preferencebased ranking algorithm running in two stages. In the first stage, the algorithm learns a preference function defined over pairs, as in a standard binary classification problem. In the second stage, it makes use of that preference function to pro ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
Abstract. This paper presents an efficient preferencebased ranking algorithm running in two stages. In the first stage, the algorithm learns a preference function defined over pairs, as in a standard binary classification problem. In the second stage, it makes use of that preference function to produce an accurate ranking, thereby reducing the learning problem of ranking to binary classification. This reduction is based on the familiar QuickSort and guarantees an expected pairwise misranking loss of at most twice that of the binary classifier derived in the first stage. Furthermore, in the important special case of bipartite ranking, the factor of two in loss is reduced to one. This improved bound also applies to the regret achieved by our ranking and that of the binary classifier obtained. Our algorithm is randomized, but we prove a lower bound for any deterministic reduction of ranking to binary classification showing that randomization is necessary to achieve our guarantees. This, and a recent result by Balcan et al., who show a regret bound of two for a deterministic algorithm in the bipartite case, suggest a tradeoff between achieving low regret and determinism in this context. Our reduction also admits an improved running time guarantee with respect to that deterministic algorithm. In particular, the number of calls to the preference function in the reduction is improved from Ω(n 2) to O(n log n). In addition, when the top k ranked elements only are required (k ≪n), as in many applications in information extraction or search engine design, the time complexity of our algorithm can be further reduced to O(k log k+n). Our algorithm is thus practical for realistic applications where the number of points to rank exceeds several thousand. 1
Ranking with kernels in Fourier space
"... In typical ranking problems the total number n of items to be ranked is relatively large, but each data instance involves only k << n items. This paper examines the structure of such partial rankings in Fourier space. Specifically, we develop a kernel–based framework for solving ranking problems, de ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
In typical ranking problems the total number n of items to be ranked is relatively large, but each data instance involves only k << n items. This paper examines the structure of such partial rankings in Fourier space. Specifically, we develop a kernel–based framework for solving ranking problems, define some canonical kernels on permutations, and show that by transforming to Fourier space, the complexity of computing the kernel between two partial rankings can be reduced from O((n−k)! 2) to O((2k) 2k+3). 1
NEW LEARNING FRAMEWORKS FOR INFORMATION RETRIEVAL
, 2011
"... Recent advances in machine learning have enabled the training of increasingly complex information retrieval models. This dissertation proposes principled approaches to formalize the learning problems for information retrieval, with an eye towards developing a unified learning framework. This will co ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
Recent advances in machine learning have enabled the training of increasingly complex information retrieval models. This dissertation proposes principled approaches to formalize the learning problems for information retrieval, with an eye towards developing a unified learning framework. This will conceptually simplify the overall development process, making it easier to reason about higher level goals and properties of the retrieval system. This dissertation advocates two complementary approaches, structured prediction and interactive learning, to learn featurerich retrieval models that can perform well in practice.
Approximation Schemes for . . .
, 2011
"... In correlation clustering, given similarity or dissimilarity information for all pairs of data items, the goal is to find a clustering of the items into similarity classes, with the fewest inconsistencies with the input. This problem is hard to approximate in general but we give arbitrarily good app ..."
Abstract
 Add to MetaCart
In correlation clustering, given similarity or dissimilarity information for all pairs of data items, the goal is to find a clustering of the items into similarity classes, with the fewest inconsistencies with the input. This problem is hard to approximate in general but we give arbitrarily good approximation algorithms (PTASs) for two interesting special cases: when there are few clusters, and when the input is generated from a natural noisy model. In the feedback arc set problem in tournaments, given comparison information (a better than b) for all pairs of data items, the goal is to find a ranking of the items with the fewest inconsistencies with the input. We give the first PTAS for this problem. We then extend our techniques to a more general class of problems called fragile dense problems.
Finding The Most Probable Ranking of Objects with Probabilistic Pairwise Preferences
 10TH INTERNATIONAL CONFERENCE ON DOCUMENT ANALYSIS AND RECOGNITION
, 2009
"... This paper discusses the ranking of a set of objects when a possibly inconsistent set of pairwise preferences is given. We consider the task of ranking objects when pairwise preferences not only can contradict each other, but in general are not binary meaning, for each pair of objects the preferenc ..."
Abstract
 Add to MetaCart
This paper discusses the ranking of a set of objects when a possibly inconsistent set of pairwise preferences is given. We consider the task of ranking objects when pairwise preferences not only can contradict each other, but in general are not binary meaning, for each pair of objects the preference is represented by a pair of nonnegative numbers that sum up to one and can be viewed as a confidence in our belief that one object is preferable to the other in the absence of any other information. We propose a probability function on the sequence of objects that includes nonbinary preferences and evaluate methods for finding the most probable ranking for this model using it to rank results of a Microsoft Online Handwriting Recognizer.
Bipartite Ranking through Minimization of Univariate Loss
, 2011
"... Minimization of the rank loss or, equivalently, maximization of the AUC in bipartite ranking calls for minimizing the number of disagreements between pairs of instances. Since the complexity of this problem is inherently quadratic in the number of training examples, it is tempting to ask how much is ..."
Abstract
 Add to MetaCart
Minimization of the rank loss or, equivalently, maximization of the AUC in bipartite ranking calls for minimizing the number of disagreements between pairs of instances. Since the complexity of this problem is inherently quadratic in the number of training examples, it is tempting to ask how much is actually lost by minimizing a simple univariate loss function, as done by standard classification methods, as a surrogate. In this paper, we first note that minimization of 0/1 loss is not an option, as it may yield an arbitrarily high rank loss. We show, however, that better results can be achieved by means of a weighted (costsensitive) version of 0/1 loss. Yet, the real gain is obtained through marginbased loss functions, for which we are able to derive proper bounds, not only for rank risk but, more importantly, also for rank regret. The paper is completed with an experimental study in which we address specific questions raised by our theoretical analysis.
How to Rank with Fewer Errors A PTAS for Feedback Arc Set in Tournaments
"... We present the first polynomial time approximation scheme (PTAS) for the minimum feedback arc set problem in tournaments. A weighted generalization gives the first PTAS for Kemeny rank aggregation. The runtime is singly exponential in 1/ǫ, improving on the conference version of this work, which was ..."
Abstract
 Add to MetaCart
We present the first polynomial time approximation scheme (PTAS) for the minimum feedback arc set problem in tournaments. A weighted generalization gives the first PTAS for Kemeny rank aggregation. The runtime is singly exponential in 1/ǫ, improving on the conference version of this work, which was doubly exponential.
Bipartite Ranking through Minimization of Univariate Loss
"... Minimization of the rank loss or, equivalently, maximization of the AUC in bipartite ranking calls for minimizing the number of disagreements between pairs of instances. Since the complexity of this problem is inherently quadratic in the number of training examples, it is tempting to ask how much is ..."
Abstract
 Add to MetaCart
Minimization of the rank loss or, equivalently, maximization of the AUC in bipartite ranking calls for minimizing the number of disagreements between pairs of instances. Since the complexity of this problem is inherently quadratic in the number of training examples, it is tempting to ask how much is actually lost by minimizing a simple univariate loss function, as done by standard classification methods, as a surrogate. In this paper, we first note that minimization of 0/1 loss is not an option, as it may yield an arbitrarily high rank loss. We show, however, that better results can be achieved by means of a weighted (costsensitive) version of 0/1 loss. Yet, the real gain is obtained through marginbased loss functions, for which we are able to derive proper bounds, not only for rank risk but, more importantly, also for rank regret. The paper is completed with an experimental study in which we address specific questions raised by our theoretical analysis. 1.