Results 1  10
of
12
Learning to rank with pairwise regularized leastsquares
 SIGIR 2007 Workshop on Learning to Rank for Information Retrieval
, 2007
"... Learning preference relations between objects of interest is one of the key problems in machine learning. Our approach for addressing this task is based on pairwise comparisons for estimation of overall ranking. In this paper, we propose a simple preference learning algorithm based on regularized le ..."
Abstract

Cited by 12 (7 self)
 Add to MetaCart
Learning preference relations between objects of interest is one of the key problems in machine learning. Our approach for addressing this task is based on pairwise comparisons for estimation of overall ranking. In this paper, we propose a simple preference learning algorithm based on regularized least squares and describe it within the kernel methods framework. Our algorithm, that we call RankRLS, minimizes a regularized leastsquares approximation of a ranking error function that counts the number of incorrectly ranked pairs of data points. We consider both primal and dual versions of the algorithm. The primal version is preferable when the dimensionality of the feature space is smaller than the number of training data points and the dual one is preferable in the opposite case. We show that both versions of RankRLS can be trained as efficiently as the corresponding versions of standard regularized leastsquares regression, despite the fact that the number of training data point pairs under consideration grows quadratically with respect to the number of individual points. As a representative example of a case where the data points outnumber features we choose the Letor dataset. For the opposite case, we choose the parse ranking task. We show that on the Letor dataset the primal RankRLS performs comparably to RankSVM and RankBoost algorithms that are used as baselines. Moreover, we show that the dual RankRLS notably outperforms the standard regularized leastsquares regression in parse ranking. We suggest that the main advantage of RankRLS is the computational efficiency both in the primal and the dual versions, especially since the efficient implementation of the latter is not straightforward, for example, for the support vector machines. 1.
Graph kernels versus graph representations: a case study in parse ranking
 In ECML/PKDD’06 workshop on Mining and Learning with Graphs (MLG’06
"... Abstract. Recently, several kernel functions designed for a data that consists of graphs have been presented. In this paper, we concentrate on designing graph representations and adapting the kernels for these graphs. In particular, we propose graph representations for dependency parses and analyse ..."
Abstract

Cited by 8 (8 self)
 Add to MetaCart
Abstract. Recently, several kernel functions designed for a data that consists of graphs have been presented. In this paper, we concentrate on designing graph representations and adapting the kernels for these graphs. In particular, we propose graph representations for dependency parses and analyse the applicability of several variations of the graph kernels for the problem of parse ranking in the domain of biomedical texts. The parses used in the study are generated with the link grammar (LG) parser from annotated sentences of BioInfer corpus. The results indicate that designing the graph representation is as important as designing the kernel function that is used as the similarity measure of the graphs. 1
Machine Learning to Automate the Assignment of Diagnosis Codes to Freetext Radiology Reports: a Method Description
"... We introduce a multilabel classification system for the automated assignment of diagnostic codes to radiology reports. The system is a cascade of text enrichment, feature selection and two classifiers. It was evaluated in the Computational Medicine Center’s 2007 Medical Natural Language Processing ..."
Abstract

Cited by 4 (0 self)
 Add to MetaCart
We introduce a multilabel classification system for the automated assignment of diagnostic codes to radiology reports. The system is a cascade of text enrichment, feature selection and two classifiers. It was evaluated in the Computational Medicine Center’s 2007 Medical Natural Language Processing Challenge and achieved a 87.7 % microaveraged F1score and third place out of 44 submissions in the task, where 45 different ICD9CM codes were present in 94 combinations. Especially the text enrichment and feature selection components are shown to contribute to our success. Our study provides insight into the development of applications for reallife usage, which are currently rare. Appearing in the Proceedings of the ICML/UAI/COLT
Transductive ranking via pairwise regularized leastsquares
 Workshop on Mining and Learning with Graphs (MLG’07
, 2007
"... Ranking data points with respect to a given preference criterion is an example of a preference learning task. Tasks of this kind are often considered as classification problems, where the training set is composed of data point pairs, in ..."
Abstract

Cited by 3 (2 self)
 Add to MetaCart
Ranking data points with respect to a given preference criterion is an example of a preference learning task. Tasks of this kind are often considered as classification problems, where the training set is composed of data point pairs, in
Matrix representations, linear transformations, and kernels for disambiguation in natural language
"... ..."
Speeding up Greedy Forward Selection for Regularized LeastSquares
"... Abstract—We propose a novel algorithm for greedy forward feature selection for regularized leastsquares (RLS) regression and classification, also known as the leastsquares support vector machine or ridge regression. The algorithm, which we call greedy RLS, starts from the empty feature set, and on ..."
Abstract

Cited by 2 (1 self)
 Add to MetaCart
Abstract—We propose a novel algorithm for greedy forward feature selection for regularized leastsquares (RLS) regression and classification, also known as the leastsquares support vector machine or ridge regression. The algorithm, which we call greedy RLS, starts from the empty feature set, and on each iteration adds the feature whose addition provides the best leaveoneout crossvalidation performance. Our method is considerably faster than the previously proposed ones, since its time complexity is linear in the number of training examples, the number of features in the original data set, and the desired size of the set of selected features. Therefore, as a side effect we obtain a new training algorithm for learning sparse linear RLS predictors which can be used for large scale learning. This speed is possible due to matrix calculus based shortcuts for leaveoneout and feature addition. We experimentally demonstrate the scalability of our algorithm compared to previously proposed implementations. I.
Efficient HoldOut for Subset of Regressors
"... Abstract. Holdout and crossvalidation are among the most useful methods for model selection and performance assessment of machine learning algorithms. In this paper, we present a computationally efficient algorithm for calculating the holdout performance for sparse regularized leastsquares (RLS) ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
Abstract. Holdout and crossvalidation are among the most useful methods for model selection and performance assessment of machine learning algorithms. In this paper, we present a computationally efficient algorithm for calculating the holdout performance for sparse regularized leastsquares (RLS) in case the method is already trained with the whole training set. The computational complexity of performing the holdout is O(H  3 + H  2 n), where H  is the size of the holdout set and n is the number of basis vectors. The algorithm can thus be used to calculate various types of crossvalidation estimates effectively. For example, when m is the number of training examples, the complexities of Nfold and leaveoneout crossvalidations are O(m 3 /N 2 +(m 2 n)/N)andO(mn), respectively. Further, since sparse RLS can be trained in O(mn 2)time for several regularization parameter values in parallel, the fast holdout algorithm enables efficient selection of the optimal parameter value. 1
Efficient AUC Maximization with Regularized
"... Area under the receiver operating characteristics curve (AUC) is a popular measure for evaluating the quality of binary classifiers, and intuitively, machine learning algorithms that maximize an approximation of AUC should have a good AUC performance when classifying new examples. However, designing ..."
Abstract
 Add to MetaCart
Area under the receiver operating characteristics curve (AUC) is a popular measure for evaluating the quality of binary classifiers, and intuitively, machine learning algorithms that maximize an approximation of AUC should have a good AUC performance when classifying new examples. However, designing such algorithms in the framework of kernel methods has proven to be challenging. In this paper, we address AUC maximization with the regularized leastsquares (RLS) algorithm also known as the leastsquares support vector machine. First, we introduce RLStype binary classifier that maximizes an approximation of AUC and has a closedform solution. Second, we show that this AUCRLS algorithm is computationally as efficient as the standard RLS algorithm that maximizes an approximation of the accuracy. Third, we compare the performance of these two algorithms in the task of assigning topic labels for newswire articles in terms of AUC. Our algorithm outperforms the standard RLS in every classification experiment conducted. The performance gains are most substantial when the distribution of the class labels is unbalanced. In conclusion, modifying the RLS algorithm to maximize the approximation of AUC does not increase the computational complexity, and this alteration enhances the quality of the classifier. 1.
Kernel methods
, 2009
"... Abstract In this paper, we introduce a framework for regularized leastsquares (RLS) type of ranking cost functions and we propose three such cost functions. Further, we propose a kernelbased preference learning algorithm, which we call RankRLS, for minimizing these functions. It is shown that Rank ..."
Abstract
 Add to MetaCart
Abstract In this paper, we introduce a framework for regularized leastsquares (RLS) type of ranking cost functions and we propose three such cost functions. Further, we propose a kernelbased preference learning algorithm, which we call RankRLS, for minimizing these functions. It is shown that RankRLS has many computational advantages compared to the ranking algorithms that are based on minimizing other types of costs, such as the hinge cost. In particular, we present efficient algorithms for training, parameter selection, multiple output learning, crossvalidation, and largescale learning. Circumstances under which these computational benefits make RankRLS preferable to RankSVM are considered. We evaluate RankRLS on four different types of ranking tasks using RankSVM and the standard RLS regression as the baselines. RankRLS outperforms the standard RLS regression and its performance is very similar to that of RankSVM, while RankRLS has several computational benefits over RankSVM.