Results 1 
6 of
6
Convex multitask feature learning
 MACHINE LEARNING
, 2007
"... We present a method for learning sparse representations shared across multiple tasks. This method is a generalization of the wellknown singletask 1norm regularization. It is based on a novel nonconvex regularizer which controls the number of learned features common across the tasks. We prove th ..."
Abstract

Cited by 145 (15 self)
 Add to MetaCart
(Show Context)
We present a method for learning sparse representations shared across multiple tasks. This method is a generalization of the wellknown singletask 1norm regularization. It is based on a novel nonconvex regularizer which controls the number of learned features common across the tasks. We prove that the method is equivalent to solving a convex optimization problem for which there is an iterative algorithm which converges to an optimal solution. The algorithm has a simple interpretation: it alternately performs a supervised and an unsupervised step, where in the former step it learns taskspecific functions and in the latter step it learns commonacrosstasks sparse representations for these functions. We also provide an extension of the algorithm which learns sparse nonlinear representations using kernels. We report experiments on simulated and real data sets which demonstrate that the proposed method can both improve the performance relative to learning each task independently and lead to a few learned features common across related tasks. Our algorithm can also be used, as a special case, to simply select – not learn – a few common variables across the tasks.
Gene expression data classification with revised kernel partial least squares algorithm
 Proceedings of the 17th International FLAIRS Conference, South Beach
, 2004
"... One important feature of the gene expression data is that the number of genes M far exceeds the number of samples N. Standard statistical methods do not work well when N < M. Development of new methodologies or modification of existing methodologies is needed for the analysis of the microarray da ..."
Abstract

Cited by 2 (2 self)
 Add to MetaCart
One important feature of the gene expression data is that the number of genes M far exceeds the number of samples N. Standard statistical methods do not work well when N < M. Development of new methodologies or modification of existing methodologies is needed for the analysis of the microarray data. In this paper, we propose a novel analysis procedure for classifying the gene expression data. This procedure involves dimension reduction using kernel partial least squares (KPLS) and classification with logistic regression (discrimination) and other standard machine learning methods. KPLS is a generalization and nonlinear version of partial least squares (PLS). The proposed algorithm was applied to five different gene expression datasets involving human tumor samples. Comparison with other popular classification methods such as support vector machines and neural networks shows that our algorithm is very promising in classifying gene expression data.
The Unbalanced Classification Problem: Detecting Breaches in Security
 DOCTORAL DISSERTATION, RENSSELAER POLYTECHNIC INSTITUTE
, 2006
"... ..."
Health monitoring of a shaft transmission system via hybrid models of PCR and PLS
"... Abstract: Prediction of motor shaft misalignment is essential for the development of effective coupling and rotating equipment maintenance information systems. It can be stated as a multivariate regression problem with illposed data. In this paper, hybrid models of principal components regression ( ..."
Abstract
 Add to MetaCart
Abstract: Prediction of motor shaft misalignment is essential for the development of effective coupling and rotating equipment maintenance information systems. It can be stated as a multivariate regression problem with illposed data. In this paper, hybrid models of principal components regression (PCR) and partial least squares regression (PLS) have been proposed for this problem. The basic idea of hybrid models is to combine the merits of PCR and PLS to develop more accurate prediction techniques. Both the principal components defined in PCR and the latent variables in PLS are involved in a hybrid model. The experimental results show that an optimal hybrid model can outperform PCR and PLS, especially when the number of predictor variables increases. It suggests that the proposed approach may be particularly useful for complex prediction tasks that need more predictor variables. Discussions for future research are also presented.
of highdimensional biomedical data
, 2005
"... Data complexity assessment in undersampled classification ..."
(Show Context)
Multivariate Regression via Stiefel Constraints
"... We introduce a new framework for regression between multidimensional spaces. Standard methods for solving this problem typically reduce the problem to onedimensional regression by choosing features in the input and/or output spaces. These methods, which include PLS (partial least squares), KDE ( ..."
Abstract
 Add to MetaCart
(Show Context)
We introduce a new framework for regression between multidimensional spaces. Standard methods for solving this problem typically reduce the problem to onedimensional regression by choosing features in the input and/or output spaces. These methods, which include PLS (partial least squares), KDE (kernel dependency estimation), and PCR (principal component regression), select features based on different apriori judgments as to their relevance. Moreover, loss function and constraints are chosen not primarily on statistical grounds, but to simplify the resulting optimisation. By contrast, in our approach the feature construction and the regression estimation are performed jointly, directly minimizing a loss function that we specify, subject to a rank constraint. A major advantage of this approach is that the loss is no longer chosen according to the algorithmic requirements, but can be tailored to the characteristics of the task at hand; the features will then be optimal with respect to this objective. Our approach also allows for the possibility of using a regularizer in the optimization. 1.