Results 1 - 10
of
15
Multi-label Prediction via Sparse Infinite CCA
"... Canonical Correlation Analysis (CCA) is a useful technique for modeling dependencies between two (or more) sets of variables. Building upon the recently suggested probabilistic interpretation of CCA, we propose a nonparametric, fully Bayesian framework that can automatically select the number of cor ..."
Abstract
-
Cited by 24 (2 self)
- Add to MetaCart
(Show Context)
Canonical Correlation Analysis (CCA) is a useful technique for modeling dependencies between two (or more) sets of variables. Building upon the recently suggested probabilistic interpretation of CCA, we propose a nonparametric, fully Bayesian framework that can automatically select the number of correlation components, and effectively capture the sparsity underlying the projections. In addition, given (partially) labeled data, our algorithm can also be used as a (semi)supervised dimensionality reduction technique, and can be applied to learn useful predictive features in the context of learning a set of related tasks. Experimental results demonstrate the efficacy of the proposed approach for both CCA as a stand-alone problem, and when applied to multi-label prediction. 1
Multi-label linear discriminant analysis
- In ECCV
"... Abstract. Multi-label problems arise frequently in image and video an-notations, and many other related applications such as multi-topic text categorization, music classification, etc. Like other computer vision tasks, multi-label image and video annotations also suffer from the difficulty of high d ..."
Abstract
-
Cited by 16 (10 self)
- Add to MetaCart
(Show Context)
Abstract. Multi-label problems arise frequently in image and video an-notations, and many other related applications such as multi-topic text categorization, music classification, etc. Like other computer vision tasks, multi-label image and video annotations also suffer from the difficulty of high dimensionality because images often have a large number of features. Linear discriminant analysis (LDA) is a well-known method for dimen-sionality reduction. However, the classical Linear Discriminant Analysis (LDA) only works for single-label multi-class classifications and cannot be directly applied to multi-label multi-class classifications. It is desirable to naturally generalize the classical LDA to multi-label formulations. At the same time, multi-label data present a new opportunity to improve classification accuracy through label correlations, which are absent in single-label data. In this work, we propose a novel Multi-label Linear Discriminant Analysis (MLDA) method to take advantage of label cor-relations and explore the powerful classification capability of the classical LDA to deal with multi-label multi-class problems. Extensive experimen-tal evaluations on five public multi-label data sets demonstrate excellent performance of our method.
Linear Dimensionality Reduction for Margin-Based Classification: High-Dimensional Data and Sensor Networks
, 2011
"... Low-dimensional statistics of measurements play an important role in detection problems, including those encountered in sensor networks. In this work, we focus on learning low-dimensional linear statistics of high-dimensional measurement data along with decision rules defined in the low-dimensional ..."
Abstract
-
Cited by 7 (2 self)
- Add to MetaCart
(Show Context)
Low-dimensional statistics of measurements play an important role in detection problems, including those encountered in sensor networks. In this work, we focus on learning low-dimensional linear statistics of high-dimensional measurement data along with decision rules defined in the low-dimensional space in the case when the probability density of the measurements and class labels is not given, but a training set of samples from this distribution is given. We pose a joint optimization problem for linear dimensionality reduction and margin-based classification, and develop a coordinate descent algorithm on the Stiefel manifold for its solution. Although the coordinate descent is not guaranteed to find the globally optimal solution, crucially, its alternating structure enables us to extend it for sensor networks with a message-passing approach requiring little communication. Linear dimensionality reduction prevents overfitting when learning from finite training data. In the sensor network setting, dimensionality reduction not only prevents overfitting, but also reduces power consumption due to communication. The learned reduced-dimensional space and decision rule is shown to be consistent and its Rademacher complexity is characterized. Experimental results are presented for a variety of datasets, including those from existing sensor networks, demonstrating the potential of our methodology in comparison with other dimensionality reduction approaches.
Multi-label subspace ensemble
- in International Conference on Artificial Intelligence and Statistics (AISTATS
, 2012
"... Abstract A challenging problem of multi-label learning is that both the label space and the model complexity will grow rapidly with the increase in the number of labels, and thus makes the available training samples insufficient for training a proper model. In this paper, we eliminate this problem ..."
Abstract
-
Cited by 4 (2 self)
- Add to MetaCart
(Show Context)
Abstract A challenging problem of multi-label learning is that both the label space and the model complexity will grow rapidly with the increase in the number of labels, and thus makes the available training samples insufficient for training a proper model. In this paper, we eliminate this problem by learning a mapping of each label in the feature space as a robust subspace, and formulating the prediction as finding the group sparse representation of a given instance on the subspace ensemble. We term this approach as "multi-label subspace ensemble (MSE)". In the training stage, the data matrix is decomposed as the sum of several low-rank matrices and a sparse residual via a randomized optimization, where each low-rank part defines a subspace mapped by a label. In the prediction stage, the group sparse representation on the subspace ensemble is estimated by group lasso. Experiments on several benchmark datasets demonstrate the appealing performance of MSE.
The Generalized Dimensionality Reduction Problem
"... The dimensionality reduction problem has been widely studied in the database literature because of its application for concise data representation in a variety of database applications. The main focus in dimensionality reduction is to represent the data in a smaller number of dimensions that the lea ..."
Abstract
-
Cited by 3 (0 self)
- Add to MetaCart
(Show Context)
The dimensionality reduction problem has been widely studied in the database literature because of its application for concise data representation in a variety of database applications. The main focus in dimensionality reduction is to represent the data in a smaller number of dimensions that the least amount of information is lost. In this paper, we study the dimensionality reduction problem from an entirely different perspective. We discuss methods to find a representation of the data so that a user-defined objective function is optimized. For example, we may desire to find a reduction of the data in which a particular kind of classifier works effectively. Another example (relevant to the similarity search domain) would be a reduction in which the cluster of k closest points provides the best distance based separation from the remaining data set. We discuss a general abstraction for the problem and provide the broad framework of an evolutionary algorithm which solves this abstraction. We test our framework on two separate instantiations of this framework and provide results illustrating the effectiveness and efficiency of our method. 1
Semi-Supervised Dimension Reduction for Multi-label Classification
"... A significant challenge to make learning techniques more suitable for general purpose use in AI is to move beyond i) complete supervision, ii) low dimensional data and iii) a single label per instance. Solving this challenge would allow making predictions for high dimensional large dataset with mult ..."
Abstract
-
Cited by 2 (1 self)
- Add to MetaCart
A significant challenge to make learning techniques more suitable for general purpose use in AI is to move beyond i) complete supervision, ii) low dimensional data and iii) a single label per instance. Solving this challenge would allow making predictions for high dimensional large dataset with multiple (but possibly incomplete) labelings. While other work has addressed each of these problems separately, in this paper we show how to address them together, namely the problem of semi-supervised dimension reduction for multi-labeled classification, SSDR-MC. To our knowledge this is the first paper that attempts to address all challenges together. In this work, we study a novel joint learning framework which performs optimization for dimension reduction and multi-label inference in semi-supervised setting. The experimental results validate the performance of our approach, and demonstrate the effectiveness of connecting dimension reduction and learning.
Multi-Label Transfer Learning with Sparse Representation
"... Abstract—Due to the visually polysemous barrier, videos and images may be annotated by multiple tags. Discovering the correlations among different tags can significantly help predicting precise labels for videos and images. Many of recent studies toward multi-label learning construct a linear subspa ..."
Abstract
-
Cited by 2 (0 self)
- Add to MetaCart
(Show Context)
Abstract—Due to the visually polysemous barrier, videos and images may be annotated by multiple tags. Discovering the correlations among different tags can significantly help predicting precise labels for videos and images. Many of recent studies toward multi-label learning construct a linear subspace embedding with encoded multi-label information, such that data points sharing many common labels tend to be close to each other in the embedded subspace. Motivated by the advances of compressive sensing research, a sparse representation that selects a compact subset to describe the input data can be more discriminative. In this paper, we propose a sparse multi-label learning method to circumvent the visually polysemous barrier of multiple tags. Our approach learns a multi-label encoded sparse linear embedding space from a related dataset, and maps the target data into the learned new representation space to achieve better annotation performance. Instead of using l1-norm penalty (lasso) to induce sparse representation, we propose to formulate the multi-label learning as a penalized least squares optimization problem with elastic-net penalty. By casting the video concept detection and image annotation tasks into a sparse multi-label transfer learning framework in this paper, ridge regression, lasso, elastic net, and the multi-label extended sparse discriminant analysis methods are, respectively, well explored and compared. Index Terms—Image annotation, multi-label learning, sparse representation, transfer learning, video concept detection. I.
The Role of Dimensionality Reduction in Classification
"... Dimensionality reduction (DR) is often used as a pre-processing step in classification, but usually one first fixes the DR mapping, possibly using label informa-tion, and then learns a classifier (a filter approach). Best performance would be obtained by optimizing the clas-sification error jointly ..."
Abstract
-
Cited by 1 (0 self)
- Add to MetaCart
Dimensionality reduction (DR) is often used as a pre-processing step in classification, but usually one first fixes the DR mapping, possibly using label informa-tion, and then learns a classifier (a filter approach). Best performance would be obtained by optimizing the clas-sification error jointly over DR mapping and classi-fier (a wrapper approach), but this is a difficult non-convex problem, particularly with nonlinear DR. Us-ing the method of auxiliary coordinates, we give a sim-ple, efficient algorithm to train a combination of non-linear DR and a classifier, and apply it to a RBF map-ping with a linear SVM. This alternates steps where we train the RBF mapping and a linear SVM as usual re-gression and classification, respectively, with a closed-
An Ensemble Multi-Label Feature Selection Algorithm Based on Information Entropy IAJIT First Online Publication
, 2012
"... Abstract: In multi-label classification, feature selection is able to remove redundant and irrelevant features, which makes the classifiers faster and improves the prediction performance of the classifiers. Currently most of feature selection algorithms in multi-label classification are dependent on ..."
Abstract
- Add to MetaCart
Abstract: In multi-label classification, feature selection is able to remove redundant and irrelevant features, which makes the classifiers faster and improves the prediction performance of the classifiers. Currently most of feature selection algorithms in multi-label classification are dependent on the concrete classifier, which leads to high computation complexity. Hence this paper proposes an ensemble multi-label feature selection algorithm based on information entropy (EMFSIE), which is independent on any concrete classifiers. Its core idea consists of: 1). we employs the information gain to evaluate the correlation between the feature and the label set; 2). to filter out useful features more effectively, we calculate the information gain in an ensemble framework and filter out useful features according to the threshold value determined by the effective factor. We validate EMFSIE on four datasets from two domains using four different multi-label classifiers. The experimental resultsand their analysis show preliminarily that EMFSIE can not only remove more than 70 % of original features, which makes the classifiers faster, but also keep the prediction performance of the classifiers as good as before, even enhance the prediction performance on three datasets underthe two-tailed paired t-tests at 0.05 significance level.
