Results 1  10
of
13
MedLDA: Maximum Margin Supervised Topic Models for Regression and Classification
"... Supervised topic models utilize document’s side information for discovering predictive low dimensional representations of documents; and existing models apply likelihoodbased estimation. In this paper, we present a maxmargin supervised topic model for both continuous and categorical response variab ..."
Abstract

Cited by 93 (27 self)
 Add to MetaCart
(Show Context)
Supervised topic models utilize document’s side information for discovering predictive low dimensional representations of documents; and existing models apply likelihoodbased estimation. In this paper, we present a maxmargin supervised topic model for both continuous and categorical response variables. Our approach, the maximum entropy discrimination latent Dirichlet allocation (MedLDA), utilizes the maxmargin principle to train supervised topic models and estimate predictive topic representations that are arguably more suitable for prediction. We develop efficient variational methods for posterior inference and demonstrate qualitatively and quantitatively the advantages of MedLDA over likelihoodbased topic models on movie review and 20 Newsgroups data sets. 1.
RankLoss Support Instance Machines for MIML Instance Annotation
"... Multiinstance multilabel learning (MIML) is a framework for supervised classification where the objects to be classified are bags of instances associated with multiple labels. For example, an image can be represented as a bag of segments and associated with a list of objects it contains. Prior wor ..."
Abstract

Cited by 25 (10 self)
 Add to MetaCart
(Show Context)
Multiinstance multilabel learning (MIML) is a framework for supervised classification where the objects to be classified are bags of instances associated with multiple labels. For example, an image can be represented as a bag of segments and associated with a list of objects it contains. Prior work on MIML has focused on predicting label sets for previously unseen bags. We instead consider the problem of predicting instance labels while learning from data labeled only at the bag level. We propose RankLoss Support Instance Machines, which optimize a regularized rankloss objective and can be instantiated with different aggregation models connecting instancelevel predictions with baglevel predictions. The aggregation models that we consider are equivalent to defining a “support instance ” for each bag, which allows efficient optimization of the rankloss objective using primal subgradient descent. Experiments on artificial and realworld datasets show that the proposed methods achieve higher accuracy than other loss functions used in prior work, e.g., Hamming loss, and recent work in ambiguous label classification.
Gibbs maxmargin topic models with fast sampling algorithms
 IN INTERNATIONAL CONFERENCE ON MACHINE LEARNING (ICML
, 2013
"... Existing maxmargin supervised topic models rely on an iterative procedure to solve multiple latent SVM subproblems with additional meanfield assumptions on the desired posterior distributions. This paper presents Gibbs maxmargin topic models by minimizing an expected margin loss, an upper bound o ..."
Abstract

Cited by 14 (10 self)
 Add to MetaCart
Existing maxmargin supervised topic models rely on an iterative procedure to solve multiple latent SVM subproblems with additional meanfield assumptions on the desired posterior distributions. This paper presents Gibbs maxmargin topic models by minimizing an expected margin loss, an upper bound of the existing margin loss derived from an expected prediction rule. By introducing augmented variables, we develop simple and fast Gibbs sampling algorithms with no restricting assumptions and no need to solve SVM subproblems for both classification and regression. Empirical results demonstrate significant improvements on time efficiency. The classification performance is also significantly improved over competitors.
Monte Carlo Methods for Maximum Margin Supervised Topic Models
"... An effective strategy to exploit the supervising side information for discovering predictive topic representations is to impose discriminative constraints induced by such information on the posterior distributions under a topic model. This strategy has been adopted by a number of supervised topic mo ..."
Abstract

Cited by 13 (8 self)
 Add to MetaCart
(Show Context)
An effective strategy to exploit the supervising side information for discovering predictive topic representations is to impose discriminative constraints induced by such information on the posterior distributions under a topic model. This strategy has been adopted by a number of supervised topic models, such as MedLDA, which employs maxmargin posterior constraints. However, unlike the likelihoodbased supervised topic models, of which posterior inference can be carried out using the Bayes ’ rule, the maxmargin posterior constraints have made Monte Carlo methods infeasible or at least not directly applicable, thereby limited the choice of inference algorithms to be based on variational approximation with strict mean field assumptions. In this paper, we develop two efficient Monte Carlo methods under much weaker assumptions for maxmargin supervised topic models based on an importance sampler and a collapsed Gibbs sampler, respectively, in a convex dual formulation. We report thorough experimental results that compare our approach favorably against existing alternatives in both accuracy and efficiency. 1
Gibbs maxmargin topic models with data augmentation
 Journal of Machine Learning Research (JMLR
"... Maxmargin learning is a powerful approach to building classifiers and structured output predictors. Recent work on maxmargin supervised topic models has successfully integrated it with Bayesian topic models to discover discriminative latent semantic structures and make accurate predictions for uns ..."
Abstract

Cited by 13 (5 self)
 Add to MetaCart
(Show Context)
Maxmargin learning is a powerful approach to building classifiers and structured output predictors. Recent work on maxmargin supervised topic models has successfully integrated it with Bayesian topic models to discover discriminative latent semantic structures and make accurate predictions for unseen testing data. However, the resulting learning problems are usually hard to solve because of the nonsmoothness of the margin loss. Existing approaches to building maxmargin supervised topic models rely on an iterative procedure to solve multiple latent SVM subproblems with additional meanfield assumptions on the desired posterior distributions. This paper presents an alternative approach by defining a new maxmargin loss. Namely, we present Gibbs maxmargin supervised topic models, a latent variable Gibbs classifier to discover hidden topic representations for various tasks, including classification, regression and multitask learning. Gibbs maxmargin supervised topic models minimize an expected margin loss, which is an upper bound of the existing margin loss derived from an expected prediction rule. By introducing augmented variables and integrating out the Dirichlet variables analytically by conjugacy, we develop simple
Scalable inference in maxmargin topic models.
 In ACM SIGKDD Conference on Knowledge Discovery and Data Mining,
, 2013
"... ABSTRACT Topic models have played a pivotal role in analyzing large collections of complex data. Besides discovering latent semantics, supervised topic models (STMs) can make predictions on unseen test data. By marrying with advanced learning techniques, the predictive strengths of STMs have been d ..."
Abstract

Cited by 6 (4 self)
 Add to MetaCart
(Show Context)
ABSTRACT Topic models have played a pivotal role in analyzing large collections of complex data. Besides discovering latent semantics, supervised topic models (STMs) can make predictions on unseen test data. By marrying with advanced learning techniques, the predictive strengths of STMs have been dramatically enhanced, such as maxmargin supervised topic models, stateoftheart methods that integrate maxmargin learning with topic models. Though powerful, maxmargin STMs have a hard nonsmooth learning problem. Existing algorithms rely on solving multiple latent SVM subproblems in an EMtype procedure, which can be too slow to be applicable to largescale categorization tasks. In this paper, we present a highly scalable approach to building maxmargin supervised topic models. Our approach builds on three key innovations: 1) a new formulation of Gibbs maxmargin supervised topic models for both multiclass and multilabel classification; 2) a simple "augmentandcollapse" Gibbs sampling algorithm without making restricting assumptions on the posterior distributions; 3) an efficient parallel implementation that can easily tackle data sets with hundreds of categories and millions of documents. Furthermore, our algorithm does not need to solve SVM subproblems. Though performing the two tasks of topic discovery and learning predictive models jointly, which significantly improves the classification performance, our methods have comparable scalability as the stateoftheart parallel algorithms for the standard LDA topic models which perform the single task of topic discovery only. Finally, an opensource implementation is also provided 1 .
Instance annotation for multiinstance multilabel learning,” Transactions on Knowledge Discovery from Data (TKDD
, 2012
"... Multiinstance multilabel learning (MIML) is a framework for supervised classification where the objects to be classified are bags of instances associated with multiple labels. For example, an image can be represented as a bag of segments and associated with a list of objects it contains. Prior wor ..."
Abstract

Cited by 3 (3 self)
 Add to MetaCart
(Show Context)
Multiinstance multilabel learning (MIML) is a framework for supervised classification where the objects to be classified are bags of instances associated with multiple labels. For example, an image can be represented as a bag of segments and associated with a list of objects it contains. Prior work on MIML has focused on predicting label sets for previously unseen bags. We instead consider the problem of predicting instance labels while learning from data labeled only at the bag level. We propose a regularized rankloss objective designed for instance annotation, which can be instantiated with different aggregation models connecting instancelevel labels with baglevel label sets. The aggregation models that we consider can be factored as a linear function of a “support instance ” for each class, which is a single feature vector representing a whole bag. Hence we name our proposed methods rankloss Support Instance Machines (SIM). We propose two optimization methods for the rankloss objective, which is nonconvex. One is a heuristic method that alternates between updating support instances, and solving a convex problem in which the support instances are treated as constant. The other is to apply the constrained concaveconvex procedure (CCCP), which can also be interpreted as iteratively updating support instances and solving a convex problem. To solve the convex problem, we employ the Pegasos framework of primal subgradient descent, and prove that it finds an suboptimal solution in runtime that is linear in the number of bags, instances, and 1 . Additionally, we
Dimensionality reduction and topic modelling: from latent semantic indexing to latent dirichlet allocation and beyond
 in Mining Text Data, C. Aggarwal and
, 2012
"... ..."
(Show Context)
Constrained Instance Clustering in MultiInstance MultiLabel Learning
"... doi:10.1016/j.patrec.2013.07.002 ..."
(Show Context)
Objectiveguided Image Annotation
, 2012
"... Automatic image annotation, which is usually formulated as a multilabel classification problem, is one of major tools to enhance the semantic understanding of web images. Many multimedia applications (e.g., tagbased image retrieval) can greatly benefit from image annotation. However, the insuffic ..."
Abstract
 Add to MetaCart
Automatic image annotation, which is usually formulated as a multilabel classification problem, is one of major tools to enhance the semantic understanding of web images. Many multimedia applications (e.g., tagbased image retrieval) can greatly benefit from image annotation. However, the insufficient performance of image annotation methods prevents these applications from being practical. On the other hand, specific measures are usually designed to evaluate how well one annotation method performs for specific objective/application, but most of image annotation methods do not consider optimization of these measures, so that they are inevitably trapped into suboptimal performance of these objective specific measures. To address this issue, we first summarize a variety of objectiveguided performance measures under a unified representation. Our analysis reveals that macroaveraging measures are very sensitive to infrequent keywords, and hamming measure is easily affected by skewed distributions. We then propose a unified multilabel learning framework, which directly optimizes a variety of objective specific measures of multilabel learning tasks. Specifically, we first present a multilayer hierarchical structure of learning hypotheses for multilabel problems based on which a variety of loss functions with respect to objectiveguided measures are defined. And then, we formulate these loss functions as relaxed surrogate functions and optimize them by structural SVMs. According to the analysis of various measures and the high time complexity of optimizing microaveraging measures, in this paper, we focus on examplebased measures which are tailormade for image annotation tasks but are seldom explored in the literature. Experiments show the consistence with the formal analysis on two widely used multilabel datasets and demonstrate the superior performance of our proposed method over stateoftheart baseline methods in terms of examplebased measures on four image annotation datasets.