Results 11  20
of
51
Variational inference in nonconjugate models
 Journal of Machine Learning Research
, 2013
"... Meanfield variational methods are widely used for approximate posterior inference in many probabilistic models. In a typical application, meanfield methods approximately compute the posterior with a coordinateascent optimization algorithm. When the model is conditionally conjugate, the coordinate ..."
Abstract

Cited by 17 (4 self)
 Add to MetaCart
(Show Context)
Meanfield variational methods are widely used for approximate posterior inference in many probabilistic models. In a typical application, meanfield methods approximately compute the posterior with a coordinateascent optimization algorithm. When the model is conditionally conjugate, the coordinate updates are easily derived and in closed form. However, many models of interest—like the correlated topic model and Bayesian logistic regression—are nonconjugate. In these models, meanfield methods cannot be directly applied and practitioners have had to develop variational algorithms on a casebycase basis. In this paper, we develop two generic methods for nonconjugate models, Laplace variational inference and delta method variational inference. Our methods have several advantages: they allow for easily derived variational algorithms with a wide class of nonconjugate models; they extend and unify some of the existing algorithms that have been derived for specific models; and they work well on realworld data sets. We studied our methods on the correlated topic model, Bayesian logistic regression, and hierarchical Bayesian logistic regression.
Geofolk: latent spatial semantics in web 2.0 social media
 In: Proc. of ACM WSDM
, 2010
"... We describe an approach for multimodal characterization of social media by combining text features (e.g. tags as a prominent example of short, unstructured text labels) with spatial knowledge (e.g. geotags and coordinates of images and videos). Our modelbased framework GeoFolk combines these two a ..."
Abstract

Cited by 17 (1 self)
 Add to MetaCart
(Show Context)
We describe an approach for multimodal characterization of social media by combining text features (e.g. tags as a prominent example of short, unstructured text labels) with spatial knowledge (e.g. geotags and coordinates of images and videos). Our modelbased framework GeoFolk combines these two aspects in order to construct better algorithms for content management, retrieval, and sharing. The approach is based on multimodal Bayesian models which allow us to integrate spatial semantics of social media in a wellformed, probabilistic manner. We systematically evaluate the solution on a subset of Flickr data, in characteristic scenarios of tag recommendation, content classification, and clustering. Experimental results show that our method outperforms baseline techniques that are based on one of the aspects alone. The approach described in this contribution can also be used in other domains such as Geoweb retrieval.
Accounting for NonGenetic Factors Improves the Power of eQTL Studies
"... Abstract. The recent availability of large scale data sets profiling single nucleotide polymorphisms (SNPs) and gene expression across different human populations, has directed much attention towards discovering patterns of genetic variation and their association with gene regulation. The influence ..."
Abstract

Cited by 15 (2 self)
 Add to MetaCart
(Show Context)
Abstract. The recent availability of large scale data sets profiling single nucleotide polymorphisms (SNPs) and gene expression across different human populations, has directed much attention towards discovering patterns of genetic variation and their association with gene regulation. The influence of environmental, developmental and other factors on gene expression can obscure such associations. We present a model that explicitly accounts for nongenetic factors so as to improve significantly the power of an expression Quantitative Trait Loci (eQTL) study. Our method also exploits the inherent block structure of haplotype data to further enhance its sensitivity. On data from the HapMap project, we find more than three times as many significant associations than a standard eQTL method. 1
Generative affine localisation and tracking
 In Advances in Neural Information Processing Systems
, 2004
"... We present an extension to the Jojic and Frey (2001) layered sprite model which allows for layers to undergo affine transformations. This extension allows for affine object pose to be inferred whilst simultaneously learning the object shape and appearance. Learning is carried out by applying an augm ..."
Abstract

Cited by 15 (1 self)
 Add to MetaCart
(Show Context)
We present an extension to the Jojic and Frey (2001) layered sprite model which allows for layers to undergo affine transformations. This extension allows for affine object pose to be inferred whilst simultaneously learning the object shape and appearance. Learning is carried out by applying an augmented variational inference algorithm which includes a global search over a discretised transform space followed by a local optimisation. To aid correct convergence, we use bottomup cues to restrict the space of possible affine transformations. We present results on a number of video sequences and show how the model can be extended to track an object whose appearance changes throughout the sequence. 1
Efficient bounds for the softmax function, applications to inference in hybrid models, Presentation at the Workshop for Approximate Bayesian Inference in Continuous/Hybrid Systems at NIPS07
, 2007
"... The softmax link is used in many probabilistic model dealing with both discrete and continuous data. However, efficient Bayesian inference for this type of model is still an open problem due to the lack of efficient upper bound for the sum of exponentials. We propose three different bounds for this ..."
Abstract

Cited by 13 (0 self)
 Add to MetaCart
(Show Context)
The softmax link is used in many probabilistic model dealing with both discrete and continuous data. However, efficient Bayesian inference for this type of model is still an open problem due to the lack of efficient upper bound for the sum of exponentials. We propose three different bounds for this function and study their approximation properties. We give a direct application to the Bayesian treatment of multiclass logistic regression and discuss its generalization to deterministic approximate inference in hybrid probabilistic graphical models. The softmax function is the extension of the sigmoid function for more than two values. Its role is of central importance in many nonlinear probabilistic models. In particular, many wellknown models deal with discrete and continuous data. Variational approximations based on the minimization of the KullbackLeibler divergence are one of the most popular tools in largescale Bayesian inference. In recent years, generic tools such as VIBES [1] have been proposed for inference and learning of graphical models using mean field approximations. For graphs having discrete nodes with continuous parents,
Structured variational distributions in VIBES
 In Proceedings Artificial Intelligence and Statistics
, 2003
"... Variational methods are becoming increasingly popular for the approximate solution of complex probabilistic models in machine learning, computer vision, information retrieval and many other fields. Unfortunately, for every new application it is necessary first to derive the specific forms of the var ..."
Abstract

Cited by 12 (3 self)
 Add to MetaCart
(Show Context)
Variational methods are becoming increasingly popular for the approximate solution of complex probabilistic models in machine learning, computer vision, information retrieval and many other fields. Unfortunately, for every new application it is necessary first to derive the specific forms of the variational update equations for the particular probabilistic model being used, and then to implement these equations in applicationspecific software. Each of these steps is both time consuming and error prone. We have therefore recently developed a general purpose inference engine called VIBES [1] (‘Variational Inference for Bayesian Networks’) which allows a wide variety of probabilistic models to be implemented and solved variationally without recourse to coding. New models are specified as a directed acyclic graph using an interface analogous to a drawing package, and VIBES then automatically generates and solves the variational equations. The original version of VIBES assumed a fully factorized variational posterior distribution. In this paper we present an extension of VIBES in which the variational posterior distribution corresponds to a subgraph of the full probabilistic model. Such structured distributions can produce much closer approximations to the true posterior distribution. We illustrate this approach using an example based on Bayesian hidden Markov models. 1
PosteriorMean SuperResolution With a Causal Gaussian Markov Random Field Prior
 IEEE Transactions on Image Processing
, 2012
"... ar ..."
(Show Context)
Expectation consistent free energies for approximate inference
 In NIPS 17
, 2005
"... We propose a novel a framework for deriving approximations for intractable probabilistic models. This framework is based on a free energy (negative log marginal likelihood) and can be seen as a generalization of adaptive TAP [1, 2, 3] and expectation propagation (EP) [4, 5]. The free energy is const ..."
Abstract

Cited by 6 (1 self)
 Add to MetaCart
We propose a novel a framework for deriving approximations for intractable probabilistic models. This framework is based on a free energy (negative log marginal likelihood) and can be seen as a generalization of adaptive TAP [1, 2, 3] and expectation propagation (EP) [4, 5]. The free energy is constructed from two approximating distributions which encode different aspects of the intractable model such a single node constraints and couplings and are by construction consistent on a chosen set of moments. We test the framework on a difficult benchmark problem with binary variables on fully connected graphs and 2D grid graphs. We find good performance using sets of moments which either specify factorized nodes or a spanning tree on the nodes (structured approximation). Surprisingly, the Bethe approximation gives very inferior results even on grids. 1
Tiger: A tuninginsensitive approach for optimally estimating gaussian graphical models
, 2012
"... We propose a new procedure for estimating high dimensional Gaussian graphical models. Our approach is asymptotically tuningfree and nonasymptotically tuninginsensitive: it requires very few efforts to choose the tuning parameter in finite sample settings. Computationally, our procedure is signifi ..."
Abstract

Cited by 6 (3 self)
 Add to MetaCart
(Show Context)
We propose a new procedure for estimating high dimensional Gaussian graphical models. Our approach is asymptotically tuningfree and nonasymptotically tuninginsensitive: it requires very few efforts to choose the tuning parameter in finite sample settings. Computationally, our procedure is significantly faster than existing methods due to its tuninginsensitive property. Theoretically, the obtained estimator is simultaneously minimax optimal for precision matrix estimation under different norms. Empirically, we illustrate the advantages of our method using thorough simulated and real examples. The R package bigmatrix implementing the proposed methods is available on the Comprehensive R Archive Network:
A variational Bayesian method for rectified factor analysis
 In Proc. 2005 IEEE Int. Joint Conf. on Neural Networks (IJCNN 2005
, 2005
"... Abstract — Linear factor models with nonnegativity constraints have received a great deal of interest in a number of problem domains. In existing approaches, positivity has often been associated with sparsity. In this paper we argue that sparsity of the factors is not always a desirable option, but ..."
Abstract

Cited by 5 (4 self)
 Add to MetaCart
(Show Context)
Abstract — Linear factor models with nonnegativity constraints have received a great deal of interest in a number of problem domains. In existing approaches, positivity has often been associated with sparsity. In this paper we argue that sparsity of the factors is not always a desirable option, but certainly a technical limitation of the currently existing solutions. We then reformulate the problem in order to relax the sparsity constraint while retaining positivity. A variational inference procedure is derived and this is contrasted to existing related approaches. Both i.i.d. and firstorder AR variants of the proposed model are provided and these are experimentally demonstrated in a realworld astrophysical application. I.