The hidden life of latent variables: Bayesian learning with mixed graph models

The hidden life of latent variables: Bayesian learning with mixed graph models (2008)

Cached

Download Links

by Ricardo Silva , Zoubin Ghahramani

Citations:

BibTeX

@MISC{Silva08thehidden,
    author = {Ricardo Silva and Zoubin Ghahramani},
    title = {The hidden life of latent variables: Bayesian learning with mixed graph models},
    year = {2008}
}

Bookmark

OpenURL

Abstract

Directed acyclic graphs (DAGs) have been widely used as a representation of conditional independence in machine learning and statistics. Moreover, hidden or latent variables are often an important component of graphical models. However, DAG models suffer from an important limitation: the family of DAGs is not closed under marginalization of hidden variables. This means that in general we cannot use a DAG to represent the independencies over a subset of variables in a larger DAG. Directed mixed graphs (DMGs) are a representation that includes DAGs as a special case, and overcomes this limitation. This paper introduces algorithms for performing Bayesian inference in Gaussian and probit DMG models. An important requirement for inference is the characterization of the distribution over parameters of the models. We introduce a new distribution for covariance matrices of Gaussian DMGs. We discuss and illustrate how several Bayesian machine learning tasks can benefit from the principle presented here: the power to model dependencies that are generated from hidden variables, but without necessarily modelling such variables explicitly.

Citations

848	Graphical Models - Lauritzen - 1996
766	Causality: Models, Reasoning, and Inference - Pearl
710	A tutorial on learning with Bayesian networks - Heckerman - 1995
656	An introduction to variational methods for graphical models - Jordan, Ghahramani, et al. - 1999
513	Bayesian Learning for Neural Networks - Neal - 1996
469	editor. Learning in Graphical Models - Jordan - 1998
384	Causation, prediction and search - Spirtes, Glymour, et al. - 1993
291	Bayesian analysis of binary and polychotomous response data - Albert, Chib - 1993
279	Constructing free-energy approximations and generalized belief propagation algorithms - Yedidia, Freeman, et al.
181	Variational algorithms for approximate Bayesian inference - Beal - 2003
156	Latent Variable Models and Factor Analysis - Bartholomew, Knott - 1999
156	Being Bayesian about network structure. a Bayesian approach to structure discovery in Bayesian networks - Friedman, Koller
129	Introduction to Monte Carlo Methods - MacKay - 1998
127	Guestrin C: Learning Structured Prediction Models: A Large Margin Approach - Taskar, Chatalbashev, et al. - 2005
118	Optimal structure identification with greedy search - Chickering
111	Correlation and causation - Wright - 1921
110	Annealed importance sampling - Neal - 2001
84	GP (2005) Causal protein-signaling networks derived from multiparameter single-cell data - Sachs, Perez, et al.
73	Decomposition by clique separators - Tarjan - 1985
58	Ancestral graph Markov models - Richardson, Spirtes
46	Experiments in stochastic computation for high-dimensional graphical models - Jones, Carvalho, et al.
37	Nonconjugate Bayesian estimation of covariance matrices and its use in hierarchical models - Daniels, Kass - 1999
28	Directed cyclic graphical representations of feedback models - Spirtes - 1995
27	Markov property for acyclic directed mixed graphs - Richardson - 2003
27	Efficient estimation of covariance selection models - Wong, Carter, et al. - 2003
26	Covariance regularization by thresholding - Bickel, Levina - 2008
26	Learning the structure of linear latent variable models - Silva, Scheines, et al.
24	Hyper inverse wishart distribution for non-decomposable graphs and its application to bayesian inference for gaussian graphical models - Roverato - 2002
23	Asymptotic model selection for naive Bayesian networks - Rusakov, Geigerm - 2005
23	Nonlinear causal discovery with additive noise models - Hoyer, Janzing, et al. - 2009
22	Relational learning with Gaussian processes - Chu, Sindhwani, et al.
20	Bayesian Estimation and Testing of Structural Equation Models.” Psychometrika - Scheines, Hoijtink, et al. - 1999
20	MCMC for doubly-intractable distributions - Murray, Ghahramani, et al. - 2006
17	Structural Equation Models With Latent Variables - Bollen - 1989
17	Decomposition by clique separators, Discrete Mathematics 55 - Tarjan - 1985
15	2004a) Iterative conditional fitting for Gaussian ancestral graph models - Drton, Richardson
13	A Monte Carlo method to compute the marginal likelihood in non decomposable graphical gaussian models. Biometrika in press. Covariance Matrix Estimation 27 - Atay-Kayis, Massam - 2005
13	Binary models for marginal independencies - Drton, Richardson - 2005
13	Probabilistic Reasoning in Expert Systems: Networks of Plausible Inference - Pearl - 1988
12	A new algorithm for maximum likelihood estimation in Gaussian graphical models for marginal independence - Drton, Richardson - 2003
12	Efficient bayesian inference for gaussian copula regression models - Pitt, Chan, et al. - 2006
11	Random effect selection in linear mixed models - Chen, Dunson - 2003
9	Inference for a covariance matrix - Brown, Le, et al. - 1994
9	Heuristic greedy search algorithms for latent variable models - Spirtes - 1997
7	Multiple testing and error control in Gaussian Graphical model selection - Drton, Perlman - 2007
6	Bayesian structural equation modeling - Palomo, Dunson, et al. - 2007
6	Nested Sampling for General Bayesian Computation - Skilling
6	An elicitation method for multivariate normal distributions - Al-Awadhi, Garthwaite - 1998
6	Graphical methods for efficient likelihood inference in gaussian covariance models - Drton, Richardson - 2008
5	Bayesian learning of measurement and structural models - Silva, Scheines - 2006