Results 1  10
of
34
Tensor decompositions for learning latent variable models
, 2014
"... This work considers a computationally and statistically efficient parameter estimation method for a wide class of latent variable models—including Gaussian mixture models, hidden Markov models, and latent Dirichlet allocation—which exploits a certain tensor structure in their loworder observable mo ..."
Abstract

Cited by 46 (6 self)
 Add to MetaCart
This work considers a computationally and statistically efficient parameter estimation method for a wide class of latent variable models—including Gaussian mixture models, hidden Markov models, and latent Dirichlet allocation—which exploits a certain tensor structure in their loworder observable moments (typically, of second and thirdorder). Specifically, parameter estimation is reduced to the problem of extracting a certain (orthogonal) decomposition of a symmetric tensor derived from the moments; this decomposition can be viewed as a natural generalization of the singular value decomposition for matrices. Although tensor decompositions are generally intractable to compute, the decomposition of these specially structured tensors can be efficiently obtained by a variety of approaches, including power iterations and maximization approaches (similar to the case of matrices). A detailed analysis of a robust tensor power method is provided, establishing an analogue of Wedin’s perturbation theorem for the singular vectors of matrices. This implies a robust and computationally tractable estimation approach for several popular latent variable models.
Likelihood ratio tests and singularities
 Ann. Statist
, 2008
"... Many statistical hypotheses can be formulated in terms of polynomial equalities and inequalities in the unknown parameters and thus correspond to semialgebraic subsets of the parameter space. We consider large sample asymptotics for the likelihood ratio test of such hypotheses in models that satisf ..."
Abstract

Cited by 12 (3 self)
 Add to MetaCart
(Show Context)
Many statistical hypotheses can be formulated in terms of polynomial equalities and inequalities in the unknown parameters and thus correspond to semialgebraic subsets of the parameter space. We consider large sample asymptotics for the likelihood ratio test of such hypotheses in models that satisfy standard probabilistic regularity conditions. We show that the assumptions of Chernoff’s theorem hold for semialgebraic sets such that the asymptotics are determined by the tangent cone at the true parameter point. At boundary points or singularities, the tangent cone need not be a linear space and limiting distributions other than chisquare distributions may arise. While boundary points often lead to mixtures of chisquare distributions, singularities give rise to nonstandard limits. We demonstrate that minima of chisquare random variables are important for locally identifiable models, and in a study of the factor analysis model with one factor, we reveal connections to eigenvalues of Wishart matrices.
Algebraic geometry of Gaussian Bayesian networks
, 2007
"... Conditional independence models in the Gaussian case are algebraic varieties in the cone of positive definite covariance matrices. We study these varieties in the case of Bayesian networks, with a view towards generalizing the recursive factorization theorem to situations with hidden variables. In ..."
Abstract

Cited by 9 (1 self)
 Add to MetaCart
(Show Context)
Conditional independence models in the Gaussian case are algebraic varieties in the cone of positive definite covariance matrices. We study these varieties in the case of Bayesian networks, with a view towards generalizing the recursive factorization theorem to situations with hidden variables. In the case when the underlying graph is a tree, we show that the vanishing ideal of the model is generated by the conditional independence statements implied by graph. We also show that the ideal of any Bayesian network is homogeneous with respect to a multigrading induced by a collection of upstream random variables. This has a number of important consequences for hidden variable models. Finally, we relate the ideals of Bayesian networks to a number of classical constructions in algebraic geometry including toric degenerations of the Grassmannian, matrix Schubert varieties, and secant varieties.
Geometry of the Restricted Boltzmann Machine
 Algebraic methods in statistics and probability II, AMS Special Session. AMS
, 2010
"... Abstract. The restricted Boltzmann machine is a graphical model for binary random variables. Based on a complete bipartite graph separating hidden and observed variables, it is the binary analog to the factor analysis model. We study this graphical model from the perspectives of algebraic statistics ..."
Abstract

Cited by 8 (1 self)
 Add to MetaCart
(Show Context)
Abstract. The restricted Boltzmann machine is a graphical model for binary random variables. Based on a complete bipartite graph separating hidden and observed variables, it is the binary analog to the factor analysis model. We study this graphical model from the perspectives of algebraic statistics and tropical geometry, starting with the observation that its Zariski closure is a Hadamard power of the first secant variety of the Segre variety of projective lines. We derive a dimension formula for the tropicalized model, and we use it to show that the restricted Boltzmann machine is identifiable in many cases. Our methods include coding theory and geometry of linear threshold functions. 1.
Equivariant Gröbner bases and the Gaussian twofactor model
, 2009
"... We show that the kernel I of the ring homomorphism R[yij  i, j ∈ N, i> j] → R[si, ti  i ∈ N] determined by yij ↦ → sisj +titj is generated by two types of polynomials: offdiagonal 3 × 3minors and pentads. This confirms a conjecture by Drton, Sturmfels, and Sullivant on the Gaussian twofacto ..."
Abstract

Cited by 6 (0 self)
 Add to MetaCart
(Show Context)
We show that the kernel I of the ring homomorphism R[yij  i, j ∈ N, i> j] → R[si, ti  i ∈ N] determined by yij ↦ → sisj +titj is generated by two types of polynomials: offdiagonal 3 × 3minors and pentads. This confirms a conjecture by Drton, Sturmfels, and Sullivant on the Gaussian twofactor model. Our proof is computational: inspired by work of Aschenbrenner and Hillar we introduce the concept of GGröbner basis, where G is a monoid acting on an infinite set of variables, and we report on a computation that yielded a finite GGröbner basis of I relative to the monoid G of strictly increasing functions N → N.
FINITENESS FOR THE kFACTOR MODEL AND CHIRALITY VARIETIES
, 2008
"... This paper deals with two families of algebraic varieties arising from applications. First, the kfactor model in statistics, consisting of n×n covariance matrices of n observed Gaussian variables that are pairwise independent given k hidden Gaussian variables. Second, chirality varieties inspired ..."
Abstract

Cited by 6 (1 self)
 Add to MetaCart
This paper deals with two families of algebraic varieties arising from applications. First, the kfactor model in statistics, consisting of n×n covariance matrices of n observed Gaussian variables that are pairwise independent given k hidden Gaussian variables. Second, chirality varieties inspired by applications in chemistry. A point in such a chirality variety records chirality measurements of all ksubsets among an nset of ligands. Both classes of varieties are given by a parameterisation, while for applications having polynomial equations would be desirable. For instance, such equations could be used to test whether a given point lies in the variety. We prove that in a precise sense, which is different for the two classes of varieties, these equations are finitely characterisable when k is fixed and n grows.
Algebraic techniques for Gaussian models
 In Prague Stochastics (M. Huskova, M. Janzura Eds
, 2006
"... ..."
(Show Context)
Bayesian learning of measurement and structural models
 23rd International Conference on Machine Learning
, 2006
"... We present a Bayesian search algorithm for learning the structure of latent variable models of continuous variables. We stress the importance of applying search operators designed especially for the parametric family used in our models. This is performed by searching for subsets of the observed vari ..."
Abstract

Cited by 5 (3 self)
 Add to MetaCart
(Show Context)
We present a Bayesian search algorithm for learning the structure of latent variable models of continuous variables. We stress the importance of applying search operators designed especially for the parametric family used in our models. This is performed by searching for subsets of the observed variables whose covariance matrix can be represented as a sum of a matrix of low rank and a diagonal matrix of residuals. The resulting search procedure is relatively efficient, since the main search operator has a branch factor that grows linearly with the number of variables. The resulting models are often simpler and give a better fit than models based on generalizations of factor analysis or those derived from standard hillclimbing methods. 1.
FINITE GRÖBNER BASES IN INFINITE DIMENSIONAL POLYNOMIAL RINGS AND APPLICATIONS
"... Abstract. We introduce the theory of monoidal Gröbner bases, a concept which generalizes the familiar notion in a polynomial ring and allows for a description of Gröbner bases of ideals that are stable under the action of a monoid. The main motivation for developing this theory is to prove finitenes ..."
Abstract

Cited by 5 (1 self)
 Add to MetaCart
(Show Context)
Abstract. We introduce the theory of monoidal Gröbner bases, a concept which generalizes the familiar notion in a polynomial ring and allows for a description of Gröbner bases of ideals that are stable under the action of a monoid. The main motivation for developing this theory is to prove finiteness results in commutative algebra and applications. A basic theorem of this type is that ideals in infinitely many indeterminates stable under the action of the symmetric group are finitely generated up to symmetry. Using this machinery, we give new streamlined proofs of some classical finiteness theorems in algebraic statistics as well as a proof of the independent set conjecture of Ho¸sten and the second author. 1.
Open problems in algebraic statistics
"... Abstract. Algebraic statistics is concerned with the study of probabilistic models and techniques for statistical inference using methods from algebra and geometry. This article presents a list of open mathematical problems in this emerging field, with main emphasis on graphical models with hidden v ..."
Abstract

Cited by 4 (2 self)
 Add to MetaCart
Abstract. Algebraic statistics is concerned with the study of probabilistic models and techniques for statistical inference using methods from algebra and geometry. This article presents a list of open mathematical problems in this emerging field, with main emphasis on graphical models with hidden variables, maximum likelihood estimation, and multivariate Gaussian distributions. These are notes from a lecture presented at the IMA in Minneapolis during the 2006/07 program on Applications of Algebraic Geometry. Key words. Algebraic statistics, contingency tables, hidden variables, Schur modules,