Results 1  10
of
70
A unified framework for highdimensional analysis of Mestimators with decomposable regularizers
"... ..."
Latent Variable Graphical Model Selection via Convex Optimization
, 2010
"... Suppose we have samples of a subset of a collection of random variables. No additional information is provided about the number of latent variables, nor of the relationship between the latent and observed variables. Is it possible to discover the number of hidden components, and to learn a statistic ..."
Abstract

Cited by 75 (5 self)
 Add to MetaCart
Suppose we have samples of a subset of a collection of random variables. No additional information is provided about the number of latent variables, nor of the relationship between the latent and observed variables. Is it possible to discover the number of hidden components, and to learn a statistical model over the entire collection of variables? We address this question in the setting in which the latent and observed variables are jointly Gaussian, with the conditional statistics of the observed variables conditioned on the latent variables being specified by a graphical model. As a first step we give natural conditions under which such latentvariable Gaussian graphical models are identifiable given marginal statistics of only the observed variables. Essentially these conditions require that the conditional graphical model among the observed variables is sparse, while the effect of the latent variables is “spread out ” over most of the observed variables. Next we propose a tractable convex program based on regularized maximumlikelihood for model selection in this latentvariable setting; the regularizer uses both the ℓ1 norm and the nuclear norm. Our modeling framework can be viewed as a combination of dimensionality reduction (to identify latent variables) and graphical modeling (to capture remaining statistical structure not attributable to the latent variables), and it consistently estimates both the number of hidden components and the conditional graphical model structure among the observed variables. These results are applicable in the highdimensional setting in which the number of latent/observed variables grows with the number of samples of the observed variables. The geometric properties of the algebraic varieties of sparse matrices and of lowrank matrices play an important role in our analysis.
Sparse Inverse Covariance Matrix Estimation Using Quadratic Approximation
"... The ℓ1 regularized Gaussian maximum likelihood estimator has been shown to have strong statistical guarantees in recovering a sparse inverse covariance matrix, or alternatively the underlying graph structure of a Gaussian Markov Random Field, from very limited samples. We propose a novel algorithm f ..."
Abstract

Cited by 64 (9 self)
 Add to MetaCart
The ℓ1 regularized Gaussian maximum likelihood estimator has been shown to have strong statistical guarantees in recovering a sparse inverse covariance matrix, or alternatively the underlying graph structure of a Gaussian Markov Random Field, from very limited samples. We propose a novel algorithm for solving the resulting optimization problem which is a regularized logdeterminant program. In contrast to other stateoftheart methods that largely use first order gradient information, our algorithm is based on Newton’s method and employs a quadratic approximation, but with some modifications that leverage the structure of the sparse Gaussian MLE problem. We show that our method is superlinearly convergent, and also present experimental results using synthetic and real application data that demonstrate the considerable improvements in performance of our method when compared to other stateoftheart methods. 1
Model selection in gaussian graphical models: Highdimensional consistency of l1regularized MLE
, 2008
"... We consider the problem of estimating the graph structure associated with a Gaussian Markov random field (GMRF) from i.i.d. samples. We study the performance of study the performance of the ℓ1regularized maximum likelihood estimator in the highdimensional setting, where the number of nodes in the ..."
Abstract

Cited by 43 (1 self)
 Add to MetaCart
(Show Context)
We consider the problem of estimating the graph structure associated with a Gaussian Markov random field (GMRF) from i.i.d. samples. We study the performance of study the performance of the ℓ1regularized maximum likelihood estimator in the highdimensional setting, where the number of nodes in the graph p, the number of edges in the graph s and the maximum node degree d, are allowed to grow as a function of the number of samples n. Our main result provides sufficient conditions on (n, p, d) for the ℓ1regularized MLE estimator to recover all the edges of the graph with high probability. Under some conditions on the model covariance, we show that model selection can be achieved for sample sizes n = Ω(d 2 log(p)), with the error decaying as O(exp(−clog(p))) for some constant c. We illustrate our theoretical results via simulations and show good correspondences between the theoretical predictions and behavior in simulations. 1
Informationtheoretic limits of selecting binary graphical models in high dimensions
 in Proc. of IEEE Intl. Symp. on Inf. Theory
, 2008
"... in high dimensions ..."
Structure estimation for discrete graphical models: Generalized covariance matrices and their inverses”. In: arXiv eprints
, 2012
"... We investigate the relationship between the structure of a discrete graphical model and the support of the inverse of a generalized covariance matrix. We show that for certain graph structures, the support of the inverse covariance matrix of indicator variables on the vertices of a graph reflects th ..."
Abstract

Cited by 18 (4 self)
 Add to MetaCart
We investigate the relationship between the structure of a discrete graphical model and the support of the inverse of a generalized covariance matrix. We show that for certain graph structures, the support of the inverse covariance matrix of indicator variables on the vertices of a graph reflects the conditional independence structure of the graph. Our work extends results that have previously been established only in the context of multivariate Gaussian graphical models, thereby addressing an open question about the significance of the inverse covariance matrix of a nonGaussian distribution. The proof exploits acombination of ideas from thegeometryof exponentialfamilies, junctiontreetheory,andconvexanalysis. Thesepopulationlevel results have various consequences for graph selection methods, both known and novel, including a novel method for structure estimation for missing or corrupted observations. We provide nonasymptotic guarantees for such methods, and illustrate the sharpness of these predictions via simulations. 1. Introduction. Graphical
HighDimensional Gaussian Graphical Model Selection: Walk Summability AND Local Separation Criterion
 JOURNAL OF MACHINE LEARNING RESEARCH
, 2012
"... We consider the problem of highdimensional Gaussian graphical model selection. We identify a set ofgraphsforwhich anefficient estimation algorithmexists, and this algorithm is based on thresholding of empirical conditional covariances. Under a set of transparent conditions, we establish structuralc ..."
Abstract

Cited by 17 (3 self)
 Add to MetaCart
(Show Context)
We consider the problem of highdimensional Gaussian graphical model selection. We identify a set ofgraphsforwhich anefficient estimation algorithmexists, and this algorithm is based on thresholding of empirical conditional covariances. Under a set of transparent conditions, we establish structuralconsistency (orsparsistency) forthe proposedalgorithm, when the number of samples n = Ω(J −2 minlogp), where p is the number of variables and Jmin is the minimum (absolute) edge potential of the graphical model. The sufficient conditions for sparsistency are based on the notion of walksummability of the model and the presence of sparse local vertex separators in the underlying graph. We also derive novel nonasymptotic necessary conditions on the number of samples required for sparsistency.
Topology selection in graphical models of autoregressive processes
 JOURNAL OF MACHINE LEARNING RESEARCH
, 2010
"... An algorithm is presented for topology selection in graphical models of autoregressive Gaussian time series. The graph topology of the model represents the sparsity pattern of the inverse spectrum of the time series and characterizes conditional independence relations between the variables. The meth ..."
Abstract

Cited by 16 (0 self)
 Add to MetaCart
An algorithm is presented for topology selection in graphical models of autoregressive Gaussian time series. The graph topology of the model represents the sparsity pattern of the inverse spectrum of the time series and characterizes conditional independence relations between the variables. The method proposed in the paper is based on an ℓ1type nonsmooth regularization of the conditional maximum likelihood estimation problem. We show that this reduces to a convex optimization problem and describe a largescale algorithm that solves the dual problem via the gradient projection method. Results of experiments with randomly generated and real data sets are also included.
Informationtheoretic bounds on model selection for Gaussian markov random fields
, 2010
"... Abstract—The problem of graphical model selection is to estimate the graph structure of an unknown Markov random field based on observed samples from the graphical model. For Gaussian Markov random fields, this problem is closely related to the problem of estimating the inverse covariance matrix of ..."
Abstract

Cited by 15 (3 self)
 Add to MetaCart
(Show Context)
Abstract—The problem of graphical model selection is to estimate the graph structure of an unknown Markov random field based on observed samples from the graphical model. For Gaussian Markov random fields, this problem is closely related to the problem of estimating the inverse covariance matrix of the underlying Gaussian distribution. This paper focuses on the informationtheoretic limitations of Gaussian graphical model selection and inverse covariance estimation in the highdimensional setting, in which the graph size p and maximum node degree d are allowed to grow as a function of the sample size n. Our first result establishes a set of necessary conditions on n(p, d) for any recovery method to consistently estimate the underlying graph. Our second result provides necessary conditions for any decoder to produce an estimate b Θ of the true inverse covariance matrix Θ satisfying ‖ b Θ − Θ ‖ < δ in the elementwise ℓ∞norm (which implies analogous results in the Frobenius norm as well). Combined with previously known sufficient conditions for polynomialtime algorithms, these results yield sharp characterizations in several regimes of interest. I.
Strong oracle optimality of folded concave penalized estimation
 Ann. Statist
, 2014
"... All intext references underlined in blue are linked to publications on ResearchGate, letting you access and read them immediately. ..."
Abstract

Cited by 13 (4 self)
 Add to MetaCart
(Show Context)
All intext references underlined in blue are linked to publications on ResearchGate, letting you access and read them immediately.