Results 1  10
of
40
A unified framework for highdimensional analysis of Mestimators with decomposable regularizers
"... ..."
A Dirty Model for Multitask Learning
 In NIPS
, 2010
"... We consider multitask learning in the setting of multiple linear regression, and where some relevant features could be shared across the tasks. Recent research has studied the use ofℓ1/ℓq norm blockregularizations withq> 1 for such blocksparse structured problems, establishing strong guarantees on ..."
Abstract

Cited by 20 (0 self)
 Add to MetaCart
We consider multitask learning in the setting of multiple linear regression, and where some relevant features could be shared across the tasks. Recent research has studied the use ofℓ1/ℓq norm blockregularizations withq> 1 for such blocksparse structured problems, establishing strong guarantees on recovery even under highdimensional scaling where the number of features scale with the number of observations. However, these papers also caution that the performance of such blockregularized methods are very dependent on the extent to which the features are shared across tasks. Indeed they show [8] that if the extent of overlap is less than a threshold, or even if parameter values in the shared features are highly uneven, then block ℓ1/ℓq regularization could actually perform worse than simple separate elementwise ℓ1 regularization. Since these caveats depend on the unknown true parameters, we might not know when and which method to apply. Even otherwise, we are far away from a realistic multitask setting: not only do the set of relevant features have to be exactly the same across tasks, but their values
Simultaneous support recovery in high dimensions: Benefits and perils of block ℓ1/ℓ∞regularization
, 2010
"... ..."
Joint Estimation of Multiple Graphical Models
, 2009
"... Gaussian graphical models explore dependence relationships between random variables, through estimation of the corresponding inverse covariance (precision) matrices. The objective of this paper is to develop an estimator for such models appropriate for heterogeneous data; specifically, data obtained ..."
Abstract

Cited by 9 (0 self)
 Add to MetaCart
Gaussian graphical models explore dependence relationships between random variables, through estimation of the corresponding inverse covariance (precision) matrices. The objective of this paper is to develop an estimator for such models appropriate for heterogeneous data; specifically, data obtained from different categories that share some common structure, but also exhibit differences. An example of such a data structure is gene networks corresponding to different subtypes of a certain disease. In this setting, estimating a single graphical model would mask the underlying heterogeneity, while estimating separate models for each category ignores the common structure. We propose a method which jointly estimates several graphical models corresponding to the different categories present in the data. The method aims to preserve the common structure, while allowing for differences between the categories. This is achieved through a hierarchical penalty that targets the removal of common zeros in the precision matrices across categories. We establish the asymptotic consistency and sparsistency of the proposed estimator in the highdimensional case, and illustrate its
On learning discrete graphical models using greedy methods
 In Neural Information Processing Systems (NIPS) (currently under review
, 2011
"... In this paper, we address the problem of learning the structure of a pairwise graphical model from samples in a highdimensional setting. Our first main result studies the sparsistency, or consistency in sparsity pattern recovery, properties of a forwardbackward greedy algorithm as applied to gener ..."
Abstract

Cited by 9 (4 self)
 Add to MetaCart
In this paper, we address the problem of learning the structure of a pairwise graphical model from samples in a highdimensional setting. Our first main result studies the sparsistency, or consistency in sparsity pattern recovery, properties of a forwardbackward greedy algorithm as applied to general statistical models. As a special case, we then apply this algorithm to learn the structure of a discrete graphical model via neighborhood estimation. As a corollary of our general result, we derive sufficient conditions on the number of samples n, the maximum nodedegreed and the problem size p, as well as other conditions on the model parameters, so that the algorithm recovers all the edges with high probability. Our result guarantees graph selection for samples scaling asn = Ω(d 2 log(p)), in contrast to existing convexoptimization based algorithms that require a sample complexity of Ω(d 3 log(p)). Further, the greedy algorithm only requires a restricted strong convexity condition which is typically milder than irrepresentability assumptions. We corroborate these results using numerical simulations at the end. 1
On Learning Discrete Graphical Models using GroupSparse
"... We study the problem of learning the graph structure associated with a general discrete graphical models (each variable can take any of m> 1 values, the clique factors have maximum size c ≥ 2) from samples, under highdimensional scaling where the number of variables p could be larger than the numbe ..."
Abstract

Cited by 6 (2 self)
 Add to MetaCart
We study the problem of learning the graph structure associated with a general discrete graphical models (each variable can take any of m> 1 values, the clique factors have maximum size c ≥ 2) from samples, under highdimensional scaling where the number of variables p could be larger than the number of samples n. We provide a quantitative consistency analysis of a procedure based on nodewise multiclass logistic regression with groupsparse regularization. We first consider general mary pairwise models – where each factor depends on at most two variables. We show that when
On Time Varying Undirected Graphs
"... The timevarying multivariate Gaussian distribution and the undirected graph associated with it, as introduced in Zhou et al. (2008), provide a useful statistical framework for modeling complex dynamic networks. In many application domains, it is of high importance to estimate the graph structure of ..."
Abstract

Cited by 4 (1 self)
 Add to MetaCart
The timevarying multivariate Gaussian distribution and the undirected graph associated with it, as introduced in Zhou et al. (2008), provide a useful statistical framework for modeling complex dynamic networks. In many application domains, it is of high importance to estimate the graph structure of the model consistently for the purpose of scientific discovery. In this paper, we show that under suitable technical conditions, the structure of the undirected graphical model can be consistently estimated in the high dimensional setting, when the dimensionality of the model is allowed to diverge with the sample size. The model selection consistency is shown for the procedure proposed in Zhou et al. (2008) and for the modified neighborhood selection procedure of Meinshausen and Bühlmann (2006). 1
Inferring multiple graphical structures
 Stat. Comput
, 2011
"... Abstract: Gaussian Graphical Models provide a convenient framework for representing dependencies between variables. Recently, this tool has received a high interest for the discovery of biological networks. The literature focuses on the case where a single network is inferred from a set of measureme ..."
Abstract

Cited by 3 (0 self)
 Add to MetaCart
Abstract: Gaussian Graphical Models provide a convenient framework for representing dependencies between variables. Recently, this tool has received a high interest for the discovery of biological networks. The literature focuses on the case where a single network is inferred from a set of measurements, but, as wetlab data is typically scarce, several assays, where the experimental conditions affect interactions, are usually merged to infer a single network. In this paper, we propose two approaches for estimating multiple related graphs, by rendering the closeness assumption into an empirical prior or group penalties. We provide quantitative results demonstrating the benefits of the proposed approaches. The methods presented in this paper are embeded in the R package simone from version 1.00 and
On the Use of Variational Inference for Learning Discrete Graphical Models
"... We study the general class of estimators for graphical model structure based on optimizing ℓ1regularized approximate loglikelihood, where the approximate likelihood uses tractable variational approximations of the partition function. We provide a messagepassing algorithm that directly computes the ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
We study the general class of estimators for graphical model structure based on optimizing ℓ1regularized approximate loglikelihood, where the approximate likelihood uses tractable variational approximations of the partition function. We provide a messagepassing algorithm that directly computes the ℓ1 regularized approximate MLE. Further, in the case of certain reweighted entropy approximations to the partition function, we show that surprisingly the ℓ1 regularized approximate MLE estimator has a closedform, so that we would no longer need to run through many iterations of approximate inference and messagepassing. Lastly, we analyze this general class of estimators for graph structure recovery, or its sparsistency, and show that it is indeed sparsistent under certain conditions. 1.
Graphical Models via Generalized Linear Models
"... Undirected graphical models, also known as Markov networks, enjoy popularity in a variety of applications. The popular instances of these models such as Gaussian Markov Random Fields (GMRFs), Ising models, and multinomial discrete models, however do not capture the characteristics of data in many se ..."
Abstract

Cited by 2 (1 self)
 Add to MetaCart
Undirected graphical models, also known as Markov networks, enjoy popularity in a variety of applications. The popular instances of these models such as Gaussian Markov Random Fields (GMRFs), Ising models, and multinomial discrete models, however do not capture the characteristics of data in many settings. We introduce a new class of graphical models based on generalized linear models (GLMs) by assuming that nodewise conditional distributions arise from exponential families. Our models allow one to estimate multivariate Markov networks given any univariate exponential distribution, such as Poisson, negative binomial, and exponential, by fitting penalized GLMs to select the neighborhood for each node. A major contribution of this paper is the rigorous statistical analysis showing that with high probability, the neighborhood of our graphical models can be recovered exactly. We also provide examples of nonGaussian highthroughput genomic networks learned via our GLM graphical models. 1