Results 1  10
of
47
Firstorder methods for sparse covariance selection
 SIAM Journal on Matrix Analysis and Applications
"... Abstract. Given a sample covariance matrix, we solve a maximum likelihood problem penalized by the number of nonzero coefficients in the inverse covariance matrix. Our objective is to find a sparse representation of the sample data and to highlight conditional independence relationships between the ..."
Abstract

Cited by 56 (1 self)
 Add to MetaCart
Abstract. Given a sample covariance matrix, we solve a maximum likelihood problem penalized by the number of nonzero coefficients in the inverse covariance matrix. Our objective is to find a sparse representation of the sample data and to highlight conditional independence relationships between the sample variables. We first formulate a convex relaxation of this combinatorial problem, we then detail two efficient firstorder algorithms with low memory requirements to solve largescale, dense problem instances.
A robust procedure for gaussian graphical model search from microarray data with p larger than n
 Journal of Machine Learning Research
, 2006
"... Learning of largescale networks of interactions from microarray data is an important and challenging problem in bioinformatics. A widely used approach is to assume that the available data constitute a random sample from a multivariate distribution belonging to a Gaussian graphical model. As a conse ..."
Abstract

Cited by 26 (3 self)
 Add to MetaCart
Learning of largescale networks of interactions from microarray data is an important and challenging problem in bioinformatics. A widely used approach is to assume that the available data constitute a random sample from a multivariate distribution belonging to a Gaussian graphical model. As a consequence, the prime objects of inference are fullorder partial correlations which are partial correlations between two variables given the remaining ones. In the context of microarray data the number of variables exceed the sample size and this precludes the application of traditional structure learning procedures because a sampling version of fullorder partial correlations does not exist. In this paper we consider limitedorder partial correlations, these are partial correlations computed on marginal distributions of manageable size, and provide a set of rules that allow one to assess the usefulness of these quantities to derive the independence structure of the underlying Gaussian graphical model. Furthermore, we introduce a novel structure learning procedure based on a quantity, obtained from limitedorder partial correlations, that we call the nonrejection rate. The applicability and usefulness of the procedure are demonstrated by both simulated and real data.
Shotgun stochastic search for “large p” regression
 Journal of the American Statistical Association
, 2007
"... Model search in regression with very large numbers of candidate predictors raises challenges for both model specification and computation, and standard approaches such as Markov chain Monte Carlo (MCMC) and stepwise methods are often infeasible or ineffective. We describe a novel shotgun stochastic ..."
Abstract

Cited by 17 (3 self)
 Add to MetaCart
Model search in regression with very large numbers of candidate predictors raises challenges for both model specification and computation, and standard approaches such as Markov chain Monte Carlo (MCMC) and stepwise methods are often infeasible or ineffective. We describe a novel shotgun stochastic search (SSS) approach that explores “interesting” regions of the resulting, very highdimensional model spaces to quickly identify regions of high posterior probability over models. We describe algorithmic and modeling aspects, priors over the model space that induce sparsity and parsimony over and above the traditional dimension penalization implicit in Bayesian and likelihood analyses, and parallel computation using cluster computers. We discuss an example from gene expression cancer genomics, comparisons with MCMC and other methods, and theoretical and simulationbased aspects of performance characteristics in largescale regression model search. We also provide software implementing the methods.
Flexible covariance estimation in graphical Gaussian models
 ANNALS OF STATISTICS
, 2008
"... In this paper, we propose a class of Bayes estimators for the covariance matrix of graphical Gaussian models Markov with respect to a decomposable graph G. Working with the WP G family defined by Letac and Massam [Ann. Statist. 35 (2007) 1278–1323] we derive closedform expressions for Bayes estimat ..."
Abstract

Cited by 16 (3 self)
 Add to MetaCart
In this paper, we propose a class of Bayes estimators for the covariance matrix of graphical Gaussian models Markov with respect to a decomposable graph G. Working with the WP G family defined by Letac and Massam [Ann. Statist. 35 (2007) 1278–1323] we derive closedform expressions for Bayes estimators under the entropy and squarederror losses. The WP G family includes the classical inverse of the hyper inverse Wishart but has many more shape parameters, thus allowing for flexibility in differentially shrinking various parts of the covariance matrix. Moreover, using this family avoids recourse to MCMC, often infeasible in highdimensional problems. We illustrate the performance of our estimators through a collection of numerical examples where we explore frequentist risk properties and the efficacy of graphs in the estimation of highdimensional covariance structures.
Smooth optimization approach for sparse covariance selection
 SIAM J. Optim
"... In this paper we first study a smooth optimization approach for solving a class of nonsmooth strictly concave maximization problems whose objective functions admit smooth convex minimization reformulations. In particular, we apply Nesterov’s smooth optimization technique [19, 21] to their dual coun ..."
Abstract

Cited by 16 (2 self)
 Add to MetaCart
In this paper we first study a smooth optimization approach for solving a class of nonsmooth strictly concave maximization problems whose objective functions admit smooth convex minimization reformulations. In particular, we apply Nesterov’s smooth optimization technique [19, 21] to their dual counterparts that are smooth convex problems. It is shown that the resulting approach has O(1 / √ ǫ) iteration complexity for finding an ǫoptimal solution to both primal and dual problems. We then discuss the application of this approach to sparse covariance selection that is approximately solved as a l1norm penalized maximum likelihood estimation problem, and also propose a variant of this approach which has substantially outperformed the latter one in our computational experiments. We finally compare the performance of these approaches with other firstorder methods, namely, Nesterov’s O(1/ǫ) smooth approximation scheme and blockcoordinate descent method studied in [9, 15] for sparse covariance selection on a set of randomly generated instances. It shows that our smooth optimization approach substantially outperforms the first method above, and moreover, its variant substantially outperforms both methods above.
Dynamic matrixvariate graphical models
 Bayesian Anal
, 2007
"... This paper introduces a novel class of Bayesian models for multivariate time series analysis based on a synthesis of dynamic linear models and graphical models. The models are then applied in the context of financial time series for predictive portfolio analysis providing a significant improvement i ..."
Abstract

Cited by 15 (2 self)
 Add to MetaCart
This paper introduces a novel class of Bayesian models for multivariate time series analysis based on a synthesis of dynamic linear models and graphical models. The models are then applied in the context of financial time series for predictive portfolio analysis providing a significant improvement in performance of optimal investment decisions.
Objective Bayesian model selection in Gaussian graphical models
, 2007
"... This paper presents a default modelselection procedure for Gaussian graphical models that involves two new developments. First, we develop a default version of the hyperinverse Wishart prior for restricted covariance matrices, called the hyperinverse Wishart gprior, and show how it corresponds t ..."
Abstract

Cited by 12 (3 self)
 Add to MetaCart
This paper presents a default modelselection procedure for Gaussian graphical models that involves two new developments. First, we develop a default version of the hyperinverse Wishart prior for restricted covariance matrices, called the hyperinverse Wishart gprior, and show how it corresponds to the implied fractional prior for covariance selection using fractional Bayes factors. Second, we apply a class of priors that automatically handles the problem of multiple hypothesis testing implied by covariance selection. We demonstrate our methods on a variety of simulated examples, concluding with a real example analysing covariation in mutualfund returns. These studies reveal that the combined use of a multiplicitycorrection prior on graphs and fractional Bayes factors for computing marginal likelihoods yields better performance than existing Bayesian methods. Some key words: covariance selection; hyperinverse Wishart distribution; fractional Bayes factors; Bayesian model selection; multiple hypothesis testing.
Multiple testing and error control in Gaussian graphical model selection
 Statistical Science
"... Abstract. Graphical models provide a framework for exploration of multivariate dependence patterns. The connection between graph and statistical model is made by identifying the vertices of the graph with the observed variables and translating the pattern of edges in the graph into a pattern of cond ..."
Abstract

Cited by 12 (2 self)
 Add to MetaCart
Abstract. Graphical models provide a framework for exploration of multivariate dependence patterns. The connection between graph and statistical model is made by identifying the vertices of the graph with the observed variables and translating the pattern of edges in the graph into a pattern of conditional independences that is imposed on the variables ’ joint distribution. Focusing on Gaussian models, we review classical graphical models. For these models the defining conditional independences are equivalent to vanishing of certain (partial) correlation coefficients associated with individual edges that are absent from the graph. Hence, Gaussian graphical model selection can be performed by multiple testing of hypotheses about vanishing (partial) correlation coefficients. We show and exemplify how this approach allows one to perform model selection while controlling error rates for incorrect edge inclusion. Key words and phrases: Acyclic directed graph, Bayesian network, bidirected graph, chain graph, concentration graph, covariance graph, DAG, graphical model, multiple testing, undirected graph. 1.
Simulation of hyperinverse Wishart distributions in graphical models
, 2007
"... We introduce and exemplify an efficient method for direct sampling from hyperinverse Wishart distributions. The method relies very naturally on the use of standard junctiontree representation of graphs, and couples these with matrix results for inverse Wishart distributions. We describe the theory ..."
Abstract

Cited by 11 (3 self)
 Add to MetaCart
We introduce and exemplify an efficient method for direct sampling from hyperinverse Wishart distributions. The method relies very naturally on the use of standard junctiontree representation of graphs, and couples these with matrix results for inverse Wishart distributions. We describe the theory and resulting computational algorithms for both decomposable and nondecomposable graphical models. An example drawn from financial time series demonstrates application in a context where inferences on a structured covariance model are required. We discuss and investigate questions of scalability of the simulation methods to higherdimensional distributions. The paper concludes with general comments about the approach, including its use in connection with existing Markov chain Monte Carlo methods that deal with uncertainty about the graphical model structure.
ALTERNATING DIRECTION METHODS FOR SPARSE COVARIANCE SELECTION
, 2009
"... The mathematical model of the widelyused sparse covariance selection problem (SCSP) is an NPhard combinatorial problem, whereas it can be well approximately by a convex relaxation problem whose maximum likelihood estimation is penalized by the L1 norm. This convex relaxation problem, however, is ..."
Abstract

Cited by 10 (1 self)
 Add to MetaCart
The mathematical model of the widelyused sparse covariance selection problem (SCSP) is an NPhard combinatorial problem, whereas it can be well approximately by a convex relaxation problem whose maximum likelihood estimation is penalized by the L1 norm. This convex relaxation problem, however, is still numerically challenging, especially for largescale cases. Recently, some efficient firstorder methods inspired by Nesterov’s work have been proposed to solve the convex relaxation problem of SCSP. This paper is to apply the wellknown alternating direction method (ADM), which is also a firstorder method, to solve the convex relaxation of SCSP. Due to the full exploitation to the separable structure of a simple reformulation of the convex relaxation problem, the ADM approach is very efficient for solving largescale SCSP. Our preliminary numerical results show that the ADM approach substantially outperforms existing firstorder methods for SCSP.