Results 1  10
of
24
Noisy matrix decomposition via convexrelaxation: Optimal rates in high dimensions
 Annals of Statistics,40(2):1171
"... We analyze a class of estimators based on convex relaxation for solving highdimensional matrix decomposition problems. The observations are noisy realizations of a linear transformation X of the sum of an (approximately) low rank matrix � ⋆ with a second matrix Ɣ ⋆ endowed with a complementary for ..."
Abstract

Cited by 17 (7 self)
 Add to MetaCart
We analyze a class of estimators based on convex relaxation for solving highdimensional matrix decomposition problems. The observations are noisy realizations of a linear transformation X of the sum of an (approximately) low rank matrix � ⋆ with a second matrix Ɣ ⋆ endowed with a complementary form of lowdimensional structure; this setup includes many statistical models of interest, including factor analysis, multitask regression and robust covariance estimation. We derive a general theorem that bounds the Frobenius norm error for an estimate of the pair ( � ⋆,Ɣ ⋆ ) obtained by solving a convex optimization problem that combines the nuclear norm with a general decomposable regularizer. Our results use a “spikiness ” condition that is related to, but milder than, singular vector incoherence. We specialize our general result to two cases that have been studied in past work: low rank plus an entrywise sparse matrix, and low rank plus a columnwise sparse matrix. For both models, our theory yields nonasymptotic Frobenius error bounds for both deterministic and stochastic noise matrices, and applies to matrices � ⋆ that can be exactly or approximately low rank, and matrices Ɣ ⋆ that can be exactly or approximately sparse. Moreover, for the case of stochastic noise matrices and the identity observation operator, we establish matching lower bounds on the minimax error. The sharpness of our nonasymptotic predictions is confirmed by numerical simulations. 1. Introduction. The
Robust Matrix Decomposition with Sparse Corruptions
"... Abstract—Suppose a given observation matrix can be decomposed as the sum of a lowrank matrix and a sparse matrix, and the goal is to recover these individual components from the observed sum. Such additive decompositions have applications in a variety of numerical problems including system identifi ..."
Abstract

Cited by 14 (1 self)
 Add to MetaCart
Abstract—Suppose a given observation matrix can be decomposed as the sum of a lowrank matrix and a sparse matrix, and the goal is to recover these individual components from the observed sum. Such additive decompositions have applications in a variety of numerical problems including system identification, latent variable graphical modeling, and principal components analysis. We study conditions under which recovering such a decomposition is possible via a combination of ℓ1 norm and trace norm minimization. We are specifically interested in the question of how many sparse corruptions are allowed so that convex programming can still achieve accurate recovery, and we obtain stronger recovery guarantees than previous studies. Moreover, we do not assume that the spatial pattern of corruptions is random, which stands in contrast to related analyses under such assumptions via matrix completion. Index Terms—Matrix decompositions, sparsity, lowrank, outliers
Robust Matrix Decomposition with Outliers
, 2010
"... Suppose a given observation matrix can be decomposed as the sum of a lowrank matrix and a sparse matrix (outliers), and the goal is to recover these individual components from the observed sum. Such additive decompositions have applications in a variety of numerical problems including system identi ..."
Abstract

Cited by 9 (1 self)
 Add to MetaCart
Suppose a given observation matrix can be decomposed as the sum of a lowrank matrix and a sparse matrix (outliers), and the goal is to recover these individual components from the observed sum. Such additive decompositions have applications in a variety of numerical problems including system identification, latent variable graphical modeling, and principal components analysis. We study conditions under which recovering such a decomposition is possible via a combination of ℓ1 norm and trace norm minimization. We are specifically interested in the question of how many outliers are allowed so that convex programming can still achieve accurate recovery, and we obtain stronger recovery guarantees than previous studies. Moreover, we do not assume that the spatial pattern of outliers is random, which stands in contrast to related analyses under such assumptions via matrix completion. 1
Copula Gaussian Graphical Models with Hidden Variables
 Proc. IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP) 2012
, 2012
"... Gaussian hidden variable graphical models are powerful tools to describe highdimensional data; they capture dependencies between observed (Gaussian) variables by introducing a suitable number of hidden variables. However, such models are only applicable to Gaussian data. Moreover, they are sensitiv ..."
Abstract

Cited by 2 (2 self)
 Add to MetaCart
Gaussian hidden variable graphical models are powerful tools to describe highdimensional data; they capture dependencies between observed (Gaussian) variables by introducing a suitable number of hidden variables. However, such models are only applicable to Gaussian data. Moreover, they are sensitive to the choice of certain regularization parameters. In this paper, (1) copula Gaussian hidden variable graphical models are introduced, which extend Gaussian hidden variable graphical models to nonGaussian data; (2) the sparsity pattern of the hidden variable graphical model is learned via stability selection, which leads to more stable results than crossvalidation and other methods to select the regularization parameters. The proposed methods are validated on synthetic and real data. Index Terms — Gaussian copula, hidden variable graphical model, stability selection, bioinformatics 1.
Learning Linear Bayesian Networks with Latent Variables
"... This work considers the problem of learning linear Bayesian networks when some of the variables are unobserved. Identifiability and efficient recovery from loworder observable moments are established under a novel graphical constraint. The constraint concerns the expansion properties of the underly ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
This work considers the problem of learning linear Bayesian networks when some of the variables are unobserved. Identifiability and efficient recovery from loworder observable moments are established under a novel graphical constraint. The constraint concerns the expansion properties of the underlying directed acyclic graph (DAG) between observed and unobserved variables in the network, and it is satisfied by many natural families of DAGs that include multilevel DAGs, DAGs with effective depth one, as well as certain families of polytrees. 1.
Subspace Identification via Convex Optimization
, 2011
"... In this thesis we consider convex optimizationbased approaches to the classical problem of identifying a subspace from noisy measurements of a random process taking values in the subspace. We focus on the case where the measurement noise is componentwise independent, known as the factor analysis m ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
In this thesis we consider convex optimizationbased approaches to the classical problem of identifying a subspace from noisy measurements of a random process taking values in the subspace. We focus on the case where the measurement noise is componentwise independent, known as the factor analysis model in statistics. We develop a new analysis of an existing convex optimizationbased heuristic for this problem. Our analysis indicates that in highdimensional settings, where both the ambient dimension and the dimension of the subspace to be identified are large, the convex heuristic, minimum trace factor analysis, is often very successful. We provide simple deterministic conditions on the underlying
Iterative reweighted algorithms for matrix rank minimization
 Journal of Machine Learning Research
"... The problem of minimizing the rank of a matrix subject to affine constraints has many applications in machine learning, and is known to be NPhard. One of the tractable relaxations proposed for this problem is nuclear norm (or trace norm) minimization of the matrix, which is guaranteed to find the m ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
The problem of minimizing the rank of a matrix subject to affine constraints has many applications in machine learning, and is known to be NPhard. One of the tractable relaxations proposed for this problem is nuclear norm (or trace norm) minimization of the matrix, which is guaranteed to find the minimum rank matrix under suitable assumptions. In this paper, we propose a family of Iterative Reweighted Least Squares algorithms IRLSp (with 0 ≤ p ≤ 1), as a computationally efficient way to improve over the performance of nuclear norm minimization. The algorithms can be viewed as (locally) minimizing certain smooth approximations to the rank function. When p = 1, we give theoretical guarantees similar to those for nuclear norm minimization, i.e., recovery of lowrank matrices under certain assumptions on the operator defining the constraints. For p < 1, IRLSp shows better empirical performance in terms of recovering lowrank matrices than nuclear norm minimization. We provide an efficient implementation for IRLSp, and also present a related family of algorithms, sIRLSp. These algorithms exhibit competitive run times and improved recovery when compared to existing algorithms for random instances of the matrix completion problem, as well as on the MovieLens movie recommendation data set.
Distributed learning of Gaussian graphical models via marginal likelihoods
 In AIStats
, 2013
"... We consider distributed estimation of the inverse covariance matrix, also called the concentration matrix, in Gaussian graphical models. Traditional centralized estimation often requires iterative and expensive global inference and is therefore difficult in large distributed networks. In this paper, ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
We consider distributed estimation of the inverse covariance matrix, also called the concentration matrix, in Gaussian graphical models. Traditional centralized estimation often requires iterative and expensive global inference and is therefore difficult in large distributed networks. In this paper, we propose a general framework for distributed estimation based on a maximum marginal likelihood (MML) approach. Each node independently computes a local estimate by maximizing a marginal likelihood defined with respect to data collected from its local neighborhood. Due to the nonconvexity of the MML problem, we derive and consider solving a convex relaxation. The local estimates are then combined into a global estimate without the need for iterative messagepassing between neighborhoods. We prove that this relaxed MML estimator is asymptotically consistent. Through numerical experiments on several synthetic and realworld data sets, we demonstrate that the twohop version of the proposed estimator is significantly better than the onehop version, and nearly closes the gap to the centralized maximum likelihood estimator in many situations. 1
ESTIMATION OF NETWORK STRUCTURES FROM PARTIALLY OBSERVED MARKOV RANDOM FIELDS
"... Abstract. We consider the estimation of highdimensional network structures from partially observed Markov random field data using a ℓ 1penalized pseudolikelihood approach. We fit a misspecified model obtained by ignoring the missing data problem. We derive an estimation error bound that highlight ..."
Abstract
 Add to MetaCart
Abstract. We consider the estimation of highdimensional network structures from partially observed Markov random field data using a ℓ 1penalized pseudolikelihood approach. We fit a misspecified model obtained by ignoring the missing data problem. We derive an estimation error bound that highlights the effect of the misspecification. We report some simulation results that illustrate the theoretical findings.