Results 1  10
of
58
A unified framework for highdimensional analysis of Mestimators with decomposable regularizers
"... ..."
Restricted strong convexity and (weighted) matrix completion: Optimal bounds with noise
, 2010
"... We consider the matrix completion problem under a form of row/column weighted entrywise sampling, including the case of uniform entrywise sampling as a special case. We analyze the associated random observation operator, and prove that with high probability, it satisfies a form of restricted strong ..."
Abstract

Cited by 34 (6 self)
 Add to MetaCart
We consider the matrix completion problem under a form of row/column weighted entrywise sampling, including the case of uniform entrywise sampling as a special case. We analyze the associated random observation operator, and prove that with high probability, it satisfies a form of restricted strong convexity with respect to weighted Frobenius norm. Using this property, we obtain as corollaries a number of error bounds on matrix completion in the weighted Frobenius norm under noisy sampling and for both exact and near lowrank matrices. Our results are based on measures of the “spikiness ” and “lowrankness ” of matrices that are less restrictive than the incoherence conditions imposed in previous work. Our technique involves an Mestimator that includes controls on both the rank and spikiness of the solution, and we establish nonasymptotic error bounds in weighted Frobenius norm for recovering matrices lying with ℓq“balls ” of bounded spikiness. Using informationtheoretic methods, we show that no algorithm can achieve better estimates (up to a logarithmic factor) over these same sets, showing that our conditions on matrices and associated rates are essentially optimal.
SOLVING A LOWRANK FACTORIZATION MODEL FOR MATRIX COMPLETION BY A NONLINEAR SUCCESSIVE OVERRELAXATION ALGORITHM
"... Abstract. The matrix completion problem is to recover a lowrank matrix from a subset of its entries. The main solution strategy for this problem has been based on nuclearnorm minimization which requires computing singular value decompositions – a task that is increasingly costly as matrix sizes an ..."
Abstract

Cited by 27 (7 self)
 Add to MetaCart
Abstract. The matrix completion problem is to recover a lowrank matrix from a subset of its entries. The main solution strategy for this problem has been based on nuclearnorm minimization which requires computing singular value decompositions – a task that is increasingly costly as matrix sizes and ranks increase. To improve the capacity of solving largescale problems, we propose a lowrank factorization model and construct a nonlinear successive overrelaxation (SOR) algorithm that only requires solving a linear least squares problem per iteration. Convergence of this nonlinear SOR algorithm is analyzed. Numerical results show that the algorithm can reliably solve a wide range of problems at a speed at least several times faster than many nuclearnorm minimization algorithms. Key words. Matrix Completion, alternating minimization, nonlinear GS method, nonlinear SOR method AMS subject classifications. 65K05, 90C06, 93C41, 68Q32
Parallel stochastic gradient algorithms for largescale matrix completion
 Mathematical Programming Computation
, 2013
"... This paper develops Jellyfish, an algorithm for solving dataprocessing problems with matrixvalued decision variables regularized to have low rank. Particular examples of problems solvable by Jellyfish include matrix completion problems and leastsquares problems regularized by the nuclear norm or ..."
Abstract

Cited by 21 (3 self)
 Add to MetaCart
This paper develops Jellyfish, an algorithm for solving dataprocessing problems with matrixvalued decision variables regularized to have low rank. Particular examples of problems solvable by Jellyfish include matrix completion problems and leastsquares problems regularized by the nuclear norm or γ2norm. Jellyfish implements a projected incremental gradient method with a biased, random ordering of the increments. This biased ordering allows for a parallel implementation that admits a speedup nearly proportional to the number of processors. On largescale matrix completion tasks, Jellyfish is orders of magnitude more efficient than existing codes. For example, on the Netflix Prize data set, prior art computes rating predictions in approximately 4 hours, while Jellyfish solves the same problem in under 3 minutes on a 12 core workstation.
Collaborative filtering in a nonuniform world: Learning with the weighted trace norm
, 2010
"... We show that matrix completion with tracenorm regularization can be significantly hurt when entries of the matrix are sampled nonuniformly, but that a properly weighted version of the tracenorm regularizer works well with nonuniform sampling. We show that the weighted tracenorm regularization i ..."
Abstract

Cited by 20 (5 self)
 Add to MetaCart
We show that matrix completion with tracenorm regularization can be significantly hurt when entries of the matrix are sampled nonuniformly, but that a properly weighted version of the tracenorm regularizer works well with nonuniform sampling. We show that the weighted tracenorm regularization indeed yields significant gains on the highly nonuniformly sampled Netflix dataset.
SpaRCS: Recovering lowrank and sparse matrices from compressive measurements
, 2011
"... We consider the problem of recovering a matrix M that is the sum of a lowrank matrix L and a sparse matrix S from a small set of linear measurements of the form y = A(M) =A(L + S). This model subsumes three important classes of signal recovery problems: compressive sensing, affine rank minimization ..."
Abstract

Cited by 13 (1 self)
 Add to MetaCart
We consider the problem of recovering a matrix M that is the sum of a lowrank matrix L and a sparse matrix S from a small set of linear measurements of the form y = A(M) =A(L + S). This model subsumes three important classes of signal recovery problems: compressive sensing, affine rank minimization, and robust principal component analysis. We propose a natural optimization problem for signal recovery under this model and develop a new greedy algorithm called SpaRCS to solve it. Empirically, SpaRCS inherits a number of desirable properties from the stateoftheart CoSaMP and ADMiRA algorithms, including exponential convergence and efficient implementation. Simulation results with video compressive sensing, hyperspectral imaging, and robust matrix completion data sets demonstrate both the accuracy and efficacy of the algorithm. 1
FAST GLOBAL CONVERGENCE OF GRADIENT METHODS FOR HIGHDIMENSIONAL STATISTICAL RECOVERY
 SUBMITTED TO THE ANNALS OF STATISTICS
, 2012
"... Many statistical Mestimators are based on convex optimization problems formed by the combination of a datadependent loss function with a normbased regularizer. We analyze the convergence rates of projected gradient and composite gradient methods for solving such problems, working within a highdi ..."
Abstract

Cited by 9 (1 self)
 Add to MetaCart
Many statistical Mestimators are based on convex optimization problems formed by the combination of a datadependent loss function with a normbased regularizer. We analyze the convergence rates of projected gradient and composite gradient methods for solving such problems, working within a highdimensional framework that allows the ambient dimension d to grow with (and possibly exceed) the sample size n. Our theory identifies conditions under which projected gradient descent enjoys globally linear convergence up to the statistical precision of the model, meaning the typical distance between the true unknown parameter θ ∗ and an optimal solution ̂ θ. By establishing these conditions with high probability for numerous statistical models, our analysis applies to a wide range of Mestimators, including sparse linear regression using Lasso; group Lasso for block sparsity; loglinear models with regularization; lowrank matrix recovery using nuclear norm regularization; and matrix decomposition
Learning with the Weighted Tracenorm under Arbitrary Sampling Distributions
"... We provide rigorous guarantees on learning with the weighted tracenorm under arbitrary sampling distributions. We show that the standard weightedtrace norm might fail when the sampling distribution is not a product distribution (i.e. when row and column indexes are not selected independently), pre ..."
Abstract

Cited by 9 (4 self)
 Add to MetaCart
We provide rigorous guarantees on learning with the weighted tracenorm under arbitrary sampling distributions. We show that the standard weightedtrace norm might fail when the sampling distribution is not a product distribution (i.e. when row and column indexes are not selected independently), present a corrected variant for which we establish strong learning guarantees, and demonstrate that it works better in practice. We provide guarantees when weighting by either the true or empirical sampling distribution, and suggest that even if the true distribution is known (or is uniform), weighting by the empirical distribution may be beneficial. 1
Principal Component Analysis with Contaminated Data: The High Dimensional
"... We consider the dimensionalityreduction problem (finding a subspace approximation of observed data) for contaminated data in the high dimensional regime, where the the number of observations is of the same magnitude as the number of variables of each observation, and the data set contains some (arb ..."
Abstract

Cited by 9 (3 self)
 Add to MetaCart
We consider the dimensionalityreduction problem (finding a subspace approximation of observed data) for contaminated data in the high dimensional regime, where the the number of observations is of the same magnitude as the number of variables of each observation, and the data set contains some (arbitrarily) corrupted observations. We propose a Highdimensional Robust Principal Component Analysis (HRPCA) algorithm that is tractable, robust to contaminated points, and easily kernelizable. The resulting subspace has a bounded deviation from the desired one, and unlike ordinary PCA algorithms, achieves optimality in the limit case where the proportion of corrupted points goes to zero. 1
Concentrationbased guarantees for lowrank matrix reconstruction
 24th Annual Conference on Learning Theory (COLT
, 2011
"... We consider the problem of approximately reconstructing a partiallyobserved, approximately lowrank matrix. This problem has received much attention lately, mostly using the tracenorm as a surrogate to the rank. Here we study lowrank matrix reconstruction using both the tracenorm, as well as the ..."
Abstract

Cited by 7 (2 self)
 Add to MetaCart
We consider the problem of approximately reconstructing a partiallyobserved, approximately lowrank matrix. This problem has received much attention lately, mostly using the tracenorm as a surrogate to the rank. Here we study lowrank matrix reconstruction using both the tracenorm, as well as the lessstudied maxnorm, and present reconstruction guarantees based on existing analysis on the Rademacher complexity of the unit balls of these norms. We show how these are superior in several ways to recently published guarantees based on specialized analysis.