Results 1  10
of
20
Learning with the Weighted Tracenorm under Arbitrary Sampling Distributions
"... We provide rigorous guarantees on learning with the weighted tracenorm under arbitrary sampling distributions. We show that the standard weightedtrace norm might fail when the sampling distribution is not a product distribution (i.e. when row and column indexes are not selected independently), pre ..."
Abstract

Cited by 18 (4 self)
 Add to MetaCart
(Show Context)
We provide rigorous guarantees on learning with the weighted tracenorm under arbitrary sampling distributions. We show that the standard weightedtrace norm might fail when the sampling distribution is not a product distribution (i.e. when row and column indexes are not selected independently), present a corrected variant for which we establish strong learning guarantees, and demonstrate that it works better in practice. We provide guarantees when weighting by either the true or empirical sampling distribution, and suggest that even if the true distribution is known (or is uniform), weighting by the empirical distribution may be beneficial. 1
Convex tensor decomposition via structured Schatten norm regularization
 IN ADVANCES IN NIPS 26
, 2013
"... We study a new class of structured Schatten norms for tensors that includes two recently proposed norms (“overlapped” and “latent”) for convexoptimizationbased tensor decomposition. We analyze the performance of “latent” approach for tensor decomposition, which was empirically found to perform bet ..."
Abstract

Cited by 15 (2 self)
 Add to MetaCart
We study a new class of structured Schatten norms for tensors that includes two recently proposed norms (“overlapped” and “latent”) for convexoptimizationbased tensor decomposition. We analyze the performance of “latent” approach for tensor decomposition, which was empirically found to perform better than the “overlapped” approach in some settings. We show theoretically that this is indeed the case. In particular, when the unknown true tensor is lowrank in a specific unknown mode, this approach performs as well as knowing the mode with the smallest rank. Along the way, we show a novel duality result for structured Schatten norms, which is also interesting in the general context of structured sparsity. We confirm through numerical simulations that our theory can precisely predict the scaling behaviour of the mean squared error.
The algebraic combinatorial approach for lowrank matrix completion
 CoRR
"... We present a novel algebraic combinatorial view on lowrank matrix completion based on studying relations between a few entries with tools from algebraic geometry and matroid theory. The intrinsic locality of the approach allows for the treatment of single entries in a closed theoretical and practic ..."
Abstract

Cited by 10 (2 self)
 Add to MetaCart
(Show Context)
We present a novel algebraic combinatorial view on lowrank matrix completion based on studying relations between a few entries with tools from algebraic geometry and matroid theory. The intrinsic locality of the approach allows for the treatment of single entries in a closed theoretical and practical framework. More specifically, apart from introducing an algebraic combinatorial theory of lowrank matrix completion, we present probabilityone algorithms to decide whether a particular entry of the matrix can be completed. We also describe methods to complete that entry from a few others, and to estimate the error which is incurred by any method completing that entry. Furthermore, we show how known results on matrix completion and their sampling assumptions can be related to our new perspective and interpreted in terms of a completability phase transition. On this revision This revision version 4 is both abridged and extended in terms of exposition and results, as compared to version 3 Király et al. (2013). The theoretical foundations are developed in a more adhoc way which allow to reach the main statements and algorithmic implications more quickly. Version 3 contains a more principled derivation of the theory, more related results (e.g., estimation of missing entries and its consistency, representations for the determinantal matroid, detailed examples), but a focus which is further away from applications. A reader who is interested in both is invited to read the main parts of version 4 first, then go through version 3 for a more detailed view on the theory. 1.
PRISMA: PRoximal Iterative SMoothing Algorithm. arXiv preprint arXiv:1206.2372
, 2012
"... Motivated by learning problems including maxnorm regularized matrix completion and clustering, robust PCA and sparse inverse covariance selection, we propose a novel optimization algorithm for minimizing a convex objective which decomposes into three parts: a smooth part, a simple nonsmooth Lipsc ..."
Abstract

Cited by 7 (2 self)
 Add to MetaCart
Motivated by learning problems including maxnorm regularized matrix completion and clustering, robust PCA and sparse inverse covariance selection, we propose a novel optimization algorithm for minimizing a convex objective which decomposes into three parts: a smooth part, a simple nonsmooth Lipschitz part, and a simple nonsmooth nonLipschitz part. We use a time variant smoothing strategy that allows us to obtain a guarantee that does not depend on knowing in advance the total number of iterations nor a bound on the domain. 1
A RankCorrected Procedure for Matrix Completion with Fixed Basis Coefficients
, 2012
"... In this paper, we address lowrank matrix completion problems with fixed basis coefficients, which include the lowrank correlation matrix completion in various fields such as the financial market and the lowrank density matrix completion from the quantum state tomography. For this class of problem ..."
Abstract

Cited by 6 (2 self)
 Add to MetaCart
In this paper, we address lowrank matrix completion problems with fixed basis coefficients, which include the lowrank correlation matrix completion in various fields such as the financial market and the lowrank density matrix completion from the quantum state tomography. For this class of problems, the efficiency of the common nuclear norm penalized estimator for recovery may be challenged. Here, with a reasonable initial estimator, we propose a rankcorrected procedure to generate an estimator of high accuracy and low rank. For this new estimator, we establish a nonasymptotic recovery error bound and analyze the impact of adding the rankcorrection term on improving the recoverability. We also provide necessary and sufficient conditions for rank consistency in the sense of Bach [3], in which the concept of constraint nondegeneracy in matrix optimization plays an important role. As a byproduct, our results provide a theoretical foundation for the majorized penalty method of Gao and Sun [25] and Gao [24] for structured lowrank matrix optimization problems.
Stochastic gradient descent, weighted sampling, and the randomized Kaczmarz algorithm
 Advances in Neural Information Processing Systems
, 2014
"... ar ..."
(Show Context)
Local LowRank Matrix Approximation
"... Matrix approximation is a common tool in recommendation systems, text mining, and computer vision. A prevalent assumption in constructing matrix approximations is that the partially observed matrix is of lowrank. We propose a new matrix approximation model where we assume instead that the matrix is ..."
Abstract

Cited by 4 (1 self)
 Add to MetaCart
(Show Context)
Matrix approximation is a common tool in recommendation systems, text mining, and computer vision. A prevalent assumption in constructing matrix approximations is that the partially observed matrix is of lowrank. We propose a new matrix approximation model where we assume instead that the matrix is locally of lowrank, leading to a representation of the observed matrix as a weighted sum of lowrank matrices. We analyze the accuracy of the proposed local lowrank modeling. Our experiments show improvements in prediction accuracy over classical approaches for recommendation tasks. 1.
Greedy bilateral sketch, completion & smoothing
 In International Conference on Artificial Intelligence and Statistics
, 2013
"... Recovering a large lowrank matrix from highly corrupted, incomplete or sparse outlier overwhelmed observations is the crux of various intriguing statistical problems. We explore the power of “greedy bilateral (GreB) ” paradigm in reducing both time and sample complexities for solving these proble ..."
Abstract

Cited by 3 (2 self)
 Add to MetaCart
(Show Context)
Recovering a large lowrank matrix from highly corrupted, incomplete or sparse outlier overwhelmed observations is the crux of various intriguing statistical problems. We explore the power of “greedy bilateral (GreB) ” paradigm in reducing both time and sample complexities for solving these problems. GreB models a lowrank variable as a bilateral factorization, and updates the left and right factors in a mutually adaptive and greedy incremental manner. We detail how to model and solve lowrank approximation, matrix completion and robust PCA in GreB’s paradigm. On their MATLAB implementations, approximating a noisy 104 × 104 matrix of rank 500 with SVD accuracy takes 6s; MovieLens10M matrix of size 69878 × 10677 can be completed in 10s from 30 % of 107 ratings with RMSE 0.86 on the rest 70%; the lowrank background and sparse moving outliers in a 120×160 video of 500 frames are accurately separated in 1s. This brings 30 to 100 times acceleration in solving these popular statistical problems. 1