Results 1  10
of
14
Robustness analysis of Hottopixx, a linear programming model for factoring nonnegative matrices
 SIAM Journal on Matrix Analysis and Applications
, 2013
"... ar ..."
(Show Context)
The why and how of nonnegative matrix factorization
 REGULARIZATION, OPTIMIZATION, KERNELS, AND SUPPORT VECTOR MACHINES. CHAPMAN & HALL/CRC
, 2014
"... ..."
(Show Context)
A Vavasis, “Semidefinite programming based preconditioning for more robust nearseparable nonnegative matrix factorization,” arXiv preprint arXiv:1310.2273
, 2013
"... ar ..."
Ellipsoidal Rounding for Nonnegative Matrix Factorization Under Noisy Separability
, 2013
"... We present a numerical algorithm for nonnegative matrix factorization (NMF) problems under noisy separability. An NMF problem under separability can be stated as one of finding all vertices of the convex hull of data points. The research interest of this paper is to find the vectors as close to the ..."
Abstract

Cited by 3 (0 self)
 Add to MetaCart
(Show Context)
We present a numerical algorithm for nonnegative matrix factorization (NMF) problems under noisy separability. An NMF problem under separability can be stated as one of finding all vertices of the convex hull of data points. The research interest of this paper is to find the vectors as close to the vertices as possible in a situation in which noise is added to the data points. Our algorithm is designed to capture the shape of the convex hull of data points by using its enclosing ellipsoid. We show that the algorithm has correctness and robustness properties from theoretical and practical perspectives; correctness here means that if the data points do not contain any noise, the algorithm can find the vertices of their convex hull; robustness means that if the data points contain noise, the algorithm can find the nearvertices. Finally, we apply the algorithm to document clustering, and report the experimental results.
Provable Algorithms for Machine Learning Problems
, 2013
"... Modern machine learning algorithms can extract useful information from text, images and videos. All these applications involve solving NPhard problems in average case using heuristics. What properties of the input allow it to be solved efficiently? Theoretically analyzing the heuristics is often v ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
(Show Context)
Modern machine learning algorithms can extract useful information from text, images and videos. All these applications involve solving NPhard problems in average case using heuristics. What properties of the input allow it to be solved efficiently? Theoretically analyzing the heuristics is often very challenging. Few results were known. This thesis takes a different approach: we identify natural properties of the input, then design new algorithms that provably works assuming the input has these properties. We are able to give new, provable and sometimes practical algorithms for learning tasks related to text corpus, images and social networks. The first part of the thesis presents new algorithms for learning thematic structure in documents. We show under a reasonable assumption, it is possible to provably learn many topic models, including the famous Latent Dirichlet Allocation. Our algorithm is the first provable algorithms for topic modeling. An implementation runs 50 times faster than latest MCMC implementation and produces comparable results. The second part of the thesis provides ideas for provably learning deep, sparse representations. We start with sparse linear representations, and give the first algorithm for dictionary learning problem with provable guarantees. Then we apply similar ideas to deep learning: under reasonable assumptions our algorithms can learn a deep network built by denoising autoencoders. The final part of the thesis develops a framework for learning latent variable models. We demonstrate how various latent variable models can be reduced to orthogonal tensor decomposition, and then be solved using tensor power method. We give a tight perturbation analysis for tensor power method, which reduces the number of samples required to learn many latent variable models. In theory, the assumptions in this thesis help us understand why intractable problems in machine learning can often be solved; in practice, the results suggest inherently new approaches for machine learning. We hope the assumptions and algorithms inspire new research problems and learning algorithms. iii
Nonnegative matrix factorization under heavy noise.
 In Proceedings of the 33nd International Conference on Machine Learning,
, 2016
"... Abstract The Noisy Nonnegative Matrix factorization (NMF) is: given a data matrix A (d × n), find nonnegative matrices B, C (d × k, k × n respy.) so that A = BC + N , where N is a noise matrix. Existing polynomial time algorithms with proven error guarantees require each column N ·,j to have l 1 ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
(Show Context)
Abstract The Noisy Nonnegative Matrix factorization (NMF) is: given a data matrix A (d × n), find nonnegative matrices B, C (d × k, k × n respy.) so that A = BC + N , where N is a noise matrix. Existing polynomial time algorithms with proven error guarantees require each column N ·,j to have l 1 norm much smaller than (BC) ·,j  1 , which could be very restrictive. In important applications of NMF such as Topic Modeling as well as theoretical noise models (eg. Gaussian with high σ), almost every column of N ·j violates this condition. We introduce the heavy noise model which only requires the average noise over large subsets of columns to be small. We initiate a study of Noisy NMF under the heavy noise model. We show that our noise model subsumes noise models of theoretical and practical interest (for eg. Gaussian noise of maximum possible σ). We then devise an algorithm TSVDNMF which under certain assumptions on B, C, solves the problem under heavy noise. Our error guarantees match those of previous algorithms. Our running time of O((n + d) 2 k) is substantially better than the O(n 3 d) for the previous best. Our assumption on B is weaker than the "Separability" assumption made by all previous results. We provide empirical justification for our assumptions on C. We also provide the first proof of identifiability (uniqueness of B) for noisy NMF which is not based on separability and does not use hard to check geometric conditions. Our algorithm outperforms earlier polynomial time algorithms both in time and error, particularly in the presence of high noise.
SelfDictionary Sparse Regression for Hyperspectral Unmixing: Greedy Pursuit and Pure Pixel Search Are Related
"... Abstract—This paper considers a recently emerged hyperspectral unmixing formulation based on sparse regression of a selfdictionary multiple measurement vector (SDMMV) model, wherein the measured hyperspectral pixels are used as the dictionary. Operating under the pure pixel assumption, this SD ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
Abstract—This paper considers a recently emerged hyperspectral unmixing formulation based on sparse regression of a selfdictionary multiple measurement vector (SDMMV) model, wherein the measured hyperspectral pixels are used as the dictionary. Operating under the pure pixel assumption, this SDMMV formalism is special in that it allows simultaneous identification of the endmember spectral signatures and the number of endmembers. Previous SDMMV studies mainly focus on convex relaxations. In this study, we explore the alternative of greedy pursuit, which generally provides efficient and simple algorithms. In particular, we design a greedy SDMMV algorithm using simultaneous orthogonal matching pursuit. Intriguingly, the proposed greedy algorithm is shown to be closely related to some existing pure pixel search algorithms, especially, the successive projection algorithm (SPA). Thus, a link between SDMMV and pure pixel search is revealed. We then perform exact recovery analyses, and prove that the proposed greedy algorithm is robust to noiseincluding its identification of the (unknown) number of endmembersunder a sufficiently low noise level. The identification performance of the proposed greedy algorithm is demonstrated through both synthetic and realdata experiments. Index Terms—Greedy pursuit, hyperspectral unmixing, number of endmembers estimation, selfdictionary sparse regression.
Successive Nonnegative Projection Algorithm for Robust Nonnegative Blind Source Separation
"... ar ..."
(Show Context)
1Robust LargeScale NonNegative Matrix Factorization Using Proximal Point Algorithm
"... A robust algorithm for nonnegative matrix factorization (NMF) is presented in this paper with the purpose of dealing with largescale data, where the separability assumption is satisfied. In particular, we modify the Linear Programming (LP) algorithm of [9] by introducing a reduced set of constrain ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
(Show Context)
A robust algorithm for nonnegative matrix factorization (NMF) is presented in this paper with the purpose of dealing with largescale data, where the separability assumption is satisfied. In particular, we modify the Linear Programming (LP) algorithm of [9] by introducing a reduced set of constraints for exact NMF. In contrast to the previous approaches, the proposed algorithm does not require the knowledge of factorization rank (extreme rays [3] or topics [7]). Furthermore, motivated by a similar problem arising in the context of metabolic network analysis [13], we consider an entirely different regime where the number of extreme rays or topics can be much larger than the dimension of the data vectors. The performance of the algorithm for different synthetic data sets are provided. I.
Robustness Analysis of Structured Ma trix Factorization via SelfDictionary MixedNorm Optimization
"... Abstract—We are interested in a lowrank matrix factorization problem where one of the matrix factors has a special structure; specifically, its columns live in the unit simplex. This problem finds applications in diverse areas such as hyperspectral unmixing, video summarization, spectrum sensing, a ..."
Abstract
 Add to MetaCart
(Show Context)
Abstract—We are interested in a lowrank matrix factorization problem where one of the matrix factors has a special structure; specifically, its columns live in the unit simplex. This problem finds applications in diverse areas such as hyperspectral unmixing, video summarization, spectrum sensing, and blind speech separation. Prior works showed that such a factorization problem can be formulated as a selfdictionary sparse optimization problem under some assumptions that are considered realistic in many applications, and convex mixed norms were employed as optimization surrogates to realize the factorization in practice. Numerical results have shown that the mixednorm approach demonstrates promising performance. In this letter, we conduct performance analysis of the mixednorm approach under noise perturbations. Our result shows that using a convex mixed norm can indeed yield provably good solutions. More importantly, we also show that using nonconvex mixed (quasi) norms is more advantageous in terms of robustness against noise. Index Terms—Matrix factorization, performance analysis, selfdictionary sparse optimization. I.