Results 1  10
of
37
Most tensor problems are NP hard
 CORR
, 2009
"... The idea that one might extend numerical linear algebra, the collection of matrix computational methods that form the workhorse of scientific and engineering computing, to numerical multilinear algebra, an analogous collection of tools involving hypermatrices/tensors, appears very promising and has ..."
Abstract

Cited by 45 (6 self)
 Add to MetaCart
The idea that one might extend numerical linear algebra, the collection of matrix computational methods that form the workhorse of scientific and engineering computing, to numerical multilinear algebra, an analogous collection of tools involving hypermatrices/tensors, appears very promising and has attracted a lot of attention recently. We examine here the computational tractability of some core problems in numerical multilinear algebra. We show that tensor analogues of several standard problems that are readily computable in the matrix (i.e. 2tensor) case are NP hard. Our list here includes: determining the feasibility of a system of bilinear equations, determining an eigenvalue, a singular value, or the spectral norm of a 3tensor, determining a best rank1 approximation to a 3tensor, determining the rank of a 3tensor over R or C. Hence making tensor computations feasible is likely to be a challenge.
Factoring nonnegative matrices with linear programs
, 2012
"... This paper describes a new approach for computing nonnegative matrix factorizations (NMFs) with linear programming. The key idea is a datadriven model for the factorization, in which the most salient features in the data are used to express the remaining features. More precisely, given a data matri ..."
Abstract

Cited by 40 (0 self)
 Add to MetaCart
(Show Context)
This paper describes a new approach for computing nonnegative matrix factorizations (NMFs) with linear programming. The key idea is a datadriven model for the factorization, in which the most salient features in the data are used to express the remaining features. More precisely, given a data matrix X, the algorithm identifies a matrix C that satisfies X ≈ CX and some linear constraints. The matrix C selects features, which are then used to compute a lowrank NMF of X. A theoretical analysis demonstrates that this approach has the same type of guarantees as the recent NMF algorithm of Arora et al. (2012). In contrast with this earlier work, the proposed method (1) has better noise tolerance, (2) extends to more general noise models, and (3) leads to efficient, scalable algorithms. Experiments with synthetic and real datasets provide evidence that the new approach is also superior in practice. An optimized C++ implementation of the new algorithm can factor a multiGigabyte matrix in a matter of minutes.
Learning topic models – going beyond SVD
 In 2012 IEEE 53rd Annual Symposium on Foundations of Computer Science
, 2012
"... ar ..."
(Show Context)
Fast and robust recursive algorithms for separable nonnegative matrix factorization. arXiv preprint arXiv:1208.1237
, 2012
"... ar ..."
(Show Context)
Fast conical hull algorithms for nearseparable nonnegative matrix factorization
 In ACM/IEEE conference on Supercomputing
, 2009
"... The separability assumption (Donoho & Stodden, 2003; Arora et al., 2012a) turns nonnegative matrix factorization (NMF) into a tractable problem. Recently, a new class of provablycorrect NMF algorithms have emerged under this assumption. In this paper, we reformulate the separable NMF problem a ..."
Abstract

Cited by 21 (1 self)
 Add to MetaCart
(Show Context)
The separability assumption (Donoho & Stodden, 2003; Arora et al., 2012a) turns nonnegative matrix factorization (NMF) into a tractable problem. Recently, a new class of provablycorrect NMF algorithms have emerged under this assumption. In this paper, we reformulate the separable NMF problem as that of finding the extreme rays of the conical hull of a finite set of vectors. From this geometricperspective, we derive new separable NMF algorithms that are highly scalable and empirically noise robust, and haveseveralotherfavorablepropertiesin relation to existing methods. A parallel implementation of our algorithm demonstrates high scalability on shared and distributedmemory machines. 1.
New Algorithms for Learning Incoherent and Overcomplete Dictionaries
, 2014
"... In sparse recovery we are given a matrix A ∈ Rn×m (“the dictionary”) and a vector of the form AX where X is sparse, and the goal is to recover X. This is a central notion in signal processing, statistics and machine learning. But in applications such as sparse coding, edge detection, compression an ..."
Abstract

Cited by 19 (2 self)
 Add to MetaCart
(Show Context)
In sparse recovery we are given a matrix A ∈ Rn×m (“the dictionary”) and a vector of the form AX where X is sparse, and the goal is to recover X. This is a central notion in signal processing, statistics and machine learning. But in applications such as sparse coding, edge detection, compression and super resolution, the dictionary A is unknown and has to be learned from random examples of the form Y = AX where X is drawn from an appropriate distribution — this is the dictionary learning problem. In most settings, A is overcomplete: it has more columns than rows. This paper presents a polynomialtime algorithm for learning overcomplete dictionaries; the only previously known algorithm with provable guarantees is the recent work of Spielman et al. (2012) who gave an algorithm for the undercomplete case, which is rarely the case in applications. Our algorithm applies to incoherent dictionaries which have been a central object of study since they were introduced in seminal work of Donoho and Huo (1999). In particular, a dictionary is µincoherent if each pair of columns has inner product at most µ/ n. The algorithm makes natural stochastic assumptions about the unknown sparse vector X, which can contain k ≤ cmin(√n/µ log n,m1/2−η) nonzero entries (for any η> 0). This is close to the best k allowable by the best sparse recovery algorithms even if one knows the dictionary A exactly. Moreover, both the running time and sample complexity depend on log 1/, where is the target accuracy, and so our algorithms converge very quickly to the true dictionary. Our algorithm can also tolerate substantial amounts of noise provided it is incoherent with respect to the dictionary (e.g., Gaussian). In the noisy setting, our running time and sample complexity depend polynomially on 1/, and this is necessary.
Approximation Limits of Linear Programs (Beyond Hierarchies)
, 2013
"... We develop a framework for proving approximation limits of polynomialsize linear programs from lower bounds on the nonnegative ranks of suitably defined matrices. This framework yields unconditional impossibility results that are applicable to any linear program as opposed to only programs generate ..."
Abstract

Cited by 17 (6 self)
 Add to MetaCart
We develop a framework for proving approximation limits of polynomialsize linear programs from lower bounds on the nonnegative ranks of suitably defined matrices. This framework yields unconditional impossibility results that are applicable to any linear program as opposed to only programs generated by hierarchies. Using our framework, we prove that O(n1/2−ɛ)approximations for CLIQUE require linear programs of size 2nΩ(ɛ). This lower bound applies to linear programs using a certain encoding of CLIQUE as a linear optimization problem. Moreover, we establish a similar result for approximations of semidefinite programs by linear programs. Our main technical ingredient is a quantitative improvement of Razborov’s rectangle corruption lemma (1992) for the high error regime, which gives strong lower bounds on the nonnegative rank of shifts of the unique disjointness matrix.
Sparse and unique nonnegative matrix factorization through data preprocessing
 Journal of Machine Learning Research
"... Nonnegative matrix factorization (NMF) has become a very popular technique in machine learning because it automatically extracts meaningful features through a sparse and partbased representation. However, NMF has the drawback of being highly illposed, that is, there typically exist many different ..."
Abstract

Cited by 14 (6 self)
 Add to MetaCart
(Show Context)
Nonnegative matrix factorization (NMF) has become a very popular technique in machine learning because it automatically extracts meaningful features through a sparse and partbased representation. However, NMF has the drawback of being highly illposed, that is, there typically exist many different but equivalent factorizations. In this paper, we introduce a completely new way to obtaining more wellposed NMF problems whose solutions are sparser. Our technique is based on the preprocessing of the nonnegative input data matrix, and relies on the theory of Mmatrices and the geometric interpretation of NMF. This approach provably leads to optimal and sparse solutions under the separability assumption of Donoho and Stodden (2003), and, for rankthree matrices, makes the number of exact factorizations finite. We illustrate the effectiveness of our technique on several image data sets.
R.: Robust nearseparable nonnegative matrix factorization using linear optimization
 Journal of Machine Learning Research
, 2014
"... ar ..."
(Show Context)
Robustness analysis of Hottopixx, a linear programming model for factoring nonnegative matrices
 SIAM Journal on Matrix Analysis and Applications
, 2013
"... ar ..."
(Show Context)