This survey provides an overview of higher-order tensor decompositions, their applications, and available software. A tensor is a multidimensional or N -way array. Decompositions of higher-order tensors (i.e., N -way arrays with N ≥ 3) have applications in psychometrics, chemometrics, signal processing, numerical linear algebra, computer vision, numerical analysis, data mining, neuroscience, graph analysis, etc. Two particular tensor decompositions can be considered to be higher-order extensions of the matrix singular value decompo- sition: CANDECOMP/PARAFAC (CP) decomposes a tensor as a sum of rank-one tensors, and the Tucker decomposition is a higher-order form of principal components analysis. There are many other tensor decompositions, including INDSCAL, PARAFAC2, CANDELINC, DEDICOM, and PARATUCK2 as well as nonnegative variants of all of the above. The N-way Toolbox and Tensor Toolbox, both for MATLAB, and the Multilinear Engine are examples of software packages for working with tensors.

Abstract. We study the decomposition of a nonnegative tensor into a minimal sum of outer product of nonnegative vectors and the associated parsimonious naïve Bayes probabilistic model. We show that the corresponding approximation problem, which is central to nonnegative parafac, will always have optimal solutions. The result holds for any choice of norms and, under a mild assumption, even Brègman divergences. hal-00410056, version 1- 16 Aug 2009 1. Dedication This article is dedicated to the memory of our late colleague Richard Allan Harshman. It is loosely organized around two of Harshman’s best known works — parafac [19] and lsi [13], and answers two questions that he posed. We target this article to a technometrics readership. In Section 4, we discussed a few aspects of nonnegative tensor factorization and Hofmann’s plsi, a variant of the lsi model co-proposed by Harshman [13]. In Section 5, we answered a question of Harshman on why the apparently unrelated construction of Bini, Capovani, Lotti, and Romani in [1] should be regarded as the first example of what he called ‘parafac degeneracy ’ [27]. Finally in Section 6, we showed that such parafac degeneracy will not happen for nonnegative approximations of nonnegative tensors, answering another question of his. 2.

This work was originally motivated by a classification of tensors proposed by Richard Harshman. In particular, we focus on simple and multiple “bottlenecks”, and on “swamps”. Existing theoretical results are surveyed, some numerical algorithms are described in details, and their numerical complexity is calculated. In particular, the interest in using the ELS enhancement in these algorithms is discussed. Computer simulations feed this discussion.

We present an algorithm for decomposing a symmetric tensor of dimension n and order d as a sum of of rank-1 symmetric tensors, extending the algorithm of Sylvester devised in 1886 for symmetric tensors of dimension 2. We exploit the known fact that every symmetric tensor is equivalently represented by a homogeneous polynomial in n variables of total degree d. Thus the decomposition corresponds to a sum of powers of linear forms. The impact of this contribution is two-fold. First it permits an efficient computation of the decomposition of any tensor of sub-generic rank, as opposed to widely used iterative algorithms with unproved convergence (e.g. Alternate Least Squares or gradient descents). Second, it gives tools for understanding uniqueness conditions, and for detecting the tensor rank. 1.

Abstract. In this paper we proposed quasi-Newton and limited memory quasi-Newton methods for objective functions defined on Grassmann manifolds or a product of Grassmann manifolds. Specifically we defined bfgs and l-bfgs updates in local and global coordinates on Grassmann manifolds or a product of these. We proved that, when local coordinates are used, our bfgs updates on Grassmann manifolds share the same optimality property as the usual bfgs updates on Euclidean spaces. When applied to the best multilinear rank approximation problem for general and symmetric tensors, our approach yields fast, robust, and accurate algorithms that exploit the special Grassmannian structure of the respective problems, and which work on tensors of large dimensions and arbitrarily high order. Extensive numerical experiments are included to substantiate our claims. Key words. Grassmann manifold, Grassmannian, product of Grassmannians, Grassmann quasi-Newton, Grassmann bfgs, Grassmann l-bfgs, multilinear rank, symmetric multilinear rank, tensor, symmetric tensor, approximations

Abstract. Recent work on eigenvalues and eigenvectors for tensors of order m ≥ 3 has been motivated by applications in blind source separation, magnetic resonance imaging, molecular conformation, and more. In this paper, we consider methods for computing real symmetric-tensor eigenpairs of the form Axm−1 = λx subject to ‖x ‖ = 1, which is closely related to optimal rank-1 approximation of a symmetric tensor. Our contribution is a novel shifted symmetric higher-order power method (SS-HOPM), which we show is guaranteed to converge to a tensor eigenpair. SS-HOPM can be viewed as a generalization of the power iteration method for matrices or of the symmetric higherorder power method. Additionally, using fixed point analysis, we can characterize exactly which eigenpairs can and cannot be found by the method. Numerical examples are presented, including examples from an extension of the method to finding complex eigenpairs. Key words. tensor eigenvalues, E-eigenpairs, Z-eigenpairs, l2-eigenpairs, rank-1 approximation, symmetric higher-order power method (S-HOPM), shifted symmetric higher-order power method

We consider the problem of determining the symmetric tensor rank for symmetric tensors with an algebraic geometry approach. We give algorithms for computing the symmetric rank for 2 × · · · × 2 tensors and for tensors of small border rank. From a geometric point of view, we describe the symmetric rank strata for some secant varieties of Veronese varieties. Key words: Symmetric tensor, tensor rank, secant variety. 1.

The concept of tensor rank, introduced in the twenties, has been popularized at the beginning of the seventies. This has allowed to carry out Factor Analysis on arrays with more than two indices. The generic rank may be seen as an upper bound to the number of factors that can be extracted from a given tensor. We explain in this short paper how to obtain numerically the generic rank of tensors of arbitrary dimensions, and compare it with the rare algebraic results already known at order three. In particular, we examine the cases of symmetric tensors, tensors with symmetric matrix slices, or tensors with free entries. Résumé La notion de rang tensoriel, proposée dans les années vingt, a été popularisée au début des années soixante-dix. Ceci a permis de mettre en oeuvre l’Analyse de Facteurs sur des tableaux de données comportant plus de deux indices. Le rang générique peut être vu comme une borne supérieure sur le nombre de facteurs pouvant être extraits d’un tenseur donné. Nous expliquons dans ce court article comment trouver numériquement le rang générique d’un tenseur de dimensions arbitraires, et le comparons aux quelques rares résultats algébriques déjà connus à l’ordre trois. Nous examinons notamment les cas des tenseurs symétriques, des tenseurs à tranches matricielles symétriques, ou des tenseurs à éléments libres.

The idea that one might extend numerical linear algebra, the collection of matrix computational methods that form the workhorse of scientific and engineering computing, to numerical multilinear algebra, an analogous collection of tools involving hypermatrices/tensors, appears very promising and has attracted a lot of attention recently. We examine here the computational tractability of some core problems in numerical multilinear algebra. We show that tensor analogues of several standard problems that are readily computable in the matrix (i.e. 2-tensor) case are NP hard. Our list here includes: determining the feasibility of a system of bilinear equations, determining an eigenvalue, a singular value, or the spectral norm of a 3-tensor, determining a best rank-1 approximation to a 3-tensor, determining the rank of a 3-tensor over R or C. Hence making tensor computations feasible is likely to be a challenge.

We discuss how recently discovered techniques and tools from compressed sensing can be used in tensor decompositions, with a view towards modeling signals from multiple arrays of multiple sensors. We show that with appropriate bounds on coherence, one could always guarantee the existence and uniqueness of a best rank-r approximation of a tensor. In particular, we obtain a computationally feasible variant of Kruskal’s uniqueness condition with coherence as a proxy for k-rank. We treat sparsest recovery and lowest-rank recovery problems in a uniform fashion by considering Schatten and nuclear norms of tensors of arbitrary order and dictionaries that comprise a continuum of uncountably many atoms. Résumé Traitement du signal multi-antenne: les décompositions tensorielles rejoignent l’échantillonnage compressé. Nous décrivons comment les techniques et outils d’échantillonnage compressé récemment découverts peuvent être utilisés dans les décompositions tensorielles, avec pour illustration une modélisation des signaux provenant de plusieurs antennes multicapteurs. Nous montrons qu’en posant des bornes appropriées sur la cohérence, on pouvait toujours garantir l’existence et l’unicité d’une meilleure approximation de rang r d’un tenseur. En particulier, nous obtenons une variante calculable de la condition d’unicité de Kruskal, où la cohérence apparaît comme une mesure du rang. Nous