This survey provides an overview of higher-order tensor decompositions, their applications, and available software. A tensor is a multidimensional or N -way array. Decompositions of higher-order tensors (i.e., N -way arrays with N ≥ 3) have applications in psychometrics, chemometrics, signal processing, numerical linear algebra, computer vision, numerical analysis, data mining, neuroscience, graph analysis, etc. Two particular tensor decompositions can be considered to be higher-order extensions of the matrix singular value decompo- sition: CANDECOMP/PARAFAC (CP) decomposes a tensor as a sum of rank-one tensors, and the Tucker decomposition is a higher-order form of principal components analysis. There are many other tensor decompositions, including INDSCAL, PARAFAC2, CANDELINC, DEDICOM, and PARATUCK2 as well as nonnegative variants of all of the above. The N-way Toolbox and Tensor Toolbox, both for MATLAB, and the Multilinear Engine are examples of software packages for working with tensors.

Linear mixtures of independent random variables (the so-called sources) are sometimes referred to as under-determined mixtures (UDM) when the number of sources exceeds the dimension of the observation space. The algorithms proposed are able to identify algebraically a UDM using the second characteristic function (c.f.) of the observations, without any need of sparsity assumption on sources. In fact, by taking higher order derivatives of the multivariate c.f. core equation, the blind identification problem is shown to reduce to a tensor decomposition. With only two sensors, the first algorithm only needs a SVD. With a larger number of sensors, the second algorithm executes an alternating least squares (ALS) algorithm. The joint use of statistics of different orders is possible, and a LS solution can be computed. Identifiability conditions are stated in each of the two cases. Computer simulations eventually demonstrate performances in the absence of sparsity, and emphasize the interest in using jointly derivatives of different orders. r 2005 Elsevier B.V. All rights reserved.

This work was originally motivated by a classification of tensors proposed by Richard Harshman. In particular, we focus on simple and multiple “bottlenecks”, and on “swamps”. Existing theoretical results are surveyed, some numerical algorithms are described in details, and their numerical complexity is calculated. In particular, the interest in using the ELS enhancement in these algorithms is discussed. Computer simulations feed this discussion.

Abstract—The PARAFAC decomposition of a higher-order tensor is a powerful multilinear algebra tool that becomes more and more popular in a number of disciplines. Existing PARAFAC algorithms are computationally demanding and operate in batch mode—both serious drawbacks for on-line applications. When the data are serially acquired, or the underlying model changes with time, adaptive PARAFAC algorithms that can track the sought decomposition at low complexity would be highly desirable. This is a challenging task that has not been addressed in the literature, and the topic of this paper. Given an estimate of the PARAFAC decomposition of a tensor at instant t, we propose two adaptive algorithms to update the decomposition at instant t +1, the new tensor being obtained from the old one after appending a new slice in the ’time ’ dimension. The proposed algorithms can yield estimation performance that is very close to that obtained via repeated application of state-of-art batch algorithms, at orders of magnitude lower complexity. The effectiveness of the proposed algorithms is illustrated using a MIMO radar application (tracking of directions of arrival and directions of departure) as an example. Index Terms—Adaptive algorithms, DOA/DOD tracking, higher-order tensor, MIMO radar, PARAllel FACtor (PARAFAC).

Since the nineties, tensors are increasingly used in Signal Processing and Data Analysis. There exist striking differences between tensors and matrices, some being advantages, and others raising difficulties. These differences are pointed out in this paper while briefly surveying the state of the art. The conclusion is that tensors are omnipresent in real life, implicitly or explicitly, and must be used even if we still know quite little about their properties.

Recently, tensor signal processing has received an increased attention, particularly in the context of wireless communication applications. The so-called PARAllel FACtor (PARAFAC) decomposition is certainly the most used tensor tool. In general, the parameter estimation of a PARAFAC decomposition is carried out by means of the iterative ALS algorithm, which exhibits the following main drawbacks: convergence towards local minima, a high number of iterations for convergence, and difficulty to take, optimally, special matrix structures into account. In this paper, we propose a noniterative parameter estimation method for a PARAFAC decomposition when one matrix factor has a Toeplitz structure, a situation that is commonly encountered in signal processing applications. We illustrate the proposed method by means of simulation results. 1.

Abstract—Detection and estimation problems in multiple-input multiple-output (MIMO) radar have recently drawn considerable interest in the signal processing community. Radar has long been a staple of signal processing, and MIMO radar presents challenges and opportunities in adapting classical radar imaging tools and developing new ones. Our aim in this article is to showcase the potential of tensor algebra and multidimensional harmonic retrieval (HR) in signal processing for MIMO radar. Tensor algebra and multidimensional HR are relatively mature topics, albeit still on the fringes of signal processing research. We show they are in fact central for target localization in a variety of pertinent MIMO radar scenarios. Tensor algebra naturally comes into play when the coherent processing interval comprises multiple pulses, or multiple transmit and receive subarrays are used (multistatic configuration). Multidimensional harmonic structure emerges for far-field uniform linear transmit/receive array configurations, also taking into account Doppler shift; and hybrid models arise in-between. This viewpoint opens the door for the application and further development of powerful algorithms and identifiability results for MIMO radar. Compared to the classical radar-imaging-based methods such as Capon or MUSIC, these algebraic techniques yield improved performance, especially for closely spaced targets, at modest complexity. Index Terms—DoA-DoD estimation, harmonic retrieval, localization, multiple-input multiple-output (MIMO) radar, tensor decomposition. I.

Abstract. A new algorithm is presented for computing a canonical rank-R tensor approximation that has minimal distance to a given tensor in the Frobenius norm, where the canonical rank-R tensor consists of the sum of R rank-one tensors. Each iteration of the method consists of three steps. In the first step, a tentative new iterate is generated by a stand-alone one-step process, for which we use alternating least squares (ALS). In the second step, an accelerated iterate is generated by a nonlinear generalized minimal residual (GMRES) approach, recombining previous iterates in an optimal way, and essentially using the stand-alone one-step process as a preconditioner. In particular, the nonlinear extension of GMRES we use that was proposed by Washio and Oosterlee in [Electron. Trans. Numer. Anal., 15 (2003), pp. 165–185] for nonlinear partial differential equation problems (which is itself related to other existing acceleration methods for nonlinear equation systems). In the third step, a line search is performed for globalization. The resulting nonlinear GMRES (N-GMRES) optimization algorithm is applied to dense and sparse tensor decomposition test problems. The numerical tests show that ALS accelerated by N-GMRES may significantly outperform stand-alone ALS when highly accurate stationary points are desired for difficult problems. Further comparison tests show that N-GMRES is competitive with the well-known nonlinear conjugate gradient method for the test problems considered and outperforms it in many cases. The proposed N-GMRES optimization algorithm is based on general concepts and may be applied to other nonlinear optimization problems. Key words. optimization canonical tensor decomposition, alternating least squares, GMRES, nonlinear

Parallel factor analysis (PARAFAC, called also CP model)) is a tensor (multiway array) factorization method which allows to find hidden factors (component matrices) from a multidimensional data. Most of the existing algorithms for the PARAFAC, especially the alternating least squares (ALS) algorithm need to compute Khatri-Rao products of tall factors and multiplication of large-scale matrices and due to this require high computational cost and large memory and are not suitable for very large-scale problems. Hence, PARAFAC for large-scale data tensors is still a challenging problem. In this paper, we propose a new approach based on a modified ALS algorithm which computes Hadamard products, instead Khatri-Rao products and employs relatively small matrices. The new algorithms are able to process extremely large-scale tensors with billions of entries. Extensive experiments confirm the validity and high performance of the developed algorithms in comparison with other well-known algorithms. Keywords: Tensor factorization, PARAFAC, Large-scale dataset, Multiway classification, Parallel computing, Alternating least squares, Hierarchical ALS (HALS)

Abstract—We present a frequency-domain technique based on PARAllel FACtor (PARAFAC) analysis that performs multichannel blind source separation (BSS) of convolutive speech mixtures. PARAFAC algorithms are combined with a dimensionality reduction step to significantly reduce computational complexity. The identifiability potential of PARAFAC is exploited to derive a BSS algorithm for the under-determined case (more speakers than microphones), combining PARAFAC analysis with time-varying Capon beamforming. Finally, a low-complexity adaptive version of the BSS algorithm is proposed that can track changes in the mixing environment. Extensive experiments with realistic and measured data corroborate our claims, including the under-determined case. Signal-to-interference ratio improvements of up to 6 dB are shown compared to state-of-the-art BSS algorithms, at an order of magnitude lower computational complexity. Index Terms—Adaptive separation, blind speech separation,, joint diagonalization, PARAllel FACtor (PARAFAC), permutation ambiguity, underdetermined case.