Results 1  10
of
36
Tensor decompositions for learning latent variable models
, 2014
"... This work considers a computationally and statistically efficient parameter estimation method for a wide class of latent variable models—including Gaussian mixture models, hidden Markov models, and latent Dirichlet allocation—which exploits a certain tensor structure in their loworder observable mo ..."
Abstract

Cited by 72 (5 self)
 Add to MetaCart
(Show Context)
This work considers a computationally and statistically efficient parameter estimation method for a wide class of latent variable models—including Gaussian mixture models, hidden Markov models, and latent Dirichlet allocation—which exploits a certain tensor structure in their loworder observable moments (typically, of second and thirdorder). Specifically, parameter estimation is reduced to the problem of extracting a certain (orthogonal) decomposition of a symmetric tensor derived from the moments; this decomposition can be viewed as a natural generalization of the singular value decomposition for matrices. Although tensor decompositions are generally intractable to compute, the decomposition of these specially structured tensors can be efficiently obtained by a variety of approaches, including power iterations and maximization approaches (similar to the case of matrices). A detailed analysis of a robust tensor power method is provided, establishing an analogue of Wedin’s perturbation theorem for the singular vectors of matrices. This implies a robust and computationally tractable estimation approach for several popular latent variable models.
A Tensor Spectral Approach to Learning Mixed Membership Community Models
"... Detecting hidden communities from observed interactions is a classical problem. Theoretical analysis of community detection has so far been mostly limited to models with nonoverlapping communities such as the stochastic block model. In this paper, we provide guaranteed community detection for a fam ..."
Abstract

Cited by 26 (4 self)
 Add to MetaCart
Detecting hidden communities from observed interactions is a classical problem. Theoretical analysis of community detection has so far been mostly limited to models with nonoverlapping communities such as the stochastic block model. In this paper, we provide guaranteed community detection for a family of probabilistic network models with overlapping communities, termed as the mixed membership Dirichlet model, first introduced in Airoldi et al. (2008). This model allows for nodes to have fractional memberships in multiple communities and assumes that the community memberships are drawn from a Dirichlet distribution. Moreover, it contains the stochastic block model as a special case. We propose a unified approach to learning communities in these models via a tensor spectral decomposition approach. Our estimator uses loworder moment tensor of the observed network, consisting of 3star counts. Our learning method is based on simple linear algebraic operations such as singular value decomposition and tensor power iterations. We provide guaranteed recovery of community memberships and model parameters, and present a careful finite sample analysis of our learning method. Additionally, our results match the best known scaling requirements for the special case of the (homogeneous) stochastic block model.
A survey on the spectral theory of nonnegative tensors
 NUMERICAL LINEAR ALGEBRA WITH APPLICATIONS
, 2013
"... This is a survey paper on the recent development of the spectral theory of nonnegative tensors and its applications. After a brief review of the basic definitions on tensors, the Heigenvalue problem and the Zeigenvalue problem for tensors are studied separately. To the Heigenvalue problem for non ..."
Abstract

Cited by 9 (5 self)
 Add to MetaCart
This is a survey paper on the recent development of the spectral theory of nonnegative tensors and its applications. After a brief review of the basic definitions on tensors, the Heigenvalue problem and the Zeigenvalue problem for tensors are studied separately. To the Heigenvalue problem for nonnegative tensors, the whole Perron–Frobenius theory for nonnegative matrices is completely extended, while to the Zeigenvalue problem, there are many distinctions and are studied carefully in details. Numerical methods are also discussed. Three kinds of applications are studied: higher order Markov chains, spectral theory of
Detection of crossing white matter fibers with highorder tensors and
"... rankk decompositions ..."
(Show Context)
A Tensor Approach to Learning Mixed Membership Community Models
"... Community detection is the task of detecting hidden communities from observed interactions. Guaranteed community detection has so far been mostly limited to models with nonoverlapping communities such as the stochastic block model. In this paper, we remove this restriction, and provide guaranteed ..."
Abstract

Cited by 6 (0 self)
 Add to MetaCart
Community detection is the task of detecting hidden communities from observed interactions. Guaranteed community detection has so far been mostly limited to models with nonoverlapping communities such as the stochastic block model. In this paper, we remove this restriction, and provide guaranteed community detection for a family of probabilistic network models with overlapping communities, termed as the mixed membership Dirichlet model, first introduced by Airoldi et al. (2008). This model allows for nodes to have fractional memberships in multiple communities and assumes that the community memberships are drawn from a Dirichlet distribution. Moreover, it contains the stochastic block model as a special case. We propose a unified approach to learning these models via a tensor spectral decomposition method. Our estimator is based on loworder moment tensor of the observed network, consisting of 3star counts. Our learning method is fast and is based on simple linear algebraic operations, e.g., singular value decomposition and tensor power iterations. We provide guaranteed recovery of community memberships and model parameters and present a careful finite sample analysis of our learning method. As an important special case, our results match the best known scaling requirements for the (homogeneous) stochastic block model.
Jacobi algorithm for the best low multilinear rank approximation of symmetric tensors
 SIAM JOURNAL ON MATRIX ANALYSIS AND APPLICATIONS
, 2013
"... The problem discussed in this paper is the symmetric best low multilinear rank approximation of thirdorder symmetric tensors. We propose an algorithm based on Jacobi rotations, for which symmetry is preserved at each iteration. Two numerical examples are provided indicating the need for such algo ..."
Abstract

Cited by 5 (1 self)
 Add to MetaCart
(Show Context)
The problem discussed in this paper is the symmetric best low multilinear rank approximation of thirdorder symmetric tensors. We propose an algorithm based on Jacobi rotations, for which symmetry is preserved at each iteration. Two numerical examples are provided indicating the need for such algorithms. An important part of the paper consists of proving that our algorithm converges to stationary points of the objective function. This can be considered an advantage of the proposed algorithm over existing symmetrypreserving algorithms in the literature.
EFFICIENTLY COMPUTING TENSOR EIGENVALUES ON A GPU
"... Abstract. The tensor eigenproblem has many important applications, and both mathematical and applicationspecific communities have taken recent interest in the properties of tensor eigenpairs as well as methods for computing them. In particular, Kolda and Mayo [3] present a generalization of the matr ..."
Abstract

Cited by 5 (3 self)
 Add to MetaCart
(Show Context)
Abstract. The tensor eigenproblem has many important applications, and both mathematical and applicationspecific communities have taken recent interest in the properties of tensor eigenpairs as well as methods for computing them. In particular, Kolda and Mayo [3] present a generalization of the matrix power method for symmetric tensors. We focus in this work on efficient implementation of their algorithm, known as the shifted symmetric higherorder power method, and on how a GPU can be used to accelerate the computation up to 70 × over a sequential implementation for an application involving many small tensor eigenproblems.
THIRD ORDER TENSORS AS OPERATORS ON MATRICES: A THEORETICAL AND COMPUTATIONAL FRAMEWORK WITH APPLICATIONS IN IMAGING ∗
"... Recent work by Kilmer and Martin, [10] and Braman [2] provides a setting in which the familiar tools of linear algebra can be extended to better understand thirdorder tensors. Continuing along this vein, this paper investigates further implications including: 1) a bilinear operator on the matrices ..."
Abstract

Cited by 5 (1 self)
 Add to MetaCart
(Show Context)
Recent work by Kilmer and Martin, [10] and Braman [2] provides a setting in which the familiar tools of linear algebra can be extended to better understand thirdorder tensors. Continuing along this vein, this paper investigates further implications including: 1) a bilinear operator on the matrices which is nearly an inner product and which leads to definitions for length of matrices, angle between two matrices and orthogonality of matrices and 2) the use of tlinear combinations to characterize the range and kernel of a mapping defined by a thirdorder tensor and the tproduct and the quantification of the dimensions of those sets. These theoretical results lead to the study of orthogonal projections as well as an effective GramSchmidt process for producing an orthogonal basis of matrices. The theoretical framework also leads us to consider the notion of tensor polynomials and their relation to tensor eigentuples defined in [2]. Implications for extending basic algorithms such as the power method, QR iteration, and Krylov subspace methods are discussed. We conclude with two examples in image processing: using the orthogonal elements generated via a GolubKahan iterative bidiagonalization scheme for facial recognition and solving a regularized image deblurring problem.
BLOCK TENSORS AND SYMMETRIC EMBEDDINGS
, 1010
"... Abstract. Well known connections exist between the singular value decomposition of a matrix A and the Schur decomposition of its symmetric embedding sym(A) = ([0A; A T 0]). In particular, if σ is a singular value of A then +σ and −σ are eigenvalues of the symmetric embedding. The top and bottom hal ..."
Abstract

Cited by 4 (0 self)
 Add to MetaCart
(Show Context)
Abstract. Well known connections exist between the singular value decomposition of a matrix A and the Schur decomposition of its symmetric embedding sym(A) = ([0A; A T 0]). In particular, if σ is a singular value of A then +σ and −σ are eigenvalues of the symmetric embedding. The top and bottom halves of sym(A)’s eigenvectors are singular vectors for A. Power methods applied to A can be related to power methods applied to sym(A). The rank of sym(A) is twice the rank of A. In this paper we develop similar connections for tensors by building on LH. Lim’s variational approach to tensor singular values and vectors. We show how to embed a general orderd tensor A into an orderd symmetric tensor sym(A). Through the embedding we relate power methods for A’s singular values to power methods for sym(A)’s eigenvalues. Finally, we connect the multilinear and outer product rank of A to the multilinear and outer product rank of sym(A). Key words. tensor, block tensor, symmetric tensor, tensor rank AMS subject classifications. 15A18, 15A69, 65F15