## Sparse image coding using a 3D non-negative tensor factorization (2005)

### Cached

### Download Links

- [www.cs.huji.ac.il]
- [www.cs.huji.ac.il]
- [www.cs.huji.ac.il]
- [mall.psy.ohio-state.edu]
- DBLP

### Other Repositories/Bibliography

Venue: | In: International Conference of Computer Vision (ICCV |

Citations: | 47 - 2 self |

### BibTeX

@INPROCEEDINGS{Hazan05sparseimage,

author = {Tamir Hazan and Simon Polak and Amnon Shashua},

title = {Sparse image coding using a 3D non-negative tensor factorization},

booktitle = {In: International Conference of Computer Vision (ICCV},

year = {2005},

pages = {50--57}

}

### Years of Citing Articles

### OpenURL

### Abstract

We introduce an algorithm for a non-negative 3D tensor factorization for the purpose of establishing a local parts feature decomposition from an object class of images. In the past such a decomposition was obtained using nonnegative matrix factorization (NMF) where images were vectorized before being factored by NMF. A tensor factorization (NTF) on the other hand preserves the 2D representations of images and provides a unique factorization (unlike NMF which is not unique). The resulting ”factors” from the NTF factorization are both sparse (like with NMF) but also separable allowing efficient convolution with the test image. Results show a superior decomposition to what an NMF can provide on all fronts — degree of sparsity, lack of ghost residue due to invariant parts and efficiency of coding of around an order of magnitude better. Experiments on using the local parts decomposition for face detection using SVM and Adaboost classifiers demonstrate that the recovered features are discriminatory and highly effective for classification. 1.

### Citations

2283 | Rapid object detection using a boosted cascade of simple features
- Viola, Jones
(Show Context)
Citation Context ...the worst efficiency would be the product of dimensions r · s, a separable filter will have an efficiency of r + s and there are filter designs with an efficiency of O(1) operations per output pixel (=-=[24, 8, 11]-=-, and references therein). It is then a matter of choosing a subset of those filters that are the most ”relevant” for the object class, i.e., provide the most accurate classification scores for a give... |

1677 | Eigenfaces vs. fisherfaces: recognition using class specific linear projection
- Belhumeur, Hespanha, et al.
- 1997
(Show Context)
Citation Context ...is to find a set of mutually orthogonal basis vectors that capture the directions of maximum variance in the data. In computer vision PCA has been used for the representation and recognition of faces =-=[21, 22, 2]-=-, recognition of 3D objects under varying pose [15], tracking of deformable objects [4] and for representations of 3D range data of heads [1]. Higher-order (tensor) decompositions, treating the traini... |

1468 |
Independent component analysis, a new concept
- Comon
- 1994
(Show Context)
Citation Context ...ge of the class is represented in terms of a small number of basis images out of a large set have been proposed [16] and closely related to that is the work on Independent Component Analysis (ICA) by =-=[7, 3]-=-. A sparse representation has also been achieved using a non-negative factorization (NMF) of the matrix V whose columns are the vectorized training images. The factorization process seeks a decomposit... |

1442 | A training algorithm for optimal margin classifiers
- Boser
- 1992
(Show Context)
Citation Context ...o the NTF with the same number of factors (a 20-fold reduction in space). The next experiment, shown in Fig. 5, used the filter responses as measurements for a Support Vector Machine (SVM) classifier =-=[5]-=-. We used the MIT CBCL face set to re6 linear poly d = 5 RBF NTF (50) 91.9% 95.3% 95.9% NMF (50) 91.6% 94% 95% NMF (20) 87.5% 90.1% 89% NMF (6) 83.2% 84.3% 86% PCA 90.8% 94% 91.7% Figure 5: Using the ... |

1111 |
Learning the parts of the objects by non-negative matrix factorization
- Lee, Seung
- 1999
(Show Context)
Citation Context ...an the number of images. The columns of W form the new basis vectors and due to the non-negativity constraint both the basis vectors and the mixing coefficients (columns of H) tend to come out sparse =-=[17, 14]-=-. In particular, [14] introduce a simple and effective iterative technique for performing the decomposition. A sparse decomposition is appealing for a number of reasons. First, the filters represent l... |

1013 | Visual Learning and Recognition of 3D Objects from Appearance - Murase, Nayar - 1995 |

1013 |
D.J.: Emergence of simple-cell receptive field properties by learning a sparse code for natural images
- Olshausen, Field
- 1996
(Show Context)
Citation Context ...courage a sparse structure, i.e., the basis images (filters) come out sparse and any image of the class is represented in terms of a small number of basis images out of a large set have been proposed =-=[16]-=- and closely related to that is the work on Independent Component Analysis (ICA) by [7, 3]. A sparse representation has also been achieved using a non-negative factorization (NMF) of the matrix V whos... |

569 | Eigentracking: Robust matching and tracking of articulated objects using a view-based representation
- Black, Jepson
- 1998
(Show Context)
Citation Context ...riance in the data. In computer vision PCA has been used for the representation and recognition of faces [21, 22, 2], recognition of 3D objects under varying pose [15], tracking of deformable objects =-=[4]-=- and for representations of 3D range data of heads [1]. Higher-order (tensor) decompositions, treating the training images as a 3D cube, have been also proposed where the idea that preserving the 2D f... |

483 |
Low-dimensional procedure for the characterization of human faces
- Sirovich, Kirby
- 1987
(Show Context)
Citation Context ...is to find a set of mutually orthogonal basis vectors that capture the directions of maximum variance in the data. In computer vision PCA has been used for the representation and recognition of faces =-=[21, 22, 2]-=-, recognition of 3D objects under varying pose [15], tracking of deformable objects [4] and for representations of 3D range data of heads [1]. Higher-order (tensor) decompositions, treating the traini... |

319 |
An informationb maximization approach to blind separation and blind deconvolutio
- Bell, Sejnowski
- 1995
(Show Context)
Citation Context ...ge of the class is represented in terms of a small number of basis images out of a large set have been proposed [16] and closely related to that is the work on Independent Component Analysis (ICA) by =-=[7, 3]-=-. A sparse representation has also been achieved using a non-negative factorization (NMF) of the matrix V whose columns are the vectorized training images. The factorization process seeks a decomposit... |

307 |
Positive matrix factorization: a non-negative factor model with optimal utilization of error estimates of data values
- Paatero, Tapper
- 1994
(Show Context)
Citation Context ...an the number of images. The columns of W form the new basis vectors and due to the non-negativity constraint both the basis vectors and the mixing coefficients (columns of H) tend to come out sparse =-=[17, 14]-=-. In particular, [14] introduce a simple and effective iterative technique for performing the decomposition. A sparse decomposition is appealing for a number of reasons. First, the filters represent l... |

306 |
Foundations of the PARAFAC procedure: Models and conditions for an "explanatory" multi-modal factor analysis
- Harshman
- 1970
(Show Context)
Citation Context ...e a reduction to SVD when the image cube is reduced at the limit to copies of a single image [19] or to enforce certain orthogonality constraints (also known as HighOrder-SVD) among the basis vectors =-=[13, 23, 26, 10]-=-. The PCA and HOSVD techniques tend to generate filters (basis elements) which have a ”holistic” form, i.e., the energy is spread throughout the filter. Techniques which encourage a sparse structure, ... |

274 | A multilinear singular value decomposition
- Lathauwer, Moor, et al.
(Show Context)
Citation Context ...e a reduction to SVD when the image cube is reduced at the limit to copies of a single image [19] or to enforce certain orthogonality constraints (also known as HighOrder-SVD) among the basis vectors =-=[13, 23, 26, 10]-=-. The PCA and HOSVD techniques tend to generate filters (basis elements) which have a ”holistic” form, i.e., the energy is spread throughout the filter. Techniques which encourage a sparse structure, ... |

262 |
Summed-area tables for texture mapping
- CROW
- 1984
(Show Context)
Citation Context ...the worst efficiency would be the product of dimensions r · s, a separable filter will have an efficiency of r + s and there are filter designs with an efficiency of O(1) operations per output pixel (=-=[24, 8, 11]-=-, and references therein). It is then a matter of choosing a subset of those filters that are the most ”relevant” for the object class, i.e., provide the most accurate classification scores for a give... |

152 | Multilinear analysis of image ensembles: Tensorfaces
- Vasilescu, Terzopoulos
- 2002
(Show Context)
Citation Context ...e a reduction to SVD when the image cube is reduced at the limit to copies of a single image [19] or to enforce certain orthogonality constraints (also known as HighOrder-SVD) among the basis vectors =-=[13, 23, 26, 10]-=-. The PCA and HOSVD techniques tend to generate filters (basis elements) which have a ”holistic” form, i.e., the energy is spread throughout the filter. Techniques which encourage a sparse structure, ... |

115 | When does non-negative matrix factorization give a correct decomposition into parts
- Donoho, Stodden
- 2003
(Show Context)
Citation Context ...d image). What is less clear is whether the NMF process will yield the underlying generative parts (even when there is a perfect fit V = W H)? in general this is not true. This point was addressed by =-=[9]-=- where they came up with a set of ”rules” that would guarantee a unique decomposition, but the set of rules does not include the common situation of invariant parts which in fact create ”ghosts” in th... |

91 | Nonnegative tensor factorization with applications to statistics and computer vision
- Shashua, Hazan
- 2005
(Show Context)
Citation Context ...thematical reasoning we can prove the convergence of the computational updates for the vectors {v m , w m } k m=1. The derivation of the general n-way array non-negative decomposition can be found in =-=[18]-=-. 2.1. NTF under Relative Entropy Following [14] we also describe positive-preserving update rule for the relative entropy cost function. Given a tensor G we consider the best positive rank-k approxim... |

57 |
Rank-one approximation to high order tensors
- Zhang, Golub
- 2001
(Show Context)
Citation Context .... This is not true with tensors in general, i.e., repeatedly subtracting the dominant rank-1 tensor is not a converging process, but only under special cases of orthogonally decomposable tensors (see =-=[27]-=-). Another striking difference, this time in favor of tensor ranks, is that unlike matrix factorization, which is generFigure 1: Representation of the image set as a 3-way array and its rank-k factor... |

54 |
Linear image coding for regression and classification using the tensor-rank principle
- Shashua, Levin
- 2001
(Show Context)
Citation Context ... techniques are based on preserving some features of Singular Value Decomposition (SVD) such as to guarantee a reduction to SVD when the image cube is reduced at the limit to copies of a single image =-=[19]-=- or to enforce certain orthogonality constraints (also known as HighOrder-SVD) among the basis vectors [13, 23, 26, 10]. The PCA and HOSVD techniques tend to generate filters (basis elements) which ha... |

54 |
Eigen faces for recognition
- Turk, Pentland
- 1991
(Show Context)
Citation Context ...is to find a set of mutually orthogonal basis vectors that capture the directions of maximum variance in the data. In computer vision PCA has been used for the representation and recognition of faces =-=[21, 22, 2]-=-, recognition of 3D objects under varying pose [15], tracking of deformable objects [4] and for representations of 3D range data of heads [1]. Higher-order (tensor) decompositions, treating the traini... |

51 | On the uniqueness of multilinear decomposition of N-way arrays
- Sidiropoulos, Bro
- 2000
(Show Context)
Citation Context ... v j⊤ . ally non-unique for any rank greater than one, a 3-valence tensor decomposition is essentially unique under mild conditions [12] and the situation actually improves in higher dimensions n > 3 =-=[20]-=-. The uniqueness property (and this before we introduce non-negativity constraints) is crucial for the sparse coding application mentioned in the previous section as an NMF is not generally unique. An... |

47 | Positive tensor factorization
- Welling, Weber
(Show Context)
Citation Context ...ases where the decomposition is orthogonal, whereas in this paper we are interested in the general multi-linear factorization with non-negative entries. A recent attempt to perform an NTF was made by =-=[25]-=- who introduced a heuristic iterative update rule which lacked a convergence proof. Their scheme was based on flattening the tensor into a matrix representation rather than working directly with the o... |

34 | Optimality, computation and interpretation of nonnegative matrix factorizations. Preprint. Available online at http://www4.ncsu.edu/∼mtchu/Research/Papers/nnmf.ps
- Chu, Diele, et al.
- 2005
(Show Context)
Citation Context ...e decomposition, but the set of rules does not include the common situation of invariant parts which in fact create ”ghosts” in the factors and contaminate the sparsity of the basis vectors (see also =-=[6]-=-). A non-negative tensor factorization (NTF) strategy will represent the input image collection as a 3-way array. A rank-k factorization would correspond to a collection of k 2 rank-1 matrices (the ba... |

20 | Real time pattern matching using projection kernels
- Hel-Or, Hel-Or
- 2003
(Show Context)
Citation Context ...the worst efficiency would be the product of dimensions r · s, a separable filter will have an efficiency of r + s and there are filter designs with an efficiency of O(1) operations per output pixel (=-=[24, 8, 11]-=-, and references therein). It is then a matter of choosing a subset of those filters that are the most ”relevant” for the object class, i.e., provide the most accurate classification scores for a give... |

5 |
Statistical approach to shape-from-shading: deriving 3d face surfaces from single 2d images
- Atick, Griffin, et al.
- 1997
(Show Context)
Citation Context ...ed for the representation and recognition of faces [21, 22, 2], recognition of 3D objects under varying pose [15], tracking of deformable objects [4] and for representations of 3D range data of heads =-=[1]-=-. Higher-order (tensor) decompositions, treating the training images as a 3D cube, have been also proposed where the idea that preserving the 2D form of images is necessary for preserving the spatial ... |

2 |
Three way arrays: rank and uniqueness of trilinear decomposition, with application to arithmetic complexity and statistics. Linear Algebra and its Applications
- Kruksal
- 1977
(Show Context)
Citation Context ... image Ai is expressed as a superposition of the rank-1 matrices u j v j⊤ . ally non-unique for any rank greater than one, a 3-valence tensor decomposition is essentially unique under mild conditions =-=[12]-=- and the situation actually improves in higher dimensions n > 3 [20]. The uniqueness property (and this before we introduce non-negativity constraints) is crucial for the sparse coding application men... |

2 |
Cramer-rao lower boubds for lowrank decomposition of multidimensional arrays
- Xianqian, Sidiropoulos
- 2001
(Show Context)
Citation Context |