This paper presents an online learning algorithm to construct from video sequences an image-based representation that is useful for recognition and tracking. For a class of objects (e.g., human faces), a generic representation of the appearances of the class is learned off-line. From video of an instance of this class (e.g., a particular person), an appearance model is incrementally learned on-line using the prior generic model and successive frames from the video. More specifically, both the generic and individual appearances are represented as an appearance manifold that is approximated by a collection of sub-manifolds (named pose manifolds) and the connectivity between them. In turn, each submanifold is approximated by a low-dimensional linear subspace while the connectivity is modeled by transition probabilities between pairs of sub-manifolds. We demonstrate that our online learning algorithm constructs an effective representation for face tracking, and its use in video-based face recognition compares favorably to the representation constructed with a batch technique. 1

We introduce a new framework, namely Tensor Canonical Correlation Analysis (TCCA) which is an extension of classical Canonical Correlation Analysis (CCA) to multidimensional data arrays (or tensors) and apply this for action/gesture classification in videos. By Tensor CCA, joint space-time linear relationships of two video volumes are inspected to yield flexible and descriptive similarity features of the two videos. The TCCA features are combined with a discriminative feature selection scheme and a Nearest Neighbor classifier for action classification. In addition, we propose a time-efficient action detection method based on dynamic learning of subspaces for Tensor CCA for the case that actions are not aligned in the space-time domain. The proposed method delivered significantly better accuracy and comparable detection speed over state-of-the-art methods on the KTH action data set as well as self-recorded hand gesture data sets. 1.

The methods for visual learning that compute a space of eigenvectors by Principal Component Analysis (PCA) traditionally require a batch computation step. Since this leads to potential problems when dealing with large sets of images, several incremental methods for the computation of the eigenvectors have been introduced. However, such learning cannot be considered as an on-line process, since all the images are retained until the final step of computation of space of eigenvectors, when their coefficients in this subspace are computed. In this paper we propose a method that allows for simultaneous learning and recognition. We show that we can keep only the coefficients of the learned images and discard the actual images and still are able to build a model of appearance that is fast to compute and open-ended. We performed extensive experimental testing which showed that the recognition rate and reconstruction accuracy are comparable to those obtained by the batch method.

Principal Component Analysis (PCA) has been of great interest in computer vision and pattern recognition. In particular, incrementally learning a PCA model, which is computationally e#cient for large scale problems as well as adaptable to reflect the variable state of a dynamic system, is an attractive research topic with numerous applications such as adaptive background modelling and active object recognition. In addition, the conventional PCA, in the sense of least mean squared error minimisation, is susceptible to outlying measurements.

Density-based modeling of visual features is very common in computer vision, either by using non-parametric techniques or through representing the underlying density function as a weighted sum of Gaussians. A number of real-time tasks such as background modeling or modeling the appearance of a moving target require sequential density estimation, where new data is incorporated in the model as it becomes available. Nevertheless, current methods for updating the density function either lack flexibility, by fixing the number of Gaussians in the mixture, or require large memory amounts, by maintaining a non-parametric representation of the density. This paper presents an efficient method for recursive density approximation that relies on the propagation of the density modes. At each time step, the modes of the density are re-estimated and a Gaussian component is assigned to each mode. The covariance of each component is derived from the Hessian matrix estimated at the mode location. To detect the modes we employ the variable-bandwidth mean shift. While the proposed density representation is memory efficient (which is typical for mixture densities), it inherits the flexibility of non-parametric methods, by allowing the number of modes to adapt in time. We show that the same mode propagation principle applies for subspaces derived from eigen analysis. Extensive experimental background modeling results demonstrate the performance of the method.

This paper presents a constructive method for deriving an updated discriminant eigenspace for classification, when bursts of data that contains new classes is being added to an initial discriminant eigenspace in the form of random chunks. Basically, we propose an incremental linear discriminant analysis (ILDA) in its two forms: a sequential ILDA; and a Chunk ILDA. In experiments, we have tested ILDA using datasets with a small number of classes and smalldimensional features, as well as datasets with a large number of classes and large-dimensional features. We have compared the proposed ILDA against the traditional batch LDA in terms of discriminability, execution time and memory usage with the increasing volume of data addition. The results show that the proposed ILDA can effectively evolve a discriminant eigenspace over a fast and large data stream, and extract features with superior discriminability in classification, when compared with other methods.

In spite of over two decades of intense research, illumination and pose invariance remain prohibitively challenging aspects of face recognition for most practical applications. The objective of this work is to recognize faces using video sequences both for training and recognition input, in a realistic, unconstrained setup in which lighting, pose and user motion pattern have a wide variability and face images are of low resolution.

In this paper we address the problem of classifying vector sets. We motivate and introduce a novel method based on comparisons between corresponding vector subspaces. In particular, there are two main areas of novelty: (i) we extend the concept of principal angles between linear subspaces to manifolds with arbitrary nonlinearities; (ii) it is demonstrated how boosting can be used for application-optimal principal angle fusion. The strengths of the proposed method are empirically demonstrated on the task of automatic face recognition (AFR), in which it is shown to outperform state-of-the-art methods in the literature.

Abstract—In multitarget tracking, the main challenge is to maintain the correct identity of targets even under occlusions or when differences between the targets are small. The paper proposes a new approach to this problem by incorporating the context information. The context of a target in an image sequence has two components: the spatial context including the local background and nearby targets and the temporal context including all appearances of the targets that have been seen previously. The paper considers both aspects. We propose a new model for multitarget tracking based on the classification of each target against its spatial context. The tracker searches a region similar to the target while avoiding nearby targets. The temporal context is included by integrating the entire history of target appearance based on probabilistic principal component analysis (PPCA). We have developed a new incremental scheme that can learn the full set of PPCA parameters accurately online. The experiments show robust tracking performance under the condition of severe clutter, occlusions, and pose changes. Index terms—Multitarget tracking, context-based tracking, probabilistic PCA. Ç 1

In this paper, we introduce a novel approach to modeling non-stationary random processes. Given a set of training samples sequentially, we can iteratively update an eigenspace to manifest the current statistics provided by each new sample. The updated eigenspace is derived more from recent samples and less from older samples, controlled by a number of decay parameters. Extensive study has been performed on how to choose these decay parameters. Other existing eigenspace updating algorithms can be regarded as special cases of our algorithm. We show the effectiveness of the proposed algorithm with both synthetic data and practical applications for face recognition. Significant improvements have been observed in recognizing face images with different variations, such as pose, expression and illumination variations. We also expect the proposed algorithm to have other applications in active recognition and modeling.