Abstract—Motion trajectories provide rich spatiotemporal information about an object’s activity. This paper presents novel classification algorithms for recognizing object activity using object motion trajectory. In the proposed classification system, trajectories are segmented at points of change in curvature, and the subtrajectories are represented by their principal component analysis (PCA) coefficients. We first present a framework to robustly estimate the multivariate probability density function based on PCA coefficients of the subtrajectories using Gaussian mixture models (GMMs). We show that GMM-based modeling alone cannot capture the temporal relations and ordering between underlying entities. To address this issue, we use hidden Markov models (HMMs) with a data-driven design in terms of number of states and topology (e.g., left-right versus ergodic). Experiments using a database of over 5700 complex trajectories (obtained from UCI-KDD data archives and Columbia University Multimedia Group) subdivided into 85 different classes demonstrate the superiority of our proposed HMM-based scheme using PCA coefficients of subtrajectories in comparison with other techniques in the literature. Index Terms—Activity recognition, Gaussian mixture models (GMMs), hidden Markov models (HMMs), trajectory modeling.

Abstract—This paper presents a novel motion trajectory-based compact indexing and efficient retrieval mechanism for video sequences. Assuming trajectory information is already available, we represent trajectories as temporal ordering of subtrajectories. This approach solves the problem of trajectory representation when only partial trajectory information is available due to occlusion. It is achieved by a hypothesis testing-based method applied to curvature data computed from trajectories. The subtrajectories are then represented by their principal component analysis (PCA) coefficients for optimally compact representation. Different techniques are integrated to index and retrieve subtrajectories, including PCA, spectral clustering, and string matching. We assume a query by example mechanism where an example trajectory is presented to the system and the search system returns a ranked list of most similar items in the dataset. Experiments based on datasets obtained from University of California at Irvine’s KDD archives and Columbia University’s DVMM group demonstrate the superiority of our proposed PCA-based approaches in terms of indexing and retrieval times and precision recall ratios, when compared to other techniques in the literature. Index Terms—Principal component analysis, spectral clustering, string Matching, trajectory retrieval. I.

The task of discovering natural groupings of input patterns, or clustering, is an important aspect of machine learning and pattern analysis. In this paper, we study the widely used spectral clustering algorithm which clusters data using eigenvectors of a similarity/affinity matrix derived from a data set. In particular, we aim to solve two critical issues in spectral clustering: (1) how to automatically determine the number of clusters, and (2) how to perform effective clustering given noisy and sparse data. An analysis of the characteristics of eigenspace is carried out which shows that (a) not every eigenvectors of a data affinity matrix is informative and relevant for clustering; (b) eigenvector selection is critical because using uninformative/irrelevant eigenvectors could lead to poor clustering results; and (c) the corresponding eigenvalues cannot be used for relevant eigenvector selection given a realistic data set. Motivated by the analysis, a novel spectral clustering algorithm is proposed which differs from previous approaches in that only informative/relevant eigenvectors are employed for determining the number of clusters and performing clustering. The key element of the proposed algorithm is a simple but effective relevance learning method which measures the relevance of an eigenvector according to how well it can separate the data set into different clusters. Our algorithm was evaluated using synthetic data sets as well as real-world data sets generated from two challenging visual learning problems. The results demonstrated that our algorithm is able to estimate the cluster number correctly and reveal natural grouping of the input data/patterns even given sparse and noisy data.

Applying advanced video technology to understand activity and intent is becoming increasingly important for intelligent video surveillance. We present a general model of a d-level dynamic Bayesian network to perform complex event recognition. The levels of the network are constrained to enforce state hierarchy while the d th level models the duration of simplest event. Moreover, in this paper we propose to use the deterministic annealing clustering method to automatically discover the states for the observable levels. We used real world data sets to show the effectiveness of our proposed method. 1.

Abstract not found

In this paper, we propose a technique for detecting possible events in outdoor areas monitored by a video surveillance system. In particular, here we focus on the time spent by an object to carry out simple events. To have a statistical representation of the times commonly required to perform certain activities, mixtures of Gaussians are maintained for each event type. Such statistics are then exploited both for the analysis of the simple activities and for discovering anomalous situations (i.e. complex events). In these cases, the system requires the attention of the human operator. A novel way of presenting results to the operator is also discussed. Experiments have been performed on a multi-camera system for parking lot security.

We present a systematic framework for arriving at audio classes for detection of crimes in elevators. We use our time series analysis framework proposed in5 to low-level features extracted from the audio of an elevator surveillance content to perform an inlier/outlier based temporal segmentation. Since suspicious events in elevators are outliers in a background of usual events, such a segmentation help bring out such events without any a priori knowledge. Then, by performing an automatic clustering on the detected outliers, we identify consistent patterns for which we can train supervised detectors. We apply the proposed framework to a colleciton of elevator surveillance audio data to systematically acquire audio classes such as banging, footsteps, nonneutral speech and normal speech etc. Based on the observation that the banging audio class and non-neutral speech class are indicative of suspicous events in the elevator data set, we are able to detect all of the suspicious activities without any misses.

It has become increasingly popular to study animal behaviors with the assistance of video recordings. The traditional manual human video annotation is a time and labor consuming process and, the observation results vary between different observers. Hence an automated video processing and behavior analysis system is desirable. We propose a framework for automatic video based behavior analysis systems, which consists of four major modules: behavior modeling, feature extraction from video sequences, basic behavior unit (BBU) discovery and complex behavior recognition. In this paper, we focus on BBU discovery using the affinity graph method on the feature data extracted from video images. We present a simple yet effective way of fusing information from multiple cameras in BBU discovery, and we present and analyze results on artificial mouse video using single, stereo and three cameras. Overall the results are encouraging.

This thesis has focused on the automatic video indexing of sports video, and in par-ticular the sub-domain of football. Televised sporting events are now commonplace especially with the arrival of dedicated digital TV channels, and as a consequence of this, large volumes of such data is generated and stored online. The current process for manually annotating video files is a time consuming and laborious task that is essen-tial for the management of large collections, especially when video is often re-used. Therefore, the development of automatic indexing tools would be advantageous for collection management, as well as the generation of a new wave of applications that are reliant on indexed video. Three main objectives were addressed successfully for football video indexing, con-centrating specifically on audio, a rich and low-dimensional information resource proven through experimentation. The first objective was an investigation into football video domain, analysing how prior knowledge can be utilised for automatic indexing. This was achieved through both inspection, and automatic content analysis, by applying the

Abstract — Activity recognition in video streams is increasingly important for both the computer vision and artificial intelligence communities. Activity recognition has many applications in security and video surveillance. Ultimately in such applications one wishes to recognize complex activities, which can be viewed as combination of simple activities. In this paper, we present a general framework of a Dlevel dynamic Bayesian network to perform complex activity recognition. The levels of the network are constrained to enforce state hierarchy while the Dth level models the duration of simplest event. Moreover, in this paper we propose to use the deterministic annealing clustering method to automatically define the simple activities, which corresponds to the low level states of observable levels in a Dynamic Bayesian Networks. We used real data sets for experiments. The experimental results show the effectiveness of our proposed method.