In this paper we introduce a novel method to detect and localize abnormal behaviors in crowd videos using Social Force model. For this purpose, a grid of particles is placed over the image and it is advected with the space-time average of optical flow. By treating the moving particles as individuals, their interaction forces are estimated using social force model. The interaction force is then mapped into the image plane to obtain Force Flow for every pixel in every frame. Randomly selected spatio-temporal volumes of Force Flow are used to model the normal behavior of the crowd. We classify frames as normal and abnormal by using a bag of words approach. The regions of anomalies in the abnormal frames are localized using interaction forces. The experiments are conducted on a publicly available dataset from University of Minnesota for escape panic scenarios and a challenging dataset of crowd videos taken from the web. The experiments show that the proposed method captures the dynamics of the crowd behavior successfully. In addition, we have shown that the social force approach outperforms similar approaches based on pure optical flow.

Extremely crowded scenes present unique challenges to video analysis that cannot be addressed with conventional approaches. We present a novel statistical framework for modeling the local spatio-temporal motion pattern behavior of extremely crowded scenes. Our key insight is to exploit the dense activity of the crowded scene by modeling the rich motion patterns in local areas, effectively capturing the underlying intrinsic structure they form in the video. In other words, we model the motion variation of local spacetime volumes and their spatial-temporal statistical behaviors to characterize the overall behavior of the scene. We demonstrate that by capturing the steady-state motion behavior with these spatio-temporal motion pattern models, we can naturally detect unusual activity as statistical deviations. Our experiments show that local spatio-temporal motion pattern modeling offers promising results in realworld scenes with complex activities that are hard for even human observers to analyze. 1.

Video-surveillance systems are becoming more and more autonomous in the detection and the reporting of abnormal events. In this context, this paper presents an approach to detect abnormal situations in crowded scenes by analyzing the motion aspect instead of tracking subjects one by one. The proposed approach estimates sudden changes and abnormal motion variations of a set of points of interest (POI). The number of tracked POIs is reduced using a mask that corresponds to hot areas of the built motion heat map. The approach detects events where local motion variation is important compared to previous events. Optical flow techniques are used to extract information such as density, direction and velocity. To demonstrate the interest of the approach, we present the results on the detection of collapsing events in real videos of airport escalator exits. 1.

A novel method for crowd flow modeling and anomaly detection is proposed for both coherent and incoherent scenes. The novelty is revealed in three aspects. First, it is a unique utilization of particle trajectories for modeling crowded scenes, in which we propose new and efficient representative trajectories for modeling arbitrarily complicated crowd flows. Second, chaotic dynamics are introduced into the crowd context to characterize complicated crowd motions by regulating a set of chaotic invariant features, which are reliably computed and used for detecting anomalies. Third, a probabilistic framework for anomaly detection and localization is formulated. The overall work-flow begins with particle advection based on optical flow. Then particle trajectories are clustered to obtain representative trajectories for a crowd flow. Next, the chaotic dynamics of all representative trajectories are extracted and quantified using chaotic invariants known as maximal Lyapunov exponent and correlation dimension. Probabilistic model is learned from these chaotic feature set, and finally, a maximum likelihood estimation criterion is adopted to identify a query video of a scene as normal or abnormal. Furthermore, an effective anomaly localization algorithm is designed to locate the position and size of an anomaly. Experiments are conducted on known crowd data set, and results show that our method achieves higher accuracy in anomaly detection and can effectively localize anomalies. 1. Introduction and Related

This paper presents a method for monitoring activities at a ticket vending machine in a video-surveillance context. Rather than relying on the output of a tracking module, which is prone to errors, the events are direclty recognized from image measurements. This especially does not require tracking. A statistical layered approach is proposed, where in the first layer, several sub-events are defined and detected using a discriminative approach. The second layer uses the result of the first and models the temporal relationships of the highlevel event using a Hidden Markov Model (HMM). Results are assessed on 3h30 hours of real video footage coming from Turin metro station. 1

Abstract not found

Abstract. Crowd behavior recognition is becoming an important research topic in video surveillance for public places. In this paper, we first discuss the crowd feature selection and extraction and propose a multiple-frame feature point detection and tracking based on the KLT tracker. We state that behavior modelling of crowd is usually coarse compared to that for individuals. Instead of developing general crowd behavior models, we propose to model crowd events for specific end-user scenarios. As a result, a same type of event may be modelled slightly differently from one scenario to another and several models are to be defined. Consequently, fast modelling is required and this is enabled by the use of an extended Scenario Recognition Engine (SRE) in our approach. Crowd event models are defined; particularly, composite events accommodating evidence accumulation allow to increase detection reliability. Tests have been conducted on real surveillance video sequences containing crowd scenes. The crowd tracking algorithm proves to be robust and gives reliable crowd motion vectors. The crowd event detection on real sequences gives reliable results of a few common crowd behaviors by simple dedicated models. Key words: Automatic video-based surveillance, crowd tracking, dedicated modelling, crowd behavior recognition 1

Video footage of real crowded scenes still poses severe challenges for automated surveillance. This paper evaluates clustering methods for finding independent dominant motion fields for an observation period based on a recently published real-time optical flow algorithm. We focus on self-tuning spectral clustering and Isomap combined with k-means. Several combinations of feature vector normalizations and distance measures (Euclidean, Mahanalobis and a general additive distance) are evaluated for four image sequences including three publicly available crowd datasets. Evaluation is based on mean accuracy obtained by comparison with a manually defined ground truth clustering. For every dataset at least one approach correctly classified more than 95 % of the flow vectors without extra tuning of parameters, providing a basis for an automatic analysis after a view-dependent setup.

Modern video surveillance systems generate diverse forms of data and to facilitate the effective exchange of these data a methodical approach is required. This thesis proposes the Video Surveillance Content Description Interface (VSCDI), a component of ISO/IEC 23000-10 – Information technology – Multimedia application format (MPEG-A) – Part 10: Video surveillance application format. The interface is designed to describe content associated with and generated by a surveillance system. In particular, a set of descriptors are included for: content-based image retrieval; user-defined Classification Schemes to impose any required description ontology; and to provide consistent descriptions across multiple sources. The VSCDI is evaluated using comparisons with other meta-data frameworks and in terms of the performance of its colour descriptor components. Two new data sets are created of pedestrians in indoor environments with multiple camera views for re-identification experiments. The experiments use a novel application of colour constancy for cross-camera comparisons. Two evaluation measures are used: the Average Normalised Mean Retrieval Rate (ANMRR) for ranked estimates; and the Information Gain metric for probabilistic estimates. Techniques are investigated for using more than one descriptor both to provide the estimate and to represent a person whose image is split into Top and Bottom clothing components. The re-identification of pedestrians is discussed in the context of providing both a coherent description of the overall scene activity and within an embedded system. 2

Enabling computers to understand human and animal behavior has the potential to revolutionize many areas that benefit society such as clinical diagnosis, humancomputer interaction, and social robotics. Critical to the understanding of human and animal behavior, and any temporally-varying phenomenon in general, is the capability to segment, classify, and cluster time series data. This thesis proposes segment-based Support Vector Machines (Seg-SVMs), a framework for supervised, weakly-supervised, and unsupervised time series analysis. Seg-SVMs outperform state-of-the-art approaches by combining three powerful ideas: energy-based structure prediction, bag-of-words representation, and maximum-margin learning. Energy-based structure prediction provides a principled mechanism for concurrent top-down recognition and bottom-up temporal localization. Bag-of-words representation provides segment-based features that tolerate misalignment errors and are computationally efficient. Maximum-margin learning, such as SVM and Structure Output SVM, has a convex learning formulation; it produces classifiers that are discriminative and less prone to over-fitting. In this thesis, we show how Seg-SVMs outperform state-of-the-art approaches for segmenting, classifying, and clustering human and animal behavior in video and accelerometer data of varying complexity. We illustrate these benefits in the problems of facial event detection, sequence labeling of human actions, and temporal clustering of animal behavior. In addition, the Seg-SVMs framework naturally provides solutions to two novel problems: early detection of human actions and weaklysupervised discovery of discriminative events.