Abstract

Anomalies in many video surveillance applications have local spatio-temporal signatures, namely, they occur over a small time window or a small spatial region. The dis-tinguishing feature of these scenarios is that outside this spatio-temporal anomalous region, activities appear nor-mal. We develop a probabilistic framework to account for such local spatio-temporal anomalies. We show that our framework admits elegant characterization of optimal deci-sion rules. A key insight of the paper is that if anomalies are lo-cal optimal decision rules are local even when the nomi-nal behavior exhibits global spatial and temporal statisti-cal dependencies. This insight helps collapse the large am-bient data dimension for detecting local anomalies. Con-sequently, consistent data-driven local empirical rules with provable performance can be derived with limited training data. Our empirical rules are based on scores functions de-rived from local nearest neighbor distances. These rules ag-gregate statistics across spatio-temporal locations & scales, and produce a single composite score for video segments. We demonstrate the efficacy of our scheme on several video surveillance datasets and compare with existing work. 1.