Abstract

We address the problem of visually detecting causal events and fitting them together into a coherent story of the action witnessed by the camera. We show that this can be done by reasoning about the motions and collisions of surfaces, using high-level causal constraints derived from psychological studies of infant visual behavior. These constraints are naive forms of basic physical laws governing substantiality, contiguity, momentum, and acceleration. We describe two implementations. One system parses instructional videos, extracting plans of action and key frames suitable for storyboarding. Since learning will play a role in making such systems robust, we introduce a new framework for coupling hidden Markov models and demonstrate its use in a second system that segments stereo video into actions in near real-time. Rather than attempt accurate low-level vision, both systems use highlevel causal analysis to integrate fast but sloppy pixelbased representations over time. The output is su...

Citations

282	Coupled Hidden Markov Models for Complex Action Recognition - Brand, Oliver, et al. - 1997
91	Real-time self-calibrating stereo person tracking using 3-d shape estimation from blob features - Azarbayejani, Pentland - 1996
62	Towards an Assembly Plan from Observation, Part I: Task Recognition with Polyhedral Objects - Ikeuchi, Suehiro - 1994
52	Coupled hidden markov models for modeling interacting processes,” MIT Media Laboratory Vision & Modeling - Brand - 1996
41	Qualitative recognition of ongoing human action sequences - Kuniyoshi, Inoue - 1993
36	Computational perception of scene dynamics - Mann, Jepson, et al. - 1996
33	A maximum-likelihood approach to visual event classification - Siskind, Morris - 1996
17	Perceiving and reasoning about objects: insights from infants - Spelke, Walle - 1993