w4 is a real time visual surveillance system for detecting and tracking multiple people and monitoring their activities in an outdoor environment. It operates on monocular gray-scale video imagery, or on video imagery from an infrared camera. W4 employs a combination of shape analysis and tracking to locate people and their parts (head, hands, feet, torso) and to create models of people's appearance so that they can be tracked through interactions such as occlusions. It can determine whether a foreground region contains multiple people and can segment the region into its constituent people and track them. W4 can also determine whether people are carrying objects, and can segment objects from their silhouettes, and construct appearance models for them so they can be identified in subsequent frames. W4 can recognize events between people and objects, such as depositing an object, exchanging bags, or removing an object. It runs at 25 Hz for 320x240 resolution images on a 400 Mhz dual-Pentium II PC.

In this paper we describe our work on 3-D modelbased tracking and recognition of human movement from real images. Our system has two major components. The first component takes real image sequences acquired from multiple views and recovers the 3-D body pose at each time instant. The poserecovery problem is formulated as a search problem and entails finding the pose parameters of a graphical human model for which its synthesized appearance is most similar to the actual appearance of the real human in the multi-view images. Currently, we use a best-first search technique and chamfer matching as a fast similarity measure between synthesized and real edge images. The second component of our system deals with the representation and recognition of human movement patterns. The recognition of human movement patterns is considered as a classification problem involving the matching of a test sequence with several reference sequences representing prototypical activities. A variation of dynamic ti...

Recently, there has been a movement in computer vision from the processing of static images to the processing of video sequences. Current research has begun to investigate the recognition of human activities taking place in the scene. Applications such as video database, virtual realityinterfaces, smart surveillance systems all have in common to track and interpret human activities. We propose a low-cost PC based real-time visual surveillance system, called W 4 , for tracking people and their body parts, and monitoring their activities in monochromatic and stereo imagery. It operates on grayscale video imagery, or on video imagery from an infrared camera. Unlike many systems for tracking people, our system make no use of color cue. Instead W 4 employs a combination of shape analysis, robust tracking techniques, silhouette based body model to locate and track the people and understand the interaction between people and objects - e.g., people exchanging objects, leaving obj...

We consider the problem of estimating the 3D motion of an articulated object, such as a robot arm or a human body, from a monocular sequence of 2D perspective views. We advocate an approach of decomposition. The object under analysis is decomposed into simpler parts, each containing a small number of links. We first estimate the motion of the simplest part(s) and then propagate the analysis to the remaining parts of the object. Human gait is used as an example; however, the approach is general. To use this decomposition approach, we need a repertoire of results for the motion of simple articulated objects: motion estimation algorithms, and uniqueness and number of solutions (especially, how many views are needed for uniqueness). With the help of techniques in algebraic geometry, we have results for a number of cases which are particularly useful for human gait analysis.

This dissertation presents an integrated human shape modeling, detection, and body part localization vision system. It demonstrates that the system can (1) detect pedestrians in various shapes, sizes, postures, partial occlusion, and clothing from a moving vehicle using stereo cameras; (2) locate the joints of a person automatically and accurately without employing any markers around the joints.

Human detection and body parts identification are important and challenging problems in computer vision. High performance human detection depends on reliable contour extraction, but contour extraction is an under constrained problem without the knowledge about the objects to be detected. This paper proposes a recursive context reasoning (RCR) approach to solving the above dilemma. A TRS 1 -invariant probabilistic model is designed to encode the shapes of the body parts and the context information --- the size and spatial relationships between body parts. A Bayesian framework is developed to perform human detection and part identification under partial occlusion. A contour reconstruction procedure is introduced to integrate the human model and the identified body parts to predict the shapes and locations of the parts missed by the contour detector; the refined contours are used to reevaluate the likelihood ratio. Therefore, contour extraction, part identification, and human detection ...