This paper presents an innovative method to interpret the content of a video scene using a depth camera. Cameras that provide distance instead of color information are part of a promising young technology but they come with many di culties: noisy signals, small resolution, and ambiguities, to cite a few. By taking advantage of the robustness to noise of a recent background subtraction algorithm, our method is able to extract useful information from the depth signals. We further enhance the robustness of the algorithm by combining this information with that of an RGB camera. In our experiments, we demonstrate this increased robustness and conclude by showing a practical example of an immersive application taking advantage of our algorithm.

technique for building databases of annotated and

Nowadays, techniques for real-time interpretation of video scenes are widespread. Amongst these techniques, the foreground segmentation is one of the favorite. It can be applied to color images as well as depth maps. The point of using depth maps is straightforward as a single color camera is not able to provide depth information. Technologies capable to acquire 3D informations are thus adequate to complement color cameras in consumer products. Practice has shown that 3D or RGB signals, taken alone, are unreliable to extract the foreground under arbitrary conditions. Therefore we combine both modalities to counter the intrinsic limitations of both modalities, which is only possible if the problems specific to a technology are handled appropriately. This paper presents a new global approach for enhanced foreground segmentation that handles limitations to 3D and RGB in a combined way.

Abstract—We present a unified framework for processing and representing images using a feature space related to local similarity. The visual data is represented by the versatile multiscale local jet feature space, possibly reduced by vector quantisation and/or represented by data structures enabling efficient nearest neighbours search (e.g. kd-trees). We demonstrate the interest of the local jet feature space processing through three fundamental low level tasks: noise reduction, motion estimation and background modelling/subtraction. We also show the potential of the framework in terms of higher level visual representation (e.g. recognition/retrieval). Keywords-vision framework; multiscale local jets; similarity space; nearest neighbours; optical flow; non local means; background modelling; I.

A key element for efficient video surveillance is situational awareness. Characteristics of human perception (e.g., inattentional blindness) as well as surveillance practice (e.g., CCTV operators have multiple responsibilities) often hinder comprehensive visual recognition of the activities in the monitored area. We support situational awareness and reduce the workload of CCTV operators by complementing the video display by an auditory display. Trajectories of moving objects extracted from surveillance video are sonified by auditory icons. These icons are interactively assigned by the user to each object category of the video and, in this way, form a sonic ecology. We use a spatial auditory display to represent location, direction and velocity of a trajectory with respect to a virtual listener. This facilitates orientation in virtual auditory space in a natural and realistic way that meets users ’ expectations. Modification areas are introduced to allow the users to define areas in which auditory icons are modified to further improve situational awareness. We put emphasis on efficient interaction between users and the auditory display to adjust the system according to the monitored area. Finally, we evaluate our approach by a user study and discuss benefits and shortcomings of the proposed sonification in the light of psychology, cognitive science, and neuroscience. 1.