Abstract

With the ubiquity of powerful, mobile computers and rapid advances in sens-ing and robot technologies, there exists a great potential for creating advanced, in-telligent computing environments. We investigate techniques for integrating passive, vision-based sensing into such environments, which include both conventional inter-faces and large-scale environments. We propose a new methodology for vision-based human-computer interaction called the Visual Interaction Cues (VICs) paradigm. VICs fundamentally relies on a shared perceptual space between the user and com-puter using monocular and stereoscopic video. In this space, we represent each inter-face component as a localized region in the image(s). By providing a clearly defined interaction locale, it is not necessary to visually track the user. Rather we model interaction as an expected stream of visual cues corresponding to a gesture. Example interaction cues are motion as when the finger moves to press a push-button, and 3D hand posture for a communicative gesture like a letter in sign language. We ex-plore both procedurally defined parsers of the low-level visual cues and learning-based

Citations