This paper is concerned with allowing the user of a wearable, portable, vision system to interact with the visual information using hand movements and gestures. Two example scenarios are explored. The first, in 2D, uses the wearer's hand to both guide an active wearable camera and to highlight objects of interest using a grasping vector. The second is based in 3D, and builds on earlier work which recovers 3D scene structure at video-rate, allowing real-time purposive redirection of the camera to any scene point. Here, a range of hand gestures are used to highlight and select 3D points within the structure and in this instance used to insert 3D graphical objects into the scene. Structure recovery, gesture recognition, scene annotation and augmentation are achieved in parallel and at video-rate.

Abstract. This paper reviews aspects of the design and construction of an active wearable camera, and describes progress in equipping it with visual processing for reactive tasks like orientation stabilisation, slaving from head movements, and 2D tracking. The paper goes on to describe a first application of frame-rate simultaneous localisation and mapping (SLAM) to the wearable camera. Though relevant for any single camera undergoing general motion, the approach has particular benefits in wearable vision, allowing extended periods of purposive fixation followed by controlled redirection of gaze to other parts of the scene. 1

We report on ongoing research into how to statistically represent the experiences of a wearable computer user for the purposes of day-to-day behavior prediction. We combine natural sensor modalities (camera, microphone, gyros) with techniques for automatic labeling from sparsely labeled data. We have also taken the next required step to build robust statistical models by beginning an extensive data collection experiment, the “I Sensed ” series, a 100 day data set consisting of full surround video, audio, and orientation.

We develop a wearable vision system that consists of a user’s visual direction sensor and stereo cameras. First, we establish a method for calibrating the system so that it can detect user’s blink points even in a real situation such that the depth of blink points changes. Next, we propose a method for detecting a gazing region of a user in terms of the planar convex polygon. In our method, the system first identifies the fixation point of a user, and then applies a stereo algorithm and robust statistics to detect his gazing region. Now the system can detect the gazing region of a user and provide him with its 3D position. 1.

Using simultaneous localization and mapping to determine the 3D surroundings and pose of a wearable or hand-held camera provides the geometrical foundation for several capabilities of value to an autonomous wearable vision system. The one explored here is the ability to incorporate recognized objects into the map of the surroundings and refer to them. Established methods for feature cluster recognition are used to identify and localize known planar objects, and their geometry is incorporated into the map of the surrounds using a minimalist representation. Continued measurement of these mapped objects improves both the accuracy of estimated maps and the robustness of the tracking system. In the context of wearable (or hand-held) vision, the system’s ability to enhance generated maps with known objects increases the map’s value to human operators, and also enables meaningful automatic annotation of the user’s surroundings. 1

In this paper we treat design philosophies and enabling technologies for ambient awareness within grids of future mobile computing/communication devices. We extensively describe the possible context sensors, their required accuracies, their use in mobile services---possibly leading to background interactions of user devices---as well as a draft of their integration into an ambient aware device. We elaborate on position sensing as one of the main aspects of context aware systems. We first describe a maximum accuracy setup for a mobile user that has the ability of Augmented Reality for indoor and outdoor applications. We then focus on a set-up for pose sensing of a mobile user, based on the fusion of several inertia sensors and DGPS. We describe the anchoring of the position of the user by using visual tracking, using a camera and image processing. We describe our experimental set-up with a background process that, once initiated by the DGPS system, continuously looks in the image for visual clues and---when found---tries to track them, to continuously adjust the inertial sensor system. We present some results of our combined inertia tracking and visual tracking system; we are able to track device rotation and position with an update rate of 10 ms with an accuracy for the rotation of about two degrees, whereas head position accuracy is in the order of a few cm at a visual clue distance of less than 3 m.

The capability of a mobile robot to determine its position in the environment (self-localization) is a prerequisite for achieving autonomous navigation. An approach is proposed for determining the absolute orientation of an autonomous robot in a system of corridors, based on the projective geom-etry and active computer vision. In the proposed approach, the common direction of longitudinal corridor edges is inferred by detecting the vanishing point of the corresponding straight line segments in the image. It is assumed that the knowledge about the vertical direction in the scene is available, so that the image coordinates of these vanishing points are considerably constrained. However, lon-gitudinal corridor edges are not visible in images acquired for many viewing directions, so that the processing in a localization procedure has to be performed on a sequence of images acquired from the given position, for regularly arranged orientations of the camera. Extensive experimentation was performed on real scenes and the obtained results are provided. Keywords: Machine vision, active vision, robot vision systems, image recognition, image edge analysis, image

Abstract — In this paper, we describe an autonomous, wearable electronic textile location awareness system that will determine a user’s location within a building given a map of that building. The system uses a moderate number of ultrasonic range transceivers as the sensing elements. Given a set of range readings from these sensors, the system attempts to match those actual readings to expected readings associated with a set of candidate locations for the wearer. These expected readings are calculated using a simulation model of the propagation of ultrasonic signals within a building. An additional algorithm is given for determining the wearer’s movement between rooms, allowing for the uncertainty associated with sensor readings in complex, multi-room environments. A wearable prototype system is described and results from this system in a variety of situations are presented. I.

Abstract- This paper introduces a novel navigation system for walking persons. The system is of particular benefit for emergency responders, who often have to enter and move around in large structures where GPS is unavailable. We refer to our system as “Personal Odometry System ” (POS). The POS measures the position of a walking person relative to a known starting position, such as the entrance to a building. This is accomplished by instrumenting one boot of the subject with a 3-axis gyroscope and a 3-axis accelerometer (collectively called “inertial measurement unit ” – IMU). This paper describes the POS hardware and explains the basics of our approach. The paper also presents extensive experimental results, which illustrate the utility and practicality of our system. I.

This paper introduces a “Personal Dead-reckoning ” (PDR) navigation system for walking persons. The system is useful for monitoring the position of emergency responders inside buildings, where GPS is unavailable. The PDR system uses a six-axes Inertial Measurement Unit attached to the user’s boot. The system’s strength lies in the use of a technique known as “Zero Velocity Update ” (ZUPT) that virtually eliminates the ill-effects of drift in the accelerometers. It works very well with different gaits, as well as on stairs, slopes, and generally on 3-dimensional terrain. This paper explains the PDR and presents extensive experimental results, which illustrate the utility and practicality of the system.