A structure from motion algorithm is described which recovers structure and camera position, modulo a projective ambiguity. Camera calibration is not required, and camera parameters such as focal length can be altered freely during motion. The structure is updated sequentially over an image sequence, in contrast to schemes which employ a batch process. A specialisation of the algorithm to recover structure and camera position modulo an affine transformation is described, together with a method to periodically update the affine coordinate frame to prevent drift over time. We describe the constraint used to obtain this specialisation. Structure is recovered from image corners detected and matched automatically and reliably in real image sequences. Results are shown for reference objects and indoor environments, and accuracy of recovered structure is fully evaluated and compared for a number of reconstruction schemes. A specific application of the work is demonstrated -- affine structure is used to compute free space maps enabling navigation through unstructured environments and avoidance of obstacles. The path planning involves only affine constructions.

Active cameras provide a navigating vehicle with the ability to fixate and track features over extended periods of time, and wide fields of view. While it is relatively straightforward to apply fixating vision to tactical, short-term navigation tasks, using serial fixation on a succession of features to provide global information for strategic navigation is more involved. However, active vision is seemingly well-suited to this task: the ability to measure features over such a wide range means that the same ones can be used as a robot makes a wide range of movements. This has advantages for map-building and localisation. The core work of this thesis concerns simultaneous localisation and map-building for a robot with a stereo active head, operating in an unknown environment and using point features in the world as visual landmarks. Importance has been attached to producing maps which are useful for extended periods of navigation. Many map-building methods fail on extended runs because ...

This paper describes the design, implementation and testing of a high speed controlled stereo "head/eye" platform which facilitates the rapid redirection of gaze in response to visual input. It details the mechanical device, which is based around geared DC motors, and describes hardware aspects of the controller and vision system, which are implemented on a reconfigurable network of general purpose parallel processors. The servo-controller is described in detail and higher level gaze and vision constructs outlined. The paper gives performance figures gained both from mechanical tests on the platform alone, and from closed loop tests on the entire system using visual feedback from a feature detector. 1. INTRODUCTION It is evident that sensing and perception are essential precursors to intelligent robotic action in dynamic and unstructured environments. It is therefore surprising to note that the principal emphasis of research into computational visual sensing and perception over the la...

Abstract. This paper presents a new algorithm for structure from motion from an arbitrary number of tracked features over an arbitrary number of images, which possesses several advantages over previous formulations. First, it is recursive, so the time complexity is independent of the number of images. The complexity is linear with the number of tracked features. The algorithm allows newly appeared features to be included, stale features to be discarded, and missing data to be handled naturally. Dynamic outlier elimination is achieved without recourse to heuristic segmentation strategies. Lastly, the algorithm can employ different kinds of tracked features, e.g. edges and corners, in the same framework. The actual structure from motion recovered is affine, which assumes limited depth variation within the field of view, but the recovery is based on a more general recursive estimation algorithm, known as the variable state dimension filter (VSDF), which we devised and applied earlier to active camera calibration. Results are presented for real image sequences, and timings for the algorithm demonstrate the feasibility for real-time implementation.

Previous publications have demonstrated, inter alia, the generation of saccades where the gaze direction of the system is rapidly redirected to a target of interest, followed by periods of smooth pursuit where fixation is maintained upon a target [6], [3]. The gaze controller, implemented as a finite state machine, selects and reselects these behaviours to provide robust performance in natural dynamic scenes over extended periods. One of the housekeeping operations of the servo-controller is to propagate odometry — in particular the axis positions or gaze angles  ¢ ¡ and  ¤ £ — around the system. For an extended pursuit sequence, the sequence ¥ ¥¨§�©� � ¥�§�©� © of gaze angles for the platform’s elevation axis and one vergence axis defines an observer-based trajectory for the target. Our first aim here is to recover an observer trajectory as the camera pursues an object. Various algorithms have been implemented to drive pursuit behaviour using active cameras, such as correlation [20], [21], deformable templates [22], feature-based affine transfer [4], [5] and segmented optical flow [6]. The last is used here, though any of the variety of methods might be employed to equal effect. Optical flow is recovered in a small central, or foveal, area of the image. Because of its small

. We investigate the general problem of accurate metrology from uncalibrated video sequences where only partial information is available. We show, via a specific example -- plotting the position of a goal-bound soccer ball -- that accurate measurements can be obtained, and that both qualitative and quantitative questions about the data can be answered. From two video sequences of an incident captured from different viewpoints, we compute a novel (overhead) view using pairs of corresponding images. Using projective constructs we determine the point at which the vertical line through the ball pierces the ground plane in each frame. Throughout we take care to consider possible sources of error and show how these may be eliminated, neglected, or we derive appropriate uncertainty measures which are propagated via a first-order analysis. 1 Introduction The 1966 World Cup Final at Wembley Stadium, between England and West Germany, produced what is arguably the best known and most controversi...

This paper demonstrates a method of using nonmetric visual information derived from an uncalibrated active vision system to navigate an autonomous vehicle through free-space regions detected in a cluttered environment. The structure of 3-space is recovered modulo an affine transformation using an uncalibrated active stereo head carried by the vehicle. The plane at infinity, necessary for recovering affine structure from projective structure, is found in a novel manner by making controlled rotations of the head. The structure is composed of 3D points obtained by detecting and matching image corners through the stereo image sequence. Considerable care has been taken to ensure that the processing is reliable, robust and automatic. Driveable regions are determined from the projection of the affine structure onto a plane parallel to the ground determined using projective constructs. Two methods of negotiating the regions are explored. The first introduces metric information to allow contro...

This paper describes an algorithm for stereo tracking using 3D affine transfer of a body-centred fixation point via the recovery of affine structure. Transfer of fixation and structure is founded on point positions in the image recovered using a corner detector, and matched over time and in stereo. The paper shows how affine transfer can be based on all the available data, not just a minimal basis set, providing further immunity to measurement noise and poor choices of basis corners. Robustness has been built in to the algorithm by enabling transfer to continue by 2D planar transfer and independent transfer if fewer spatio-temporal matches are available. The paper also shows that affine transfer can proceed without explicit calculation of the structure, allowing a fixed-latency tracking feedback loop. The method is implemented at video rates in the visual feedback loop of a four axis active camera platform, and experimental results are shown from stereo tracking of moving objects, with...

Zoom interacts strongly with both vision and control processes in an active visual system, causing problems for many commonly used tracking methods. This paper demonstrates the use of affine transfer to track while zooming, using clusters of corner features. Affine transfer not only is fundamentally invariant to zoom but also provides a natural mechanism to allow features to appear and disappear while tracking, events which will occur as detail sharpens and dissolves during zooming. The paper demonstrates offline 3D affine transfer during zoom for objects undergoing substantial rotation, and describes real-time experiments using 2D affine transfer while zooming and tracking using an active camera platform. 1

This thesis presents a real-time tracking and saccade system that is designed to be the basis for a more complete vision system for a humanoid robot. It consists of several semi-independent functional units including a tracking system, a saccade system, and a calibration system. These three units are designed to be simple and run at video rate on a single digital signal processor chip. Each system recognizes algorithmic failures in the other systems and corrects for these failures. Thesis Supervisor: Lynn Andrea Stein Title: Class of 1957 Associate Professor of Computer Science Acknowledgments This thesis would not have been possible without the support of several people. In particular, Chris Barnhart made the whole thing realizable when he let me borrow his prototype C31 board, and then didn't shout bloody murder when I completely walked off with it. It's quite possibly the most stable piece of hardware I've ever seen. A similar thank you goes to Matt Scheuring for building Odie ju...