In this paper we show that the optical flow, a 2-D field that can be associated with the variation of the image brightness pattern, and the 2-D motion field, the projection on the image plane of the 3-D velocity field of a moving scene, are in general different, unless very special conditions are satisfied. The optical flow, therefore, is ill-suited for computing structure from motion and for reconstructing the 3-D velocity field, problems that require an accurate estimate of the 2-D motion field. We then suggest a different use of the optical flow. We argue that stable qualitative properties of the 2-D motion field give useful information about the 3-D velocity field and the 3-D structure of the scene, and that they can be usually obtained from the optical flow. To support this approach we show how the (smoothed) optical flow and 2-D motion field, interpreted as vector fields tangent to flows of planar dynamical systems, may have the same qualitative properties from the point of view of the theory of structural stability of dynamical systems. () Massachusetts Institute of Technology 1986 This report describes research done within the Artificial Intelligence Laboratory. Support for the A.I. Laboratory's artificial intelligence research is provided in part by the Advanced Research Projects Agency of the Depart- ment of Defense under Oilice of Naval Research contract N00014-S5-K-0124. Support for this research is also provided by a grant from the Oilice of Naval Research, Engineering Psychology Division and by gift of the Artificial Intelligence Center of Hughes Aircraft Corporation to T. Poggio.

In the past decade, a large number of robots has been built that explicitly implement biological navigation behaviours. We review these biomimetic approaches using a framework that allows for a common description of biological and technical navigation behaviour. The review shows that biomimetic systems make significant contributions to two fields of research: First, they provide a real world test of models of biological navigation behaviour; second, they make new navigation mechanisms available for technical applications, most notably in the field of indoor robot navigation. While simpler insect navigation behaviours have been implemented quite successfully, the more complicated way-finding capabilities of vertebrates still pose a challenge to current systems. ©2000 Elsevier Science B.V. All rights reserved.

: We discuss some of the fundamental issues in the design of highly-parallel, dense, low-power motion sensors in analog VLSI. Since photoreceptor circuits are an integral part of all visual motion sensors, we discuss how the sizing of photosensitive areas can affect the performance of such systems. We review the classic gradient and correlation algorithms and give a survey of analog motion-sensing architectures inspired by them. We calculate how the measurable speed range scales with signal-tonoise ratio for a classic Reichardt sensor with a fixed time constant. We show how this speed range may be improved using a nonlinear filter with an adaptive time constant, constructed out of a diode and a capacitor, and present data from a velocity sensor based on such a filter. Finally, we describe how arrays of such velocity sensors can be employed to compute the heading direction of a moving subject and to estimate the time-to-contact between the sensor and a moving object. Keywords: motion se...

\Gamma\GammaAn integrated circuit with on-chip photoreceptors is described, that computes the bi-directional velocity of a visual stimulus moving along a given axis in the focal plane by measuring the time delay of its detection at two positions. Due to the compactness of the circuit, a dense array of such motion-sensing elements can be monolithically integrated to estimate the velocity field of an image and to extract higher-level image features through local or global interaction. Index Terms\Gamma\Gammamotion estimation, velocity sensor, optical flow, analog VLSI, robot vision. Compact Integrated Motion Sensor with Three-Pixel Interaction 3 I. Introduction A variety of image-processing tasks, such as segmentation and estimation of depth, can be considerably simplified in dynamic scenes if motion data is available. Furthermore, mobile systems rely on motion information for the computation of important parameters of ego-motion, such as time to contact and focus of expansion. Since...

Lu and Sperling [Vision Res. 35, 2697 (1995)] proposed that human visual motion perception is served by three separate motion systems: a first-order system that responds to moving luminance patterns, a second-order system that responds to moving modulations of feature types—stimuli in which the expected luminance is the same everywhere but an area of higher contrast or of flicker moves, and a third-order system that computes the motion of marked locations in a ‘‘salience map,’ ’ that is, a neural representation of visual space in which the locations of important visual features (‘‘figure’’) are marked and ‘‘ground’ ’ is unmarked. Subsequently, there have been some strongly confirmatory reports: different gain-control mechanisms for first- and second-order motion, selective impairment of first- versus second- and/or third-order motion by different brain injuries, and the classification of new third-order motions, e.g., isoluminant chromatic motion. Various procedures have successfully discriminated between second- and third-order motion (when first-order motion is excluded): dual tasks, second-order reversed phi, motion competition, and selective adaptation. Meanwhile, eight apparent contradictions to the three-systems theory have been proposed. A review and reanalysis here of the new evidence, pro and con, resolves the challenges and yields a more clearly defined and significantly strengthened theory. © 2001 Optical Society of America OCIS codes: 330.4150.

The real time computation of motion from real images using a single chip with integrated sensors is a hard problem. We present two analog VLSI schemes that use pulse domain neuromorphic circuits to compute motion. Pulses of variable width, rather than graded potentials, represent a natural medium for evaluating temporal relationships. Both algorithms measure speed by timing a moving edge in the image. Our first model is inspired by Reichardt's algorithm in the fly and yields a non-monotonic response vs. velocity curve. We present data from a chip that implements this model. Our second algorithm yields a monotonic response vs. velocity curve and is currently being translated into silicon. 1 Introduction Analog VLSI chips for the real time computation of visual motion have been the focus of much active research because of their importance as sensors for robotic applications. Correlation schemes such as those described in (Delbruck, 1993) have been found to be more robust than gradient s...

Although artificial and biological systems face similar sensorimotor control problems, until today only a few attempts have been made to implement specific biological control structures on robots. Nevertheless, the process of designing the sensorimotor control of a robot can contribute to our understanding of these mechanisms and can provide the basis of a critical evaluation of existing biological models. Flies have developed a specialized visuomotor control for tasks such as course stabilization, fixation and approach towards stationary objects, tracking of moving objects and landing, which are based on the analysis of visual motion information. Theoretical and experimental results suggest that in flies the visuomotor control for course stabilization as well as fixation and approach towards stationary objects may be implemented at least partially by one common sensory circuit. We present agents with a visuomotor controller that regulates the two behaviors of course stabilization and object fixation. To test this controller under real world conditions, we implemented it on a miniature robot. We have been able to show that in addition to course stabilization and object fixation, the robot also approaches stationary

The mechanisms by which animals manage sensorimotor integration and coordination of different behaviors can be investigated in robot models. In previous work the first author has built a robot that localizes sound based on close modeling of the auditory and neural system in the cricket. It is known that the cricket combines its response to sound with other sensorimotor activities such as an optomotor reflex and reactions to mechanical stimulation of the antennae and cerci. Behavioral evidence suggests some ways these behaviors may be integrated. We have tested the addition of an optomotor response, using an analog VLSI circuit developed by the second author, to the sound localizing behavior and have shown that it can, as in the cricket, improve the directness of the robot's path to sound. In particular it substantially improves behavior when the robot is subject to a motor disturbance. Our aim is to better understand how the insect brain functions in controlling complex combinations of behavior, with the hope that this will also suggest novel mechanisms for sensory integration on robots.

Abstract. Sensing visual motion gives a creature valuable information about its interactions with the environment. Flies in particular use visual motion information to navigate through turbulent air, avoid obstacles, and land safely. Mobile robots are ideal candidates for using this sensory modality to enhance their performance, but so far have been limited by the computational expense of processing video. Also, the complex structure of natural visual scenes poses an algorithmic challenge for extracting useful information in a robust manner. We address both issues by creating a small, low-power visual sensor with integrated analog parallel processing to extract motion in real-time. Because our architecture is based on biological motion detectors, we gain the advantages of this highly evolved system: A design that robustly and continuously extracts relevant information from its visual environment. We show that this sensor is suitable for use in the real world, and demonstrate its ability to compensate for an imperfect motor system in the control of an autonomous robot. The sensor attenuates open-loop rotation by a factor of 31 with less than 1 mW power dissipation.

Arthropods exhibit highly efficient solutions to sensorimotor navigation problems. They thus provide a source of inspiration and ideas to robotics researchers. At the same time, attempting to re-engineer these mechanisms in robot hardware and software provides useful insights into how the natural systems might work. This paper reviews three examples of arthropod sensorimotor control systems that have been implemented and tested on robots. First we discuss visual control mechanisms of flies, such as the optomotor reflex and collision avoidance, that have been replicated in analog VLSI (very large scale integration) hardware and used to produce corrective behavior in robot vehicles. Then, we present a robot model of auditory localization in the cricket; and discuss integration of this behavior with the optomotor behavior previously described. Finally we present a model of olfactory search in the moth, which makes use of several sensory cues, and has also been tested using robot hardware. We discuss some of the similarities and differences of the solutions obtained.