We present an incremental focus of attention (IFA) architecture for adding robustness to software-based, real-time, motion trackers. The framework provides a structure which, when given the entire camera image to search, efficiently focuses the attention of the system into a narrow set of configurations that includes the target configuration. IFA offers a means for automatic tracking initialization and reinitialization when environmental conditions momentarily deteriorate and cause the system to lose track of its target. Systems based on the framework degrade gracefully as various assumptions about the environment are violated. In particular, when constructed with multiple tracking algorithms of varying precision, the failure of a single algorithm causes another, less precise algorithm to take over, thereby allowing the system to return approximate information on feature location or configuration. 1 Introduction Robustness is a hallmark of intelligent behavior and a desirable proper...

We present the Incremental Focus of Attention (IFA) architecture for robust, adaptive, realtime motion tracking. IFA systems combine several visual search and vision-based tracking algorithms into a layered hierarchy. The architecture controls the transitions between layers and executes algorithms appropriate to the visual environment at hand: When conditions are good, tracking is accurate and precise; as conditions deteriorate, more robust, yet less accurate algorithms take over; when tracking is lost altogether, layers cooperate to perform a rapid search for the target in order to recover it and continue tracking. Implemented IFA systems are extremely robust to most common types of temporary visual disturbances. They resist minor visual perturbances and recover quickly after full occlusions, illumination changes, major distractions, and target disappearances. Analysis of the algorithm's recovery times are supported by simulation results and experiments on real data. In particular, ex...

One of the classical topics in neural networks is winner-take-all (WTA), which has been widely used in unsupervised (competitive) learning, cortical processing, and attentional control. Because of global connectivity, WTA networks, however, do not encode spatial relations in the input, and thus cannot support sensory and perceptual processing where spatial relations are important. We propose a new architecture that maintains spatial relations between input features. This selection network builds on LEGION (Locally Excitatory Globally Inhibitory Oscillator Networks) dynamics and slow inhibition. In an input scene with many objects (patterns), the network selects the largest object. This system can be easily adjusted to select several largest objects, which then alternate in time. We further show that a two-stage selection network gains efficiency by combining selection with parallel removal of noisy regions. The network is applied to select the most salient object in real images. As a s...

A novel model of attentive visual motion processing is presented. A new feedforward motion-processing pyramid is described whose motivation lies in the neurobiology of primate motion processes. On this structure the Selective Tuning (ST) model for visual attention is implemented and demonstrated, showing how it can localize and label simple motion patterns. There are three main contributions: 1) we present a new feed-forward motion processing hierarchy, the first to include a multi-level decomposition of processing including local spatial derivatives of velocity as a separate layer; 2) we present examples of how ST can operate on this hierarchy to localize and label motion patterns; and, 3) we present a new solution to aspects of the feature binding problem and show it to be sufficient for the task of grouping motion features into coherent object motion. This feature grouping (or binding) is accomplished using a top-down attentional selection mechanism that does not depend on a single location-based saliency representation.

Selective visual attention to objects and locations depends both on deliberate behavioral goals that regulate even early visual representations (goal-directed influences) and on autonomous neural responses to sensory input (stimulus-driven influences). In this chapter, I argue that deliberate goal-directed attentional strategies are always constrained by involuntary, ”hard-wired computations, and that an appropriate research strategy is to delineate the nature of the interactions imposed by these constraints. To illustrate the inter-action between goal-directed and stimulus-driven attentional control, four domains of visual selection are reviewed. First, selection by location is both spatially and temporally limited, reflecting in part early visual representations of the scene. Second, selection by feature is an available attentional strategy, but it appears to be mediated by location, and feature salience alone does not govern the deployment of attention. Third, early visual seg-mentation processes that parse a scene into perceptual object representations enable object-based selection, but they also enforce selection of entire objects, and not just isolated features. And fourth, the appearance of a new perceptual object captures attention in a stimulus-driven fashion, but even this is subject to some top-down attentional control.

The Incremental Focus of Attention (IFA) framework provides a structure in which different algorithms for visual motion tracking can be assembled into a single system which tracks an object with the best combination of precision, speed, and robustness. IFA puts different tracking algorithms in a hierarchy based on speed and precision and supplies a means for determining which algorithm should perform in different circumstances. Precision is traded for robustness and speed when ideal environmental conditions are not met, and conversely, when conditions are favorable, tracking proceeds at the greatest precision offered by the component trackers. Implementations of IFA systems provide robust visual tracking which returns as much tracking information as possible given the constraints of the environment. Keywords: robust visual motion tracking, flexible computation Introduction The aim of visual motion tracking is to recover the state of a target object as it moves in a sequence of image...

An agent performing a task in an environment must be able to selectively attend to visual stimuli. This ability is of critical importance for adaptive behavior in (vision-based) biological and artificial agents. In this paper we present a connectionist model of how visual attention can serve an agent to perform its task. The model is embedded in a mobile robot. Visual stimuli are segregated by means of synchronization of spiking neurons. They then enter a selection process, the result of which determines what region of the visual field the robot will attend and consequently react to. Results from the behavior of the robot as well as the underlying neuronal dynamics are presented, and limitations as well as future extensions of the model are discussed. I. INTRODUCTION An agent performing a task in an environment must be able to select objects upon which to act. This requires (a) a segregation mechanism which binds image features belonging to objects and separates them from the backgroun...

A model for cortex information processing is described which depends upon a combination of population, rate and temporal coding for action potential spikes. In this model, binding of information derived from one visual attention object occurs because attention causes a slight ( ~ 1 millisecond) shift in each spike in the sensory inputs derived from the object towards the nearest peak to the spike in a 40 Hz modulation frequency. This frequency modulation results in preferential processing of the information derived from the attention object. Simulations of populations of leaky integrator neurons with both excitatory and inhibitory connectivity demonstrate that this preferential processing occurs with physiologically reasonable synaptic integration times, and allows object categorization on time scales consistent with human cognitive processing.

Visual processes do not amount to a simple filtering process performed by a series of hierarchical modules. They allow to select the items immediately useful for the current action from the information included in the external scene. To perform this selection, attentional top-down controls must combine with bottom-up information issued from the retina. In the prospect to understand how these informations are fused together, a computational model of the first steps of the visual process able to account for the pre-attentional and attentional mechanisms involved in visual search has been developed. This model, called Competitive Search, integrates the dynamical aspects of a local dynamical architecture. It accounts for 'pop-out' and attentional phenomena involved in the search for conjunctive targets without introducing ad hoc hypothetical mechanisms such as the attentional spotlight hypothesis. It suggests that such metaphors, issued from the conventional cognitive psychology, may in fa...

n is gazecontingent. To match visual attention, computer systems must adapt dynamically to relay only relevant information at the point of regard. Gaze-contingent systems have been used in human-computer interaction, image/video processing, virtual environments, and telerobotic navigation. Due to the high cost of real-time processing required to match eye movements, these systems are not widespread. A critical obstacle for gaze-contingent systems is the inherent delay of eye tracking and the subsequent display of a foveally projected Region Of Interest (ROI). 1 This problem is alleviated by anticipating and/or cueing visual attention. The latter approach is obtrusive in the sense that the system may divert attention from its natural course. The former strategy is unobtrusive but requires computationally 1 Also referred to as Areas Of Interest (AOIs). expensive analysis of underlying scenery. Since the role of low-level vision