this paper, we propose a Bayesian theory of hierarchical cortical computation based both on (a) the mathematical and computational ideas of computer vision and pattern the- ory and on (b) recent neurophysiological experimental evidence. We ,2 have proposed that Grenander's pattern theory 3 could potentially model the brain as a generafive model in such a way that feedback serves to disambiguate and 'explain away' the earlier representa- tion. The Helmholtz machine 4, 5 was an excellent step towards approximating this proposal, with feedback implementing priors. Its development, however, was rather limited, dealing only with binary images. Moreover, its feedback mechanisms were engaged only during the learning of the feedforward connections but not during perceptual inference, though the Gibbs sampling process for inference can potentially be interpreted as top-down feedback disambiguating low level representations? Rao and Ballard's predictive coding/Kalman filter model 6 did integrate generafive feedback in the perceptual inference process, but it was primarily a linear model and thus severely limited in practical utility. The data-driven Markov Chain Monte Carlo approach of Zhu and colleagues 7, 8 might be the most successful recent application of this proposal in solving real and difficult computer vision problems using generafive models, though its connection to the visual cortex has not been explored. Here, we bring in a powerful and widely applicable paradigm from artificial intelligence and computer vision to propose some new ideas about the algorithms of visual cortical process- ing and the nature of representations in the visual cortex. We will review some of our and others' neurophysiological experimental data to lend support to these ideas

this paper, we describe a hierarchical network model of visual recognition that explains these experimental observations by using a form of the extended Kalman filter as given by the Minimum Description Length (MDL) principle. The model dynamically combines input-driven bottom-up signals with expectation-driven top-down signals to predict current recognition state. Synaptic weights in the model are adapted in a Hebbian manner according to a learning rule also derived from the MDL principle. The resulting prediction/learning scheme can be viewed as implementing a form of the Expectation-Maximization (EM) algorithm. The architecture of the model posits an active computational role for the reciprocal connections between adjoining visual cortical areas in determining neural response properties. In particular, the model demonstrates the possible role of feedback from higher cortical areas in mediating neurophysiological effects due to stimuli from beyond the classical receptive field. Si

A neural network model is developed to explain how visual thalamocortical interactions give rise to boundary percepts such as illusory contours and surface percepts such as filled-in brightnesses. Top-down feedback interactions are needed in addition to bottom-up feed-forward interactions to simulate these data. One feedback loop is modeled between lateral geniculate nucleus (LGN) and cortical area VI, and another within cortical areas VI and V2. The first feedback loop realizes a matching process which enhances LGN cell activities that are consistent with those of active cortical cells, and suppresses LGN activities that are not. This corticogeniculate feedback, being endstopped and oriented, also enhances LGN ON cell activations at the ends of thin dark lines, thereby leading to enhanced cortical brightness percepts when the lines group into closed illusory contours. The second feedback loop generates boundary representations, including illusory contours, that coherently bind distributed cortical features together. Brightness percepts form within the surface representations through a diffusive filling-in process that is contained by resistive gating signals from the boundary representations. The model is used to simulate illusory contours and surface brightnesses induced by Ehrenstein disks, Kanizsa squares, Glass patterns, and cafe wall patterns in single contrast, reverse contrast, and mixed contrast configurations. These examples illustrate how boundary

In the classical feed-forward, modular view of visual processing, the primary visual cortex (area V1) is a module that serves to extract local features such as edges and bars. Representation and recognition of objects are thought to be functions of higher extrastriate cortical areas. This paper presents neurophysiological data that show the later part of V1 neurons' responses reflecting higher order perceptual computations related to Ullman's (Cognition 1984;18:97 -- 159) visual routines and Marr's (Vision NJ: Freeman 1982) full primal sketch, 2 1 2 D sketch and 3D model. Based on theoretical reasoning and the experimental evidence, we propose a possible reinterpretation of the functional role of V1. In this framework, because of V1 neurons' precise encoding of orientation and spatial information, higher level perceptual computations and representations that involve high resolution details, fine geometry and spatial precision would necessarily involve V1 and be reflected in the later...

We have studied the mechanism of contour perception by recording from neurons in the visual cortex of alert rhesus monkeys. In order to assess the relationship between neural signals and perception, we compared the responses to edges and lines with the responses to patterns in which human observers perceive a contour where no line or edge is given (anomalous contour), such as the border between gratings of thin lines offset by half a cycle. With only one exception out of 60, orientation-selective neurons in area Vl did not signal the anomalous contour. Many neurons failed to re-spond to this stimulus at all, others responded according to the orientation of the grating lines. In area V2, 45 of 103 neurons (44%) signaled the orientation of the anomalous contour. Sixteen did so without signaling the orientation of the inducing lines. Some responded better to anomalous

... This article models how these interactions help visual cortex to realize: (1) the binding process whereby cortex groups distributed data into coherent object representations; (2) the attentional process whereby cortex selectively processes important events; and (3) the developmental and learning processes whereby cortex shapes its circuits to match environmental constraints. New computational ideas about feedback systems suggest how neocortex develops and learns in a stable way, and why top-down attention requires converging bottom-up inputs to fully activate cortical cells, whereas perceptual groupings do not.

Recent neurophysiological studies have shown that primary visual cortex, or V1, does more than passively process image features using the feedforward filters suggested by Hubel and Wiesel. It also uses horizontal interactions to group features preattentively into object representations, and feedback interactions to selectively attend to these groupings. All neocortical areas, including V1, are organized into layered circuits. We present a neural model showing how the layered circuits in areas V1 and V2 enable feedforward, horizontal, and feedback interactions to complete perceptual groupings over positions that do not receive contrastive visual inputs, even while attention can only modulate or prime positions that do not receive such inputs. Recent neurophysiological data about how grouping and attention occur and interact in V1 are simulated and explained, and testable predictions are made. These simulations show how attention can selectively propagate along an object grouping and protect it from competitive masking, and how contextual stimuli can enhance or suppress groupings in a contrast-sensitive manner.

Spatial attention: Visual selection and deployment over space The attentional spotlight and spatial cueing Attentional shifts, splits, and resolution Object-based Selection The visual search paradigm Top-down and bottom-up control of attention Inhibitory mechanisms of attention Invalid cueing Negative priming Inhibition of return Temporal attention: Visual selection and deployment over time Single target search Attentional blink and attentional dwell time Repetition blindness NEURAL MECHANISMS OF SELECTION Single-cell physiological method Event-related potentials Functional imaging: PET and fMRI

How does the visual system generate percepts of moving forms? How does this happen when the forms are emergent percepts, such as illusory contours or segregated textures, and the motion percept is apparent motion between the emergent forms? We develop a neural model of form-motion interactions to explain and simulate parametric properties of psychophysical motion data and to make predictions about how the parallel cortical processing streams V1 ! MT and V1 ! V2 ! MT control form-motion interactions. The model explains how an illusory contour can move in apparent motion to another illusory contour or to a luminance-derived contour; how illusory contour persistence relates to the upper ISI threshold for apparent motion; and how upper and lower ISI thresholds for seeing apparent motion between two flashes decrease with stimulus duration and narrow with spatial separation (Korte's laws). The model accounts for these data by suggesting how the persistence of a boundary segmentation in the V...

This article develops the FACADE theory of 3-dimensional (3-D) vision and figure-ground separation to explain data concerning how 2-dimensional pictures give rise to 3-D percepts of occluding and occluded objects. The model describes how geometrical and contrastive properties of a picture can either cooperate or compete when fonning the boundaries and surface representations that subserve conscious percepts. Spatially long-range cooperation and spatially short-range competition work together to separate the boundaries of occluding figures from their occluded neighbors. This boundary ownership process is sensitive to image T junctions at which occluded figures contact occluding figures. These boundaries control the filling-in of color within multiple depth-sensitive surface representations. Feedback between surface and boundary representations strengthens consistent boundaries while inhibiting inconsistent ones. Both the boundary and the surface representations of occluded objects may be amodally completed, while the surface representations of unoccluded objects become visible through modal completion. Functional roles for conscious modal and amodal representations in object recognition, spatial attention, and reaching behaviors are discussed. Model interactions are interpreted in tenns of visual, temporal, and parietal cortices. The human urge to represent the three-dimensional (3-D)