this paper, we propose a Bayesian theory of hierarchical cortical computation based both on (a) the mathematical and computational ideas of computer vision and pattern the- ory and on (b) recent neurophysiological experimental evidence. We ,2 have proposed that Grenander's pattern theory 3 could potentially model the brain as a generafive model in such a way that feedback serves to disambiguate and 'explain away' the earlier representa- tion. The Helmholtz machine 4, 5 was an excellent step towards approximating this proposal, with feedback implementing priors. Its development, however, was rather limited, dealing only with binary images. Moreover, its feedback mechanisms were engaged only during the learning of the feedforward connections but not during perceptual inference, though the Gibbs sampling process for inference can potentially be interpreted as top-down feedback disambiguating low level representations? Rao and Ballard's predictive coding/Kalman filter model 6 did integrate generafive feedback in the perceptual inference process, but it was primarily a linear model and thus severely limited in practical utility. The data-driven Markov Chain Monte Carlo approach of Zhu and colleagues 7, 8 might be the most successful recent application of this proposal in solving real and difficult computer vision problems using generafive models, though its connection to the visual cortex has not been explored. Here, we bring in a powerful and widely applicable paradigm from artificial intelligence and computer vision to propose some new ideas about the algorithms of visual cortical process- ing and the nature of representations in the visual cortex. We will review some of our and others' neurophysiological experimental data to lend support to these ideas

Abstract not found

This thesis describes an effort to construct a scene understanding system that is able to analyze the content of real images. While constructing the system we had to provide solutions to many of the fundamental questions that every student of object recognition deals with daily. These include the choice of data set, the choice of success measurement, the representation of the image content, the selection of inference engine, and the representation of the relations between objects. The main test-bed for our system is the CBCL StreetScenes data base. It is a carefully labeled set of images, much larger than any similar data set available at the time it was collected. Each image in this data set was labeled for 9 common classes such as cars, pedestrians, roads and trees. Our system represents each image using a set of features that are based on a model of the human visual system constructed in our lab. We demonstrate that this biologically motivated image representation, along with its extensions, constitutes an effective representation for object detection, facilitating unprecedented levels of detection

Recent experimental work has shown that the primate visual system can analyze complex natural scenes in only 100-150 ms. Such data, when combined with anatomical and physiological knowledge, seriously constrains current models of visual processing. In particular, it suggests that a lot of processing can be achieved using a single feed-forward pass through the visual system, and that each processing layer probably has no more than around 10 ms before the next stage has to respond. In this time, few neurons will have generated more than one spike, ruling out most conventional rate coding models. We have been exploring the possibility of using the fact that strongly activated neurons tend to fire early and that information can be encoded in the order in which a population of cells fire. These ideas have been tested using SpikeNet, a computer program that simulates the activity of very large networks of asynchronously firing neurons. The results have been extremely promising, and we have been able to develop artificial visual systems capable of processing complex natural scenes in real time using standard computer hardware (see

You might find this additional information useful... This article cites 103 articles, 33 of which you can access free at:

Contents . . . . . . What and where visual streams: from [2] E.g. motion processing: from [7] from [4] Fast brain: how fast ? backward connection may be faster than forward from [2] Following [6], let us review that the visual cortices can be considered as a hierarchy of cortical levels with reciprocal extrinsic corticocortical connections among the constituent cortical areas [5]. The notion of a hierarchy depends upon a distinction between forward and backward extrinsic connections. This distinction rests upown di#erent laminar specificity [9, 10] Forwards connections Backwards connections Sparse axonal bifurcations Abundant axonal bifurcation Topographically organized Di#use topography Originate in supragranular layers 2/3 Originate in bilaminar/infragranular layers 5/6 Terminate largely in layer 4 Terminate predominantly in layer 1, but all layers except 4 Postsynaptic e#ects through fast Modulatory a#erents activate slow AMPA (1.3-2.4 ms decay) and GABAA (6 ms decay) recept

■ Research on the spatio-temporal dynamics of visual object recognition suggests a recurrent, interactive model whereby an initial feedforward sweep through the ventral stream to prefrontal cortex is followed by recurrent interactions. However, critical questions remain regarding the factors that mediate the degree of recurrent interactions necessary for meaningful object recognition. The novel prediction we test here is that recurrent interactivity is driven by increasing semantic integration demands as defined by the complexity of semantic information required by the task and driven by the stimuli. To test this prediction, we recorded magnetoencephalography data while participants named living and nonliving objects during two naming tasks. We found that the spatio-temporal dynamics of neural activity were modulated by the level of semantic integration required. Specifically, source reconstructed time courses and phase synchronization measures showed increased recurrent interactions as a function of semantic integration demands. These findings demonstrate that the cortical dynamics of object processing are modulated by the complexity of semantic information required from the visual input. ■

Multivariate information-theoretic measures reveal directed information structure and task relevant changes in fMRI connectivity

Abstract not found

Aneuralnetworkmodelforobjectrecognitionin cluttered scenes using motion and binocular