this paper, we propose a Bayesian theory of hierarchical cortical computation based both on (a) the mathematical and computational ideas of computer vision and pattern the- ory and on (b) recent neurophysiological experimental evidence. We ,2 have proposed that Grenander's pattern theory 3 could potentially model the brain as a generafive model in such a way that feedback serves to disambiguate and 'explain away' the earlier representa- tion. The Helmholtz machine 4, 5 was an excellent step towards approximating this proposal, with feedback implementing priors. Its development, however, was rather limited, dealing only with binary images. Moreover, its feedback mechanisms were engaged only during the learning of the feedforward connections but not during perceptual inference, though the Gibbs sampling process for inference can potentially be interpreted as top-down feedback disambiguating low level representations? Rao and Ballard's predictive coding/Kalman filter model 6 did integrate generafive feedback in the perceptual inference process, but it was primarily a linear model and thus severely limited in practical utility. The data-driven Markov Chain Monte Carlo approach of Zhu and colleagues 7, 8 might be the most successful recent application of this proposal in solving real and difficult computer vision problems using generafive models, though its connection to the visual cortex has not been explored. Here, we bring in a powerful and widely applicable paradigm from artificial intelligence and computer vision to propose some new ideas about the algorithms of visual cortical process- ing and the nature of representations in the visual cortex. We will review some of our and others' neurophysiological experimental data to lend support to these ideas

This article concerns the nature of evoked brain responses and the principles underlying their generation. We start with the premise that the sensory brain has evolved to represent or infer the causes of changes in its sensory inputs. The problem of inference is well formulated in statistical terms. The statistical fundaments of inference may therefore afford important constraints on neuronal implementation. By formulating the original ideas of Helmholtz on perception, in terms of modern-day statistical theories, one arrives at a model of perceptual inference and learning that can explain a remarkable range of neurobiological facts. It turns out that the problems of inferring the causes of sensory input (perceptual inference) and learning the relationship between input and cause (perceptual learning) can be resolved using exactly the same principle. Specifically, both inference and learning rest on minimizing the brain’s free energy, as defined in statistical physics. Furthermore, inference and learning can proceed in a biologically plausible fashion. Cortical responses can be seen as the brain’s attempt to minimize the free energy induced by a stimulus and thereby encode the most likely cause of that stimulus. Similarly, learning emerges from changes in synaptic efficacy that minimize the free energy, averaged over all stimuli encountered. The underlying scheme rests on empirical Bayes and hierarchical models

As people examine their world, the proximal stimulus changes position on their retinae with every saccade, but they perceive the world as being stable. This phenomenon of visual stability was explored by making changes in natural, full-color pictures during selected saccades as observers examined them in preparation for a recognition test. In Experiment 1, the pictures were displaced up, down, left, or right by 0.3, 0.6, or 1.2°. In Experiment 2, the pictures were expanded or contracted by 10 % or 20%. As a secondary task, subjects pressed a button when a change was detected. Three results from previous studies with simpler stimuli did not generalize. Evidence suggests that subjects ' detection of image changes primarily involves the use of local information in the region of the eyes ' landing position. A saccade target theory of visual stability is proposed. Making a saccadic eye movement causes a displacement of the light pattern across the retinae. If a similar retinal displacement occurs during an eye fixation, there is perception of movement, that is, the world appears to jump. However, the same pattern of motion on the retinae, occurring as a consequence of making a saccade, is not perceived and the world appears stable. 1 This phenomenon, referred to traditionally as space constancy, and which we will call visual stability, permits people to visually explore the world with a moving sensory matrix without misattributing selfinduced stimulus motion on the matrix to the world itself. How the visual system achieves this stability has been a matter of speculation and research since Helmholtz (18667 1963) discussed the problem.

We propose a representation of images in which a global, but not a local topology is defined. The topology is restricted to resolutions up to the extent of the local region of interest (ROI). Although the ROI's may contain many pixels, there is no spatial order on the pixels within the ROI, the only information preserved is the histogram of pixel values within the ROI's. This can be considered as an extreme case of a textel (texture element) image: The histogram is the limit of texture where the spatial order has been completely disregarded. We argue that locally orderless images are ubiquitous in perception and the visual arts. Formally, the orderless images are most aptly described by three mutually intertwined scale spaces. The scale parameters correspond to the pixellation ("inner scale"), the extent of the ROI's ("outer scale") and the resolution in the histogram ("tonal scale"). We describe how to construct locally orderless images, how to render them, and how to use them in a variety of local and global image processing operations.

Subjects first detected a target embedded in a stream of letters presented at the left of fixation and then, as quickly as possible, shifted their attention to a stream of numerals at the right of fixation. They attempted to report, in order, the four earliest occurring numerals after the target. Numerals appeared at rates of 4.6, 6.9, 9.2, and 13.4/s. Scaling analyses were made of(a) item scores, P~(r), the probability of a numeral from stimulus position i appearing in response position r, r = (1, 2, 3, 4), and (b) order scores, P~nj, the probability that a numeral from stimulus position i appeared earlier in the response than one from stimulus position j. For all subjects, targets, and numeral rates, the relative position of numerals in the response sequence showed clustering, disorder, and folding. Reported numerals tended to cluster around a stimulus position 400 ms after the target. The numerals were reported in an apparently haphazard order--at high numeral rates, inverted iBj pairs were as frequent as correct pairs. The actual order of report resulted from a mixture of correctly ordered numerals with numerals ordered in the direction opposite to their order of presentation (folding around the cluster center). These results are quantitatively described by a strength theory of order (precedence) and are efficiently predicted by a computational attention gating model (AGM). The AGM makes quantitatively correct predictions of over 500 values ofPl(r), P~Bj in 12 conditions with two attention

Image displacement fields---optical flow fields, stereo disparity fields, normal flow fields---due to rigid motion possess a global geometric structure which is independent of the scene in view. Motion vectors of certain lengths and directions are constrained to lie on the imaging surface at particular loci whose location and form depends solely on the 3D motion parameters. If optical flow fields or stereo disparity fields are considered, then equal vectors are shown to lie on conic sections. Similarly, for normal motion fields, equal vectors lie within regions whose boundaries also constitute conics. By studying various properties of these curves and regions and their relationships, a characterization of the structure of rigid motion fields is given. The goal of this paper is to introduce a concept underlying the global structure of image displacement fields. This concept gives rise to various constraints that could form the basis of algorithms for the recovery of visual information f...

Abstract not found

A taxonomy is described that relates different navigational behaviours in a hierarchical and compositional way. Elementary navigation tactics are combined to tactical navigation in routes; landmarks in space are contrasted to routemarks in networks of passages. Survey knowledge comes in at the level of strategic navigation. The Bremen Autonomous Wheelchair is then presented as a vehicle for experimentation in robotics, both to model biologically plausible navigational behaviours and to develop efficient navigational mechanisms for a technical application. The implementation on the autonomous system is based on the use of basic behaviours and the identification of routemarks. The actual recognition of artificial routemarks is described and early results of the current work on the identification of natural 3D-marks are presented. 1

Abstract: Why and how people perceive the visual world as continuous and stable, despite the gross changes of its retinal projection that occur with each saccade, is one of the classic problems in perception. In the present paper, we argue that an important factor of visual stability and transsaccadic perception is formed by the reafferent visual information, i.e., the visual display that is present when the eyes land. After a review of some of the relevant theoretical, behavioural and physiological research on space constancy, saccadic suppression and transsaccadic memory, three experiments are presented. In a first experiment, we study the effect of an extended horizontal bar covering the target area for a short period after the saccade on saccadic suppression of image displacement. The results show that the bar acts just like a temporary blanking of the saccade target, leading to a strong reduction of saccadic suppression. In the second experiment, we show that any object that is present immediately after the saccade can establish a spatial reference, even if it is dissimilar to the saccade target. In a third experiment we study, with a similar approach, the effect of blanking and postsaccadic information on transsaccadic integration of form information. The data demonstrate that a localized postsaccadic object tends to replace the content of transsaccadic memory.

We advance a dominant neural strategy for facilitating conceptual thought. Concepts are groupings of “object ” attributes. Once the brain learns such critical groupings, the “object” attributes are inhibited from conscious awareness. We see the whole, not the parts. The details are inhibited when the concept network is activated, ie. the inhibition is dynamic and can be switched on and off. Autism is suggested to be the state of retarded concept formation. Our model predicts the possibility of accessing nonconscious information by artificially disinhibiting (turning off) the inhibiting networks associated with concept formation, using transcranial magnetic brain stimulation (TMS). For example, this opens the door for the restoration of perfect pitch, for recalling detail, for acquiring accent-free second languages beyond puberty, and even for enhancing creativity. The model further shows how unusual autistic savant skills as well as certain psychopathologies can be due respectively to privileged or inadvertent access to information that is normally inhibited from conscious awareness.