this paper, we propose a Bayesian theory of hierarchical cortical computation based both on (a) the mathematical and computational ideas of computer vision and pattern the- ory and on (b) recent neurophysiological experimental evidence. We ,2 have proposed that Grenander's pattern theory 3 could potentially model the brain as a generafive model in such a way that feedback serves to disambiguate and 'explain away' the earlier representa- tion. The Helmholtz machine 4, 5 was an excellent step towards approximating this proposal, with feedback implementing priors. Its development, however, was rather limited, dealing only with binary images. Moreover, its feedback mechanisms were engaged only during the learning of the feedforward connections but not during perceptual inference, though the Gibbs sampling process for inference can potentially be interpreted as top-down feedback disambiguating low level representations? Rao and Ballard's predictive coding/Kalman filter model 6 did integrate generafive feedback in the perceptual inference process, but it was primarily a linear model and thus severely limited in practical utility. The data-driven Markov Chain Monte Carlo approach of Zhu and colleagues 7, 8 might be the most successful recent application of this proposal in solving real and difficult computer vision problems using generafive models, though its connection to the visual cortex has not been explored. Here, we bring in a powerful and widely applicable paradigm from artificial intelligence and computer vision to propose some new ideas about the algorithms of visual cortical process- ing and the nature of representations in the visual cortex. We will review some of our and others' neurophysiological experimental data to lend support to these ideas

this paper, we describe a hierarchical network model of visual recognition that explains these experimental observations by using a form of the extended Kalman filter as given by the Minimum Description Length (MDL) principle. The model dynamically combines input-driven bottom-up signals with expectation-driven top-down signals to predict current recognition state. Synaptic weights in the model are adapted in a Hebbian manner according to a learning rule also derived from the MDL principle. The resulting prediction/learning scheme can be viewed as implementing a form of the Expectation-Maximization (EM) algorithm. The architecture of the model posits an active computational role for the reciprocal connections between adjoining visual cortical areas in determining neural response properties. In particular, the model demonstrates the possible role of feedback from higher cortical areas in mediating neurophysiological effects due to stimuli from beyond the classical receptive field. Si

& What happens in the brain when you conjure up a mental image in your mind’s eye? We tested whether the particular regions of extrastriate cortex activated during mental imagery depend on the content of the image. Using functional magnetic resonance imaging (fMRI), we demonstrated selective activation within a region of cortex specialized for face perception during mental imagery of faces, and selective activation within a place-selective cortical region during imagery of places. In a further study, we compared the activation for imagery and perception in these regions, and found greater response magnitudes for perception than for imagery of the same items. Finally, we found that it is possible to determine the content of single cognitive events from an inspection of the fMRI data from individual imagery trials. These findings strengthen evidence that imagery and perception share common processing mechanisms, and demonstrate that the specific brain regions activated during mental imagery depend on the content of the visual image. &

We investigated grapheme--colour synaesthesia and found that: (1) The induced colours led to perceptual grouping and pop-out, (2) a grapheme rendered invisible through `crowding' or lateral masking induced synaesthetic colours --- a form of blindsight --- and (3) peripherally presented graphemes did not induce colours even when they were clearly visible. Taken collectively, these and other experiments prove conclusively that synaesthesia is a genuine perceptual phenomenon, not an effect based on memory associations from childhood or on vague metaphorical speech. We identify different subtypes of number--colour synaesthesia and propose that they are caused by hyperconnectivity between colour and number areas at different stages in processing; lower synaesthetes may have cross-wiring (or cross-activation) within the fusiform gyrus, whereas higher synaesthetes may have cross-activation in the angular gyrus. This hyperconnectivity might be caused by a genetic mutation that causes defective pruning of connections between brain maps. The mutation may further be expressed selectively (due to transcription factors) in the fusiform or angular gyri, and this may explain the existence of different forms of synaesthesia. If expressed very diffusely, there may be extensive cross-wiring between brain regions that represent abstract concepts, which would explain the link between creativity, metaphor and synaesthesia (and the higher incidence of synaesthesia among artists and poets). Also, hyperconnectivity between the sensory cortex and amygdala would explain the heightened aversion synaesthetes experience when seeing numbers printed in the `wrong' colour. Lastly, kindling (induced hyperconnectivity in the temporal lobes of temporal lobe epilepsy [TLE] patients) may explain the purp...

A recent eyetracking experiment has indicated that, while staring at a blank white display, participants engaged in imagery tend to make eye movements that mimic the directionality of spatial expressions in the speech stream (Spivey & Geng, 2000). This result is consistent with a spatial mental models account of language comprehension (e.g., Johnson-Laird, 1983), adds a motor component to evidence for activation of perceptual mechanisms during visual imagery (e.g., Kosslyn, Thompson, Kim, & Alpert, 1995), and fits with claims regarding the embodiment of cognition (e.g., Varela, Thompson, & Rosch, 1991). However, some methodological concerns remain. We report some preliminary observations, and a controlled experiment, in which these methodological concerns are resolved. We demonstrate that, even when the speech includes no instructions to imagine anything, and even when participants ’ eyes are closed, participants tend to make eye movements in the same direction (and especially along the same axis) as the described scene when listening to a spatially extended scene description..

Below is the unedited, uncorrected final draft of a BBS target article that has been accepted for publication. This preprint has been prepared for potential commentators who wish to nominate themselves for formal commentary invitation. Please DO NOT write a commentary until you receive a formal invitation. If you are invited to submit a commentary, a copyedited, corrected version of this paper will be posted.

Can theories of mental imagery, conscious mental contents, developed within cognitive science throw light on the obscure (but culturally very significant) concept of imagination? Three extant views of mental imagery are considered: quasi-pictorial, description, and perceptual activity theories. The first two face serious theoretical and empirical difficulties. The third is (for historically contingent reasons) little known, theoretically underdeveloped, and empirically untried, but has real explanatory potential. It rejects the “traditional ” symbolic computational view of mental contents, but is compatible with recent situated cognition and active vision approaches in robotics. This theory is developed and elucidated. Three related key aspects of imagination (non-discursiveness, creativity, and seeing as) raise difficulties for the other theories. Perceptual activity theory presents imagery as non-discursive and relates it closely to seeing as. It is thus well placed to be the basis for a general theory of imagination and its role in creative thought.

& The goal of this study was to investigate the neurocognitive processes of mental imagery in deductive reasoning. Behavioral studies yielded four sorts of verbal relations: (1) visuospatial relations that are easy to envisage both visually and spatially; (2) visual relations that are easy to envisage visually but hard to envisage spatially; (3) spatial relations that are hard to envisage visually but easy to envisage spatially; and (4) control relations that are hard to envisage both visually and spatially. In three experiments, visual relations slowed the process of reasoning in comparison with control relations, whereas visuospatial and spatial relations yielded inferences comparable to those of control relations. An experiment using functional magnetic resonance imaging showed that in the absence of any correlated visual input (problems were presented acoustically via headphones), all types of reasoning problems evoked activity in the left middle temporal gyrus, in the right superior parietal cortex, and bilaterally in the precuneus. In the prefrontal cortex, increased activity was found in the middle and inferior frontal gyri. However, only the problems based on visual relations also activated areas of the visual association cortex corresponding to V2. The results indicate that cortical activity during reasoning depends on the nature of verbal relations. All relations elicit mental models that underlie reasoning, but visual relations in addition elicit visual images. This account resolves inconsistencies in the previous literature. &... it is clear that imagery plays a key role in reasoning. (Kosslyn,1994, p.404)

ABSTRACT. Drawings are an integral part of the dialogue a designer conducts with him or herself during design. They are a kind of external representation, a cognitive tool developed to facilitate information processing. Drawings differ from images in that they reflect conceptualizations, not perceptions, of reality. For a particular domain, sketches use a small set of segments or elements that map the critical elements of the domain. It is proposed that the choice of and representation of elements and the order in which they are drawn reflect the way that domain is schematized and conceptualized. Support for this claim is gathered from research on sketch maps, graphs, and geometric analogy solution. Design without drawing seems inconceivable. One view of the role of sketching in design is an iterative, cyclical, dialectic view where sketches serve to instantiate design ideas as well as to stimulate new ones. In each cycle, designers express their ideas externally, on paper, and then examine, interpret, and perhaps reinterpret them. This inspection of the drawings may inspire changes in design ideas, which are put down again on paper to be reexamined again, reconceived, redrawn, reexamined, and so on (for expansion of these ideas, see Goldschmidt, 1989, 1991, 1994;

The interaction between the vision of colors and odor determination is investigated through lexical analysis of experts ’ wine tasting comments. The analysis shows that the odors of a wine are, for the most part, represented by objects that have the color of the wine. The assumption of the existence of a perceptual illusion between odor and color is confirmed by a psychophysical experiment. A white wine artificially colored red with an odorless dye was olfactory described as a red wine by a panel of 54 tasters. Hence, because of the visual information, the tasters discounted the olfactory information. Together with recent psychophysical and neuroimaging data, our results suggest that the above perceptual illusion occurs during the verbalization phase of odor determination. © 2001 Academic Press Key Words: odor identification; color; perceptual illusion; lexical analysis; psychophysic; wine.