Abstract not found

Human and non-human primates excel at visual recognition tasks. The primate visual system exhibits a strong degree of selectivity while at the same time being robust to changes in the input image. We have developed a quantitative theory to account for the computations performed by the feedforward path in the ventral stream of the primate visual cortex. Here we review recent predictions by a model instantiating the theory about physiological observations in higher visual areas. We also show that the model can perform recognition tasks on datasets of complex natural images at a level comparable to psychophysical measurements on human observers during rapid categorization tasks. In sum, the evidence suggests that the theory may provide a framework to explain the first 100–150 ms of visual object recognition. The model also constitutes a vivid example of how computational models can interact with experimental observations in order to advance our understanding of a complex phenomenon. We conclude by suggesting a number of open questions, predictions, and specific experiments for visual physiology and psychophysics.

Single-unit and multiunit recordings in primates have already established that decision making involves at least two general stages of neural processing: representation of evidence from early sensory areas and accumulation of evidence to a decision threshold from decision-related regions. However, the relay of information from early sensory to decision areas, such that the accumulation process is instigated, is not well understood. Using a cued paradigm and single-trial analysis of electroencephalography (EEG), we previously reported on temporally specific components related to perceptual decision making. Here, we use information derived from our previous EEG recordings to inform the analysis of fMRI data collected for the same behavioral task to ascertain the cortical origins of each of these EEG components. We demonstrate that a cascade of events associated with perceptual decision making takes place in a highly distributed neural network. Of particular importance is an activation in the lateral occipital complex implicating perceptual persistence as a mechanism by which object decision making in the human brain is instigated. Key words: EEG; fMRI; spatiotemporal analysis; perceptual decision making; perceptual persistence; lateral occipital complex

The emotional content of visual images can be parameterized along two dimensions: valence (pleasantness) and arousal (intensity of emotion). In this study we ask how these distinct emotional dimensions affect the short-term memory of human observers viewing a rapid stream of images and trying to remember their content. We show that valence and arousal modulate short-term memory as independent factors. Arousal influences dramatically the average speed of data accumulation in memory: Higher arousal results in faster accumulation. Valence has a more interesting effect: While a picture is being viewed, information from positive and neutral scenes accumulates in memory at a constant rate, whereas information from negative scenes is encoded slowly at first, then increasingly faster. We provide evidence showing that neither differences in low-level image properties nor differences in the ability to apprehend the meaning of images at short exposures can account for the observed results, and propose that the effects are specific to the short-term memory mechanism. We interpret this pattern of results to mean that information accumulation in short-term memory is a controlled process, whose gain is modulated by valence and arousal acting as endogenous attentional cues.

Abstract—Advances in neural signal and image acquisition as well as in multivariate signal processing and machine learning are enabling a richer and more rigorous understanding of the neural basis of human decision-making. Decision-making is essentially characterized behaviorally by the variability of the decision across individual trials—e.g., error and response time distributions. To infer the neural processes that govern decision-making requires identifying neural correlates of such trial-to-trial behavioral variability. In this paper, we review efforts that utilize signal processing and machine learning to enable single-trial analysis of neural signals acquired while subjects perform simple decision-making tasks. Our focus is on neuroimaging data collected noninvasively via electroencephalograpy (EEG) and functional magnetic resonance imaging (fMRI). We review the specific framework for extracting decision-relevant neural components from the neuroimaging data, the goal being to analyze the trial-to-trial variability of the neural signal along these component directions and to relate them to elements of the decision-making process. We review results for perceptual decision-making and discrimination tasks, including paradigms in which EEG variability is used to inform an fMRI analysis. We discuss how single-trial analysis reveals aspects of the underlying decision-making networks that are unobservable using traditional trial-averaging methods. Index Terms—Decision-making, electroencephalography, functional magnetic resonance imaging, machine learning, single-trial analysis. I.

journal homepage: www.elsevier.com/locate/visres Automatic computation of an image’s statistical surprise predicts performance

The present paper argues for the notion that when attention is spread across the visual field the first sweep of information through the brain visual selection is completely stimulus-driven. Only later in time, through recurrent feedback processing, volitional control based on expectancy and goal set will bias visual selection in a top-down manner. Here we review behavioral evidence as well as evidence from ERP, fMRI, TMS and single cell recording consistent with stimulus-driven selection. Alternative viewpoints that assume a large role for top-down processing are discussed. It is argued that in most cases evidence supporting top-down control on visual selection in fact demonstrates top-down control on processes occurring later in time, following initial selection. We conclude that top-down knowledge regarding non-spatial features of the objects cannot alter the initial selection priority. Only by adjusting the size of the attentional window, the initial sweep of information through the brain may be altered in a top-down way. top-down & bottom-up control 3

■ Everyday vision requires robustness to a myriad of environmental factors that degrade stimuli. Foreground clutter can occlude objects of interest, and complex lighting and shadows can decrease the contrast of items. How does the brain recognize visual objects despite these low-quality inputs? On the basis of predictions from a model of object recognition that contains excitatory feedback, we hypothesized that recurrent processing would promote robust recognition when objects were degraded by strengthening bottom–up signals that were weakened because of occlusion and contrast reduction. To test this hypothesis, we used backward masking to interrupt the processing of partially occluded and contrast reduced images during a categorization experiment. As predicted by the model, we found significant interactions between the mask and occlusion and the mask and contrast, such that the recognition of heavily degraded stimuli was differentially impaired by masking. The model provided a close fit of these results in an isomorphic version of the experiment with identical stimuli. The model also provided an intuitive explanation of the interactions between the mask and degradations, indicating that masking interfered specifically with the extensive recurrent processing necessary to amplify and resolve highly degraded inputs, whereas less degraded inputs did not require much amplification and could be rapidly resolved, making them less susceptible to masking. Together, the results of the experiment and the accompanying model simulations illustrate the limits of feedforward vision and suggest that object recognition is better characterized as a highly interactive, dynamic process that depends on the coordination of multiple brain areas. ■

This document is a penultimate draft. The final version was published as a chapter in Machine Learning for