We use a simple modification to a conventional SLR camera to capture images of several hundred scenes in colour (RGB) and near-infrared (NIR). We show that the addition of near-infrared information leads to significantly improved performance in a scene-recognition task, and that the improvements are greater still when an appropriate 4-dimensional colour representation is used. In particular we propose MSIFT – a multispectral SIFT descriptor that, when combined with a kernel based classifier, exceeds the performance of state-of-the-art scene recognition techniques (e.g., GIST) and their multispectral extensions. We extensively test our algorithms using a new dataset of several hundred RGB-NIR scene images, as well as benchmarking against Torralba’s scene categorization dataset. Figure 1: Examples from our database of RGB-NIR images. Notice that the NIR band exhibits noticeable differences at the scene level: sky and water are dark, foliage is bright, and details are more clearly resolvable in haze. 1.

Research, 45(13), 1707–1724) reported, in contradiction to several earlier studies, that photographs of human faces can be searched for efficiently (i.e., ‘‘pop out’’) among photographs of other objects (as long as these objects are not ‘‘too similar’ ’ to faces). An apparent search asymmetry between faces and other categories (houses, cars) pointed to the existence of a specialized ‘‘face map’’. Findings of impaired performance for scrambled images were presented as evidence that this face pop out is a high-level, ‘‘holistic’ ’ effect. While the main pop-out effect cannot be disputed, several choices made in that study in terms of experiment design, analysis and interpretation are questionable. After discussing these issues, I report novel experiments which show that (i) the face pop-out effect can be replicated, but under controlled conditions there is no asymmetry between faces and other objects (cars); (ii) inverting pictures and hence disrupting holistic face processing has only a minor effect on search performance; (iii) finally, search becomes inefficient when Fourier amplitude information (which carries global low-level statistical properties of images) is made irrelevant, and only phase information (carrying contour localization) can be used to detect faces. These results imply, contrary to the target article, that the face pop-out effect is mostly based on low-level factors.

Human subjects are extremely efficient at categorizing natural scenes, despite the fact that different classes of natural scenes often share similar image statistics. Thus far, however, it is unknown where and how complex natural scene categories are encoded and discriminated in the brain. We used functional magnetic resonance imaging (fMRI) and distributed pattern analysis to ask what regions of the brain can differentiate natural scene categories (such as forests vs mountains vs beaches). Using completely different exemplars of six natural scene categories for training and testing ensured that the classification algorithm was learning patterns associated with the category in general and not specific exemplars. We found that area V1, the parahippocampal place area (PPA), retrosplenial cortex (RSC), and lateral occipital complex (LOC) all contain information that distinguishes among natural scene categories. More importantly, correlations with human behavioral experiments suggest that the information present in the PPA, RSC, and LOC is likely to contribute to natural scene categorization by humans. Specifically, error patterns of predictions based on fMRI signals in these areas were significantly correlated with the behavioral errors of the subjects. Furthermore, both behavioral categorization performance and predictions from PPA exhibited a significant decrease in accuracy when scenes were presented up-down inverted. Together these results suggest that a network of regions, including the PPA, RSC, and LOC, contribute to the human ability to categorize natural scenes.

Human superiority over computers in identifying natural objects like clouds, water, grass etc. comes from two capabilities: the capability to maintain a growing knowledge base pertaining to natural object attributes as experienced over a period of time, and the capability to tolerate variations inherently present in the attributes of natural objects. In this paper we aim to mimic both these capabilities for efficient machine identification of local objects in natural scene imagery through a combined application of Case Based Reasoning (CBR) and Fuzzy Logic (FL).Training data of natural objects is used offline to build a case base of various natural object attributes. The imprecision in the definition of natural objects is accommodated by employing fuzzy linguistic variables for object attribute description in the case base as well as in the similarity calculation and case retrieval procedures. When presented with a new image, the reasoning system applies fuzzy similarity concept to retrieve approximately similar cases from the case base and output the corresponding object label. Experimental results endorse validity of proposed approach.