We show how high-level scene properties can be inferred from classification of low-level image features, specifically for the indoor-outdoor scene retrieval problem. We systematically studied the features: (1) histograms in the Ohta color space (2) multiresolution, simultaneous autoregressive model parameters (3) coefficients of a shift-invariant DCT. We demonstrate that performance is improved by computing features on subblocks, classifying these subblocks, and then combining these results in a way reminiscent of "stacking." State of the art single-feature methods are shown to result in about 75-86 % performance, while the new method results in 90.3 % correct classification, when evaluated on a diverse database of over 1300 consumer images provided by Kodak.

In this paper, we present an approach to color image understanding that can be used to segment and analyze sur- faces with color variations due to highlights and shading. The work is based on a theory-the Dichromatic Reflec- tion Model-which describes the color of the reflected light as a mixture of light from surface reflection (highlights) and body reflection (object color). In the past, we have shown how the dichromatic theory can be used to separate a color image into two intrinsic reflection images: an image of just the highlights, and the original image with the highlights removed. At that time, the algorithm could only be applied to hand-segmented images. This paper shows how the same reflection model can be used to include color image segmentation into the image analysis. The result is a color image understanding system, capable of generating physical descriptions of the reflection processes occurring in the scene. Such descriptions include the intrinsic reflection images, an image segmenta- tion, and symbolic information about the object and highlight colors. This line of research can lead to based image understanding methods that are both more reliable and more useful than traditional methods.

Digital library access is driven by features, but features are often context-dependent and noisy, and their relevance for a query is not always obvious. This paper describes an approach for utilizing many data-dependent, user-dependent, and task-dependent features in a semi-automated tool. Instead of requiring universal similarity measures or manual selection of relevant features, the approach provides a learning algorithm for selecting and combining groupings of the data, where groupings can be induced by highlyspecialized and context-dependent features. The selection process is guided by arichexample-based interaction with the user. The inherent combinatorics

Abstract. This paper studies a statistical skin-color model and its adaptation. It is revealed that (1) human skin colors cluster in a small region in a color space; (2) the variance of a skin color cluster can be reduced by intensity normalization, and (3) under a certain lighting condition, a skin-color distribution can be characterized by amultivariate normal distribution in the normalized color space. We then propose an adaptive model to characterize human skin-color distributions for tracking human faces under di erent lighting conditions. The parameters of the model are adapted based on the maximum likelihood criterion. The model has been successfully applied to a real-time face tracker and other applications. 1

This paper demonstrates a new application of computer vision to digital libraries -- the use of texture for annotation, the description of content. Vision-based annotation assists the user in attaching descriptions to large sets of images and video. If a user labels a piece of an image as "water," a texture model can be used to propagate this label to other "visually similar" regions. However, a serious problem is that no single model has been found to be good enough to reliably match human perception of similarity in pictures. Rather than using one model, the system described here knows several texture models, and is equipped with the ability to choose the one which "best explains" the regions selected by the user for annotating. If none of these models suffices, then it creates new explanations by combining models. Examples are given of annotations propagated by the system on natural scenes. The system provides an average gain of four to one in label prediction over a set of 98 image...

Image segmentation is very essential and critical to image processing and pattern recognition. This survey provides a summary of color image segmentation techniques available now. Basically, color segmentation approaches are based on monochrome segmentation approaches operating in di erent color spaces. Therefore, we rst discuss the major segmentation approaches for segmenting monochrome images: histogram thresholding, characteristic feature clustering, edge detection, region-based methods, fuzzy techniques, neural networks, etc. � then review some major color representation methods and their advantages/disadvantages� nally summarize the color image segmentation techniques using di erent color representations. The usage of color models for image segmentation is also discussed. Some novel approaches such as fuzzy method and physics based method are investigated as well.

We present two solutions for the scale selection problem in computer vision. The first one is completely nonparametric and is based on the the adaptive estimation of the normalized density gradient. Employing the sample point estimator, we define the Variable Bandwidth Mean Shift, prove its convergence, and show its superiority over the fixed bandwidth procedure. The second technique has a semiparametric nature and imposes a local structure on the data to extract reliable scale information. The local scale of the underlying density is taken as the bandwidth which maximizes the magnitude of the normalized mean shift vector. Both estimators provide practical tools for autonomous image and quasi real-time video analysis and several examples are shown to illustrate their effectiveness. 1 Motivation for Variable Bandwidth The efficacy of Mean Shift analysis has been demonstrated in computer vision problems such as tracking and segmentation in [5, 6]. However, one of the limitations of the mean shift procedure as defined in these papers is that it involves the specification of a scale parameter. While results obtained appear satisfactory, when the local characteristics of the feature space differs significantly across data, it is difficult to find an optimal global bandwidth for the mean shift procedure. In this paper we address the issue of locally adapting the bandwidth. We also study an alternative approach for data-driven scale selection which imposes a local structure on the data. The proposed solutions are tested in the framework of quasi real-time video analysis. We review first the intrinsic limitations of the fixed bandwidth density estimation methods. Then, two of the most popular variable bandwidth estimators, the balloon and the sample point, are introduced and...

In this paper, we present anapproach to colorimage understandingthat accountsforcolorvariationsdue to highlights and shading. We demonstrate that the reflected light from every point on a dielectric object. such as plastic, can be described asa linearcombination of the object color and the highlight color. The colors of all light rays reflected from one object then form a planar cluster in the color space.The shapeof this cluster is determined by the object and highlight colors and by the object shape and illumination geometry. We present a method that exploits the difference between object color and highlight color to separate the color of every pixel into a matte component and a highlight component.This generates two intrinsic images, one showing the scene without highlights, and the other one showing only the highlights. The intrinsic images may be a useful tool for a variety of algorithms in computer vision. such as stereo vision, motion analysis, shape from shading,and shapefrom highlights. Ourmethod combines the analysis of matte and highlight reflection with a sensor model that accounts for camera limitations. This enables us to successfully run our algorithm on real images taken in a laboratory setting. We show and discuss the results.

Digital libraries of images and video are rapidly growing in size and availability. To avoid the expense and limitations of text, there is considerable interest in navigation by perceptual and other automatically extractable attributes. Unfortunately, the relevance of an attribute for a query is not always obvious. Queries which go beyond explicit color, shape, and positional cues must incorporate multiple features in complex ways. This dissertation uses machine learning to automatically select and combine features to satisfy a query, based on positive and negative examples from the user. The learning algorithm does not just learn during the course of one session: it learns continuously, across sessions. The learner improves its learning ability by dynamically modifying its inductive bias, based on experience over multiple sessions. Experiments demonstrate the ability to assist image classification, segmentation, and annotation (labeling of image regions). The common theme of this work...

We describe an algorithm for detecting human faces and facial features, such as the location of the eyes, nose, and mouth. First, a supervised pixel-based color classifier is employed to mark all pixels that are within a prespecified distance of "skin color," which is computed from a training set of skin patches. This color-classification map is then smoothed by Gibbs random field model-based filters to define skin regions. An ellipse model is fit to each disjoint skin region. Finally, we introduce symmetry-based cost functions to search the center of the eyes, tip of nose, and center of mouth within ellipses whose aspect ratio is similar to that of a face. Face detection facial feature detection image segmentation shape classification Gibbs random fields 1 Introduction Automatic detection and recognition of faces from still images and video is an active research area. A complete facial image analysis system should be able to localize faces in a given image, identify and pin-point fac...