We introduce a new distance between two distributions that we call the Earth Mover’s Distance (EMD), which reflects the minimal amount of work that must be performed to transform one distributioninto the other by moving “distribution mass ” around. This is a special case of the transportation problem from linear optimization, for which efficient algorithms are available. The EMD also allows for partial matching. When used to compare distributions that have the same overall mass, the EMD is a true metric, and has easy-to-compute lower bounds. In this paper we focus on applications to image databases, especially color and texture. We use the EMD to exhibit the structure of color-distribution and texture spaces by means of Multi-Dimensional Scaling displays. We also propose a novel approach to the problem of navigating through a collection of color images, which leads to a new paradigm for image database search. 1

This article presents a statistical theory for texture modeling. This theory combines filtering theory and Markov random field modeling through the maximum entropy principle, and interprets and clarifies many previous concepts and methods for texture analysis and synthesis from a unified point of view. Our theory characterizes the ensemble of images I with the same texture appearance by a probability distribution f (I) on a random field, and the objective of texture modeling is to make inference about f (I), given a set of observed texture examples. In our theory, texture modeling consists of two steps. (1) A set of filters is selected from a general filter bank to capture features of the texture, these filters are applied to observed texture images, and the histograms of the filtered images are extracted. These histograms are estimates of the marginal distributions of f (I). This step is called feature extraction. (2) The maximum entropy principle is employed to derive a distribution p(I), which is restricted to have the same marginal distributions as those in (1). This p(I) is considered as an estimate of f (I). This step is called feature fusion. A stepwise algorithm is proposed to choose filters from a general filter bank. The resulting model, called FRAME (Filters, Random fields And Maximum Entropy), is a Markov random field (MRF) model, but with a much enriched vocabulary and hence much stronger descriptive ability than the previous MRF models used for texture modeling. Gibbs sampler is adopted to synthesize texture images by drawing typical samples from p(I), thus the model is verified by seeing whether the synthesized texture images have similar visual appearances

We apply a Bayesian method for inferring an optimal basis to the problem of finding efficient image codes for natural scenes. The basis functions learned by the algorithm are oriented and localized in both space and frequency, bearing a resemblance to two-dimensional Gabor functions, and increasing the number of basis functions results in a greater sampling density in position, orientation, and scale. These properties also resemble the spatial receptive fields of neurons in the primary visual cortex of mammals, suggesting that the receptive-field structure of these neurons can be accounted for by a general efficient coding principle. The probabilistic framework provides a method for comparing the coding efficiency of different bases objectively by calculating their probability given the observed data or by measuring the entropy of the basis function coefficients. The learned bases are shown to have better coding efficiency than traditional Fourier and wavelet bases. This framework also provides a Bayesian solution to the problems of image denoising and filling in of missing pixels. We demonstrate that the results obtained by applying the learned bases to these problems are improved over those obtained with traditional techniques. © 1999 Optical Society of America [S0740-3232(99)03107-5] OCIS codes: 000.5490, 100.2960, 100.3010.

With an increasing emphasis on security, automated personal identification based on biometrics has been receiving extensive attention over the past decade. Iris recognition, as an emerging biometric recognition approach, is becoming a very active topic in both research and practical applications. In general, a typical iris recognition system includes iris imaging, iris liveness detection, and recognition. This paper focuses on the last issue and describes a new scheme for iris recognition from an image sequence. We first assess the quality of each image in the input sequence and select a clear iris image from such a sequence for subsequent recognition. A bank of spatial filters, whose kernels are suitable for iris recognition, is then used to capture local characteristics of the iris so as to produce discriminating texture features. Experimental results show that the proposed method has an encouraging performance. In particular, a comparative study of existing methods for iris recognition is conducted on an iris image database including 2,255 sequences from 213 subjects. Conclusions based on such a comparison using a nonparametric statistical method (the bootstrap) provide useful information for further research.

We extend the geometric framework introduced in Sochen et al. (IEEE Trans. on Image Processing, 7(3):310–318, 1998) for image enhancement. We analyze and propose enhancement techniques that selectively smooth images while preserving either the multi-channel edges or the orientation-dependent texture features in them. Images are treated as manifolds in a feature-space. This geometrical interpretation lead to a general way for grey level, color, movies, volumetric medical data, and color-texture image enhancement. We first review our framework in which the Polyakov action from high-energy physics is used to develop a minimization procedure through a geometric flow for images. Here we show that the geometric flow, based on manifold volume minimization, yields a novel enhancement procedure for color images. We apply the geometric framework and the general Beltrami flow to feature-preserving denoising of images in various spaces. Next, we introduce a new method for color and texture enhancement. Motivated by Gabor’s geometric image sharpening method (Gabor, Laboratory Investigation, 14(6):801–807, 1965), we present a geometric sharpening procedure for color images with texture. It is based on inverse diffusion across the multi-channel edge, and diffusion along the edge.

A biometric system provides automatic identification of an individual based on a unique feature or characteristic possessed by the individual. Iris recognition is regarded as the most reliable and accurate biometric identification system available. Most commercial iris recognition systems use patented algorithms developed by Daugman, and these algorithms are able to produce perfect recognition rates. However, published results have usually been produced under favourable conditions, and there have been no independent trials of the technology. The work presented in this thesis involved developing an ‘open-source ’ iris recognition system in order to verify both the uniqueness of the human iris and also its performance as a biometric. For determining the recognition performance of the system two databases of digitised greyscale eye images were used. The iris recognition system consists of an automatic segmentation system that is based on the Hough transform, and is able to localise the circular iris and pupil region, occluding eyelids and eyelashes, and reflections. The extracted iris region was then

In this letter we propose a facGE feature extracA-W tecracA whic utilizes polynomial clynomial derived from 2D DiscHWE Cosine Transform (DCT)cT)2:EEB8 obtained from horizontally and vertic:2) neighbouringblochb Fac authenticing2 results on the VidTIMIT database suggest that the proposed feature set is superior (in terms of robustness to illuminationclumin anddiscAB:2)AH8# ability) to features extracs2 using four popular methods: Princs:2 Component Analysis (PCA), PCA with histogram equalizationpre-procion2AB 2D DCT and 2D Gabor wavelets; the results also suggest that histogram equalizationpre-procion2A inc-proc the error rate and o#ers no help against illuminationcuminat Moreover, the proposed feature set is over 80 times faster toc2GWW# than features based on Gabor wavelets. Further experiments on the Weizmann database also show that the proposed approac is more robust than 2D Gabor wavelets and 2D DCT coefficients.

In this paper, we investigate the use of two types of features extracted from face images for recognizing facial expressions. The first type is the geometric positions of a set of fiducial points on a face. The second type is a set of multi-scale and multi-orientation Gabor wavelet coefficients extracted from the face image at the fiducial points. They can be used either independently or jointly. The architecture we developed is based on a two-layer perceptron. The recognition performance with different types of features has been compared, which shows that Gabor wavelet coefficients are much more powerful than geometric positions. Furthermore, since the first layer of the perceptron actually performs a nonlinear reduction of the dimensionality of the feature space, we have also studied the desired number of hidden units, i.e., the appropriate dimension to represent a facial expression in order to achieve a good recognition rate. It turns out that five to seven hidden units are probably...

Abstract—A system that could automatically analyze the facial actions in real time has applications in a wide range of different fields. However, developing such a system is always challenging due to the richness, ambiguity, and dynamic nature of facial actions. Although a number of research groups attempt to recognize facial action units (AUs) by improving either the facial feature extraction techniques or the AU classification techniques, these methods often recognize AUs or certain AU combinations individually and statically, ignoring the semantic relationships among AUs and the dynamics of AUs. Hence, these approaches cannot always recognize AUs reliably, robustly, and consistently. In this paper, we propose a novel approach that systematically accounts for the relationships among AUs and their temporal evolutions for AU recognition. Specifically, we use a dynamic Bayesian network (DBN) to model the relationships among different AUs. The DBN provides a coherent and unified hierarchical probabilistic framework to represent probabilistic relationships among various AUs and to account for the temporal changes in facial action development. Within our system, robust computer vision techniques are used to obtain AU measurements. Such AU measurements are then applied as evidence to the DBN for inferring various AUs. The experiments show that the integration of AU relationships and AU dynamics with AU measurements yields significant improvement of AU recognition, especially for spontaneous facial expressions and under more realistic environment including illumination variation, face pose variation, and occlusion. Index Terms—Facial action unit recognition, facial expression analysis, Facial Action Coding System, Bayesian networks. 1

We present in this paper an Independent Gabor Features (IGF) method and its application to face recognition. The novelty of the IGF method comes from (i) the derivation of independent Gabor features in the feature extraction stage, and (ii) the development of an IGF features-based Probabilistic Reasoning Model (PRM) classification method in the pattern recognition stage. In particular, the IGF method first derives a Gabor feature vector from a set of downsampled Gabor wavelet representations of face images, then reduces the dimensionality of the vector by means of Principal Component Analysis (PCA), and finally defines the independent Gabor features based on the Independent Component Analysis (ICA). The independence property of these Gabor features facilitates the application of the PRM method for classification. The rationale behind integrating the Gabor wavelets and the ICA is two-fold. On the one hand, the Gabor transformed face images exhibit strong characteristics of spatial locality, scale and orientation selectivity. These images can thus produce salient local features that are most suitable for face recognition. On the other hand, ICA would further reduce redundancy and represent independent features explicitly. These independent features are most useful for subsequent pattern discrimination and associative recall. Experiments on face recognition using the FERET and the Chengjun Liu is with the Department of Computer Science, New Jersey Institute of Technology, Newark, NJ 07102. E-mail: liu@cs.njit.edu.