In this letter we propose a facGE feature extracA-W tecracA whic utilizes polynomial clynomial derived from 2D DiscHWE Cosine Transform (DCT)cT)2:EEB8 obtained from horizontally and vertic:2) neighbouringblochb Fac authenticing2 results on the VidTIMIT database suggest that the proposed feature set is superior (in terms of robustness to illuminationclumin anddiscAB:2)AH8# ability) to features extracs2 using four popular methods: Princs:2 Component Analysis (PCA), PCA with histogram equalizationpre-procion2AB 2D DCT and 2D Gabor wavelets; the results also suggest that histogram equalizationpre-procion2A inc-proc the error rate and o#ers no help against illuminationcuminat Moreover, the proposed feature set is over 80 times faster toc2GWW# than features based on Gabor wavelets. Further experiments on the Weizmann database also show that the proposed approac is more robust than 2D Gabor wavelets and 2D DCT coefficients.

Identity verification systems are an important part of our every day life. A typical example is the Automatic Teller Machine (ATM) which employs a simple identity verification scheme: the user is asked to enter their secret password after inserting their ATM card; if the password matches the one prescribed to the card, the user is allowed access to their bank account. This scheme suffers from a major drawback: only the validity of the combination of a certain possession (the ATM card) and certain knowledge (the password) is verified. The ATM card can be lost or stolen, and the password can be compromised. Thus new verification methods have emerged, where the password has either been replaced by, or used in addition to, biometrics such as the person's speech, face image or fingerprints. Apart from the ATM example described above, biometrics can be applied to other areas, such as telephone & internet based banking, airline reservations & check-in, as well as forensic work and law enforcement applications. Biometric systems

This article first provides an review of important concepts in the field of information fusion, followed by a review of important milestones in audio--visual person identification and verification. Several recent adaptive and nonadaptive techniques for reaching the verification decision (i.e., to accept or reject the claimant), based on speech and face information, are then evaluated in clean and noisy audio conditions on a common database; it is shown that in clean conditions most of the nonadaptive approaches provide similar performance and in noisy conditions most exhibit a severe deterioration in performance; it is also shown that current adaptive approaches are either inadequate or utilize restrictive assumptions. A new category of classifiers is then introduced, where the decision boundary is fixed but constructed to take into account how the distributions of opinions are likely to change due to noisy conditions; compared to a previously proposed adaptive approach, the proposed classifiers do not make a direct assumption about the type of noise that causes the mismatch between training and testing conditions.

This report provides an overview of important concepts in the field of information fusion, followed by a review of literature pertaining to audio-visual person identification & verification. Several recent adaptive and non-adaptive techniques for reaching the verification decision (i.e., to accept or reject the claimant), based on audio and visual information, are evaluated in clean and noisy conditions on a common database using a text-independent setup. It is shown that in clean conditions all the non-adaptive approaches provide similar performance; in noisy conditions they exhibit deterioration in their performance. It is also shown that current adaptive approaches are either inadequate or utilize restrictive assumptions. A new category of classifiers is then introduced, where the decision surface is fixed but constructed to take into account the effects of noisy conditions, providing a good trade-off between performance in clean and noisy conditions. Keywords: multi-modal, fusion, person recognition, person verification, face recognition, face verification, speaker recognition, speaker verification, noise resistance, adaptability.

In this paper, we demonstrate that use ofa recently proposed feature set, termed Maximum Auto-Correlation Values, which utilizes information from the source part of the speech sigchf sighfqjEjkf improves the robustness of a text independent identity veri#cation system. We also propose an adaptive fusion technique forintegzEfAT of audio and visual information in a multi-modal veri#cation system. The proposed technique explicitly measures the quality ofthe speech sigchf adjusting the amount ofcontribution ofthe speech modality to the #nal veri#cation decision. Results on the VidTIMIT database indicate that the proposed approach outperformsexisting adaptive and non-adaptive fusion techniques. For a wide rang of audio SNRs, the performance of the multi-modal systemutilizing the proposed technique is always found to be better than the performance ofthe face modality.

In this report we extend the recently proposed DCT-mod2 feature extraction technique (which utilizes polynomial coefficients derived from 2D DCT coefficients obtained from horizontally & vertically neighbouring blocks) via the use of various windows and diagonally neighbouring blocks. We also evaluate enhanced PCA, where traditional PCA feature extraction is combined with DCT-mod2. Results using test images corrupted by a linear and a non-linear illumination change, white Gaussian noise and compression artefacts, show that use of diagonally neighbouring blocks and windowing is detrimental to robustness against illumination changes while being useful for increasing robustness against white noise and compression artefacts. We also show that the enhanced PCA technique retains all the positive aspects of traditional PCA (that is robustness against white noise and compression artefacts) while also being robust to illumination direction changes; moreover, enhanced PCA outperforms PCA with histogram equalisation pre-processing.

We address the pose mismatch problem which can occur in face verification systems that have only a single (frontal) face image available for training. In the framework of a Bayesian classifier based on mixtures of gaussians, the problem is tackled through extending each frontal face model with artificially synthesized models for non-frontal views. The synthesis methods are based on several implementations of Maximum Likelihood Linear Regression (MLLR), as well as standard multi-variate linear regression (LinReg). All synthesis techniques rely on prior information and learn how face models for the frontal view are related to face models for non-frontal views. The synthesis and extension approach is evaluated by applying it to two face verification systems: a holistic system (based on PCA-derived features) and a local feature system (based on DCT-derived features). Experiments on the FERET database suggest that for the holistic system, the LinReg based technique is more suited than the MLLR based techniques; for the local feature system, the results show that synthesis via a new MLLR implementation obtains better performance than synthesis based on traditional MLLR. The results further suggest that extending frontal models considerably reduces errors. It is also shown that the local feature system is less affected by view changes than the holistic system; this can be attributed to the parts based representation of the face, and, due to the classifier based on mixtures of gaussians, the lack of constraints on spatial relations between the face parts, allowing for deformations and movements of face areas.

In this paper, we propose a novel ensemble-based approach to boost performance of traditional Linear Discriminant Analysis (LDA)-based methods used in face recognition. The ensemble-based approach is based on the recently emerged technique known as “boosting”. However, it is generally believed that boosting-like learning rules are not suited to a strong and stable learner such as LDA. To break the limitation, a novel weakness analysis theory is developed here. The theory attempts to boost a strong learner by increasing the diversity between the classifiers created by the learner, at the expense of decreasing their margins, so as to achieve a trade-off suggested by recent boosting studies for a low generalization error. In addition, a novel distribution accounting for the pairwise class dis-criminant information is introduced for effective interaction between the booster and the LDA-based learner. The integration of all these methodologies proposed here leads to the novel ensemble-based discriminant learning approach, capable of taking advantage of both the boosting and LDA techniques. Promising experimental results obtained on various difficult face recognition scenarios demonstrate the effectiveness of the proposed approach. We believe that this work is especially beneficial in extending the boosting framework to accommodate general (strong/weak) learners.

In the framework of a face verification system using a Gaussian Mixture Model (GMM) based classifier, we address the problem of non-frontal face verification (when only a single (frontal) training image is available) by augmenting a client's frontal face model with artificially synthesized models for non-frontal views. Several techniques are proposed for the synthesis: "difference between two universal background models" (UBMdiff), Maximum Likelihood Linear Regression (MLLR) based, Maximum Likelihood Shift (MLS) based and standard multi-variate linear regression (LinReg) based. All techniques rely on prior information and learn how a generic face model for the frontal view is related to generic face models at non-frontal views. The synthesis and augmentation approach is evaluated by applying it to two face verification systems: PCA based and DCTmod2 based [32]; the two systems are a representation of holistic and local feature approaches, respectively. Results from experiments on the FERET database suggest that the LinReg technique (which is based on a common relation between two sets of points) is more suited to the PCA based system compared to the other techniques (which in effect are "single point to single point" transforms in the PCA based system). For the DCTmod2 based system, the results suggest that the proposed MLS technique (where the shift of the means is found under a maximum likelihood constraint) is more suitable than MLLR (due to a lower number of free parameters) and UBMdiff (due to lack of heuristics). The results further suggest that frontal model augmentation has beneficial effects for both PCA and DCTmod2 based systems. The results also suggest that the standard DCTmod2 based system is less affected by out-of-plane rotations than the corresponding ...

In this paper we propose thediscxzx cscx transform (DCT) mod 2 feature set, whic utilizes polynomial clynomial derived from 2D DCT cfz#:TxfI obtained from spatially neighboringblochb Fac veri#cring results on the multi-session VidTIMIT database suggest that the DCT-mod 2 feature set is superior (in terms of robustness to illuminationdirecina creci and discTMfIkVkx: ability) to features extracsf using three popular methods:eigenfack princfac cincfac analysis, 2D DCT and 2D Gabor wavelets. Moreover,creover to Gabor wavelets, the DCT-mod 2 feature set is over 80 times faster to cofVAAk Additional experiments on the Weizmann database also show that the DCT-mod 2approac is more robust than 2D Gabor wavelets and 2D DCTcfzxxx#fIk ? 2003 Elsevier Scvier B.V. All rights reserved.