Results 1  10
of
56
An introduction to variable and feature selection
 Journal of Machine Learning Research
, 2003
"... Variable and feature selection have become the focus of much research in areas of application for which datasets with tens or hundreds of thousands of variables are available. ..."
Abstract

Cited by 1100 (17 self)
 Add to MetaCart
(Show Context)
Variable and feature selection have become the focus of much research in areas of application for which datasets with tens or hundreds of thousands of variables are available.
SPECTRUM ESTIMATION FOR LARGE DIMENSIONAL COVARIANCE MATRICES USING RANDOM MATRIX THEORY
 SUBMITTED TO THE ANNALS OF STATISTICS
"... Estimating the eigenvalues of a population covariance matrix from a sample covariance matrix is a problem of fundamental importance in multivariate statistics; the eigenvalues of covariance matrices play a key role in many widely techniques, in particular in Principal Component Analysis (PCA). In ma ..."
Abstract

Cited by 62 (4 self)
 Add to MetaCart
Estimating the eigenvalues of a population covariance matrix from a sample covariance matrix is a problem of fundamental importance in multivariate statistics; the eigenvalues of covariance matrices play a key role in many widely techniques, in particular in Principal Component Analysis (PCA). In many modern data analysis problems, statisticians are faced with large datasets where the sample size, n, is of the same order of magnitude as the number of variables p. Random matrix theory predicts that in this context, the eigenvalues of the sample covariance matrix are not good estimators of the eigenvalues of the population covariance. We propose to use a fundamental result in random matrix theory, the MarčenkoPastur equation, to better estimate the eigenvalues of large dimensional covariance matrices. The MarčenkoPastur equation holds in very wide generality and under weak assumptions. The estimator we obtain can be thought of as “shrinking ” in a non linear fashion the eigenvalues of the sample covariance matrix to estimate the population eigenvalues. Inspired by ideas of random matrix theory, we also suggest a change of point of view when thinking about estimation of highdimensional vectors: we do not try to estimate directly the vectors but rather a probability measure that describes them. We think this is a theoretically more fruitful way to think about these problems. Our estimator gives fast and good or very good results in extended simulations. Our algorithmic approach is based on convex optimization. We also show that the proposed estimator is consistent.
Testing for Homogeneity with Kernel Fisher Discriminant Analysis
"... We propose to investigate test statistics for testing homogeneity based on kernel Fisher discriminant analysis. Asymptotic null distributions under null hypothesis are derived, and consistency against fixed alternatives is assessed. Finally, experimental evidence of the performance of the proposed a ..."
Abstract

Cited by 25 (12 self)
 Add to MetaCart
(Show Context)
We propose to investigate test statistics for testing homogeneity based on kernel Fisher discriminant analysis. Asymptotic null distributions under null hypothesis are derived, and consistency against fixed alternatives is assessed. Finally, experimental evidence of the performance of the proposed approach on both artificial and real datasets is provided. 1
Tradeoffs in the Empirical Evaluation of Competing Algorithm Designs
"... Abstract. We propose an empirical analysis approach for characterizing tradeoffs between different methods for comparing a set of competing algorithm designs. Our approach can provide insight into performance variation both across candidate algorithms and across instances. It can also identify the b ..."
Abstract

Cited by 11 (2 self)
 Add to MetaCart
(Show Context)
Abstract. We propose an empirical analysis approach for characterizing tradeoffs between different methods for comparing a set of competing algorithm designs. Our approach can provide insight into performance variation both across candidate algorithms and across instances. It can also identify the best tradeoff between evaluating a larger number of candidate algorithm designs, performing these evaluations on a larger number of problem instances, and allocating more time to each algorithm run. We applied our approach to a study of the rich algorithm design spaces offered by three highlyparameterized, stateoftheart algorithms for satisfiability and mixed integer programming, considering six different distributions of problem instances. We demonstrate that the resulting algorithm design scenarios differ in many ways, with important consequences for both automatic and manual algorithm design. We expect that both our methods and our findings will lead to tangible improvements in algorithm design methods.
Detecting Correlation in Stock Market
 Physica A: Statistical Mechanics and its Applications, Volume 344, Issues 12
, 2004
"... We present a new method for detecting dependencies in the stock market. In order to find hidden correlations in the daily returns, we build cross prediction models and use the normalized modeling error as a generalized correlation measure that extends the concept of the classical correlation matrix. ..."
Abstract

Cited by 7 (5 self)
 Add to MetaCart
We present a new method for detecting dependencies in the stock market. In order to find hidden correlations in the daily returns, we build cross prediction models and use the normalized modeling error as a generalized correlation measure that extends the concept of the classical correlation matrix.
VFOLD CROSSVALIDATION IMPROVED: VFOLD PENALIZATION
 SUBMITTED TO THE ANNALS OF STATISTICS
, 2008
"... We study the efficiency of Vfold crossvalidation (VFCV) for model selection from the nonasymptotic viewpoint, and suggest an improvement on it, which we call “Vfold penalization”. Considering a particular (though simple) regression problem, we prove that VFCV with a bounded V is suboptimal for m ..."
Abstract

Cited by 5 (2 self)
 Add to MetaCart
We study the efficiency of Vfold crossvalidation (VFCV) for model selection from the nonasymptotic viewpoint, and suggest an improvement on it, which we call “Vfold penalization”. Considering a particular (though simple) regression problem, we prove that VFCV with a bounded V is suboptimal for model selection, because it “overpenalizes ” all the more that V is large. Hence, asymptotic optimality requires V to go to infinity. However, when the signaltonoise ratio is low, it appears that overpenalizing is necessary, so that the optimal V is not always the larger one, despite of the variability issue. This is confirmed by some simulated data. In order to improve on the prediction performance of VFCV, we define a new model selection procedure, called “Vfold penalization” (penVF). It is a Vfold subsampling version of Efron’s bootstrap penalties, so that it has the same computational cost as VFCV, while being more flexible. In a heteroscedastic regression framework, assuming the models to have a particular structure, we prove that penVF satisfies a nonasymptotic oracle inequality with a leading constant that tends to 1 when the sample size goes to infinity. In particular, this implies adaptivity to the smoothness of the regression function, even with a highly heteroscedastic noise. Moreover, it is easy to overpenalize with penVF, independently from the V parameter. A simulation study shows that this results in a significant improvement on VFCV in nonasymptotic situations.
Tropical Implicitization
, 2010
"... In recent years, tropical geometry has developed as a theory on its own. Its two main aims are to answer open questions in algebraic geometry and to give new proofs of celebrated classical results. The main subject of this thesis is concerned with the former: the solution of implicitization problems ..."
Abstract

Cited by 5 (1 self)
 Add to MetaCart
In recent years, tropical geometry has developed as a theory on its own. Its two main aims are to answer open questions in algebraic geometry and to give new proofs of celebrated classical results. The main subject of this thesis is concerned with the former: the solution of implicitization problems via tropical geometry. We develop new and explicit techniques that completely solve these challenges in four concrete examples. We start by studying a family of challenging examples inspired by algebraic statistics and machine learning: the restricted Boltzmann machines F(n, k). These machines are highly structured projective varieties in tensor spaces. They correspond to a statistical model encoded by the complete bipartite graph Kk,n, by marginalizing k of the n+ k binary random variables. In Chapter 2, we investigate this problem in the most general setting. We conjecture a formula for the expected dimension of the model, verifying it in all relevant cases. We also study inference functions and their interplay with tropicalization of polynomial maps. In Chapter 3, we focus on the particular case F(4, 2), answering a question by Drton, Sturmfels and Sullivant regarding the degree (and Newton polytope) of the homogeneous equation in 16 variables defining this model. We show that its degree is 110 and compute
Feature selection for satellite image indexing
 In ESAEUSC: Image Information Mining
, 2005
"... Lots of feature extraction procedures can be found in the literature. We propose to automatically select the best ones using automatic supervised feature selection algorithms. We demonstrate the efficiency of such a methodology in comparing texture features sets (using Haralick coefficients, Gaussia ..."
Abstract

Cited by 4 (2 self)
 Add to MetaCart
(Show Context)
Lots of feature extraction procedures can be found in the literature. We propose to automatically select the best ones using automatic supervised feature selection algorithms. We demonstrate the efficiency of such a methodology in comparing texture features sets (using Haralick coefficients, Gaussian Markov Random Fields, several wavelets,...) computed on satellite images. We illustrate the fact that geometrical features and texture features have to be combined to enhance the discriminative power of the selected features set. Key words: Feature Selection, supervised classification, satellite image indexing.
Regularized allpole models for speaker verification under noisy environments
 IEEE Sig. Proc. Lett
, 2012
"... Regularization of linear prediction based melfrequency cepstral coefficient (MFCC) extraction in speaker verification is considered. Commonly, MFCCs are extracted from the discrete Fourier transform (DFT) spectrum of speech frames. In our recent study, it was shown that replacing the DFT spectrum ..."
Abstract

Cited by 4 (3 self)
 Add to MetaCart
(Show Context)
Regularization of linear prediction based melfrequency cepstral coefficient (MFCC) extraction in speaker verification is considered. Commonly, MFCCs are extracted from the discrete Fourier transform (DFT) spectrum of speech frames. In our recent study, it was shown that replacing the DFT spectrum estimation step with the conventional and temporally weighted linear prediction (LP) and their regularized versions increases the recognition performance considerably. In this paper, we provide a through analysis on the regularization of conventional and temporally weighted LP methods. Experiments on the NIST 2002 corpus indicate that regularized allpole methods yield large improvements on recognition accuracy under additive factory and babble noise conditions in terms of both equal error rate (EER) and minimum detection cost function (MinDCF). 1.
Prediction of the InterObserver Visual Congruency (IOVC) and application to image ranking
"... This paper proposes an automatic method for predicting the interobserver visual congruency (IOVC). The IOVC reflects the congruence or the variability among different subjects looking at the same image. Predicting this congruence is of interest for image processing applications where the visual per ..."
Abstract

Cited by 3 (2 self)
 Add to MetaCart
(Show Context)
This paper proposes an automatic method for predicting the interobserver visual congruency (IOVC). The IOVC reflects the congruence or the variability among different subjects looking at the same image. Predicting this congruence is of interest for image processing applications where the visual perception of a picture matters such as website design, advertisement, etc. This paper makes several new contributions. First, a computational model of the IOVC is proposed. This new model is a mixture of lowlevel visual features extracted from the input picture where model’s parameters are learned by using a large eyetracking database. Once the parameters have been learned, it can be used for any new picture. Second, regarding lowlevel visual feature extraction, we propose a new scheme to compute the depth of field of a picture. Finally, once the training and the feature extraction have been carried out, a score ranging from 0 (minimal congruency) to 1 (maximal congruency) is computed. A value of 1 indicates that observers would focus on the same locations and suggests that the picture presents strong locations of interest. A second database of eye movements is used to assess the performance of the proposed model. Results show that our IOVC criterion outperforms the Feature Congestion measure [33]. To illustrate the interest of the proposed model, we have used it to automatically rank personalized photograph.