Results 1  10
of
48
Statistical Comparisons of Classifiers over Multiple Data Sets
, 2006
"... While methods for comparing two learning algorithms on a single data set have been scrutinized for quite some time already, the issue of statistical tests for comparisons of more algorithms on multiple data sets, which is even more essential to typical machine learning studies, has been all but igno ..."
Abstract

Cited by 328 (0 self)
 Add to MetaCart
While methods for comparing two learning algorithms on a single data set have been scrutinized for quite some time already, the issue of statistical tests for comparisons of more algorithms on multiple data sets, which is even more essential to typical machine learning studies, has been all but ignored. This article reviews the current practice and then theoretically and empirically examines several suitable tests. Based on that, we recommend a set of simple, yet safe and robust nonparametric tests for statistical comparisons of classifiers: the Wilcoxon signed ranks test for comparison of two classifiers and the Friedman test with the corresponding posthoc tests for comparison of more classifiers over multiple data sets. Results of the latter can also be neatly presented with the newly introduced CD (critical difference) diagrams.
An extension on ―statistical comparisons of classifiers over multiple data sets‖ for all pairwise comparisons
 Journal of Machine Learning Research
"... In a recently published paper in JMLR, Demˇsar (2006) recommends a set of nonparametric statistical tests and procedures which can be safely used for comparing the performance of classifiers over multiple data sets. After studying the paper, we realize that the paper correctly introduces the basic ..."
Abstract

Cited by 75 (17 self)
 Add to MetaCart
(Show Context)
In a recently published paper in JMLR, Demˇsar (2006) recommends a set of nonparametric statistical tests and procedures which can be safely used for comparing the performance of classifiers over multiple data sets. After studying the paper, we realize that the paper correctly introduces the basic procedures and some of the most advanced ones when comparing a control method. However, it does not deal with some advanced topics in depth. Regarding these topics, we focus on more powerful proposals of statistical procedures for comparing n×n classifiers. Moreover, we illustrate an easy way of obtaining adjusted and comparable pvalues in multiple comparison procedures.
Studies of tropical tuna swimming performance in a large water tunnel
 I. Energetics. J. exp. Biol
, 1994
"... The body temperatures (Tb) of nine yellowfin tuna (Thunnus albacares) were monitored while fish swam in a large water tunnel at controlled velocities (U) and ambient temperatures (Ta). Monitoring Tb during step changes in Ta at constant U permitted estimation of the thermal rate coefficient (k), an ..."
Abstract

Cited by 31 (4 self)
 Add to MetaCart
The body temperatures (Tb) of nine yellowfin tuna (Thunnus albacares) were monitored while fish swam in a large water tunnel at controlled velocities (U) and ambient temperatures (Ta). Monitoring Tb during step changes in Ta at constant U permitted estimation of the thermal rate coefficient (k), an index of heat transfer. In the yellowfin, k is dependent on both Ta and the direction of the thermal gradient (i.e. whether Ta is greater or less than Tb). Modulation of k in response to Ta was further demonstrated during tests in which U was varied; the elevation of Tb in response to equal increases in U was 3–4 times less at 30 ˚C than at 25 and 20 ˚C. These experiments demonstrate that the yellowfin tuna can modulate heat transfer. This ability could prevent overheating during intense activity, retard heat loss during a descent into cool water and permit increased heat gain upon returning to warm surface waters (i.e. when Tb<Ta).
Fast and scalable local kernel machines
 J. Mach Learn Res
, 2009
"... A computationally efficient approach to local learning with kernel methods is presented. The Fast Local Kernel Support Vector Machine (FaLKSVM) trains a set of local SVMs on redundant neighbourhoods in the training set and an appropriate model for each query point is selected at testing time accord ..."
Abstract

Cited by 6 (0 self)
 Add to MetaCart
(Show Context)
A computationally efficient approach to local learning with kernel methods is presented. The Fast Local Kernel Support Vector Machine (FaLKSVM) trains a set of local SVMs on redundant neighbourhoods in the training set and an appropriate model for each query point is selected at testing time according to a proximity strategy. Supported by a recent result by Zakai and Ritov (2009) relating consistency and localizability, our approach achieves high classification accuracies by dividing the separation function in local optimisation problems that can be handled very efficiently from the computational viewpoint. The introduction of a fast local model selection further speedsup the learning process. Learning and complexity bounds are derived for FaLKSVM, and the empirical evaluation of the approach (with data sets up to 3 million points) showed that it is much faster and more accurate and scalable than stateoftheart accurate and approximated SVM solvers at least for non highdimensional data sets. More generally, we show that locality can be an important factor to sensibly speedup learning approaches and kernel methods, differently from other recent techniques that tend to dismiss local information in order to improve scalability.
FR3: A fuzzy rule learner for inducing reliable classifiers
 IEEE Transactions Fuzzy Systems
, 2009
"... This paper introduces a fuzzy rulebased classification method called FR3, which is short for ..."
Abstract

Cited by 6 (1 self)
 Add to MetaCart
(Show Context)
This paper introduces a fuzzy rulebased classification method called FR3, which is short for
FURIA: An Algorithm For Unordered Fuzzy Rule Induction
"... This paper introduces a novel fuzzy rulebased classification method called FURIA, which is short for Fuzzy Unordered Rule Induction Algorithm. FURIA extends the wellknown RIPPER algorithm, a stateoftheart rule learner, while preserving its advantages, such as simple and comprehensible rule sets ..."
Abstract

Cited by 3 (0 self)
 Add to MetaCart
(Show Context)
This paper introduces a novel fuzzy rulebased classification method called FURIA, which is short for Fuzzy Unordered Rule Induction Algorithm. FURIA extends the wellknown RIPPER algorithm, a stateoftheart rule learner, while preserving its advantages, such as simple and comprehensible rule sets. In addition, it includes a number of modifications and extensions. In particular, FURIA learns fuzzy rules instead of conventional rules and unordered rule sets instead of rule lists. Moreover, to deal with uncovered examples, it makes use of an efficient rule stretching method. Experimental results show that FURIA significantly outperforms the original RIPPER, as well as other classifiers such as C4.5, in terms of classification accuracy. 1
Is an ordinal class structure useful in classifier learning
 Int. J. of Data Mining, Modelling and Management
, 2008
"... In recent years, a number of machine learning algorithms have been developed for the problem of ordinal classification. These algorithms try to exploit, in one way or the other, the order information of the problem, essentially relying on the assumption that the ordinal structure of the set of class ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
(Show Context)
In recent years, a number of machine learning algorithms have been developed for the problem of ordinal classification. These algorithms try to exploit, in one way or the other, the order information of the problem, essentially relying on the assumption that the ordinal structure of the set of class labels is also reflected in the topology of the instance space. The purpose of this paper is to investigate, on an experimental basis, the validity of this assumption. Moreover, we seek to answer the question to what extent existing techniques and learning algorithms for ordinal classification are able to exploit order information, and which properties of these techniques are important in this regard.
LETTER Why Not Use an Oracle When You Got One?
, 2006
"... Abstract—The primary goal of predictive modeling is to achieve high accuracy when the model is applied to novel data. For certain problems this requires the use of complex techniques like neural networks or ensembles, resulting in opaque models that are hard or impossible to interpret. For some doma ..."
Abstract

Cited by 2 (1 self)
 Add to MetaCart
(Show Context)
Abstract—The primary goal of predictive modeling is to achieve high accuracy when the model is applied to novel data. For certain problems this requires the use of complex techniques like neural networks or ensembles, resulting in opaque models that are hard or impossible to interpret. For some domains this is unacceptable, since models need to be comprehensible. To achieve comprehensibility, accuracy is often sacrificed by using simpler techniques; a tradeoff termed the accuracy vs. comprehensibility tradeoff. Another, frequently studied, alternative is rule extraction; i.e. the activity where another, transparent, model is generated from the opaque model. In this paper it is argued that existing rule extraction algorithms do not use all information available, and typically should benefit from also using oracle data; i.e. test set instances, together with corresponding predictions from the opaque model. The experiments, using fifteen publicly available data sets, clearly show that rules extracted using either just oracle data or training data augmented with oracle data, will explain the predictions significantly better than rules extracted in the standard way; i.e. using training data only. Keywords—Rule extraction, neural networks, accuracy vs. comprehensibility tradeoff 1.
E.: FuzzyUCS: A Michiganstyle Learning FuzzyClassifier System for Supervised Learning
 IEEE Transactions on Evolutionary Computation
"... learning tasks. FuzzyUCS is inspired by UCS, an ..."
(Show Context)
Analysis of the KDD cup 2009: Fast scoring on a large orange customer database
 In KDD cup 2009, in press, volume 8. JMLR W&CP
"... We organized the KDD cup 2009 around a marketing problem with the goal of identifying data mining techniques capable of rapidly building predictive models and scoring new entries on a large database. Customer Relationship Management (CRM) is a key element of modern marketing strategies. The KDD Cup ..."
Abstract

Cited by 2 (2 self)
 Add to MetaCart
(Show Context)
We organized the KDD cup 2009 around a marketing problem with the goal of identifying data mining techniques capable of rapidly building predictive models and scoring new entries on a large database. Customer Relationship Management (CRM) is a key element of modern marketing strategies. The KDD Cup 2009 offered the opportunity to work on large marketing databases from the French Telecom company Orange to predict the propensity of customers to switch provider (churn), buy new products or services (appetency), or buy upgrades or addons proposed to them to make the sale more profitable (upselling). The challenge started on March 10, 2009 and ended on May 11, 2009. This challenge attracted over 450 participants from 46 countries. We attribute the popularity of the challenge to several factors: (1) A generic problem relevant to the Industry (a classification problem), but presenting a number of scientific and technical challenges of practical interest including: a large number of training examples (50,000) with a large number of missing values (about 60%) and a large number of features (15,000), unbalanced class proportions (fewer than 10 % of the examples of the positive class), noisy data, presence of categorical variables