Results 11  20
of
535
Tree Induction for Probabilitybased Ranking
, 2002
"... Tree induction is one of the most effective and widely used methods for building classification models. However, many applications require cases to be ranked by the probability of class membership. Probability estimation trees (PETs) have the same attractive features as classification trees (e.g., c ..."
Abstract

Cited by 158 (4 self)
 Add to MetaCart
(Show Context)
Tree induction is one of the most effective and widely used methods for building classification models. However, many applications require cases to be ranked by the probability of class membership. Probability estimation trees (PETs) have the same attractive features as classification trees (e.g., comprehensibility, accuracy and efficiency in high dimensions and on large data sets). Unfortunately, decision trees have been found to provide poor probability estimates. Several techniques have been proposed to build more accurate PETs, but, to our knowledge, there has not been a systematic experimental analysis of which techniques actually improve the probabilitybased rankings, and by how much. In this paper we first discuss why the decisiontree representation is not intrinsically inadequate for probability estimation. Inaccurate probabilities are partially the result of decisiontree induction algorithms that focus on maximizing classification accuracy and minimizing tree size (for example via reducederror pruning). Larger trees can be better for probability estimation, even if the extra size is superfluous for accuracy maximization. We then present the results of a comprehensive set of experiments, testing some straghtforward methods for improving probabilitybased rankings. We show that using a simple, common smoothing methodthe Laplace correctionuniformly improves probabilitybased rankings. In addition, bagging substantioJly improves the rankings, and is even more effective for this purpose than for improving accuracy. We conclude that PETs, with these simple modifications, should be considered when rankings based on classmembership probability are required.
Explicitly representing expected cost: an alternative to ROC representation
 KDD
, 2000
"... This paper proposes an alternative to ROC representation, in which the expected cost of a classier is represented explicitly. This expected cost representation maintains many of the advantages of ROC representation, but is easier to understand. It allows the experimenter to immediately see the rang ..."
Abstract

Cited by 93 (11 self)
 Add to MetaCart
(Show Context)
This paper proposes an alternative to ROC representation, in which the expected cost of a classier is represented explicitly. This expected cost representation maintains many of the advantages of ROC representation, but is easier to understand. It allows the experimenter to immediately see the range of costs and class frequencies where a particular classier is the best and quantitatively how much better it is than other classiers. This paper demonstrates there is a point/line duality between the two representations. A point in ROC space representing a classier becomes a line segment spanning the full range of costs and class frequencies. This duality produces equivalent operations in the two spaces, allowing most techniques used in ROC analysis to be readily reproduced in the cost space.
SJ: BindN: a webbased tool for efficient prediction of DNA and RNA binding sites in amino acid sequences
 Nucleic Acids Res
"... BindN ..."
(Show Context)
Tree induction vs. logistic regression: A learningcurve analysis
 CEDER WORKING PAPER #IS0102, STERN SCHOOL OF BUSINESS
, 2001
"... Tree induction and logistic regression are two standard, offtheshelf methods for building models for classi cation. We present a largescale experimental comparison of logistic regression and tree induction, assessing classification accuracy and the quality of rankings based on classmembership pr ..."
Abstract

Cited by 85 (17 self)
 Add to MetaCart
(Show Context)
Tree induction and logistic regression are two standard, offtheshelf methods for building models for classi cation. We present a largescale experimental comparison of logistic regression and tree induction, assessing classification accuracy and the quality of rankings based on classmembership probabilities. We use a learningcurve analysis to examine the relationship of these measures to the size of the training set. The results of the study show several remarkable things. (1) Contrary to prior observations, logistic regression does not generally outperform tree induction. (2) More specifically, and not surprisingly, logistic regression is better for smaller training sets and tree induction for larger data sets. Importantly, this often holds for training sets drawn from the same domain (i.e., the learning curves cross), so conclusions about inductionalgorithm superiority on a given domain must be based on an analysis of the learning curves. (3) Contrary to conventional wisdom, tree induction is effective atproducing probabilitybased rankings, although apparently comparatively less so foragiven training{set size than at making classifications. Finally, (4) the domains on which tree induction and logistic regression are ultimately preferable canbecharacterized surprisingly well by a simple measure of signaltonoise ratio.
Methods and uncertainties in bioclimatic envelope modelling under climate change. Prog
 Phys. Geogr
, 2006
"... Abstract: Potential impacts of projected climate change on biodiversity are often assessed using singlespecies bioclimatic ‘envelope ’ models. Such models are a special case of species distribution models in which the current geographical distribution of species is related to climatic variables so ..."
Abstract

Cited by 65 (3 self)
 Add to MetaCart
Abstract: Potential impacts of projected climate change on biodiversity are often assessed using singlespecies bioclimatic ‘envelope ’ models. Such models are a special case of species distribution models in which the current geographical distribution of species is related to climatic variables so to enable projections of distributions under future climate change scenarios. This work reviews a number of critical methodological issues that may lead to uncertainty in predictions from bioclimatic modelling. Particular attention is paid to recent developments of bioclimatic modelling that address some of these issues as well as to the topics where more progress needs to be made. Developing and applying bioclimatic models in a informative way requires good understanding of a wide range of methodologies, including the choice of modelling technique, model validation, collinearity, autocorrelation, biased sampling of explanatory variables, scaling and impacts of nonclimatic factors. A key challenge for future research is integrating factors such as land cover, direct CO2 effects, biotic interactions and dispersal mechanisms into speciesclimate models. We conclude that, although bioclimatic envelope models have a number of important advantages, they need to be applied only when users of models have a thorough understanding of their limitations and uncertainties.
Cost curves: an improved method for visualizing classifier performance
 Machine Learning
, 2006
"... Abstract. This paper introduces cost curves, a graphical technique for visualizing the performance (error rate or expected cost) of 2class classifiers over the full range of possible class distributions and misclassification costs. Cost curves are shown to be superior to ROC curves for visualizing ..."
Abstract

Cited by 63 (7 self)
 Add to MetaCart
(Show Context)
Abstract. This paper introduces cost curves, a graphical technique for visualizing the performance (error rate or expected cost) of 2class classifiers over the full range of possible class distributions and misclassification costs. Cost curves are shown to be superior to ROC curves for visualizing classifier performance for most purposes. This is because they visually support several crucial types of performance assessment that cannot be done easily with ROC curves, such as showing confidence intervals on a classifier’s performance, and visualizing the statistical significance of the difference in performance of two classifiers. A software tool supporting all the cost curve analysis described in this paper is available from the authors.
Benchmarking AnomalyBased Detection Systems
, 2000
"... Anomaly detection is a key element of intrusiondetection and other detection systems in which perturbations of normal behavior suggest the presence of intentionally or unintentionally induced attacks, faults, defects, etc. Because most anomaly detectors are based on probabilistic algorithms that exp ..."
Abstract

Cited by 63 (6 self)
 Add to MetaCart
(Show Context)
Anomaly detection is a key element of intrusiondetection and other detection systems in which perturbations of normal behavior suggest the presence of intentionally or unintentionally induced attacks, faults, defects, etc. Because most anomaly detectors are based on probabilistic algorithms that exploit the intrinsic structure, or regularity, embedded in data logs, a fundamental question is whether or not such structure influences detection performance. If detector performance is indeed a function of environmental regularity, it would be critical to match detectors to environmental characteristics. In intrusiondetection settings, however, this is not done, possibly because such characteristics are not easily ascertained. This paper introduces a metric for characterizing structure in data environments, and tests the hypothesis that intrinsic structure influences probabilistic detection. In a series of experiments, an anomalydetection algorithm was applied to a benchmark suite of 165 c...
ROC analysis of statistical methods used in functional MRI: Individual Subjects. NeuroImage 9
, 1999
"... The complicated structure of fMRI signals and associated noise sources make it difficult to assess the validity of various steps involved in the statistical analysis of brain activation. Most methods used for fMRI analysis assume that observations are independent and that the noise can be treated as ..."
Abstract

Cited by 59 (7 self)
 Add to MetaCart
(Show Context)
The complicated structure of fMRI signals and associated noise sources make it difficult to assess the validity of various steps involved in the statistical analysis of brain activation. Most methods used for fMRI analysis assume that observations are independent and that the noise can be treated as white gaussian noise. These assumptions are usually not true but it is difficult to assess how severely these assumptions are violated and what are their practical consequences. In this study a direct comparison is made between the power of various analytical methods used to detect activations, without reference to estimates of statistical significance. The statistics used in fMRI are treated as metrics designed to detect activations and are not interpreted probabilistically. The receiver operator characteristic (ROC) method is used to compare the efficacy of various steps in calculating an activation map in the study of a single subject based on optimizing the ratio of the number of detected activations to the number of falsepositive findings. The main findings are as follows: Preprocessing. The removal of intensity drifts and highpass filtering applied on the voxel timecourse level is beneficial to the efficacy of analysis. Temporal normalization of the global image intensity, smoothing in the temporal domain, and lowpass filtering do not improve power of analysis. Choices of statistics. the crosscorrelation coefficient and tstatistic, as well as nonparametric Mann–Whitney statistics, prove to be the most effective and are similar in performance, by our criterion. Task design. the proper design of task protocols is shown to be crucial. In an alternating block design the optimal block length is be approximately 18 s. Spatial clustering. an initial spatial smoothing of images is more efficient than cluster filtering of the statistical parametric activation maps. � 1999 Academic Press 1.
Minimum cut model for spoken lecture segmentation
 In Proceedings of the Annual Meeting of the Association for Computational Linguistics (COLINGACL 2006
, 2006
"... We consider the task of unsupervised lecture segmentation. We formalize segmentation as a graphpartitioning task that optimizes the normalized cut criterion. Our approach moves beyond localized comparisons and takes into account longrange cohesion dependencies. Our results demonstrate that global a ..."
Abstract

Cited by 58 (8 self)
 Add to MetaCart
(Show Context)
We consider the task of unsupervised lecture segmentation. We formalize segmentation as a graphpartitioning task that optimizes the normalized cut criterion. Our approach moves beyond localized comparisons and takes into account longrange cohesion dependencies. Our results demonstrate that global analysis improves the segmentation accuracy and is robust in the presence of speech recognition errors. 1
Development of a European multimodel ensemble system for seasonal to interannual prediction
, 2004
"... S easonal timescale climate predictions are nowmade routinely at a number of operationalmeteorological centers around the world, using comprehensive coupled models of the atmosphere, oceans, and land surface (e.g., Stockdale et al. 1998; Mason et al. 1999; Kanamitsu et al. 2002; Alves et al. 2002). ..."
Abstract

Cited by 55 (7 self)
 Add to MetaCart
(Show Context)
S easonal timescale climate predictions are nowmade routinely at a number of operationalmeteorological centers around the world, using comprehensive coupled models of the atmosphere, oceans, and land surface (e.g., Stockdale et al. 1998; Mason et al. 1999; Kanamitsu et al. 2002; Alves et al. 2002). This development can be traced back to a revolution in our understanding of the coupled ocean– atmosphere system in the second half of the twentieth century (Neelin et al. 1998), to the development and deployment of specialized buoys to observe and measure the evolution of nearsurface waters in the