## Efficient AUC Learning Curve Calculation

Citations: | 1 - 0 self |

### BibTeX

@MISC{Bouckaert_efficientauc,

author = {Remco R. Bouckaert},

title = {Efficient AUC Learning Curve Calculation},

year = {}

}

### OpenURL

### Abstract

Abstract. A learning curve of a performance measure provides a graphical method with many benefits for judging classifier properties. The area under the ROC curve (AUC) is a useful and increasingly popular performance measure. In this paper, we consider the computational aspects of calculating AUC learning curves. A new method is provided for incrementally updating exact AUC curves and for calculating approximate AUC curves for datasets with millions of instances. Both theoretical and empirical justifications are given for the approximation. Variants for incremental exact and approximate AUC curves are provided as well. 1

### Citations

3170 | Data Mining: Practical machine learning tools and techniques, 2nd edn - Witten, Frank - 2005 |

299 |
Measuring the accuracy of diagnostic systems
- Swets
- 1988
(Show Context)
Citation Context ...erator characteristics (ROC) graphs allow for easily visualizing performance of classifiers. ROC graphs stem from electronics [3], but has been widely adopted by the medical decision making community =-=[2, 8]-=-. Recently, the machine learning community has taken an interest as well [4, 5] 1 . Probabilistic classifiers, such as naive Bayes, are often selected based on accuracy, where the class with the highe... |

237 |
Signal detection theory and ROC analysis
- Egan
- 1975
(Show Context)
Citation Context ...nd approximate AUC curves are provided as well. 1 Introduction Receiver operator characteristics (ROC) graphs allow for easily visualizing performance of classifiers. ROC graphs stem from electronics =-=[3]-=-, but has been widely adopted by the medical decision making community [2, 8]. Recently, the machine learning community has taken an interest as well [4, 5] 1 . Probabilistic classifiers, such as naiv... |

159 | ROC Graphs: Notes and Practical Considerations for Data Mining Researchers,” HP Laboratories technical report
- Fawcett
- 2003
(Show Context)
Citation Context ... classifiers. ROC graphs stem from electronics [3], but has been widely adopted by the medical decision making community [2, 8]. Recently, the machine learning community has taken an interest as well =-=[4, 5]-=- 1 . Probabilistic classifiers, such as naive Bayes, are often selected based on accuracy, where the class with the highest probability is taken to be the class prediction. However, classification per... |

64 | The geometry of ROC space: understanding machine learning metrics through ROC isometrics
- Flach
- 2003
(Show Context)
Citation Context ... classifiers. ROC graphs stem from electronics [3], but has been widely adopted by the medical decision making community [2, 8]. Recently, the machine learning community has taken an interest as well =-=[4, 5]-=- 1 . Probabilistic classifiers, such as naive Bayes, are often selected based on accuracy, where the class with the highest probability is taken to be the class prediction. However, classification per... |

47 | Improving accuracy and cost of two-class and multi-class probabilistic classifiers using ROC curves
- Lachiche, Flach
- 2003
(Show Context)
Citation Context ...uracy, where the class with the highest probability is taken to be the class prediction. However, classification performance can be increased by optimally choosing the threshold for selecting a class =-=[6]-=-, so that for instance a binary class the positive class is selected when the classifier predicts at least a 0.6 probability for this class, and otherwise selects the negative class. One of the benefi... |

10 |
The use of ROC curves in test performance evaluation
- Beck, Shultz
- 1986
(Show Context)
Citation Context ...erator characteristics (ROC) graphs allow for easily visualizing performance of classifiers. ROC graphs stem from electronics [3], but has been widely adopted by the medical decision making community =-=[2, 8]-=-. Recently, the machine learning community has taken an interest as well [4, 5] 1 . Probabilistic classifiers, such as naive Bayes, are often selected based on accuracy, where the class with the highe... |

3 |
Saharon Rosset. Pointwise ROC confidence bounds: An empirical evaluation
- Macskassy, Provost
- 2005
(Show Context)
Citation Context ...r research is the question how to efficiently combine a number of ROC curves in order to create a single ROC curve from various curves obtained from a cross validation run on the data as addressed in =-=[7]-=-. Acknowledgements I thank all Waikato University Machine Learning group members for his stimulating discussions on this topic, especially Richard Kirkby for making his Moa system for experimenting on... |

1 | Resampling Methods for the Area Under the ROC Curve
- Bandos, Rockette, et al.
(Show Context)
Citation Context ... from the model P (y, x1, x2, x3) described above. The exact AUC was calculated as described in Section 2. The top line shows the AUC estimate based on a sample from the outcomes drawn from the model =-=[1]-=-. Though this approach does not require visiting all outcomes, it does require sorting. The bottom line is the average error obtained with the bin based approach. The maximum error found is indicated ... |