Abstract

We analyse the biases of eleven measures for estimating the quality of the multi-valued attributes. The values of information gain, J- measure, gini-index, and relevance tend to linearly increase with the number of values of an attribute. The values of gain-ratio, distance measure, Relief , and the weight of evidence decrease for informative attributes and increase for irrelevant attributes. The bias of the statistic tests based on the chi-square distribution is similar but these functions are not able to discriminate among the attributes of different quality. We also introduce a new function based on the MDL principle whose value slightly decreases with the increasing number of attribute's values. 1 Introduction In top down induction of decision trees various impurity functions are used to estimate the quality of attributes in order to select the "best" one to split on. However, various heuristics tend to overestimate the multi-valued attributes. One possible approach to this proble...

Citations

3143	Classification and Regression Trees - Breiman, Friedman, et al. - 1984
2888	Induction of Decision Trees - Quinlan - 1986
1466	Numerical Recipes in C. The Art of Scientific Computing - PRESS, FLANNERY, et al. - 1988
1412	An Introduction to Kolmogorov Complexity and Its Applications - Li, Vitanyi - 1997
277	LA: A practical approach to feature selection - Kira, Rendell - 1992
231	Estimating attributes: analysis and extensions of RELIEF - Kononenko - 1994
72	Rule Induction Using Information Theory - Smyth, Goodman - 1991
31	Bias in information-based measures in decision tree induction - White, Liu - 1994
24	A method for attribute selection in inductive learning systems - Baim - 1988
14	ID3 revisited: A distance based criterion for attribute selection - Mantaras - 1989
2	A universal prior for integers and estimation by minimum description length - J - 1983
1	Personal Models of Rationality. J. of Statistical Planning and - Michie - 1989
1	A method for attribute selection in inductive learning systems - Bairn - 1988
1	H Friedman R A Olshen and - J - 1984
1	and P Vilanyi An introduction to holmogoroi Complexity and its applications - Li - 1993
1	Personal Models of Rationalit} J of Statistical Planning and - Michie - 1989
1	A Teukolsky V\ T \ettenug B P Flannerv Numerical recipes m C The art of scientific computing - Press, S - 1988
1	Rissanen J A universal prior for integers and estimation by minimum description length Annals of Statistics - R - 1983