Results 1  10
of
14
Automatic Construction of Decision Trees from Data: A MultiDisciplinary Survey
 Data Mining and Knowledge Discovery
, 1997
"... Decision trees have proved to be valuable tools for the description, classification and generalization of data. Work on constructing decision trees from data exists in multiple disciplines such as statistics, pattern recognition, decision theory, signal processing, machine learning and artificial ne ..."
Abstract

Cited by 164 (1 self)
 Add to MetaCart
(Show Context)
Decision trees have proved to be valuable tools for the description, classification and generalization of data. Work on constructing decision trees from data exists in multiple disciplines such as statistics, pattern recognition, decision theory, signal processing, machine learning and artificial neural networks. Researchers in these disciplines, sometimes working on quite different problems, identified similar issues and heuristics for decision tree construction. This paper surveys existing work on decision tree construction, attempting to identify the important issues involved, directions the work has taken and the current state of the art. Keywords: classification, treestructured classifiers, data compaction 1. Introduction Advances in data collection methods, storage and processing technology are providing a unique challenge and opportunity for automated data exploration techniques. Enormous amounts of data are being collected daily from major scientific projects e.g., Human Genome...
Lookahead and Pathology in Decision Tree Induction
 Proceedings of the 14th International Joint Conference on Artificial Intelligence
, 1995
"... The standard approach to decision tree induction is a topdown, greedy algorithm that makes locally optimal, irrevocable decisions at each node of a tree. In this paper, we study an alternative approach, in which the algorithms use limited lookahead to decide what test to use at a node. We systemati ..."
Abstract

Cited by 51 (2 self)
 Add to MetaCart
(Show Context)
The standard approach to decision tree induction is a topdown, greedy algorithm that makes locally optimal, irrevocable decisions at each node of a tree. In this paper, we study an alternative approach, in which the algorithms use limited lookahead to decide what test to use at a node. We systematically compare, using a very large number of decision trees, the quality of decision trees induced by the greedy approach to that of trees induced using lookahead. The main results of our experiments are: (i) the greedy approach produces trees that are just as accurate as trees produced with the much more expensive lookahead step; and (ii) decision tree induction exhibits pathology, in the sense that lookahead can produce trees that are both larger and less accurate than trees produced without it. 1. Introduction The standard algorithm for constructing decision trees from a set of examples is greedy induction  a tree is induced topdown with locally optimal choices made at each node, with...
Minimaxing: Theory and practice
 AI Magazine
, 1988
"... Empirical evidence suggests that searching deeper in game trees using the minimax propagation rule usually improves the quality of decisions significantly. However, despite many recent theoretical analyses of the effects of minimax lookahead, however, this phenomenon has still not been convincingly ..."
Abstract

Cited by 4 (0 self)
 Add to MetaCart
(Show Context)
Empirical evidence suggests that searching deeper in game trees using the minimax propagation rule usually improves the quality of decisions significantly. However, despite many recent theoretical analyses of the effects of minimax lookahead, however, this phenomenon has still not been convincingly explained. Instead, much attention has been given to socalled pathological behavior, which occurs under certain assumptions. This article supports the view that pathology is a direct result of these underlying theoretical assumptions. Pathology does not occur in practice, because these assumptions do
A Theory of Game Trees
 Proceedings of the National Conference on Artificial Intelligence
, 1983
"... A theory of heuristic game tree search and evaluation functions for estimating minimax values is developed. The result is quite different from the traditional minimsx approach to game playing, and it leads to productpropagation rules for backing up values when subpositions in the game are indepen ..."
Abstract

Cited by 3 (3 self)
 Add to MetaCart
A theory of heuristic game tree search and evaluation functions for estimating minimax values is developed. The result is quite different from the traditional minimsx approach to game playing, and it leads to productpropagation rules for backing up values when subpositions in the game are independent. In this theory Nau’s paradox is avoided and deeper searching leads to better moves if one has reasonable evaluation functions. I
The Effect of Mobility on Minimaxing of Game Trees with Random Leaf Values
"... Random minimaxing, introduced by Beal and Smith [3], is the process of using a random static evaluation function for scoring the leaf nodes of a full width game tree and then computing the best move using the standard minimax procedure. The experiments carried out by Beal and Smith, using random min ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
Random minimaxing, introduced by Beal and Smith [3], is the process of using a random static evaluation function for scoring the leaf nodes of a full width game tree and then computing the best move using the standard minimax procedure. The experiments carried out by Beal and Smith, using random minimaxing in Chess, showed that the strength of play increases as the depth of the lookahead is increased. We investigate random minimaxing from a combinatorial point of view in an attempt to gain a better understanding of the utility of the minimax procedure and a theoretical justification for the results of Beal and Smith's experiments. The concept of domination is central to our theory. Intuitively, one move by white dominates another move when choosing the former move would give less choice for black when it is black's turn to move, and subsequently more choice for white when it is white's turn to move. We view domination as a measure of mobility and show that when one move dominates another then its probability of being chosen is higher. We then investigate when the probability of a "good" move relative to the probability of a "bad" move" increases with the depth of search. We show that there exist situations when increased depth of search is "beneficial" but that this is not always the case. Under the assumption that each move is either "good" or "bad", we are able to state su#cient conditions to ensure that increasing the depth of search increases the strength of play of random minimaxing. If the semantics of the game under consideration match these assumptions then it is fair to say that random minimaxing appears to follow a reasonably "intelligent" strategy. In practice domination does not always occur, so it remains an open problem to find a more general measure of mo...
Towards an Understanding of Minimaxing of Game Trees with Random Leaf Values
"... Random minimaxing, introduced by Beal and Smith [1], is the process of using a random static evaluation function for scoring the leaf nodes of a full width game tree and then computing the best move using the standard minimax procedure. The experiments carried out by Beal and Smith, using random min ..."
Abstract
 Add to MetaCart
(Show Context)
Random minimaxing, introduced by Beal and Smith [1], is the process of using a random static evaluation function for scoring the leaf nodes of a full width game tree and then computing the best move using the standard minimax procedure. The experiments carried out by Beal and Smith, using random minimaxing in Chess, showed that the strength of play increases as the depth of the lookahead is increased. We investigate random minimaxing from a combinatorial point of view in order to gain a better understanding of the utility of the minimax procedure and thus obtain a theoretical justification for the results of Beal and Smith's experiments. The concept of domination is central to our theory. Intuitively, a move by white dominates another move when this move gives less choice for black when it is black's turn to move, and subsequently more choice for white when it is white's turn to move. We view domination as a measure of mobility and show that when one move dominates another then its pro...
Investigations of the Greedy Heuristic for Classification Tree Induction
"... Most existing methods for automatic construction of classification trees utilize the greedy heuristic: trees are constructed one node at a time with no looking ahead or backtracking, choosing locally optimal splits to divide the data. This heuristic is commonly believed to work well in practice and ..."
Abstract
 Add to MetaCart
(Show Context)
Most existing methods for automatic construction of classification trees utilize the greedy heuristic: trees are constructed one node at a time with no looking ahead or backtracking, choosing locally optimal splits to divide the data. This heuristic is commonly believed to work well in practice and is widely used, even though it is known to produce necessarily suboptimal trees. In this paper, we attempt to systematically study the effectiveness of the greedy heuristic. In a largescale experimental study, we compare greedily induced trees with the corresponding optimal trees while varying several control variables. We also contrast the greedy heuristic with a seemingly superior alternative: limited lookahead search. We design tens of thousands of synthetic data sets for our experiments and build trees using two popular goodness measures. We also use realworld data. Two main observations from the experiments are: (1) the greedy heuristic produces trees that are consistently close to the...
Is RealValued Minimax Pathological?
"... Deeper searches in gameplaying programs relying on the minimax principle generally produce better results. Theoretical analyses, however, suggest that in many cases minimaxing amplifies the noise introduced by the heuristic function used to evaluate the leaves of the game tree, leading to what is k ..."
Abstract
 Add to MetaCart
Deeper searches in gameplaying programs relying on the minimax principle generally produce better results. Theoretical analyses, however, suggest that in many cases minimaxing amplifies the noise introduced by the heuristic function used to evaluate the leaves of the game tree, leading to what is known as pathological behavior, where deeper searches produce worse results. In most minimax models analyzed in previous research, positions ’ true values and sometimes also heuristic values were only losses and wins. In contrast to this, a model is proposed in this paper that uses real numbers for both true and heuristic values. This model did not behave pathologically in the experiments performed. The mechanism that causes deeper searches to produce better evaluations is explained. A comparison with chess is made, indicating that the model realistically reflects position evaluations in chessplaying programs. Conditions under which the pathology might appear in a realvalue model are also examined. The essential difference between our realvalue model and the common twovalue model, which causes the pathology in the twovalue model, is identified. Most previous research reports that the pathology tends to disappear when there are dependences between the values of sibling nodes in a game tree. In this paper, another explanation is presented which indicates that in the twovalue models the error of the heuristic evaluation was not modeled realistically.