Results 1  10
of
11
Lookahead and Pathology in Decision Tree Induction
 Proceedings of the 14th International Joint Conference on Artificial Intelligence
, 1995
"... The standard approach to decision tree induction is a topdown, greedy algorithm that makes locally optimal, irrevocable decisions at each node of a tree. In this paper, we study an alternative approach, in which the algorithms use limited lookahead to decide what test to use at a node. We systemati ..."
Abstract

Cited by 54 (2 self)
 Add to MetaCart
(Show Context)
The standard approach to decision tree induction is a topdown, greedy algorithm that makes locally optimal, irrevocable decisions at each node of a tree. In this paper, we study an alternative approach, in which the algorithms use limited lookahead to decide what test to use at a node. We systematically compare, using a very large number of decision trees, the quality of decision trees induced by the greedy approach to that of trees induced using lookahead. The main results of our experiments are: (i) the greedy approach produces trees that are just as accurate as trees produced with the much more expensive lookahead step; and (ii) decision tree induction exhibits pathology, in the sense that lookahead can produce trees that are both larger and less accurate than trees produced without it. 1. Introduction The standard algorithm for constructing decision trees from a set of examples is greedy induction  a tree is induced topdown with locally optimal choices made at each node, with...
Lookaheadbased Algorithms for Anytime Induction of Decision Trees
 In ICML’04
, 2004
"... The majority of the existing algorithms for learning decision trees are greedya tree is induced topdown, making locally optimal decisions at each node. In most cases, however, the constructed tree is not globally optimal. Furthermore, the greedy algorithms require a fixed amount of time and are no ..."
Abstract

Cited by 19 (4 self)
 Add to MetaCart
(Show Context)
The majority of the existing algorithms for learning decision trees are greedya tree is induced topdown, making locally optimal decisions at each node. In most cases, however, the constructed tree is not globally optimal. Furthermore, the greedy algorithms require a fixed amount of time and are not able to generate a better tree if additional time is available. To overcome this problem, we present two lookaheadbased algorithms for anytime induction of decision trees, thus allowing tradeoff between tree quality and learning time. The first one is depthk lookahead, where a larger time allocation permits larger k. The second algorithm uses a novel strategy for evaluating candidate splits; a stochastic version of ID3 is repeatedly invoked to estimate the size of the tree in which each split results, and the one that minimizes the expected size is preferred. Experimental results indicate that for several hard concepts, our proposed approach exhibits good anytime behavior and yields significantly better decision trees when more time is available.
An approximate dynamic programming approach to multidimensional knapsack problems
 Management Sci
, 2002
"... We present an Approximate Dynamic Programming (ADP)approach for the multidimensional knapsack problem (MKP). We approximate the value function (a) using parametric and nonparametric methods and (b)using a baseheuristic. We propose a new heuristic which adaptively rounds the solution of the linear p ..."
Abstract

Cited by 19 (0 self)
 Add to MetaCart
We present an Approximate Dynamic Programming (ADP)approach for the multidimensional knapsack problem (MKP). We approximate the value function (a) using parametric and nonparametric methods and (b)using a baseheuristic. We propose a new heuristic which adaptively rounds the solution of the linear programming relaxation. Our computational study suggests: (a)the new heuristic produces high quality solutions fast and robustly, (b)state of the art commercial packages like CPLEX require significantly larger computational time to achieve the same quality of solutions, (c)the ADP approach using the new heuristic competes successfully with alternative heuristic methods such as genetic algorithms, (d)the ADP approach based on parametric and nonparametric approximations, while producing reasonable solutions, is not competitive. Overall, this research illustrates that the baseheuristic approach is a promising computational approach for MKPs worthy of further investigation. 1.
Anytime learning of decision trees
 Journal of Machine Learning Research
"... The majority of existing algorithms for learning decision trees are greedy—a tree is induced topdown, making locally optimal decisions at each node. In most cases, however, the constructed tree is not globally optimal. Even the few nongreedy learners cannot learn good trees when the concept is diff ..."
Abstract

Cited by 13 (3 self)
 Add to MetaCart
(Show Context)
The majority of existing algorithms for learning decision trees are greedy—a tree is induced topdown, making locally optimal decisions at each node. In most cases, however, the constructed tree is not globally optimal. Even the few nongreedy learners cannot learn good trees when the concept is difficult. Furthermore, they require a fixed amount of time and are not able to generate a better tree if additional time is available. We introduce a framework for anytime induction of decision trees that overcomes these problems by trading computation speed for better tree quality. Our proposed family of algorithms employs a novel strategy for evaluating candidate splits. A biased sampling of the space of consistent trees rooted at an attribute is used to estimate the size of the minimal tree under that attribute, and an attribute with the smallest expected tree is selected. We present two types of anytime induction algorithms: a contract algorithm that determines the sample size on the basis of a pregiven allocation of time, and an interruptible algorithm that starts with a greedy tree and continuously improves subtrees by additional sampling. Experimental results indicate that, for several hard concepts, our proposed approach exhibits good anytime behavior and yields significantly better decision trees when more time is available.
Decision Trees For Classification: A Review And Some New Results
"... Introduction Topdown induction of decision trees is a simple and powerful method of inferring classication rules from a set of labeled examples 1 . Each node of the tree implements a decision rule that splits the examples into two or more partitions. New nodes are created to handle each of the p ..."
Abstract

Cited by 4 (0 self)
 Add to MetaCart
Introduction Topdown induction of decision trees is a simple and powerful method of inferring classication rules from a set of labeled examples 1 . Each node of the tree implements a decision rule that splits the examples into two or more partitions. New nodes are created to handle each of the partitions and a node is considered terminal or a leaf node based on a stopping criteria. This standard approach to decision tree construction thus corresponds to a topdown greedy algorithm that makes locally optimal decisions at each node. There are two advantages that decision trees have over many other methods of classication methods. The rst is that the sequence of decisions made from the root node to the eventual labeling of a test input is easy to follow. This gives them an intuitive appeal that other methods of classication such as
Electric distribution network multiobjective design using a problemspecific genetic algorithm
 IEEE Trans. Power Del
, 2006
"... ..."
(Show Context)
Utilitybased Channel Assignment and Topology Control in Wireless Mesh Networks
"... Abstract—We define a utilitybased framework for joint channel assignment and topology control in multirate multiradio wireless mesh networks, and present a greedy algorithm for solving the corresponding optimization problem. Key features of the proposed approach are the support for different targ ..."
Abstract

Cited by 2 (1 self)
 Add to MetaCart
(Show Context)
Abstract—We define a utilitybased framework for joint channel assignment and topology control in multirate multiradio wireless mesh networks, and present a greedy algorithm for solving the corresponding optimization problem. Key features of the proposed approach are the support for different target objectives, which are expressed as utility functions of the MAC layer throughput, and the efficient utilization of wired network gateways, while guaranteeing that for every mesh node there exists a path to a gateway. Investigations show the influence of different target objectives on the channel assignment and network topology, demonstrate the proposed approach’s load balancing properties in mesh networks containing multiple gateways, show the effect of 802.11a adjacent channel interference on the channel assignment, and compare the performance of the proposed procedure with a ratebased channel assignment scheme. I.
Application of a Modelfree Algorithm for the Packing of Irregular Shaped Objects in Semiconductor Manufacture
"... A Robotic System is being developed to automate the crucible packing process in the CZ semiconductor wafer production. It requires the delicate manipulation and packing of highly irregular shaped polycrystalline silicon nuggets, into a fragile glass crucible. Here an online algorithm is presented t ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
A Robotic System is being developed to automate the crucible packing process in the CZ semiconductor wafer production. It requires the delicate manipulation and packing of highly irregular shaped polycrystalline silicon nuggets, into a fragile glass crucible. Here an online algorithm is presented to plan the packing. It uses a method called Virtual Trial and Error. The online algorithm handles large numbers of highly irregularly shaped object of different sizes without requiring the object models. Working with the 3D range maps of objects, it is computationally fast enough to be applied in realtime to practical industrial applications, such as the CZ wafer manufacture. Simulation results show that it compares well with the human performance. The integrated system is shown to achieve high production rates, required precision and cost effectiveness. 1 Introduction During the widely used CZ semiconductor production process, highly irregular shaped polycrystalline silicon nuggets are...
A Modelfree Algorithm for the Packing of Highly Irregular Shaped Objects: with Application to CZ Semiconductor Manufacture
"... A Robotic System is being developed to automate the crucible packing in the CZ semiconductor wafer production. It requires the delicate manipulation and packing of highly irregular shaped polycrystalline silicon nuggets, into a fragile glass crucible. Here an online algorithm is presented to plan t ..."
Abstract
 Add to MetaCart
A Robotic System is being developed to automate the crucible packing in the CZ semiconductor wafer production. It requires the delicate manipulation and packing of highly irregular shaped polycrystalline silicon nuggets, into a fragile glass crucible. Here an online algorithm is presented to plan the packing. It uses a method called Virtual Trial and Error. The online algorithm handles large numbers of highly irregularly shaped object of different sizes without requiring the object models. Working with raw vision data it is computationally fast enough to be applied in realtime to practical industrial applications, such as the CZ wafer manufacture. Simulation results show that it will compare well with the human performance. 1. Introduction During the widely used CZ semiconductor production process, highly irregular shaped polycrystalline silicon nuggets are packed into a large quartz crucible, see Figure 1 [7]. Each highly irregularly shaped nugget is unique, with weights ranging ...
Investigations of the Greedy Heuristic for Classification Tree Induction
"... Most existing methods for automatic construction of classification trees utilize the greedy heuristic: trees are constructed one node at a time with no looking ahead or backtracking, choosing locally optimal splits to divide the data. This heuristic is commonly believed to work well in practice and ..."
Abstract
 Add to MetaCart
(Show Context)
Most existing methods for automatic construction of classification trees utilize the greedy heuristic: trees are constructed one node at a time with no looking ahead or backtracking, choosing locally optimal splits to divide the data. This heuristic is commonly believed to work well in practice and is widely used, even though it is known to produce necessarily suboptimal trees. In this paper, we attempt to systematically study the effectiveness of the greedy heuristic. In a largescale experimental study, we compare greedily induced trees with the corresponding optimal trees while varying several control variables. We also contrast the greedy heuristic with a seemingly superior alternative: limited lookahead search. We design tens of thousands of synthetic data sets for our experiments and build trees using two popular goodness measures. We also use realworld data. Two main observations from the experiments are: (1) the greedy heuristic produces trees that are consistently close to the...