Results 1  10
of
23
A System for Induction of Oblique Decision Trees
 Journal of Artificial Intelligence Research
, 1994
"... This article describes a new system for induction of oblique decision trees. This system, OC1, combines deterministic hillclimbing with two forms of randomization to find a good oblique split (in the form of a hyperplane) at each node of a decision tree. Oblique decision tree methods are tuned espe ..."
Abstract

Cited by 251 (13 self)
 Add to MetaCart
This article describes a new system for induction of oblique decision trees. This system, OC1, combines deterministic hillclimbing with two forms of randomization to find a good oblique split (in the form of a hyperplane) at each node of a decision tree. Oblique decision tree methods are tuned especially for domains in which the attributes are numeric, although they can be adapted to symbolic or mixed symbolic/numeric attributes. We present extensive empirical studies, using both real and artificial data, that analyze OC1's ability to construct oblique trees that are smaller and more accurate than their axisparallel counterparts. We also examine the benefits of randomization for the construction of oblique decision trees. 1. Introduction Current data collection technology provides a unique challenge and opportunity for automated machine learning techniques. The advent of major scientific projects such as the Human Genome Project, the Hubble Space Telescope, and the human brain mappi...
Automatic Construction of Decision Trees from Data: A MultiDisciplinary Survey
 Data Mining and Knowledge Discovery
, 1997
"... Decision trees have proved to be valuable tools for the description, classification and generalization of data. Work on constructing decision trees from data exists in multiple disciplines such as statistics, pattern recognition, decision theory, signal processing, machine learning and artificial ne ..."
Abstract

Cited by 146 (1 self)
 Add to MetaCart
Decision trees have proved to be valuable tools for the description, classification and generalization of data. Work on constructing decision trees from data exists in multiple disciplines such as statistics, pattern recognition, decision theory, signal processing, machine learning and artificial neural networks. Researchers in these disciplines, sometimes working on quite different problems, identified similar issues and heuristics for decision tree construction. This paper surveys existing work on decision tree construction, attempting to identify the important issues involved, directions the work has taken and the current state of the art. Keywords: classification, treestructured classifiers, data compaction 1. Introduction Advances in data collection methods, storage and processing technology are providing a unique challenge and opportunity for automated data exploration techniques. Enormous amounts of data are being collected daily from major scientific projects e.g., Human Genome...
Hybrid neural systems: from simple coupling to fully integrated neural networks
 Neural Computing Surveys
, 1999
"... This paper describes techniques for integrating neural networks and symbolic components into powerful hybrid systems. Neural networks have unique processing characteristics that enable tasks to be performed that would be di cult or intractable for a symbolic rulebased system. However, a standalone ..."
Abstract

Cited by 29 (7 self)
 Add to MetaCart
This paper describes techniques for integrating neural networks and symbolic components into powerful hybrid systems. Neural networks have unique processing characteristics that enable tasks to be performed that would be di cult or intractable for a symbolic rulebased system. However, a standalone neural network requires an interpretation either by ahuman or a rulebased system. This motivates the integration of neural/symbolic techniques within a hybrid system. Anumber of integration possibilities exist: some systems consist of neural network components performing symbolic tasks while other systems are composed of several neural networks and symbolic components, each component acting as a selfcontained module communicating with the others. Other hybrid systems are able to transform subsymbolic representations into symbolic ones and viceversa. This paper providesanoverview and evaluation of the state of the artofseveral hybrid neural systems for rulebased processing. 1
Transferring Previously Learned BackPropagation Neural Networks To New Learning Tasks
, 1993
"... ..."
How to Make Best Use of Evolutionary Learning
 in Complex Systems: From Local Interactions to Global Phenomena
, 1996
"... Evolutionary learning has been developing rapidly in the last decade. It is a powerful and general learning approach which has been used successfully in both symbolic systems, e.g., rulebased systems, and subsymbolic systems, e.g., artificial neural networks. However, most evolutionary learning sys ..."
Abstract

Cited by 22 (13 self)
 Add to MetaCart
Evolutionary learning has been developing rapidly in the last decade. It is a powerful and general learning approach which has been used successfully in both symbolic systems, e.g., rulebased systems, and subsymbolic systems, e.g., artificial neural networks. However, most evolutionary learning systems have paid little attention to the fact that they are populationbased learning. The common practice is to select the best individual in the last generation as the final learned system. Such practice in essence treats these learning systems as optimisation ones. This paper emphasises the difference between a learning system and an optimisation one, and shows that such difference requires a different approach to populationbased learning and that the current practice of selecting the best individual as the learned system is not the best choice. The paper then argues that a population contains more information than the best individual and thus should be used as the final learned system. Tw...
Decision Trees Can Initialize RadialBasis Function Networks
, 1998
"... Successful implementations of radialbasis function (RBF) networks for classification tasks must deal with architectural issues, the burden of irrelevant attributes, scaling, and some other problems. This paper addresses these issues by initializing RBF networks with decision trees that define relat ..."
Abstract

Cited by 20 (1 self)
 Add to MetaCart
Successful implementations of radialbasis function (RBF) networks for classification tasks must deal with architectural issues, the burden of irrelevant attributes, scaling, and some other problems. This paper addresses these issues by initializing RBF networks with decision trees that define relatively pure regions in the instance space; each of these regions then determines one basis function. The resulting network is compact, easy to induce, and has favorable classification accuracy. Keywords Pattern recognition, neural networks, radialbasis functions, decision trees I. Introduction A system that learns to recognize concepts accepts pairs [x; c(x)], where x = [x 1 ; x 2 ; : : : ; xn ] is a vector describing an example, and c(x) is a concept label. The variables x i are referred to as attributes. The space occupied by all possible examples that can be described by the given set of attributes is called the instance space. A concept is a binary function, c : R n ! f\Gamma1; 1g...
Omnivariate Decision Trees
"... Univariate decision trees at each decision node consider the value of only one feature leading to axisaligned splits. In a linear multivariate decision tree, each decision node divides the input space into two with a hyperplane. In a nonlinear multivariate tree, a multilayer perceptron at each node ..."
Abstract

Cited by 14 (8 self)
 Add to MetaCart
Univariate decision trees at each decision node consider the value of only one feature leading to axisaligned splits. In a linear multivariate decision tree, each decision node divides the input space into two with a hyperplane. In a nonlinear multivariate tree, a multilayer perceptron at each node divides the input space arbitrarily, at the expense of increased complexity and higher risk of overfitting. We propose omnivariate trees where the decision node may be univariate, linear, or nonlinear depending on the outcome of comparative statistical tests on accuracy thus matching automatically the complexity of the node with the subproblem defined by the data reaching that node. Such an architecture frees the designer from choosing the appropriate node type, doing model selection automatically at each node. Our simulation results indicate that such a decision tree induction method generalizes better than trees with the same types of nodes everywhere and induces small trees.
Realtime learning capability of neural networks
 IEEE Trans. Neural Networks
, 2006
"... Abstract—In some practical applications of neural networks, fast response to external events within an extremely short time is highly demanded and expected. However, the extensively used gradientdescentbased learning algorithms obviously cannot satisfy the realtime learning needs in many applicat ..."
Abstract

Cited by 14 (8 self)
 Add to MetaCart
Abstract—In some practical applications of neural networks, fast response to external events within an extremely short time is highly demanded and expected. However, the extensively used gradientdescentbased learning algorithms obviously cannot satisfy the realtime learning needs in many applications, especially for largescale applications and/or when higher generalization performance is required. Based on Huang’s constructive network model, this paper proposes a simple learning algorithm capable of realtime learning which can automatically select appropriate values of neural quantizers and analytically determine the parameters (weights and bias) of the network at one time only. The performance of the proposed algorithm has been systematically investigated on a large batch of benchmark realworld regression and classification problems. The experimental results demonstrate that our algorithm can not only produce good generalization performance but also have realtime learning and prediction capability. Thus, it may provide an alternative approach for the practical applications of neural networks where realtime learning and prediction implementation is required. Index Terms—Backpropagation (BP), extreme learning machine, feedforward networks, generalization performance,NN, realtime learning, realtime prediction. I.
Constructive Learning Techniques for Designing Neural Network Systems
, 1997
"... Contents 1. Introduction. 2. Classification. 2.1 Introduction. 2.2 The Pocket algorithm. 2.3 Tower and Cascade architectures. 2.4 Tree architectures: the Upstart algorithm. 2.5 Constructing tree and cascade architectures using dichotomies. 2.6 Constructing neural networks with a single hidden layer. ..."
Abstract

Cited by 13 (0 self)
 Add to MetaCart
Contents 1. Introduction. 2. Classification. 2.1 Introduction. 2.2 The Pocket algorithm. 2.3 Tower and Cascade architectures. 2.4 Tree architectures: the Upstart algorithm. 2.5 Constructing tree and cascade architectures using dichotomies. 2.6 Constructing neural networks with a single hidden layer. 2.7 Summary. 3. Regression. 3.1 Introduction. 3.2 The Cascade Correlation Algorithm. 3.3 Node creation and node splitting algorithms. 3.4 Constructing RBF networks. 3.5 Summary. 4. Constructing Modular Architectures. 4.1 Introduction. 4.2 Neural Decision Trees. 4.3 Other approaches to constructing modular networks. 5. Reducing Network Complexity. 5.1 Introduction. 5.2 Pruning Procedures. 5.3 Summary. 6. Conclusion. 7. Appendix: algorithms for singlenode learning. 1 1 Introduction Neural networks have been applied to a wide range of application domains such as control, telecommun
Learning NonLinearly Separable Boolean Functions With Linear Threshold Unit Trees and MadalineStyle Networks
 In AAAI93 [2
"... This paper investigates an algorithm for the construction of decisions trees comprised of linear threshold units and also presents a novel algorithm for the learning of nonlinearly separable boolean functions using Madalinestyle networks which are isomorphic to decision trees. The construction of su ..."
Abstract

Cited by 8 (1 self)
 Add to MetaCart
This paper investigates an algorithm for the construction of decisions trees comprised of linear threshold units and also presents a novel algorithm for the learning of nonlinearly separable boolean functions using Madalinestyle networks which are isomorphic to decision trees. The construction of such networks is discussed, and their performance in learning is compared with standard BackPropagation on a sample problem in which many irrelevant attributes are introduced. Littlestone's Winnow algorithm is also explored within this architecture as a means of learning in the presence of many irrelevant attributes. The learning ability of this Madalinestyle architecture on nonoptimal (larger than necessary) networks is also explored. Introduction We initially examine a nonincremental algorithm that learns binary classification tasks by producing decision trees of linear threshold units (LTU trees). This decision tree bears some similarity to the decision trees produced by ID3 (Quinlan 19...