Results 1 
6 of
6
Inductive Learning of Treebased Regression Models
, 1999
"... This thesis explores different aspects of the induction of treebased regression models from data. The main goal of this study is to improve the predictive accuracy of regression trees, while retaining as much as possible their comprehensibility and computational efficiency. Our study is divided in ..."
Abstract

Cited by 17 (2 self)
 Add to MetaCart
This thesis explores different aspects of the induction of treebased regression models from data. The main goal of this study is to improve the predictive accuracy of regression trees, while retaining as much as possible their comprehensibility and computational efficiency. Our study is divided in three main parts. In the first part we describe in detail two different methods of growing a regression tree: minimising the mean squared error and minimising the mean absolute deviation. Our study is particularly focussed on the computational efficiency of these tasks. We present several new algorithms that lead to significant computational speed ups. We also describe an experimental comparison of both methods of growing a regression tree highlighting their different application goals. Pruning is a standard procedure within treebased models whose goal is to provide a good compromise for achieving simple and comprehensible models with good predictive accuracy. In the second part of our study we describe a series of new techniques for pruning by selection from a series of alternative pruned trees. We carry out an extensive set of experiments comparing different methods of pruning, which show that our proposed techniques are able to significantly outperform the predictive accuracy of current state of the art pruning algorithms in a large set of regression domains. In the final part of our study we present a new type of treebased models that we refer to as local regression trees. These hybrid models integrate treebased regression with local modelling techniques. We describe different types of local regression trees and show that these models are able to significantly outperform standard regression trees in terms of predictive accuracy. Through a large set of experiments we prove the competitiveness of local regression trees when compared to existing regression techniques.
Data Fitting with RuleBased Regression
 In Proceedings of the 2nd international workshop on Artificial Intelligence Techniques (AIT'95
, 1995
"... . In the classical regression theory we try to build one functional model to fit a set of data. In noisy and complex domains this methodology can be highly unreliable and/or demand too complex functional models. Piecewise regression models provide means to overcome these difficulties. Some existing ..."
Abstract

Cited by 10 (3 self)
 Add to MetaCart
. In the classical regression theory we try to build one functional model to fit a set of data. In noisy and complex domains this methodology can be highly unreliable and/or demand too complex functional models. Piecewise regression models provide means to overcome these difficulties. Some existing approaches to piecewise regression are based on regression trees. However, rules are known to be more powerful descriptive languages than trees. This paper describes the rule learning system R R 2 2 . This system learns a set of regression rules from a classical machine learning data set. Regression rules are IFTHEN rules that have regression models in the conclusion. The conditional part of these rules determines the domain of applicability of the respective model. We believe that by adopting a rulebased formalism, R R 2 2 will outperform regression trees. The initial set of experiments that we have conducted in artificial data sets show that R R 2 2 compares reasonably to other ma...
Filtering Large Propositional Rule Sets While Retaining Classifier Performance
, 1999
"... Data mining is the problem of inducing models from data. Models have both a descriptive and a predictive aspect. Descriptive models can be inspected and used for knowledge discovery. Models consisting of decision rules  such as those produced by methods from Pawlak's rough set theory  are in pri ..."
Abstract

Cited by 9 (0 self)
 Add to MetaCart
Data mining is the problem of inducing models from data. Models have both a descriptive and a predictive aspect. Descriptive models can be inspected and used for knowledge discovery. Models consisting of decision rules  such as those produced by methods from Pawlak's rough set theory  are in principle descriptive, but in practice the induced models are too large to be inspected. In this thesis, extracting descriptive models from already induced complex models is considered. According to the principle of Occam's razor, the simplest of two models both consistent with the observed data should be chosen. A descriptive model can be found by simplifying a complex model while retaining predictive performance. The approach taken in this thesis is rule filtering; postpruning of complete rules from a model. Two methods for finding highperformance subsets of a set of rules are investigated. The first is to use a genetic algorithm to search the space of subsets. The second method is to creat...
On the Strength of Incremental Learning
, 1999
"... . This paper provides a systematic study of incremental learning from noisefree and from noisy data, thereby distinguishing between learning from only positive data and from both positive and negative data. Our study relies on the notion of noisy data introduced in [22]. The basic scenario, nam ..."
Abstract

Cited by 7 (4 self)
 Add to MetaCart
. This paper provides a systematic study of incremental learning from noisefree and from noisy data, thereby distinguishing between learning from only positive data and from both positive and negative data. Our study relies on the notion of noisy data introduced in [22]. The basic scenario, named iterative learning, is as follows. In every learning stage, an algorithmic learner takes as input one element of an information sequence for a target concept and its previously made hypothesis and outputs a new hypothesis. The sequence of hypotheses has to converge to a hypothesis describing the target concept correctly. We study the following refinements of this scenario. Bounded examplememory inference generalizes iterative inference by allowing an iterative learner to additionally store an a priori bounded number of carefully chosen data elements, while feedback learning generalizes it by allowing the iterative learner to additionally ask whether or not a particular data ele...
Chapter 6 Conclusions
, 199
"... ares (LS) regression trees generated with this simplification are very efficient in computational terms. These techniques can easily handle data sets with hundreds of thousands of cases, in few seconds. In effect, our simulation studies confirmed a clearly linear dependence of the computation time o ..."
Abstract
 Add to MetaCart
ares (LS) regression trees generated with this simplification are very efficient in computational terms. These techniques can easily handle data sets with hundreds of thousands of cases, in few seconds. In effect, our simulation studies confirmed a clearly linear dependence of the computation time on the number of cases. This can be regarded as a crucial property when facing large regression problems, which was the main motivation behind our study of LS trees. With respect to least absolute deviation (LAD) trees we have presented a theoretical analysis of this methodology, leading to a series of algorithms that ensure high computational efficiency in the task of finding the best split for each tree node. We have also attempted to prove that a theorem by Breiman et al. (1984) concerning the issue of finding the best split for discrete variables was also applicable to LAD trees. Although we were not able to obtain a proof of its validity we have encountered a counterexam
Chapter 6 Conc l us i ons
"... Least squares (LS) regression trees generated with this simplification are very efficient in computational terms. These techniques can easily handle data sets with hundreds of thousands of cases, in few seconds. In effect, our simulation studies confirmed a clearly linear dependence of the computati ..."
Abstract
 Add to MetaCart
Least squares (LS) regression trees generated with this simplification are very efficient in computational terms. These techniques can easily handle data sets with hundreds of thousands of cases, in few seconds. In effect, our simulation studies confirmed a clearly linear dependence of the computation time on the number of cases. This can be regarded as a crucial property when facing large regression problems, which was the main motivation behind our study of LS trees. With respect to least absolute deviation (LAD) trees we have presented a theoretical analysis of this methodology, leading to a series of algorithms that ensure high computational efficiency in the task of finding the best split for each tree node. We have also attempted to prove that a theorem by Breiman et al. (1984) concerning the issue of finding the best split for discrete variables was also applicable to LAD trees. Although we were not able to obtain a proof of its validity we have encountered a co