Results 1  10
of
17
Separateandconquer rule learning
 Artificial Intelligence Review
, 1999
"... This paper is a survey of inductive rule learning algorithms that use a separateandconquer strategy. This strategy can be traced back to the AQ learning system and still enjoys popularity as can be seen from its frequent use in inductive logic programming systems. We will put this wide variety of ..."
Abstract

Cited by 136 (29 self)
 Add to MetaCart
This paper is a survey of inductive rule learning algorithms that use a separateandconquer strategy. This strategy can be traced back to the AQ learning system and still enjoys popularity as can be seen from its frequent use in inductive logic programming systems. We will put this wide variety of algorithms into a single framework and analyze them along three different dimensions, namely their search, language and overfitting avoidance biases.
Rulebased Machine Learning Methods for Functional Prediction
 Journal of Artificial Intelligence Research
, 1995
"... We describe a machine learning method for predicting the value of a realvalued function, given the values of multiple input variables. The method induces solutions from samples in the form of ordered disjunctive normal form (DNF) decision rules. A central objective of the method and representation ..."
Abstract

Cited by 41 (3 self)
 Add to MetaCart
We describe a machine learning method for predicting the value of a realvalued function, given the values of multiple input variables. The method induces solutions from samples in the form of ordered disjunctive normal form (DNF) decision rules. A central objective of the method and representation is the induction of compact, easily interpretable solutions. This rulebased decision model can be extended to search efficiently for similar cases prior to approximating function values. Experimental results on realworld data demonstrate that the new techniques are competitive with existing machine learning and statistical methods and can sometimes yield superior regression performance. 1. Introduction The problem of approximating the values of a continuous variable is described in the statistical literature as regression. Given samples of output (response) variable y and input (predictor) variables x = fx 1 :::x n g, the regression task is to find a mapping y = f(x). Relative to the spac...
Logic regression
 Journal of Computational and Graphical Statistics
, 2003
"... Logic regression is an adaptive regression methodology that attempts to construct predictors as Boolean combinations of binary covariates. In many regression problems a model is developed that relates the main effects (the predictors or transformations thereof) to the response, while interactions ar ..."
Abstract

Cited by 37 (11 self)
 Add to MetaCart
Logic regression is an adaptive regression methodology that attempts to construct predictors as Boolean combinations of binary covariates. In many regression problems a model is developed that relates the main effects (the predictors or transformations thereof) to the response, while interactions are usually kept simple (two to threeway interactions at most). Often, especially when all predictors are binary, the interaction between many predictors may be what causes the differences in response. This issue arises, for example, in the analysis of SNP microarray data or in some data mining problems. In the proposed methodology, given a set of binary predictors we create new predictors such as “X1, X2, X3, and X4 are true, ” or “X5 or X6 but not X7 are true. ” In more speci � c terms: we try to � t regression models of the form g(E[Y]) = b0 + b1L1 + ¢ ¢ ¢ + bnLn, where Lj is any Boolean expression of the predictors. The Lj and bj are estimated simultaneously using a simulated annealing algorithm. This article discusses how to � t logic regression models, how to carry out model selection for these models, and gives some examples.
Predicting Equity Returns from Securities Data
, 1995
"... Our experiments with capital markets data suggest that the domain can be effectively modeled by classification rules induced from available historical data for the purpose of making gainful predictions for equityinvestments. New classification techniques developed at IBM Research, including minimal ..."
Abstract

Cited by 28 (5 self)
 Add to MetaCart
Our experiments with capital markets data suggest that the domain can be effectively modeled by classification rules induced from available historical data for the purpose of making gainful predictions for equityinvestments. New classification techniques developed at IBM Research, including minimal rule generation (RMINI) and contextual feature analysis, seem robust enough for consistently extracting useful information from noisy domains such as financial markets. We will briefly introduce the rationale for our minimal rule generation technique, and the motivation for the use of contextual information in analyzing features. We will then describe our experience from several experiments with the S&P 500 data, illustrating the general methodology, and the results of correlations and simulated managed investment based on classification rules generated by RMINI. Wewillsketchhow the rules for classifications can be effectively used for numerical prediction, and eventually to an investment ...
Functional Models for Regression Tree Leaves
, 1997
"... This paper presents a study about functional models for regression tree leaves. We evaluate experimentally several alternatives to the averages commonly used in regression trees. We have implemented a regression tree learner (HTL) that is able to use several alternative models in the tree leaves. We ..."
Abstract

Cited by 25 (3 self)
 Add to MetaCart
This paper presents a study about functional models for regression tree leaves. We evaluate experimentally several alternatives to the averages commonly used in regression trees. We have implemented a regression tree learner (HTL) that is able to use several alternative models in the tree leaves. We study the effect on accuracy and the computational cost of these alternatives. The experiments carried out on 11 data sets revealed that it is possible to significantly outperform the "naive" averages of regression trees. Among the four alternative models that we evaluated, kernel regressors were usually the best in terms of accuracy. Our study also indicates that by integrating regression trees with other regression approaches we are able to overcome the limitations of individual methods both in terms of accuracy as well as in computational efficiency. 1 INTRODUCTION In this paper we present an empirical evaluation of alternative regression models for the leaves of decision trees that dea...
Inductive Learning of Treebased Regression Models
, 1999
"... This thesis explores different aspects of the induction of treebased regression models from data. The main goal of this study is to improve the predictive accuracy of regression trees, while retaining as much as possible their comprehensibility and computational efficiency. Our study is divided in ..."
Abstract

Cited by 17 (2 self)
 Add to MetaCart
This thesis explores different aspects of the induction of treebased regression models from data. The main goal of this study is to improve the predictive accuracy of regression trees, while retaining as much as possible their comprehensibility and computational efficiency. Our study is divided in three main parts. In the first part we describe in detail two different methods of growing a regression tree: minimising the mean squared error and minimising the mean absolute deviation. Our study is particularly focussed on the computational efficiency of these tasks. We present several new algorithms that lead to significant computational speed ups. We also describe an experimental comparison of both methods of growing a regression tree highlighting their different application goals. Pruning is a standard procedure within treebased models whose goal is to provide a good compromise for achieving simple and comprehensible models with good predictive accuracy. In the second part of our study we describe a series of new techniques for pruning by selection from a series of alternative pruned trees. We carry out an extensive set of experiments comparing different methods of pruning, which show that our proposed techniques are able to significantly outperform the predictive accuracy of current state of the art pruning algorithms in a large set of regression domains. In the final part of our study we present a new type of treebased models that we refer to as local regression trees. These hybrid models integrate treebased regression with local modelling techniques. We describe different types of local regression trees and show that these models are able to significantly outperform standard regression trees in terms of predictive accuracy. Through a large set of experiments we prove the competitiveness of local regression trees when compared to existing regression techniques.
Regression by Classification
 In Proceedings of SBIA’96, Borges
, 1996
"... We present a methodology that enables the use of existent classification inductive learning systems on problems of regression. We achieve this goal by transforming regression problems into classification problems. This is done by transforming the range of continuous goal variable values into a set o ..."
Abstract

Cited by 13 (3 self)
 Add to MetaCart
We present a methodology that enables the use of existent classification inductive learning systems on problems of regression. We achieve this goal by transforming regression problems into classification problems. This is done by transforming the range of continuous goal variable values into a set of intervals that will be used as discrete classes. We provide several methods for discretizing the goal variable values. These methods are based on the idea of performing an iterative search for the set of final discrete classes. The search algorithm is guided by a Nfold cross validation estimation of the prediction error resulting from using a set of discrete classes. We have done extensive empirical evaluation of our discretization methodologies using C4.5 and CN2 on four real world domains. The results of these experiments show the quality of our discretization methods compared to other existing methods. Our method is independent of the used classification inductive system. The method is...
RAMP: Rules Abstraction for Modeling and Prediction
 IBM Research Division, IBM Research Division, T. J. Watson Research Center, Yorktown Heights, NY
, 1995
"... ion for Modeling and Prediction C. Apte, S.J. Hong, J. Lepre, S. Prasad, and B. Rosen IBM Research Division Technical Report RC20271 RAMP: Rules Abstraction for Modeling and Prediction Chidanand Apte, Se June Hong, Jorge Lepre, Seema Prasad, and Barry Rosen IBM T.J. Watson Research Center Y ..."
Abstract

Cited by 10 (3 self)
 Add to MetaCart
ion for Modeling and Prediction C. Apte, S.J. Hong, J. Lepre, S. Prasad, and B. Rosen IBM Research Division Technical Report RC20271 RAMP: Rules Abstraction for Modeling and Prediction Chidanand Apte, Se June Hong, Jorge Lepre, Seema Prasad, and Barry Rosen IBM T.J. Watson Research Center Yorktown Heights, NY 10598 January 12, 1996 Abstract Generating accurate and robust models is crucial to the successful use and deployment of classifiers on a large scale. Rule induction, i.e., generating decision rule models from data, is often a preferred approach to classification modeling and prediction, due to the enhanced explanatory capability and interpretability of decision rules. The RAMP system for rules abstraction and modeling is evolving with accuracy and robustness as primary goals. The system provides the following key capabilities: 1) feature analysis and selection based upon contextual merits technique, 2) "optimal" discretization of numerical features, 3) generation of m...
Data Fitting with RuleBased Regression
 In Proceedings of the 2nd international workshop on Artificial Intelligence Techniques (AIT'95
, 1995
"... . In the classical regression theory we try to build one functional model to fit a set of data. In noisy and complex domains this methodology can be highly unreliable and/or demand too complex functional models. Piecewise regression models provide means to overcome these difficulties. Some existing ..."
Abstract

Cited by 10 (3 self)
 Add to MetaCart
. In the classical regression theory we try to build one functional model to fit a set of data. In noisy and complex domains this methodology can be highly unreliable and/or demand too complex functional models. Piecewise regression models provide means to overcome these difficulties. Some existing approaches to piecewise regression are based on regression trees. However, rules are known to be more powerful descriptive languages than trees. This paper describes the rule learning system R R 2 2 . This system learns a set of regression rules from a classical machine learning data set. Regression rules are IFTHEN rules that have regression models in the conclusion. The conditional part of these rules determines the domain of applicability of the respective model. We believe that by adopting a rulebased formalism, R R 2 2 will outperform regression trees. The initial set of experiments that we have conducted in artificial data sets show that R R 2 2 compares reasonably to other ma...
Attribute Selection for Modeling
, 1997
"... Modelling a target attribute by other attributes in the data is perhaps the most traditional data mining task. When there are many attributes in the data, one needs to know which of the attribute (s) are relevant for modelling the target, either as a group or the one feature that is most appropriate ..."
Abstract

Cited by 8 (0 self)
 Add to MetaCart
Modelling a target attribute by other attributes in the data is perhaps the most traditional data mining task. When there are many attributes in the data, one needs to know which of the attribute (s) are relevant for modelling the target, either as a group or the one feature that is most appropriate to select within the model construction process in progress. There are many approaches for selecting the attribute(s) in machine learning. We examine various important concepts and approaches that are used for this purpose and contrast their strengths. Discretization of numeric attributes is also discussed for its use is prevalentinmany modelling techniques. Keywords: attribute quality measures, impurity function, discretization, classification, regression 1 1 Introduction A precondition to any data mining is data itself. The purpose of data mining is to explore the data and to eventually discover certain relationships, rules, correlations etc. that can give some insights about the data ...