## MML Inference of Oblique Decision Trees (2004)

### Cached

### Download Links

- [www.csse.monash.edu.au]
- [www.cs.monash.edu.au]
- DBLP

### Other Repositories/Bibliography

Venue: | In Lecture Notes in Artificial Intelligence (LNAI) 3339 (Springer), Proc. 17th Australian Joint Conf. on AI |

Citations: | 9 - 5 self |

### BibTeX

@INPROCEEDINGS{Tan04mmlinference,

author = {Peter J. Tan and David L. Dowe},

title = {MML Inference of Oblique Decision Trees},

booktitle = {In Lecture Notes in Artificial Intelligence (LNAI) 3339 (Springer), Proc. 17th Australian Joint Conf. on AI},

year = {2004},

pages = {1082--1088},

publisher = {Springer}

}

### OpenURL

### Abstract

Abstract. We propose a multivariate decision tree inference scheme by using the minimum message length (MML) principle (Wallace and Boulton, 1968; Wallace and Dowe, 1999). The scheme uses MML coding as an objective (goodness-of-fit) function on model selection and searches with a simple evolution strategy. We test our multivariate tree inference scheme on UCI machine learning repository data sets and compare with the decision tree programs C4.5 and C5. The preliminary results show that on average and on most data-sets, MML oblique trees clearly perform better than both C4.5 and C5 on both “right”/“wrong ” accuracy and probabilistic prediction- and with smaller trees, i.e., less leaf nodes. 1

### Citations

3892 |
Classification and Regression Trees
- Breiman, JH, et al.
- 1984
(Show Context)
Citation Context ...ght”/“wrong” accuracy and probabilistic prediction - and with smaller trees, i.e., less leaf nodes. 1 Introduction While there are a number of excellent decision tree learning algorithms such as CART =-=[2]-=-, C4.5 and C5 [13], much research effort has been continuously directed to finding new and improved tree induction algorithms. Most decision tree algorithms only test on one attribute at internal node... |

2862 |
UCI repository of machine learning databases
- BLAKE, C
- 1998
(Show Context)
Citation Context ... these trees. 3 Experiments 3.1 Comparing and scoring probabilistic predictions To evaluate our new oblique decision tree scheme, we run experiments on nine data sets selected from the UCI Repository =-=[1]-=-. The performance of our scheme is compared with those of C4.5 and C5 [13]. In addition to the traditional right/wrong accuracy, we are also keen to compare the probabilistic performances[17, sec 5.1]... |

311 |
An Information Measure for Classification
- Wallace, Boulton
- 1968
(Show Context)
Citation Context ...space with hyperplanes of arbitrary slope - rather than only parallel to the co-ordinate axes. We propose an oblique decision tree inference scheme by using the minimum message length (MML) principle =-=[19, 21, 20, 17]-=-. Test results show our new oblique decision tree inference algorithms find smaller trees with better (or near identical) accuracy compared to the standard univariate schemes, C4.5 and C5. 2 MML Infer... |

293 |
Inferring decision tree using the Minimum Description Length principle
- Quinlan, Riverst
- 1989
(Show Context)
Citation Context ...schemes, C4.5 and C5. 2 MML Inference of Multivariate Decision Trees MML inference [19, 21, 8, 20, 17, 4, 5, 18] has been successfully implemented in [22] to infer univariate decision trees (refining =-=[14]-=-) and in [12, 16, 17] to infer univariate decision graphs, with the most recent decision graphs [16, 17] clearly out-performing both C4.5 and C5 [13] on both real-world and artificial data-sets on a r... |

187 |
Estimation and inference by compact coding
- Wallace, Freeman
- 1987
(Show Context)
Citation Context ...space with hyperplanes of arbitrary slope - rather than only parallel to the co-ordinate axes. We propose an oblique decision tree inference scheme by using the minimum message length (MML) principle =-=[19, 21, 20, 17]-=-. Test results show our new oblique decision tree inference algorithms find smaller trees with better (or near identical) accuracy compared to the standard univariate schemes, C4.5 and C5. 2 MML Infer... |

104 | Minimum message length and Kolmogorov complexity,” The Computer Journal,vol.42,no.4
- Wallace, Dowe
- 1999
(Show Context)
Citation Context ...space with hyperplanes of arbitrary slope - rather than only parallel to the co-ordinate axes. We propose an oblique decision tree inference scheme by using the minimum message length (MML) principle =-=[19, 21, 20, 17]-=-. Test results show our new oblique decision tree inference algorithms find smaller trees with better (or near identical) accuracy compared to the standard univariate schemes, C4.5 and C5. 2 MML Infer... |

86 |
2005): Statistical and Inductive Inference by Minimum Message
- Wallace
(Show Context)
Citation Context ...inference algorithms find smaller trees with better (or near identical) accuracy compared to the standard univariate schemes, C4.5 and C5. 2 MML Inference of Multivariate Decision Trees MML inference =-=[19, 21, 8, 20, 17, 4, 5, 18]-=- has been successfully implemented in [22] to infer univariate decision trees (refining [14]) and in [12, 16, 17] to infer univariate decision graphs, with the most recent decision graphs [16, 17] cle... |

84 |
Coding decision trees
- Wallace, Patrick
- 1993
(Show Context)
Citation Context ...cal) accuracy compared to the standard univariate schemes, C4.5 and C5. 2 MML Inference of Multivariate Decision Trees MML inference [19, 21, 8, 20, 17, 4, 5, 18] has been successfully implemented in =-=[22]-=- to infer univariate decision trees (refining [14]) and in [12, 16, 17] to infer univariate decision graphs, with the most recent decision graphs [16, 17] clearly out-performing both C4.5 and C5 [13] ... |

83 | system for induction of oblique decision trees
- Kasif, Salzberg
- 1994
(Show Context)
Citation Context ...onal hyper-cube (from Fig. 1a). 2.2 Search for the optimal hyperplane In order to perform faster searches for optimal multivariate splits, we do not use the search heuristic used in OC1 [10] and SADT =-=[9]-=-. Instead, we implement a simple evolution strategy as the preliminary search heuristic for our scheme. A similar approach has appeared in [3], in which promising results were reported. The search pro... |

33 |
On growing better decision trees from data
- Murthy, Salzberg
- 1995
(Show Context)
Citation Context ... the D-dimensional hyper-cube (from Fig. 1a). 2.2 Search for the optimal hyperplane In order to perform faster searches for optimal multivariate splits, we do not use the search heuristic used in OC1 =-=[10]-=- and SADT [9]. Instead, we implement a simple evolution strategy as the preliminary search heuristic for our scheme. A similar approach has appeared in [3], in which promising results were reported. T... |

20 |
General Bayesian networks and asymmetric languages
- Comley, Dowe
- 2003
(Show Context)
Citation Context ...inference algorithms find smaller trees with better (or near identical) accuracy compared to the standard univariate schemes, C4.5 and C5. 2 MML Inference of Multivariate Decision Trees MML inference =-=[19, 21, 8, 20, 17, 4, 5, 18]-=- has been successfully implemented in [22] to infer univariate decision trees (refining [14]) and in [12, 16, 17] to infer univariate decision graphs, with the most recent decision graphs [16, 17] cle... |

15 |
Message length as an effective ockham’s razor in decision tree induction
- Needham, Dowe
- 2001
(Show Context)
Citation Context ... The performance of our scheme is compared with those of C4.5 and C5 [13]. In addition to the traditional right/wrong accuracy, we are also keen to compare the probabilistic performances[17, sec 5.1] =-=[7, 6, 11, 16]-=- of the learning algorithms. In a lot of domains, like oncological and other medical data, not only the class predictions but also the probability associated with each class is essential. In some doma... |

13 | Using evolutionary algorithms to induce oblique decision trees
- Cantú-Paz, Kamath
- 2000
(Show Context)
Citation Context ...do not use the search heuristic used in OC1 [10] and SADT [9]. Instead, we implement a simple evolution strategy as the preliminary search heuristic for our scheme. A similar approach has appeared in =-=[3]-=-, in which promising results were reported. The search process in our scheme can be summarized as follows. Assuming the linear discriminant function in our scheme takes the form �d i=1 wixi < wd+1, fo... |

12 |
Information-theoretic football tipping
- Dowe, Farr, et al.
- 1996
(Show Context)
Citation Context ... The performance of our scheme is compared with those of C4.5 and C5 [13]. In addition to the traditional right/wrong accuracy, we are also keen to compare the probabilistic performances[17, sec 5.1] =-=[7, 6, 11, 16]-=- of the learning algorithms. In a lot of domains, like oncological and other medical data, not only the class predictions but also the probability associated with each class is essential. In some doma... |

11 |
Decision tree models of bushfire activity
- Dowe, Krusel
- 1994
(Show Context)
Citation Context ... The performance of our scheme is compared with those of C4.5 and C5 [13]. In addition to the traditional right/wrong accuracy, we are also keen to compare the probabilistic performances[17, sec 5.1] =-=[7, 6, 11, 16]-=- of the learning algorithms. In a lot of domains, like oncological and other medical data, not only the class predictions but also the probability associated with each class is essential. In some doma... |

11 |
Kolmogorov complexity, minimum message length and inverse learning
- Dowe, Wallace
- 1998
(Show Context)
Citation Context ...inference algorithms find smaller trees with better (or near identical) accuracy compared to the standard univariate schemes, C4.5 and C5. 2 MML Inference of Multivariate Decision Trees MML inference =-=[19, 21, 8, 20, 17, 4, 5, 18]-=- has been successfully implemented in [22] to infer univariate decision trees (refining [14]) and in [12, 16, 17] to infer univariate decision graphs, with the most recent decision graphs [16, 17] cle... |

10 |
Inferring decision graphs
- Oliver, Wallace
- 1991
(Show Context)
Citation Context ... and C5. 2 MML Inference of Multivariate Decision Trees MML inference [19, 21, 8, 20, 17, 4, 5, 18] has been successfully implemented in [22] to infer univariate decision trees (refining [14]) and in =-=[12, 16, 17]-=- to infer univariate decision graphs, with the most recent decision graphs [16, 17] clearly out-performing both C4.5 and C5 [13] on both real-world and artificial data-sets on a range of test criteria... |

10 |
Hypersensitivity to Perturbations in the quantum kicked top,” Phys
- Schack, D’Ariano, et al.
- 1994
(Show Context)
Citation Context ... that a randomly picked vector falls into the set is given by Vθ VT , where Vθ is the volume of a partial sphere of radius θ and VT is the total volume of the unit sphere. The value of Vθ VT is given =-=[15]-=- by (sin θ)2(D−1) , so the information required to specify the set of the vectors is − log((sin θ) 2(D−1) ). By specifying one data point on each side of the hyperplane hc, two hyperplanes which are p... |

10 | MML inference of decision graphs with multi-way joins
- Tan, Dowe
- 2002
(Show Context)
Citation Context ... and C5. 2 MML Inference of Multivariate Decision Trees MML inference [19, 21, 8, 20, 17, 4, 5, 18] has been successfully implemented in [22] to infer univariate decision trees (refining [14]) and in =-=[12, 16, 17]-=- to infer univariate decision graphs, with the most recent decision graphs [16, 17] clearly out-performing both C4.5 and C5 [13] on both real-world and artificial data-sets on a range of test criteria... |

6 |
Minimum Message Length, MDL and Generalised Bayesian Networks with Asymmetric Languages, Chapter 11
- Comley, Dowe
- 2005
(Show Context)
Citation Context |