this paper we are interested in discrete prediction problems for a decision-theoretic setting, where the

This paper addresses the problem of learning Bayesian network structures from data by using an information theoretic dependency analysis approach. Based on our three-phase construction mechanism, two efficient algorithms have been developed. One of our algorithms deals with a special case where the node ordering is given, the algorithm only require ) ( 2 N O CI tests and is correct given that the underlying model is DAG-Faithful [Spirtes et. al., 1996]. The other algorithm deals with the general case and requires ) ( 4 N O conditional independence (CI) tests. It is correct given that the underlying model is monotone DAG-Faithful (see Section 4.4). A system based on these algorithms has been developed and distributed through the Internet. The empirical results show that our approach is efficient and reliable. 1 Introduction The Bayesian network is a powerful knowledge representation and reasoning tool under conditions of uncertainty. A Bayesian network is a directed acyclic graph ...

In this paper we examine the use of Bayesian networks (BNs) for improving weather prediction, applying them to the problem of predicting sea breezes. We compare a pre-existing Bureau of Meteorology rule-based system with an elicited BN and others learned by two data mining programs, TETRAD II [Spirtes et al., 1993] and Causal MML [Wallace and Korb, 1999]. These Bayesian nets are shown to significantly outperform the rule-based system in predictive accuracy.

The main contributions of this thesis are a description of Minimum Message Length inductive inference, the comparison of this theory with other theories of inductive inference and applications of Minimum Message Length inference to regression models, mixture models and segmentation models.

Weak causal relationships and small sample size pose two significant difficulties to the automatic discovery of causal models from observational data. This paper examines the influence of weak causal links and varying sample sizes on the discovery of causal models. The experimental results illustrate the effect of larger sample sizes for discovering causal models reliably and the relevance of the strength of causal links and the complexity of the original causal model. We present indicative evidence of the superior robustness of MML (Minimum Message Length) methods to standard significance tests in the recovery of causal links. The comparative results show that the MML-CI (the MML Causal Inducer) causal discovery system finds better models than TETRAD II given small samples from linear causal models. The experimental results also reveal that MML-CI finds weak links with smaller sample sizes than can TETRAD II.

This paper presents an ensemble MML approach for the discovery of causal models. The component learners are formed based on the MML causal induction methods. Six different ensemble causal induction algorithms are proposed. Our experiential results reveal that (1) the ensemble MML causal induction approach has achieved an improved result compared with any single learner in terms of learning accuracy and correctness; (2) Among all the ensemble causal induction algorithms examined, the weighted voting without seeding algorithm outperforms all the rest; (3) It seems that the ensembled CI algorithms could alleviate the local minimum problem. The only drawback of this method is that the time complexity is increased by \delta times, where \delta is the ensemble size.

This paper formulates the problem of learning Bayesian network structures from data as determining the structure that best approximates the probability distribution indicated by the data. A new metric, Penalized Mutual Information metric, is proposed, and a evolutionary algorithm is designed to search for the best structure among alternatives. The experimental results show that this approach is reliable and promising. 1. Problem Definition and PMI Metric Bayesian network is a powerful knowledge representation and reasoning tool under uncertainty [1]. However, the construction of a Bayesian network manually is usually time-consuming and subject to mistakes. Therefore, algorithms for automatic learning, that occasionally use the information provided by an expert, can be of great help [2]. Considering the fact that any Bayesian network for domain U uniquely determines a joint probability function over the domain U, the problem of structure learning of Bayesian network, can be viewed as finding the best approximate decomposition of the target distribution determined by the data. Let p ˆ be the probability distribution function represented by a Bayesian network, the KL difference [3] between p ˆ and the target probability distribution function p over the domain is KL( p( v, v1, , vn

How to assess the performance of machine learning algorithms is a problem of increasing interest and urgency as the data mining application of myriad algorithms grows. The standard approach of employing predictive accuracy has rightly been losing favor in the AI community. The alternative of cost-sensitive metrics provides a far better approach, given the availability of useful cost functions. For situations where no useful cost function can be found we need other alternatives to predictive accuracy. We propose that information-theoretic reward functions be applied. The first such proposal for assessing specifically machine learning algorithms was made by Kononenko and Bratko [1]. Here we improve upon our alternative Bayesian metric [2], which provides a fair betting assessment of any machine learner. We include an empirical analysis of various Bayesian classification learners, ranging from Naive Bayes learners to causal discovery algorithms.

presented at the 22nd Conference on Hurricane and Tropical Meteorology, Fort Collins -- Colorado, 19 -- 23 May 1997 1 Introduction The existing tropical cyclone intensity forecasting schemes (SHIFOR [Jarvinen and Neumann, 1979], SHIPS [DeMaria and Kaplan, 1994], SHIFOR94 [Landsea, 1995], TIPS [Fitzpatrick, 1995]) were built using the conventional multiple linear regression method. This method relies upon statistical significance test techniques which the chosen models prone to overfit the data. This inherent tendency of overfitting makes the separation of the limited available data into the training and test data sets imperative. In this abstract, a Bayesian approach using the Minimum Message Length (MML) principle [Wallace and Freeman, 1987] is applied to tropical cyclone intensity change forecasting. The MML technique builds regression models by taking a balance between the complexity of the models and the goodness of fit as a performance criterion. Because of this balancing mech...

orrelation and Causation ffl Independence in Linear Models ffl References Reading: ffl S. Wright (1934) The Method of Path Coefficients. Lecture 10 Korb 5 Learning Bayesian Nets (LBN) Motivation: As applications of BNs have proliferated, interest has exploded in automating the learning of them. Why? Same old reason (e.g., for success of neural nets): the "knowledge bottleneck" Bayesian Intelligent Decision Support (BIDS): 1. Elicitation: BNs, CPTs, utilities 2. Learning: ffl Structure ffl Parameters ffl Adaptation 3. Validation & Sensitivity Analysis Lecture 10 Korb 6 BIDS Lifecycle Industrial Use Collection of Statistics Collection of experimental results Field Testing Alpha/Beta Testing Validation Sensitivity Analysis Accuracy Testing Refinement U