Recursive graphical models usually underlie the statistical modelling concerning probabilistic expert systems based on Bayesian networks. This paper defines a version of these models, denoted as recursive exponential models, which have evolved by the desire to impose sophisticated domain knowledge onto local fragments of a model. Besides the structural knowledge, as specified by a given model, the statistical modelling may also include expert opinion about the values of parameters in the model. It is shown how to translate imprecise expert knowledge into approximately conjugate prior distributions. Based on possibly incomplete data, the score and the observed information are derived for these models. This accounts for both the traditional score and observed information, derived as derivatives of the log-likelihood, and the posterior score and observed information, derived as derivatives of the log-posterior distribution. Throughout the paper the specialization int...

This paper compares three methods --- em algorithm, Gibbs sampling, and Bound and Collapse (bc) --- to estimate conditional probabilities from incomplete databases in a controlled experiment. Results show a substantial equivalence of the estimates provided by the three methods and a dramatic gain in efficiency using bc. Reprinted from: Proceedings of Uncertainty 99: Seventh International Workshop on Artificial Intelligence and Statistics, Morgan Kaufmann, San Mateo, CA, 1999. Address: Marco Ramoni, Knowledge Media Institute, The Open University, Milton Keynes, United Kingdom MK7 6AA. phone: +44 (1908) 655721, fax: +44 (1908) 653169, email: m.ramoni@open.ac.uk, url: http://kmi.open.ac.uk/people/marco. Learning Conditional Probabilities from Incomplete Data: An Experimental Comparison Marco Ramoni Knowledge Media Institute The Open University Paola Sebastiani Statistics Department The Open University Abstract This paper compares three methods --- em algorithm, Gibbs sampling, an...

Abstract not found

Classification is one of the major tasks in knowledge discovery and data mining. Naive Bayes classifier, in spite of its simplicity, has proven surprisingly effective in many practical applications. In real datasets, noise is inevitable, because of the imprecision of measurement or privacy preserving mechanisms. In this paper, we develop a new approach, LinEar-Equation-based noise-aWare bAYes classifier (LEEWAY), for learning the underlying naive Bayes classifier from noisy observations. Using

Bayesian networks are emerging into the genomic arena as a general modeling tool able to unravel the cellular mechanism, to identify genotypes that confer susceptibility to disease, and to lead to diagnostic models. This chapter reviews the foundations of Bayesian networks and shows their application to the analysis of various types of genomic data, from genomic markers to gene expression data. The examples will highlight the potential of this methodology but also the current limitations and we will describe new research directions that hold the promise to make Bayesian networks a fundamental tool for genome data

Many applications require that we learn the parameters of a model from data. EM (ExpectationMaximization) is a method for learning the parameters of probabilistic models with missing or hidden data. There are instances in which this method is slow to converge. Therefore, several accelerations have been proposed to improve the method. None of the proposed acceleration methods are theoretically dominant and experimental comparisons are lacking. In this paper, we present the different proposed accelerations and compare them experimentally. From the results of the experiments, we argue that some acceleration of EM is always possible, but that which acceleration is superior depends on properties of the problem. 1 INTRODUCTION There are many applications in artificial intelligence and statistics that require the fitting of a parametric model to data. It is often desired to find the maximum-likelihood (ML) or maximum-a-posteriori-probability (MAP) model of the data. When all ...

Under complete data, there are closed-form maximum likelihood estimators for mixed Bayesian networks composed of discrete models [1], conditional Gaussian models [2] and conditional Gaussian regression models [2]. We describe an extension to Lauritzen' expectation-maximisation (EM) algorithm [3], which estimates the parameters of discrete networks from incomplete data, to the more general case of mixed continuous and discrete variable networks. A simple mixed network that is easy to manipulate is the leaf node continuous Bayesian network (LNCBN). Fast algorithms for estimation and marginalisation of LNCBNs are described.

Bayesian networks have gained increasing attention in recent years. One key issue in Bayesian networks (BNs) is parameter learning. When training data is incomplete or sparse or when multiple hidden nodes exist, learning parameters in Bayesian networks (BNs) becomes extremely difficult. Under these circumstances, the learning algorithms are required to operate in a high-dimensional search space and they could easily get trapped among copious local maxima. This paper presents a learning algorithm to incorporate domain knowledge into the learning to regularize the otherwise ill-posed problem, to limit the search space, and to avoid local optima. Unlike the conventional approaches that typically exploit the quantitative domain knowledge such as prior probability distribution, our method systematically incorporates qualitative constraints on some of the parameters into the learning process. Specifically, the problem is formulated as a constrained optimization problem, where an objective function is defined as a combination of the likelihood function and penalty functions constructed from the qualitative domain knowledge. Then, a gradient-descent procedure is systematically integrated with the E-step and M-step of the EM algorithm, to estimate the parameters iterativelyuntil it converges. The experiments with both synthetic data and real data for facial action recognition show 2 our algorithm improves the accuracy of the learned BN parameters significantly over the conventional EM algorithm. I.

Abstract. This paper introduces a new method, called the robust Bayesian estimator (RBE), to learn conditional probability distributions from incomplete data sets. The intuition behind the RBE is that, when no information about the pattern of missing data is available, an incomplete database constrains the set of all possible estimates and this paper provides a characterization of these constraints. An experimental comparison with two popular methods to estimate conditional probability distributions from incomplete data—Gibbs sampling and the EM algorithm—shows a gain in robustness. An application of the RBE to quantify a naive Bayesian classifier from an incomplete data set illustrates its practical relevance. Keywords: