This paper provides algorithms that use an information-theoretic analysis to learn Bayesian network structures from data. Based on our three-phase learning framework, we develop efficient algorithms that can effectively learn Bayesian networks, requiring only polynomial numbers of conditional independence (CI) tests in typical cases. We provide precise conditions that specify when these algorithms are guaranteed to be correct as well as empirical evidence (from real world applications and simulation tests) that demonstrates that these systems work efficiently and reliably in practice.

In this paper we perform a comparison among FSS-EBNA, a randomized, populationbased and evolutionary algorithm, and two genetic and other two sequential search approaches in the well known Feature Subset Selection (FSS) problem. In FSS-EBNA, the FSS problem, stated as a search problem, uses the EBNA (Estimation of Bayesian Network Algorithm) search engine, an algorithm within the EDA (Estimation of Distribution Algorithm) approach. The EDA paradigm is born from the roots of the GA community in order to explicitly discover the relationships among the features of the problem and not disrupt them by genetic recombination operators. The EDA paradigm avoids the use of recombination operators and it guarantees the evolution of the population of solutions and the discovery of these relationships by the factorization of the probability distribution of best individuals in each generation of the search. In EBNA, this factorization is carried out by a Bayesian network induced by a chea...

This paper addresses the problem of learning Bayesian network structures from data by using an information theoretic dependency analysis approach. Based on our three-phase construction mechanism, two efficient algorithms have been developed. One of our algorithms deals with a special case where the node ordering is given, the algorithm only require ) ( 2 N O CI tests and is correct given that the underlying model is DAG-Faithful [Spirtes et. al., 1996]. The other algorithm deals with the general case and requires ) ( 4 N O conditional independence (CI) tests. It is correct given that the underlying model is monotone DAG-Faithful (see Section 4.4). A system based on these algorithms has been developed and distributed through the Internet. The empirical results show that our approach is efficient and reliable. 1 Introduction The Bayesian network is a powerful knowledge representation and reasoning tool under conditions of uncertainty. A Bayesian network is a directed acyclic graph ...

This report describes the riso project, a system for unified prediction and diagnosis in HVAC systems based on a class of probabilistic models called belief networks. Progress has been made in both theoretical and practical problems: a scheme for the representation of belief networks with heterogeneous conditional distributions has been devised, an algorithm for inference in a polytree network with arbitrary distributions has been implemented, and software for distributed belief networks has been implemented. After reviewing the motivation for the use of belief networks, the heterogeneous polytree algorithm is described and several important details are discussed. Distributed belief networks as a framework for reasoning under uncertainty in functionally and geographically distributed systems are then described. Several interesting questions arise in connection with distributed belief networks, such as the control of communication, publishing information in probabilistic form, and copin...

This paper describes riso, an implementation of distributed belief network software. Distributed belief networks are a natural extension of ordinary belief networks in which the belief network is composed of subnetworks running on separate processors. In keeping with the distributed computational model, no single processor has information about the structure of the entire distributed belief network, and inferences are to be computed using only local quantities. A general policy is proposed for publishing information as belief networks. A modeling language for the representation of distributed belief networks has been devised, and software has been implemented to compile the modeling language and carry out inferences. Belief networks may contain arbitrary conditional distributions, and new types of distributions can be defined without modifying the existing inference software. In inference, an exact result is computed if a rule is known for combining incoming partial results, and if an ...

This paper provides an overview of distributed belief networks and how they can model engineering systems. In extending the usual single-processor computational model to multiple processors, several interesting problems arise, which must be solved for the successful implementation of a distributed belief network. In keeping with the distributed computational model, no single processor has information about the structure of the entire distributed belief network, and inferences are to be computed using only local quantities. However, this leads to difficulties (which have not yet been resolved) when there are loops in the distributed belief network which contain nodes in two or more component networks. Also, temporal dependencies in one network lead to temporal dependencies in any other network connected to some of its variables; this may lead to intractable dependencies, especially if some of the dependencies involve different time scales. While these problems of induced dependencies ha...

We present FSS-EBNA, a new randomized, population-based and evolutionary algorithm which deals with the well known FSS problem on Data Mining applications. In FSS-EBNA, the FSS problem, stated as a search problem, uses the EBNA (Estimation of Bayesian Network Algorithm) search engine, an algorithm within the EDA (Estimation of Distribution Algorithm) approach. The EDA paradigm was born from the roots of the GA community in order to explicitly discover the relationships among the features of the problem and not disrupt them by genetic recombination operators. The EDA paradigm avoids the use of recombination operators and it guarantees the evolution of the population of solutions and the discovery of these relationships by the factorization of the probability distribution of best individuals in each generation of the search. In EBNA, this factorization is carried out by a Bayesian network induced by a cheap local search mechanism. Promising results on a set of real domains are achieved...

This paper presents a new sequential algorithm to answer the question about the existence of a causal explanation for a set of independence statements (a dependency model), which is consistent with a given set of background knowledge. Emphasis is placed on generality, efficiency and ease of parallelization of the algorithm. From this sequential algorithm, an efficient, scalable, and easy to implement parallel algorithm with very little inter-processor communication is derived.