In learning belief networks, the single link lookahead search is widely adopted to reduce the search space. We show that there exists a class of probabilistic domain models which displays a special pattern of dependency. We analyze the behavior of several learning algorithms using different scoring metrics such as the entropy, conditional independence, minimal description length and Bayesian metrics. We demonstrate that single link lookahead search procedures (employed in these algorithms) cannot learn these models correctly. Thus, when the underlying domain model actually belongs to this class, the use of a single link search procedure will result in learning of an incorrect model. This may lead to inference errors when the model is used. Our analysis suggests that if the prior knowledge about a domain does not rule out the possible existence of these models, a multi-link lookahead search or other heuristics should be used for the learning process. 1 INTRODUCTION As many effective pr...

Numerous probability models have been suggested for information retrieval (IR) over the years. These models have been applied to try to manage the inherent uncertainty in IR, for instance, document and query representation, relevance feedback, and evaluating the effectiveness of IR system. On the other hand, Bayesian networks have become an established probabilistic framework for uncertainty management in artificial intelligence. In this

Abstract—We consider a common task in multiagent systems where agents need to estimate the state of an uncertain domain so that they can act accordingly. If each agent only has partial knowledge about the domain and local observations, how can the agents accomplish the task with a limited amount of communication? Multiply sectioned Bayesian networks (MSBNs) provide an effective and exact framework for such a task but also impose a set of constraints. Are there simpler frameworks with the same performance but with less constraints? We identify a small set of high level choices which logically imply the key representational choices leading to MSBNs. The result addresses the necessity of constraints of the framework. It facilitates comparisons with related frameworks and provides guidance to potential extensions of the framework. (Keywords: multiagent system, decentralized interpretation, communication, organization structure, uncertain reasoning, probabilistic reasoning, belief network, Bayesian network) I.

Data dependencies are used in database schema design to enforce the correctness of a database as well as to reduce redundant data. These dependencies are usually determined from the semantics of the attributes and are then enforced upon the relations. This paper describes a bottom-up procedure for discovering multivalued dependencies (MVDs) in observed data without knowing `a priori the relationships amongst the attributes. The proposed algorithm is an application of the technique we designed for learning conditional independencies in probabilistic reasoning. A prototype system for automated database schema design has been implemented. Experiments were carried out to demonstrate both the effectiveness and efficiency of our method. 1

Automatic generation of Bayesian network (BNs) structures (directed acyclic graphs) is an important step in experimental study of algorithms for inference in BNs and algorithms for learning BNs from data. Previously known simulation algorithms do not guarantee connectedness of generated structures or even successful genearation according to a user specification. We propose a simple, efficient and well-behaved algorithm for automatic generation of BN structures. The performance of the algorithm is demonstrated experimentally. Keywords: directed acyclic graph, graph theory, simulation, Bayesian network. 1

. Most algorithms to learn belief networks use single-link lookahead search to be efficient. It has been shown that such search procedures are problematic when applied to learning pseudo-independent (PI) models. Furthermore, some researchers have questioned whether PI models exist in practice. We present two non-trivial PI models which derive from a social study dataset. For one of them, the learned PI model reached ultimate prediction accuracy achievable given the data only, while using slightly more inference time than the learned non-PI model. These models provide evidence that PI models are not simply mathematical constructs. To develop efficient algorithms to learn PI models effectively we benefit from studying and understanding such models in depth. We further analyze how multiple PI submodels may interact in a larger domain model. Using this result, we show that the RML algorithm for learning PI models can learn more complex PI models than previously known. Keywor...

A pseudo-independent (PI) domain is a problem domain where a proper subset of a set of collectively dependent variables displays marginal independence.

Learning belief networks from large domains can be expensive even with single-link lookahead search (SLLS). Since a SLLS cannot learn correctly in a class of problem domains, multi-link lookahead search (MLLS) is needed which further increases the computational complexity. In our experiment, learning in some difficult domains over more than a dozen variables took days. In this paper, we study how to use parallelism to speed up SLLS for learning in large domains and to tackle the increased complexity of MLLS for learning in difficult domains. We propose a natural decomposition of the learning task for parallel processing. We investigate two strategies for job allocation among processors to further improve load balancing and efficiency of the parallel system. For learning from very large datasets, we present a regrouping of the available processors such that slow data access through the file system can be replaced by fast memory access. Experimental results in a distributed memory MIMD c...

Estimation of Distribution Algorithms (EDA) have been proposed as an extension of genetic algorithms. Our algorithm FDA assumes that the function to be optimized is additively decomposed (ADF). The interaction graph GADF is used to create exact or approximate factorizations of the Boltzmann distribution. Using Gibbs sampling instead of probabilistic logic sampling is investigated. We also discuss the algorithm LFDA which learns a Bayesian network from data. For both algorithms estimates of the necessary sample size N to find the optimum are derived. The bounds are based on statistical learning theory and PAC learning. If the assumptions of a factorization theorem are fulfilled, the upper bound of the sample size N of FDA is of order O(n ln n) where n is the size of the problem. The computational complexity per generation is O(N ∗ n). For LFDA a bound cannot be proven because the network learned might be far from optimal. In many applications the optimal network is not necessary for converge to the global optima. For the 2D Ising model only 60 % of the edges of GADF need to be contained in the learned graph. Bounds can be obtained for two new learning methods. The first one learns factor graphs instead of Bayesian networks, the second one detects the structure of the function by computing its Walsh or Fourier coefficients. The computational complexity to compute the Walsh coefficients is O(n 2 ln n). The networks computed by FDA and LFDA are analyzed for a set of benchmark functions.

. One alternative to manual acquisition of belief networks from domain experts is automatic learning of these networks from data. Common algorithms for learning belief networks employ a single-link lookahead search. It is unclear, however, what types of domain models are learnable by such algorithms and what types of models will escape. We conjecture that these learning algorithms that use a single-link search are specializations of a simple algorithm which we call LIM. We put forward arguments that support such a conjecture, and then provide an axiomatic characterization of models learnable by LIM. The characterization coupled with the conjecture identifies models that are definitely learnable and definitely unlearnable by a class of learning algorithms. It also identifies models that are highly likely to escape these algorithms. Research to formally prove the conjecture is ongoing. Keywords: knowledge acquisition, learning, knowledge discovery. 1 Introduction Belief networks [9, 5...