In learning belief networks, the single link lookahead search is widely adopted to reduce the search space. We show that there exists a class of probabilistic domain models which displays a special pattern of dependency. We analyze the behavior of several learning algorithms using different scoring metrics such as the entropy, conditional independence, minimal description length and Bayesian metrics. We demonstrate that single link lookahead search procedures (employed in these algorithms) cannot learn these models correctly. Thus, when the underlying domain model actually belongs to this class, the use of a single link search procedure will result in learning of an incorrect model. This may lead to inference errors when the model is used. Our analysis suggests that if the prior knowledge about a domain does not rule out the possible existence of these models, a multi-link lookahead search or other heuristics should be used for the learning process. 1 INTRODUCTION As many effective pr...

Several scoring metrics are used in different search procedures for learning probabilistic networks. We study the properties of cross entropy in learning a decomposable Markov network. Though entropy and related scoring metrics were widely used, its `microscopic' properties and asymptotic behavior in a search have not been analyzed. We present such a `microscopic' study of a minimum entropy search algorithm, and show that it learns an I-map of the domain model when the data size is large. Search procedures that modify a network structure one link at a time have been commonly used for efficiency. Our study indicates that a class of domain models cannot be learned by such procedures. This suggests that prior knowledge about the problem domain together with a multi-link search strategy would provide an effective way to uncover many domain models.

Numerous probability models have been suggested for information retrieval (IR) over the years. These models have been applied to try to manage the inherent uncertainty in IR, for instance, document and query representation, relevance feedback, and evaluating the effectiveness of IR system. On the other hand, Bayesian networks have become an established probabilistic framework for uncertainty management in artificial intelligence. In this

Data dependencies are used in database schema design to enforce the correctness of a database as well as to reduce redundant data. These dependencies are usually determined from the semantics of the attributes and are then enforced upon the relations. This paper describes a bottom-up procedure for discovering multivalued dependencies (MVDs) in observed data without knowing `a priori the relationships amongst the attributes. The proposed algorithm is an application of the technique we designed for learning conditional independencies in probabilistic reasoning. A prototype system for automated database schema design has been implemented. Experiments were carried out to demonstrate both the effectiveness and efficiency of our method. 1

. In this paper we propose an algorithm for structure learning in predictive expert systems based on a probabilistic network representation. The idea is to have the "simplest" structure (minimum number of links) with acceptable predictive capability. The algorithm starts by building a tree structure based on measuring mutual information between pairs of variables, and then it adds links as necessary to obtain certain predictive performance. We have applied this method for ozone prediction in M'exico City, where the ozone level is used as a global indicator for the air quality in different parts of the city. It is important to predict the ozone level a day, or at least several hours in advance, to reduce the health hazards and industrial losses that occur when the ozone reaches emergency levels. We obtained as a first approximation a tree-structured dependency model for predicting ozone in one part of the city. We observe that even with only three parameters, its estimations are accepta...

Automatic generation of Bayesian network (BNs) structures (directed acyclic graphs) is an important step in experimental study of algorithms for inference in BNs and algorithms for learning BNs from data. Previously known simulation algorithms do not guarantee connectedness of generated structures or even successful genearation according to a user specification. We propose a simple, efficient and well-behaved algorithm for automatic generation of BN structures. The performance of the algorithm is demonstrated experimentally. Keywords: directed acyclic graph, graph theory, simulation, Bayesian network. 1

. Most algorithms to learn belief networks use single-link lookahead search to be efficient. It has been shown that such search procedures are problematic when applied to learning pseudo-independent (PI) models. Furthermore, some researchers have questioned whether PI models exist in practice. We present two non-trivial PI models which derive from a social study dataset. For one of them, the learned PI model reached ultimate prediction accuracy achievable given the data only, while using slightly more inference time than the learned non-PI model. These models provide evidence that PI models are not simply mathematical constructs. To develop efficient algorithms to learn PI models effectively we benefit from studying and understanding such models in depth. We further analyze how multiple PI submodels may interact in a larger domain model. Using this result, we show that the RML algorithm for learning PI models can learn more complex PI models than previously known. Keywor...

Learning belief networks from large domains can be expensive even with single-link lookahead search (SLLS). Since a SLLS cannot learn correctly in a class of problem domains, multi-link lookahead search (MLLS) is needed which further increases the computational complexity. In our experiment, learning in some difficult domains over more than a dozen variables took days. In this paper, we study how to use parallelism to speed up SLLS for learning in large domains and to tackle the increased complexity of MLLS for learning in difficult domains. We propose a natural decomposition of the learning task for parallel processing. We investigate two strategies for job allocation among processors to further improve load balancing and efficiency of the parallel system. For learning from very large datasets, we present a regrouping of the available processors such that slow data access through the file system can be replaced by fast memory access. Experimental results in a distributed memory MIMD c...

. One alternative to manual acquisition of belief networks from domain experts is automatic learning of these networks from data. Common algorithms for learning belief networks employ a single-link lookahead search. It is unclear, however, what types of domain models are learnable by such algorithms and what types of models will escape. We conjecture that these learning algorithms that use a single-link search are specializations of a simple algorithm which we call LIM. We put forward arguments that support such a conjecture, and then provide an axiomatic characterization of models learnable by LIM. The characterization coupled with the conjecture identifies models that are definitely learnable and definitely unlearnable by a class of learning algorithms. It also identifies models that are highly likely to escape these algorithms. Research to formally prove the conjecture is ongoing. Keywords: knowledge acquisition, learning, knowledge discovery. 1 Introduction Belief networks [9, 5...

Common algorithms for learning belief networks employ a single-link lookahead search. It is unclear, however, what types of domain models are learnable by such algorithms and what types of models will escape. We provide an axiomatic characterization of models learnable by a class of learning algorithms that use a single-link search. The characterization identifies models that are definitely learnable and definitely unlearnable by the entire class of algorithms. It also identifies models that are highly likely to escape current learning algorithms. A comparison between forward and backward single-link search is also presented.