## Probabilistic Network Construction Using the Minimum Description Length Principle (1994)

Citations: | 30 - 1 self |

### BibTeX

@MISC{Bouckaert94probabilisticnetwork,

author = {Remco R. Bouckaert},

title = {Probabilistic Network Construction Using the Minimum Description Length Principle},

year = {1994}

}

### OpenURL

### Abstract

Probabilistic networks can be constructed from a database of cases by selecting a network that has highest quality with respect to this database according to a given measure. A new measure is presented for this purpose based on a minimum description length (MDL) approach. This measure is compared with a commonly used measure based on a Bayesian approach both from a theoretical and an experimental point of view. We show that the two measures have the same properties for infinite large databases. For smaller databases, however, the MDL measure assigns equal quality to networks that represent the same set of independencies while the Bayesian measure does not. Preliminary test results suggest that an algorithm for learning probabilistic networks using the minimum description length approach performs comparably to a learning algorithm using the Bayesian approach. However, the former is slightly faster.

### Citations

7493 |
Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference
- Pearl
- 1988
(Show Context)
Citation Context ...fers a mathematically sound formalism for representing probabilistic information. Efficient algorithms have been designed for making inferences with information represented in a probabilistic network =-=[9, 12, 15]-=-. In various domains the framework has been applied successfully [1, 2, 7] indicating its practical use. Constructing a probabilistic network for a given domain by hand is a time consuming task: the d... |

1349 |
Local computations with probabilities on graphical structures and their application to expert systems (with discussion
- Lauritzen, Spiegelhalter
- 1988
(Show Context)
Citation Context ...fers a mathematically sound formalism for representing probabilistic information. Efficient algorithms have been designed for making inferences with information represented in a probabilistic network =-=[9, 12, 15]-=-. In various domains the framework has been applied successfully [1, 2, 7] indicating its practical use. Constructing a probabilistic network for a given domain by hand is a time consuming task: the d... |

1140 | A Bayesian Method for the Induction of Probabilistic Networks from Data
- Cooper, Herskovits
- 1992
(Show Context)
Citation Context ...etwork. Learning algorithms for probabilistic networks developed so far can be divided into algorithms based on non-Bayesian approaches [4, 17, 22, 23, 24] and algorithms based on a Bayesian approach =-=[5, 10, 14, 21]-=-. The non-Bayesian approaches employ statistical tests on databases for deciding on the existence of arcs in the probabilistic network under construction. The Bayesian approach assumes a prior probabi... |

685 | Approximating discrete probability distribution with dependence trees
- Chow, Liu
- 1968
(Show Context)
Citation Context ...elp shorten this build and test cycle by suggesting an initial network. Learning algorithms for probabilistic networks developed so far can be divided into algorithms based on non-Bayesian approaches =-=[4, 17, 22, 23, 24]-=- and algorithms based on a Bayesian approach [5, 10, 14, 21]. The non-Bayesian approaches employ statistical tests on databases for deciding on the existence of arcs in the probabilistic network under... |

562 | Concrete Mathematics
- Graham, Knuth, et al.
- 1994
(Show Context)
Citation Context ...two terms of expression (5), giving, \Gamma log q 2N ij ` N ij e 'N ij + r i X k=1 log q 2N ijk ` N ijk e 'N ijk : (6) Note that since for larger x, p 2x( x e ) x has a relative error of about 1 12x (=-=[6]-=- p.112) we introduce an O(1) error. Expression (6) equals, \Gamma 1 2 log 2 \Gamma ` N ij + 1 2 ' log N ij +N ij log e+ r i X k=1 ae 1 2 log 2 + ` N ijk + 1 2 ' log N ijk \Gamma N ijk log e oe : 9 Now... |

541 |
Stochastic complexity
- Rissanen
- 1987
(Show Context)
Citation Context ...purpose of removing arcs from a given network structure. 4 A Minimum Description Length Approach Another way to judge the quality of a network structure is by the minimum description length principle =-=[18, 19]-=- which stems from coding theory where the aim is to create a network structure that describes the database as accurately as possible with as few symbols as possible. 4.1 The MDL Measure The MDL princi... |

268 |
Stochastic Complexity and Modeling
- Rissanen
- 1986
(Show Context)
Citation Context ...purpose of removing arcs from a given network structure. 4 A Minimum Description Length Approach Another way to judge the quality of a network structure is by the minimum description length principle =-=[18, 19]-=- which stems from coding theory where the aim is to create a network structure that describes the database as accurately as possible with as few symbols as possible. 4.1 The MDL Measure The MDL princi... |

258 | Bayesian graphical models for discrete data
- Madigan, York
- 1995
(Show Context)
Citation Context ...etwork. Learning algorithms for probabilistic networks developed so far can be divided into algorithms based on non-Bayesian approaches [4, 17, 22, 23, 24] and algorithms based on a Bayesian approach =-=[5, 10, 14, 21]-=-. The non-Bayesian approaches employ statistical tests on databases for deciding on the existence of arcs in the probabilistic network under construction. The Bayesian approach assumes a prior probabi... |

247 |
The ALARM monitoring system: A case study with two probabilistic inference techniques for belief networks
- Beinlich, Suermondt, et al.
- 1989
(Show Context)
Citation Context ...tion. Efficient algorithms have been designed for making inferences with information represented in a probabilistic network [9, 12, 15]. In various domains the framework has been applied successfully =-=[1, 2, 7]-=- indicating its practical use. Constructing a probabilistic network for a given domain by hand is a time consuming task: the domain knowledge of one or more experts must be modelled in the formalism o... |

199 |
Bayesian analysis in expert systems
- Spiegelhalter, Dawid, et al.
- 1993
(Show Context)
Citation Context ...etwork. Learning algorithms for probabilistic networks developed so far can be divided into algorithms based on non-Bayesian approaches [4, 17, 22, 23, 24] and algorithms based on a Bayesian approach =-=[5, 10, 14, 21]-=-. The non-Bayesian approaches employ statistical tests on databases for deciding on the existence of arcs in the probabilistic network under construction. The Bayesian approach assumes a prior probabi... |

195 |
Propagating uncertainty in Bayesian networks by probabilistic logic sampling
- Henrion
- 1988
(Show Context)
Citation Context ...andom number in the unit interval and P (x i = 1j i = w ij ) calculated by 1 \Gamma P (x i = 0j i = w ij ). With the resulting probabilistic network, a set of cases was generated using logic sampling =-=[8]-=- to constitute a database D. Both K2 and K3 were applied to this database, with the node ordering used for generating the network structure. This procedure was repeated for various database sizes. The... |

98 | Causal networks: semantics and expressiveness
- Verma, Pearl
- 1988
(Show Context)
Citation Context ...elp shorten this build and test cycle by suggesting an initial network. Learning algorithms for probabilistic networks developed so far can be divided into algorithms based on non-Bayesian approaches =-=[4, 17, 22, 23, 24]-=- and algorithms based on a Bayesian approach [5, 10, 14, 21]. The non-Bayesian approaches employ statistical tests on databases for deciding on the existence of arcs in the probabilistic network under... |

90 |
MUNINâ€”A causal probabilistic network for interpretation of electromyographic findings
- Andreassen, Woldbye, et al.
- 1987
(Show Context)
Citation Context ...tion. Efficient algorithms have been designed for making inferences with information represented in a probabilistic network [9, 12, 15]. In various domains the framework has been applied successfully =-=[1, 2, 7]-=- indicating its practical use. Constructing a probabilistic network for a given domain by hand is a time consuming task: the domain knowledge of one or more experts must be modelled in the formalism o... |

84 | Toward normative expert systems: Part I. The Pathfinder project
- Heckerman, Horvitz, et al.
- 1992
(Show Context)
Citation Context ...tion. Efficient algorithms have been designed for making inferences with information represented in a probabilistic network [9, 12, 15]. In various domains the framework has been applied successfully =-=[1, 2, 7]-=- indicating its practical use. Constructing a probabilistic network for a given domain by hand is a time consuming task: the domain knowledge of one or more experts must be modelled in the formalism o... |

54 |
Graphical and recursive models for contingency tables
- Wermuth, Lauritzen
- 1983
(Show Context)
Citation Context ...elp shorten this build and test cycle by suggesting an initial network. Learning algorithms for probabilistic networks developed so far can be divided into algorithms based on non-Bayesian approaches =-=[4, 17, 22, 23, 24]-=- and algorithms based on a Bayesian approach [5, 10, 14, 21]. The non-Bayesian approaches employ statistical tests on databases for deciding on the existence of arcs in the probabilistic network under... |

52 |
Counting unlabeled acyclic digraphs
- Robinson
- 1977
(Show Context)
Citation Context ...uctures from data. We observe that the number of different network structures over n nodes is given by the recursive formula G(0) = 1, G(n) = P n i=1 (\Gamma1) i+1 ( n i )2 i(n\Gamma1) G(n \Gamma i), =-=[20]-=-. For example, for n = 10 there are approximately 4:2 \Theta 10 18 different network structures. As this number is exponential in the number of nodes, it is not feasible from a computational point of ... |

49 |
The recovery of causal poly-tree from statistical data
- Rebane, Pearl
- 1987
(Show Context)
Citation Context |

35 |
An introduction to algorithms for inference in belief nets
- Henrion
- 1990
(Show Context)
Citation Context ...fers a mathematically sound formalism for representing probabilistic information. Efficient algorithms have been designed for making inferences with information represented in a probabilistic network =-=[9, 12, 15]-=-. In various domains the framework has been applied successfully [1, 2, 7] indicating its practical use. Constructing a probabilistic network for a given domain by hand is a time consuming task: the d... |

29 |
The logic of influence diagrams
- PEARL, GEIGER, et al.
- 1988
(Show Context)
Citation Context ...uctures B S and B S 0 , we have M S = M S 0 if and only if x a \Gamma\Gammax b 2 B S , x a \Gamma\Gammax b 2 B S 0 , and x, y, z vorms a v-node in B S if and only if x, y, z vorms a v-node in B S 0 , =-=[16]-=-. Note that the condition x a \Gamma\Gammax b 2 B S , x a \Gamma\Gammax b 2 B S 0 implies that both B S and B S 0 have the same underlying undirected graph, however, the direction of the arcs may not ... |

27 |
Computer-based probabilistic-network construction
- Herskovits
- 1991
(Show Context)
Citation Context |

17 |
Diagnostic systems created by model selection methods: A case study
- Lauritzen, Thiesson, et al.
- 1994
(Show Context)
Citation Context ... that due to this property, the MDL principle gives a natural stopping criterion for heuristics that search for network structures. Approaches based on information criteria as proposed in for example =-=[11, 13]-=-, apply a quality measure that is closely related to the MDL measure: the log N term is replaced by another function and the prior distribution on probabilistic network structures is assumed uniform. ... |

15 |
BIFROST|block recursive models induced from relevant knowledge, observations, and statistical techniques
- Hjsgaard, Thiesson
- 1995
(Show Context)
Citation Context ... that due to this property, the MDL principle gives a natural stopping criterion for heuristics that search for network structures. Approaches based on information criteria as proposed in for example =-=[11, 13]-=-, apply a quality measure that is closely related to the MDL measure: the log N term is replaced by another function and the prior distribution on probabilistic network structures is assumed uniform. ... |

11 |
Optimizing causal orderings for generating dags from data
- Bouckaert
- 1992
(Show Context)
Citation Context ... but automated learning is often applied to avoid participation of expensive experts. An alternative is to start with a random ordering, apply K2 with this ordering, and to optimize this ordering. In =-=[3]-=- an 6 algorithm has been presented for optimizing an ordering for this purpose of removing arcs from a given network structure. 4 A Minimum Description Length Approach Another way to judge the quality... |