## A Parallel Learning Algorithm for Bayesian Inference Networks

Venue: | IEEE Transactions on Knowledge and Data Engineering |

Citations: | 4 - 0 self |

### BibTeX

@ARTICLE{Lam_aparallel,

author = {Wai Lam and Alberto Maria Segre},

title = {A Parallel Learning Algorithm for Bayesian Inference Networks},

journal = {IEEE Transactions on Knowledge and Data Engineering},

year = {},

volume = {2002},

pages = {93--105}

}

### OpenURL

### Abstract

We present a new parallel algorithm for learning Bayesian inference networks from data. Our learning algorithm exploits both properties of the MDL-based score metric, and a distributed, asynchronous, adaptive search technique called nagging. Nagging is intrinsically fault tolerant, has dynamic load balancing features, and scales well. We demonstrate the viability, effectiveness, and scalability of our approach empirically with several experiments using on the order of 20 machines. More specifically, we show that our distributed algorithm can provide optimal solutions for larger problems as well as good solutions for Bayesian networks of up to 150 variables. Keywords: Machine Learning, Bayesian Networks, Minimum Description Length Principle, Distributed Systems Support for this research was provided by the Office of Naval Research through grant N0014-94-1-1178, and by the Advanced Research Project Agency through Rome Laboratory Contract Number F30602-93-C-0018 via Odyssey Research As...

### Citations

1140 | A Bayesian Method for the Induction of Probabilistic Networks from Data
- Cooper, Herskovits
- 1992
(Show Context)
Citation Context ...nd prone to error. Clearly, any mechanism that can help automate this task would be beneficial. One technique to cope with this problem is to learn the network model from data pertinent to the domain =-=[12, 8, 4, 19, 24, 28]-=-. Unfortunately, since this problem is believed to be NPcomplete [7], learning larger models requires exponentially increasing computational resources. In this paper, we present a new distributed solu... |

953 | Learning Bayesian networks: The combination of knowledge and statistical data
- Heckerman, Geiger, et al.
- 1995
(Show Context)
Citation Context ...nd prone to error. Clearly, any mechanism that can help automate this task would be beneficial. One technique to cope with this problem is to learn the network model from data pertinent to the domain =-=[12, 8, 4, 19, 24, 28]-=-. Unfortunately, since this problem is believed to be NPcomplete [7], learning larger models requires exponentially increasing computational resources. In this paper, we present a new distributed solu... |

609 |
The computational complexity of probabilistic inference using Bayesian belief networks models approach
- Cooper
(Show Context)
Citation Context ...eficial. One technique to cope with this problem is to learn the network model from data pertinent to the domain [12, 8, 4, 19, 24, 28]. Unfortunately, since this problem is believed to be NPcomplete =-=[7]-=-, learning larger models requires exponentially increasing computational resources. In this paper, we present a new distributed solution to the Bayesian network learning problem that exploits idle or ... |

326 | A Bayesian model of plan recognition
- Charniak, Goldman
- 1993
(Show Context)
Citation Context ...LLIPATH, which is in used by several hundred medical and clinical sites. ffl Other Applications: Other application areas include, for example, software maintenance [5], natural language understanding =-=[6]-=-, troubleshooting [11]. Despite these successful system deployments, systems designers who intend to use Bayesian networks -- like designers of knowledge-based systems in general -- encounter the know... |

247 |
The ALARM monitoring system: A case study with two probabilistic inference techniques for belief networks
- Beinlich, Suermondt, et al.
- 1989
(Show Context)
Citation Context ...ructures were used to generate experimental data sets. The first data set is based on the ALARM Bayesian inference network used to model realworld anesthesia problems in an operating room environment =-=[2]-=-. This network consists of 37 variables and 46 arcs. The 10,000 case data set associated with this network is commonly used as a benchmark; the input variable ordering used in our experiments is ident... |

220 |
Equivalence and synthesis of causal models
- Verma, Pearl
- 1990
(Show Context)
Citation Context ...ches for this learning problem are characterized by employing conditional independence relations exemplified by the SGS and PC algorithms [23, 22] as well as the techniques developed by Pearl et. al. =-=[29, 19]-=-. In this section, we briefly describe a simple, score-based, solution to the Bayesian network structure learning problem that is based on the result of our previous work [16]. 3.1 The Minimum Descrip... |

215 | A Theory of Inferred Causation
- Pearl, Verma
(Show Context)
Citation Context ...nd prone to error. Clearly, any mechanism that can help automate this task would be beneficial. One technique to cope with this problem is to learn the network model from data pertinent to the domain =-=[12, 8, 4, 19, 24, 28]-=-. Unfortunately, since this problem is believed to be NPcomplete [7], learning larger models requires exponentially increasing computational resources. In this paper, we present a new distributed solu... |

199 | Learning Bayesian belief networks: An approach based on the MDL principle
- Lam, Bacchus
- 1994
(Show Context)
Citation Context ...cting a network structure based on the input data and ordering constraints. 3 A Simple Serial Solution One family of techniques for learning Bayesian network structures is based on a scoring approach =-=[8, 12, 16, 27]-=-. This approach is characterized by devising a score metric for a candidate network structure, and searching the space of network structures for the best-scoring structure. Since there are exponential... |

86 |
An algorithm for fast recovery of sparse causal graphs
- Spirtes, Glymour, et al.
- 1991
(Show Context)
Citation Context ...ge) length of a network structure. Alternatively, other approaches for this learning problem are characterized by employing conditional independence relations exemplified by the SGS and PC algorithms =-=[23, 22]-=- as well as the techniques developed by Pearl et. al. [29, 19]. In this section, we briefly describe a simple, score-based, solution to the Bayesian network structure learning problem that is based on... |

79 | DecisionTheoretic Troubleshooting
- Breese, Heckerman
- 1999
(Show Context)
Citation Context ...used by several hundred medical and clinical sites. ffl Other Applications: Other application areas include, for example, software maintenance [5], natural language understanding [6], troubleshooting =-=[11]-=-. Despite these successful system deployments, systems designers who intend to use Bayesian networks -- like designers of knowledge-based systems in general -- encounter the knowledge engineering bott... |

76 | Display of Information for Time-Critical Decision Making
- Horvitz, Barry
- 1995
(Show Context)
Citation Context ...a [9]. ffl Intelligent Decision Making: The Vista system, developed at NASA Mission Control Center, interprets live telemetry data and assesses the operation of the space shuttle's propulsion systems =-=[13]-=-. ffl Process monitoring: General Electric's GEMS expert system monitors power generation equipment performance [17]. ffl Medical Diagnosis: The PATHFINDER system [10] performs diagnosis of lymph node... |

56 | Learning Bayesian networks with discrete variables from data
- Spirtes, Meek
- 1995
(Show Context)
Citation Context |

52 |
Counting unlabeled acyclic digraphs
- Robinson
- 1977
(Show Context)
Citation Context ...ate network structure, and searching the space of network structures for the best-scoring structure. Since there are exponential number of candidate network structures for a given number of variables =-=[20]-=-, finding an optimal solution is infeasible even for problems of moderate size. Thus scoring systems generally resort to greedy or heuristic search methods that find reasonable -- but suboptimal -- so... |

52 | Learning Bayesian belief networks based on the MDL principle: An efficient algorithm using the branch and bound technique
- Suzuki
- 1996
(Show Context)
Citation Context |

48 | Applying Bayesian networks to information retrieval - Fung, Favero - 1995 |

42 |
Properties of Bayesian belief network learning algorithms
- Bouckaert
- 1994
(Show Context)
Citation Context ...escription length is that the more accurate the network structure, 2 Other researchers have proposed alternative metrics for network description length that differ from the one we have just described =-=[27, 3]-=-. We note that the parallelization scheme we propose in subsequent sections of this paper is also applicable to systems based on many of these alternative metrics. the smaller its data description len... |

35 | Using causal information and local measures to learn Bayesian belief networks
- Lain, Bacchus
- 1993
(Show Context)
Citation Context ... is a function representing the total description length, L total (B), of a candidate network structure B. In related work, we have designed a scheme for efficiently computing this description length =-=[15]-=- of a candidate network structure B by decomposing it by individual variable X i . With some overloading of the notation L total , we can say: L total (B) = X X i 2X L total (X i ; \Upsilon X i ) wher... |

33 |
A construction of Bayesian networks from databases based on an MDL scheme
- Suzuki
- 1993
(Show Context)
Citation Context ...cting a network structure based on the input data and ordering constraints. 3 A Simple Serial Solution One family of techniques for learning Bayesian network structures is based on a scoring approach =-=[8, 12, 16, 27]-=-. This approach is characterized by devising a score metric for a candidate network structure, and searching the space of network structures for the best-scoring structure. Since there are exponential... |

29 | Structure and Chance: Melding Logic and Probability for Software Debugging
- Burnell, Horvitz
- 1995
(Show Context)
Citation Context ...d to a commercial system called INTELLIPATH, which is in used by several hundred medical and clinical sites. ffl Other Applications: Other application areas include, for example, software maintenance =-=[5]-=-, natural language understanding [6], troubleshooting [11]. Despite these successful system deployments, systems designers who intend to use Bayesian networks -- like designers of knowledge-based syst... |

24 |
causality from probability
- Spirtes, Glymour, et al.
(Show Context)
Citation Context ...ge) length of a network structure. Alternatively, other approaches for this learning problem are characterized by employing conditional independence relations exemplified by the SGS and PC algorithms =-=[23, 22]-=- as well as the techniques developed by Pearl et. al. [29, 19]. In this section, we briefly describe a simple, score-based, solution to the Bayesian network structure learning problem that is based on... |

16 |
Bayesian network refinement via machine learning approach
- Lam
(Show Context)
Citation Context ...ptimal -- solutions quickly. As for the score metrics, examples such as the BD and K2 metrics in [12, 8] are the relative posterior probability of a network structure, while the score metrics used in =-=[14, 27]-=- are cost functions representing the description (message) length of a network structure. Alternatively, other approaches for this learning problem are characterized by employing conditional independe... |

15 |
Toward Normative Expert Systems: The Pathfinder Project
- Heckerman, Horvitz, et al.
- 1992
(Show Context)
Citation Context ...ace shuttle's propulsion systems [13]. ffl Process monitoring: General Electric's GEMS expert system monitors power generation equipment performance [17]. ffl Medical Diagnosis: The PATHFINDER system =-=[10]-=- performs diagnosis of lymph node pathology for over 60 diseases: it has been recently transferred to a commercial system called INTELLIPATH, which is in used by several hundred medical and clinical s... |

13 |
ARCO1: An application of belief networks to the oil market
- Abramson
- 1991
(Show Context)
Citation Context ...earch Laboratory employs Bayesian networks to perform ship classification from raw sensor images [18]. ffl Forecasting: The ARCO1 system is able to both reason about and forecast the crude oil market =-=[1]-=-. ffl Information Retrieval: A Bayesian network is used to retrieve documents relevant to a particular information need from a huge collection of information stored in electronic media [9]. ffl Intell... |

7 |
Nagging: A distributed adversarial search-pruning technique applied to first-order logic
- Sturgill, Segre
- 1997
(Show Context)
Citation Context ...master's search along by determining whether a portion of the master's search space is infeasible. Readers interested in a more complete treatment of nagging and its formal properties are referred to =-=[26]-=-. Nagging has a number of desirable properties. For example, it is intrinsically fault tolerant, since losing a nagger due to communication or hardware problems will not compromise the master's soluti... |

6 |
Commercialization of EPRI's Generator Expert Monitoring
- Morjaia, Rink, et al.
- 1993
(Show Context)
Citation Context ...telemetry data and assesses the operation of the space shuttle's propulsion systems [13]. ffl Process monitoring: General Electric's GEMS expert system monitors power generation equipment performance =-=[17]-=-. ffl Medical Diagnosis: The PATHFINDER system [10] performs diagnosis of lymph node pathology for over 60 diseases: it has been recently transferred to a commercial system called INTELLIPATH, which i... |

6 | Using hundreds of workstations to solve first-order logic problems
- Segre, Sturgill
- 1994
(Show Context)
Citation Context ...h may result in lower overall performance. 4.2 A Second Parallel Solution Based on our previous work on nagging, a parallel asynchronous search pruning technique for first-order logic theorem proving =-=[25, 21]-=-, we now propose a distributed strategy that addresses the shortcomings of the simple partitioning scheme just described. Nagging employs two types of processes; a master process which attempts to sol... |

6 |
A Novel Asynchronous Parallelization Scheme for First-Order Logic
- Sturgill, Segre
- 1994
(Show Context)
Citation Context ...h may result in lower overall performance. 4.2 A Second Parallel Solution Based on our previous work on nagging, a parallel asynchronous search pruning technique for first-order logic theorem proving =-=[25, 21]-=-, we now propose a distributed strategy that addresses the shortcomings of the simple partitioning scheme just described. Nagging employs two types of processes; a master process which attempts to sol... |

4 |
Application of a real-time control strategy for Bayesian belief networks to ship classification problem solving
- MUSMAN, CHANG, et al.
- 1993
(Show Context)
Citation Context ... and flexible knowledge representation scheme: ffl Image Understanding: A system developed at Naval Research Laboratory employs Bayesian networks to perform ship classification from raw sensor images =-=[18]-=-. ffl Forecasting: The ARCO1 system is able to both reason about and forecast the crude oil market [1]. ffl Information Retrieval: A Bayesian network is used to retrieve documents relevant to a partic... |

3 |
Theory refinement on Baysian networks
- Buntine
- 1991
(Show Context)
Citation Context |