Results 1  10
of
18
Bayesian Adaptive Sampling for Variable Selection and Model Averaging
"... For the problem of model choice in linear regression, we introduce a Bayesian adaptive sampling algorithm (BAS), that samples models without replacement from the space of models. For problems that permit enumeration of all models BAS is guaranteed to enumerate the model space in 2 p iterations where ..."
Abstract

Cited by 9 (4 self)
 Add to MetaCart
For the problem of model choice in linear regression, we introduce a Bayesian adaptive sampling algorithm (BAS), that samples models without replacement from the space of models. For problems that permit enumeration of all models BAS is guaranteed to enumerate the model space in 2 p iterations where p is the number of potential variables under consideration. For larger problems where sampling is required, we provide conditions under which BAS provides perfect samples without replacement. When the sampling probabilities in the algorithm are the marginal variable inclusion probabilities, BAS may be viewed as sampling models “near ” the median probability model of Barbieri and Berger. As marginal inclusion probabilities are not known in advance we discuss several strategies to estimate adaptively the marginal inclusion probabilities within BAS. We illustrate the performance of the algorithm using simulated and real data and show that BAS can outperform Markov chain Monte Carlo methods. The algorithm is implemented in the R package BAS available at CRAN.
Darwinian Evolution in Parallel Universes: A Parallel Genetic Algorithm for Variable Selection
"... The need to identify a few important variables that affect a certain outcome of interest commonly arises in various industrial engineering applications. The genetic algorithm (GA) appears to be a natural tool for solving such a problem. In this article we first demonstrate that the GA is actually no ..."
Abstract

Cited by 5 (1 self)
 Add to MetaCart
The need to identify a few important variables that affect a certain outcome of interest commonly arises in various industrial engineering applications. The genetic algorithm (GA) appears to be a natural tool for solving such a problem. In this article we first demonstrate that the GA is actually not a particularly effective variable selection tool, and then propose a very simple modification. Our idea is to run a number of GAs in parallel without allowing each GA to fully converge, and to consolidate the information from all the individual GAs in the end. We call the resulting algorithm the parallel genetic algorithm (PGA). Using a number of both simulated and real examples, we show that the PGA is an interesting as well as highly competitive and easytouse variable selection tool.
Detecting Poor Convergence of Posterior Samplers due to Multimodality
"... Computation in Bayesian statistical models is often performed using sampling techniques such as Markov chain Monte Carlo (MCMC) or adaptive Monte Carlo methods. The convergence of the sampler to the posterior distribution is typically assessed using a set of standard diagnostics; recent draft Food ..."
Abstract

Cited by 3 (1 self)
 Add to MetaCart
Computation in Bayesian statistical models is often performed using sampling techniques such as Markov chain Monte Carlo (MCMC) or adaptive Monte Carlo methods. The convergence of the sampler to the posterior distribution is typically assessed using a set of standard diagnostics; recent draft Food and Drug Administration guidelines for the use of Bayesian statistics in medical device trials, for instance, advocate this approach for validating computations. We give several examples showing that this approach may be insufficient when the posterior distribution is multimodal–that lack of convergence due to posterior multimodality can be undetected using the standard convergence diagnostics, including the GelmanRubin diagnostic that was introduced for exactly this problem. We show that the poor convergence can be detected by modifying a validation technique that was originally proposed for detecting coding errors in MCMC soft
Distributed Evolutionary Monte Carlo with Applications to Bayesian Analysis

, 2005
"... Sampling from multimodal and high dimensional target distribution posits a great challenge in Bayesian analysis. This paper combines the attractive features of the distributed genetic algorithm and the Markov Chain Monte Carlo, resulting in a new Monte Carlo algorithm Distributed Evolutionary Monte ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
Sampling from multimodal and high dimensional target distribution posits a great challenge in Bayesian analysis. This paper combines the attractive features of the distributed genetic algorithm and the Markov Chain Monte Carlo, resulting in a new Monte Carlo algorithm Distributed Evolutionary Monte Carlo (DEMC) for realvalued problems. DEMC evolves a population of the Markov chains through genetic operators to explore the target function efficiently. The promising potential of the DEMC algorithm is illustrated by applying it to multimodal samples, Bayesian Neural Network and logistic regression inference.
Convergence Rate of Markov Chain Methods for Genomic Motif Discovery
, 2011
"... We analyze the convergence rate of a popular Gibbs sampling method used for statistical discovery of gene regulatory binding motifs in DNA sequences. This sampler satisfies a very strong form of ergodicity (uniform). However, we show that, due to multimodality of the posterior distribution, the rate ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
We analyze the convergence rate of a popular Gibbs sampling method used for statistical discovery of gene regulatory binding motifs in DNA sequences. This sampler satisfies a very strong form of ergodicity (uniform). However, we show that, due to multimodality of the posterior distribution, the rate of convergence often decreases exponentially as a function of the length of the DNA sequence. Specifically, we show that this occurs whenever there is more than one true repeating pattern in the data. In practice there are typically multiple, even numerous, such patterns in biological data, the goal being to detect the most wellconserved and frequentlyoccurring of these. Our findings match empirical results, in which the motifdiscovery Gibbs sampler has exhibited such poor convergence that it is used only for finding modes of the posterior distribution (candidate motifs) rather than for obtaining samples from that distribution. Ours appear to be the first meaningful bounds on the convergence rate of a Markov chain method for sampling from a multimodal posterior distribution, as a function of statistical quantities like the number of observations. Keywords: Gibbs sampler; DNA; slow mixing; spectral gap; binding motifs; multimodal.
Chapter 8 Regulatory Motif Discovery: from Decoding to MetaAnalysis
"... Gene transcription is regulated by interactions between transcription factors and their target binding sites in the genome. A motif is the sequence pattern recognized by a transcription factor to mediate such interactions. With the availability of highthroughput genomic data, computational identifi ..."
Abstract
 Add to MetaCart
Gene transcription is regulated by interactions between transcription factors and their target binding sites in the genome. A motif is the sequence pattern recognized by a transcription factor to mediate such interactions. With the availability of highthroughput genomic data, computational identification of transcription factor binding motifs has become a major research problem in computational biology and bioinformatics. In this chapter, we present a series of Bayesian approaches to motif discovery. We start from a basic statistical framework for motif finding, extend it to the identification of cisregulatory modules, and then discuss methods that combine motif finding with phylogenetic footprinting, gene expression or ChIPchip data, and nucleosome positioning information. Simulation studies and applications to biological data sets are presented to illustrate the utility of these methods.
Project with the Breast Cancer Surveillance Consortium, incl. Joann
"... Biostatistics), Alan Gelfand (Duke) When a mammogram is performed, a radiologist decides whether to recall the patient for further testing There is concern about inconsistency in this recall decision between radiologists Database of 500,000+ mammograms Demographic characteristics of the patient Outc ..."
Abstract
 Add to MetaCart
Biostatistics), Alan Gelfand (Duke) When a mammogram is performed, a radiologist decides whether to recall the patient for further testing There is concern about inconsistency in this recall decision between radiologists Database of 500,000+ mammograms Demographic characteristics of the patient Outcome of the mammogram (false +, true +, false, or true) Radiologist data Practice characteristics Demographic characteristics Concerns about malpractice
Population Stochastic Approximation MCMC Algorithm and its Weak Convergence
, 2010
"... In this paper, we propose a population stochastic approximation MCMC (SAMCMC) algorithm, and establish its weak convergence (toward a normal distribution) under mild conditions. The theory of weak convergence established for the population SAMCMC algorithm is also applicable for general single chain ..."
Abstract
 Add to MetaCart
In this paper, we propose a population stochastic approximation MCMC (SAMCMC) algorithm, and establish its weak convergence (toward a normal distribution) under mild conditions. The theory of weak convergence established for the population SAMCMC algorithm is also applicable for general single chain SAMCMC algorithms. Based on the theory, we then give an explicit ratio for the convergence rates of the population SAMCMC algorithm and the single chain SAMCMC algorithm. The theoretical results are illustrated by a population stochastic approximation Monte Carlo (SAMC) algorithm with a multimodal example. Our results, in both theory and numerical examples, suggest that the population SAMCMC algorithm can be more efficient than the single chain SAMCMC algorithm. This is of interest for practical applications.
Variable Architecture Bayesian Neural Networks
"... monte carlo algorithm. A crucial problem which arises when dealing with Bayesian neural networks is that of determining their most appropriate size, expressed in terms of number of computational units and/or connections. In fact, too small a network may not be able to learn the sample data, whereas ..."
Abstract
 Add to MetaCart
monte carlo algorithm. A crucial problem which arises when dealing with Bayesian neural networks is that of determining their most appropriate size, expressed in terms of number of computational units and/or connections. In fact, too small a network may not be able to learn the sample data, whereas one that is too large may give rise to overfitting phenomena and cause poor “generalization ” performance. A few solutions have been proposed in the literature to solve this problem, such as the use of a geometric prior probability on the number of hidden units (Müller and Rios Insua, 1998), thereby favouring smallersize networks, and a reversible jump algorithm to move between architectures having a different number of hidden units (Rios Insua and Müller, 1998). In this work we propose a variable architecture model where inputtohidden connections and, therefore, hidden units are selected by using a variant of the Evolutionary Monte Carlo (EMC) algorithm developed by Liang and Wong (2000). The
Some connections between Bayesian and nonBayesian methods for regression model selection
, 2001
"... In this article, we study the connections between Bayesian methods and nonBayesian methods for variable selection in multiple linear regression. We show that each ofthe nonBayesian criteria, FPE; AIC; Cp and adjusted R 2, has its Bayesian correspondence under an appropriate prior setting. The theo ..."
Abstract
 Add to MetaCart
In this article, we study the connections between Bayesian methods and nonBayesian methods for variable selection in multiple linear regression. We show that each ofthe nonBayesian criteria, FPE; AIC; Cp and adjusted R 2, has its Bayesian correspondence under an appropriate prior setting. The theoretical results are