## On the utility of graphics cards to perform massively parallel simulation of advanced Monte Carlo methods (2010)

Venue: | Journal of Computational and Graphical Statistics |

Citations: | 20 - 5 self |

### BibTeX

@ARTICLE{Lee10onthe,

author = {Anthony Lee and Christopher Yau and Michael B. Giles and Arnaud Doucet and Christopher C. Holmes},

title = {On the utility of graphics cards to perform massively parallel simulation of advanced Monte Carlo methods},

journal = {Journal of Computational and Graphical Statistics},

year = {2010},

pages = {769--789}

}

### OpenURL

### Abstract

We present a case-study on the utility of graphics cards to perform massively parallel simulation of advanced Monte Carlo methods. Graphics cards, containing multiple Graphics Processing Units (GPUs), are self-contained parallel computational devices that can be housed in conventional desktop and laptop computers. For certain classes of Monte Carlo algorithms they offer massively parallel simulation, with the added advantage over conventional distributed multi-core processors that they are cheap, easily accessible, easy to maintain, easy to code, dedicated local devices with low power consumption. On a canonical set of stochastic simulation examples including population-based Markov chain Monte Carlo methods and Sequential Monte Carlo methods, we find speedups from 35 to 500 fold over conventional single-threaded computer code. Our findings suggest that GPUs have the potential to facilitate the growth of statistical modelling into complex data rich domains through the availability of cheap and accessible many-core computation. We believe the speedup we observe should motivate wider

### Citations

1073 | Finite Mixture Models - Mclachlan, Peel - 2000 |

441 |
Monte Carlo Strategies in Scientific Computing
- Liu
- 2001
(Show Context)
Citation Context ...erately from parallelization. However, the bulk of the speedup will generally come from the parallelization of the evolution and weighting steps. Therefore, using criteria like effective sample size (=-=Liu 2008-=-) to avoid resampling at every time step can also improve speedup. 83.3 Sequential Monte Carlo Samplers SMC samplers (Del Moral et al. 2006) are a more general class of methods that utilize a sequenc... |

212 |
Markov chain Monte Carlo maximum likelihood. Pages 156–163 in Computing science and statistics
- Geyer
- 1991
(Show Context)
Citation Context ...tationary MCMC kernels without affecting the stationary distribution of the joint chain Tierney (1994), we can allow certain types of interaction between the subchains which can speed up convergence (=-=Geyer 1991-=-; Hukushima and Nemoto 1996). In general, we apply a series of kernels that act on subsets of the variables. For the sake of clarity, let us denote the number of second-stage kernels by R and the kern... |

162 | Annealed Importance Sampling - Neal - 2001 |

115 | Computational and inferential difficulties with mixture posterior distributions
- Celeux, Hurn, et al.
- 2000
(Show Context)
Citation Context ...which should all 13be represented in the samples. Basic random-walk MCMC and importance sampling methods typically fail to provide a correct approximation of the posterior for practical values of N (=-=Celeux et al. 2000-=-). It should be noted that while it might not be necessary to sample from all the symmetric modes in the case of a mixture model, the successful traversal of all the modes suggests that the sampler wo... |

93 |
Blind deconvolution via sequential imputations
- Liu, Chen
- 1995
(Show Context)
Citation Context ...erately from parallelization. However, the bulk of the speedup will generally come from the parallelization of the evolution and weighting steps. Therefore, using criteria like effective sample size (=-=Liu and Chen 1995-=-) to avoid resampling at every time step can also improve speedup. 3.3 Sequential Monte Carlo Samplers SMC samplers (Del Moral et al. 2006) are a more general class of methods that utilize a sequence ... |

76 | A Tutorial on Particle Filtering and Smoothing: Fifteen years later - Doucet, Johansen |

67 | Exchange Monte Carlo method and application to spin glass simulations
- Hukushima, Nemoto
(Show Context)
Citation Context ...MC kernels without affecting the stationary distribution of the joint chain Tierney (1994), we can allow certain types of interaction between the subchains which can speed up convergence (Geyer 1991; =-=Hukushima and Nemoto 1996-=-). In general, we apply a series of kernels that act on subsets of the variables. For the sake of clarity, let us denote the number of second-stage kernels by R and the kernels themselves as K1,...,KR... |

61 | Accelerating molecular modeling applications with graphics processors - Stone, Phillips, et al. |

58 | 2004: Monte Carlo Statistical Methods: Second Edition - Robert, Casella |

57 | An Object-Oriented Random-Number Package With Many Long Streams and Substreams - L’Ecuyer, Simard, et al. |

52 |
1999b. Time varying covariances: a factor stochastic volatility approach
- Pitt, Shephard
(Show Context)
Citation Context ...ng a dynamic latent factor model. In such models, all the variances and covariances are modelled through a low dimensional stochastic volatility structure driven by common factors (Liu and West 2000; =-=Pitt and Shephard 1999-=-). We consider here a factor stochastic volatility model most similar to that proposed in Liu and West (2000): yt ∼ N(Bft,Ψ) ft ∼ N(0,Ht) xt ∼ N(Φxt−1,U) 18where Ψ � diag(ψ1,...,ψM) Ht � diag(exp(xt)... |

45 | Xorshift RNGs - Marsaglia |

44 | The accuracy of floating point summation - Higham - 1993 |

44 | Nonequilibrium measurements of free energy differences for microscopically reversible markovian systems - Crooks - 1998 |

31 | Parameter Sets for Combined Multiple Recursive Random Number Generators, Shorter version - L’Ecuyer, Good - 1999 |

24 | 2007b) On population-based simulation for static inference - Jasra, Stephens, et al. |

22 | Parallel computing and monte carlo algorithms. Far East Journal of Theoretical Statistics 4:207–236 - Rosenthal |

16 | Accelerating molecular dynamic simulation on graphics processing units - Friedrichs, Eastman, et al. |

10 | Many-core algorithms for statistical phylogenetics - Suchard, Rambaut - 2009 |

8 | Particle Markov chain Monte Carlo (with discussion - Andrieu, Doucet, et al. |

8 | The incomplete beta function law for parallel tempering sampling of classical canonical systems - PREDESCU, PREDESCU, et al. - 2004 |

7 | Bayesian inference for mixture models via Monte Carlo - Jasra - 2005 |

7 | Handbook of Parallel Computing and Statistics - Kontoghiorghes - 2006 |

1 | Parallel Processing in Markov chain Monte Carlo Simulation by Pre-Fetching - Brockwell - 2006 |

1 | Nonequilibrium measurements of free energy differences for microscopically reversible Markovian systems - E - 1998 |