Results 11  20
of
114
Ascentbased Monte Carlo EM
, 2004
"... The EM algorithm is a popular tool for maximizing likelihood functions in the presence of missing data. Unfortunately, EM often requires the evaluation of analytically intractable and highdimensional integrals. The Monte Carlo EM (MCEM) algorithm is the natural extension of EM that employs Monte Ca ..."
Abstract

Cited by 12 (7 self)
 Add to MetaCart
The EM algorithm is a popular tool for maximizing likelihood functions in the presence of missing data. Unfortunately, EM often requires the evaluation of analytically intractable and highdimensional integrals. The Monte Carlo EM (MCEM) algorithm is the natural extension of EM that employs Monte Carlo methods to estimate the relevant integrals. Typically, a very large Monte Carlo sample size is required to estimate these integrals within an acceptable tolerance when the algorithm is near convergence. Even if this sample size were known at the onset of implementation of MCEM, its use throughout all iterations is wasteful, especially when accurate starting values are not available. We propose a datadriven strategy for controlling Monte Carlo resources in MCEM. The proposed algorithm improves on similar existing methods by: (i) recovering EM’s ascent (i.e., likelihoodincreasing) property with high probability, (ii) being more robust to the impact of user defined inputs, and (iii) handling classical Monte Carlo and Markov chain Monte Carlo within a common framework. Because of (i) we refer to the algorithm as “Ascentbased MCEM”. We apply Ascentbased MCEM to a variety of examples, including one where it is used to dramatically accelerate the convergence of deterministic EM.
QuasiMonte Carlo sampling to improve the efficiency of Monte Carlo EM
 Computational Statistics and Data Analysis
, 2005
"... In this paper we investigate an efficient implementation of the Monte Carlo EM algorithm based on QuasiMonte Carlo sampling. The Monte Carlo EM algorithm is a stochastic version of the deterministic EM (ExpectationMaximization) algorithm in which an intractable Estep is replaced by a Monte Carlo ..."
Abstract

Cited by 11 (5 self)
 Add to MetaCart
(Show Context)
In this paper we investigate an efficient implementation of the Monte Carlo EM algorithm based on QuasiMonte Carlo sampling. The Monte Carlo EM algorithm is a stochastic version of the deterministic EM (ExpectationMaximization) algorithm in which an intractable Estep is replaced by a Monte Carlo approximation. QuasiMonte Carlo methods produce deterministic sequences of points that can significantly improve the accuracy of Monte Carlo approximations over purely random sampling. One drawback to deterministic QuasiMonte Carlo methods is that it is generally difficult to determine the magnitude of the approximation error. However, in order to implement the Monte Carlo EM algorithm in an automated way, the ability to measure this error is fundamental. Recent developments of randomized QuasiMonte Carlo methods can overcome this drawback. We investigate the implementation of an automated, datadriven Monte Carlo EM algorithm based on randomized QuasiMonte Carlo methods. We apply this algorithm to a geostatistical model of online purchases and find that it can significantly decrease the total simulation effort, thus showing great potential for improving upon the efficiency of the classical Monte Carlo EM algorithm. Key words and phrases: Monte Carlo error; lowdiscrepancy sequence; Halton sequence; EM algorithm; geostatistical model.
The EM Algorithm, Its Stochastic Implementation and Global Optimization: Some Challenges and Opportunities for OR
, 2006
"... The EM algorithm is a very powerful optimization method and has reached popularity in many fields. Unfortunately, EM is only a local optimization method and can get stuck in suboptimal solutions. While more and more contemporary data/model combinations yield more than one optimum, there have been on ..."
Abstract

Cited by 10 (4 self)
 Add to MetaCart
(Show Context)
The EM algorithm is a very powerful optimization method and has reached popularity in many fields. Unfortunately, EM is only a local optimization method and can get stuck in suboptimal solutions. While more and more contemporary data/model combinations yield more than one optimum, there have been only very few attempts at making EM suitable for global optimization. In this paper we review the basic EM algorithm, its properties and challenges and we focus in particular on its stochastic implementation. The stochastic EM implementation promises relief to some of the contemporary data/model challenges and it is particularly wellsuited for a wedding with global optimization ideas since most global optimization paradigms are also based on the principles of stochasticity. We review some of the challenges of the stochastic EM implementation and propose a new algorithm that combines the principles of EM with that of the Genetic Algorithm. While this new algorithm shows some promising results for clustering of an online auction database of functional objects, the primary goal of this work is to bridge a gap between the field of statistics, which is home to extensive research on the EM algorithm, and the field of operations research, in which work on global optimization thrives, and to stir new ideas for joint research between the two.
Personalized Recommendation of User Comments via Factor Models
"... In recent years, the amount of usergenerated opinionated texts (e.g., reviews, user comments) continues to grow at a rapid speed: featured news stories on a major event easily attract thousands of user comments on a popular online News service. How to consume subjective information of this volume b ..."
Abstract

Cited by 9 (0 self)
 Add to MetaCart
(Show Context)
In recent years, the amount of usergenerated opinionated texts (e.g., reviews, user comments) continues to grow at a rapid speed: featured news stories on a major event easily attract thousands of user comments on a popular online News service. How to consume subjective information of this volume becomes an interesting and important research question. In contrast to previous work on review analysis that tried to filter or summarize information for a generic average user, we explore a different direction of enabling personalized recommendation of such information. For each user, our task is to rank the comments associated with a given article according to personalized user preference (i.e., whether the user is likely to like or dislike the comment). To this end, we propose a factor model that incorporates ratercomment and raterauthor interactions simultaneously in a principled way. Our full model significantly outperforms strong baselines as well as related models that have been considered in previous work. 1
Hierarchical Models: A Current Computational Perspective
 JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION
, 2000
"... Hierarchical models (HMs) provide a flexible framework for modeling data. The ongoing development of techniques like the EM algorithm and Markov chain Monte Carlo has enabled statisticians to make use of increasingly more complicated HMs over the last few decades. In this article, we consider Bay ..."
Abstract

Cited by 9 (1 self)
 Add to MetaCart
Hierarchical models (HMs) provide a flexible framework for modeling data. The ongoing development of techniques like the EM algorithm and Markov chain Monte Carlo has enabled statisticians to make use of increasingly more complicated HMs over the last few decades. In this article, we consider Bayesian and frequentist versions of a general, twostage HM, and describe several examples from the literature that illustrate its versatility. Some key aspects of the computational techniques that are currently used in conjunction with this HM are then examined in the context of McCullagh and Nelder's (1989) salamander data. Several areas that are ripe for new research are identified.
Analysis of spatial data using generalized linear mixed models and Langevintype Markov chain Monte Carlo
, 2000
"... Markov chain Monte Carlo methods are useful in connection with inference and prediction for spatial generalized linear mixed models, where the unobserved random effects constitute a spatially correlated Gaussian random field. We point out that socalled Langevintype updates are useful for Metropoli ..."
Abstract

Cited by 9 (3 self)
 Add to MetaCart
Markov chain Monte Carlo methods are useful in connection with inference and prediction for spatial generalized linear mixed models, where the unobserved random effects constitute a spatially correlated Gaussian random field. We point out that socalled Langevintype updates are useful for MetropolisHastings simulation of the posterior distribution of the random eects given the data. Furthermore, we discuss the use of improper priors in Bayesian analysis of spatial generalized linear mixed models with particular emphasis on the socalled Poissonlog normal model. For this and certain other models nonparametric estimation of the covariance function of the Gaussian field is also studied. The methods are applied to various data sets including counts of weed plants on a field.
Printed in Great Britain
"... The joint modeling of a longitudinal disease progression marker and the failure time process in the presence of cure ..."
Abstract

Cited by 9 (0 self)
 Add to MetaCart
The joint modeling of a longitudinal disease progression marker and the failure time process in the presence of cure
MONTE CARLO LIKELIHOOD INFERENCE FOR MISSING DATA MODELS
"... We describe a Monte Carlo method to approximate the maximum likelihood estimate (MLE), when there are missing data and the observed data likelihood is not available in closed form. This method uses simulated missing data that are independent and identically distributed and independent of the observe ..."
Abstract

Cited by 8 (3 self)
 Add to MetaCart
(Show Context)
We describe a Monte Carlo method to approximate the maximum likelihood estimate (MLE), when there are missing data and the observed data likelihood is not available in closed form. This method uses simulated missing data that are independent and identically distributed and independent of the observed data. Our Monte Carlo approximation to the MLE is a consistent and asymptotically normal estimate of the minimizer θ ∗ of the KullbackLeibler information, as both Monte Carlo and observed data sample sizes go to infinity simultaneously. Plugin estimates of the asymptotic variance are provided for constructing confidence regions for θ ∗. We give LogitNormal generalized linear mixed model examples, calculated using an R package. AMS 2000 subject classifications. Primary 62F12; secondary 65C05. Key words and phrases. Asymptotic theory, Monte Carlo, maximum likelihood, generalized
Negative Binomial Loglinear Mixed Models
"... The Poisson loglinear model is a common choice for explaining variability in counts. However, in many practical circumstances the restriction that the mean and variance are equal is not realistic. Overdispersion with respect to the Poisson distribution can be modeled explicitly by integrating with r ..."
Abstract

Cited by 7 (1 self)
 Add to MetaCart
(Show Context)
The Poisson loglinear model is a common choice for explaining variability in counts. However, in many practical circumstances the restriction that the mean and variance are equal is not realistic. Overdispersion with respect to the Poisson distribution can be modeled explicitly by integrating with respect to a mixture distribution, and use of the conjugate gamma mixing distribution leads to a negative binomial loglinear model. This paper extends the negative binomial loglinear model to the case of dependent counts, where dependence among the counts is handled by including linear combinations of random eects in the linear predictor. If we assume that the vector of random eects is multivariate normal, then arbitrary forms of dependence can be modeled by appropriate specication of the covariance structure. Although the likelihood function for the resulting model is not tractable, maximum likelihood estimates (and standard errors) can be found using the NLMIXED procedure in SAS or, in more complicated examples, using a Monte Carlo EM algorithm. An alternate approach is to leave the random eects completely unspecied and attempt to estimate them using nonparametric maximum likelihood. The methodologies are illustrated with several examples. Key words and phrases: Monte Carlo EM; NLMIXED procedure; Nonparametric maximum likelihood; Overdispersion; Random eects; Booth and Hobert's research supported by NSF Grant DMS0072827. Casella's research supported by NSF Grant DMS9971586. 1 1