Results 11  20
of
51
A new look at statespace models for neural data
 Journal of Computational Neuroscience
, 2010
"... State space methods have proven indispensable in neural data analysis. However, common methods for performing inference in statespace models with nonGaussian observations rely on certain approximations which are not always accurate. Here we review direct optimization methods that avoid these appro ..."
Abstract

Cited by 28 (19 self)
 Add to MetaCart
State space methods have proven indispensable in neural data analysis. However, common methods for performing inference in statespace models with nonGaussian observations rely on certain approximations which are not always accurate. Here we review direct optimization methods that avoid these approximations, but that nonetheless retain the computational efficiency of the approximate methods. We discuss a variety of examples, applying these direct optimization techniques to problems in spike train smoothing, stimulus decoding, parameter estimation, and inference of synaptic properties. Along the way, we point out connections to some related standard statistical methods, including spline smoothing and isotonic regression. Finally, we note that the computational methods reviewed here do not in fact depend on the statespace setting at all; instead, the key property we are exploiting involves the bandedness of certain matrices. We close by discussing some applications of this more general point of view, including Markov chain Monte Carlo methods for neural decoding and efficient estimation of spatiallyvarying firing rates.
A survey of Monte Carlo algorithms for maximizing the likelihood of a twostage hierarchical model
, 2001
"... Likelihood inference with hierarchical models is often complicated by the fact that the likelihood function involves intractable integrals. Numerical integration (e.g. quadrature) is an option if the dimension of the integral is low but quickly becomes unreliable as the dimension grows. An alternati ..."
Abstract

Cited by 10 (4 self)
 Add to MetaCart
Likelihood inference with hierarchical models is often complicated by the fact that the likelihood function involves intractable integrals. Numerical integration (e.g. quadrature) is an option if the dimension of the integral is low but quickly becomes unreliable as the dimension grows. An alternative approach is to approximate the intractable integrals using Monte Carlo averages. Several dierent algorithms based on this idea have been proposed. In this paper we discuss the relative merits of simulated maximum likelihood, Monte Carlo EM, Monte Carlo NewtonRaphson and stochastic approximation. Key words and phrases : Eciency, Monte Carlo EM, Monte Carlo NewtonRaphson, Rate of convergence, Simulated maximum likelihood, Stochastic approximation All three authors partially supported by NSF Grant DMS0072827. 1 1
Maximum likelihood estimation via the ECM algorithm: Computing the asymptotic variance
, 1994
"... Abstract: This paper provides detailed theory, algorithms, and illustrations for computing asymptotic variancecovariance matrices for maximum likelihood estimates using the ECM algorithm (Meng and Rubin (1993)). This Supplemented ECM (SECM) algorithm is developed as an extension of the Supplemented ..."
Abstract

Cited by 9 (2 self)
 Add to MetaCart
Abstract: This paper provides detailed theory, algorithms, and illustrations for computing asymptotic variancecovariance matrices for maximum likelihood estimates using the ECM algorithm (Meng and Rubin (1993)). This Supplemented ECM (SECM) algorithm is developed as an extension of the Supplemented EM (SEM) algorithm (Meng and Rubin (1991a)). Explicit examples are given, including one that demonstrates SECM, like SEM, has a powerful internal error detecting system for the implementation of the parent ECM or of SECM itself.
Crossfertilizing strategies for better EM mountain climbing and DA field exploration: A graphical guide book
, 2009
"... In recent years, a variety of extensions and refinements have been developed for data augmentation based model fitting routines. These developments aim to extend the application, improve the speed, and/or simplify the implementation of data augmentation methods, such as the deterministic EM algorith ..."
Abstract

Cited by 8 (5 self)
 Add to MetaCart
In recent years, a variety of extensions and refinements have been developed for data augmentation based model fitting routines. These developments aim to extend the application, improve the speed, and/or simplify the implementation of data augmentation methods, such as the deterministic EM algorithm for mode finding and stochastic Gibbs sampler and other auxiliaryvariable based methods for posterior sampling. In this overview article we graphically illustrate and compare a number of these extensions all of which aim to maintain the simplicity and computation stability of their predecessors. We particularly emphasize the usefulness of identifying similarities between the deterministic and stochastic counterparts as we seek more efficient computational strategies. We also demonstrate the applicability of data augmentation methods for handling complex models
Markov Chain Monte Carlo Methods in Biostatistics
 Statistical Methods in Medical Research 5:339355
, 1996
"... this article, we review some important general methods for Markov chain Monte Carlo ..."
Abstract

Cited by 7 (0 self)
 Add to MetaCart
this article, we review some important general methods for Markov chain Monte Carlo
Maximum Likelihood Estimation of Factor Analysis Using the ECME Algorithm with Complete and Incomplete Data
 Statist. Sinica
, 1998
"... Factor analysis is a standard tool in educational testing contexts, which can be fit using the EM algorithm (Dempster, Laird, and Rubin, 1977). An extension of EM, called the ECME algorithm (Liu and Rubin, 1994), can be used to obtain ML estimates more efficiently in factor analysis models. ECME has ..."
Abstract

Cited by 5 (2 self)
 Add to MetaCart
Factor analysis is a standard tool in educational testing contexts, which can be fit using the EM algorithm (Dempster, Laird, and Rubin, 1977). An extension of EM, called the ECME algorithm (Liu and Rubin, 1994), can be used to obtain ML estimates more efficiently in factor analysis models. ECME has an Estep, identical to the Estep of EM, but instead of EM's Mstep, it has a sequence of CM (conditional maximization) steps, each of which maximizes Either the constrained expected completedata loglikelihood, as with the ECM algorithm (Meng and Rubin, 1993), or the constrained actual loglikelihood. For factor analysis, we use two CM steps: the first maximizes the expected completedata loglikelihood over the factor loadings given fixed uniquenesses, and the second maximizes the actual likelihood over the uniquenesses given fixed factor loadings. We also describe EM and ECME for ML estimation of factor analysis from incomplete data, which arise in applications of factor analysis in educational testing contexts.
An Akaike Information Criterion for Model Selection in the Presence of Incomplete Data
, 1997
"... We derive and investigate a variant of AIC, the Akaike information criterion, for model selection in settings where the observed data is incomplete. Our variant is based on the motivation provided for the PDIO ("predictive divergence for incomplete observation models") criterion of Shimodaira (1994, ..."
Abstract

Cited by 4 (1 self)
 Add to MetaCart
We derive and investigate a variant of AIC, the Akaike information criterion, for model selection in settings where the observed data is incomplete. Our variant is based on the motivation provided for the PDIO ("predictive divergence for incomplete observation models") criterion of Shimodaira (1994, in Selecting Models from Data: Artificial Intelligence and Statistics IV, Lecture Notes in Statistics 89, SpringerVerlag, New York, 2129). However, our variant differs from PDIO in its "goodnessoffit" term. Unlike AIC and PDIO, which require the computation of the observeddata empirical loglikelihood, our criterion can be evaluated using only completedata tools, readily available through the EM algorithm and the SEM ("supplemented" EM) algorithm of Meng and Rubin (1991, Journal of the American Statistical Association 86, 899909). We compare the performance of our AIC variant to that of both AIC and PDIO in simulations where the data being modeled contains missing values. The results indicate that our criterion is less prone to overfitting than AIC and less prone to underfitting than PDIO. AMS Subject Classification: 62B10, 94A17, 62E25. Keywords: AIC, EM algorithm, information theory, KullbackLeibler information, model selection criteria, PDIO criterion, SEM algorithm. Abbreviated Title: An AIC for Model Selection with Incomplete Data. 1.
On computing the largest fraction of missing information for the EM algorithm and the worst linear function for data augmentation
"... We address the problem of computing the largest fraction of missing information for the EM algorithm and the worst linear function for data augmentation. These are the largest eigenvalue and its associated eigenvector for the Jacobian of the EM operator at a maximum likelihood estimate, which are im ..."
Abstract

Cited by 4 (0 self)
 Add to MetaCart
We address the problem of computing the largest fraction of missing information for the EM algorithm and the worst linear function for data augmentation. These are the largest eigenvalue and its associated eigenvector for the Jacobian of the EM operator at a maximum likelihood estimate, which are important for assessing convergence in iterative simulation. An estimate of the largest fraction of missing information is available from the EM iterates � this is often adequate since only a few gures of accuracy are needed. In some instances the EM iteration also gives an estimate of the worst linear function. We showthat improved estimates can be essential for proper inference. In order to obtain improved estimates e ciently, weuse the power method for eigencomputation. Unlike eigenvalue decomposition, the power method computes only the largest eigenvalue and eigenvector of a matrix, it can take advantage of a good eigenvector estimate as an initial value and it can be terminated after only a few gures of accuracy are achieved. Moreover, the matrix products needed in the power method can be computed by extrapolation, obviating the need to form the Jacobian of the EM operator. We give results of simulation studies on multivariate normal data showing that this approach becomes more e cient as the data dimension increases than methods that use a nitedi erence approximation to the Jacobian, which is the only generalpurpose alternative available.
To Center or Not To Center: That Is Not The Question
 in progress) Paul Baines 101909 Bayesian Computation in ColorMagnitude Diagrams
, 2009
"... For a broad class of multilevel models, there exist two wellknown competing parameterizations, the centered parametrization (CP) and the noncentered parametrization (NCP), for effective MCMC implementation. Much literature has been devoted to the questions of when to use which and how to compromi ..."
Abstract

Cited by 3 (0 self)
 Add to MetaCart
For a broad class of multilevel models, there exist two wellknown competing parameterizations, the centered parametrization (CP) and the noncentered parametrization (NCP), for effective MCMC implementation. Much literature has been devoted to the questions of when to use which and how to compromise between them via partial CP/NCP. This paper introduces an alternative strategy for boosting MCMC efficiency via simply interweaving— but not alternating—the two parameterizations. This strategy has the surprising property that failure of both the CP and NCP chains to converge geometrically does not prevent the interweaving algorithm from doing so. It achieves this seemingly magical property by taking advantage of the discordance of the two parameterizations, namely, the sufficiency of CP and the ancillarity of NCP, to substantially reduce the Markovian dependence, especially when the original CP and NCP form a “beauty and beast ” pair (i.e., when one chain mixes far more rapidly than the other). The ancillaritysufficiency reformulation of the CPNCP dichotomy allows us to borrow insight from the wellknown Basu’s theorem on the independence of (complete) sufficient and ancillary statistics, albeit a Bayesian version of Basu’s
On the estimation of nonrandom signal coefficients from jittered samples
 IEEE Trans. Signal Process
, 2011
"... Abstract—This paper examines the problem of estimating the parameters of a bandlimited signal from samples corrupted by random jitter (timing noise) and additive, independent identically distributed (i.i.d.) Gaussian noise, where the signal lies in the span of a finite basis. For the presented class ..."
Abstract

Cited by 3 (2 self)
 Add to MetaCart
Abstract—This paper examines the problem of estimating the parameters of a bandlimited signal from samples corrupted by random jitter (timing noise) and additive, independent identically distributed (i.i.d.) Gaussian noise, where the signal lies in the span of a finite basis. For the presented classical estimation problem, the Cramér–Rao lower bound (CRB) is computed, and an ExpectationMaximization (EM) algorithm approximating the maximum likelihood (ML) estimator is developed. Simulations are performed to study the convergence properties of the EM algorithm and compare the performance both against the CRB and a basic linear estimator. These simulations demonstrate that by postprocessing the jittered samples with the proposed EM algorithm, greater jitter can be tolerated, potentially reducing onchip ADC power consumption substantially. Index Terms—Analogtodigital converters, Cramér–Rao bound, EM algorithm, jitter, maximum likelihood estimator,