@MISC{Lebanon09asymptoticefficiency, author = {Guy Lebanon}, title = {Asymptotic Efficiency of the Maximum Likelihood Estimator}, year = {2009} }

Share

OpenURL

Abstract

In this note we provide a short proof based on [1] and [2] for the asymptotic normality and efficiency of the multivariate maximum likelihood estimator (mle). Asymptotic efficiency refers to the situation when the asymptotic variance equals the inverse Fisher information which is the best possible variance (Cramer-Rao lower bound). It is assumed that the reader is familiar with the notes on Relative Efficiency, Efficiency, and the Fisher Information and Consistency of the Maximum Likelihood Estimator. We assume that (i) X1, X2,... are sampled iid from pθ0, which is assumed to be continuous and twice differentiable in θ. We also assume that (ii) the parameter space Θ ⊂ Rk is a convex open set. For a function g: Rk → R we denote by ∇g(z) its gradient k × 1 vector and by ∇2g(z) the k × k matrix containing its second order derivatives i.e. [∇2g(z)]ij = ∂2 ∂zi∂zj g(z). The Fisher information is defined as the k × k matrix J(θ) = E pθ {∇θ log pθ(X)(∇θ log pθ(X)) ⊤}. It is well known that J(θ) = −E pθ {∇2 θ log pθ(X)} and E pθ {∇θ log pθ(X)} = 0 (see the note Relative Efficiency, Efficiency, and the Fisher Information for the one dimensional case). The following theorem establishes the asymptotic normality of the mle and its efficiency by demonstrating that the asymptotic variance is the inverse Fisher information. Proposition 1 (Cramer). In addition to the assumptions above, we assume that (iii) there exists a function K(x) such that E pθ K(X) < ∞ and each component of ∇θ log pθ(x) is bounded in absolute value by K(x) 0 uniformly in some neighborhood of θ0, (iv) J(θ0) is a positive definite matrix, and (v) identifiability i.e. pθ ≡ pθ0 ⇔ θ = θ0. Then there exists a strongly consistent sequence ˆ θn of likelihood (local) maximizers for which the following convergence in distribution holds n ( θn