Abstract-In the absence of knowledge of the true density function, Bayesian models take the joint density function for a sequence of n random variables to be an average of densities with respect to a prior. We examine the relative entropy distance D,, between the true density and the Bayesian density and show that the asymptotic distance is (d/2Xlogn)+ c, where d is the dimension of the parameter vector. Therefore, the relative entropy rate D,,/n converges to zero at rate (logn)/n. The constant c, which we explicitly identify, depends only on the prior density function and the Fisher information matrix evaluated at the true parameter value. Consequences are given for density estima-tion, universal data compression, composite hypothesis testing, and stock-market portfolio selection. 1.

We study two estimators of the mean function of a counting process based on "panel count data". The setting for "panel count data" is one in which n independent subjects, each with a counting process with common mean function, are observed at several possibly di erent times during a study. Following a model proposed by Schick and Yu (1997), we allow the number of observation times, and the observation times themselves, to be random variables. Our goal is to estimate the mean function of the counting process. We show that the estimator of the mean function proposed by Sun and Kalbfleisch (1995) can be viewed as a pseudo-maximum likelihood estimator when a nonhomogeneous Poisson process model is assumed for the counting process. We establish consistency of both the nonparametric pseudo maximum likelihood estimator of Sun and Kalbfleisch (1995) and the full maximum likelihood estimator, even if the underlying counting process is not a Poisson process. We also derive the asymptotic distribution of both estimators at a xed time t, and compare the resulting theoretical relative e ciency with nite sample relative efficiency by way of a limited monte-carlo study.

∗ Signatures are on file in the Graduate School. This dissertation presents two topics from opposite disciplines: one is from a parametric realm and the other is based on nonparametric methods. The first topic is a jackknife maximum likelihood approach to statistical model selection and the second one is a convex hull peeling depth approach to nonparametric massive multivariate data analysis. The second topic includes simulations and applications on massive astronomical data. First, we present a model selection criterion, minimizing the Kullback-Leibler distance by using the jackknife method. Various model selection methods have been developed to choose a model of minimum Kullback-Liebler distance to the true model, such as Akaike information criterion (AIC), Bayesian information criterion (BIC), Minimum description length (MDL), and Bootstrap information criterion. Likewise, the jackknife method chooses a model of minimum Kullback-Leibler distance through bias reduction. This bias, which is inevitable in model

this paper, we study the behaviour of the posterior distribution as the sample size n tends to infinity where the dimension of the parameter space p = p n is also allowed to grow to infinity with n. This problem is of significant practical importance since in data analysis, one often uses a delicate model (i.e., with a large number of parameters) if one has enough data. In other words, one allows the dimension of the parameter to grow with the sample size. Moreover, nonparametric models can be approximated by parametric models with increasing dimension as discussed by Shibata (1981) and Diaconis and Freedman (1993). The frequentist version of this problem, namely consistency and asymptotic normality of M-estimates has been studied by Huber (1973), Yohai and Maronna (1979), Ringland (1983) and Portnoy (1984, 1985, 1986). In this paper we show that, under certain growth restrictions on the dimension depending on the design variables, the posterior distributions concentrate in the neighbourhoods of the true value of the parameter and admit a normal approximation. It seems that the present paper is the first attempt to study Bayesian asymptotic properties in models of increasing dimension. We observe that the condition required on the growth rate of the dimension p n is more stringent than its frequentist counterparts. Though no claim is made about the necessity of this condition on the growth of p n , we believe that there are at least three reasons to expect some difficulties if p n grows very fast with n. First, there is a long tail area which may substantially contribute to the posterior probabilities although the likelihood is small there. Secondly, our choice of the L

Let X k be a sequence of iid random variables taking values in a finite set, and consider the problem of estimating the law of X 1 in a Bayesian framework. We prove that the sequence of posterior distributions satisfies a large deviation principle, and give an explicit expression for the rate function. As an application, we obtain an asymptotic formula for the predictive probability of ruin in the classical gambler's ruin problem. 1 Introduction and preliminaries Let X be a Hausdorff topological space with Borel oe-algebra B, and let ¯ n be a sequence of probability measures on (X ; B). A rate function is a nonnegative lower semicontinuous function on X . We say that the sequence ¯ n satisfies the large deviation principle (LDP) with rate function I, if for all B 2 B, \Gamma inf x2B ffi I(x) lim inf n 1 n log ¯ n (B) lim sup n 1 n log ¯ n (B) \Gamma inf x2 ¯ B I(x): Here B ffi and ¯ B denote the interior and closure of B, respectively. Let\Omega be a finite set and ...

In learning problems available information is usually divided into two categories: examples of function values (or training data) and prior information (e.g. a smoothness constraint).

Semiparametric maximum likelihood estimators have recently been proposed for a class of two-phase, outcome dependent sampling models. All of them were "restricted" maximum likelihood estimators, in the sense that the maximization is carried out only over distributions concentrated on the observed values of the covariate vectors. In this paper, the authors give conditions for consistency of these restricted maximum likelihood estimators. They also consider the corresponding unrestricted maximization problems, in which the "absolute" maximum likelihood estimators may then have support on additional points in the covariate space. Their main consistency result also covers these unrestricted maximum likelihood estimators, when they exist for all sample sizes.

The problem of estimating parameters of randomly perturbed PDE's is considered. ML estimators based on discrete time sampling of M observable Fourier coefficients of the random field governed by the stochastic PDE in question are studied. Necessary and sufficient conditions are given for the consistency, asymptotic normality and asymptotic efficiency of the ML estimators when M !1. These conditions are given in terms of simple properties of the operators involved in the equation and are easy to check. Key words: parameter estimation, SPDE, ML Introduction We consider the problem of estimating the unknown scalar parameter ` from partial observations of the random field u(t; x) governed by the equation @u=@t + (A 0 + `A 1 )u = S(t; x) (1) where x 2 G; G is a bounded region in the d-dimensional Euclidean space R d ; A k ; k = 0; 1 are linear operators and S(t; x) is the Gaussian "white noise" in t. We assume that only the amplitudes um (t n ); n = 1; : : : ; N m = 1; : : : ; M (2...

Based on concentration probability of estimators about a true parameter, third-order asymptotic efficiency of the first-order bias-adjusted MLE within the class of first-order bias-adjusted estimators has been well established in a variety of probability models. In this paper we consider the class of second-order bias-adjusted Fisher consistent estimators of a structural parameter vector on the basis of an i.i.d. sample drawn from a curved exponential-type distribution, and study the asymptotic concentration probability, about a true parameter vector, of these estimators up to the fifth-order. In particular, (i) we show that third-order efficient estimators are always fourth-order efficient; (ii) a necessary and sufficient condition for fifth-order efficiency is provided; and finally (iii) the MLE is shown to be fifth-order efficient. Key Words and Phrases: Bias-adjustment, curved exponential distributions, Edgeworth expansion, maximum likelihood estimator, Fisher-consistency. AMS 199...

: It is shown (Proposition (3.9)) that the asymptotic information bound which is valid for the estimation of a parameter in the structure (mixture) model remains valid in the functional model (incidental nuisance parameters) if only perturbation symmetric estimators (Definition (3.6)) are admitted. Perturbation symmetry is a property which is closely related to permutation symmetry (Theorem (3.4)). In particular, equicontinuous functions of empirical processes are perturbation symmetric (Theorem (3.3)). Thus, the results of this paper continue a discussion initiated by Bickel and Klaassen, [2], Pfanzagl, [14], and Strasser, [21], on permutation symmetry of estimators and the exclusion of superefficiency in the functional model. 1 AMS 1991 subject classifications. Primary: 62F12; secondary: 62B15, 62C15 2 Key words and phrases. Asymptotic efficiency of estimates, incidental nuisance parameters, structure model, functional model, permutation symmetric estimates, perturbation symmetric...