We provide a classification of graphical models according to their representation as exponential families. Undirected graphical models with no hidden variables are linear exponential families (LEFs), directed acyclic graphical (DAG) models and chain graphs with no hidden variables, including DAG models with several families of local distributions, are curved exponential families (CEFs) and graphical models with hidden variables are stratified exponential families (SEFs). A SEF is a finite union of CEFs of various dimensions satisfying some regularity conditions. The main results of this paper are that graphical models are SEFs and that many graphical models are not CEFs. That is, roughly speaking, graphical models when viewed as exponential families correspond to a set of smooth manifolds of various dimensions and usually not to a single smooth manifold. These results are discussed in the context of model selection. Keywords : Bayesian networks, graphical models, hidden variables, cur...

We present a new class of statistical models designed for life history analysis of plants and animals. They allow joint analysis of data on survival and reproduction over multiple years, allow for variables having different statistical distributions, and correctly account for the dependence of variables on earlier variables (for example, that a dead individual stays dead and cannot reproduce). We illustrate their utility with an analysis of data taken from an experimental study of Echinacea angustifolia sampled from remnant prarie populations in western Minnesota. Statistically, they are graphical models with some resemblance to generalized linear models and survival analysis. They have directed acyclic graphs with nodes having no more than one parent. The conditional distribution of each node given the parent is a oneparameter exponential family with the parent variable the sample size. The model may be heterogeneous, each node having a different exponential family. We show that the joint distribution is a flat exponential family and derive its canonical parameters, Fisher information, and other properties. These models are implemented in an R package ‘aster ’ available from CRAN.

We consider a log-concave density f in R m satisfying certain weak conditions, particularly on the Hessian matrix of \Gamma log f . For such a density, we prove tail exactness of the multivariate saddlepoint approximation. The proof is based on a local limit theorem for the exponential family generated by f . However, the result refers not to asymptotic behaviour under repeated sampling, but to a limiting property at the boundary of the domain of f . Our approach does not apply any complex analysis, but relies totally on convex analysis and exponential models arguments. Running headline: Multivariate saddlepoint approximations. AMS 1991 Subject Classifications: primary: 62E17, 62E20, 62F11, 62H10 secondary: 60E10, 60F05 Keywords: asymptotic normality, convex analysis, exponential models, local limit theorem, moment generating function, Legendre transform, saddlepoint approximation. Department of Mathematical Sciences, Aarhus University, DK-8000 Aarhus C, Denmark, email: atsoebn@mi...

url: www.stat.umn.edu/geyer/gdor Abstract: When in a full exponential family the maximum likelihood estimate (MLE) does not exist, the MLE may exist in the Barndorff-Nielsen completion of the family. We propose a practical algorithm for finding the MLE in the completion based on repeated linear programming using the R contributed package rcdd and illustrate it with two generalized linear model examples. When the MLE for the null hypothesis lies in the completion, likelihood ratio tests of model comparison are almost unchanged from the usual case. Only the degrees of freedom need to be adjusted. When the MLE lies in the completion, confidence intervals are changed much more from the usual case. The MLE of the natural parameter can be thought of as having gone to infinity in a certain direction, which we call a generic direction of recession. We propose a new one-sided confidence interval which says how close to infinity the natural parameter may be. This maps to one-sided confidence intervals for mean values showing how close to the

This paper presents recursive cavity modeling—a principled, tractable approach to approximate, near-optimal inference for large Gauss–Markov random fields. The main idea is to subdivide the random field into smaller subfields, constructing cavity models which approximate these subfields. Each cavity model is a concise, yet faithful, model for the surface of one subfield sufficient for near-optimal inference in adjacent subfields. This basic idea leads to a tree-structured algorithm which recursively builds a hierarchy of cavity models during an “upward pass ” and then builds a complementary set of blanket models during a reverse “downward pass. ” The marginal statistics of individual variables can then be approximated using their blanket models. Model thinning plays an important role, allowing us to develop thinned cavity and blanket models thereby providing tractable approximate inference. We develop a maximum-entropy approach that exploits certain tractable representations of Fisher information on thin chordal graphs. Given the resulting set of thinned cavity models, we also develop a fast preconditioner, which provides a simple iterative method to compute optimal estimates. Thus, our overall approach combines recursive inference, variational learning and iterative estimation. We demonstrate the accuracy and scalability of this approach in several challenging, large-scale remote sensing problems.

This paper compares the classical concept of assumed density filters (ADF) with a new class of approximate filters, the projection filters (PF). It is shown that the concept of ADF is inconsistent in the sense that the resulting filters vary according to the choice of a stochastic calculus, i.e. Ito or McShane--Fisk--Stratonovich (MFS). The PF are based on the differential geometric approach to statistics and are well defined. It is shown that in the context of the exponential families, the PF coincide with MFS--based ADF. An example is provided, which shows that this does not hold in general, for non--exponential families of densities. Keywords : Nonlinear filtering, finite dimensional filtering, Stratonovich stochastic differential equations, projection filter, assumed density filter, differential geometry and statistics, Fisher metric, Hellinger metric, exponential family. AMS 1991 subject classifications : Primary 60G35 ; secondary 62B10. x This work was partially supported by the...

General methods for obtaining maximum likelihood estimates in exponential families are demonstrated using a constrained autologistic model for estimating relatedness from DNA fingerprint data. The novel features are the use of constrained optimization and two new algorithms for maximum likelihood estimation. The first, the "phase I " algorithm determines the support of the MLE in the closure of the exponential family (a distribution in the family conditioned on a face of the convex support of the natural statistic) when the MLE does not exist in the traditional sense (a point in the natural parameter space). The second, the maximum Monte Carlo likelihood algorithm uses the Metropolis algorithm or the Gibbs sampler to obtain estimates when exact calculation of the likelihood is not possible. Separate papers on each algorithm accompany

Log-linear models are a widely accepted tool for modeling discrete data given in a contingency table. Although their parameters reflect the interaction structure in the joint distribution of all variables, they do not give information about structures appearing in the margins of the table. This is in contrast to multivariate logistic parameters recently introduced by Glonek & McCullagh (1995). They have as parameters the highest order log odds ratios derived from the joint table and from each marginal table. The link between the cell probabilities and the multivariate logistic parameters is given in Glonek & McCullagh in an algebraic fashion. In this paper we focus on this link, showing that it is derived by general parameter transformations in exponential families. In particular, the connection between the natural, the expectation and the mixed parameterization in exponential families (Barndorff-Nielsen, 1978) is used. This also yields the derivatives of the likelihood equation and shows properties of the Fisher matrix. Further emphasis is paid to the analysis of independence hypotheses in margins of a contingency table.

When in a full exponential family the maximum likelihood estimate (MLE) does not exist, the MLE may exist in the Barndorff-Nielsen completion of the family (Barndorff-Nielsen, 1978; Brown, 1986; Geyer, 1990). A practical algorithm for finding the MLE in the completion using repeated linear programming was proposed in the author’s unpublished thesis (Geyer, 1990) and used in Geyer and Thompson (1992). Now we propose a slightly different method, also using repeated linear programming with the R contributed package rcdd (Geyer and Meeden, 2008), which makes straightforward the calculation of the MLE in the Barndorff-Nielsen completion for any models satisfying a condition of Brown (1986) and for which some R function can calculate the MLE when it does exist, for example, generalized linear models (GLM) and aster models (Geyer et al., 2007; Geyer, 2008). In this technical report we give details of two GLM examples. Likelihood ratio tests of model comparison are almost unchanged from the usual case. Only the degrees of freedom need be adjusted when the MLE for the null hypothesis lies in the completion rather than the original family. Confidence intervals are changed much more. When the MLE for the natural parameter does not exist, it can be thought of as having gone to infinity in a certain direction, which we call a generic direction of recession. Here we propose a new kind of one-sided confidence interval, not involving asymptotic approximation, for how close to infinity the true unknown natural parameter value may be. This maps to a one-sided confidence interval for the mean value parameter showing how close to the boundary of its support it may be. 1 R Package Rcdd We use the R statistical computing environment (R Development Core Team, 2008) in our analysis. It is free software and can be obtained from

We study the problem of existence of finite-dimensional filters in discrete time. Discrete time filtering problems can be seen as dynamic generalizations of Bayesian statistics, when the unknown parameter becomes an unobserved (state/signal) process. The problem of existence of finite-dimensional filters corresponds, in Bayesian statistics, to the existence to a finitely parametrized family of conjugate distributions and this makes the exponential class of distributions relevant also for filtering. Given an observation model with distribution belonging to the exponential class and a conjugate family of filter distributions together with its dynamics, we give conditions under which there exists an unobserved state/signal process leading to the given filter distribution with its dynamics. We also present an approach to actually determine the state process dynamics. The methodology is based on the inverse Laplace transform. Key Words: Dynamic Bayes formula, exponential families, finite d...