Abstract. Multivariate Gaussian graphical models are defined in terms of Markov properties, i.e., conditional independences associated with the underlying graph. Thus, model selection can be performed by testing these conditional independences, which are equivalent to specified zeroes among certain (partial) correlation coefficients. For concentration graphs, covariance graphs, acyclic directed graphs, and chain graphs (both LWF and AMP), we apply Fisher’s z-transformation, ˇ Sidák’s correlation inequality, and Holm’s step-down procedure, to simultaneously test the multiple hypotheses obtained from the Markov properties. This leads to a simple method for model selection that controls the overall error rate for incorrect edge inclusion. In practice, we advocate partitioning the simultaneous p-values into three disjoint sets, a significant set S, an indeterminate set I, and a non-significant set N. Then our SIN model selection method selects two graphs, a graph whose edges correspond to the union of S and I, and a more conservative graph whose edges correspond to S only. Prior information about the presence and/or absence of particular edges can be incorporated readily. 1.

Ancestral graph models, introduced by Richardson and Spirtes (2002), generalize both Markov random fields and Bayesian networks to a class of graphs with a global Markov property that is closed under conditioning and marginalization. By design, ancestral graphs encode precisely the conditional independence structures that can arise from Bayesian networks with selection and unobserved (hidden/latent) variables. Thus, ancestral graph models provide a potentially very useful framework for exploratory model selection when unobserved variables might be involved in the data-generating process but no particular hidden structure can be specified. In this paper, we present the Iterative Conditional Fitting (ICF) algorithm for maximum likelihood estimation in Gaussian ancestral graph models. The name reflects that in each step of the procedure a conditional distribution is estimated, subject to constraints, while a marginal distribution is held fixed. This approach is in duality to the well-known Iterative Proportional Fitting algorithm, in which marginal distributions are fitted while conditional distributions are held fixed. 1

Graphical models with bi-directed edges (↔) represent marginal independence: the absence of an edge between two vertices indicates that the corresponding variables are marginally independent. In this paper, we consider maximum likelihood estimation in the case of continuous variables with a Gaussian joint distribution, sometimes termed a covariance graph model. We present a new fitting algorithm which exploits standard regression techniques and establish its convergence properties. Moreover, we contrast our procedure to existing estimation algorithms. 1

The AMP Markov property is a recently proposed alternative Markov property for chain graphs. In the case of continuous variables with a joint multivariate Gaussian distribution, it is the AMP rather than the earlier introduced LWF Markov property that is coherent with data-generation by natural block-recursive regressions. In this paper, we show that maximum likelihood estimates in Gaussian AMP chain graph models can be obtained by combining generalized least squares and iterative proportional fitting to an iterative algorithm. In an appendix, we give useful convergence results for iterative partial maximization algorithms that apply in particular to the described algorithm. Key words: AMP chain graph, graphical model, iterative partial maximization, multivariate normal distribution, maximum likelihood estimation 1

Covariance matrices which can be arranged in tridiagonal form are called covariance chains. They are used to clarify some issues of parameter equivalence and of independence equivalence for linear models in which a set of latent variables influences a set of observed variables. For this purpose, orthogonal decompositions for covariance chains are derived first in explicit form. Covariance chains are also contrasted to concentration chains, for which estimation is explicit and simple. For this purpose, maximum-likelihood equations are derived first for exponential families when some parameters satisfy zero value constraints. From these equations explicit estimates are obtained, which are asymptotically efficient, and they are applied to covariance chains. Simulation results confirm the satisfactory behaviour of the explicit covariance chain estimates also in moderate-size samples.

In graphical modelling, a bi-directed graph encodes marginal independences among random variables that are identified with the vertices of the graph. We show how to transform a bi-directed graph into a maximal ancestral graph that (i) represents the same independence structure as the original bi-directed graph, and (ii) minimizes the number of arrowheads among all ancestral graphs satisfying (i). Here the number of arrowheads of an ancestral graph is the number of directed edges plus twice the number of bi-directed edges. In Gaussian models, this construction can be used for more efficient iterative maximization of the likelihood function and to determine when maximum likelihood estimates are equal to empirical counterparts.

Abstract: Let µ be a p-dimensional vector, and let Σ1 and Σ2 be p×p positive definite covariance matrices. On being given random samples of sizes N1 and N2 from independent multivariate normal populations Np(µ,Σ1) and Np(µ, Σ2), respectively, the Behrens-Fisher problem is to solve the likelihood equations for estimating the unknown parameters µ, Σ1, and Σ2. We shall prove that for N1, N2> p there are, almost surely, exactly 2p + 1 complex solutions of the likelihood equations. For the case in which p = 2, we utilize Monte Carlo simulation to estimate the relative frequency with which a typical Behrens-Fisher problem has multiple real solutions; we find that multiple real solutions occur infrequently. Key words and phrases: Behrens-Fisher problem, Bézout’s theorem, maximum likelihood estimation, maximum likelihood degree. 1

A multivariate Gaussian graphical Markov model for an undirected graph G, also called a covariance selection model or concentration graph model, is defined in terms of the Markov properties, i.e., conditional independences associated with G, which in turn are equivalent to specified zeroes among the set of pairwise partial correlation coe#cients. By means of Fisher's z-transformation and Sidak's correlation inequality, conservative simultaneous confidence intervals for the entire set of partial correlations can be obtained, leading to a simple method for model selection that controls the overall error rate for incorrect edge inclusion. The simultaneous p-values corresponding to the partial correlations are partitioned into three disjoint sets, a significant set S, an indeterminate set I, and a non-significant set N. Our SIN model selection method selects two graphs, a graph GSI whose edges correspond to the set I, and a more conservative graph GS whose edges correspond to S only. Prior information about the presence and/or absence of particular edges can be incorporated readily. Similar considerations apply to covariance graph models, which are defined in terms of marginal independence rather than conditional independence. 1.

In recursive linear models, the multivariate normal joint distribution of all variables exhibits a dependence structure induced by recursive systems of linear structural equations. Such models appear in particular in seemingly unrelated regressions, structural equation modelling, simultaneous equation systems, and in Gaussian graphical modelling. We show that recursive linear models that are ‘bow-free’ are well-behaved statistical models, namely, they are everywhere identifiable and form curved exponential families. Here, ‘bow-free ’ refers to models satisfying the condition that if a variable x occurs in the structural equation for y, then the errors for x and y are uncorrelated. For the computation of maximum likelihood estimates in ‘bow-free ’ recursive linear models we introduce the Residual Iterative Conditional Fitting (RICF) algorithm. Compared to existing algorithms RICF is easily implemented requiring only least squares computations, has clear convergence properties, and finds parameter estimates in closed form whenever possible. KEY WORDS: Linear structural equation model; curved exponential family; maximum likelihood estimation; residual iterative conditional fitting; bow-free acyclic path diagrams; BAP. 1

Abstract. We consider normal ≡ Gaussian seemingly unrelated regressions (SUR) models with incomplete data (ID). Imposing a natural minimal set of conditional independence con-straints, we find restricted SUR/ID models for which the likelihood function and the parameter space factors into the product of the likelihood functions and the parameter spaces of standard complete data multivariate analysis of variance models. Hence, the restricted model has a uni-modal likelihood and permits explicit likelihood inference. The restricted model may be used to directly model the data actually observed. Alternatively, the maximum likelihood estimates in the restricted model can yield improved starting values for iterative methods to maximize the likelihood of the unrestricted SUR/ID model. In the development of our methodology, we review and extend existing results for complete data SUR models and the multivariate ID problem. The results are presented in the framework of both lattice conditional independence models and graphical Markov models based on acyclic directed graphs. Date: October 1, 2003. Key words and phrases. Acyclic directed graph, graphical model, incomplete data, lattice conditional indepen-dence model, MANOVA, maximum likelihood estimator, multivariate analysis, multivariate linear model, missing data, seemingly unrelated regressions.