Results 11  20
of
179
Least squares model averaging
 Econometrica
, 2007
"... This paper considers the problem of selection of weights for averaging across leastsquares estimates obtained from a set of models. Existing model average methods are based on exponential AIC and BIC weights. In distinction, this paper proposes selecting the weights by minimizing a Mallows ’ criteri ..."
Abstract

Cited by 40 (11 self)
 Add to MetaCart
This paper considers the problem of selection of weights for averaging across leastsquares estimates obtained from a set of models. Existing model average methods are based on exponential AIC and BIC weights. In distinction, this paper proposes selecting the weights by minimizing a Mallows ’ criterion, the latter an estimate of the average squared error from the model average fit. We show that our new Mallows ’ Model Average (MMA) estimator is asymptotically optimal in the sense of achieving the lowest possible squared error in a class of discrete model average estimators. In a simulation experiment we show that the MMA estimator compares favorably with those based on AIC and BIC weights. The proof of the main result is an application of Li (1987). Research supported by the National Science Foundation. I gratefully thank the CoEditor (Whitney Newey), three referees, and Benedickt Potscher for helpful comments.
Verification, validation and predictive capability in computational engineering and physics
 Hopkins University
, 2002
"... Computer simulations of physical processes are being relied on to an increasing degree for design, performance, reliability, and safety of engineered systems. Computational analyses have addressed the operation of systems at design conditions, offdesign conditions, and accident scenarios. For examp ..."
Abstract

Cited by 39 (3 self)
 Add to MetaCart
(Show Context)
Computer simulations of physical processes are being relied on to an increasing degree for design, performance, reliability, and safety of engineered systems. Computational analyses have addressed the operation of systems at design conditions, offdesign conditions, and accident scenarios. For example, the safety aspects of products or systems can represent an important, sometimes dominant, element of numerical simulations. The potential legal and liability costs of hardware failures can be staggering to a company, the environment, or the public. This consideration is especially crucial, given that we may be interested in highconsequence systems that cannot ever be physically tested, including the catastrophic failure of a fullscale containment building for a nuclear power plant, explosive damage to a highrise office building, ballistic missile defense systems, and a nuclear weapon involved in a transportation accident. Developers of computer codes, analysts who use the codes, and decision makers who rely on the results of the analyses face a critical question: How should confidence in modeling and simulation be critically assessed? Verification and validation (V&V) of computational simulations are the primary methods for building and quantifying this confidence. Briefly, verification is the assessment of the accuracy of the solution to a computational model. Validation is the assessment
Improved learning of Bayesian networks
 Proc. of the Conf. on Uncertainty in Artificial Intelligence
, 2001
"... Two or more Bayesian network structures are Markov equivalent when the corresponding acyclic digraphs encode the same set of conditional independencies. Therefore, the search space of Bayesian network structures may be organized in equivalence classes, where each of them represents a different set o ..."
Abstract

Cited by 39 (6 self)
 Add to MetaCart
(Show Context)
Two or more Bayesian network structures are Markov equivalent when the corresponding acyclic digraphs encode the same set of conditional independencies. Therefore, the search space of Bayesian network structures may be organized in equivalence classes, where each of them represents a different set of conditional independencies. The collection of sets of conditional independencies obeys a partial order, the socalled “inclusion order.” This paper discusses in depth the role that the inclusion order plays in learning the structure of Bayesian networks. In particular, this role involves the way a learning algorithm traverses the search space. We introduce a condition for traversal operators, the inclusion boundary condition, which, when it is satisfied, guarantees that the search strategy can avoid local maxima. This is proved under the assumptions that the data is sampled from a probability distribution which is faithful to an acyclic digraph, and the length of the sample is unbounded. The previous discussion leads to the design of a new traversal operator and two new learning algorithms in the context of heuristic search and the Markov Chain Monte Carlo method. We carry out a set of experiments with synthetic and realworld data that show empirically the benefit of striving for the inclusion order when learning Bayesian networks from data.
Bayesian Model Averaging in proportional hazard models: Assessing the risk of a stroke
 Applied Statistics
, 1997
"... Evaluating the risk of stroke is important in reducing the incidence of this devastating disease. Here, we apply Bayesian model averaging to variable selection in Cox proportional hazard models in the context of the Cardiovascular Health Study, a comprehensive investigation into the risk factors for ..."
Abstract

Cited by 37 (5 self)
 Add to MetaCart
Evaluating the risk of stroke is important in reducing the incidence of this devastating disease. Here, we apply Bayesian model averaging to variable selection in Cox proportional hazard models in the context of the Cardiovascular Health Study, a comprehensive investigation into the risk factors for stroke. We introduce a technique based on the leaps and bounds algorithm which e ciently locates and ts the best models in the very large model space and thereby extends all subsets regression to Cox models. For each independent variable considered, the method provides the posterior probability that it belongs in the model. This is more directly interpretable than the corresponding Pvalues, and also more valid in that it takes account of model uncertainty. Pvalues from models preferred by stepwise methods tend to overstate the evidence for the predictive value of a variable. In our data Bayesian model averaging predictively outperforms standard model selection methods for assessing
The Maintenance of Uncertainty
 in Control Systems
, 1997
"... It is important to remain uncertain, of observation, model and law. For the Fermi Summer School, Criticisms Requested email : lenny@maths.ox.ac.uk, Contents 1 ..."
Abstract

Cited by 35 (6 self)
 Add to MetaCart
(Show Context)
It is important to remain uncertain, of observation, model and law. For the Fermi Summer School, Criticisms Requested email : lenny@maths.ox.ac.uk, Contents 1
Regression with Multiple Candidate Models: Selecting or Mixing?
 STATISTICA SINICA
, 1999
"... Model averaging provides an alternative to model selection. An algorithm ARM rooted in information theory is proposed to combine different regression models/methods. A simulation is conducted in the context of linear regression to compare its performance with familiar model selection criteria AIC ..."
Abstract

Cited by 33 (9 self)
 Add to MetaCart
Model averaging provides an alternative to model selection. An algorithm ARM rooted in information theory is proposed to combine different regression models/methods. A simulation is conducted in the context of linear regression to compare its performance with familiar model selection criteria AIC and BIC, and also with some Bayesian model averaging (BMA) methods. The simulation suggests
Maximum likelihood Bayesian averaging of alternative conceptualmathematical models, Stochastic Environ
 Res. Risk Assess
, 2003
"... [1] Hydrologic analyses typically rely on a single conceptualmathematical model. Yet hydrologic environments are open and complex, rendering them prone to multiple interpretations and mathematical descriptions. Adopting only one of these may lead to statistical bias and underestimation of uncertain ..."
Abstract

Cited by 30 (1 self)
 Add to MetaCart
(Show Context)
[1] Hydrologic analyses typically rely on a single conceptualmathematical model. Yet hydrologic environments are open and complex, rendering them prone to multiple interpretations and mathematical descriptions. Adopting only one of these may lead to statistical bias and underestimation of uncertainty. Bayesian model averaging (BMA) [Hoeting et al., 1999] provides an optimal way to combine the predictions of several competing models and to assess their joint predictive uncertainty. However, it tends to be computationally demanding and relies heavily on prior information about model parameters. Neuman [2002, 2003] proposed a maximum likelihood version (MLBMA) of BMA to render it computationally feasible and to allow dealing with cases where reliable prior information is lacking. We apply MLBMA to seven alternative variogram models of log air permeability data from singlehole pneumatic injection tests in six boreholes at the Apache Leap Research Site (ALRS) in central Arizona. Unbiased ML estimates of variogram and drift parameters are obtained using adjoint state maximum likelihood cross validation [Samper and Neuman, 1989a] in conjunction with universal kriging and generalized least squares. Standard information criteria provide an ambiguous ranking of
A conceptual framework for predictability studies
 J. Climate
, 1999
"... A conceptual framework is presented for a unified treatment of issues arising in a variety of predictability studies. The predictive power (PP), a predictability measure based on information–theoretical principles, lies at the center of this framework. The PP is invariant under linear coordinate tra ..."
Abstract

Cited by 23 (0 self)
 Add to MetaCart
A conceptual framework is presented for a unified treatment of issues arising in a variety of predictability studies. The predictive power (PP), a predictability measure based on information–theoretical principles, lies at the center of this framework. The PP is invariant under linear coordinate transformations and applies to multivariate predictions irrespective of assumptions about the probability distribution of prediction errors. For univariate Gaussian predictions, the PP reduces to conventional predictability measures that are based upon the ratio of the rms error of a model prediction over the rms error of the climatological mean prediction. Since climatic variability on intraseasonal to interdecadal timescales follows an approximately Gaussian distribution, the emphasis of this paper is on multivariate Gaussian random variables. Predictable and unpredictable components of multivariate Gaussian systems can be distinguished by predictable component analysis, a procedure derived from discriminant analysis: seeking components with large PP leads to an eigenvalue problem, whose solution yields uncorrelated components that are ordered by PP from largest to smallest. In a discussion of the application of the PP and the predictable component analysis in different types of predictability studies, studies are considered that use either ensemble integrations of numerical models or autoregressive models fitted to observed or simulated data. An investigation of simulated multidecadal variability of the North Atlantic illustrates the proposed methodology. Reanalyzing an ensemble of integrations of the Geophysical Fluid Dynamics Laboratory coupled general circulation model confirms and refines earlier findings. With an autoregressive model fitted to a single integration of the same model, it is demonstrated that similar conclusions can be reached without resorting to computationally costly ensemble integrations. 1.
A comparison of scientific and engineering criteria for Bayesian model selection
, 1996
"... Given a set of possible models for variables X and a set of possible parameters for each model, the Bayesian “estimate ” of the probability distribution for X given observed data is obtained by averaging over the possible models and their parameters. An oftenused approximation for this estimate is ..."
Abstract

Cited by 21 (0 self)
 Add to MetaCart
Given a set of possible models for variables X and a set of possible parameters for each model, the Bayesian “estimate ” of the probability distribution for X given observed data is obtained by averaging over the possible models and their parameters. An oftenused approximation for this estimate is obtained by selecting a single model and averaging over its parameters. The approximation is useful because it is computationally efficient, and because it provides a model that facilitates understanding of the domain. A common criterion for model selection is the posterior probability of the model. Another criterion for model selection, proposed by San Martini and Spezzafari (1984), is the predictive performance of a model for the next observation to be seen. From the standpoint of domain understanding, both criteria are useful, because one identifies the model that is most likely, whereas the other identifies the model that is the best predictor of the next observation. To highlight the difference, we refer to the posteriorprobability and alternative criteria as the scientific criterion (SC) and engineering criterion (EC), respectively. When we are interested in predicting the next observation, the modelaveraged estimate is at least as good as that produced by EC, which itself is at least as good as the estimate produced by SC. We show experimentally that, for Bayesiannetwork models containing discrete variables only, the predictive performance of the model average can be significantly better than those of single models selected by either criterion, and that differences between models selected by the two criterion can be substantial. Keywords: model selection, model averaging, Bayesian selection criteria
Prediction and Retrospective Analysis of Soccer Matches in a League
, 1997
"... A common discussion subject for the male part of the population in particular, is the prediction of next weekend's soccer matches, especially for the local team. Knowledge of offensive and defensive skills is valuable in the decision process before making a bet at a bookmaker. In this article w ..."
Abstract

Cited by 19 (0 self)
 Add to MetaCart
A common discussion subject for the male part of the population in particular, is the prediction of next weekend's soccer matches, especially for the local team. Knowledge of offensive and defensive skills is valuable in the decision process before making a bet at a bookmaker. In this article we take an applied statistician's approach to the problem, suggesting a Bayesian dynamic generalised linear model to estimate the time dependent skills of all teams in a league, and to predict next weekend's soccer matches. The problem is more intricate than it may appear at first glance, as we need to estimate the skills of all teams simultaneously as they are dependent. It is now possible to deal with such inference problems using the iterative simulation technique known as Markov Chain Monte Carlo. We will show various applications of the proposed model based on the English Premier League and Division 1 199798; Prediction with application to betting, retrospective analysis of the final ranking...