Results 1 
8 of
8
Severe Testing as a Basic Concept in a NeymanPearson Philosophy of Induction
 BRITISH JOURNAL FOR THE PHILOSOPHY OF SCIENCE
, 2006
"... Despite the widespread use of key concepts of the Neyman–Pearson (N–P) statistical paradigm—type I and II errors, significance levels, power, confidence levels—they have been the subject of philosophical controversy and debate for over 60 years. Both current and longstanding problems of N–P tests s ..."
Abstract

Cited by 35 (14 self)
 Add to MetaCart
Despite the widespread use of key concepts of the Neyman–Pearson (N–P) statistical paradigm—type I and II errors, significance levels, power, confidence levels—they have been the subject of philosophical controversy and debate for over 60 years. Both current and longstanding problems of N–P tests stem from unclarity and confusion, even among N–P adherents, as to how a test’s (predata) error probabilities are to be used for (postdata) inductive inference as opposed to inductive behavior. We argue that the relevance of error probabilities is to ensure that only statistical hypotheses that have passed severe or probative tests are inferred from the data. The severity criterion supplies a metastatistical principle for evaluating proposed statistical inferences, avoiding classic fallacies from tests that are overly sensitive, as well as those not sensitive enough to particular errors and discrepancies.
CurveFitting, the Reliability of Inductive Inference and the ErrorStatistical Approach,” forthcoming Philosophy of Science
"... The main aim of this paper is to revisit the curvefitting problem using the reliability of inductive inference as a primary criterion for the ‘fittest ’ curve. Viewed from this perspective, it is argued that a crucial concern with the current framework for addressing the curvefitting problem is, o ..."
Abstract

Cited by 4 (4 self)
 Add to MetaCart
The main aim of this paper is to revisit the curvefitting problem using the reliability of inductive inference as a primary criterion for the ‘fittest ’ curve. Viewed from this perspective, it is argued that a crucial concern with the current framework for addressing the curvefitting problem is, on the one hand, the undue influence of the mathematical approximation perspective, and on the other, the insufficient attention paid to the statistical modeling aspects of the problem. Using goodnessoffit as the primary criterion for best, the mathematical approximation perspective undermines the reliability of inference objective by giving rise to selection rules which pay insufficient attention to ‘capturing the regularities in the data’. A more appropriate framework is offered by the errorstatistical approach, where(i)statistical adequacy provides the criterion for assessing when a curve captures the regularities in the data adequately, and (ii) the relevant error probabilities canbeusedtoassessthereliabilityof inductive inference. Broadly speaking, the fittest curve (statistically adequate) is not determined by the smallness if its residuals, tempered by simplicity or other pragmatic criteria, but by the nonsystematic (e.g. whitenoise) nature of its residuals. The advocated errorstatistical arguments are illustrated by comparing the Kepler and Ptolemaic models on empirical grounds. ∗ Forthcoming in Philosophy of Science, 2007. † I’m grateful to Deborah Mayo and Clark Glymour for many valuable suggestions and comments on an earlier draft of the paper; estimating the Ptolemaic model was the result of Glymour’s prompting and encouragement. 1 1
2006c) “The CurveFitting Problem, Akaiketype Model Selection, and the Error Statistical Approach.” Virginia Tech working paper
"... The curvefitting problem is often viewed as an exemplar which encapsulates the multitude of dimensions and issues associated with inductive inference, including underdetermination and the reliability of inference. The prevailing view is that the ‘fittest ’ curve is one which provides the optimal tr ..."
Abstract

Cited by 3 (2 self)
 Add to MetaCart
The curvefitting problem is often viewed as an exemplar which encapsulates the multitude of dimensions and issues associated with inductive inference, including underdetermination and the reliability of inference. The prevailing view is that the ‘fittest ’ curve is one which provides the optimal tradeoff between goodnessoffit and simplicity, with the Akaike Information Criterion (AIC) the preferred method. The paper argues that the AICtype procedures do not provide an adequate solution to the curve fitting problem because (a) they have no criterion to assess when a curve captures the regularities in the data inadequately, and (b) they are prone to unreliable inferences. The thesis advocated is that for more satisfactory answers one needs to view the curvefitting problem in the context of errorstatistical approach where (i) statistical adequacy provides a criterion for selecting the fittest curve and (ii) the error probabilities can be used to calibrate the reliability of inductive inference. This thesis is illustrated by comparing the Kepler and Ptolemaic models in terms of statistical adequacy, showing that the latter does not ‘save the phenomena ’ as often claimed. This calls into question the view concerning the pervasiveness of the problem of underdetermination; statistically adequate ‘fittest ’ curves are rare, not common. ∗Thanks are due to Deborah Mayo and Clark Glymour for valuable suggestions and comments on an earlier draft of the paper. 1 1
Akaiketype criteria and the reliability of inference: Model selection versus statistical model specification
 Journal of Econometrics
"... Over the last two decades or so, the Akaike Information Criterion (AIC) and its various modifications/extensions have found wide applicability in econometrics as objective procedures which can be used to select parsimonious statistical models. The aim of this paper is to argue that these model selec ..."
Abstract

Cited by 3 (3 self)
 Add to MetaCart
Over the last two decades or so, the Akaike Information Criterion (AIC) and its various modifications/extensions have found wide applicability in econometrics as objective procedures which can be used to select parsimonious statistical models. The aim of this paper is to argue that these model selection procedures invariably give rise to unreliable inferences primarily because their selection within a prespecified family of models, (a) assumes away the problem of model validation, and (b) ignores the relevant error probabilities. The paper argues for a return to the original statistical model specification problem, as envisaged by Fisher (1922), where the task is understood as one of selecting a statistical model in such a way so as to render the particular data a truly typical realization of the stochastic process underlying the model in question. This problem can be addressed by evaluating a statistical model in terms of its statistical adequacy, i.e. whether it accounts for the chance regularities in the data, as opposed to trading goodnessoffit against parsimony. Thanks are due to Steven Durlauf, Andros Kourtellos, Nikitas Pittis and two anonymous referees for their constructive comments and suggestions that helped to improve the paper substantially. 1 1
Revisiting the Statistical Foundations of Panel Data Modeling
, 2011
"... Despite the impressive developments in panel data modeling, the statistical foundations of such models are rather weak in so far that they are inadequate for securing the reliability and precision of inference. In statistical induction we learn from data about phenomena of interest when we employ re ..."
Abstract
 Add to MetaCart
Despite the impressive developments in panel data modeling, the statistical foundations of such models are rather weak in so far that they are inadequate for securing the reliability and precision of inference. In statistical induction we learn from data about phenomena of interest when we employ reliable and incisive inference procedures. When one invokes either untested probabilistic assumptions, or/and broad (including nonparametric) premises, one has to rely on crude approximations (asymptotic) for evaluating the relevant error probabilities. This invariably leads to imprecise inference of unknown reliability because one does not know how closely the actual error probabilities approximate the assumed nominal ones, or how effective the inference procedures are! The primary objective of the paper is to revisit these probabilistic foundations with a view to: (a) recast the error assumptions in terms of the probabilistic structure of the observable processes underlying the data, (b) provide a complete and internally consistent set of testable probabilistic assumptions for several statistical models for panel data, and (c) propose pertinent interpretations for the individualspecific (fixed or random) and timespecific effects. It is shown that the current interpretations of the individualspecific effects (fixed or random) need to be reconsidered in light of the implicit statistical parameterizations in terms of the observable stochastic processes involved. This provides a more appropriate framework for securing learning from panel data by bringing out the neglected facets of empirical modeling which include specification, misspecification testing and respecification. 1 1
IMS Lecture Notes–Monograph Series
, 2006
"... Where do statistical models come from? Revisiting the problem of specification ..."
Abstract
 Add to MetaCart
Where do statistical models come from? Revisiting the problem of specification
Statistical Model Specification and Validation: Statistical vs. Substantive Information
, 2011
"... Statistical model specification and validation raise crucial foundational problems whose pertinent resolution holds the key to learning from data by securing the reliability of frequentist inference. The paper questions the judiciousness of several current practices, including the theorydriven appr ..."
Abstract
 Add to MetaCart
Statistical model specification and validation raise crucial foundational problems whose pertinent resolution holds the key to learning from data by securing the reliability of frequentist inference. The paper questions the judiciousness of several current practices, including the theorydriven approach, and the Akaiketype model selection procedures, arguing that they often lead to unreliable inferences. This is primarily due to the fact that goodnessoffit/prediction measures and other substantive and pragmatic criteria are of questionable value when the estimated model is statistically misspecified. Foisting one’s favorite model on the data often yields estimated models which are both statistically and substantively misspecified, but one has no way to delineate between the two sources of error and apportion blame. The paper argues that these problems also arise in seemingly purely statistical procedures like the Akaiketype model selection and Crossvalidation. The paper argues that the error statistical approach can address this Duhemian ambiguity by distinguishing between statistical and substantive premises and viewing empirical modeling in a piecemeal way with a view to delineate the various issues more effectively. Many thanks are due to my colleague, Deborah Mayo, for numerous constructive criticisms and suggestions that improved the paper considerably. Special thanks are also due to David F. Hendry for valuable comments and suggestions.