Results 1 
8 of
8
Publication Bias: The “FileDrawer Problem” in Scientific Inference
 The Journal of Scientific Exploration
, 2000
"... It is human nature for “the affirmative or active to effect more than the negative or privative. So that a few times hitting, or presence, countervails ofttimes failing or absence.” —Francis Bacon, The Advancement of Learning Abstract—Publication bias arises whenever the probability that a study is ..."
Abstract

Cited by 13 (0 self)
 Add to MetaCart
It is human nature for “the affirmative or active to effect more than the negative or privative. So that a few times hitting, or presence, countervails ofttimes failing or absence.” —Francis Bacon, The Advancement of Learning Abstract—Publication bias arises whenever the probability that a study is published depends on the statistical significance of its results. This bias, often called the filedrawer effect because the unpublished results are imagined to be tucked away in researchers ’ file cabinets, is a potentially severe impediment to combining the statistical results of studies collected from the literature. With almost any reasonable quantitative model for publication bias, only a small number of studies lost in the file drawer will produce a significant bias. This result contradicts the wellknown failsafe filedrawer (FSFD) method for setting limits on the potential harm of publication bias, widely used in social, medical, and psychic research. This method incorrectly treats the file drawer as unbiased and almost always misestimates the seriousness of publication bias. A large body of not only psychic research, but medical and social science studies as well, has mistakenly relied on this method to validate claimed discoveries. Statistical combination can be trusted only if it is known with certainty that all studies that have been carried out are included. Such certainty is virtually impossible to achieve in literature surveys. Key words: publication bias — metaanalysis — file drawer effect — statistics
A Nonparametric "Trim and Fill" Method of Assessing Publication Bias in Metaanalysis
"... Metaanalysis collects and synthesizes results from individual studies to estimate an overall effect size. If published studies are chosen, say through a literature review, an inherent selection bias may arise, since for example, studies may tend to be published more readily if they are statisticall ..."
Abstract

Cited by 7 (2 self)
 Add to MetaCart
Metaanalysis collects and synthesizes results from individual studies to estimate an overall effect size. If published studies are chosen, say through a literature review, an inherent selection bias may arise, since for example, studies may tend to be published more readily if they are statistically significant, or deemed to be more `interesting' in terms of the impact of their outcomes. We develop a simple rankbased data augmentation technique, formalizing the use of funnel plots, to estimate and adjust for the numbers and outcomes of missing studies. Several nonparametric estimators are proposed for the number of missing studies, and their properties are developed analytically and through simulations. We apply the method to simulated and epidemiological data sets, and show it is both effective and consistent with other criteria in the literature. Corresponding author's email address: tweedie@stat.colostate.edu Key words: Metaanalysis; Publication bias; Missing studies; File dra...
Adjustment for Publication and Quality Bias in Bayesian Metaanalysis
, 1997
"... this paper we extend a Bayesian method described in Givens, Smith and Tweedie (1997), which covers the situation where publication is due solely to significance levels, to a stratified model which allows for other aspects to be taken into account. Estimation uses the data augmentation principle (Tan ..."
Abstract

Cited by 2 (1 self)
 Add to MetaCart
this paper we extend a Bayesian method described in Givens, Smith and Tweedie (1997), which covers the situation where publication is due solely to significance levels, to a stratified model which allows for other aspects to be taken into account. Estimation uses the data augmentation principle (Tanner, 1991); specifically, we construct an algorithm which imputes latent sets of missing studies into a metaanalysis, in accordance with the probabilities that they are missing in given significance ranges, or quality ranges, or the like. We then apply this model to a set of studies collected by DelgadoRodriguez et al. (1992), who examine studies on the association between the use of oral contraceptives and cervical cancer. We investigated the effect of publication bias due to significance levels alone in LaFleur et al., (1996). Here we are able to consider the effect of the quality assessments made by DelgadoRodriguez et al. (1992), and show that this makes a considerable difference in the final evaluation of the data set. Specifically, after allowing for these biases the estimated relative risk of cervical cancer from oral contraceptive use is reduced considerably, indicating that a suppression bias against studies of insignificance or poor quality may seriously distort the results of an ordinary metaanalysis of these data. 2 Adjusting for Publication Bias
Practical Estimates of the Effect of Publication Bias in MetaAnalysis
"... this paper we present a new technique, the `Trim and Fill' method developed in [16,17], for estimating and adjusting for the numbers and outcomes of such missing studies. It has the advantage of being computationally simple, and in practical situations seems to perform better than existing methods. ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
this paper we present a new technique, the `Trim and Fill' method developed in [16,17], for estimating and adjusting for the numbers and outcomes of such missing studies. It has the advantage of being computationally simple, and in practical situations seems to perform better than existing methods. To illustrate this new approach, in this article we apply the trim and fill method to an existing metaanalysis of the effect of using gangliosides in reducing case fatality in acute stroke. The data are taken from the 1998 Cochrane Database of Systematic Reviews, which contains several hundred overviews of clinical trials which can be assessed using metaanalyses.
Assessing Sensitivity to Data Problems in Epidemiological Metaanalyses
"... this paper we restrict comment to the following specific issues, although there are many more to which similar approaches will apply: (i) the problem of comparability of data and study design, since for the metaanalysis to be meaningfully interpreted, we must not combine "apples and oranges"; (ii) ..."
Abstract
 Add to MetaCart
this paper we restrict comment to the following specific issues, although there are many more to which similar approaches will apply: (i) the problem of comparability of data and study design, since for the metaanalysis to be meaningfully interpreted, we must not combine "apples and oranges"; (ii) the effect of "publication bias", recognising that failure to obtain all relevant studies, both published and unpublished, may result in a quite distorted metaanalysis; (iii) the possible existence of systematic errors in individual studies, since these flow to bias in the overall analysis, and some account must be taken of them. Clearly all of these (and many other problems) are of concern in principle in any metaanalysis, but it is not obvious whether they will cause real problems in any one specific application. Our theme in this paper is that, by using sensitivity analyses based on the collection of real data and comparison of more and less sophisticated models, one can develop useful information on the extent and effect of such problems, rather than merely expressing concern about their existence. We shall see that for each of (i)(iii), we can develop approaches based on data which describe the best and worst case scenarios (and the intermediate ones also), and use these to quantify whether further steps must be taken to protect the metaanalysis from incorrect inferences. Although other measures of relationship are possible, in epidemiology it is common to consider the effect of the relationship to be measured by the relative risk (RR), given conceptually as the ratio
Trim and Fill: A Simple Funnel Plot Based Method of Testing and Adjusting for Publication Bias in Metaanalysis
"... this paper we investigate three properties of this approach: (a) We consider which of the estimators have the better mean square error (MSE) properties: we show in Section 3 that the estimators R 0 and L 0 are both better than Q 0 , but that there are values of the number of observed and missing stu ..."
Abstract
 Add to MetaCart
this paper we investigate three properties of this approach: (a) We consider which of the estimators have the better mean square error (MSE) properties: we show in Section 3 that the estimators R 0 and L 0 are both better than Q 0 , but that there are values of the number of observed and missing studies for which each is better than the other; (b) We use the distributional properties of the estimators to formulate tests for the existence of publication bias, and compare these with recent methods of Begg (1994) and Egger et al (1997): the resulting tests, in Section 5, appear to be quite powerful if there are more than 56 missing studies; (c) We compare the properties of the iterated version of the algorithm (used in practice) with those of the analytic versions in Taylor and Tweedie (1998): using simulations, we see in Section 4.2 that the iteration does not adversely affect accuracy in general, and so the analytic descriptions of the tests and estimators can be used in the iteration. We then apply the iterative algorithm and the test methods, in Section 6, to a number of sets of studies, and show that they are consistent with the Begg (1994) and Egger et al (1997) approaches, and also to the more complex Bayesian method in Givens et al (1997). These data sets include metaanalyses of the risk of Chlamydia trachomatis from oral contraceptive (OC) use, described by Cottingham and Hunter (1992) and analyzed by Begg (1994); studies of the association between lung cancer and passive smoking, collected and analyzed in Tweedie et al (1996); and a psychometric metaanalysis of IQ scores and teacher expectancy, collected by Raudenbush (1984), and analyzed by Begg (1994) and Gleser and Olkin (1996). 1.2 Funnel plots and quantitative methods
Environmental Tobacco Smoke
"... ather higher than this. The conclusion was subsequently challenged in the courts by the tobacco manufacturers, suppliers and growers, and the EPA lost the case (which is currently under appeal with a final ruling pending). Although some of the reasons for this judgment were procedural, the courts r ..."
Abstract
 Add to MetaCart
ather higher than this. The conclusion was subsequently challenged in the courts by the tobacco manufacturers, suppliers and growers, and the EPA lost the case (which is currently under appeal with a final ruling pending). Although some of the reasons for this judgment were procedural, the courts ruled in effect that the EPA had not established causality, and had not used appropriate statistical methods in reaching its evaluation. The intense nature of this and other debates on ETS worldwide (another example being the Australian NH&MRC review of ETS and its relationship to a variety of diseases [9]) has focused attention on some interesting and possibly unique statistical or environmetrical aspects of the arguments used. These include: (a) To what degree can statistics be used in supporting causal assertions? In particular, is a relative risk of 1.20 strong enough for such a conclusion (especially when, for example, effects of exposures to electromagnetic fields from powerlines hav
Presented at the
, 1999
"... It is human nature for “the affirmative or active to effect more than the negative or privative. So that a few times hitting, or presence, countervails ofttimes failing or absence.” ..."
Abstract
 Add to MetaCart
It is human nature for “the affirmative or active to effect more than the negative or privative. So that a few times hitting, or presence, countervails ofttimes failing or absence.”