## Publication Bias in Meta-Analysis: A Bayesian Data-Augmentation Approach to Account for Issues Exemplified in the Passive Smoking Debate (1997)

Venue: | Statistical Science |

Citations: | 15 - 6 self |

### BibTeX

@ARTICLE{Givens97publicationbias,

author = {Geof Givens and D. D. Smith and R. L. Tweedie},

title = {Publication Bias in Meta-Analysis: A Bayesian Data-Augmentation Approach to Account for Issues Exemplified in the Passive Smoking Debate},

journal = {Statistical Science},

year = {1997},

volume = {12},

pages = {12--221}

}

### OpenURL

### Abstract

`Publication bias' is a relatively new statistical phenomenon that only arises when one attempts through a meta-analysis to review all studies, significant or insignificant, in order to provide a total perspective on a particular issue. This has recently received some notoriety as an issue in the evaluation of the relative risk of lung cancer associated with passive smoking, following legal challenges to a 1992 EPA analysis which concluded that such exposure is associated with significant excess risk of lung cancer. We introduce a Bayesian approach which estimates and adjusts for publication bias. Estimation is based on a data augmentation principle within a hierarchical model, and the number and outcomes of unobserved studies are simulated using Gibbs sampling methods. This technique yields a quantitative adjustment for the passive smoking meta-analysis. We estimate that there may be both negative and positive but insignificant studies omitted, and that failing to allow for these woul...

### Citations

9193 | Maximum likelihood from incomplete data via the EM algorithm
- Dempster, Laird, et al.
- 1977
(Show Context)
Citation Context ..., thus creating a `complete' dataset for analysis. Data augmentation is a technique which has proven useful in a range of Bayesian and likelihood problems, including applications of the EM algorithm (=-=Dempster et al., 1977-=-) and the IP algorithm (Tanner and Wong, 1987). The premise of data augmentation is that the `observed data' Y can be thought of as a partial realization of the random variable X = (Y ; Z), where a co... |

4100 |
Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images
- Geman, Geman
- 1984
(Show Context)
Citation Context ...rd hierarchical Bayesian model, and the posterior for \Delta can no longer be derived in a tractable analytical form. Instead, numerical techniques must be used, and we use a Gibbs sampling strategy (=-=Geman and Geman, 1984-=-) to obtain approximate samples from the desired posterior distribution. Gibbs sampling techniques, which have been very successful at solving a wide variety of similar problems in Bayesian estimation... |

659 |
Statistical methods for meta-analysis
- Hedges, Olkin
- 1985
(Show Context)
Citation Context ...alysis seeks to combine the analyses from all relevant individual studies into a single statistical analysis with an overall estimate and confidence interval for effect size (Cooper and Hedges, 1994; =-=Hedges and Olkin, 1985-=-). Ideally, greater statistical power can be achieved through meta-analysis than through any one individual study, since data from a greater number of subjects are used, and in recent years there has ... |

659 |
The Calculation of Posterior Distributions by Data Augmentation. With Discussion and With a Reply by the Authors
- Tanner, Wong
- 1987
(Show Context)
Citation Context ...ysis. Data augmentation is a technique which has proven useful in a range of Bayesian and likelihood problems, including applications of the EM algorithm (Dempster et al., 1977) and the IP algorithm (=-=Tanner and Wong, 1987-=-). The premise of data augmentation is that the `observed data' Y can be thought of as a partial realization of the random variable X = (Y ; Z), where a complete realization, X, of X is called the `co... |

393 |
Bayesian computation via the Gibbs sampler and related Markov Chain Monte Carlo methods (with disussion
- Smith, Roberts
- 1993
(Show Context)
Citation Context ...to obtain approximate samples from the desired posterior distribution. Gibbs sampling techniques, which have been very successful at solving a wide variety of similar problems in Bayesian estimation (=-=Smith and Roberts, 1993-=-; Besag and Green, 1993), can be used to obtain a sample from a desired distribution by simulating realizations from a Markov chain whose stationary distribution is equal to the target distribution. H... |

117 | How many iterations in the Gibbs sampler - Raftery, Lewis - 1992 |

87 |
Summing up: The science of reviewing research
- Light, Pillemer
- 1984
(Show Context)
Citation Context ... 1992; Dear and Begg, 1992), and the number of missing studies (Gleser and Olkin, 1996; Eberly and Casella, 1996) based on such data have been proposed but perhaps the most common is the funnel plot (=-=Light and Pillemer, 1984-=-; Vandenbroucke, 1988; Thompson, 1993; Mengersen et al., 1995), which is a graphical method to display possible publication bias. It shows the relationship between the estimated value of \Delta and th... |

67 |
Spatial statistics and Bayesian computation (with discussion
- Besag, Green
- 1993
(Show Context)
Citation Context ...ples from the desired posterior distribution. Gibbs sampling techniques, which have been very successful at solving a wide variety of similar problems in Bayesian estimation (Smith and Roberts, 1993; =-=Besag and Green, 1993-=-), can be used to obtain a sample from a desired distribution by simulating realizations from a Markov chain whose stationary distribution is equal to the target distribution. Here, the target distrib... |

48 |
Handbook of Research Synthesis. Russell Sage Foundation
- Cooper, LV
- 1994
(Show Context)
Citation Context ...ases to be valid. Meta-analysis seeks to combine the analyses from all relevant individual studies into a single statistical analysis with an overall estimate and confidence interval for effect size (=-=Cooper and Hedges, 1994-=-; Hedges and Olkin, 1985). Ideally, greater statistical power can be achieved through meta-analysis than through any one individual study, since data from a greater number of subjects are used, and in... |

47 | Comment: One long run with diagnostics: Implementation strategies for Markov Chain Monte Carlo - Raftery, Lewis - 1992 |

44 | T: Nonsmoking wives of heavy smokers have a higher risk of lung cancer: A study from Japan. Br Med J - Hirayama |

31 | The Number of Iterations, Convergence Diagnostics, and Generic Metropolis Algorithms - Raftery, Lewis - 1995 |

30 |
Modeling Publication Selection Effects in Meta-Analysis
- Hedges
- 1992
(Show Context)
Citation Context ...s of bias, including publication bias : : : could explain any association claimed by the EPA between ETS and lung cancer" (Bero et al., 1994, p. 133). In any meta-analysis, a well documented conc=-=ern (Hedges, 1992-=-; Dear and Begg, 1992; Sterling et al., 1995) is the need to have available all relevant information. It is clearly crucial to attempt to collect at least all published studies, and if possible, one s... |

26 | Bayesian meta-analysis. In - DuMouchel - 1990 |

21 |
An approach to assessing publication bias prior to performing a meta-analysis. Stat Sci
- Dear, Begg
- 1992
(Show Context)
Citation Context ...luding publication bias : : : could explain any association claimed by the EPA between ETS and lung cancer" (Bero et al., 1994, p. 133). In any meta-analysis, a well documented concern (Hedges, 1=-=992; Dear and Begg, 1992-=-; Sterling et al., 1995) is the need to have available all relevant information. It is clearly crucial to attempt to collect at least all published studies, and if possible, one should also search for... |

17 | Incorporating variability in estimates of heterogeneity in the random effects model in meta-analysis
- Biggerstaff, Tweedie
- 1997
(Show Context)
Citation Context ... = 1 P (oe 2 j + �� 2 ) \Gamma1 : (4) In the FE model we take �� 2 = 0 in these equations and in the RE model there are various momentbased and maximum likelihood approaches giving estimates o=-=f �� 2 (Biggerstaff and Tweedie, 1996-=-); in both models it is assumed that the oe 2 j are known, either from estimates based on the raw data in the individual papers or from published estimates in those papers. The results of meta-analyse... |

15 |
Selection models and the file drawer problem
- Iyengar, Greenhouse
- 1988
(Show Context)
Citation Context ...C, Gibbs sampling, environmental tobacco smoke, ETS, passive smoking, lung cancer, file drawer problem. 1 Introduction 1.1 The publication bias problem Publication bias, or the `file-drawer problem' (=-=Iyengar and Greenhouse, 1988-=-), is in some sense a new statistical phenomenon which runs counter to the way in which the scientific method has developed over the past century. One of the key historical contributions of statistica... |

15 |
Publication Decisions Revisited: The Effect of the Outcome of Statistical Tests on the Decision to Publish and Vice Versa
- Sterling, Rosenbaum, et al.
- 1995
(Show Context)
Citation Context ...as : : : could explain any association claimed by the EPA between ETS and lung cancer" (Bero et al., 1994, p. 133). In any meta-analysis, a well documented concern (Hedges, 1992; Dear and Begg, 1=-=992; Sterling et al., 1995-=-) is the need to have available all relevant information. It is clearly crucial to attempt to collect at least all published studies, and if possible, one should also search for unpublished studies su... |

13 |
The impact of method choice on meta-analysis
- Mengersen, Tweedie, et al.
- 1995
(Show Context)
Citation Context ...which assess the benefits, drawbacks and problems of meta-analysis techniques (see for example Mosteller and Chalmers, 1992; Felson, 1992; Chalmers, 1991; NRC Report, 1992; Thompson and Pocock, 1991; =-=Mengersen et al., 1995-=-). One of the most frequently considered aspects is the need for collection of all studies, especially taking into account the possibility that some studies might not get to the peer reviewed publicat... |

12 |
Environmental Tobacco Smoke and Mortality
- LEE
- 1992
(Show Context)
Citation Context ...thorities will consider them established (Doll, 1986; Wynder, 1987). We have not gone into these issues here, but the possibility that an observed association might be caused by such factors as diet (=-=Lee, 1992-=-) or misclassification bias (Lee, 1992; Tweedie et al., 1994) certainly should deserve more attention if the excess risk is reduced as it seems to be when allowing for publication bias. The estimate o... |

11 |
Tainted truth: The manipulation of fact in
- Crossen
- 1994
(Show Context)
Citation Context ...r PhD or MS dissertations; or studies are suppressed by those who do not wish to have results appear that are against their own vested interests, political beliefs, or funding source's interests (see =-=Crossen, 1994-=-, p. 19). With these possible reasons for publication bias, it is clearly hard to ensure that all studies will be found even by diligent search procedures. Sterling et al. (1995) discuss recent indica... |

10 |
An Assessment of Publication Bias Using a Sample of Published Clinical Trials
- Berlin, Begg, et al.
- 1989
(Show Context)
Citation Context ...otentially severe distortions in meta-analyses. Publication bias is not incorporated in the combined estimates in Table 1. A number of ways of attempting to assess the possibility of missing studies (=-=Berlin et al., 1989-=-; Hedges, 1992; Dear and Begg, 1992), and the number of missing studies (Gleser and Olkin, 1996; Eberly and Casella, 1996) based on such data have been proposed but perhaps the most common is the funn... |

9 |
Meta-analysis: methods for combining independent studies
- Olkin
- 1992
(Show Context)
Citation Context ...r can be achieved through meta-analysis than through any one individual study, since data from a greater number of subjects are used, and in recent years there has been an enormous increase (see e.g. =-=Olkin, 1992-=-) in the use of meta-analysis in many areas in order to obtain overall evaluations of association when individual studies are equivocal. Studies for a meta-analysis are usually collected through a rev... |

8 |
Some progress and problems in meta-analysis of clinical trials
- Mosteller, Chalmers
- 1992
(Show Context)
Citation Context ...t al. (1996). 2.4 Publication Bias and the Funnel Plot A large number of discussion papers have appeared which assess the benefits, drawbacks and problems of meta-analysis techniques (see for example =-=Mosteller and Chalmers, 1992-=-; Felson, 1992; Chalmers, 1991; NRC Report, 1992; Thompson and Pocock, 1991; Mengersen et al., 1995). One of the most frequently considered aspects is the need for collection of all studies, especiall... |

8 |
Probing encountered data, meta analysis and weighted distribution methods
- Patil, Taillie
- 1989
(Show Context)
Citation Context ...h this approach. The fixed interval approach seems to provide an adequate, stable estimate. Direct parametric modeling of publication probability has also been proposed (Iyengar and Greenhouse, 1988; =-=Patil and Taillie, 1989-=-). Larose and Dey (1995) survey and compare several alternative parametric models. This parametric approach seems to yield some of the same benefits as our approach, including tighter posterior confid... |

7 | Lung cancer in Japan: Effects of nutrition and exposure to ETS - HIRAYAMA - 1984 |

7 |
Can meta-analyses be trusted? Lancet 338
- THOMPSON, POCOCK
- 1991
(Show Context)
Citation Context ...ssion papers have appeared which assess the benefits, drawbacks and problems of meta-analysis techniques (see for example Mosteller and Chalmers, 1992; Felson, 1992; Chalmers, 1991; NRC Report, 1992; =-=Thompson and Pocock, 1991-=-; Mengersen et al., 1995). One of the most frequently considered aspects is the need for collection of all studies, especially taking into account the possibility that some studies might not get to th... |

7 |
Mengersen K: Bayesian meta-analysis, with application to studies of ETS and lung cancer. Lung Cancer
- Tweedie, Scott, et al.
- 1996
(Show Context)
Citation Context ...s does make a difference to the 95% CI although not in any meaningful way to the estimate of RR itself; the estimate of b �� 2 = 0:023 in this case is insufficient to make a great deal of differen=-=ce (Tweedie et al., 1996). C-=-learly one source of between study variation that might lead to a requirement for heterogeneity in the Y j (expressed through �� 2 ? 0) is the use of studies from different countries. The analysis... |

7 |
Garbage in, garbage out: Can statisticians quantify the effects of poor data
- Tweedie, Mengersen, et al.
- 1994
(Show Context)
Citation Context ...986; Wynder, 1987). We have not gone into these issues here, but the possibility that an observed association might be caused by such factors as diet (Lee, 1992) or misclassification bias (Lee, 1992; =-=Tweedie et al., 1994-=-) certainly should deserve more attention if the excess risk is reduced as it seems to be when allowing for publication bias. The estimate of excess risk is also central in evaluating the problem that... |

6 |
Preexisting lung disease and lung cancer among nonsmoking women
- Alavanja, Brownson, et al.
- 1992
(Show Context)
Citation Context ...tobacco smoke---is a potential carcinogen, the disease is lung cancer, and the hypothesis of concern is RR ? 1. Between 9% to 20% of lung cancer cases occur in non-smokers (Schneiderman et al., 1989; =-=Alavanja et al., 1992-=-). Until the early 1980's, epidemiological studies had not reported any noticeable increase in the incidence of lung cancer among non-smokers who were exposed to ETS. This changed starting in 1981 whe... |

6 |
Problems induced by metaanalysis
- CHALMERS
- 1991
(Show Context)
Citation Context ...unnel Plot A large number of discussion papers have appeared which assess the benefits, drawbacks and problems of meta-analysis techniques (see for example Mosteller and Chalmers, 1992; Felson, 1992; =-=Chalmers, 1991-=-; NRC Report, 1992; Thompson and Pocock, 1991; Mengersen et al., 1995). One of the most frequently considered aspects is the need for collection of all studies, especially taking into account the poss... |

5 | Passive smoking in the workplace: Classical and bayesian meta-analyses
- Biggerstaff, Tweedie, et al.
- 1994
(Show Context)
Citation Context ...dy variance,soe 2 j , is exactly correct, so p(oe 2 ) is degenerate for observed studies. For �� 2 , we use an exponential prior with mean 0.031, based on a meta-analysis of studies on workplace E=-=TS (Biggerstaff et al., 1994-=-). We take an improper uniform prior for Z and our initial prior for w (before being scaled by the largest) is that of 3 uniform random variates on (0:5; 1] for I 1 ; I 2 ; I 3 , and uniform on (0:3; ... |

5 |
Models for Estimating the Number of Unpublished Studies.” Statistics in Medicine
- Gleser, Olkin
- 1996
(Show Context)
Citation Context ...combined estimates in Table 1. A number of ways of attempting to assess the possibility of missing studies (Berlin et al., 1989; Hedges, 1992; Dear and Begg, 1992), and the number of missing studies (=-=Gleser and Olkin, 1996-=-; Eberly and Casella, 1996) based on such data have been proposed but perhaps the most common is the funnel plot (Light and Pillemer, 1984; Vandenbroucke, 1988; Thompson, 1993; Mengersen et al., 1995)... |

5 | Non-parametric classes of weight functions to model publication bias
- Paul
- 1995
(Show Context)
Citation Context ... is based on the idea (Hedges, 1992) that these are the common ranges in which editors and researchers might decide to change the probabilities of publication. Other researchers (Dear and Begg, 1992; =-=Paul, 1995-=-) have considered methods for estimating the endpoints and number of such intervals, rather than fixing them in advance. This may permit a more flexible, data-based determination of how the probabilit... |

4 |
Publication bias and public health policy on environmental tobacco smoke
- Bero, Glantz, et al.
- 1994
(Show Context)
Citation Context ...mpanies filed a lawsuit against the EPA, claiming that " : : : various sources of bias, including publication bias : : : could explain any association claimed by the EPA between ETS and lung canc=-=er" (Bero et al., 1994-=-, p. 133). In any meta-analysis, a well documented concern (Hedges, 1992; Dear and Begg, 1992; Sterling et al., 1995) is the need to have available all relevant information. It is clearly crucial to a... |

4 | Bayesian assessment of publication bias in meta-analyses of cervical cancer and oral contraceptives
- LaFleur, Taylor, et al.
- 1996
(Show Context)
Citation Context ...p-value increases. Such a constraint is much harder to put in place in the frequentist setting (Dear, 1995), and we note that in other circumstances we have found that it seems to be worth enforcing (=-=LaFleur et al., 1996-=-). Using prior distributions and the complete data likelihood, univariate conditional posterior distributions can be derived. We use p(q j \Delta) to represent the conditional posterior distribution o... |

4 |
Controversies in meta-analysis: the case of the trials of serum cholesterol reduction
- THOMPSON
- 1993
(Show Context)
Citation Context ...missing studies (Gleser and Olkin, 1996; Eberly and Casella, 1996) based on such data have been proposed but perhaps the most common is the funnel plot (Light and Pillemer, 1984; Vandenbroucke, 1988; =-=Thompson, 1993-=-; Mengersen et al., 1995), which is a graphical method to display possible publication bias. It shows the relationship between the estimated value of \Delta and the size of the study, measured by, say... |

4 | Lung cancer and exposure to ETS - TRICHOPOULOS, KALANDIDI, et al. - 1981 |

3 |
Bias in meta-analytic research
- Felson
- 1992
(Show Context)
Citation Context ...Bias and the Funnel Plot A large number of discussion papers have appeared which assess the benefits, drawbacks and problems of meta-analysis techniques (see for example Mosteller and Chalmers, 1992; =-=Felson, 1992-=-; Chalmers, 1991; NRC Report, 1992; Thompson and Pocock, 1991; Mengersen et al., 1995). One of the most frequently considered aspects is the need for collection of all studies, especially taking into ... |

3 |
Lung cancer that is not attributable to smoking: letter to the editor
- Schneiderman, Davis, et al.
- 1989
(Show Context)
Citation Context ... substance---environmental tobacco smoke---is a potential carcinogen, the disease is lung cancer, and the hypothesis of concern is RR ? 1. Between 9% to 20% of lung cancer cases occur in non-smokers (=-=Schneiderman et al., 1989-=-; Alavanja et al., 1992). Until the early 1980's, epidemiological studies had not reported any noticeable increase in the incidence of lung cancer among non-smokers who were exposed to ETS. This chang... |

3 | Lung cancer and exposure to ETS. Conclusion of Greek study - TRICHOPOULOS, KALANDIDI, et al. - 1983 |

3 |
Passive smoking and lung cancer: a publication bias
- Vandenbroucke
- 1988
(Show Context)
Citation Context ...), and the number of missing studies (Gleser and Olkin, 1996; Eberly and Casella, 1996) based on such data have been proposed but perhaps the most common is the funnel plot (Light and Pillemer, 1984; =-=Vandenbroucke, 1988-=-; Thompson, 1993; Mengersen et al., 1995), which is a graphical method to display possible publication bias. It shows the relationship between the estimated value of \Delta and the size of the study, ... |

3 | Assessment of the Association Between Passive Smoking and Lung Cancer - VARELA - 1987 |

2 | Invited Commentary: Confounding, measurement error, and publication bias in studies of passive smoking - Kawachi, Colditz - 1996 |

2 |
Mausner & Bahn Epidemiology--An Introductory Text, second edition, W.B
- Mausner, Kramer
- 1985
(Show Context)
Citation Context ...on the disease incidence. In general, epidemiological studies are necessarily observational, rather than controlled experiments. In the two most common study designs, cohort and case-control studies (=-=Mausner and Kramer, 1985-=-), subjects are categorized in a 2 \Theta 2 cross-classification table. Each subject is classified as either exposed to the possible toxic agent or not exposed. Each subject is also classified based o... |

2 | Adjustment for publication and quality bias in Bayesian meta-analysis
- Smith, Givens, et al.
- 1999
(Show Context)
Citation Context ...t for other selection biases, such as those based on differing study quality, for covariates influencing selection bias, and for additional hierarchical strata in the model; we pursue this elsewhere (=-=Smith et al., 1997-=-). Clearly, this approach will also be applicable in many areas other than the epidemiological context in which we illustrate it. 3 Meta-Analysis Allowing for Publication Bias 3.1 The Data Augmentatio... |

1 |
The Aetiology of the Spanish Toxic Shock Syndrome: Interpretation of the Epidemiological Evidence. Report to the WHO Regional Office for Europe
- Doll
- 1986
(Show Context)
Citation Context ...ave led to it than if it is large: the values of the RR (or even of the lower bound on the confidence interval on the RR) need to be at least 2 before many authorities will consider them established (=-=Doll, 1986-=-; Wynder, 1987). We have not gone into these issues here, but the possibility that an observed association might be caused by such factors as diet (Lee, 1992) or misclassification bias (Lee, 1992; Twe... |

1 |
Estimating the number of unseen studies
- Eberly, Casella
- 1996
(Show Context)
Citation Context ...ble 1. A number of ways of attempting to assess the possibility of missing studies (Berlin et al., 1989; Hedges, 1992; Dear and Begg, 1992), and the number of missing studies (Gleser and Olkin, 1996; =-=Eberly and Casella, 1996-=-) based on such data have been proposed but perhaps the most common is the funnel plot (Light and Pillemer, 1984; Vandenbroucke, 1988; Thompson, 1993; Mengersen et al., 1995), which is a graphical met... |

1 | Australia NRC Committee on Applied and Theoretical Statistics - Party, Canberra - 1992 |

1 | Regulation - Draft - 1994 |