Skip to content
Psych Dictionary

Publication bias

  • Concept · Research methods and meta-analysis
  • Strong evidence
  • Published October 20, 2026
  • 11 sources
  • 4 min read

In short

Publication bias is the distortion of the published evidence that occurs when studies with positive or statistically significant results are more likely to be published, or reported in full, than studies with null results. In psychotherapy for depression it has inflated published effects by about a quarter or more.

Key facts

Field
Research methods and meta-analysis
Also known as
Study publication bias; Outcome reporting bias; File drawer problem

State of the evidence

Strong evidence

Cohort studies that followed trials from protocol to publication show that significant results are more likely to be published and fully reported, and that 40% to 62% of trials changed, added or omitted a primary outcome[1]. In psychotherapy for adult depression, statistical adjustment reduced the mean effect from 0.67 to 0.42[2], and adding unpublished NIH-funded trials reduced it by 25%[3]. Statistical detection methods have known limits[4][5].

Evidence labels summarise the research literature; they are not recommendations. How labels are assigned

Publication bias is the distortion of the available evidence that arises when whether, how fully or how quickly a study is published depends on its results. Studies that report positive or statistically significant results are more likely to be published, and within a published study, significant outcomes are more likely to be reported in full[1]. The result is a literature, and meta-analyses built on it, that overstate how well a treatment works.

Two forms are usually distinguished. Study publication bias concerns whole studies that are never published. Outcome reporting bias concerns studies that are published but report their outcomes selectively, for example by changing, adding or omitting primary outcomes relative to the protocol[1].

Evidence that results drive publication

The most direct evidence comes from following studies from approval of their protocols to publication. A systematic review of 16 such cohort studies found strong evidence of an association between significant results and publication. In three studies, statistically significant outcomes had 2.2 to 4.7 times the odds of being fully reported, and comparisons of publications with their protocols found that 40% to 62% of studies had at least one primary outcome that was changed, introduced or omitted[1].

Psychotherapy trials show the same pattern of incomplete registration and selective reporting. Of 112 randomized trials published between 2010 and 2014 in five high-impact clinical psychology journals, 67 were registered, 27 prospectively, and only 13 were correctly registered and reported; seven of those 13 showed evidence of selective outcome reporting. Overall, only five of the 112 trials (4.5%) were free of selective outcome reporting[6].

Funnel plots and asymmetry tests

Most published estimates of publication bias are statistical, inferred from the pattern of results within a meta-analysis. A funnel plot shows each study’s effect against its size or precision. Without bias, small studies scatter widely and large studies cluster, forming a symmetrical funnel; if small studies with null results are missing, the plot is lopsided. Egger and colleagues proposed a regression test of this asymmetry. Applied to 75 published meta-analyses, it indicated bias in 38% of those from leading general medical journals and 13% of Cochrane reviews. The authors cautioned that the test’s ability to detect bias is limited when a meta-analysis rests on a small number of small trials[7].

Asymmetry is not proof of publication bias. Later recommendations stressed that funnel plot asymmetry has a number of other possible causes and should not be equated with publication bias[4].

Trim-and-fill and its limits

The trim-and-fill method goes a step further and estimates what the missing studies might have shown. It uses simple rank-based techniques on the funnel plot to estimate how many studies are missing and the effect they might have had, and adds them to produce an adjusted estimate. In its authors’ simulations, the adjusted estimates were approximately correct and confidence intervals substantially improved[8].

The method performs less well when studies differ in what they estimate. A simulation study found that when meta-analyses are heterogeneous, trim-and-fill may adjust for publication bias where none exists, for example when some precise studies have effects far from the average or when trials’ power calculations create an inverse relation between effect and sample size. The authors concluded that the funnel plot itself is inappropriate for heterogeneous meta-analyses and pointed to selection models as an alternative warranting study[5].

Publication bias in psychotherapy for depression

Psychotherapy for adult depression provides both a statistical and a direct estimate. Cuijpers and colleagues examined 117 randomized trials with 175 comparisons between psychotherapy and control conditions. The mean effect size was 0.67; after trim-and-fill imputed 51 missing studies it fell to 0.42, and both the rank-correlation and Egger tests of funnel plot asymmetry were highly significant. They concluded that the effects of psychotherapy for adult depression appear to be overestimated considerably because of publication bias[2].

Driessen and colleagues then measured the problem directly. They identified the grants the US National Institutes of Health had awarded between 1972 and 2008 to fund randomized trials of psychological treatment for major depressive disorder, and checked whether each had been published. Thirteen of the 55 funded trials that began (23.6%) had produced no publication. Adding data from unpublished trials to the published ones reduced the effect against control conditions from g = 0.52 to 0.39, a 25% reduction, and the authors noted that outcome reporting bias, which they could not examine quantitatively, might inflate the published estimate further[3]. The wider consequences for estimates of how well psychotherapy works are discussed in how psychotherapy works.

Preregistration and registered reports

The main remedies act before results are known. Preregistration means defining the research questions and analysis plan before observing outcomes, which distinguishes analyses that test predictions from those generated after the fact[9]. Trial registries apply the same principle to clinical research. Among large cardiovascular trials funded by the US National Heart, Lung, and Blood Institute, 57% of those published before 2000 reported a significant benefit on the primary outcome, against 8% of those published after 2000, and prior registration of the trial was strongly associated with the trend toward null findings[10].

Registered reports go further: peer review and the decision to publish take place before the results are known. In a comparison of 71 registered reports with 152 standard hypothesis-testing articles in psychology, 96% of the standard articles reported positive results for their first hypothesis, against 44% of the registered reports. The authors suggested that reduced publication bias, reduced inflation of false-positive rates, or both, are plausible explanations[11]. These reforms are part of the wider response to the replication crisis.

Sources11

Numbered in order of first citation.

  1. 1.Dwan, K., Altman, D. G., Arnaiz, J. A., Bloom, J., Chan, A. W., Cronin, E., Decullier, E., Easterbrook, P. J., Von Elm, E., Gamble, C., Ghersi, D., Ioannidis, J. P., Simes, J., & Williamson, P. R. (2008). Systematic review of the empirical evidence of study publication bias and outcome reporting bias. PLoS ONE, 3(8), e3081. https://doi.org/10.1371/journal.pone.0003081
  2. 2.Cuijpers, P., Smit, F., Bohlmeijer, E., Hollon, S. D., & Andersson, G. (2010). Efficacy of cognitive–behavioural therapy and other psychological treatments for adult depression: meta-analytic study of publication bias. British Journal of Psychiatry, 196(3), 173–178. https://doi.org/10.1192/bjp.bp.109.066001
  3. 3.Driessen, E., Hollon, S. D., Bockting, C. L., Cuijpers, P., & Turner, E. H. (2015). Does publication bias inflate the apparent efficacy of psychological treatment for major depressive disorder? A systematic review and meta-analysis of US National Institutes of Health-funded trials. PLOS ONE, 10(9), e0137864. https://doi.org/10.1371/journal.pone.0137864
  4. 4.Sterne, J. A., Sutton, A. J., Ioannidis, J. P., Terrin, N., Jones, D. R., Lau, J., Carpenter, J., Rücker, G., Harbord, R. M., Schmid, C. H., Tetzlaff, J., Deeks, J. J., Peters, J., Macaskill, P., Schwarzer, G., Duval, S., Altman, D. G., Moher, D., & Higgins, J. P. (2011). Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials. BMJ, 343(jul22 1), d4002. https://doi.org/10.1136/bmj.d4002
  5. 5.Terrin, N., Schmid, C. H., Lau, J., & Olkin, I. (2003). Adjusting for publication bias in the presence of heterogeneity. Statistics in Medicine, 22(13), 2113–2126. https://doi.org/10.1002/sim.1461
  6. 6.Bradley, H. A., Rucklidge, J. J., & Mulder, R. T. (2017). A systematic review of trial registration and selective outcome reporting in psychotherapy randomized controlled trials. Acta Psychiatrica Scandinavica, 135(1), 65–77. https://doi.org/10.1111/acps.12647
  7. 7.Egger, M., Davey Smith, G., Schneider, M., & Minder, C. (1997). Bias in meta-analysis detected by a simple, graphical test. BMJ, 315(7109), 629–634. https://doi.org/10.1136/bmj.315.7109.629
  8. 8.Duval, S., & Tweedie, R. (2000). Trim and fill: A simple funnel‐plot–based method of testing and adjusting for publication bias in meta‐analysis. Biometrics, 56(2), 455–463. https://doi.org/10.1111/j.0006-341X.2000.00455.x
  9. 9.Nosek, B. A., Ebersole, C. R., DeHaven, A. C., & Mellor, D. T. (2018). The preregistration revolution. Proceedings of the National Academy of Sciences, 115(11), 2600–2606. https://doi.org/10.1073/pnas.1708274114
  10. 10.Kaplan, R. M., & Irvin, V. L. (2015). Likelihood of null effects of large NHLBI clinical trials has increased over time. PLOS ONE, 10(8), e0132382. https://doi.org/10.1371/journal.pone.0132382
  11. 11.Scheel, A. M., Schijen, M. R. M. J., & Lakens, D. (2021). An excess of positive results: Comparing the standard psychology literature with registered reports. Advances in Methods and Practices in Psychological Science, 4(2). https://doi.org/10.1177/25152459211007467

Further reading

  • Dwan, K., Altman, D. G., Arnaiz, J. A., Bloom, J., Chan, A. W., Cronin, E., Decullier, E., Easterbrook, P. J., Von Elm, E., Gamble, C., Ghersi, D., Ioannidis, J. P., Simes, J., & Williamson, P. R. (2008). Systematic review of the empirical evidence of study publication bias and outcome reporting bias. PLoS ONE, 3(8), e3081. https://doi.org/10.1371/journal.pone.0003081
  • Sterne, J. A., Sutton, A. J., Ioannidis, J. P., Terrin, N., Jones, D. R., Lau, J., Carpenter, J., Rücker, G., Harbord, R. M., Schmid, C. H., Tetzlaff, J., Deeks, J. J., Peters, J., Macaskill, P., Schwarzer, G., Duval, S., Altman, D. G., Moher, D., & Higgins, J. P. (2011). Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials. BMJ, 343(jul22 1), d4002. https://doi.org/10.1136/bmj.d4002

Cite this entry

  • APA 7

    Psych Dictionary. (2026, October 20). Publication bias. In Psych Dictionary. https://www.psychdictionary.com/concepts/publication-bias

  • MLA 9

    “Publication Bias.” Psych Dictionary, 20 Oct. 2026, www.psychdictionary.com/concepts/publication-bias.

Written by Editorial team. First published October 20, 2026. Editorial standards

Educational reference material. Nothing on this site is a diagnosis, a treatment recommendation or a substitute for care from a qualified clinician.