Review Question - QID 5576

A funnel plot is the most commonly used statistical test for detection of publication bias in meta-analyses.

Publication bias occurs because studies with a non-significant result, so-called negative studies, have a higher likelihood of being rejected than positive studies, and are oftentimes not even submitted for publication. Funnel plots, which plot the effect size of a study against a measure of the study’s size are used to detect this bias. This method is based on the fact that larger studies have smaller variability, whereas small studies, which are more numerous, have larger variability. Thus the plot of a sample of studies without publication bias will produce a symmetrical, inverted-funnel shaped scatter, whereas a biased sample will result in a skewed plot.

Vavken et al. reviewed orthopaedic meta-analyses in order to determine whether publication bias was assessed and to evaluate its effect on the outcomes of these meta-analyses. They found that only 35% of all orthopaedic meta-analyses published between 1992 and 2008 in English and German assessed publication bias. Adjustment for publication bias did not produce significantly different results, but the magnitude of the pooled estimates in the affected meta-analyses changed by 29% on average.

Illustration A depicts a symmetrical funnel plot with no evidence for publication bias. Illustration B shows a skewed funnel plot suggesting publication bias, as it is missing studies in the lower left corner, i.e. ‘‘negative studies’’. Illustration C depicts a forest plot comparing the incidence of squeaking between ceramic-on-ceramic (COC) and ceramic-on-polyethylene (COP). Illustration D is an example of a ROC curve examining the probability of DVT.

Incorrect Answers:
Answer 1: Forest plots illustrate the relative strength of treatment effects in multiple studies, NOT funnel plots.
Answer 3: A graph of the sensitivity versus 1-specificity of a diagnostic test is referred to as a receiver operating characteristic (ROC) curve and is typically used to determine the accuracy of diagnostic tests.
Answer 4: A power analysis is used to determine the sample size required to detect an effect of a given size with a given degree of confidence prior to the start of a clinical study.
Answer 5: A multiple regression analysis is used to predict the unknown value of a variable from the known value of two or more variables in a study.