Outcome reporting bias in randomised trials

Thesis



Background

Selective reporting of outcomes within a published study based on their nature or direction can result in systematic differences between reported and unreported data. Direct evidence of outcome reporting bias is limited to case reports.

Objective

To study empirically the nature of outcome reporting bias in randomised controlled trials (RCTs).

Methods

Three cohorts of RCTs were identified: PubMed-indexed RCTs published in December 2000; trial protocols approved by a Danish ethics committee from 1994-95; and trial protocols funded by a government agency in Canada from 1990-98. Data on reported and unreported outcomes were recorded from all trial publications and a survey of authors. An outcome was considered incompletely reported if insufficient data were presented for meta-analysis. Odds ratios relating the completeness of outcome reporting to statistical significance were calculated for each trial, and then pooled using a random effects meta-analysis. Protocols and publications were also reviewed for discrepancies in primary outcome reporting.

Results

519 trials with 10,557 outcomes, 102 trials with 3613 outcomes, and 48 trials with 1390 outcomes were identified for the PubMed, ethics committee, and funding agency cohorts respectively. 22%-35% of outcomes per parallel group study were, on average, incompletely reported for meta-analysis. Fully reported outcomes had a two- to three-fold higher odds of being statistically significant compared to incompletely reported outcomes. The most common reasons given for omitting outcomes included a lack of clinical importance, lack of statistical significance, and space constraints. Major discrepancies between primary outcomes in protocols and publications were found in one half of trials.

Discussion and conclusions

The reporting of trial outcomes is frequently inadequate for meta-analysis; is biased to favour statistical significance; and is inconsistent with pre-specified protocol outcomes. Unacknowledged modifications to outcomes specified in trial protocols constitute scientific misconduct. Meta-analyses may therefore produce inflated and unreliable estimates of treatment effect.

Authors: Chan, A; Chan, An-Wen

• Alternative title: implications for systematic reviews
• Publication date: 2003
• DOI: 10.5287/ora-kzepdq4j6
• Type of award: DPhil
• Level of award: Doctoral
• Awarding institution: University of Oxford
• Language: English
• Subjects: Clinical trials; Systematic reviews (Medical research); Medicine; Evaluation; Evidence-based medicine; Research