Analysis of characteristics of randomized clinical trials in leukemia that are associated with how results are reported

Journal article

Background

Since many trials are small, systematic reviews are essential for obtaining statistically reliable results. However, some trials are better-reported than others. Non-publication or delayed publication could lead to bias in a review. We identify trial characteristics affecting how quickly or widely results of randomized trials are reported, and hence how likely the trial is to be found by reviewers.

Methods

We analyzed all randomized trials in childhood acute lymphoblastic leukemia that began before 1988 and all articles for these trials published before 2000, as identified by the Childhood Acute Lymphoblastic Leukaemia (ALL) Collaborative Group secretariat. This was the set of 149 trials included in the Second International Collaborative Workshop on Childhood ALL Studies at the end of 1992, comprising 243 randomized comparisons. We used multiple linear regression to analyze time to first mention or to first reporting of results (time to publication), logistic regression for whether a randomization was ever mentioned or reported, and Poisson regression for frequency of mentions or publications.

Results

Collectively, the articles mentioned 217 randomizations, with results reported for 188. Highly statistically significant results were published faster, each tenfold reduction in the p-value (e.g., going from 0.5 to 0.05 or from 0.05 to 0.005) resulting in publication on average 20 months earlier (95% confidence interval 6-34, p=0.005), non-statistically significant results from trials outside North America and Europe took on average 55 months longer than those without these characteristics (95% CI 22-88, p=0.001), and results from trials in high income countries were more likely to reach publication at some point than were results from other countries (odds ratio 7.8, 95% CI 2.4-25.3, p=0.0006). Randomizations in high income countries were mentioned 73 months earlier than those in middle or low income countries (95% CI 51-94, p<0.0001), were more likely to ever be mentioned (OR 13.1, 95% CI 2.1-80.9, p=0.006), and were mentioned more frequently (incidence ratio 2.5, 95% CI 1.4-4.5, p=0.003), as were North American trials compared with those conducted elsewhere (IR 1.3, 95% CI 1.1-1.6, p=0.01).

Conclusions

Systematic reviewers should not rely solely on published reports, but should use additional ways of finding trials in order to minimize biases related to results and other trial characteristics. This relates both to published reports of trial results and to mentions of trials in the literature.

Authors: Burrett, J; Lunn, D

• Publication status: Published
• Peer review status: Peer reviewed
• Journal: Journal of Evidence-Based Medicine
• Volume: 4
• Issue: 4
• Pages: 217-224
• Publication date: 2011-11-01
• Edition: Accepted Manuscript
• DOI: 10.1111/j.1756-5391.2011.01157.x.
• EISSN: 1756-5391
• Language: English
• Keywords: systematic review; meta-analysis; randomized trial; childhood leukaemia; time-lag bias; publication bias
• Subjects: Haematology; Statistics; Medical sciences; Paediatrics