ROB-MEN: a tool to assess risk of bias due to missing evidence in network meta-analysis

Journal article

Publication bias refers to a systematic deviation from the truth in the results of a meta-analysis due to the higher likelihood for published studies to be included in meta-analyses than unpublished studies. Publication bias can lead to misleading recommendations for decision- and policy-making. In this education review, we introduce, explain, and provide solutions to the pervasive misuses and misinterpretations of publication bias that afflict evidence syntheses in sport and exercise medicine. Publication bias is more routinely assessed by visually inspecting funnel plot asymmetry, although it has been consistently deemed unreliable, leading to the development of statistical tests to assess publication bias. However, statistical tests of publication bias (i) cannot rule out alternative explanations for funnel plot asymmetry (e.g., between-study heterogeneity, choice of metric, chance), and (ii) are grossly underpowered, even when using an arbitrary minimum threshold of ≥10 studies. We performed a cross-sectional, meta-research investigation of how publication bias was assessed in systematic reviews with meta-analysis published in the top two sport and exercise medicine journals throughout 2021. This analysis highlights that publication bias is frequently misused and misinterpreted, even in top tier journals. Due to conceptual and methodological problems when assessing and interpreting publication bias, preventive strategies (e.g., pre-registration, disclosing protocol deviations, and reporting all study findings regardless of direction or magnitude) offer the best and most efficient solution to mitigate the misuse and misinterpretation of publication bias. Because true publication bias is very difficult to determine, we recommend that future publications use the term “risk of publication bias”

Authors: Chiocchia, V; Nikolakopoulou, A; Higgins, JPT; Page, MJ; Papakonstantinou, T

• Publication status: Published
• Peer review status: Peer reviewed
• Publisher: BioMed Central
• Journal: BMC Medicine
• Volume: 19
• Issue: 1
• Pages: 304-304
• Article number: 304
• Publication date: 2021-11-22
• DOI: 10.1186/s12916-021-02166-3
• EISSN: 1741-7015
• ISSN: 1741-7015
• Language: English
• Keywords: Statistics; Missing data; Medicine; Meta-analysis; Internal medicine; MEDLINE; Publication bias