Early warning scores for detecting deterioration in adult hospital patients: a systematic review and critical appraisal of methodology

Journal article

Objective: To provide an overview and critical appraisal of early warning scores for adult hospital patients.

Design: Systematic review.

Data sources: Medline, CINAHL, PsycInfo, and Embase until June 2019. Eligibility criteria for study selection. Studies describing the development or external validation of an early warning score for adult hospital inpatients.

Results: 13171 references were screened. 95 articles were included in the review. 11 studies were development only, 23 were development and external validation, and 61 were external validation only. Most early warning scores were developed for use in the US (n=13, 38%), followed by UK (n=10, 29%). Death was the most frequent prediction outcome for development (n=10, 44%) and validation (n=66, 79%) studies, with a variety of different prediction time horizons. The most common predictors were respiratory rate (n=30, 88%), heart rate (n=28, 83%), oxygen saturation, temperature and systolic blood pressure (all n=24, 71%). Age (n=13, 38%) and sex (n=3, 9%) were less commonly included. Key details of the analysis populations were often not reported either in development (n=12, 41%) or validation (n=33, 39%) studies. Small sample sizes and numbers of event patients were common in both model development and external validation. Missing data was often discarded, with only one study using multiple imputation. Only nine of the early warning scores that were developed were presented in sufficient detail to allow individualised risk prediction. Internal validation was carried out in 19 studies, but rarely using recommended approaches such as bootstrapping or cross-validation (n=4, 17%). Model performance was frequently assessed using discrimination (development n=15, 94%; validation n=69, 82%), whilst calibration was rarely assessed (validation n=13, 15%). All included studies were rated at high risk of bias.

Conclusions: Early warning scores are widely used prediction models, and often mandated in daily clinical practice to identify early clinical deterioration in hospital patients. Despite wide use, this study found many early warning scores in clinical use have methodological weaknesses. They may not perform as well as expected, and therefore have a potentially highly detrimental effect on patient care. Future work in this area should focus on following recommended approaches for developing and evaluating early warning scores, as well as investigating the impact and safety of using these scores in clinical practice.

Authors: Gerry, S; Bonnici, T; Birks, J; Kirtley, S; Vibrdee, P

• Publication status: Published
• Peer review status: Peer reviewed
• Publisher: BMJ Publishing Group
• Journal: BMJ
• Volume: 369
• Issue: 2020
• Article number: m1501
• Publication date: 2020-05-20
• Acceptance date: 2020-03-25
• DOI: 10.1136/bmj.m1501
• EISSN: 1756-1833
• ISSN: 1759-2151
• Language: English
• Keywords: FFR