Real-world evaluation of AI-driven COVID-19 triage for emergency admissions: External validation & operational assessment of lab-free and high-throughput screening solutions

Abstract


Background
Reducing nosocomial transmission of SARS-CoV-2 has been identified as a priority in safeguarding patient and healthcare staff safety, particularly as individuals with existing medical conditions are at greatest risk of severe illness and death [1][2][3][4][5] .However, as the early clinical course of infection is often characterised by weaklyspecific symptoms, and can be asymptomatic, viral testing is necessary to identify cases and is mandated for all UK hospital admissions. 4e mainstay of testing is batch-processed laboratory polymerase chain assay (PCR), which is imperfectly sensitive and requires specialist equipment [6][7][8] .Turnaround times have shortened throughout the pandemic, typically to within 12-24h in hospitals in high-and middle-income countries, but the interim uncertainty around patients' COVID-19 status may contribute to treatment delays and postpone transfer to wards, thereby contributing to nosocomial transmission and operational strain.Novel rapid testing solutions have been adopted, including point-of-care (POC) PCR, loop mediated isothermal amplification, and lateral flow antigen testing (LFD), despite limitations in throughput and sensitivity 9,10 .Where POC PCR is available, use is typically constrained to time-critical decisions, such as surrounding emergency or transplant surgery, due to supply 11,12 .Moreover, although LFD is lab-free and highly specific (>99.5%) 13,14, allowing for a role in community case-finding 15 , multiple reports show more limited sensitivity (~40%-70%) [16][17][18] leading up to the U.S. Food and Drug Administration issuing a Class 1 recall on June 10, 2021 19 .A recent study evaluating performance amongst unscheduled hospital admissions confirmed high specificity (99.6%), but relatively low sensitivity (62%) 10 .
We recently demonstrated that an artificial-intelligence (AI) screening test (CURIAL-1.0)can rapidly detect COVID-19 amongst patients being admitted to hospital, by recognising SARS-CoV-2-induced abnormalities in routinely collected data 20 .A strength of our approach is the use of readily available blood test, blood gas & physiological measurements which are typically collected within 1h of presentation to hospitals in high-and middle-income countries, without requiring patient exposure to ionising radiation 21,22 .Explainability analyses revealed that features most informative to predictions were components of the Full Blood Count (FBC) and vital signs (Basophil count, Eosinophil count and Oxygen requirements), offering promise for clinically-guided optimisation to reduce prediction time.
Whereas many studies have investigated AI applications for diagnosis and prognosis during the pandemic [23][24][25][26] , key reviews highlight sector-wide methodological and reporting concerns that threaten generalisability, questioning the suitability of many models to-date for clinical use [27][28][29] .Reviewing the contribution of AI to the COVID-19 response, a recent editorial highlighted the promise of CURIAL-1.0amongst other solutions to support patients during the pandemic, discussing the importance of highquality validation studies inclusive of diverse patient populations 30 .Moreover, additional work quantifying benefits in the real-world clinical setting would demonstrate the clinical added-value of such approaches.
Accordingly, in this study we investigate efficacy, generalisability and real-world operational benefits of AI-driven COVID-19 screening in emergency departments, using insight from explainability analyses to improve generalisability and exploit .advances in near-patient diagnostics for lab-free COVID-19 screening 31 .First, we externally validate two models with rationalised predictors, optimised separately for throughput (CURIAL-Lab; vital signs & routine blood tests) and speed (CURIAL-Rapide; vital signs & FBC only), across three independent UK NHS hospital trusts and prospectively for the second wave of the UK COVID-19 epidemic at Oxford University Hospitals.Next, we propose and investigate a novel clinical triage pathway using our AI models to enhance sensitivity of rapid antigen test (LFD)based triage for unscheduled admissions.Lastly, we deploy CURIAL-Rapide alongside an approved point-of-care FBC analyser (OLO; SightDiagnostics, Israel) to provide rapid lab-free COVID-19 screening in the John Radcliffe Hospital's Emergency Department (Oxford, UK), evaluating real-world performance and operational characteristics at a time of falling COVID-19 community prevalence in the UK 32

Diagnostic models to identify patients presenting with COVID-19:
We updated our previously described model, designed to identify patients presenting to hospital with COVID-19 using vital signs, blood gas and routine laboratory blood tests (CURIAL-1.0 20), with additional training data.The model was trained previously using clinical data from patients presenting to emergency and acute medical services at Oxford University Hospitals (OUH) between Dec 1, 2017 and April 19, 2020; additional data on all COVID-19-positive patient presentations to June 30, 2020 were added (Appendix B) 20 .This was performed to encompass all COVID-19 cases presenting to OUH during the 'first wave' of the COVID-19 pandemic (Supplementary Figure S1).OUH consist of four teaching hospitals, serving a population of 600 000, and provides tertiary referral services to the surrounding region.Routine blood tests were full blood count (FBC), urea, creatinine and electrolytes (U&Es), liver function tests (LFTs), coagulation and C-reactive protein (CRP), due to their ubiquity within existing emergency care pathways and rapid results, typically within around 1 h.
Next, we eliminated predictors with lower relative feature importances to improve generalisability across hospitals.CURIAL-Lab uses a focussed subset of routinely performed blood tests (FBC, U&Es, liver function tests, CRP) and vital signs (Table 1), eliminating the coagulation panel and blood gas which are not universally performed and are less informative 20 .Separately, we optimised for result-time, developing a minimalist model (CURIAL-Rapide) considering only predictors that can be obtained by the patient bedside (FBC and vital signs).We selected the FBC due to recent approval of a point-of-care haematology analyser (OLO, SightDiagnostics, Israel) with a result-time of 10 minutes, and as explanability analyses showed FBC components were most informative 31 .Models were trained using the OUH first-wave dataset described above.A combined algorithm to enhance the sensitivity of LFD testing Next, we investigated whether CURIAL-Rapide, CURIAL-Lab and CURIAL-1.0could enhance the sensitivity of LFDs for identifying COVID-19 amongst patients being admitted to hospital.We proposed and retrospectively evaluated a novel clinical triage pathway (Figure 1) labelling patients as COVID-19-suspected where they had either a positive CURIAL result or a positive LFD result.Due to high specificity, in our pathway patients with positive LFDs can be streamed directly to a COVID-19positive clinical area, meanwhile patients with a negative LFD but a positive CURIAL result would be managed in an enhanced-precautions area pending PCR adjudication.The pathway aimed to enhanced negative predictive value for patients receiving both negative LFD and CURIAL results, reduce the false-negative rate and therefore supporting safe and rapid triage directly to a 'green' zone.
We assessed performance of this novel pathway retrospectively for all unscheduled admissions to OUH where patients received LFD testing, from introduction on December 23, 2020 to March 6, 2021.Comparison between the performances of CURIAL-Lab, CURIAL-Rapide and CURIAL-1.0alone are performed with integrated clinical pathway for each model using McNemar's Chi-Square test.
. The study protocol for model development, external and prospective validation was approved by the National Health Service (NHS) Health Research Authority (IRAS ID 281832) and sponsored by the University of Oxford.

Prospective evaluation of CURIAL-Rapide in a lab-free clinical pathway
.
To prospectively assess operational and predictive performance of CURIAL-Rapide in a lab-free setting, we deployed two OLO rapid haematology analysers [SightDiagnostics, Tel Aviv] in the Emergency Department (ED) at the John Radcliffe Hospital, Oxford, as part of an OUH-approved service evaluation (Ulysses ID: 6907) 31 .We simultaneously aimed to improve routine clinical care by reducing the time for routine blood test results to become available in ED.The analysis plan and data requirements for the CURIAL-Rapide evaluation were determined prospectively and registered with the Trust service evaluation database.
We estimated a suitable review-point using Buderer's standard formulae 35 .Predicting a sensitivity of CURIAL-Rapide of 80% (matching model calibration), specificity of 75%, and prevalence of COVID-19 at 15% amongst patients in ED, we estimated a minimum sample size of 410 enrolled patients to determine sensitivity and 85 patients to determine specificity (95% confidence, precision 10% 36 ).We therefore planned to review model performance once 500 patients had been enrolled, to allow for missing or invalid confirmatory tests.
The service evaluation operated from February 18, 2021 to May 10, 2021 between 8am and 8pm for patients meeting the eligibility criteria.Patients eligible were aged over 18, attending the ED with an acute illness and streamed to a bedded clinical area, and had consented to receive full blood count analysis and vital signs as part of their emergency care plan.We selected patients allocated to bedded clinical areas as non-ambulatory patients typically have higher acuity of illness, therefore being more likely to benefit from faster blood test results and having a higher probability of admission.Patients were identified on arrival using the FirstNet system (Cerner Millennium, Cerner, UK).
Eligible patients were enrolled for additional near-patient, lab-free full blood count analysis using the OLO, which in conjunction with vital signs were used to generate CURIAL-Rapide predictions.OLO FBC results were uploaded immediately to the EHR, making results available to clinicians and supporting routine care.We excluded patients with an invalid OLO result and no subsequent successful result, thereby ensuring data completeness.Routine COVID-19 testing was performed in line with trust policies, with LFDs (Innova SARS-CoV-2 Antigen Rapid Qualitative Test) performed in the department and paired multiplex PCR on-premises in a dedicated laboratory (ThermoFisher TaqPath).Patients who did not have confirmatory testing for COVID-19 within routine care were excluded from performance evaluation.
We recorded patients' arrival time to the hospital, measurement time of vital signs, and result times for LFD, PCR, OLO and laboratory FBC analysis.We also recorded the first-attending clinician's triage impression of COVID-19 status, using the locally adopted Green/Amber/Blue categorization system (Green representing a patient whose illness has no features of COVID-19, amber representing an illness with features potentially consistent with COVID-19, and blue representing laboratory confirmed COVID-19 infection) 37,38

Role of the funding source:
The funders of the study had no role in study design, data collection, data analysis, data interpretation, or writing of the manuscript. .

Results:
Patient Populations  A sensitivity analysis to assess susceptibility of our models to imputation strategy demonstrated consistent performance across multiple imputations (Table 3).We therefore performed subsequent evaluation using a model trained solely using a single imputation strategy (population median). .

Comparing CURIAL triage performance with LFDs & evaluating a combined CURIAL-LFD clinical pathway to enhance LFD sensitivity
We applied CURIAL-Rapide and CURIAL-Lab to the first-performed blood tests and vital signs for 3,207 patients admitted to OUH, receiving LFD testing from introduction on December 23, 2020 to March 6, 2021 (Figure 3, Supplementary Table S6).Patient eligibility is demonstrated in Supplementary Figure S2.One patient with an invalid LFD result was excluded from analysis.LFDs achieved a high specificity of 99.8% (99.6 -99.9) during triage, but poor sensitivity of 56.9% (51.7 -62.0).

Deployment & operational evaluation of CURIAL-Rapide at Oxford University Hospitals NHS Foundation Trust
520 patients were enrolled to the OLO/CURIAL-Rapide service evaluation between February 18, 2021 and May 10, 2021 (baseline characteristics, Table 2).436 patients received confirmatory PCR testing within routine care, of whom 10 returned positive results (prevalence 2.3%).This reflected the falling prevalence of COVID-19 in the UK, coinciding with governmental restrictions and the national vaccination programme 32 .348 patients received LFDs within routine care, with 4 positive results.Two patients with indeterminate PCR results were excluded from analysis, although both had a negative LFD result and were triaged by the assessing clinician to a COVID-19-free clinical pathway.A summary of OLO results and vital signs are shown in Supplementary Tables S2-S3.survival analyses (Figure 4a) showed CURIAL-Rapide results were available sooner than LFDs (log rank test, p<0.0001) and PCR results (p<0.0001).The median timeto-result for full blood count analysis was shorter with near-patient OLO analysis (44:00 min, 31:00-63:00 min) than laboratory analysis (76:00 min, 58:00-100:00 mins; p <0.0001), confirming an improvement to routine care.
In one presentation, a patient given a 'negative' CURIAL-Rapide prediction went on to have a positive SARS-CoV-2 PCR test, although they had a negative LFD result and were triaged to a COVID-19-free ('green') clinical area.The patient did not have COVID-19 symptoms on this presentation.We noted that the patient had also been enrolled to the service evaluation 10 days prior; on that occasion having a positive CURIAL-Rapide prediction, positive Lateral Flow Test and a positive PCR test.This raises the possibility of a latent positive PCR result, detecting non-infectious residual viral fragments, on the date of the second presentation 39,40 .
Rates of COVID-19 status misclassification were comparable between CURIAL-Rapide and clinician judgement (McNemar's Exact test; p=0.91).Moreover, of the 53 patients who were triaged to a 'COVID-19-suspected' (amber) pathway by the attending clinician but went on to test negative by PCR, 31 patients (58.5%) had a negative CURIAL-Rapide prediction demonstrating that the AI system could reduce operational strain by expediting clinical exclusion of infection.As all patients with positive LFD results also had positive CURIAL-Rapide predictions, a combined CURIAL-Rapide/LFD pathway did not impact classifier performance in this evaluation.

Discussion
National health policy recognises that effective in-hospital triage is necessary to safeguard patient and staff safety, mandating COVID-19 testing for all admissions 4 .However, despite significant innovation leading to new near-patient testing options, alongside reduced PCR result-times to typically within 12-24 h, there remain significant performance and logistical limitations that contribute to nosocomial transmission and operational strain.While many hospitals have adopted LFDs within acute admissions pathways 10 , our study confirms a limited sensitivity (56.9%) indicating a clinically-meaningful false negative rate (Figure 3, Supplementary Table S6) [14][15][16]41 .
In this study we demonstrate generalisability, efficacy, and real-world operational benefits of AI-driven COVID-19 screening in the acute care setting.Whereas rapid molecular testing options are frequently rationed 11,12 , we show that a high-throughput AI-solution, CURIAL-Lab, rapidly excludes COVID-19 using routine data and generalises across three independent hospital groups (Figure 2).Moreover, we improve upon the speed of existing rapid testing solutions, demonstrating a median result-time of 45 minutes (32-64 min; CURIAL-Rapide) from patients' first arrival in an emergency department using near-patient haematology analysis.This decentralised approach may support time-critical decision making and assist triage in remote and primary care settings where laboratory facilities are less readily available.
In our external and prospective validation of CURIAL-Lab & CURIAL-Rapide, model performance was consistently high across four UK hospital groups (CURIAL-Lab: AUROC range 0.858-0.881;CURIAL-Rapide 0.836-0.854),with high negative predictive values confirming suitability as tests-of-exclusion for COVID-19.CURIAL-Lab expectedly achieved marginally superior performance to CURIAL-Rapide, representing a trade-off between result-time and specificity which would favour different clinical use-cases.Strengths of the validation include geographic breadth, including over 72,000 patients across three regions of the UK (Midlands, South East, and East of England), and temporal breadth across both waves of the UK COVID-19 epidemic therefore including vaccinated patients and patients with Coronavirus variants, across a range of prevalences (4.27%-12.2%).Moreover, the validation study considers the broad range of confirmatory COVID-19 tests (PCR, SAMBA-II & Panther) utilised across different centres, and address sector-wide concerns highlighted in key reviews of COVID-19 diagnostic and prognostic models by using external, representative cohorts of all unscheduled adult admissions 27,28 .Notable limitations include that the external validation is solely UK-based and the limitations of the confirmatory testing methods, with PCR testing having been shown to be imperfectly sensitive 6,7 .We were unable to quantify the number of vaccinated patients as we could not link our de-identified hospital datasets with vaccination records.
Our study finds that CURIAL-Rapide & CURIAL-Lab achieve significantly higher sensitivity and NPV than LFDs, improving upon standard-care by reducing risk of a COVID-19-positive patient being streamed to a COVID-19-free clinical area.Moreover, our study is the first to validate an application of an AI test to enhance sensitivity of LFDs in a real-world clinical setting, with our combined clinical pathway .
(Figure 1; CURIAL-Lab/LFD) achieving both high sensitivity of 85.6% (81.6 -88.9) and overall classification performance with AUROC of 0.925 (0.905 -0.945).A significant beneficiary population includes patients streamed to COVID-19-free ('green') clinical areas on receiving negative LFD and CURIAL results, which are performed in parallel and available much sooner than PCR results.Moreover, our pathway identifies an enriched subpopulation at greater risk of testing positive for COVID-19 (negative LFD & positive CURIAL-Lab/Rapide compared to negative LFD alone), therefore enabling prioritisation for rapid confirmatory testing where availability is limited.Limitations of this analysis include that patients may have been inadvertently excluded if LFDs were recorded incorrectly on the EHR, and the analysis was performed only for a single trust as other participating sites did not electronically record LFD results.
We report the fastest result-time to date for AI-driven COVID-19 screening in a hospital emergency department, using lab-free haematology analysis to achieve a median reduction of 16 minutes (26.3%) over LFDs.Significantly, by demonstrating that CURIAL-Rapide correctly excluded COVID-19 for 58.5% of negative patients who were triaged by a clinician to a 'COVID-19-suspected' (amber) clinical area, we show a role for AI screening in reducing delays in transfers to wards.A strength of our service evaluation is its real-world context and operational focus, assessing time from first-arrival to result availability, and demonstrating added clinical value by comparison to LFD results and clinician impressions.In this study, we address the need for evidence of clinical utility that AI tools such as CURIAL offer to the pandemic response 30 ; demonstrating both operational and safety improvements to standard-care.
A significant limitation of our service evaluation is that, although the a priori target sample size was achieved, the desired precision and power levels were not achieved for the metric of sensitivity due to sharply falling prevalence of COVID-19 in the UK, associated with the national vaccination programme and public health measures 32,42 .The evaluation was, however, adequate to determine specificity.As a service evaluation, we used a convenience series, with OLO operation limited to daytime and evening hours (8am-8pm) for logistical reasons.Moreover, although routine LFD testing was hospital policy, 33% of enrolled patients did not have a coded result in their EHR using the pre-specified form raising the possibility that these may have been recorded elsewhere or communicated verbally.As all patients who were LFD positive also had a positive CURIAL-Rapide result in our study, a larger evaluation is needed to assess whether integrating LFD results could further improve performance of CURIAL-Rapide in this context.Further evaluation would assess performance as a clinical decision support aid and for sensitivity across coronavirus variants.
A major strength of the CURIAL-Rapide and CURIAL-Lab solutions is the use of clinical data that is readily available and routinely collected for all patients admitted to hospital.Our approach optimises generalisability with CURIAL-Lab, applicable to virtually all unscheduled patient admissions to hospital, thereby facilitating COVID-  S1.

Normalisation
Data normalisation was implemented to mitigate overfitting and to avoid the reliance of the model on measurement units.Categorical data were handled by encoding as "1-hot" variables.Where a lab value was reported as being below the threshold of detection of the laboratory assay, the value was replaced with a numerical zero value.Where values were reported as being above the threshold of detection, clinically appropriate values were selected to maintain the significance of the high result.A summary of first-performed blood tests, vital signs, and blood gasses on arrival to hospital are shown Supplementary Tables S2-S4.

Missing Data
Multiple imputation strategies, population median, population mean, and age-based imputation, were separately used to impute missing data initially.As a sensitivity analysis to assess for effects of imputation strategy on model performance, we assessed performance of models trained using each imputation method prospectively for all patients attending emergency departments and acute medical services across OUH during the second-wave of the COVID-19 pandemic (October 01, 2020 and March 06, 2021; Table 2).Mean performance was reported alongside SD in Table 2, with narrow standard deviations in all performance metrics demonstrating resilience to imputation method.We therefore subsequently only used models trained with missing data imputed using population median, reporting results alongside 95% confidence intervals (CIs).

Model Training & Prospective Evaluation:
We repeated training and optimisation of our eXtreme Gradient BOOSTed tree model (XGBoost) to discriminate COVID-19-positive cases from pre-pandemic COVID-19-negative controls, for each of the three feature-sets (Table 1) 44 .During training using 'first wave' case, controls were matched for age, gender, and ethnicity at a ratio of 1:20.For missing data, we initially used three independent imputation

Comparison with Lateral Flow Tests
We considered any positive lateral flow test which was followed by a positive PCR test within a +/-48hr window of a patient being admitted to hospital to represent a true positive infection.As previously, model predictions were generated using blood tests performed from the first blood draw on arrival and first-recorded vital signs.In the integrated clinical pathway (Figure 1), patients were considered COVID-19suspected if they had either a positive LFD result or CURIAL prediction.Results are show in Figure 3 and Supplementary Table S6.samples were not processed for 2 of the 520 patients, owing to sample or labelling errors.For paired samples, we compared time-to-result between each test using a one-tailed Wilcoxon Signed Rank test.We additionally performed a Kaplan-Meier survival analysis (Figure 4).
We report our study in compliance with the "Standards for Reporting Diagnostic accuracy studies" (STARD) standards 46,47 .

Figure 1 :
Figure 1: A rapid COVID-19 screening pathway for Emergency Department arrivals (a) routine blood tests and vital signs recordings are performed on patients' arrival to the emergency department, either using rapid point-of-care haematology analysers (~10 minutes; CURIAL-Rapide), or the existing laboratory analysis (~1h; CURIAL-Lab).(b) Realtime algorithmic analysis of blood tests & vital signs allows early, high-confidence identification of negative patients for safe streaming to COVID-19-Free clinical areas.(c) Patients with positive screening test results are admitted to enhanced precautions (amber) areas, pending confirmatory PCR result.(d) Patients testing positive via Lateral Flow Test can be streamed directly to COVID-19 ('red') clinical areas.Arrow thickness represents patient flow.

Figure 2 :
Figure 2: Performance of CURIAL-1.0,CURIAL-Lab & CURIAL-Rapide during external validation at three independent UK Hospitals trusts.All models were calibrated during training to achieve 90% sensitivity.Error bars show 95% confidence intervals.Numerical results are shown in Supplementary TableS5.

Figure 3 :
Figure 3: Performance characteristics of (a) INNOVA SARS-CoV2 Rapid Antigen Tests, (b) CURIAL-Rapide & CURIAL-Lab, calibrated during training to a sensitivity of 80%, and (c) combined clinical pathways considering either a positive CURIAL-Rapide/CURIAL-Lab result or a positive LFD test as a COVID-19 suspected case, at Oxford University Hospitals NHS Foundation Trust between December 23, 2020 & March 6, 2021.Error bars show 95% confidence intervals.Numerical results are shown in Supplementary TableS6.

Table 6 :
Operational and performance characteristics of (a) CURIAL-Rapide, (b) INNOVA SARS-CoV-2 Rapid Antigen Testing and (c) clinical triage by the first-attending clinician calculated against laboratory RT-PCR testing.Results are reported alongside interquartile range (time-to-result) or 95% confidence intervals.

Figure 4 :
Figure 4: (a) Kaplan-Meier plots of time-to-result in hours from patient arrival in the Emergency Department for (i) CURIAL-Rapide, (ii) INNOVA SARS-CoV-2 Rapid-Antigen Tests and (iii) PCR swabs tests (Numbers at risk: 520, 348 and 436 respectively).CURIAL-Rapide results were available sooner than LFD testing (log rank test, p<0.0001) and PCR test results (p<0.001).(b) Receiver operating characteristic curve showing performance of (i) CURIAL-Rapide (ii) clinical judgement of the first-attending clinician, and (iii) INNOVA SARS-CoV-2 rapid antigen testing, against a PCR reference standard.

Figure 1 :
Figure 1: A rapid COVID-19 screening pathway for Emergency Department arrivals (a) routine blood tests and vital signs recordings are performed on patients' arrival to the emergency department, either using rapid point-of-care haematology analysers (~10 minutes; CURIAL-Rapide), or the existing laboratory analysis (~1h; CURIAL-Lab).(b) Realtime algorithmic analysis of blood tests & vital signs allows early, high-confidence identification of negative patients for safe streaming to COVID-19-Free clinical areas.(c) Patients with positive screening test results are admitted to enhanced precautions (amber) areas, pending confirmatory PCR result.(d) Patients testing positive via Lateral Flow Test can be streamed directly to COVID-19 ('red') clinical areas.Arrow thickness represents patient flow.

Figure 2 :
Figure 2: Performance of CURIAL-1.0,CURIAL-Lab & CURIAL-Rapide during external validation at three independent UK Hospitals trusts.All models were calibrated during training to achieve 90% sensitivity.Error bars show 95% confidence intervals.Numerical results are shown in Supplementary TableS5.

Figure 3 :
Figure 3: Performance characteristics of (a) INNOVA SARS-CoV2 Rapid Antigen Tests, (b) CURIAL-Rapide & CURIAL-Lab, calibrated during training to a sensitivity of 80%, and (c) combined clinical pathways considering either a positive CURIAL-Rapide/CURIAL-Lab result or a positive LFD test as a COVID-19 suspected case, at Oxford University Hospitals NHS Foundation Trust between December 23, 2020 & March 6, 2021.Error bars show 95% confidence intervals.Numerical results are shown in Supplementary TableS6.

.
CC-BY 4.0 International license It is made available under a perpetuity.is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprintThe copyright holder for this this version posted November 6, 2021.; https://doi.org/10.1101/2021.08.24.

Figure 4 :
Figure 4: (a) Kaplan-Meier plots of time-to-result in hours from patient arrival in the Emergency Department for (i) CURIAL-Rapide, (ii) INNOVA SARS-CoV-2 Rapid-Antigen Tests and (iii) PCR swabs tests (Numbers at risk: 520, 348 and 436 respectively).CURIAL-Rapide results were available sooner than LFD testing (log rank test, p<0.0001) and PCR test results (p<0.001).(b) Receiver operating characteristic curve showing performance of (i) CURIAL-Rapide (ii) clinical judgement of the firstattending clinician, and (iii) INNOVA SARS-CoV-2 rapid antigen testing, against a PCR reference standard.

which was not certified by peer review) preprint
.

External validation at independent NHS trusts:
Patients with PCR-confirmed COVID-19 in the 10 days preceding attendance were adjudicated to the 'Blue' (COVID-19-confirmed) stream.Patients with no features of COVID-19 and no documented clinical suspicion were adjudicated to the 'Green' stream.We generated CURIAL-Rapide predictions using OLO results and vital signs, comparing predictions against results of confirmatory PCR testing.CURIAL-Rapide predictions were not made available to the attending clinician so as not to influence the clinical triage category or decisions to proceed to confirmatory testing for patients being discharged.Availability time for CURIAL-Rapide was the later of OLO result time and vital signs recording time as both are required to generate a prediction.The time-to-result for PCR, lateral flow, and CURIAL-Rapide tests were calculated as the time from a patient's first arrival in the ED to the time of a test result being available on the EHR.
. Where COVID-19 triage category had not been documented by the first-assessing physician, adjudication was performed through clinical review of notes using rules-based determination.Patients having documentation of a new continuous cough, temperature ≥37.8°C, or loss or change in sense of smell or taste were adjudicated as an 'amber' (COVID-19-suspected) stream, matching UK Government guidance on definition of a possible COVID-19 .case 8.We selected time-to-result as our primary outcome, recognising the role of rapid test results in reducing nosocomial transmission.Our secondary outcomes were sensitivity, specificity, PPV and NPV for CURIAL-Rapide & LFDs, and AUROC for CURIAL-Rapide, assessed against PCR results.Further detail is provided in Appendix D.

Table 2 :
Summary population characteristics for (a) OUH pre-pandemic and COVID-7%; chi-square p<0.0001) and reported a South Asian ethnicity (13.2% versus 0.5% and 2.0%; chi-square p<0.0001).Prevalence of COVID-19 was higher in the Bedfordshire cohort owing to the evaluation period matching the timeline of the second wave of the UK COVID-19 epidemic (11.1% versus 5.29% (PUH) and 4.27 (UHB); Fisher's exact test p<0.0001& <0.0001).Summaries of vital signs, index routine blood tests and blood gases are presented in Supplementary Tables S2-S4.

Table 3 :
Evaluation of the performance of (a) CURIAL-Rapide and (b) CURIAL-Lab, calibrated during training to achieve sensitivities of 80% and 90%, on an independent prospective set of all admissions to OUH during the second-wave of COVID-19, between October 1, 2020 and March 6, 2021.(c) Benchmark performance of CURIAL-1.0(Soltan et al. 2020) on the prospective set.Mean values are reported alongside SD across three imputation methods.

Table S5 .
23,28,43ning without significant additional cost.Where faster exclusion of COVID-19 is helpful for operational or treatment reasons, CURIAL-Rapide can provide faster results by eliminating the need for blood sample transportation and laboratory processing, at an approximate cost of around ~£9 (~$12.50;inclusive of device rental and consumables).By contrast, alternative strategies for AI-assisted COVID-.19diagnosislargelyfocus on chest imaging23,28,43, which involve patient exposure to ionising radiation and have higher costs.Following successful COVID-19 vaccination programmes, falling community prevalence may reduce cost-effectiveness of universal PCR-testing for unscheduled admissions.Our results suggest that CURIAL-Lab could deliver significant cost savings by reducing the number of routine PCR tests by >85% (where prevalence <2%) while achieving high NPV, utilising data that would be collected within the routine course of a patients' care.47.Liu, X., Cruz Rivera, S., Moher, D., Calvert, M. J. & Denniston, A. K. Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension.Nature medicine vol.26 1364-1374 (2020).As previously, we included all patients attending acute and emergency care settings at Oxford University Hospitals NHS foundation trust who received routine blood tests on arrival, considering presentations before December 1, 2019, and thus before the pandemic, as the COVID-19-negative (control) cohort.We considered presentations during the 'first wave' of the UK COVID-19 pandemic (December 1, 2019 to June 30, 2020) with PCR confirmed SARS-CoV-2 infection as the COVID-19-positive (cases) cohort.We excluded patients who opted out of electronic health record (EHR) research and those who did not receive laboratory blood tests or were younger than 18 years of age.Due to incomplete penetrance of testing during the first wave of the pandemic, and imperfect sensitivity of the PCR test, there is uncertainty in the viral status of patients presenting during the pandemic who were untested or tested negative.We therefore selected a pre-pandemic control cohort during training to ensure absence of disease in patients labelled as COVID-19-negative.Clinical features extracted for each presentation included first-performed blood tests, blood gases, vital signs measurements and PCR testing for SARS-CoV-2 (Abbott Architect [Abbott, Maidenhead, UK], TaqPath [Thermo Fisher Scientific, Massachusetts, USA] and Public Health England-designed RNA-dependent RNA polymerase assays).A list of extracted clinical features is shown in Supplementary Table

which was not certified by peer review) preprint
. CC-BY 4.0 International license It is made available under a perpetuity. is the author/funder, who has granted medRxiv a license to display the preprint in (The copyright holder for this this version posted November 6, 2021.; https://doi.org/10.1101/2021.08.24.21262376 doi: medRxiv preprint