Ability of the respiratory ECMO survival prediction (RESP) score to predict survival for patients with COVID-19 ARDS and non-COVID-19 ARDS: a single-center retrospective study

Abstract

The respiratory ECMO survival prediction (RESP) score is used to predict survival for patients managed with extracorporeal membrane oxygenation (ECMO), but its performance in patients with Coronavirus Disease 2019 (COVID-19) acute respiratory distress syndrome (ARDS) is unclear. We evaluated the ability of the RESP score to predict survival for patients with both non-COVID 19 ARDS and COVID-19 ARDS managed with ECMO at our institution. Receiver operating characteristic area under the curve (AUC) analysis found the RESP score reasonably predicted survival in patients with non-COVID-19 ARDS (AUC 0.76, 95% CI 0.68–0.83), but not patients with COVID-19 ARDS (AUC 0.54, 95% CI 0.41–0.66).

Background

The respiratory ECMO survival prediction (RESP) score is a clinical decision support tool used to predict survival for patients with respiratory failure supported with VV-ECMO [1]. Current guidelines recommend using the RESP score to identify patients with ARDS most likely to benefit from VV-ECMO support [2]. However, the ability of the RESP score to predict outcomes in patients with ARDS caused by COVID-19 is less clear [3,4,5].

Methods

We conducted a single center retrospective observational cohort study comparing the ability of the RESP score to predict survival for patients with COVID-19 ARDS and patients with ARDS from other causes. The Duke Health System Institutional Review Board approved the study with a waiver of informed consent (IRB Pro00090196) prior to data collection. All patients supported with VV-ECMO in the Duke University Hospital Medical Intensive Care Unit (MICU) between January 1, 2009, and December 31, 2021, were eligible for inclusion. Patients were excluded if they were supported with ECMO for indications other than ARDS, were placed on ECMO as a bridge to lung transplant or post-lung transplant, were supported with ECMO > 48 h prior to admission at our institution, or were < 18 years old at the time of cannulation. Data were collected by primary chart review and included baseline demographic and clinical data at the time of cannulation, variables for calculation of Sequential Organ Failure Assessment (SOFA) and RESP scores, etiology of ARDS (i.e., COVID-19 ARDS or non-COVID-19 ARDS), and survival to hospital discharge. The primary aim was to assess the ability of the RESP score to predict survival to discharge using the area under the receiver operating characteristic curve (ROC AUC) and assess the association between the RESP score and survival.

Demographic and clinical characteristics are presented using the median with 25th and 75th percentiles (Q1, Q3) or count (percentage). A ROC curve was constructed for both COVID-19 ARDS and non-COVID-19 ARDS patients. The ROC AUC and 95% confidence interval (CI) are reported, and DeLong’s method was used to test for differences between the AUCs. A logistic regression model including an interaction term between RESP score and COVID-19 status was fit to determine the association between RESP scores and survival to discharge. The odds ratio (OR) and 95% CI are presented for COVID-19 ARDS and non-COVID-19 ARDS patients, as well as the p-value for interaction. R version 4.2.0 and SAS version 9.4 (SAS Institute, Inc., Cary, NC) were used for all analyses, and a p-value < 0.05 was considered statistically significant.

Results

There were 344 patients supported with VV-ECMO during the study period. Of these, 257 met inclusion criteria, including 175 with non-COVID-19 ARDS and 82 with COVID-19 ARDS (Additional file 1: Fig. S1). Baseline data for both cohorts are summarized in Table 1. The median (Q1, Q3) RESP score was similar between the non-COVID-19 ARDS and COVID-19 ARDS cohorts (3.0 [1.0, 5.0] vs 3.0 [2.0, 5.0]). The frequency of components composing the RESP score for each group are shown in Table 2. The ROC AUC for RESP score predicting survival to discharge was 0.54 (95% CI 0.41–0.66) for the COVID-19 ARDS cohort and 0.76 (95% CI 0.68–0.83) for the non-COVID-19 ARDS cohort, a statistically significant difference (p = 0.003) (Fig. 1). Higher RESP scores were significantly associated with survival to discharge in the non-COVID-19 ARDS cohort (OR 1.36, 95% CI 1.21–1.53, p < 0.001) but not in the COVID-19 ARDS cohort (OR 1.09, 95% CI 0.89–1.33, p = 0.39) (Additional file 1: Figs. S2, S3), though testing for an association between hospital survival and COVID-19 status was not significant (p-interaction = 0.065).

Table 1 Patient demographics and clinical characteristics

Table 2 RESP Score components by cohort

Fig. 1

Respiratory ECMO survival prediction score receiver operating characteristic curves. RESP score ROC curves for patients with (A) COVID-19 ARDS and (B) non-COVID-19 ARDS. The AUC for patients with COVID-19 ARDS was 0.54 (95% CI 0.41–0.66) and for patients with non-COVID-19 ARDS was 0.76 (95% CI 0.68–0.83). DeLong’s test for difference between AUCs was significant (p-value = 0.003). RESP  respiratory ECMO survival prediction, ROC  receiver operating characteristic, ARDS  acute respiratory distress syndrome, AUC  area under the curve

Discussion

Our results suggest the RESP score does not accurately predict in-hospital survival for patients with COVID-19 ARDS managed with VV-ECMO. In our COVID-19 ARDS cohort, the RESP score had a poor discriminative ability to predict survival and was not significantly associated with survival.

The reasons for the poor performance of the RESP score in our COVID-19 patients are unclear. Differences in the pathophysiology between COVID-19 ARDS and non-COVID-19 ARDS may reduce the clinical benefit of ECMO support for patients with COVID-19 ARDS. Alternatively, clinical variables not contained in the RESP score may better predict outcomes for patients with COVID-19 ARDS supported with ECMO. It is also possible that because the two study cohorts were treated exclusively in separate, consecutive time periods that differences in outcomes are related to changes in patient care (e.g. staffing shortages, increased patient volumes, different management practices) and not true differences between the cohorts.

Our study has several limitations. Its retrospective design makes it difficult to control for unmeasured confounding. Additionally, as our primary hypothesis was testing the discriminative ability of the previously published RESP score using a ROC curve, we did not adjust for other potential causes of poor outcomes in our models. As all patients were treated at a single center, its external validity may be limited.

In conclusion, the RESP score did not predict survival in patients with COVID-19 ARDS at our high volume ECMO center. Further studies are needed to confirm these findings in larger cohorts of patients with COVID-19 ARDS, especially those patients treated outside the height of the pandemic when shortages in medical staff and resources may have contributed to poor outcomes. Novel clinical decision support tools may be needed to identify patients with COVID-19 ARDS likely to benefit from VV-ECMO support in the future.