FDA-authorized AI/ML tool for sepsis prediction: Development and validation

Document Type

Article

Publication Date

11-27-2024

Publication Title

NEJM AI

Abstract

Background

Sepsis is a life-threatening condition that demands prompt treatment for improved patient outcomes. Its heterogenous presentation makes early detection challenging, highlighting the need for effective risk assessment tools. Artificial intelligence (AI) models could potentially identify patients with sepsis, but none have previously been authorized by the U.S. Food and Drug Administration (FDA) for commercial use. This study outlines the development and validation of the Sepsis ImmunoScore, the first FDA-authorized AI-based software designed to identify patients at risk of sepsis.

Methods

In this prospective study, we enrolled adult patients (18+ years of age) suspected of infection, as indicated by a blood culture order, from five U.S. institutions between April 2017 and July 2022. The participants were divided into an algorithm development cohort (n=2366), an internal validation cohort (n=393), and an external validation cohort (n=698). The primary end point was sepsis presence (as defined by Sepsis-3) within 24 hours of test initiation. Secondary end points included length of hospital stay, intensive care unit (ICU) admission within 24 hours, mechanical ventilation use within 24 hours, vasopressor use within 24 hours, and in-hospital mortality.

Results

The Sepsis ImmunoScore demonstrated high diagnostic accuracy, with an area under the curve of 0.85 (0.83 to 0.87) in the derivation cohort, 0.80 (0.74 to 0.86) in internal validation, and 0.81 (0.77 to 0.86) in external validation. The scores were categorized into four sepsis risk levels with corresponding likelihood ratios: low (0.1), medium (0.5), high (2.1), and very high (8.3). These risk categories also predicted in-hospital mortality: low (0.0%), medium (1.9%), high (8.7%), and very high (18.2%) in the external validation cohort. Similar trends were observed for other metrics, such as length of hospital stay, ICU utilization, mechanical ventilation, and vasopressor use.

Conclusions

The Sepsis ImmunoScore demonstrated high accuracy for the identification and prediction of sepsis and critical illness metrics that could enable prompt identification of patients at high risk of sepsis and adverse outcomes, potentially improving clinical decision-making and patient outcomes. (Funded by the Defense Threat Reduction Agency and others.)

Volume

1

Issue

12

First Page

AIoa2400867

DOI

10.1056/AIoa2400867

Link to Full Text

Share

COinS