Abstract
Clinicians urgently need reliable and stable tools to predict the severity of COVID-19 infection for hospitalized patients to enhance the utilization of hospital resources and supplies. Published COVID-19 related guidelines are frequently being updated, which impacts its utilization as a stable go-to resource for informing clinical and operational decision-making processes. In addition, many COVID-19 patient-level severity prediction tools that were developed during the early stages of the pandemic failed to perform well in the hospital setting due to many challenges including data availability, model generalization, and clinical validation. This study describes the experience of a large tertiary hospital system network in the Middle East in developing a real-time severity prediction tool that can assist clinicians in matching patients with appropriate levels of needed care for better management of limited health care resources during COVID-19 surges. It also provides a new perspective for predicting patients' COVID-19 severity levels at the time of hospital admission using comprehensive data collected during the first year of the pandemic in the hospital. Unlike many previous studies for a similar population in the region, this study evaluated 4 machine learning models using a large training data set of 1386 patients collected between March 2020 and April 2021. The study uses comprehensive COVID-19 patient-level clinical data from the hospital electronic medical records (EMR), vital sign monitoring devices, and Polymerase Chain Reaction (PCR) machines. The data were collected, prepared, and leveraged by a panel of clinical and data experts to develop a multi-class data-driven framework to predict severity levels for COVID-19 infections at admission time. Finally, this study provides results from a prospective validation test conducted by clinical experts in the hospital. The proposed prediction framework shows excellent performance in concurrent validation (n=462 patients, March 2020-April 2021) with highest discrimination obtained with the random forest classification model, achieving a macro- and micro-average area under receiver operating characteristics curve (AUC) of 0.83 and 0.87, respectively. The prospective validation conducted by clinical experts (n=185 patients, April-May 2021) showed a promising overall prediction performance with a recall of 78.4-90.0% and a precision of 75.0-97.8% for different severity classes.