Validation of the APACHE IV Score for ICU Mortality Prediction in Dr. Sardjito Hospital During the Pandemic Era

Introduction: ICU service quality must continuously improve to provide better patient service. One of these improvement efforts is the use of a risk prediction system to predict mortality rates in the ICU by utilizing risk factors. This system helps healthcare services perform evaluations and comparative audits of intensive services, which can also aid with more targeted planning. APACHE IV is considered to have good validity. However, its predictive capabilities may change over time due to various factors, such as the pandemic, where changes in the case mix may affect its predictive abilities. Therefore, this research tests the validity of APACHE IV on the Indonesian population through Dr. Sardjito Hospital patients. The findings can be utilized for future use and risk stratification, and ICU quality benchmarking. Objectives: This study aims to assess the validity of the APACHE IV score in ICU Mortality prediction in Dr. Sardjito Hospital for medical patients, surgical patients, and patients with both cases during the pandemic. Materials and Method: This study used retrospective data from 336 patients at Dr. Sardjito Hospital Yogyakarta from the 1st of January 2020 to the 31st of December 2021. All data required for calculating the APACHE IV score was collected, and the patient’s observed ICU Mortality was used. The model’s predictive validity is measured by finding the discrimination and calibration of the APACHE IV score and comparing it to the observed ICU mortality. Validation was also conducted separately for medical and surgical cases. Results: APACHE IV shows good discrimination ability in all cases (AUC-ROC 95% CI: 0.819 [0.772-0.866]) but poor calibration (p = 0.023) for mortality prediction in the ICU. For medical cases, the discrimination ability is poor but still acceptable (AUC-ROC 95% CI: 0.698 [0.614-0.782]), and in surgical cases, the discrimination ability is good (AUC-ROC 95% CI: 0.848 [0.776-0.921]). Both cases showed good calibration (p: medical = 0.569, surgical = 0.579) in predicting mortality during the pandemic. Conclusion: APACHE IV showed good discrimination but poor calibration ability for predicting mortality for all ICU patients during the pandemic era. Mortality prediction for surgical cases showed good discrimination and calibration. However, medical cases showed poor discrimination but good calibration.