The performance of national COVID-19 ‘Symptom Checkers’: A comparative case simulation study

ABSTRACT Introduction The response to COVID-19 differs from nation to nation. There are likely a number of factors one can attribute to such disparity, not least of which is differing healthcare models and approaches. Here, we examine the COVID-19 community triage pathways employed by four nations, specifically comparing the safety and efficacy of national online ‘symptom checkers’ utilised within the triage pathway. Methods A simulation study was conducted on current, nationwide, patient-led symptom checkers from four countries (Singapore, Japan, USA and UK). 52 cases were simulated to approximate typical COVID-19 presentations (mild, moderate, severe and critical), and COVID-19 mimickers (e.g. sepsis and bacterial pneumonia). The same simulations were applied to each of the four country’s symptom checkers, and the recommendations to refer on for medical care or to stay home were recorded and compared. Results The symptom checkers from Singapore and Japan advised onward healthcare contact for the majority of simulations (88% and 77% respectively). The USA and UK symptom checkers triaged 38% and 44% of cases to healthcare contact, respectively. Both the US and UK symptom checkers consistently failed to identify severe COVID-19, bacterial pneumonia and sepsis, triaging such cases to stay home. Conclusion Our results suggest that whilst ‘symptom checkers’ may be of use to the healthcare COVID-19 response, there is the potential for such patient-led assessment tools to worsen outcomes by delaying appropriate clinical assessment. The key features of the well performing symptom checkers are discussed. SUMMARY What is already known? The availability and use of symptom checkers are increasing. Symptom checkers are currently in use at a national level to help in the healthcare response to COVID-19. There is limited evidence to support the effectiveness or safety of symptom checkers as triage tools during a pandemic response. What does this paper add? This study compares performance of symptom checkers across different countries, revealing marked variation between national symptom checkers. The symptom checkers employed by Japan and Singapore are twice as likely to triage cases onward for clinical assessment than those of the US or UK. The US and UK symptom checkers frequently triaged simulated cases of sepsis, bacterial pneumonia and severe COVID-19 to stay home with no further healthcare contact. We discuss the key aspects of the well-performing triage systems.