The validation of a mobile sensor-based neurobehavioral assessment with machine learning analytics

Abstract Background Mild cognitive impairment is a common yet complex condition that is often underdiagnosed in the early stages. Miro Health is a mobile platform for the self-administration of sensor-based cognitive and behavioral assessments that was developed to measure factors typical of legacy neuropsychological tests in addition to behaviors that are currently left to subjective clinical impression such as eye movement, language, and processing speed. Objective Studies were conducted to measure Miro Health’s concurrent validity, test-retest reliability, and amnestic MCI classification performance. Method Spearman correlations were calculated to estimate the concurrent validity of Miro Health variables with legacy neuropsychological test variables using data from 160 study participants. Fifty-nine healthy controls were assessed at three time points to evaluate the test-retest reliability of Miro Health scores. Reliability was quantified with the scores’ intraclass correlations. Learning effects were measured as trends. In addition, a machine learning algorithm combined Miro Health variable scores into a Risk Score designed to distinguish 65 healthy controls (HC), 38 MCI participants (21 amnestic MCI (aMCI), and 17 non-amnestic MCI (naMCI)). Results Significant correlations of Miro Health variables with legacy neuropsychological test variables were observed. Longitudinal studies show agreement of subsequent measurements and minimal learning effects. The Risk Score distinguished aMCI from healthy controls with an Area Under the Receiver Operator Curve (AUROC) of 0.97; the naMCI participants and controls were separated with an AUROC of 0.80, and the combined MCI group (aMCI + naMCI) was separated from healthy controls with an AUROC of 0.89. Conclusion Miro Health includes valid and reliable versions of variable scores that are analogous to legacy neuropsychological variable scores and a machine-learning derived risk score that effectively distinguishes HCs and individuals with MCI.