Developing Deep Learning Methods for Cognitive Impairment Detection Based on Non-invasive Data

Cognitive impairment detection is on the rise to help reduce the burden of healthcare costs on institutions and individuals. Mild Cognitive Impairment (MCI) is an early stage of cognitive decline progressing to Alzheimer’s disease (AD) or AD-related Dementia (ADRD). Detecting the early stages of AD/ADRD is crucial for early interventions among older adults to mitigate cognitive decline over time. However, the current diagnostic methods are often costly and/or invasive, such as MRI and PET scans. Thus, the search for non-invasive and cost-effective screening tools for the early detection of cognitive impairment using speech, language, visual, and motor data is growing. These methods are intended to identify at-risk individuals for further clinical evaluation, not to serve as a standalone diagnosis. Cognitively impaired older adults face challenges in direct communication (i.e., speech and facial expressions) in natural settings. These challenges have been studied in clinical trials. However, deploying Deep Learning (DL) models to capture these challenges and detect the cognitive status of potential patients remains underexplored. This Ph.D. dissertation presents research studies on developing DL models for cost-effective and non-invasive data to detect the participants’ cognitive status. Through the analysis of non-invasive data, including facial expressions, head poses, language, and neurological assessment responses, I explored several approaches to early detection of cognitive decline. In addition, I developed digital twins to project the future behavior and predict the cognitive condition of the participants. Firstly, the subtle changes in participants’ facial expressions or head movements could carry insights into the participants’ cognitive status. Based on these observations, I extracted the facial expressions and head poses of the participants’ responses during the interviews to analyze the facial features of the participants. Also, I deployed several machine-learning models to predict the participants’ cognitive conditions. Moving forward, I implemented a Convolutional AutoEncoder (CAE) to extract holistic, unlabeled facial features of the participants. This approach was adopted due to the limited availability of pre-trained DL models for extracting facial features from elderly subjects. Then, I created a DL framework that utilizes samples of the video frames by capturing the temporal information of the samples using a Transformers encoder to detect the cognitive conditions from the facial features. This work shows the feasibility of utilizing facial videos of older adults in a home setting as an early, non-invasive detection method for MCI. Additionally, the interaction patterns between potential cognitively impaired subjects and trained staff can reveal cognitive conditions. Thus, I proposed the Spatial-to-Temporal Attention Module (STAM) to detect the I-CONECT study participants’ cognitive conditions (MCI vs. normal cognition (NC)) using facial and interaction features. The interaction features of the facial features improved the prediction performance compared with applying facial features solely. Overall, the results show that spatiotemporal facial features modeled using DL algorithms have a discriminating power for the detection of MCI. Finally, to advance the detection of cognitive impairment conditions, I developed deep learning digital twins using a conditional convolutional autoencoder (cVAE) to predict participants’ future data and cognitive conditions at subsequent visits. Specifically, this method utilized participants’ non-invasive neurological assessment data or speech transcripts from picture description tasks. These frameworks forecast participants’ future data and predict their cognitive conditions at subsequent visits.