Investigating Eye-Tracking Features in a Machine Learning Classifier to Assess Cognitive Workload during Simulation Based Training

Cognitive workload can be defined based on the relationship between task requirement and human mental capacity i.e., the number of resources demanded by a task compared to the mental effort required by a person to adequately engage with that task and complete it. One of the current measures to assess cognitive workload are questionnaires, such as the NASA Task Load Index, which ask participants to rate their levels of applied effort, frustration, and temporal demand, among other factors. However, questionnaires include subjective ratings and are intrusive to gather, as the data needs to be collected during or immediately after the task. This makes it difficult to compare objective cognitive workload across participants. In recent years, physiological measures such as eye-tracking, heart and respiration rates have been explored as a complementary assessment method for a continuous and objective measurement of cognitive workload. This thesis focuses on utilizing eye-tracking to identify biomarkers to assess cognitive workload that is manipulated by changes in task load during unmanned aerial systems (UAS) operator training. The objective is to use eye-tracking-derived biomarkers, in addition to behavioral data and questionnaire ratings through machine learning (ML), to determine which feature, or combination of features, would serve as the best predictor of cognitive workload. To achieve this objective, raw eye-tracking data was collected from participants (n = 35), who underwent both 'low' and 'high' workload conditions in simulated real-world UAS tasks. A software toolbox containing custom user-written codes was developed to first denoise and filter out confounding factors and artifacts. The algorithms to extract eye tracking based features including but not limited to blink rate, average pupil diameter change, and saccade velocity were implemented. Finally, Linear Discriminant Analysis (LDA), Support Vector Machine (SVM), and Logistic Regression (LR) as ML classifiers were trained using a combination of features. To evaluate the performance of the models, F1-scores and test accuracies were extracted from each classifier using Leave-One-Group-Out Cross Validation. Combining peak saccade velocity, not-scan percentage, and performance self-ratings into an SVM classifier revealed the highest accuracy and F1-scores. The development of an ML classifier allows for robust and improved accuracy in the classification of differing mental workload states. Future research avenues are identified to enhance the model's predictive capabilities.