Emotion recognition through advanced signal fusion and kolmogorov-arnold networks

Emotion recognition is an important aspect of affective computing that has many applications including human computer interaction and mental health care. In this thesis, Kolmogorov-Arnold Networks (KANs) are explored for the purpose of multimodal emotion recognition from EEG (Electroencephalogram) and eye movement signals of the SEED-V dataset. Both modalities were represented by Differential Entropy (DE) features and a two-branch KAN architecture was used which composed of seven KAN variants, namely, Efficient KAN, Jacobi KAN, Hermite KAN, Fourier KAN, Chebyshev KAN, RBF KAN, and Wavelet KAN (with different wavelet types including DOG, Morlet, Mexican hat, and Meyer). The strengths of two modalities were combined through late fusion for the final classification. The results show that Wavelet KANs outperformed other variants and Wavelet KAN (DOG) and Wavelet KAN (Morlet) achieved accuracies higher than 96%. The KAN models effectively identified the localized and transient feautres in EEG and eye movement signals, which was also reflected in the smooth training convergence and nearly diagonal confusion matrices. Poor performance was observed with Fourier KAN while polynomial-based and RBF KANs gave fairly good results. Efficient KAN had good accuracy but had some problems with validation instability. This work also shows that flexible non-linear, multi scale models are needed for modeling non-stationary physiological signals. The findings also indicate that future work should consider hybrid KAN architectures and more sophisticated fusion strategies to improve the robustness of emotion recognition systems. The proposed framework emphasizes the potential of wavelet-based KANs as a foundation for more adaptive, accurate affective computing solutions.