Shades of Smiles: Creating Variants of Smiles from Neutral Images of Real Individuals - Method and Validation

Abstract Despite their crucial role in emotion research, facial expression databases primarily contain stimuli of basic emotions, rather than subtle, socially nuanced ones. Here, we introduce a rigorous yet easily applicable pipeline for synthesizing fine-grained emotional expressions—specifically, reward, affiliative, and dominance smiles—from neutral photographs, guided by Facial Action Coding System (FACS) principles and the perspective that expressions serve as social signals. From neutral images of 90 individuals (45 female, 45 male), we generated five expressions each (including neutral and disgust), and mirrored each image to address hemiface biases. In an online validation study, 13 participants rated each image on emotional content, arousal, and plausibility, yielding 26 ratings per expression and model. Rating results show that (1) reward smiles were most reliably recognized, while affiliative and dominance smiles tended to be confused with related expressions and that (2) arousal and plausibility ratings varied systematically across expression types and model gender. In summary, the suggested expression generation pipeline and the validated stimuli offer a robust method and a scalable dataset for research on fine-grained emotion recognition and emotional communication.