Face parsing and blending-based analysis model for makeup style transfer

Abstract Makeup is a fundamental element of traditional opera culture, embodying profound artistic connotations and aesthetic values. Incorporating patterns and decorative elements into makeup has also become an essential aspect of contemporary society. Makeup transfer technology enables the reproduction of existing makeup designs onto specified facial images. However, current methodologies face several challenges, including incomplete pattern replication and suboptimal results, particularly in transferring patterned makeup. Consequently, this paper proposes a novel makeup transfer model, termed Opera-MT, which integrates facial parsing with image-fusion techniques. The Opera-MT model first maps facial images onto UV space to establish a pixel-accurate correspondence between the reference face and the source face, effectively addressing issues related to pattern distortion. Subsequently, it integrates facial parsing techniques with Poisson image fusion methods to preserve the texture details of the pattern, thereby overcoming the challenge of incomplete pattern migration in the reference makeup. Finally, the model utilizes a color transfer network to apply the color style of the reference makeup to the source face, resulting in the final output. The effectiveness of Opera-MT has been validated on three datasets, including our collected PM dataset and two publicly available datasets, namely the MT and makeup datasets. The results demonstrate that Opera-MT successfully captures makeup color and pattern texture, producing high-quality, realistic, and natural results. Our code will be available at https://github.com/xiaozhang0927/Opera-MT.