A Deep Learning Framework for Face Sketch Synthesis Using Generative Adversarial Network

Abstract Face sketch synthesis phenomenon, a kind of image-image translation, generates synthesized face/sketch with wide range of applications pertaining law enforcement and entertainment to mention few. Despite the compelling results produced by many existing methods of late, there are still challenges due to deformation and blurred effects on facial components resulting in unrealistic face/sketch. To overcome this problem, in this paper, we proposed a novel framework known as Deep Face-Sketch Synthesis Framework (DFSSF). The framework is realized with different building blocks including an algorithm known as Deep Face-Sketch Synthesis for High Perceptual Quality (DFSS-HPQ). The framework is based on the architecture of Generative Adversarial Network (GAN) which exploits facial structures and a novel labelling mechanism. It takes paired inputs compromising of face images and sketches. The framework also considers extraction of GANs with heterogeneity from inputs. Afterwards, they are stacked to obtain additional features that can be effectively used to rectify defects if any. Two algorithms known as Hybrid GAN for Face Sketch Synthesis (HGAN-FSS) and Stacked Hybrid GAN for Face Sketch Synthesis (SHGAN-FSS) are proposed. We used two frequently used datasets namely CUFS and CUFSF having samples collected from 606 and 1194 persons respectively. The proposed framework is built using Python data science platform. Empirical results of the framework are evaluated and compared with traditional face sketch methods, deep learning models and deep learning models based on GANs. The proposed framework showed better performance over the state of the art in presence of different styles, lighting conditions and head poses.