Multi-Instance Sketch to Image Synthesis with Progressive Generative Adversarial Networks

Abstract

Real-world images usually contain multiple objects, as a result, generating an image from a multi-instance sketch is an attractive research topic. However, existing generative networks usually produce a similar texture on different instances for those methods focus on learning the distribution of the whole image. To address this problem, we propose a progressive instance texture reserved generative approach to generate more convincible images by decoupling the generation of the instances and the whole image. Specifically, we create an instance generator to synthesize the primitive color distribution and the detailed texture for each instance. Then, an image generator is designed to combine all of these instances to synthesize an image retaining texture and color. Besides, to generate more significant details, such as eyes, ears, and so on, we propose a novel technique called discriminative sketch augmentation, which can provide structural constraint by obtaining the sketch of the discriminative region. The extensive experiments demonstrate that our model not only generates convincing images but also achieves higher inception score and lower Fréchet Inception Distance on the MS-COCO dataset.