Controllable image synthesis methods, applications and challenges: a comprehensive survey

Abstract

Controllable Image Synthesis (CIS) is a methodology that allows users to generate desired images or manipulate specific attributes of images by providing precise input conditions or modifying latent representations. In recent years, CIS has attracted considerable attention in the field of image processing, with significant advances in consistency, controllability and harmony. However, several challenges still remain, particularly regarding the fine-grained controllability and interpretability of synthesized images. In this paper, we comprehensively and systematically review the CIS from problem definition, taxonomy and evaluation systems to existing challenges and future research directions. First, the definition of CIS is given, and several representative deep generative models are introduced in detail. Second, the existing CIS methods are divided into three categories according to the different control manners used and discuss the typical work in each category critically. Furthermore, we introduce the public datasets and evaluation metrics commonly used in image synthesis and analyze the representative CIS methods. Finally, we present several open issues and discuss the future research direction of CIS.