Facial Mask Region Completion Using StyleGAN2 with a Substitute Face of the Same Person

Abstract

In recent years, there has been a worldwide outbreak of coronaviruses, and people are wearing facial masks more and more often. In many cases, people wear masks even when taking photos of themselves, and when photos with the lower half of the face hidden are uploaded to web pages or social networking sites, it is difficult to convey the attractiveness of the photographed persons. In this study, we propose a method to complete the masked region in a face using StyleGAN2, a kind of Generative Adversarial Networks (GAN). In the proposed method, we prepare an image of the same person who is not wearing a mask, and change the orientation and contour of the face of the person in the image to match those of the target image using StyleGAN2. Then, the image with the changed orientation is combined with the target image in which the person is wearing the mask to produce an image in which the mask region is completed.