A Two-Stage Image Inpainting Technique for Old Photographs Based on Transfer Learning

Abstract

To address the challenge of sparse old photo datasets, we apply transfer learning to image inpainting tasks. Specifically, we improve a two-stage image inpainting network that focuses on collaborative subtasks. We also design a transform module based on the cross-aggregation of windows to improve long-distance contextual information acquisition in image inpainting and enhance the integrity of images in terms of structure and texture. Our improved two-stage network has a significantly better repair performance compared to that of the current common inpainting methods. We further apply transfer learning techniques by utilizing the improved two-stage image inpainting network as the base network and decoupling the generator into a feature extractor and classifier, which consist of an encoder and a decoder, respectively. We obtain a domain-invariant feature extractor through minimax game training using source and target domain data. This feature extractor can be combined with the original encoder to restore old photo images. To verify the effectiveness of our approach, we conducted comparative experiments. Our results show that the PSNR, SSIM, and FID indexes of the model using transfer learning are 11.8%, 2.96%, and 44.4% higher than those without transfer learning, respectively. These findings suggest that applying transfer learning techniques can be an effective solution to address the challenge of sparse old photo datasets in image inpainting tasks.