Refining eye synthetic images via coarse-to-fine adversarial networks for appearance-based gaze estimation

Abstract

Recently, several models have achieved great success in terms of reducing the gap between synthetic and real image distributions with large unlabeled real data. However, collecting such large amounts of real data costs a lot of labouring and training them requires high memory. To reduce the gap with less real data, we propose a coarse-to-fine refine eye image method combining coarse model net and fine model net through adversarial training. Coarse model net is a feed-forward convolutional neural network aiming to transform synthetic eye images into coarse images. Fine model net is a modified Generative Adversarial Networks (GANs) which add realism to coarse images using unlabeled real data. Experimental results show that the proposed method achieves similar distributions as recent work but decreasing real data at least one order of magnitude. In addition, a significant accuracy improvement for gaze estimation with refined synthetic eye images is observed.