Evaluating visual impressions based on gaze analysis and deep learning: A case study of attractiveness evaluation of streets in densely built-up wooden residential area

Abstract

This paper examines the possibility of impression evaluation based on gaze analysis of subjects and deep learning, using an example of evaluating street attractiveness in densely built-up wooden residential areas. Firstly, the relationship between the subjects' gazing tendency and their evaluation of street image attractiveness is analysed by measuring the subjects' gaze with an eye tracker. Next, we construct a model that can estimate an attractiveness evaluation result using convolutional neural networks (CNNs), combined with the method of gradient-weighted class activation mapping (Grad-CAM) - these in in visualizing which street components can contribute to evaluating attractiveness. Finally, we discuss the similarity between the subjects' gaze tendencies and activation heatmaps created by Grad-CAM.