I don’t have that much data! reusing user behavior models for websites from different domains

Abstract

User behavior models see increased usage in automated evaluation and design of user interfaces (UIs). Obtaining training data for the models is costly, since it generally requires the involvement of human subjects. For interaction’s subjective quality parameters, like aesthetic impressions, it is even inevitable. In our paper, we study applicability of trained user behavior models between different domains of websites. We collected subjective assessments of Aesthetics, Complexity and Orderliness from 137 human participants for more than 3000 homepages from 7 domains, and used them to train 21 artificial neural network (ANN) models. The input neurons were 32 quantitative metrics obtained via computer vision-based analysis of the homepages screenshots. Then, we tested how well each ANN model can predict subjective assessments for websites from other domains, and correlated the changes in prediction accuracies with the pairwise distances between the domains. We found that the Complexity scale was rather domain-independent, whereas “foreign-domain” models for Aesthetics and Orderliness had on average greater prediction errors for other domains, by 60% and 45%, respectively. The results of our study provide web designers and engineers with a first framework to assess the reusability and difference in prediction accuracy of the models, for more informed decisions.