Venue prediction for social images by exploiting rich temporal patterns in lbsns

Abstract

Location (or equivalently, “venue”) is a crucial facet of user generated images in social media (aka. social images) to describe the events of people’s daily lives. While many existing works focus on predicting the venue category based on image content, we tackle the grand challenge of predicting the specific venue of a social image. Simply using the visual content of a social image is insufficient for this purpose due its high diversity. In this work, we leverage users’ check-in histories in location-based social networks (LBSNs), which contain rich temporal movement patterns, to complement the limitations of using visual signals alone. In particular, we explore the transition patterns on successive check-ins and periodical patterns on venue categories from users’ check-in behaviors in Foursquare. For example, users tend to check-in, to cinemas nearby after having meals at a restaurant (transition patterns), and frequently check-in, to churches on every Sunday morning (periodical patterns). To incorporate such rich temporal patterns into the venue prediction process, we propose a generic embedding model that fuses the visual signal from image content and various temporal signal from LBSN check-in histories. We conduct extensive experiments on Instagram social images, demonstrating that by properly leveraging the temporal patterns latent in Foursquare check-ins, we can significantly boost the accuracy of venue prediction.