Friends with Faces: How Social Networks Can Enhance Face Recognition and Vice Versa

Abstract

The friendship relation, a social relation among individuals, is one of the primary relations modeled in some of the worlds largest online social networking sites, such as FaceBook. On the other hand, the co-occurrence relation, as a relation among faces appearing in pictures, is one that is easily detectable using modern face detection techniques. These two relations, though appearing in different realms (social vs. visual sensory), have a strong correlation: faces that co-occur in photos often belong to individuals who are friends. Using real-world data gathered from Facebook, which were gathered as part of the FaceBots project, the worlds first physical face-recognizing and conversing robot that can utilize and publish information on Facebook was established. We present here methods as well as results for utilizing this correlation in both directions. Both algorithms for utilizing knowledge of the social context for faster and better face recognition are given, as well as algorithms for estimating the friendship network of a number of individuals given photos containing their faces. The results are quite encouraging. In the primary example, doubling of the recognition accuracy as well as a sixfold improvement in speed is demonstrated. Various improvements, interesting statistics, as well as an empirical investigation leading to predictions of scalability to much bigger data sets are discussed.