Photo Sleuth: Identifying Historical Portraits with Face Recognition and Crowdsourced Human Expertise

Abstract

Identifying people in historical photographs is important for preserving material culture, correcting the historical record, and creating economic value, but it is also a complex and challenging task. In this article, we focus on identifying portraits of soldiers who participated in the American Civil War (1861-65), the first widely photographed conflict. Many thousands of these portraits survive, but only 10%-20% are identified. We created Photo Sleuth, a web-based platform that combines crowdsourced human expertise and automated face recognition to support Civil War portrait identification. Our mixed-methods evaluations of Photo Sleuth one month and 11 months after its public launch showed that it helped users successfully identify unknown portraits and provided a sustainable model for volunteer contribution. We also discuss implications for crowd-AI interaction and person identification pipelines.