Consolidating user relevance criteria: A meta‐ethnography of empirical studies

Abstract

The purpose of this study is to bring together empirical studies on user‐based relevance criteria and to provide a conceptual framework for the organization of factors affecting relevance judgments and information selection. Through systematic analysis applying a meta‐ethnographic approach we cluster compatible relevance criteria definitions and resolve semantic differences of the same named criteria with diverse definitions. The consolidated criteria are factored into four dimensions: situational factors pertaining to user's context, task, and cognitive and affective states, user criteria pertaining to users' perceptions and needs, document information elements (DIEs) pertaining to information attributes, and value judgments pertaining to what an information object means to the user). For further investigation, we propose a set of matrices to depict the relationships between dimensions as an effective tool for developing a better understanding of the relationship and interaction between users' relevance factors. These relationships can guide IR system design and improvements.