Approaching multimedia retrieval from a polyrepresentative perspective

Abstract

Multimedia documents such as videos, images, or music are characterized by an amount of different qualities that can become relevant during a search task. These qualities are seldom reflected as a whole by retrieval models. Thus, we present a new query model, which fully supports the principle of polyrepresentation by taking advantage of quantum logic. We offer means to model document relevance as a cognitive overlap from various features describing a multimedia document internally. Using our query model, the combination of the aforementioned polyrepresentative features is supported by the mechanisms of a Boolean algebra. In addition, these overlaps can be personalized by user preferences during a machine-based learning supported relevance feedback process. The input for the relevance feedback is based on qualitative judgments between documents, which are known from daily life, to keep the cognitive load on users low. We further discuss how our model contributes to the unification of different aspects of polyrepresentation into one sound theory. © 2012 Springer-Verlag.