Capturing the semantics of user interaction: A review and case study

Abstract

In many retrieval domains there exists a problematic gap between what computers can describe and what humans are capable of perceiving. This gap is most evident in the indexing of multimedia data such as images, video and sound where the low-level features are too semantically deficient to be of use from a typical users’ perspective. On the other hand, users possess the ability to quickly examine and summarise these documents, even subconsciously. Examples include specifying relevance between a query and results, rating preferences in film databases, purchasing items from online retailers, and even browsing web sites. Data from these interactions, captured and stored in log files, can be interpreted to have semantic meaning, which proves indispensable when used in a collaborative setting where users share similar preferences or goals. In this chapter we summarise techniques for efficiently exploiting user interaction in its many forms for the generation and augmentation of semantic data in large databases. This user interaction can be applied to improve performance in recommender and information retrieval systems. A case study is presented which applies a popular technique, latent semantic analysis, to improve retrieval on an image database.