Design and evaluation of a system to support collaborative search

Abstract

We describe a collaborative search system called Results Space to support small groups of users in conducting asynchronous collaborative searches. We discuss the design of the system and present results from a laboratory evaluation. We also describe the development of an asynchronous collaborative task scenario (based on a task from the TREC Robust test collection) designed to elicit naturalistic behaviors with the system. Our results show that participants used the collaborative features not just to avoid duplication of effort, but also to check and refine collaborators' work, to gain a general understanding of collaborator's actions, and to get ideas for new queries. Although we expected participants to use the collaborative awareness mechanisms to find and rate new items, we found that participants were more likely to make ratings for result items that had been previously rated by their collaborators. Considered together, these results suggest a range of tactics and behaviors that collaborative search tools should support.