Users' perception of the performance of a filtering system

Abstract

Although filtering electronic information is spreading rapidly, very few studies examined users' perceptions about the success of filtering. Users at the Boeing Company participated in a study which collected data through observation, verbal protocols, questionnaire, and interviews. Data analysis used four levels of relevance to assess the importance, and frequency of use, of thirteen criteria for relevance, and fourteen for non relevance, that are not topics or subject matters. Results showed that perceived precision ratios for filtered information were higher then the ratios for non-filtered information, but not significantly and could still be improved even though most respondents were satisfied with these ratios. Developing methods to create and maintain useful profiles, and finding ways to incorporate relevance as well as non-relevance criteria into profiles, are necessary to improve the performance of filtering mechanisms. Copyright 1997 ACM.