Design of a high performance information filtering system

Abstract

A high performance information filtering system has three main requirements: it must be effective in supplying users with useful information, it must do so in a timely fashion, and it must be able to handle a large throughput of information and a large number of user profiles efficiently. These three requirements pose a difficult problem, and to our knowledge no existing system is capable of meeting all three. In this paper we describe a system which combines a number of techniques from other information retrieval and filtering systems, and is capable of providing high performance on a typical workstation platform. We provide estimates of computing resource usage, and show that our system is also scalable.