Open architecture for distributed search systems⋆

Abstract

This paper describes an open architecture for distributed Internet search engines and the experience derived from implementation of a conforming prototype. The architecture enables competing collaboration of independent information retrieval service providers. It allows integration of multiple cheap private servers into a powerful distributed system, which still guards independence and commercial interests of every player. Special emphasis was made on demonstrating the ability of the architecture to make effective use of latest advances in information retrieval technology. Prototype implementation has proved the feasibility of the approach. It has also exposed a wide area of optimisations desirable at the component level. The source code of the prototype is publicly available.