A comparison of two persistent storage tools for implementing a search engine

Abstract

This paper compares two Java-based persistent storage mechanisms: a commercially available object-oriented database (OODB) system1 and the persistent storage tool PJama [1] on the basis of their suitability for implementing the index component of an experimental World Wide Web search engine called WWLib-TNG. Persistence is provided to the builder component of the search engine, which constructs a catalogue of Web pages. The searcher component of the engine searches the catalogue. The implementation of the builder using PJama and using an OODB were compared with respect to time taken to construct a catalogue, efficient use of disk space and scalability. The implementations of the searcher were compared on response time and scalability. The comparison showed that for this application PJama performs better than the OODB. Compared with the OODB, PJama gave 300% better build performance and was more scalable.