Focused crawling using fictitious play

Abstract

A new probabilistic approach for focused crawling in hierarchically ordered information repositories is presented in this paper. The model is suitable for searching the World Wide Web and it is based on the fictitious play model from the theory of learning in games. The leading idea of the play is that players (software agents) are competing of the resources so that the search focuses on areas where relevant information can be found more likely. The model is basically a coordination model of the agent population but it is also possible to plug different features into the model, e.g. features for the user’s relevance feedback or semantic links between documents. Additionally, the method is highly scalable and the efficient parallel implementation of the method is possible.