Ant focused crawling algorithm

Abstract

This paper presents a new algorithm for hypertext graph crawling. Using an ant as an agent in a hypertext graph significantly limits amount of irrelevant hypertext documents which must be downloaded in order to download a given number of relevant documents. Moreover, during all time of the crawling, artificial ants do not need a queue to central control crawling process. The proposed algorithm, called the Focused Ant Crawling Algorithm, for hypertext graph crawling, is better than the Shark-Search crawling algorithm and the algorithm with best-first search strategy utilizing a queue for the central control of the crawling process. © 2008 Springer-Verlag Berlin Heidelberg.