This paper deals with the problem of constructing an intelligent Focused Crawler, i.e. a system that is able to retrieve documents of a specific topic from the Web. The crawler must contain a component which assigns visiting priorities to the links, by estimating the probability of leading to a relevant page in the future. Reinforcement Learning was chosen as a method that fits this task nicely, as it provides a method for rewarding intermediate states to the goal. Initial results show that a crawler trained with Reinforcement Learning is able to retrieve relevant documents after a small number of steps.
CITATION STYLE
Grigoriadis, A., & Paliouras, G. (2004). Focused crawling using temporal difference-learning. In Lecture Notes in Artificial Intelligence (Subseries of Lecture Notes in Computer Science) (Vol. 3025, pp. 142–153). Springer Verlag. https://doi.org/10.1007/978-3-540-24674-9_16
Mendeley helps you to discover research relevant for your work.