Deep questions in the “Deep or Hidden”web

Abstract

The Hidden Web is a part of the Web that consists mainly of the information inside databases, i.e., anything behind an interactive electronic form (search interfaces), which cannot be accessed by the conventional Web crawlers [1, 2, 8]. However, there have been well-defined, effective, and efficient methods for accessing Deep Web contents. One of these methods for accessing the Hidden Web employs an approach similar to ‘traditional’ crawling but aims at extracting the data behind the search interfaces or forms residing in databases. The paper brings insight into the various steps, a crawler must perform to access the contents in the Hidden Web. We structure the problem area and analyze what aspects have already been covered by previous research and what needs to be done.