Identifying Content Blocks from Web Documents

  • Debnath S
  • Mitra P
  • Giles C
  • 15

    Readers

    Mendeley users who have this article in their library.
  • 34

    Citations

    Citations of this article.

Abstract

Intelligent information processing systems, such as digital libraries or search engines index web-pages according to their informative content. However, web-pages contain several non-informative contents, e.g., navigation sidebars, advertisements, copyright notices, etc. It is very important to separate the informative “primary content blocks” from these non-informative blocks. In this paper, two algorithms, FeatureExtractor and K-FeatureExtractor are proposed to identify the “primary content blocks” based on their features. None of these algorithms require any supervised learning, but still can identify the “primary content blocks” with high precision and recall. While operating on several thousand web-pages obtained from 15 different websites, our algorithms significantly outperform the Entropy-based algorithm proposed by Lin and Ho [14] in both precision and run-time.

Author-supplied keywords

  • data mining
  • electronic publishing
  • information systems

Get free article suggestions today

Mendeley saves you time finding and organizing research

Sign up here
Already have an account ?Sign in

Find this document

Authors

  • Sandip Debnath

  • Prasenjit Mitra

  • C Giles

Cite this document

Choose a citation style from the tabs below

Save time finding and organizing research with Mendeley

Sign up for free