Web contents extracting for web-based learning

Abstract

Web mining has been applied to improve web-based learning. Content-based Web mining usually focuses on main contents of web page. This paper proposes a novel approach to automatically extract main contents from web pages. Compared with existed studies, the method may determine whether a web page contains main contents, and then extracts main contents without using DOM-Tree and template. Main contributions include: (1) Introducing a new concept of Block and proposing a method to partition web page to blocks. Main contents and noise contents may be well partitioned into different blocks. (2) Introducing a concept of Web Page Block Distribution and studying its feature. Based on Block Distribution, we may effectively determine whether the web page contain main contents, and then extract main contents via outlier analysis. Experiments demonstrate utility and feasibility of the method. © 2008 Springer-Verlag Berlin Heidelberg.