WebSifter: An ontological web-mining agent for e-business

Abstract

The World Wide Web provides access to a great deal of information on a vast array of subjects. A user can begin a search for information by selecting a Web page and following the embedded links from page to page looking for clues to the desired information. An alternative method is to use one of the Web-based search engines to select the Web pages that refer to the general subject of the information desired. In either case, a vast amount of information is retrieved. The quantity can be overwhelming, and much of the information may be marginally relevant or completely irrelevant to the user's request. We present a methodology, architecture, and proof-of-concept prototype for query construction and results analysis that provides the user with a ranking of choices based on the user's determination of importance. The user initially designs the query with assistance from the user's profile, a thesaurus, and previously constructed queries acting as a taxonomy of the information requirements. After the query has returned its results, decision analytic methods and information source reliability information are used in conjunction with the expanded taxonomy to rank the solution candidates. © 2003 by International Federation for Information Processing.