A context aware notification architecture based on distributed focused crawling in the big data era

Abstract

The amount of data created in various sources over the Web is tremendously increasing. Trying to keep track of relevant sources is an increasingly time-consuming task. The traditional way of accessing information over the Web is pull-based. Users need to query data sources in certain time intervals where an important piece of information can be lately recognized or even missed completely. Technologies including RSS help users to get push-based notifications from websites. Discovering the relevant information without a notification overload is still not possible with existing technologies. Despite some promising efforts in push-based architectures to solve this problem, they fall short to meet the requirements in the big data era. In this study, by leveraging the latest advancements in distributed computing and big data analytics technologies, we use a focused crawling approach to propose a context aware notification architecture for people to find desired information at its most valuable state.