Using Cloud Services to Improve Weather Forecasting Based on Weather Big Data Scraped From Web Sources

Abstract

Big Data (BD) scraping systems are among the recommended approaches for large-scale web data extraction. However, these systems for collecting large amounts of data face many challenges, including processing, storage, and data extraction reliability. Due to its potentials, cloud computing is becoming a viable solution to support BD scraping systems. This paper tenders a cloud based-web scraping framework for weather BD extraction and analysis. The aim is to extract weather BD from web sources, analyze this data and use it for visualization and forecasting purposes, and this by enabling elastic and on-demand resources. The framework is implemented using Selenium and Amazon Web Services and tested with Morocco weather data. The suggested cloud-based scrapper’s performance and scalability analysis reveals that it provides more efficiency in terms of data collecting and analysis, as well as forecast quality, due to its capacity to leverage cloud resources.