Accelerating the Internet in the presence of Big Data: Reducing user delays by leveraging historical user request patterns for web caching

Abstract

Approximately 4 billion people have access to the Internet, additionally 23 billion devices are connected as of 2018. This has allowed for a substantial growth in data collection which has allowed for Big Data to flourish. The continued increase in user, devices, and Big Data usage has created a significant intensification in Internet traffic. This in turn has the potential to increase user delays when accessing data on the Internet. There are a number of ways to help reduce user latency, web caching is able to reduce web user delays in addition to reducing network traffic and the load on web servers. In this study we propose a proxy level web caching mechanism leveraging historical web patterns to help reduce user latency and accelerate the Internet. In addition we survey the state of the art of other caching approaches. Our investigation shows that using historical patterns as part of a proxy caching mechanisms in large scale networks can significantly shorten the latency for users in this era of Big Data