Adaptive partitioning using partial replication for sensor data

Abstract

There is a huge increase in IoT network size and applications. It has increased the amount of the IoT data that needs to be handled by the applications. State-of-the art workload based static partitioning methods scale poorly and often result in poor execution times as not all the queries are favoured by initial partition created. This work proposes an adaptive partitioning method that adapts the system to workload changes by reproducing the most frequent pattern among nodes. The scheme also adapts when new triples or properties are added into a system by ensuring proper placement of new triples in an appropriate partition by leveraging subject-object joins. The performance of this adaptive partitioning method is evaluated against the existing static partitioning scheme. The performance of the system for different query types such as linear, star, administrative and snowflakes are analysed. The experimental results verify that the adaptive partitioning method is scalable, adjusts to categories of dynamism and results in faster query execution by minimizing inter-node communication. Although Algorithm Execution Time (AET) for adaptive partitioning is greater than static partitioning, Query Execution Time (QET) increases at much faster rate for static partitioning for scaled data. Adaptive partitioning accelerates queries by 60% compared to static partitioning when averaged over types of queries.