Research of Benchmarking and Selection for TSDB

Abstract

With the increasing use of sensor and IoT technologies, sensor stream data is generated and consumed at an unprecedented scale. Traditional storage mechanisms represented by relational database systems become more and more difficult to adapt to the store, query, update and other operations of large-scale sensor stream data. This, in turn, has led to the emergence of a new kind of complementary non-relational data store subsumed under the term time series database (TSDB). However, the heterogeneity and diversity of numerous TSDBs impede the well-informed comparison and selection for a given application context. A thorough survey shows that current benchmarks for TSDBs are few and they still need improvement in workload implementation based on real business requirements, data generator based on real-world data and fine-grained performance metrics. How to implement a benchmarking tool for TSDBs according to different tradeoffs in IoT scenarios becomes a key challenge, which will be addressed in this paper. Firstly, we propose a benchmarking platform TS_Store_Test, which integrates five well-known TSDBs using the micro-services mechanism. Meanwhile, we integrated and extend Prometheus to capture the performance metrics in a refined manner. Based on TS_Store_Test, the execution efficiency of some workloads from technical and business perspectives is tested using the real hydrological sensor data. Experimental results demonstrate the usability and scalability of TS_Store_Test, and also show the performance differences of different TSDBs for sensor stream data. Finally, TS_Store_Test is compared with other NoSQL benchmarking suits.