Application-Based Benchmarking on Redis and MongoDB for Trip Planning using GTFS Data

Abstract

Benchmarking serves as the foundation for selecting a database in any project. The available benchmarking tools evaluate system performance by subjecting it to random data and a set of arbitrary operations, without considering the specific characteristics of the application. The problem with these tools is that they reflect unrealistic benchmarks as they do not consider the nature, sequence, and type of queries the application will send to the database. In this paper, we introduced the approach of benchmarking the database based on the nature of interaction and queries between the application and database, and we built a benchmarking tool using Java to benchmark Redis and MongoDB as databases for a trip planning application with GTFS data of Budapest local transport data. Our study involved comparing the performance of both databases under ten different stress levels by simulating the number of querying clients. The results show that both database's performance is slightly decreased while increasing the number of clients (stress). However, Redis shows better performance compared to MongoDB.