An Ingestion and Analytics Architecture for IoT Applied to Smart City Use Cases

Abstract

As sensors are adopted in almost all fields of life, the Internet of Things (IoT) is triggering a massive influx of data. We need efficient and scalable methods to process this data to gain valuable insight and take timely action. Existing approaches which support both batch processing (suitable for analysis of large historical data sets) and event processing (suitable for real-time analysis) are complex. We propose the hut architecture, a simple but scalable architecture for ingesting and analyzing IoT data, which uses historical data analysis to provide context for real-time analysis. We implement our architecture using open source components optimized for Big Data applications and extend them, where needed. We demonstrate our solution on two real-world smart city use cases in transportation and energy management.