Datalyzer: Streaming data applications made easy

Abstract

Nowadays, streaming data are continuously generated from thousands of sources, including social networks, mobile apps, sensors, e-commerce transactions, and many more. Hence, it becomes very useful to build applications able to process these data, with the purpose of filtering interesting parts, monitor their run-time evolution, persist valuable chunks, trigger events upon certain conditions are met and provide analytics. While several frameworks and systems have emerged to create this kind of applications, these systems tend to be low-level, based on complicated APIs, challenging to install and configure for end-users, and requiring from high performant hardware for their execution. Our goal is to lower the entry level to develop, deploy and run streaming applications. To accomplish this goal, we propose Datalyzer, an approach to create streaming data applications on the cloud based on a visual language. This way, Datalyzer provides a facility to describe streaming data sources in an open way, and a visual language to describe the execution flow of the streaming application. Datalyzer is based on model-based development principles, where code is generated automatically, and then compiled, deployed and executed on the cloud. As a proof of concept, we describe a case study in enterprise systems, and how it can be built using our prototype tool.