Pervasive sensor systems offer unbounded possibilities for monitoring and tracking objects, machines, and spaces. To maximize the benefit from a sensor system, sensor data requires efficient preprocessing and analysis. Big data techniques make distributed processing of huge amounts of data fast and cost-effective, making them a practical necessity for sensor data. However, the realtime requirements and the sheer velocity and volume of data from large sensor systems require a dedicated approach to designing the data processing pipeline. This paper discusses viewpoints and requirements for designing a sensor data pipeline, with specific focus on data input, live preprocessing, and storage.
Ronkainen, J., & Iivari, A. (2015). Designing a data management pipeline for pervasive sensor communication systems. In Procedia Computer Science (Vol. 56, pp. 183–188). Elsevier B.V. https://doi.org/10.1016/j.procs.2015.07.193