Towards industrial assistance systems: Experiences of applying multi-sensor fusion in harsh environments

Abstract

With the availability of miniaturized low cost sensors and the general availability and easy applicability of algorithms for activity recognition, we investigate how various sensors can be deployed in a harsh environment, the industrial shop-floor. We review related work and provide an in-depth review of our own experiences were sensors wer used to enable recognition of activity, task progress and also mental and cognitive states of assembly workers. The recognition process is based on stationary (RGBD cameras, stereo vision depth sensors) and wearable devices (IMUs, GSR, ECG, mobile eye tracker). We describe in detail the used sensors, the challenges of fusing the data from these various sources together in real-time and how to interpret that data semantically.