Interactive spatiotemporal reasoning

Abstract

Spatiotemporal reasoning involves pattern recognition in space and time. It is a complex process that has been dominated by manual analytics. In this chapter, we explore the new method that combines computer vision, multi-physics simulation and human-computer interaction. The objective is to bridge the gap among the three with visual transformation algorithms for mapping the data from an abstract space to an intuitive one, which includes shape correlation, periodicity, cellular shape dynamics, and spatial Bayesian machine learning. We tested this approach with the case studies of tracking and predicting oceanographic objects. In testing with 2,384 satellite image samples from SeaWiFS, we found that the interactive visualization increases robustness in object tracking and positive detection accuracy in object prediction. We also found that the interactive method enables the user to process the image data at less than 1 min per image versus 30 min per image manually. As a result, our test system can handle at least ten times more data sets than traditional manual analysis. The results also suggest that minimal human interactions with appropriate computational transformations or cues may significantly increase the overall productivity.