Newtonian Physics Informed Neural Network (NwPiNN) for Spatio-Temporal Forecast of Visual Data

Abstract

Machine intelligence is at great height these days and has been evident with its effective provenance in almost all domains of science and technology. This work will focus on one handy and profound application of machine intelligence-time series forecast, and that too on visual data points, i.e., our objective is to predict future visual data points, given a subtle lag to work on. For the same, we would propose a deep learner, Newtonian physics informed neural network (NwPiNN) with the critical modelling capabilities of the physics informed neural networks, modelled on the laws of Newtonian physics. For computational efficacy, we would work on the gray-scale values of pixels. Since the variation in data pixel values is not only provoked by the pixel gray values but also by the velocity component of each pixel, the final prediction of the model would be a weighted average of the gray value forecast and the kinematics of each pixel, as modelled by the PINN. Besides its’ proposal, NwPiNN is subjected to benchmark visual dataset, and compared with existing models for visual time series forecast, like ConvLSTM, and CNN-LSTM, and in most of the occasions, NwPiNN is found to outperform its preliminaries.