Robust Image Stitching and Reconstruction of Rolling Stocks Using a Novel Kalman Filter with a Multiple-Hypothesis Measurement Model

Abstract

This work introduces a novel algorithm for the reconstruction of rolling stocks from a sequence of images. The research aims at producing an accurate and wide image model that can be used as a Digital Twin (DT) for diagnosis, fault prediction, maintenance, and other monitoring operations. When observing large surfaces with nearly constant textures, metallic reflections, and repetitive patterns, motion estimation algorithms based on whole image error minimization and feature pairing with Random Sampling and Consensus (RANSAC) or Least Median of Squares (LMedS) fail to provide appropriate associations. To overcome such an issue, we propose a custom Kalman Filter (KF) modified by adding multiple input-noise sources represented as a Gaussian mixture distribution (GM), and specific algorithms to select appropriate data and variance to use for state prediction and correction. The proposed algorithm has been tested on images of train vessels, having a high number of windows, and large metallic paintings with constant or repetitive patterns. The approach here presented showed to be robust in the presence of high environmental disturbances and a reduced number of features. A large set of rolling stocks has been collected during a six months campaign. The set was employed to demonstrate the validity of the proposed algorithm by comparing the reconstructed twin versus known data. The system showed an overall accuracy in length estimation above 99%.