A new training strategy for spatial transform networks (STN’s)

Abstract

Spatial transform networks (STN) are widely used since they can transform images captured from different viewpoints to obtain an objective image. These networks use an image captured from any viewpoint as input and the desired image as a label. Usually, these images are segmented, but this could lead to convergence problems if the percentage of overlap between the segmented images is quite low. In this paper, we propose a new training method to facilitate the convergence of a STN in these cases, even when there is no overlap between the object’s projections in the two images. This new strategy is based on the incorporation of the distance transformation images to the training, thus increasing the useful image information to provide gradients in the loss function. This new training strategy has been applied to a real case, with images of Caenorhabditis elegans, and to a simulated case, which uses artificial images to ensure that there is no overlap between the images used for the assays. In the assays carried out with these datasets, we have shown that the training convergence is strengthened, reaching a precision level for IoU metric of 0.862 and 0.984, respectively, and the computational cost has been maintained compared to the assay with segmented images, for the real case.