On the reprojection of 3D and 2D scenes without explicit model selection

Abstract

It is known that recovering projection matrices from planar configurations is ambiguous, thus, posing the problem of model selection - is the scene planar (2D) or non-planar (3D)? For a 2D scene one would recover a homography matrix, whereas for a 3D scene one would recover the fundamental matrix or trifocal tensor. The task of model selection is especially problematic when the scene is neither 2D nor 3D - for example a “thin” volume in space. In this paper we show that for certain tasks, such as reprojection, there is no need to select a model. The ambiguity that arises from a 2D scene is orthogonal to the reprojection process, thus if one desires to use multilinear matching constraints for transferring points along a sequence of views it is possible to do so under any situation of 2D, 3D or “thin” volumes.