3D reconstruction from tangent-of-sight measurements of a moving object seen from a moving camera

Abstract

Consider the situation of a monocular image sequence with known ego-motion observing a 3D point moving simultaneously but along a path of up to second order, i.e. it can trace a line in 3D or a conic shaped path. We wish to reconstruct the 3D path from the projection of the tangent to the path at each time instance. This problem is analogue to the “trajectory triangulation" of lines and conic sections recently introduced in [1,3], but instead of observing a point projection we observe a tangent projection and thus obtain a far simpler solution to the problem. We show that the 3D path can be solved in a natural manner, and linearly, using degenerate quadric envelopes - specifically the disk quadric. Our approach works seamlessly with both linear and second order paths, thus there is no need to know in advance the shape of the path as with the previous approaches for which lines and conics were treated as distinct. Our approach is linear in both straight line and conic paths, unlike the non-linear solution associated with point trajectory [3]. We provide experiments that show that our method behaves extremely well on a wide variety of scenarios, including those with multiple moving objects along lines and conic shaped paths.