A neural network-based algorithm for 3D multispectral scanning applied to multimedia

Abstract

We describe a new stereoscopic system based on a multi-spectral camera and an LCD-Projector. The novel concept we want to show consists in the use of multispectral information for 3D-scenes reconstruction. Each 3D point is linked to a curve representing the spectral reflectance. This latter is a physical representation of the matter and presents the advantage over color information, which is perceptual, that it is independent from both illuminant and observer. We first present an easy methodology to geometrically and spectrally calibrate such a system. We then describe an algorithm for recovering 3D coordinates based on triangulation and an algorithm for reflectance curves reconstruction based on neural networks. The results are encouraging, they confirm the feasibility of such a system and in the same time enable some multimedia applications like simulating illumination change. © Springer-Verlag Berlin Heidelberg 2005.