Enhancing a sequence of facial images by combining multiple undersampled and compressed images

Abstract

The enhancement of a sequence of low-resolution images of a human face into a single higher resolution composite poses a real problem in the fields of forensics and security. This is due to the substantial variations in appearance that faces undergo with changing illumination, orientation, scale, facial expressions and occlusions. In addition, faces often appear small in surveillance imagery because of the wide fields of view that are typically used and the relatively large distance between the cameras and the scene. A device-independent algorithm for the estimation of an enhanced resolution image from multiple low-resolution and non-coplanar compressed images having arbitrary motion is proposed in this paper. The basic algorithm has previously been described but its application to enhancing images of objects, in this case a human face, with a range of scales and orientations is the extension detailed here. The task of obtaining a super-resolved image from a dynamic, undersampled and degraded image sequence can take advantage of the additional spatio-temporal data available in the image sequence. In particular, camera and scene motion lead to frames containing similar, but not identical information. While the images may look similar to the eye, the grey-scale values of the individual pixels are sufficiently different to allow reconstruction of a super-resolved image with wider bandwidth than that of any of the individual low-resolution frames. The arbitrarily distorted images must bewarped, firstly, to a common orientation so that a rigorous least squares area-based matching technique can then compute the registration parameters needed for their accurate combination. The warping is an iterative process relying on manual intervention. Once all the lowresolution images are brought into registration and comply with pre-established image correlation criteria, they are combined using the algorithm to recover the desired high resolution composite. In addition to improving the spatial resolution of the undersampled sequence, the method may also attenuate compression artefacts. © 2006 The Authors. Journal Compilation © The Remote Sensing and Photogrammetry Society and Blackwell Pubilishing Ltd.