FACE DETECTION IN COLOR IMAGES Rein-Lien Hsu��, Mohamed Abdel-Mottaleb*, and Anil K. Jain �� �� Dept. of Computer Science & Engineering, Michigan State University, MI 48824 * Philips Research, 345 Scarborough Rd., Briarcliff Manor, NY 10510 Email: {hsureinl, jain}@cse.msu.edu, mohamed.abdel-mottaleb@philips.com ABSTRACT Human face detection is often the first step in applications such as video surveillance, human computer interface, face recognition, and image database management. We propose a face detection algorithm for color images in the presence of varying lighting conditions as well as complex backgrounds. Our method detects skin regions over the entire image, and then generates face candidates based on the spatial arrangement of these skin patches. The algorithm constructs eye, mouth, and boundary maps for verifying each face candidate. Experimental results demonstrate successful detection over a wide variety of facial variations in color, position, scale, rotation, pose, and expression from several photo collections. 1. INTRODUCTION Various approaches to face detection are discussed in [10]. These approaches utilize techniques such as neural networks, machine learning, (deformable) template matching, Hough transform, motion extraction, and color analysis. The neural network-based [11] and view-based [14] approaches require a large number of face and non- face training examples, and are designed to find frontal faces in grayscale images. A recent view-based approach [12] extends the detection of frontal faces to profile views using a learning technique. Model-based approaches are widely used in tracking faces and often assume that the initial locations of faces are known. Skin color provides an important cue for face detection. However, the color- based approaches face difficulties in robust detection of skin colors in the presence of complex background and variations in lighting conditions. We propose a face detection algorithm which is able to handle a wide variety of variations in color images. 2. FACE DETECTION ALGORITHM The use of color information can simplify the task of face localization in complex environments [10, 3]. An overview of our face detection algorithm is depicted in Fig. 1, which contains two major modules: (i) face localization for finding face candidates and (ii) facial feature detection for verifying detected face candidates. Major modules of the algorithm are briefly described below. 2.1. Lighting compensation and skin tone detection The appearance of the skin-tone color can change due to different lighting conditions. We introduce a lighting compensation technique that uses ���reference white��� to normalize the color appearance. We regard pixels with top 5 percent of the luma (nonlinear gamma-corrected luminance) values as the reference white if the number of these reference-white pixels is larger than 100. The R, G, and B components of a color image are also adjusted in the same way as these reference-white pixels are scaled to the gray level of 255. Modeling skin color requires choosing an appropriate color space and a cluster associated with skin color in this space. Based on Terrillon et al.���s [15] comparison of the nine color spaces for face detection, we use the YCbCr space since it is widely used in video compression standards. Since the skin-tone color depends on luminance, we nonlinearly transform the YCbCr color space to make the skin cluster luma-independent. This also enables robust detection of dark and light skin tone colors. A parametric ellipse in the nonlinearly transformed Cb-Cr color subspace is used as a model of skin color. Figure 2 shows an example of skin detection. Figure 1: Face detection algorithm.