Professional Documents
Culture Documents
PDF 139
PDF 139
ABSTRACT
The appearance of skin colors in the images depends among other things, on the camera, the calibration of the camera, and the
illumination under which the image was taken. In this study, we investigate how the skin colors appear in the chromaticity
coordinates of different color spaces like HSV/HSL, normalized rgb, YES and TSL. For this purpose, we have taken images
of faces under 16 different illumination/camera calibration conditions using simulated illuminants (Horizon, A, fluorescent
TL84 and daylight) with different RGB cameras (1CCD web cameras and a 3CCD camera). In the making of this series of 16
images, first the selected camera was calibrated to one of the four light sources and an image was taken. After that the light
source was changed to the other light sources and at each time the person was imaged. The process was repeated to the other
two light sources. The same procedure was done for all four light sources and for each cameras. The skin regions were extract-
ed from these images and this skin data was then converted to different color spaces. We inspected how the chromaticities of
different skin color groups in these color spaces overlap in images taken in all 16 different cases and only in those cases in
which the selected camera was calibrated to the current illuminant. These investigations were also made between different
cameras. In addition to this, we examined the overlapping of all skin chromaticities from the different skin color groups be-
tween cameras.
Keywords: color spaces, skin color, skin locus, varying illumination, RGB camera
1. INTRODUCTION
Skin color has been used increasingly for detecting faces and hands in images and image sequences taken under varying im-
aging conditions because the color is a powerful and often computationally inexpensive cue. It is also robust towards changes
in orientation and scaling and can tolerate well occlusion. A disadvantage of the color cue is its sensitivity to illumination color
changes and, especially in the case of RGB, sensitivity to illumination intensity. One way to increase tolerance toward inten-
sity changes in images is to transform the image RGB into a color space whose intensity and chromaticity are separate and use
only chromaticity part for detection.
Such procedure has been followed in many papers; the range of skin color has been modelled as a probability distribution in
Tint-Saturation-Lightness (TSL), Normalized Color Coordinates (NCC), Hue-Saturation-Value (HSV), Luminance-Inphase-
Quadrature (YIQ - also called as NTSC color space) or YES eg. 1,2,3,4,5. In all cases, the illumination color change is assumed
to be mildeg. 1,2,3,4,5. Störring and coworkers considered a wider range of illumination color variation and defined the range
of possible skin colors in NCC they called “skin locus”6. We have demonstrated in several tracking experiments that the skin
locus can be used in robust skin detection as a chromaticity constraint to select pixels for adapting skin color histogram to
illumination changes7,8.
In this study, we map image RGB of skin areas to the chromaticity coordinates of 17 different color spaces to determine the
range of skin color. Facial images where captured under four calibration conditions from four light sources whose color tem-
peratures range from 2300K to 6500K. Under such conditions, skin color seen by a camera can change drastically from reddish
to bluish9. Color spaces are evaluated on the basis of: (1) how different skin color groups (Asian, Caucasian) overlap, when
while the camera is calibrated to one illuminant, the current illuminant color temperature changes, and (2) how big the area of
skin color histogram is compared to the size of the whole space. If the skin locus in a color space occupies a contiguous region
*Correspondence: Email: jbm@ee.oulu.fi; WWW: http://www.ee.oulu.fi/EE/Info.Laboratory/; Tel. 358 8 553 2809; Fax: 358 8 553 2612
and at the same time shows very good overlap within skin groups, then that color space is potentially useful for adaptive skin
color-based tracking.
2. METHODS
2.1. Data
Images of faces were captured and saved in RGB format (RAW BMP) under 16 different illumination/camera calibration con-
ditions using simulated illuminants (Horizon sunlight, Incandescent A, Fluorescent TL84 and Daylight D65) with different
cameras: 1CCD web cameras (Alaris, Nogatech and Winnov) and a 3CCD camera (Sony). In the creation of this 16-image
series9, the selected camera was first calibrated to one of the four light sources and an image was taken. While keeping the
calibration to the first illuminant, the current light source is changed and another image is taken. This is repeated for all illu-
minants. The camera is then calibrated to the next illuminant and the above steps are repeated until all illuminants have been
used to calibrate the camera. Examples of images may be seen at http://www.ee.oulu.fi/research/imag/color.
For Sony camera, ten Asians and Caucasians were imaged9 whereas for the other three cameras, images of only one person
for each of these groups were taken. Skin areas of these images were extracted manually by using defined polygons around
skin regions. The RGB data was transformed then to different color spaces.
The color spaces in this study may be divided into two groups: (1) those which can be obtained by a linear transform from
RGB, and (2) those which follow a nonlinear transform of RGB. The color spaces which are a linear transform from RGB are
I1I2I3 (Ohta’s features), YES, YIQ, YUV, YCrCb1 (Rec 601-5), and YCbCr2 (Rec 709). The nonlinear transforms are NCC
rgb or Normalized Color Coordinates rgb, modified rgb, ln-chromaticity (referred in this paper as log-chroma color space),
P1P2, l1l2l3, ratio between channels (G/R, B/R and B/G), HSV, HSL, modified ab, TSL and Yuv. Before transformations, the
RGB data was scaled between 0 and 1 except in the case of logchroma in which case the range was 0-255. Some transformation
needed special handle for certain RGB values: 1. pixels with 0 intensity - a special handling was needed for transformation to
NCC rgb and these pixels were removed before transformation to modified rgb, P1P2, uv and modified ab; 2. pixels with ach-
romatic color - transformations to HSV and HSL needed special handle for calculating output for achromatic colors and it was
necessary to remove these pixels before making transformation to l1l2l3; 3. pixels with 0 green channel - they were removed
before transformation to modified ab color space and 4. one or more channel have 0 value - for transformation to logchroma
space these pixels were removed. After the transformation, in most of the cases, the data was quantized approximately 1%
accuracy of the full range.
I1I2I3 or Ohta’s features10 (Eqs. 1-3) were first introduced for segmentation as optimized color features.
(R + G + B) (1)
I 1 = -----------------------------
3
I 2′ = ( R – B ) (2)
( 2G – R – B ) (3)
I 3′ = --------------------------------
2
YIQ space (Eqs. 7-9) which is derived from YUV can be used optionally by the NTSC composite TV standard.
Y = 0.30R + 0.59G + 0.11B (7)
I = 0.60R – 0.28G + 0.31B (8)
Q = 0.21 – 0.52G + 0.31B (9)
YUV space (Eqs.10-12) is the basic format for composite color television standard for NTSC, PAL and SECAM. uv space
(Eqs. 13-14) has been presented as luminance normalized UV color information.
Y = 0.299R + 0.587G + 0.114B (10)
U = – 0. 147R – 0.289G + 0.437B (11)
V = 0.615R – 0.515 G – 0.100B (12)
u = U⁄Y (13)
v = V ⁄Y (14)
General formulae for calculating YCbCr is shown in Eqs. 15-17 and specific coefficients for different standards are in Table 1.
Y = c R+c G+c B (15)
1 2 3
B–Y
Cb = ------------------ (16)
2 – 2c
3
R–Y (17)
Cr = ------------------
2 – 2c
1
Table 1: Coefficients for two YCbCr standards
The overlap computation was based on the bins which skin colors occupied. The overlap is percentage of the number of skin
color pixel common for both group divided by the amount of all bins with skin colors. The overlap was calculated for (1) all
4 calibrated cases (exception was Nogatech camera in which it was done only for 3 calibrated cases because under daylight
illumination the calibration was not successful) and (2) all 16 cases.
The skin locus in each color space can be described by one or more functions which define a region showing the range of
possible skin colors. However, the skin locus is only partially occupied at any one time with high probability when only one
person is considered. The skin locus is NOT a skin color model which changes during tracking. The skin locus can be used as
a chromaticity constraint in selecting pixels for training an adaptive skin color model.
Size of the color space was determined from the upper and lower boundary value of chromaticity coordinates of the color space
assuming uniformly distributed bins. It should be noted however that chromaticities of TSL space does not fill the whole area
and the modified ab space itself is nonuniform.
3. RESULTS
Alaris and Nogatech have a brightness (or gain) control whereas Sony and Winnov do not. Therefore it is possible due to the
lack of the brightness control that one channel have very low values. The results can be seen from Fig.1: very low values in
one channel increases the size of the locus. Therefore we must either use the whole locus in order to be able to track under
these illumination intensity and color changes or limit the tracking only certain illumination intensity range. Although here the
brightness control can help to track under wider range of illumination intensity change, it can also cause tracking failure if the
background is very bright (colors bleach).
0.8 0.8
0.6 0.6
g
g
0.4 0.4
0.2 0.2
0 0
0 0.5 1 0 0.5 1
r r
a) b)
Fig.1. Winnov locus in NCC rgb a) original data and b) only data in which each channel value is higher than 25/255.
The shape of locus for HSV and HSL are similar because the main difference for calculating their chromaticity coordinates
lies in how the saturation is calculated. A common mistake in displaying HSX distribution (where X=V, L, B or I) is to show
the distribution in cartesian coordinates. In fact, HS space must be viewed in polar coordinates, with H corresponding to angle
and S corresponding to the radius. Otherwise, skin locus in HS represented in cartesian coordinates will appear difficult to
model as shown in Figure 2.a whereas skin locus presented in the polar coordinates (Fig. 2.b) is easier to describe.
0.5 0.8
150 30
0.6
180 0
S
0.4
210 330
0.2
240 300
270
0
0 100 200 300
H H
a) b)
Fig.2 HSV in a) polar coordinates and b) cartesian coordinates.
3.3. Uniformity of chromaticity space
Chromaticity coordinates of HSV and HSL are not uniform in cartesian coordinate system. In TSL color spaces, there can be
nonuniformity present even in the cases where the camera was calibrated to the light sources (Fig. 3 a) but this is not so obvious
with all 16 cases.
Alaris − TS Alaris − TS
1 1
0.8 0.8
0.6 0.6
S
S
0.4 0.4
0.2 0.2
0 0
0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1
T T
a) b)
Fig. 3. TSL color spaces: a) only calibrated cases and b) all 16 different combinations.
The skin colors are presented in Fig. 4 in several different color spaces: YCbCr, l1l2l3, logchroma, ratio, P1P2, I1I2I3 (Ohta),
YES, modified rgb, NCC rb and gb, YIQ, YUV, Yuv, and modified ab. The skin colors are from all 16 cases for both Asian
and Caucasian persons as perceived by the Nogatech. For some color spaces, like log-chroma the input data range is basically
limited due to the definition of color space coordinates. Based on these images, we make a suggestion for functions which can
be used for modelling the skin locus (Table 3.).
4. a) 4. b) 4. c)
4. d) 4. e) 4. f)
4. g) 4. h) 4. i)
4. j) 4. k) 4. l)
4. m) 4. n) 4. o)
4. p) 4. q) 4. r)
Fig.4: a) locus of YCbCR. The two different versions of YCbCr produced similar locus.; b) locus of l1l2 chromaticity coordi-
nates which is also similar for the other combinations l1l2l3 chromaticity coordinates.; c) the locus of logchroma color space;
4d) locus of P1P2; 4e)-4g) loci of different chromaticity coordinates of ratio space; 4h) locus of Ohta’s I1I2I3; 4i)-k) loci of
different chromaticity coordinates of modified rgb; 4l) locus of NCC rb; 4m) locus of NCC gb; 4n) locus of YES; 4o) locus
of YIQ; 4p) locus of YUV; 4q) locus of Yuv; and 4r) locus of modified ab.
3.5. Overlap between the data of different skin color groups and size of skin locus in color space
The percentage of the overlap between skin colors extracted from images of Asian and Caucasian subjects taken with different
cameras is shown in Table 2. From this table, it can be seen that l1l2l3 give quite high results for almost every cases but on
the other hand the skin locus fulfils big part of the whole space (Table 3) in the Sony camera and therefore range modelling is
difficult.
In the Table 3. we investigate how much the different cameras overlap. The overlapness of all four cameras was not very big,
but when we make the comparison between cameras with and without brightness control, the overlapness increases. Next we
considered with what functions the skin chromaticity constraint can be defined; the results are also shown in Table 3.
Table 2: overlapping percentage between different skin group
4. Conclusions
The overlap in chromaticity of both Asian and Caucasian skin colors depends on the camera but it is relatively good with all
cameras considered in this study. The overlap can be increased by using brightness control or by rejecting pixels with low
channel values. As can be seen from skin locus images, none of these pixel wise color spaces offer invariance for illumination
color change. Therefore, the skin locus is useful tool for increasing tolerance for illumination color changes and non-uniform-
ity. For most of the color spaces, if the proper coordinate system is used in displaying a chromaticity space, the skin locus may
be modelled by one or two functions of up to quadratic order only.
5. ACKNOWLEDGEMENTS
This research was partially sponsored by Finnish Graduate school in Electronics, Telecommunications and Automation (GE-
TA)
6. REFERENCES
[1]. J.-C. Terrillon, M.N. Shirazi, H. Fukamachi and S. Akamatsu, "Comparative performance of different skin chrominance
models and chrominance spaces for the automatic detection of human faces in color images", Proc. of the fourth
international conference on automatic face and gesture recognition, IEEE Computer Society, Grenoble, France, pp.
54-61, 2000.
[2]. B.D. Zarit, B.J. Super, and F.K.H.Quek, "Comparison of five color models in skin pixel classification", Proc. of Int.
Workshop on Recognition, Analysis, and Tracking of Faces and Gestures in Real-Time Systems, IEEE Computer
Society, Corfu, Greece, pp. 58-63, 1999.
[3]. D. Comaniciu, and V. Ramesh, “Robust detection and tracking of human faces with an active camera”, Proc. Third IEEE
International Workshop on Visual Surveillance, IEEE Computer Society, Dublin, Ireland, pp. 11 -18, 2000.
[4]. J. Yang, and A. Waibel, “A real-time face tracker”, Proc. the Third IEEE workshop on Applications of Computer Vision,
IEEE Computer Society, pp.142-147, Florida, USA, 1996.
[5]. E. Saber, and A.M. Tekalp, “Face detection and facial feature extraction using color, shape and symmetry-based cost
function”, Proc. the 13th Int. Conf. Pattern recognition Vol. 3 Track C: Applications and robotic systems, 3, Vienna,
Austria, pp. 654-658, 1996.
[6]. M. Störring, H. Andersen, and E. Granum, “Skin color detection under changing lighting condition”, 7th Symposium of
Intelligent Robotics Systems, pp. 187-195, Coimbra, Portugal, 1999.
[7]. M. Soriano, B. Martinkauppi, S. Huovinen, and M. Laaksonen, “Using the skin locus to cope with changing illumination
conditions in color-based face tracking”. Proc. IEEE Nordic Signal Processing Symposium (NORSIG),Kolmården,
Sweden, pp. 383-386, 2000.
[8]. M. Soriano, B. Martinkauppi, S. Huovinen, and M. Laaksonen, “Skin detection in video under changing illumination
conditions”, Proc. 15th International Conference on Pattern Recognition, pp. 839-842, Barcelona, Spain, 2000.
[9]. E. Marszalec, B. Martinkauppi, M. Soriano, and M. Pietikäinen, "Physics-based face database for color research", J.
Electronic Imaging, 9(1), pp. 32-38, 2000.
[10]. Y.-I. Ohta, T. Kanade, and T. Sakai, “Color information for region segmentation”, Computer Graphics and Image
Processing, 13(3), pp. 222-241, July 1980.
[11]. G. Wyszecki, and W.S. Stiles, Color Science: Concepts and Methods, Quantitative Data and Formulae, 2nd edn, John
Wiley, New York, 1982.
[12]. S. Tominaga, “Illuminant estimation of natural scenes from color images”, Int. Conf. Color in Graphics and Image
processing, Cépaduès-éditions, Saint-Etienne, France, pp.35-40, 2000.
[13]. J. Berens, and G.D. Finlayson, “Log-opponent chromaticity coding of colour space”, 15th Int. Conf. Pattern Recognition
Vol. 1. Computer Vision and Image Analysis,1, pp.206-211, Barcelona, Spain, 2000.
[14]. C. Vertan, M. Ciuc, and N. Boujemaa, “On the introduction of a chrominance spectrum and its applications”, Int. Conf.
Color in Graphics and Image processing, Cépaduès-éditions, pp. 214-218, Saint-Etienne, France, 2000.
[15]. T. Gevers, and A.W.M. Smeulders, “Color-based object recognition”, Pattern Recognition, 32(3), pp. 453-464, 1999.
[16]. J.D. Foley, A. van Dam, S.K. Feiner, and J.F. Hughes, “Second edition in C computer graphics principles and practice”,
Addison-Wesley, New York, 1996.
[17]. S. Kawato, and J. Ohya, "Real-time detection of nodding and head-shaking by directly detecting and tracking the between
eyes", Proc. of the 4th international conference on automatic face and gesture recognition, pp. 40-45, Grenoble,
France, 2000.