MATHEMATICAL MODEL OF SKIN COLOUR

FOR FACE DETECTION

Setiawan Hadia , Adang Suwandib , Iping Suprianab , Farid Wazdib

a
Universitas Padjadjaran, Bandung, Indonesia
b
Institut Teknologi Bandung, Indonesia

Abstract. Skin colour in digital image is important element that is very useful
for preprocessing task in skin-based automatic image detection. In this paper, skin
colour was investigated and its representation in several colour spaces has been
explored. A mathematical model for the representation has been developed and
successfully implemented for automatic single frontal face detection conjuncted
with mathematics morphological and segmentation image processing filters and
simple 4-neigbourhood image measurement algorithm. The experiment has been
conducted based from our proposed method using face database images that have
been collected from several sources, such as standard FERET Colour face database
and local native face database.

Key-words: Skin, color, face, detection

1 Introduction
Skin colour is an important element in detecting image that contain skin or skin-
like region. Skin colour can be used to detect faces [5, 8, 9] or hands [13, 28], in
dynamic images as well as in still images. Skin colour has also been used to detect
images of naked people for Internet content altering [3, 6, 7]. In the field of health
and disease, skin colour can also be used to analyse medical images. For example,
the ability to segment an image using skin colour can aid in the diagnosis of skin
cancer [17].

Skin becomes an interesting feature in the field of image detection especially in

face detection due to (1) it covers most of the face image area, (2) skin of dif-
ferent people appears to vary over a wide range, however the differ is much less
in colour (chromaticity) than brightness [18]. In the computational perspective,
detection of skin area in digital image are more practical and easy to implement.
In addition, skin color chromaticity distribution from different ethnic groups lie in
the same Gaussian distribution [12]. In our research, skin model is investigated
in three colour spaces, rg space, HSB space and YCbCr space, and represented in
histogram and skin colour distribution.

Skin colour detection will give result of skin region and skin-like region. False
detection of skin-like colour region can degrade the performance of skin colour
detection. To cover this problem, skin colour detection must be followed with
 Setiawan Hadi, Adang Suwandi A., Iping Supriana S., Farid Wazdi

other detection techniques. In our approach, skin color detection in three colour
spaces was conducted and compared using skin detector that has been created
semi-manually based on the statistical skin model that has been developed from
the previous step. The result of skin color detection is followed with two morpho-
logical filters, those are erosion, for thinning the image and removing the noise
with salt and pepper technique, and dilation, for compacting the binary image
representation.

In the following pages, we will describe skin color distribution in the explored color
spaces. Description about colour in digital images can be read in [4], meanwhile
elaboration of colour spaces and their comparison can be found in detail in [27].

2 Skin Colour Distribution

A color histogram is a distribution of colors in the color space and has long been
used by the computer vision community in image understanding. For example,
analysis of color histogram has been a key tool in applying physics-based models
to computer vision. It has been shown that color histograms are stable object
representations unaffected by occlusion and changes in view, and that they can
be used to differentiate among a large number of objects. In the mid-1980s, it
was recognized that the color histogram for a single in homogeneous surface with
highlights will have a planar distribution in color space. It has since been shown
that the colors do not fall randomly in a plane, but form clusters at specific points.
It has been further observed that (1) human skin colors cluster in a small region in
a color space; (2) human skin colors differ more in intensity than in colors, and (3)
under a certain lighting condition, a skin- color distribution can be characterized
by a multivariate normal distribution in the normalized color space [25]. The
figure 1 shows a face image and the skin color distribution in the RGB, HSI and
YCbCr colour spaces.

3 Mathematical Model of Skin Colour

Literature research [6, 20, 12] shown that there are three approaches of skin colour
modelling, those are empirical approach, statistical approach and adaptive ap-
proach. A simple empirical approach to colour modelling in general is to use
a representative sample of pixels for some target colour to generate a histogram
in the selected colour space. A threshold is then selected. Any pixel belonging
to a histogram region above that threshold is then classified as being the target
colour. The main disadvantages of the histogram approach, when compared to
the statistical approach, are that the resulting model is not as compact, and clas-
sifying pixels is more computationally demanding [13]. In addition, the size of the
sampling bins used to create the histogram affects the performance of the model.
The main advantages of the histogram approach are that it is relatively simple
to implement, can be applied to non-Gaussian distributions, and does not require
detailed knowledge of the skin colour distribution. In the statistical models,
 MATHEMATICAL MODEL OF SKIN COLOUR FOR FACE DETECTION

Figure 1: Sample image and its skin colour distribution in colour spaces

the normalised rg colour space and various hue and saturation based colour spaces
result in a skin colour distribution that, is well modelled with a bivariate Gaussian
distribution [19, 25, 23]. There is contradiction about the best statistical model to
use for skin colour. Most researchers use a single distribution, while some [24, 26]
claim that a single multivariate distribution is inadequate, and a Gaussian mix-
ture model is more appropriate. Other researchers use a Gaussian mixture model,
yet observe that a single Gaussian distribution is often adequate [14, 15]. Some
researchers have used adaptive skin colour models to overcome the limitations
between fixed and statistical colour models. Jordao et al. [8] used the variance of
the skin colour distribution in the HSV colour space to detect regions in an image
as being skin coloured based on the variance of colour within the region. Yang and
Waibel [22] used a statistical model in the normalised rg colour space, and allowed
the model parameters to adapt from one frame to the next in a video sequence.
Raja, McKenna, and Gong [15] used an adaptive Gaussian mixture model that
modified the model parameters from frame to frame by resampling a window of
the image, while ignoring frames that were too different to the current model.
Similarly, Rowley [16] used a Gaussian model in the normalised rg colour space
that resampled areas around the centre of the face to adapt the model parameters
from frame to frame.

In our approach, skin color is modelled using mean and covariance of chrominant
color. If we use normalize rg space, then the value of r and g are calculated.
If HSB, h element and s element are considered. Similar with that, Cb and Cr
components are used for modelling skin colour in YCbCr colour space.
 Setiawan Hadi, Adang Suwandi A., Iping Supriana S., Farid Wazdi

3.1 Mean chromaticity calculation

In the HSB colour space, hue ranges from [0 . . . 1), and wraps around so that 0
≡ 1. The mean hue is therefore not calculated in the usual way. First, the hue
is mapped onto the range [0 . . . 2π) radians, and then treated as a point on the
unit circle in polar coordinates. Hence, it is represented by the polar coordinates
(θ, 1), where θ = 2πh, and h is the hue. This point is then converted into Cartesian
coordinates, so that for every hue, there is a Cartesian point (x, y) where x = cos(θ)
and y = sin(θ). The mean x and y values for the sample of n pixels are then
calculated in the usual way:
n n
1
1

x= xi y= yi . (1)
n i=1 n i=1

The mean (x, y) location is then used to derive an angle in polar coordinates which
is then mapped onto the range [0..2π), by adding or subtracting multiples of 2π as
necessary, and then mapped onto the range [0, 1) by dividing by 2π. Saturation
ranges from [0..1], and does not wrap around, so the mean saturation is calculated
in the usual way. The calculation of the angle θ, mean hue ans saturation are
n
y θ 1

θ = arctan( ) h= s= si (2)
x 2π n i=1

For calculations in the normalised rg colour space, the chromaticity is defined as

(r, g) instead of (h, s), and the mean chromaticity is calculated in the usual way:
n n
1
1

r= ri g= gi (3)
n i=1 n i=1

3.2 Covariance chromaticity calculation

After calculating the mean chromaticity, the covariance matrix is calculated, with
the additional constraint that the distance from any hue to the mean hue cannot
exceed 0.5. This takes into account the fact that hue wraps around and conse-
quently the maximum distance between any hue and the mean hue is 0.5. For a
sample containing n pixels, the covariance matrix for chromaticity is
 
Chh Chs
C= (4)
Chs Css
where
n n n
1
1
1

Chh = ∆hi ∆hi Chs = ∆hi ∆si Css = ∆si ∆si (5)
n i=1 n i=1 n i=1

and the distances to the mean hue and saturation is defined as

⎧
⎨ hi − h iff |(hi − h)| < 0.5,
∆hi = hi − h + 1 iff (hi − h) ≤ −0.5, (6)
⎩
hi − h − 1 iff (hi − h) ≥ 0.5
 MATHEMATICAL MODEL OF SKIN COLOUR FOR FACE DETECTION

and
∆si = si − s. (7)
Chromaticity is defined as (r, g) for the normalised rg colour space instead of (h, s).
The covariance matrix is calculated in the same way as for the HSB colour space
by substituting r for h, g for s, r for h, and g for s. The absolute value of h is not
constrained when using the normalised rg colour space because this space does not
wrap around.

3.3 Measuring Distance from the Population

The Mahalanobis distance metric D can be used to measure the distance between
a point and a population of points in a way that is sensitive to changes in the
variance along the principal axes of the distribution for the population. It is
defined [10] as 
D(
p) = (  T C −1 (
p − m) p − m)
 (8)

where p = (h, s) is the chromaticity of the pixel being classified, m

 = (h, s) is
the mean chromaticity for the population and C −1 the inverse of the covariance
matrix of chromaticity for the population.
As stated earlier, the covariance matrix is
 
Chh Chs
C= (9)
Chs Css

If (Chh Css − Chs Chs ) = 0 (which in practise is usually the case), the covariance
matrix is invertible, and the invers [2] is
 
−1 1 Css −Chs
C = (10)
Chh Css − Chs Chs −Chs Chh

This calculation is the same for the rg colour space using the appropriate chro-
maticity values and covariance matrix.

4 Face Detection
Face detection can be considered as part of face recognition implementation. Lit-
erature shows many researches have been conducted in face detection area that
have been reported in survey papers [11, 21]. Skin colour has been used in research
as a preprocessing step in face detection. Many methods have been developed and
reported in the literature that use skin colour to detect skin and skin-like region
in digital image.

Our approach in skin detection has been implemented in three colour spaces,
normalized rg, HSB and YCbCr colour spaces. Figure 2 shows the schematic
approach of our proposed face detection based on the skin colour representation.
 Setiawan Hadi, Adang Suwandi A., Iping Supriana S., Farid Wazdi

Training Images

Skin model
generation Input Image

Skin colour
Detection (Image
distribution in
Matching)
chrominant space

Detection result

Figure 2: Face Detection Scheme

4.1 Generating skin colour model

To generate skin colour model, mean chromaticity of each training image was
calculated for every colour space. Combination of all mean chromaticities for
every colour space was calculated using formula 11 and assumed as a general skin
model.

n
1
Mk = Ti (11)
δ
i=1 i

1 1 1 1
Mk = T1 + T2 + · · · + Tn−1 + Tn (12)
δ1 δ2 δn−1 δn
Mk is skin model-k, meanwhile Ti is tripixel RGBof skin image-i (training image).
δi is parameter is interval between 1 and n. If δi = n the process is averaging.
The amout of training image is presented by n.

Images that will be used as training images are segmented and cropped semi-
manually. Figure 3 and figure 4 illustrated sample images before segmentation
and the result of segmentation. Figure 5 show combined images of segmented
images and their statistical measurement characteristics (mean chromaticities) and
visualization in histogram.
Figure 6 shows distribution of skin color in an image for every colour space. It is
clearly seen that skin cluster is located in a small area or region (the white small
region) of the whole skin distribution (the black large region).
MATHEMATICAL MODEL OF SKIN COLOUR FOR FACE DETECTION

Figure 3: Sample images BEFORE segmentation

Figure 4: Sample images AFTER segmentation

Figure 5: Mean Chromaticities

Figure 6: Skin Colour Distribution in Color Spaces

 Setiawan Hadi, Adang Suwandi A., Iping Supriana S., Farid Wazdi

4.2 Improved skin-based face detection

Our proposed algorithm is based on the skin colour representation of face images.
However, this method is insufficient due to pixel that is false detected and supposed
as skin pixel but it is not part of the skin. In addition, our focus is on the face
detection, so the skin pixels that are detected but not part of face, has to be
eliminated. Our basic detection algorithm is

Pskin (i, j) = Pskin (i, j) ∈ DMk ∀ P (i, j) ∧ ∀ Rn (13)

where Pskin (i, j) is probability of pixel P as skin pixel if included in distribution

skin model DMk for every colour spaces Rn .

To localize face-only image, two morphological filters are utilized. First, Erosion
filter, which is mainly used to remove skin-like detected pixels that can be con-
sidered as noise. After image is cleaned, we have to compact the image, by filling
the holes in image using dilation filter. The result of this implementation of two
filters are image that has clear and compact representation. Detail description on
mathematical foundation and its practical implementation of these filters can be
found a lot in literatures. After face can be localized, a 4-neighbourhood ellipse
generator algorithm is implemented to mark the face region. Figure 7 shows step
by step our proposed face detection.

Figure 7: Proposed face detection, step by step

5 Result of experiment
Experiment has been conducted using several face databases that have been col-
lected in various ways. FDB-08 is obtained officially from FERET face database
NIST which consists of two DVD-ROM (10 Gigabytes); for this research we just
use small number of pictures. FDB-02 was created using 3.3 megapixels digital
camera Minolta S304, consists of single photograph of academic and administra-
tive staffs of Mathematics Department UNPAD. FDB-07 is set of face images
which have been taken from extension student candidates of UNPAD using Fuji
3.3 megapixels MPIX digital camera. Other database sets have been collected from
various multimedia resources and internet. Table 1 shows the detail specification
of databases used in the experiment.
 MATHEMATICAL MODEL OF SKIN COLOUR FOR FACE DETECTION

Table 1: Face Databases

Group Image Size Number of images Graphic Type Colour Bits
FDB-01 640x480 754 JPEG 24
FDB-02 1280x960 70 JPEG 24
FDB-03 Vary 285 JPEG 24
FDB-04 Vary 150 JPG, GIF 24, 8, 7
FDB-05 Vary 123 JPG, GIF 24, 8, 4
FDB-06 Vary 32 JPG, GIF 24, 8
FDB-07 140x160 2475 JPG, GIF 24
FDB-08 512x768 114 PPM 24

Table 2: Result of Experiment

rg space HSB space YCbCr space
Dataset
Ins Outs %Acc Ins Outs %Acc Ins Outs %Acc
FDB-01 103 521 17 127 503 20 160 513 24
FDB-02 52 18 74 48 22 69 57 13 81
FDB-03 149 114 57 135 128 51 153 110 58
FDB-04 103 46 69 95 54 64 92 57 62
FDB-05 72 50 59 67 55 55 74 48 61
FDB-06 21 5 81 21 5 81 18 8 69
FDB-07 1371 1021 57 1135 1221 48 1376 897 61
FDB-08 59 55 52 64 50 56 68 46 60

The goal of this experiment is to measure the detection performance of our pro-
posed face detection algorithm. In this report, we use simple qualitative perfor-
mance measurement by using human-eye visual examination. The referenced skin
model was generated from FDB-02 face database (see figure 7). We define the
result into two categories, (i) the target face is in ellipse and (ii) target face is
outside the ellipse. The result can be shown in table 2. In advance measurement,
a face detection system system makes two types of errors [1]: (i) mistaking mea-
surement of non face region which is detected as face (called false match or false
accept), and (ii) mistaking in measurement of a face region which is not detected
as face (called false non-match or false reject). There is a trade-off between false
match rate (FMR) and false non-match rate (FNMR) in every face detection sys-
tem. In fact, both FMR and FNMR are functions of the system threshold t ; if t
is decreased to make the system more tolerant to input variations and noise, then
FMR increases. On the other hand, if t is raised to make the system more secure,
then FNMR increases accordingly. The system performance at all the operating
points (thresholds, t ) can be depicted in the form of a Receiver Operating Char-
acteristic (ROC) curve.
 Setiawan Hadi, Adang Suwandi A., Iping Supriana S., Farid Wazdi

6 Concluding Remarks
We have presented the theoretical background of mathematical skin colour mod-
elling and its practical implementation in the area of face detection. The proposed
approach utilizes of a skin-color detector to detect the skin region, as the first step
(initial) detection. To localize the face region, morphological filters, erosion and
dilation, are used, conjuncted with 4-neigbourhood ellipse generation algorithm.
Experiment has been performed in three colour spaces using more than 1 Giga-
bytes data of face databases.

It is shown that the result was not very accurate in general. This problem arised
due to the use of only one skin model (from FDB-02) which perhaps not appro-
priate for other face database sets. Nonetheless, the future research will involve
the use of other techniques to enhance and assist in face localization and also to
perform face tracking such as: detection of symmetricity and skewness of faces(by
using feature point detection), multiple face detection (using clustering), general
skin detector generation , non frontal face detection, advance measurement of face
detection accuracy using ROC curve, interactive detection of face(s), 3D genera-
tion of detected face(s), and other enhancement.

References
[1] Anil K. Jain, Arun Ross and Salil Prabhakar. An Introduction to Biometric
Recognition. IEEE Transactions on Circuits and Systems for Video Technol-
ogy, vol. 14, no. 1, pp. 4–20, Jan 2004.

[2] Anton, H. Elementary Linear Algebra. 4th ed. John Wiley & Sons, Inc, New
York, USA, 1984.

[3] Forsyth, D. A. and Fleck, M. M. Automatic detection of human nudes. In-

ternational Journal of Computer Vision, vol. 32 63–77, Aug 1999.

[4] Setiawan Hadi. Laporan Kemajuan Penelitian - Pengembangan Metode 1.

Departemen Informatika ITB, Jun 2005.

[5] Hunke, M. and Waibel, A. Face locating and tracking for human computer
interaction. In Proceedings of the 28th Asilomar Conference on Signals, Sys-
tems & Computers, Nov 1994.

[6] Jones, M. J. and Rehg, J. M. Statistical color models with application to skin
detection. Tech. Rep. CRL-98/11, Compaq - Cambridge Research Laboratory,
Dec 1998.

[7] Jones, M. J. and Rehg, J. M. Statistical color models with application to

skin detection. In Proceedings of the IEEE Computer Society Conference on
Computer Vision and Pattern Recognition, vol. 3656, 1999.
 MATHEMATICAL MODEL OF SKIN COLOUR FOR FACE DETECTION

[8] Jordao L., Perrone M., Costeira J. P. and Santos-Victor J. Active face and
feature tracking. In Proceedings of the 10th International Conference on Im-
age Analysis and Processing, Sep 1999.

[9] Kim, S.-H. and Kim, H.-G. Face detection using multi-modal information.
In Proceedings of the 4th IEEE International Conference on Automatic Face
and Gesture Recognition, pp. 14–19, Mar 2000.

[10] Manly, B. F. J. Multivariate Statistical Methods: A Primer. Chapman and

Hall, 1986.

[11] Ming-Hsuan Yang, David J. Kriegman, Narendra Ahuja. Detecting Faces

in Images: A Survey. IEEE Transaction on Pattern Analysis and Machine
Intelligence, vol. 24, no. 1, pp. 34–58, Jan 2002.

[12] David Thomas John O’Mara. Automated Facial Metrology. PhD Dissertaion
Univ. of Western Australia, Feb 2002.

[13] Pavlovic V. I., Sharma R. and Huang T. S. Visual interpretation of hand

gestures for human-computer interaction: A review. IEEE Transactions on
Pattern Analysis and Machine Intelligence, vol. 7, no. 19, pp. 677–695, 1997.

[14] Raja Y., McKenna S. J. and Gong S. Segmentation and tracking using colour
mixture models. In Proceedings of the Asian Conference on Computer Vision,
vol. 1 607–614, 1998.

[15] Raja Y., McKenna S. J. and Gong S. Tracking and segmenting people in
varying lighting conditions using colour. In Proceedings of the 3rd IEEE
International Conference on Automatic Face and Gesture Recognition, pp.
228–233, Apr 1998.

[16] Rowley, H. A. Neural network-based face detection. Tech. Rep. CMU-CS-99-

117, Carnegie Mellon University, May 1999.

[17] Schmid, P. Segmentation & symmetry measure for image analysis: Applica-
tion to digital dermatoscopy. PhD thesis, École Polytechnique Fédérale de
Lausanne, 1999.

[18] Mariań Sedlaćek. Evaluation of RGB and HSV models in Human Faces De-
tection. Slovak University of Technology.

[19] Terrillon, J. and Akamatsu, S. Comparative performance of different chromi-

nance spaces for color segmentation and detection of human faces in complex
scene images. In Vision Interface ’99, 1999.

[20] Terrillon J., David M. and Akamatsu S. Detection of human faces in complex
scene images by use of a skin color model and of invariant Fourier-Mellin
moments. In Proceedings of the 14th International Conference on Pattern
Recognition, vol. 2 1350–1355, Aug 1998.
 Setiawan Hadi, Adang Suwandi A., Iping Supriana S., Farid Wazdi

[21] W Zhao, R Chellappa, A Rosenfeld and P J Phillips. Face Recognition: A

Literature Survey. ACM Computing Surveys, vol. 35, no. 4, pp. 399–458, Dec
2003.
[22] Yang, J. and Waibel, A. A real-time face tracker. In Proceedings of the IEEE
Workshop on Applications of Computer Vision, pp. 142–147, 1996.
[23] Yang, M. and Ahuja, N. Detecting human faces in color images. In Proceedings
of the 1998 IEEE International Conference on Image Processing, vol. 1 127–
130, Oct 1998.
[24] Yang, M. and Ahuja, N. Gaussian mixture model for human skin color and
its applications in image and video databases. In Proceedings of SPIE-the
International Society for Optical Engineering: Conference on Storage and
Retrieval for Image and Video Databases VII, vol. 3656 458–466, 1999.
[25] Yang J., Lu W. and Waibel A. Skin-color modeling and adaptation. Tech.
Rep. CMU-CS-97-146, Carnegie Mellon University, May 1997.
[26] Yang M., Ahuja N. and Kriegman D. Face detection using mixtures of lin-
ear subspaces. In Proceedings of the 4th IEEE International Conference on
Automatic Face and Gesture Recognition, pp. 70–76, 2000.
[27] Zarit B. D., Super B. J. and Quek F. K. H. Comparison of five color models in
skin pixel classification. In Proceedings of the IEEE International Workshop
on Recognition, Analysis, and Tracking of Faces and Gestures in Real-time
Systems, 1998.
[28] Zhu X., Yang J. and Waibel A. Segmenting hands of arbitrary color. In
Proceedings of the 4th IEEE International Conference on Automatic Face
and Gesture Recognition, pp. 448–455., 2000.

Setiawan Hadi: S3 student at Department of Informatics, Institut Teknologi Bandung,

Jl. Ganesha 10 Bandung 40132, Indonesia.
Lecturer at Dept. of Mathematics UNPAD, Phone/Fax: +62 +22 779 4696.
E-mail: setiawanhadi@ieee.org

Adang Suwandi Ahmad: Professor at Department of Electrical Engineering, Institut

Teknologi Bandung, Jl. Ganesha 10 Bandung 40132, Indonesia.
E-mail: asa@isrg.itb.ac.id

Iping Supriana Suwardi: Senior Lecturer at Department of Informatics, Institut

Teknologi Bandung, Jl. Ganesha 10 Bandung 40132, Indonesia.
E-mail: iping@informatika.org

Farid Wazdi: Senior Lecturer at Department of Informatics, Institut Teknologi Ban-

dung, Jl. Ganesha 10 Bandung 40132, Indonesia.
E-mail: farid@informatika.org