Professional Documents
Culture Documents
Salah 2021
Salah 2021
Salah 2021
https://doi.org/10.1007/s00371-021-02108-3
ORIGINAL ARTICLE
Abstract
Human skin detection, which is one of the important pre-processing phases, has a wide range of applications such as face
tracking, skin diseases, video surveillance, web content filtering, and so on. Skin detection is a challenging problem because
skin color can vary dramatically in its appearance due to many factors such as illumination conditions, pose variations, race,
aging, and complex background. Several methods dealing with skin detection assume that skin pixels can be extracted from
background colors according to some thresholding rules related to a specific color model. Nevertheless, it is a complex task
to recognize skin pixels under the challenging factors aforementioned. In the recent era, the success of deep convolutional
neural network (CNN) has strongly influenced the field of computer vision. However, we could find only a few researches that
apply deep learning methods to deal with the skin detection problem. This paper presents a novel approach based on CNN
for skin detection. Extensive experiments show that the proposed approach exceeds the best result for other state-of-the-art
methods.
Keywords Skin detection · Color models · Deep learning · Convolutional neural networks
123
K. B. Salah et al.
2 Related work
Different color models are applied in many applications such Fig. 1 Illustration of the HSV color model
as TV broadcasting, image processing, computer graphics,
and computer vision [16–18]. The most common ones are
YCbCr and HSV. The choice of color model is the primary – H is the hue: corresponds to the color perception;
process for modeling skin color and further for classification – S is the saturation: describes the color purity, simply its
[19]. According to [20], HSV is classified as a perceptual vivid character or dull;
uniform model and YCbCr as an orthogonal model, and – V is the value: indicates the light quantity in the color, if
YCbCr is chosen as the most suited for skin detection prob- it has light or dark appearance.
lem because orthogonal models are characterized by lower
correlation between components. In the HSV color model, the intensity is represented by the
component V. This component must be abandoned in the
2.1.1 YCbCr color model process of skin detection. Only the H and S components are
maintained which represent the chrominance. The skin color
This color model is often used in image compression. The detection, according to the HSV model, is realized by the
luminance (also called Luma) is presented by the component conversion of the original image from the RGB model to the
Y and calculated by a weighted sum of the components of R, HSV model respecting the following thresholds [23].
G, and B [21]
((Cr >= 133) and (Cr <= 173)). (5) A fair comparison between works based on deep learning
for skin detection is difficult due to the unavailability of
2.1.2 HSV color model a common benchmark. A fully convolutional neural net-
work method for skin segmentation was present in [24]. The
HSV is a color model called “natural”, in other words, close authors make experiments on many CNN structures to deter-
to the physiological perception of color by the human eye. mine the best one. A handcrafted skin dataset was provided
It decomposes colors according to physiological criteria as in their study consisting of three well-known ECU, SFA, and
illustrated in Fig. 1: Pratheepan datasets. The first and second datasets were used
123
A novel approach for human skin detection using convolutional neural network
can detect, localize, and extract the human skin region of the
input image. All steps of the proposed approach are shown
in Fig. 5.
123
K. B. Salah et al.
123
A novel approach for human skin detection using convolutional neural network
Fig. 5 Framework of the proposed skin detection algorithm. The skin map is constructed progressively and displayed in steps 0–9. Each pixel of
the image is predicted with the trained model
123
K. B. Salah et al.
123
A novel approach for human skin detection using convolutional neural network
Fig. 8 Example of results of human skin detection with different methods on SFA dataset. a Original image, b skin detection with YCbCr, c skin
detection with HSV, d skin detection with HSCBCR, and e skin detection with the proposed approach
123
K. B. Salah et al.
Fig. 9 Experimental results on Pratheepan dataset. a Original image, b skin detection with the proposed approach, c ground truth image, and d
skin map obtained with the proposed approach
123
A novel approach for human skin detection using convolutional neural network
Table 4 Performances of different skin detection methods on SFA fundamental objective of this study was to explore the poten-
dataset tial of CNN learning model for skin pixel classification. A
Accuracy Recall F-measure Mcc IOU comparison is carried out among the existing methods and
our approach. As exhibited in experiments, the proposed
YCbCr 0.8898 0.8700 0.9200 0.7655 0.6988
method outperforms state-of-the-art methods in terms of
HSV 0.8597 0.8200 0.8900 0.7220 0.6488
recall, accuracy, precision, and F-measure in different con-
HSCbCr 0.8477 0.8000 0.8800 0.7060 0.6323
ditions: complex background, variation in the illumination
Ours 0.9239 0.9600 0.9500 0.8126 0.7601
level, and ethnicity. The comparison was performed using
The bold is used to identify which method exceeds in term of value in the SFA and Pratheepan datasets. We have trained the CNN
that metric with skin and non-skin patches from the SFA dataset instead
of whole-image based strategy. We found that this training
Table 5 Evaluation results on Pratheepan dataset
strategy finds skin color as well as its texture. Then, when we
Methods Accuracy Precision Recall F-measure integrate the trained model in the global algorithm for human
Bayesian [2] 0.8237 0.6881 0.8972 0.7788
skin detection, it was efficient in rejecting non-skin pixels in
FSD [25] 0.8255 0.8077 0.6851 0.7414
various situations.
LASD [26] 0.8361 0.7954 0.8275 0.8111
FPSD [27] 0.8419 0.7387 0.8991 0.8070 Declarations
DSPF [28] 0.8521 0.7543 0.8436 0.7964
Conflict of interest The authors declared that they have no conflicts of
SPSD [29] 0.8782 0.7659 0.9328 0.8412
interest to this work.
Patch-VGG [30] 0.9299 0.8563 0.8750 0.8655
Patch-NiN [30] 0.9334 0.8802 0.8972 0.8886
Image-VGG [30] 0.9313 0.8577 0.9069 0.8816
Image-NiN [30] 0.9484 0.9003 0.8912 0.8957 References
FCNN [24] 0.9499 0.8480 0.8981 0.8678
1. Chen, W., Wang, K., Jiang, H., et al.: Skin color modeling for
Ours 0.9357 0.9801 0.9730 0.9765
face detection and segmentation: a review and a new approach.
The bold is used to identify which method exceeds in term of value in Multimed. Tools Appl. 75, 839–862 (2016)
that metric 2. Jones, M.J., Rehg, J.M.: Statistical color models with application
to skin detection. Int. J. Comput. Vis. 46, 81–96 (2002). https://
doi.org/10.1023/A:1013200319198
5 Conclusion 3. Zafarifar, B., Bellers, E.B., de With P.H.: Application and eval-
uation of texture-adaptive skin detection in TV image enhance-
ment. In: IEEE International Conference on Consumer Electronics
In this work, a novel approach based on convolutional neu- (ICCE), pp. 88–91 (2013). https://doi.org/10.1109/ICCE.2013.
ral networks for human skin detection is presented. The 6486807
123
K. B. Salah et al.
4. Rautaray, S.S., Agrawal, A.: Vision based hand gesture recognition 22. Chai, D., Ngan, K.N.: Face segmentation using skin-color map in
for human computer interaction: a survey. Artif. Intell. Rev. 43, 1– videophone applications. IEEE Trans. Circuits Syst. Video Tech-
54 (2015). https://doi.org/10.1007/s10462-012-9356-9 nol. 9(4), 551–564 (1999)
5. Zhang, Z., Gunes, H., Piccardi, M.: Head detection for video 23. Chitra, S., Balakrishnan, G.: Comparative study for two color
surveillance based on categorical hair and skin colour models. In: spaces HSCbCr and YCbCr in skin color detection. Appl. Math.
IEEE International Conference on Image Processing, pp. 1137– Sci. 6, 4229–4238 (2012)
1140 (2009) 24. Ma, C., Shih, H.: Human skin segmentation using fully convo-
6. Schaefer, G., Tait, R., Zhu, S.Y.: Overlay of thermal and visual lutional neural networks. Nara (2018). https://doi.org/10.1109/
medical images using skin detection and image registration. In: GCCE.2018.8574747
International Conference of the IEEE Engineering in Medicine and 25. Tan, W.R., Chan, C.S., Yogarajah, P., Condell, J.: A fusion approach
Biology Society, NY, vol. 2, pp. 965–967 (2006). https://doi.org/ for efficient human skin detection. IEEE Trans. Ind. Inform. 8, 138–
10.1109/IEMBS.2006.259275 147 (2012)
7. Devi, M.S., Bajaj, P.R.: Driver fatigue detection based on eye 26. Hwang, I., Lee, S.H., Min, B., Cho, N.I.: Luminance adapted skin
tracking. In: First International Conference on Emerging Trends color modeling for the robust detection of skin areas. In: Proceed-
in Engineering and Technology, pp. 649–652 (2008). https://doi. ings of IEEE ICIP, pp. 2622–2625 (2013)
org/10.1109/ICETET.2008.17 27. Kawulok, M.: Fast propagation based skin regions segmentation in
8. Fang, R., Pouyanfar, S., Yang, Y., Chen, S.-C., Iyengar, S.: Com- color images. In: Proceedings of IEEEFG, pp. 1–7 (2013)
putational health informatics in the bigdata age: a survey. ACM 28. Kawulok, M., Kawulok, J., Nalepa, J.: Spatial based skin detection
Comput. Surv. 49, 12 (2016) using discriminative skin presence features. Pattern Recognit. Lett.
9. Erdem, C.E., Ulukaya, S., Karaali, A., Erdem, A.T.: Combining 41, 3–13 (2014)
Haar feature and skin color based classifiers for face detection. In: 29. Hwang, I., Kim, Y., Cho, N.I.: Skin detection based on multi-seed
IEEE International Conference on Acoustics, Speech and Signal propagation in a multi-layer graph for regional and color consis-
Processing Proceedings, pp. 1497–1500 (2011) tency. In: IEEE ICASSP (2017). https://doi.org/10.1109/ICASSP.
10. Zhu, Q., Cheng, K.T., Wu, C.T., Wu, Y.L.: Adaptive learning of an 2017.7952361
accurate skin-color model. In: IEEE International Conference on 30. Kim, Y., Hwang, I., Cho, N.I.: Convolutional neural networks and
Automatic Face and Gesture Recognition, pp. 37–42 (2004) training strategies for skin detection. In: IEEE ICIP (2017). https://
11. Al-Tairi, Z., Wirza, R., Saripan, M.I., Sulaiman, P.: Skin segmen- doi.org/10.1109/ICIP.2017.8297017
tation using YUV and RGB color spaces. J. Inf. Process. Syst. 10, 31. Lecun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature
283–299 (2014) 521(7553), 436–444 (2015)
12. Rahman, M.A., Edy Purnama, I.K., Purnomo, M.H.: Simple 32. Ahmad, J., Muhammad, K., Bakshi, S., Baik, S.W.: Object-oriented
method of human skin detection using HSV and YCbCr color convolutional features for fine-grained image retrieval in large
spaces. In: International Conference on Intelligent Autonomous surveillance datasets. Future Gener. Comput. Syst. 81, 314–330
Agents, Networks and Systems, pp. 58–61 (2015) (2018)
13. Bin Abdul Rahman, N.A., Wei, K.C., See, J.: RGB-HCbCr skin 33. Mudassar, R., Muhammad, S., Mussarat, Y., Attique, K.M.,
colour model for human face detection. In: Proceedings of The Tanzila, S., Lawrence, F.S.: Appearance based pedestrians’ gender
MMU International Symposium on Information and Communica- recognition by employing stacked auto encoders in deep learning.
tions Technologies, pp. 90–96 (2006) Future Gener. Comput. Syst. 88, 28–39 (2018)
14. Hajiarbabi, M., Agah, A.: Face detection in color images using skin 34. Hong, T.J., Bhandary, S.V., Sobha, S., Yuki, H., Akanksha, B.,
segmentation. J. Autom. Mob. Robot. Intell. Syst. 8, 41–51 (2014) Raghavendra, U., et al.: Age-related macular degeneration detec-
15. Li, Y., Wang, Z., Yang, X., et al.: Efficient convolutional hierar- tion using deep convolutional neural network. Future Gener.
chical autoencoder for human motion prediction. Vis. Comput. 35, Comput. Syst. 87, 127–135 (2018)
1143–1156 (2019). https://doi.org/10.1007/s00371-019-01692-9 35. Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhut-
16. Ganesan, P., Rajini, V.: YIQ color space based satellite image dinov, R.: Dropout: a simple way to prevent neural networks from
segmentation using modified FCM clustering and histogram equal- overfitting. J. Mach. Learn. Res. 15, 1929–1958 (2014)
ization. In: Advances in Electrical Engineering (ICAEE), pp. 9–11 36. Casati, J.P.B., Moraes, D.R., Rodrigues, E.L.L.: SFA: a human
(2014) skin image database based on FERET and AR facial images. In:
17. Ganesan, P., Rajini, V.: Assessment of satellite image segmentation IX Workshop de Visao Computational, Rio de Janeiro (2013)
in RGB and HSV color space using image quality measures. In: 37. Yogarajah, P., Condell, J., Curran, K., Cheddad, A., McKevitt, P.: A
Advances in Electrical Engineering (ICAEE), pp. 9–11 (2014) dynamic threshold approach for skin segmentation in color images.
18. Ganesan, P., Rajini, V.: Value based semi automatic segmentation of In: Proceedings of IEEE ICIP, pp. 2225–2228 (2010)
satellite images using HSV color space, histogram equalization and
modified FCM clustering algorithm. In: Green Computing. Com-
munication and Conservation of Energy (ICGCE), p. 77 (2013)
Publisher’s Note Springer Nature remains neutral with regard to juris-
19. Nikolskaia, K., Ezhova, N., Sinkov, A., Medvedev, M.: Skin detec-
dictional claims in published maps and institutional affiliations.
tion technique based on HSV color model and SLIC segmentation
method. In: CEUR Workshop Proceedings, pp. 1323–1355 (2018)
20. Kakumanu, P., Makrogiannis, S., Bourbakis, N.: A survey of skin-
color modeling and detection methods. Pattern Recognit. (2007).
https://doi.org/10.1016/j.patcog.2006.06.010
21. Sack, H., Meinel, C.: Digitale Kommunikation: Vernetzen, Multi-
media, Sicherheit. Springer, Berlin (2009)
123
A novel approach for human skin detection using convolutional neural network
123