# Pixel-wise Skin Segmentation

21 October 2017
Yehya Abouelnaga, Hesham M. Eraqi, Mohamed N. Moustafa
The American University in Cairo

## Introduction

Skin segmentation was a challenging problem (in [1]) to solve due to the different lighting conditions. We use a Multivariate Gaussian Naive Bayes classifier to develop a pixel-wise skin segmentation model that operates on an HSV colorspace. Our model is very similar to [2] except that we don’t use a histogram. Instead, we use a normal distribution.

\begin{split} \text{Model}(x) & = \frac{\mathbb{P}(skin \mid x)}{\mathbb{P}(skin \mid x) + \mathbb{P}(non\text{-}skin \mid x)}
& = \frac{\mathbb{P}(x \mid skin)}{\mathbb{P}(x \mid skin) + \mathbb{P}(x \mid non\text{-}skin)} \end{split}

Note that $\mathbb{P}(skin) = \mathbb{P}(non\text{-}skin) = 0.5$ (i.e. we don’t make any assumptions about existence of skin pixels in the image).

We trained our model using the UCI Skin Segmentation dataset [3]. $\mu_{skin}$, $\Sigma_{skin}$, $\mu_{non-skin}$, and $\Sigma_{non-skin}$ are evaluated from labeled skin/non-skin values in [3]. Now, we have a normal distribution for both skin and non-skin pixels ready for use. In order to evaluate the model (i.e. skin-segment images), for each pixel, $x$, in the input image, we transform the RGB values to HSV, and then, feed them to our model. Then, we get a probability heat map of skin in the image. We classify a pixel to a “skin” when $\text{Model}(x) > 0.5$. Then, we cluster the skin pixels into objects and remove those with a small number of pixels. In other words, we don’t expect neither faces nor hands to have a very small number of pixels.

## Implementation

We implemented the skin segmentation in Tensorflow so that we could integrate it directly with our Tensorflow-based deep learning (i.e. AlexNet and InceptionV3) models. Also, porting that code into Tensorflow enabled us to train and test the skin segmentation on our GPUs.

## Results

The following are skin-segmented images using our pixel-wise skin segmentation implementation. The model is sensitive to lighting conditions and clothes colors. Adding more context-based information would probably help us improve our skin prediction accuracy.

## References

1. [1]Y. Abouelnaga, H. M. Eraqi, and M. N. Moustafa, “Real-time Distracted Driver Posture Classification,” arXiv preprint arXiv:1706.09498, 2017.
2. [2]S. L. Phung, A. Bouzerdoum, and D. Chai, “Skin segmentation using color and edge information,” in Signal Processing and Its Applications, 2003. Proceedings. Seventh International Symposium on, 2003, vol. 1, pp. 525–528.
3. [3]R. Bhatt and A. Dhall, “Skin segmentation dataset,” UCI Machine Learning Repository, 2010.