The code and data in this repository was used to generate the results in

  Carl Scheffler and Jean-Marc Odobez, ``Joint adaptive colour modelling
  and skin, hair and clothing segmentation using coherent probabilistic
  index maps'', in Proceedings of the British Machine Vision Conference,
  2011.

The instructions below describe how to reproduce the results of the above
article and how to use the color model in your own work. Most files also
have more detailed comments in them.

This repository is made available under the terms of the GNU General Public
License version 3 -- see Section 8 below and the COPYING file for details.
If you use this library in your work and write a paper or technical report
on it, please cite the article above.


Table of contents
-----------------
 1. Training data provided here
 2. How to label new images
 3. Compiling the code
 4. How to train the model
 5. How to run the experiments in the article
 6. Dependencies
 7. References
 8. License


1. Training data provided here
------------------------------
The training data for this model has an external component and an internal
component. The external component is the Compaq skin database [1], which
can be obtained by sending an email to Dr Mike Jones <mjones@merl.com>.
Once you have downloaded the database, replace the compaq_database sub-
directory distributed here with the downloaded database.

The internal component of the database contains the labels for some of the
images from the Compaq database. Image pixels were labelled as skin, hair,
clothing or background. The label images are located in the compaq_labels
directory.

There is a list file for each of the input and label image sets -- provided
in input_list.txt and label_list.txt respectively. These index files are
used by the training script.


2. How to label new images
--------------------------
Should you wish to create your own labelled dataset, you will need to
supply the input RGB images, label images and an image list file for each
of these two sets. Use the training data set provided (Section 1) as an
example.

To make a label image for a given input image:

 * Create a label image that has exactly the same dimensions as your input
   RGB image.

 * The color of each pixel in the label image determines the class of the
   pixel. Use the following colors to label the four classes.

    o Skin:       pure yellow (#FFFF00)
    o Hair:       pure red    (#FF0000)
    o Clothing:   pure green  (#00FF00)
    o Background: pure blue   (#0000FF)
   
   All other colors will be ignored (i.e., the label is assumed to be
   unknown).

 * Save the label image in a RGB format and using lossless compression. The
   PNG format is a good choice. Note that using lossy compression will
   distort the colors of your label image, causing many pixels to be
   labelled as "unknown". In particular, do *not* use the JPEG format for
   label images. (It is ok if your input images are in this format.)


3. Compiling the code
---------------------
The software provided here is a mixture of Python and C++ code. To compile
the C++ code and create the Python wrapper around it, you should only need
to run "make". See Section 6 for information on software dependencies.


4. How to train the model
-------------------------
The fully trained model is already provided with this repository. The
binary files storing the model data can be found in the storage/ directory.

To retrain the model, run the following command.

  python train.py --input_list input_list.txt --label_list label_list.txt

By default this will *not* retrain the skin and background color models
since these rely on the Compaq database, which needs to be downloaded
separately, has thousands of images and takes a long time to compute. Use
the --force flag to retrain the skin and background models too. Note that
when retraining the skin and background models from the Compaq database,
you will see a lot of warning messages. This is because a lot of the images
have damaged headers or image data. You can safely ignore the warnings.

Should you wish to train using other labelled data (see Section 2 for
labelling instructions), replace the input and label list filenames with
your own.

Running train.py with the --help flag will print full usage information.

Once training is done, you can run plot_prior_palettes.py to visualize the
skin, hair and background colour distributions. Run plot_prior_pim.py to
visualize the per-pixel class prior for each of the skin, hair, clothing
and background classes.


5. How to run the experiments in the article
--------------------------------------------
The run_experiment.py script accepts a number of hyper-parameters as
command line arguments and computes the accuracy of the skin model in a
N-fold experiment. Run

  python run_experiment.py --help

for a complete list of hyper-parameter options. You can also specify the
names of two text files that contain the paths to input and label images.
By default the fully labelled Compaq database images will be used.

Note that running an experiment will update the trained model -- in
particular the PIM prior and the hair color model using a subset of the
full training data set. Run the command in Section 3 to retrain the
standard model (i.e. using all available training data).

Figure 6a in the paper was produced by calling

  python run_experiments_mrf.py

which calls the run_experiment.py script, and then

  python plot_mrf_results.py

Figure 6b in the paper was produced with the output from the following two
calls.

  python run_experiment.py --folds 5 --kmrf 3 --skin cont --hair cont
  python run_experiment.py --folds 5 --kmrf 3 --skin disc --hair disc

Figure 7 in the paper was produced by running

  python segment_single_image.py

on the two input images shown in the figure. The script takes one command
line argument, namely the path to the image to segment. You can optionally
specify the MRF field strength and whether the skin and hair models are
continuous or discrete. Run

  python segment_single_image.py --help

for more details.


6. Dependencies
---------------
The software provided here is a mixture of Python and C++ code. The Python
code depends on the python-opencv and python-imaging libraries. All Python
scripts were tested on Python 2.6.5 under Ubuntu 10.04 LTS. The C++ code
depends on the OpenCV library. To build the Python bindings for the C++
FaceColorModel class, you will also need SWIG.


7. References
-------------
 [1] M.J. Jones and J.M. Rehg, ``Statistical color models with application
     to skin detection'', International Journal of Computer Vision,
     46(1):81-96, 2002.


8. License
----------
FaceColorModel: A repository for training and testing a face color
segmentation and adaptation model. This library provides scripts for
training the model from labelled images and a C++ class, with a Python
wrapper, for model adaptation and pixel class labelling in images.

Copyright (c) 2011 Idiap Research Institute, http://www.idiap.ch/
Written by Carl Scheffler <carl.scheffler@gmail.com>

FaceColorModel is free software: you can redistribute it and/or modify it
under the terms of the GNU General Public License version 3 as published
by the Free Software Foundation.

FaceColorModel is distributed in the hope that it will be useful, but
WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
for more details.

You should have received a copy of the GNU General Public License along
with FaceColorModel. If not, see <http://www.gnu.org/licenses/>.
