Running baselines in grandtest protocol of WMCA

In the baseline experiments, initially experiments are done separately for all four channels and score fusion is performed. Running experiments for each channel is described below. The feature based baselines implemented are Image Quality Measure + Logistic Regression (IQM+LR), Local Binary Pattern + Logistic Regression (LBP+LR), and Haralick-RDWT-SVM (H-SVM).

For the baselines using Haralick-SVM you need to have mahotas packsge. It is available from conda-forge channel and can be installed as

conda install -c https://conda.anaconda.org/conda-forge mahotas

The steps to reproduce the results for grandtest protocol are listed below.

A. Color channel

1.A.1. IQM - LR

./bin/spoof.py \
wmca-color \
iqm-lr \
--sub-directory <PATH_TO_BASELINE_RESULTS>/color/ -vvv --grid idiap

python bin/scoring.py -df \
<PATH_TO_BASELINE_RESULTS>/color/grandtest-color-50/scores/scores-dev -ef \
<PATH_TO_BASELINE_RESULTS>/color/grandtest-color-50/scores/scores-eval

1.A.2 Haralick- SVM

./bin/spoof.py \
wmca-color \
haralick-svm \
--skip-preprocessing \
--preprocessed-directory <PATH_TO_BASELINE_RESULTS>/color/preprocessed \
--sub-directory <PATH_TO_BASELINE_RESULTS>/color/haralicksvm/ -vvv --grid idiap


python bin/scoring.py -df \
<PATH_TO_BASELINE_RESULTS>/color/haralicksvm/grandtest-color-50/scores/scores-dev -ef \
<PATH_TO_BASELINE_RESULTS>/color/haralicksvm/grandtest-color-50/scores/scores-eval

B. Depth channel

1.B.1 LBP-LR

./bin/spoof.py \
wmca-depth \
lbp-lr-depth \
--sub-directory <PATH_TO_BASELINE_RESULTS>/depth/ -vvv --grid idiap

python bin/scoring.py -df \
<PATH_TO_BASELINE_RESULTS>/depth/grandtest-depth-50/scores/scores-dev -ef \
<PATH_TO_BASELINE_RESULTS>/depth/grandtest-depth-50/scores/scores-eval

1.B.2 Haralick SVM

./bin/spoof.py \
wmca-depth \
haralick-svm \
--skip-preprocessing \
--preprocessed-directory <PATH_TO_BASELINE_RESULTS>/depth/preprocessed \
--sub-directory <PATH_TO_BASELINE_RESULTS>/depth/haralicksvm/ -vvv --grid idiap

python bin/scoring.py -df \
<PATH_TO_BASELINE_RESULTS>/depth/haralicksvm/grandtest-depth-50/scores/scores-dev -ef \
<PATH_TO_BASELINE_RESULTS>/depth/haralicksvm/grandtest-depth-50/scores/scores-eval

C. Infrared channel

1.C.1 LBP-LR

./bin/spoof.py \
wmca-infrared \
lbp-lr-infrared \
--sub-directory <PATH_TO_BASELINE_RESULTS>/infrared/ -vvv --grid idiap

1.C.2 Haralick SVM

./bin/spoof.py \
wmca-infrared \
haralick-svm \
--skip-preprocessing \
--preprocessed-directory <PATH_TO_BASELINE_RESULTS>/infrared/preprocessed \
--sub-directory <PATH_TO_BASELINE_RESULTS>/infrared/haralicksvm/ -vvv --grid idiap

python bin/scoring.py -df \
<PATH_TO_BASELINE_RESULTS>/infrared/haralicksvm/grandtest-infrared-50/scores/scores-dev -ef \
<PATH_TO_BASELINE_RESULTS>/infrared/haralicksvm/grandtest-infrared-50/scores/scores-eval

D. Thermal channel

1.D.1 LBP-LR

./bin/spoof.py \
wmca-thermal \
lbp-lr-thermal \
--sub-directory <PATH_TO_BASELINE_RESULTS>/thermal/ -vvv --grid idiap

python bin/scoring.py -df \
<PATH_TO_BASELINE_RESULTS>/thermal/grandtest-thermal-50/scores/scores-dev -ef \
<PATH_TO_BASELINE_RESULTS>/thermal/grandtest-thermal-50/scores/scores-eval

1.D.2 Haralick SVM

./bin/spoof.py \
wmca-thermal \
haralick-svm \
--skip-preprocessing \
--preprocessed-directory <PATH_TO_BASELINE_RESULTS>/thermal/preprocessed \
--sub-directory <PATH_TO_BASELINE_RESULTS>/thermal/haralicksvm/ -vvv --grid idiap

python bin/scoring.py -df \
<PATH_TO_BASELINE_RESULTS>/thermal/haralicksvm/grandtest-thermal-50/scores/scores-dev -ef \
<PATH_TO_BASELINE_RESULTS>/thermal/haralicksvm/grandtest-thermal-50/scores/scores-eval

E. Score fusion (Haralick-SVM)

./bin/python bin/mean_fusion.py -c \
<PATH_TO_BASELINE_RESULTS>/color/haralicksvm/grandtest-color-50/scores/scores-dev -s \
<PATH_TO_BASELINE_RESULTS>/mean_fusion/grandtest/haralick/


python bin/scoring.py -df \
<PATH_TO_BASELINE_RESULTS>/mean_fusion/grandtest/haralick/scores_mean_fused_dev -ef \
<PATH_TO_BASELINE_RESULTS>/mean_fusion/grandtest/haralick/scores_mean_fused_eval

F. Score fusion (IQM-LBP-LR)

./bin/python bin/mean_fusion.py -c \
<PATH_TO_BASELINE_RESULTS>/color/grandtest-color-50/scores/scores-dev -s \
<PATH_TO_BASELINE_RESULTS>/mean_fusion/grandtest/

python bin/scoring.py -df \
<PATH_TO_BASELINE_RESULTS>/mean_fusion/grandtest/scores_mean_fused_dev -ef \
<PATH_TO_BASELINE_RESULTS>/mean_fusion/grandtest/scores_mean_fused_eval

BASELINES in LOO protocols (for score fusion)

For the experiments first do individual experiments for all channels, after that perform fusion. This needs to be repeated for all the protocols

A. Color

unseen_protocols=[‘’,’-LOO_fakehead’,’-LOO_flexiblemask’,’-LOO_glasses’,’-LOO_papermask’,’-LOO_prints’,’-LOO_replay’,’-LOO_rigidmask’]

The protocols can be easily indexed as

PROTOCOL = ‘grandtest-color-50’+unseen_protocols[0]

./bin/spoof.py \
wmca-color \
iqm-lr \
--protocol grandtest-color-50-LOO_fakehead \
--skip-preprocessing --skip-extraction --skip-extractor-training  --allow-missing-files \
--extracted-directory <PATH_TO_BASELINE_RESULTS>/color/extracted/ \
--grid idiap \
--sub-directory <PATH_TO_BASELINE_RESULTS>/LOO/color/grandtest-color-50-LOO_fakehead/

B. Depth

./bin/spoof.py \
wmca-depth \
lbp-lr-depth \
--protocol grandtest-depth-50-LOO_fakehead \
--skip-preprocessing --skip-extraction --skip-extractor-training  --allow-missing-files \
--extracted-directory <PATH_TO_BASELINE_RESULTS>/depth/extracted/ \
--grid idiap \
--sub-directory <PATH_TO_BASELINE_RESULTS>/LOO/depth/grandtest-depth-50-LOO_fakehead/

C. Infrared

./bin/spoof.py \
wmca-infrared \
lbp-lr-infrared \
--protocol grandtest-infrared-50-LOO_fakehead \
--skip-preprocessing --skip-extraction --skip-extractor-training  --allow-missing-files \
--extracted-directory <PATH_TO_BASELINE_RESULTS>/infrared/extracted/ \
--grid idiap \
--sub-directory <PATH_TO_BASELINE_RESULTS>/LOO/infrared/grandtest-infrared-50-LOO_fakehead/

D. Thermal

./bin/spoof.py \
wmca-thermal \
lbp-lr-thermal \
--protocol grandtest-thermal-50-LOO_fakehead \
--skip-preprocessing --skip-extraction --skip-extractor-training  --allow-missing-files \
--extracted-directory <PATH_TO_BASELINE_RESULTS>/thermal/extracted/ \
--grid idiap \
--sub-directory <PATH_TO_BASELINE_RESULTS>/LOO/thermal/grandtest-thermal-50-LOO_fakehead/

E. Score fusion

The protocols are, ‘-LOO_fakehead’,’-LOO_flexiblemask’,’-LOO_glasses’,’-LOO_papermask’,’-LOO_prints’,’-LOO_replay’,’-LOO_rigidmask’

./bin/python bin/mean_fusion.py -c \
<PATH_TO_BASELINE_RESULTS>/LOO/color/grandtest-color-50-LOO_fakehead/grandtest-color-50-LOO_fakehead/scores/scores-dev

python bin/scoring.py -df \
<PATH_TO_BASELINE_RESULTS>/mean_fusion/grandtest-color-50-LOO_fakehead/fusionmine/scores_mean_fused_dev -ef \
<PATH_TO_BASELINE_RESULTS>/mean_fusion/grandtest-color-50-LOO_fakehead/fusionmine/scores_mean_fused_eval

Similarly, repeat the experiment for all protocols.

BASELINES in LOO protocols (Haralick-SVM)

Color

./bin/spoof.py wmca-color haralick-svm --protocol grandtest-color-50-LOO_fakehead --skip-preprocessing --skip-extraction --skip-extractor-training  --allow-missing-files --extracted-directory <PATH_TO_BASELINE_RESULTS>/color/haralicksvm/extracted/ --grid idiap --sub-directory <PATH_TO_BASELINE_RESULTS>/LOO/color/haralicksvm/grandtest-color-50-LOO_fakehead/

Repeat the same procedure for all four channels.

Score fusion Haralick-SVM

Once scores from all channels are available. Run the following command.

./bin/python bin/mean_fusion.py -c <PATH_TO_BASELINE_RESULTS>/LOO/color/haralicksvm/grandtest-color-50-LOO_fakehead/grandtest-color-50-LOO_fakehead/scores/scores-dev -s <PATH_TO_BASELINE_RESULTS>/mean_fusion/haralick/grandtest-color-50-LOO_fakehead/

Scoring for Haralick Mean fusion

python bin/scoring.py -df <PATH_TO_BASELINE_RESULTS>/mean_fusion/haralick/grandtest-color-50-LOO_fakehead/scores_mean_fused_dev -ef <PATH_TO_BASELINE_RESULTS>/mean_fusion/haralick/grandtest-color-50-LOO_fakehead/scores_mean_fused_eval

Similarly, repeat the experiment for all LOO sub-protocols.