.. -*- coding: utf-8 -*-
.. _bob.ip.binseg.benchmarkresults:


==================
Benchmark Results
==================

F1 Scores
===========

* Benchmark results for models: DRIU, HED, M2UNet and U-Net.
* Models are trained and tested on the same dataset using the train-test split as indicated in :ref:`bob.ip.binseg.datasets`
* Standard-deviations across all test images are indicated in brakets

+--------------------------------------------+------------------------------------------------+---------------------------------------------+-------------------------------------------+----------------------------------------------+---------------------------------------------+
| F1 (std)                                   | :ref:`bob.ip.binseg.configs.datasets.chasedb1` | :ref:`bob.ip.binseg.configs.datasets.drive` | :ref:`bob.ip.binseg.configs.datasets.hrf` | :ref:`bob.ip.binseg.configs.datasets.iostar` | :ref:`bob.ip.binseg.configs.datasets.stare` |
+--------------------------------------------+------------------------------------------------+---------------------------------------------+-------------------------------------------+----------------------------------------------+---------------------------------------------+
| :ref:`bob.ip.binseg.configs.models.driu`   | `0.810 (0.021) <driu_chasedb1.pth_>`_          | `0.820 (0.014) <driu_drive.pth_>`_          | `0.783 (0.055) <driu_hrf.pth_>`_          | `0.825 (0.020) <driu_iostar.pth_>`_          | `0.827 (0.037) <driu_stare.pth_>`_          |
+--------------------------------------------+------------------------------------------------+---------------------------------------------+-------------------------------------------+----------------------------------------------+---------------------------------------------+
| :ref:`bob.ip.binseg.configs.models.hed`    | 0.810 (0.022)                                  | 0.817 (0.013)                               | 0.783 (0.058)                             | 0.825 (0.020)                                | 0.823 (0.037)                               |
+--------------------------------------------+------------------------------------------------+---------------------------------------------+-------------------------------------------+----------------------------------------------+---------------------------------------------+
| :ref:`bob.ip.binseg.configs.models.m2unet` | `0.802 (0.019) <m2unet_chasedb1.pth_>`_        | `0.803 (0.014) <m2unet_drive.pth_>`_        | `0.780 (0.057) <m2unet_hrf.pth_>`_        | `0.817 (0.020) <m2unet_iostar.pth_>`_        | `0.815 (0.041) <m2unet_stare.pth_>`_        |
+--------------------------------------------+------------------------------------------------+---------------------------------------------+-------------------------------------------+----------------------------------------------+---------------------------------------------+
| :ref:`bob.ip.binseg.configs.models.unet`   | 0.812 (0.020)                                  | 0.822 (0.015)                               | 0.788 (0.051)                             | 0.818 (0.019)                                | 0.829 (0.042)                               |
+--------------------------------------------+------------------------------------------------+---------------------------------------------+-------------------------------------------+----------------------------------------------+---------------------------------------------+



.. include:: links.rst