.. -*- coding: utf-8 -*- .. _bob.ip.binseg.covdresults: ========================== COVD- and COVD-SLL Results ========================== In addition to the M2U-Net architecture, we also evaluated the larger DRIU network and a variation of it that contains batch normalization (DRIU BN) on COVD- and COVD-SSL. Perhaps surprisingly, for the majority of combinations, the performance of the DRIU variants are roughly equal or worse than the M2U-Net. We anticipate that one reason for this could be overparameterization of large VGG16 models that are pretrained on ImageNet. F1 Scores =========== Comparison of F1-micro-scores (std) of DRIU and M2U-Net on COVD- and COVD-SSL. Standard deviation across test-images in brackets. +---------------------------------------------------------+--------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+ | F1 score | :ref:`bob.ip.binseg.configs.models.driu`/:ref:`bob.ip.binseg.configs.models.driussl` | :ref:`bob.ip.binseg.configs.models.driubn`/:ref:`bob.ip.binseg.configs.models.driubnssl` | :ref:`bob.ip.binseg.configs.models.m2unet`/:ref:`bob.ip.binseg.configs.models.m2unetssl` | +---------------------------------------------------------+--------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+ | :ref:`bob.ip.binseg.configs.datasets.covd-drive` | 0.788 (0.018) | 0.797 (0.019) | `0.789 (0.018) `_ | +---------------------------------------------------------+--------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+ | :ref:`bob.ip.binseg.configs.datasets.covd-drive_ssl` | 0.785 (0.018) | 0.783 (0.019) | `0.791 (0.014) `_ | +---------------------------------------------------------+--------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+ | :ref:`bob.ip.binseg.configs.datasets.covd-stare` | 0.778 (0.117) | 0.778 (0.122) | `0.812 (0.046) `_ | +---------------------------------------------------------+--------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+ | :ref:`bob.ip.binseg.configs.datasets.covd-stare_ssl` | 0.788 (0.102) | 0.811 (0.074) | `0.820 (0.044) `_ | +---------------------------------------------------------+--------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+ | :ref:`bob.ip.binseg.configs.datasets.covd-chasedb1` | 0.796 (0.027) | 0.791 (0.025) | `0.788 (0.024) `_ | +---------------------------------------------------------+--------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+ | :ref:`bob.ip.binseg.configs.datasets.covd-chasedb1_ssl` | 0.796 (0.024) | 0.798 (0.025) | `0.799 (0.026) `_ | +---------------------------------------------------------+--------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+ | :ref:`bob.ip.binseg.configs.datasets.covd-hrf` | 0.799 (0.044) | 0.800 (0.045) | `0.802 (0.045) `_ | +---------------------------------------------------------+--------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+ | :ref:`bob.ip.binseg.configs.datasets.covd-hrf_ssl` | 0.799 (0.044) | 0.784 (0.048) | `0.797 (0.044) `_ | +---------------------------------------------------------+--------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+ | :ref:`bob.ip.binseg.configs.datasets.covd-iostar` | 0.791 (0.021) | 0.777 (0.032) | `0.793 (0.015) `_ | +---------------------------------------------------------+--------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+ | :ref:`bob.ip.binseg.configs.datasets.covd-iostar_ssl` | 0.797 (0.017) | 0.811 (0.074) | `0.785 (0.018) `_ | +---------------------------------------------------------+--------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+ M2U-Net Precision vs. Recall Curves =================================== Precision vs. recall curves for each evaluated dataset. Note that here the F1-score is calculated on a macro level (see paper for more details). .. figure:: img/pr_CHASEDB1.png :scale: 50 % :align: center :alt: model comparisons CHASE_DB1: Precision vs Recall curve and F1 scores .. figure:: img/pr_DRIVE.png :scale: 50 % :align: center :alt: model comparisons DRIVE: Precision vs Recall curve and F1 scores .. figure:: img/pr_HRF.png :scale: 50 % :align: center :alt: model comparisons HRF: Precision vs Recall curve and F1 scores .. figure:: img/pr_IOSTARVESSEL.png :scale: 50 % :align: center :alt: model comparisons IOSTAR: Precision vs Recall curve and F1 scores .. figure:: img/pr_STARE.png :scale: 50 % :align: center :alt: model comparisons STARE: Precision vs Recall curve and F1 scores