.. =*= coding: utf=8 =*= .. _bob.bio.demographics.meds.analysis: ======== MEDS II ======== Follow below the demographic differentials considering some FR baselines using the :ref:`Fairness Discrepancy Rate ` . .. figure:: meds.png :align: center :alt: Fairness Discrepancy Rate FDR and FMR/FNMR trade-offs (:download:`pdf `) Facenet-sanderberg ------------------ `Facenet `_ model. FMR === =========== ===== ===== ====== ===== ===== ===== Ethnicities 1e-06 1e-05 0.0001 0.001 0.01 0.1 White-Black 0 0 0 0 0 0.019 White-White 0 0 0 0 0.004 0.067 Black-Black 0 0 0 0.002 0.032 0.289 Black-White 0 0 0 0 0 0.017 =========== ===== ===== ====== ===== ===== ===== FNMR ==== =========== ===== ===== ====== ===== ===== === Ethnicities 1e-06 1e-05 0.0001 0.001 0.01 0.1 White-White 0.131 0.131 0.119 0.06 0.012 0 Black-Black 0.08 0.08 0.08 0.013 0 0 =========== ===== ===== ====== ===== ===== === iResnet 100 ----------- `Arcface (iResnet 101) model `_ ported to pytorch. FMR === =========== ===== ===== ====== ===== ===== ===== Ethnicities 1e-06 1e-05 0.0001 0.001 0.01 0.1 White-Black 0 0 0 0 0.003 0.065 White-White 0 0 0 0 0.01 0.117 Black-Black 0 0 0 0.002 0.019 0.144 Black-White 0 0 0 0 0.003 0.071 =========== ===== ===== ====== ===== ===== ===== FNMR ==== =========== ===== ===== ====== ===== ==== === Ethnicities 1e-06 1e-05 0.0001 0.001 0.01 0.1 White-White 0.012 0.012 0 0 0 0 Black-Black 0.004 0.004 0 0 0 0 =========== ===== ===== ====== ===== ==== === iResnet 50 ---------- `Arcface (iResnet 50) model `_ ported to pytorch. FMR === =========== ===== ===== ====== ===== ===== ===== Ethnicities 1e-06 1e-05 0.0001 0.001 0.01 0.1 White-Black 0 0 0 0 0.002 0.051 White-White 0 0 0 0.001 0.008 0.109 Black-Black 0 0 0 0.002 0.022 0.161 Black-White 0 0 0 0 0.002 0.056 =========== ===== ===== ====== ===== ===== ===== FNMR ==== =========== ===== ===== ====== ===== ===== ===== Ethnicities 1e-06 1e-05 0.0001 0.001 0.01 0.1 White-White 0.071 0.071 0.06 0.06 0.036 0.024 Black-Black 0.093 0.093 0.089 0.04 0.009 0 =========== ===== ===== ====== ===== ===== ===== idiap_resnet50_arcface_2021 --------------------------- Arcface (resnet 50 **not the iResnet**) model trained using tensorflow. FNMR ==== =========== ===== ===== ====== ===== ===== ===== Ethnicities 1e-06 1e-05 0.0001 0.001 0.01 0.1 White-Black 0 0 0 0 0.003 0.047 White-White 0 0 0 0 0.011 0.124 Black-Black 0 0 0.001 0.003 0.022 0.153 Black-White 0 0 0 0 0.002 0.042 =========== ===== ===== ====== ===== ===== ===== FMR === =========== ===== ===== ====== ===== ===== ===== Ethnicities 1e-06 1e-05 0.0001 0.001 0.01 0.1 White-White 0.143 0.143 0.131 0.095 0.036 0 Black-Black 0.16 0.16 0.133 0.093 0.04 0.004 =========== ===== ===== ====== ===== ===== ===== idiap_resnet50_arcface_20210521 ------------------------------- Arcface (resnet 50 **not the iResnet**) model trained using tensorflow. This one is more recent. FMR === =========== ===== ===== ====== ===== ===== ===== Ethnicities 1e-06 1e-05 0.0001 0.001 0.01 0.1 White-Black 0 0 0 0 0.001 0.039 White-White 0 0 0 0.001 0.01 0.118 Black-Black 0 0 0 0.002 0.024 0.183 Black-White 0 0 0 0 0.001 0.047 =========== ===== ===== ====== ===== ===== ===== FNMR ==== =========== ===== ===== ====== ===== ===== ===== Ethnicities 1e-06 1e-05 0.0001 0.001 0.01 0.1 White-White 0.155 0.155 0.143 0.06 0.036 0.012 Black-Black 0.116 0.116 0.107 0.049 0.022 0 =========== ===== ===== ====== ===== ===== ===== VGG - 16 -------- VGG 16 Face model FMR === =========== ===== ===== ====== ===== ===== ===== Ethnicities 1e-06 1e-05 0.0001 0.001 0.01 0.1 White-Black 0 0 0 0 0 0.012 White-White 0 0 0 0 0.002 0.033 Black-Black 0 0 0.001 0.004 0.04 0.306 Black-White 0 0 0 0 0 0.012 =========== ===== ===== ====== ===== ===== ===== FNMR ==== =========== ===== ===== ====== ===== ===== ===== Ethnicities 1e-06 1e-05 0.0001 0.001 0.01 0.1 White-White 0.56 0.56 0.381 0.214 0.119 0.024 Black-Black 0.551 0.551 0.356 0.2 0.093 0.009 =========== ===== ===== ====== ===== ===== ===== arcface_insightface ------------------- Original `Arcface (iResnet 101) model `_. FMR === =========== ===== ===== ====== ===== ===== ===== Ethnicities 1e-06 1e-05 0.0001 0.001 0.01 0.1 White-Black 0 0 0 0 0.003 0.065 White-White 0 0 0 0 0.01 0.117 Black-Black 0 0 0 0.002 0.019 0.144 Black-White 0 0 0 0 0.003 0.071 =========== ===== ===== ====== ===== ===== ===== FNMR ==== =========== ===== ===== ====== ===== ==== === Ethnicities 1e-06 1e-05 0.0001 0.001 0.01 0.1 White-White 0.012 0.012 0 0 0 0 Black-Black 0.004 0.004 0 0 0 0 =========== ===== ===== ====== ===== ==== === idiap_inc_res_v2_2018 ------------------------------------------- Inception-resnet v2 model trained in [TFP18]_: FMR === =========== ===== ===== ====== ===== ===== ===== Ethnicities 1e-06 1e-05 0.0001 0.001 0.01 0.1 White-Black 0 0 0 0 0 0.027 White-White 0 0 0 0 0.006 0.09 Black-Black 0 0 0 0.003 0.028 0.236 Black-White 0 0 0 0 0 0.023 =========== ===== ===== ====== ===== ===== ===== FNMR ==== =========== ===== ===== ====== ===== ===== === Ethnicities 1e-06 1e-05 0.0001 0.001 0.01 0.1 White-White 0.155 0.155 0.131 0.06 0.012 0 Black-Black 0.147 0.147 0.111 0.027 0.004 0 =========== ===== ===== ====== ===== ===== ===