Fairness Discrepancy RateΒΆ

Here we discuss the fairness discrepancy rate (FDR) proposed in:

@article{de2020fairness,
   title={Fairness in Biometrics: a figure of merit to assess biometric verification systems},
   author={de Freitas Pereira, Tiago and Marcel, S{\'e}bastien},
   journal={arXiv preprint arXiv:2011.02395},
   year={2020}
}

In this work, a biometric verification system is considered fair if statistical parity between groups is reached in terms of both FMR (False Match Rate) and FNMR (False Non Match Rate) for a given decision threshold \(\tau\). More formally, given a set of demographic groups \(\mathcal{D}=\{d_1,d_2,...,d_n\}\), and \(\tau = \text{FMR}_{x}\), a biometric verification system is considered fair with respect to FMR if the following premisse holds:

\[\text{FMR}^{d_i}(\tau) \geq \text{FMR}^{d_j}(\tau) - \epsilon \text{ } \forall d_i,d_j \in D\]

Such premisse can be written with the following equation:

\[A(\tau) = \max(|\text{FMR}^{d_i}(\tau)- \text{FMR}^{d_j}(\tau)|) \leq \epsilon \text{ } \forall d_i, d_j \in \mathcal{D}\]

Conversely, in terms of \(\text{FNMR}\), a biometric verification system is considered fair if the following premisse holds:

\[\text{FNMR}^{d_i}(\tau) \geq \text{FNMR}^{d_j}(\tau) \text{ } \forall d_i,d_j \in D$.\]

Such premisse can be written with the following equation:

\[B(\tau) = \max(|\text{FNMR}^{d_i}(\tau)- \text{FNMR}^{d_j}(\tau)|) \leq \epsilon \text{ } \forall d_i, d_j \in \mathcal{D}.\]

Since A and B are functions of \(\tau\), both can be summarized in one figure of merit, that we refer as Fairness Discrepancy Rate (FDR) which is defined as:

\[FDR(\tau) = 1- (\alpha A(\tau) + (1-\alpha) B(\tau)),\]

where \(\alpha\) is a hyper-parameter that defines the weight of \(A(\tau)\) in the figure of merit (the importance of False Matches).

To see how FDR behaves in situations of fair/unfair score distributions, please check this link.