.. author: Yannick Dayer <yannick.dayer@idiap.ch>
.. date: Mon 09 May 2022 13:48:48 UTC+02


.. _bob.bio.spear.mobio-audio-female:

================================
 Mobio Dataset, female subjects
================================

Dataset Description
-------------------

Mobio is a collection of English voice recordings. The set of female subjects contains:

+--------------------+------------+--------------+
|                    | Identities | Sample count |
+--------------------+------------+--------------+
| train              | 13         | 2406         |
+-------+------------+------------+--------------+
|       | references |            | 90           |
|       +------------+            +--------------+
| dev   | probes     | 18         | 1890         |
+-------+------------+------------+--------------+
|       | references |            | 100          |
|       +------------+            +--------------+
| eval  | probes     | 20         | 2100         |
+-------+------------+------------+--------------+

GMM
---

.. table:: [Min. criterion: EER] Threshold on Development set: 7.550647e-01

    =====================  ==================  =================
    ..                     Development         Evaluation
    =====================  ==================  =================
    Failure to Acquire     0.0%                0.0%
    False Match Rate       20.6% (6632/32130)  7.8% (3093/39900)
    False Non Match Rate   20.6% (390/1890)    26.8% (562/2100)
    False Accept Rate      20.6%               7.8%
    False Reject Rate      20.6%               26.8%
    Half Total Error Rate  20.6%               17.3%
    =====================  ==================  =================

Command used to generate scores::

    $ bob bio pipeline -d mobio-audio-female gmm-mobio -g dev -g eval -l sge -o results/gmm_mobio_female

On 128\ [#nodes]_ CPU nodes on the SGE Grid: Ran in 15 minutes (11 minutes of training).

ISV
---

.. table:: [Min. criterion: EER] Threshold on Development set: 3.483318e-01

    =====================  ==================  ==================
    ..                     Development         Evaluation
    =====================  ==================  ==================
    Failure to Acquire     0.0%                0.0%
    False Match Rate       14.7% (4710/32130)  20.3% (8103/39900)
    False Non Match Rate   14.7% (277/1890)    17.4% (366/2100)
    False Accept Rate      14.7%               20.3%
    False Reject Rate      14.7%               17.4%
    Half Total Error Rate  14.7%               18.9%
    =====================  ==================  ==================

Command used to generate scores::

    $ bob bio pipeline -d mobio-audio-female -p isv-voxforge -g dev -g eval -l sge -o results/isv_mobio_female

On 128\ [#nodes]_ CPU nodes on the SGE Grid: Ran in 8 minutes (3 minutes of training).

Speechbrain ECAPA-TDNN
----------------------

.. table:: [Min. criterion: EER] Threshold on Development set: -5.091601e-01

    =====================  ================  ==================
    ..                     Development       Evaluation
    =====================  ================  ==================
    Failure to Acquire     0.0%              0.0%
    False Match Rate       1.9% (616/32130)  10.8% (4307/39900)
    False Non Match Rate   1.9% (36/1890)    2.5% (52/2100)
    False Accept Rate      1.9%              10.8%
    False Reject Rate      1.9%              2.5%
    Half Total Error Rate  1.9%              6.6%
    =====================  ================  ==================


Command used to generate scores::

    $ bob bio pipeline -d mobio-audio-female -p speechbrain-ecapa-voxceleb -g dev -g eval -l sge -o results/speechbrain_mobio_female

On 128\ [#nodes]_ CPU nodes on the SGE Grid: 12 minutes (no training).


.. rubric:: Footnotes

.. [#nodes] The number of nodes is a requested maximum amount and can vary depending on
    the number of jobs currently running on the grid as well as the scheduler's load
    estimation. The execution time can then also vary.