Evaluation of a binary anti-spoofing system
Algorithms have at least one input and one output. All algorithm endpoints are organized in groups. Groups are used by the platform to indicate which inputs and outputs are synchronized together. The first group is automatically synchronized with the channel defined by the block in which the algorithm is deployed.
| Endpoint Name | Data Format | Nature |
|---|---|---|
| scores_test | system/float/1 | Input |
| label_test | system/text/1 | Input |
| Endpoint Name | Data Format | Nature |
|---|---|---|
| scores_dev | system/float/1 | Input |
| label_dev | system/text/1 | Input |
Analyzers may produce any number of results. Once experiments using this analyzer are done, you may display the results or filter experiments using criteria based on them.
| Name | Type |
|---|---|
| far_test | float32 |
| number_of_negatives_dev | int32 |
| frr_test | float32 |
| far_dev | float32 |
| eer | float32 |
| number_of_negatives_test | int32 |
| roc_test | plot/scatter/1 |
| roc_dev | plot/scatter/1 |
| threshold | float32 |
| frr_dev | float32 |
| hter | float32 |
| number_of_positives_test | int32 |
| number_of_positives_dev | int32 |
xxxxxxxxxximport numpyimport bobclass Algorithm: def __init__(self): self.positives_dev = [] self.negatives_dev = [] self.positives_test = [] self.negatives_test = [] def process(self, inputs, outputs): # accumulate the dev scores while inputs['scores_dev'].hasMoreData(): inputs['scores_dev'].next() inputs['label_dev'].next() data_dev = inputs['scores_dev'].data.value label_dev = inputs['label_dev'].data.text if label_dev == 'real': self.positives_dev.append(data_dev) else: self.negatives_dev.append(data_dev) # accumulate the test scores data_test = inputs['scores_test'].data.value label_test = inputs['label_test'].data.text if label_test == 'real': self.positives_test.append(data_test) else: self.negatives_test.append(data_test) # once all values are received, evaluate the scores if not(inputs.hasMoreData()): eer_threshold = bob.measure.eer_threshold(self.negatives_dev, self.positives_dev) far_dev, frr_dev = bob.measure.farfrr(self.negatives_dev, self.positives_dev, eer_threshold) far_test, frr_test = bob.measure.farfrr(self.negatives_test, self.positives_test, eer_threshold) roc_points_dev = bob.measure.roc(self.negatives_dev, self.positives_dev, 100) roc_points_test = bob.measure.roc(self.negatives_test, self.positives_test, 100) # writes the output back to the platform outputs.write({ 'eer': numpy.float32((far_dev+frr_dev) / 2.), 'hter': numpy.float32((far_test+frr_test) / 2.), 'far_dev': numpy.float32(far_dev), 'frr_dev': numpy.float32(frr_dev), 'far_test': numpy.float32(far_test), 'frr_test': numpy.float32(frr_test), 'number_of_positives_dev': numpy.int32(len(self.positives_dev)), 'number_of_negatives_dev': numpy.int32(len(self.negatives_dev)), 'number_of_positives_test': numpy.int32(len(self.positives_test)), 'number_of_negatives_test': numpy.int32(len(self.negatives_test)), 'threshold': numpy.float32(eer_threshold), 'roc_dev': { "data": [ { "label": "development", "x": roc_points_dev[0], "y": roc_points_dev[1], } ] }, 'roc_test': { "data": [ { "label": "test", "x": roc_points_test[0], "y": roc_points_test[1], } ] }, }) return TrueThe code for this algorithm in Python
The ruler at 80 columns indicate suggested POSIX line breaks (for readability).
The editor will automatically enlarge to accomodate the entirety of your input
Use keyboard shortcuts for search/replace and faster editing. For example, use Ctrl-F (PC) or Cmd-F (Mac) to search through this box
An algorithm that implements standard metrics for antispoofing evaluation.
Specifically, it returns:
This implementation relies on the 'measure' package from the Bob library. See http://www.idiap.ch/software/bob/docs/releases/last/sphinx/html/measure/ for more details.
| Updated | Name | Databases/Protocols | Analyzers | |||
|---|---|---|---|---|---|---|
| smarcel/ivana7c/simple-antispoofing-updated/1/antispoof-chi2-expA-rr22 | replay/1@countermeasure | ivana7c/spoofing_eer/1 | ||||
| ivana7c/ivana7c/simple-antispoofing-updated/1/antispoof-chi2-expA | replay/1@countermeasure | ivana7c/spoofing_eer/1 |
This table shows the number of times this algorithm has been successfully run using the given environment. Note this does not provide sufficient information to evaluate if the algorithm will run when submitted to different conditions.