BEAT - Kanma/iqm_spoof_eer

This algorithm is a legacy one. The API has changed since its implementation. New versions and forks will need to be updated.

This algorithm is an analyzer. It can only be used on analysis blocks.

Endpoint Groups 2

Algorithms have at least one input and one output. All algorithm endpoints are organized in groups. Groups are used by the platform to indicate which inputs and outputs are synchronized together. The first group is automatically synchronized with the channel defined by the block in which the algorithm is deployed.

Group: test

Endpoint Name	Data Format	Nature
scores_test	system/array_1d_floats/1	Input
label_test	system/text/1	Input

Group: dev

Endpoint Name	Data Format	Nature
scores_dev	system/array_1d_floats/1	Input
label_dev	system/text/1	Input

Results 15

Analyzers may produce any number of results. Once experiments using this analyzer are done, you may display the results or filter experiments using criteria based on them.

Name	Type
test_frr	float32
dev_eer	float32
dev_roc	plot/isoroc/1
test_roc	plot/isoroc/1
test_numNegatives	int32
dev_numNegatives	int32
dev_scoreDistribution	plot/bar/1
test_numPositives	int32
dev_far	float32
test_hter	float32
dev_numPositives	int32
test_scoreDistribution	plot/bar/1
test_far	float32
dev_frr	float32
dev_eer_threshold	float32

import numpy
import bob.measure

# same functionality as earlier version but using Bob_v2

class Algorithm:

def __init__(self):
        self.positives_dev = None
        self.negatives_dev = None
        self.positives_test = []
        self.negatives_test = []
        self.n_bins = 50

def process(self, inputs, outputs):

# accumulate the dev scores
        if self.positives_dev is None:
            self.positives_dev = []
            self.negatives_dev = []

group = inputs.groupOf('scores_dev')
            while group.hasMoreData():
              group.next()
              data_dev = inputs['scores_dev'].data.value
              label_dev = inputs['label_dev'].data.text
              if label_dev == 'real':
                self.positives_dev.append(data_dev)
              else:  
                self.negatives_dev.append(data_dev)

# accumulate the test scores
        data_test = inputs['scores_test'].data.value
        label_test = inputs['label_test'].data.text
        if label_test == 'real':
            self.positives_test.append(data_test)
        else:  
            self.negatives_test.append(data_test)

# once all values are received, evaluate the scores
        if not(inputs.hasMoreData()):
            self.positives_dev = numpy.vstack(self.positives_dev).flatten()
            self.negatives_dev = numpy.vstack(self.negatives_dev).flatten()
            
            self.positives_test = numpy.vstack(self.positives_test).flatten()
            self.negatives_test = numpy.vstack(self.negatives_test).flatten()
            
            dev_hist_pos, dev_bin_pos = numpy.histogram(self.positives_dev, self.n_bins)
            dev_hist_neg, dev_bin_neg = numpy.histogram(self.negatives_dev, self.n_bins)
            
            test_hist_pos, test_bin_pos = numpy.histogram(self.positives_test, self.n_bins)
            test_hist_neg, test_bin_neg = numpy.histogram(self.negatives_test, self.n_bins)
            
            eer_threshold = bob.measure.eer_threshold(self.negatives_dev, self.positives_dev)
            far_dev, frr_dev   = bob.measure.farfrr(self.negatives_dev, self.positives_dev, eer_threshold)
            far_test, frr_test = bob.measure.farfrr(self.negatives_test, self.positives_test, eer_threshold)
            roc_points_dev     = bob.measure.roc(self.negatives_dev, self.positives_dev, 100)
            roc_points_test    = bob.measure.roc(self.negatives_test, self.positives_test, 100)

# writes the output back to the platform
            outputs.write({
                'dev_eer': numpy.float32((far_dev+frr_dev) / 2.),
                'test_hter': numpy.float32((far_test+frr_test) / 2.),
                'dev_far': numpy.float32(far_dev),
                'dev_frr': numpy.float32(frr_dev),
                'test_far': numpy.float32(far_test),
                'test_frr': numpy.float32(frr_test),
                'dev_numPositives': numpy.int32(len(self.positives_dev)),
                'dev_numNegatives': numpy.int32(len(self.negatives_dev)),
                'test_numPositives': numpy.int32(len(self.positives_test)),
                'test_numNegatives': numpy.int32(len(self.negatives_test)),
                'dev_eer_threshold': numpy.float32(eer_threshold),
                'dev_roc': {
                        "data":
                        [   
                            {                              
                                "label": "roc_dev",
                                "false_positives": roc_points_dev[1],
                                "false_negatives": roc_points_dev[0],
                                "number_of_positives": numpy.uint64(len(self.positives_dev)),
                                "number_of_negatives": numpy.uint64(len(self.negatives_dev)),
                            }   
                        ]   
                    },
                'test_roc': {
                    "data":
                        [   
                            {   
                                "label": "roc_test",
                                "false_positives": roc_points_test[1],
                                "false_negatives": roc_points_test[0],
                                "number_of_positives": numpy.uint64(len(self.positives_test)),
                                "number_of_negatives": numpy.uint64(len(self.negatives_test)),
                            }   
                        ]   
                    },
                
                'dev_scoreDistribution': {
                    "data":
                        [
                            {
                                "label": "negative scores",
                                "x": dev_bin_neg[:-1],
                                "y": dev_hist_neg,
                            },
                            {
                                "label": "positive scores",
                                "x": dev_bin_pos[:-1],
                                "y": dev_hist_pos,
                            }
                        ]
                    },
                'test_scoreDistribution': {
                    "data":
                        [
                            {
                                "label": "negative scores",
                                "x": test_bin_neg[:-1],
                                "y": test_hist_neg,
                            },
                            {
                                "label": "positive scores",
                                "x": test_bin_pos[:-1],
                                "y": test_hist_pos,
                            }
                        ]
                    },
            })
        return True

The code for this algorithm in Python
The ruler at 80 columns indicate suggested POSIX line breaks (for readability).
The editor will automatically enlarge to accomodate the entirety of your input
Use keyboard shortcuts for search/replace and faster editing. For example, use Ctrl-F (PC) or Cmd-F (Mac) to search through this box

An algorithm that implements standard metrics for antispoofing evaluation.

Specifically, it returns:

eer: the equal error rate (EER) on the development set.
hter: the half total error rate (HTER) on the test set
far_dev: the false acceptance rate (FAR) on the development set
frr_dev: the false rejection rate (FRR) on the development set
far_test: the false acceptance rate (FAR) on the test set
frr_test: the false rejection rate (FRR) on the test set
number_of_positives_dev: the number of positive trials on the development set
number_of_negatives_dev: the number of negative trials on the development set
number_of_positives_test: the number of positive trials on the test set
number_of_negatives_test: the number of negative trials on the test set
threshold: the threshold at the equal error rate on the development set
roc_dev: the receiver operating characteristic (ROC) curve on the development set
roc_test: the receiver operating characteristic (ROC) curve on the test set

This implementation relies on the 'measure' package from the Bob library. See http://www.idiap.ch/software/bob/docs/releases/last/sphinx/html/measure/ for more details.

Experiments

Attestation:

Privacy:

Status:

			Updated	Name	Databases/Protocols	Analyzers
				smarcel/ivana7c/simple-antispoofing-updated/1/replay2-antispoofing-lbp-histograms-fix	replay/3@grandtest	Kanma/iqm_spoof_eer_analyzer/1

This table shows the number of times this algorithm has been successfully run using the given environment. Note this does not provide sufficient information to evaluate if the algorithm will run when submitted to different conditions.

algorithms Kanma iqm_spoof_eer_analyzer 1

Endpoint Groups 2

Group: test

Group: dev

Results 15

Experiments

algorithms

Kanma

iqm_spoof_eer_analyzer

1