Tools implemented in bob.bio.base

Summary

Base Classes

bob.bio.base.grid.Grid([grid_type, ...])

This class is defining the options that are required to submit parallel jobs to the SGE grid, or jobs to the local queue.

Implementations

Preprocessors

class bob.bio.base.preprocessor.Preprocessor(**kwargs)

This is the base class for all preprocessors. It defines the minimum requirements for all derived proprocessor classes.

Parameters:

kwargs : key=value pairs

A list of keyword arguments to be written in the __str__() function.

__call__(data, annotations) → dara

This is the call function that you have to overwrite in the derived class. The parameters that this function will receive are:

Parameters:

data : object

The original data that needs preprocessing, usually a numpy.ndarray, but might be different.

annotations : {} or None

The annotations (if any) that belongs to the given data; as a dictionary. The type of the annotation depends on your kind of problem.

Returns:

data : object

The preprocessed data, usually a numpy.ndarray, but might be different.

__str__() → info

This function returns all parameters of this class (and its derived class).

Returns:

info : str

A string containing the full information of all parameters of this (and the derived) class.

read_data(data_file) → data

Reads the preprocessed data from file. In this base class implementation, it uses bob.bio.base.load() to do that. If you have different format, please overwrite this function.

Parameters:

data_file : str or bob.io.base.HDF5File

The file open for reading or the name of the file to read from.

Returns:

data : object (usually numpy.ndarray)

The preprocessed data read from file.

read_original_data(original_file_name) → data

Reads the original data (usually something like an image) from file. In this base class implementation, it uses bob.io.base.load() to do that. If you have different format, please overwrite this function.

Parameters:

original_file_name : str

The file name to read the original data from.

Returns:

data : object (usually numpy.ndarray)

The original data read from file.

write_data(data, data_file)

Writes the given preprocessed data to a file with the given name. In this base class implementation, we simply use bob.bio.base.save() for that. If you have a different format (e.g. not images), please overwrite this function.

Parameters:

data : object

The preprocessed data, i.e., what is returned from __call__().

data_file : str or bob.io.base.HDF5File

The file open for writing, or the name of the file to write.

Extractors

class bob.bio.base.extractor.Extractor(requires_training=False, split_training_data_by_client=False, **kwargs)

This is the base class for all feature extractors. It defines the minimum requirements that a derived feature extractor class need to implement.

If your derived class requires training, please register this here.

Parameters

requires_training : bool

Set this flag to True if your feature extractor needs to be trained. In that case, please override the train() and load() methods

split_training_data_by_client : bool

Set this flag to True if your feature extractor requires the training data to be split by clients. Ignored, if requires_training is False

kwargs : key=value pairs

A list of keyword arguments to be written in the __str__() function.

__call__(data) → feature

This function will actually perform the feature extraction. It must be overwritten by derived classes. It takes the (preprocessed) data and returns the features extracted from the data.

Parameters

data : object (usually numpy.ndarray)

The preprocessed data from which features should be extracted.

Returns:

feature : object (usually numpy.ndarray)

The extracted feature.

__str__() → info

This function returns all parameters of this class (and its derived class).

Returns:

info : str

A string containing the full information of all parameters of this (and the derived) class.

load(extractor_file)

Loads the parameters required for feature extraction from the extractor file. This function usually is only useful in combination with the train() function. In this base class implementation, it does nothing.

Parameters:

extractor_file : str

The file to read the extractor from.

read_feature(feature_file)

Reads the extracted feature from file. In this base class implementation, it uses bob.bio.base.load() to do that. If you have different format, please overwrite this function.

Parameters:

feature_file : str or bob.io.base.HDF5File

The file open for reading or the name of the file to read from.

Returns:

feature : object (usually numpy.ndarray)

The feature read from file.

train(training_data, extractor_file)

This function can be overwritten to train the feature extractor. If you do this, please also register the function by calling this base class constructor and enabling the training by requires_training = True.

Parameters:

training_data : [object] or [[object]]

A list of preprocessed data that can be used for training the extractor. Data will be provided in a single list, if split_training_features_by_client = False was specified in the constructor, otherwise the data will be split into lists, each of which contains the data of a single (training-)client.

extractor_file : str

The file to write. This file should be readable with the load() function.

write_feature(feature, feature_file)

Writes the given extracted feature to a file with the given name. In this base class implementation, we simply use bob.bio.base.save() for that. If you have a different format, please overwrite this function.

Parameters:

feature : object

The extracted feature, i.e., what is returned from __call__().

feature_file : str or bob.io.base.HDF5File

The file open for writing, or the name of the file to write.

class bob.bio.base.extractor.Linearize(dtype=None)

Bases: bob.bio.base.extractor.Extractor.Extractor

Extracts features by simply concatenating all elements of the data into one long vector.

If a dtype is specified in the contructor, it is assured that the resulting

If the dtype parameter is given, it specifies the data type that is enforced for the features.

__call__(data) → data

Takes data of arbitrary dimensions and linearizes it into a 1D vector; enforcing the data type, if desired.

Parameters:

data : numpy.ndarray

The preprocessed data to be transformed into one vector.

Returns:

data : 1D numpy.ndarray

The extracted feature vector, of the desired dtype (if specified).

__str__() → info

This function returns all parameters of this class (and its derived class).

Returns:

info : str

A string containing the full information of all parameters of this (and the derived) class.

read_feature(feature_file)

Reads the extracted feature from file. In this base class implementation, it uses bob.bio.base.load() to do that. If you have different format, please overwrite this function.

Parameters:

feature_file : str or bob.io.base.HDF5File

The file open for reading or the name of the file to read from.

Returns:

feature : object (usually numpy.ndarray)

The feature read from file.

write_feature(feature, feature_file)

Writes the given extracted feature to a file with the given name. In this base class implementation, we simply use bob.bio.base.save() for that. If you have a different format, please overwrite this function.

Parameters:

feature : object

The extracted feature, i.e., what is returned from __call__().

feature_file : str or bob.io.base.HDF5File

The file open for writing, or the name of the file to write.

Algorithms

class bob.bio.base.algorithm.Algorithm(performs_projection=False, requires_projector_training=True, split_training_features_by_client=False, use_projected_features_for_enrollment=True, requires_enroller_training=False, multiple_model_scoring='average', multiple_probe_scoring='average', **kwargs)

This is the base class for all biometric recognition algorithms. It defines the minimum requirements for all derived algorithm classes.

Call the constructor in derived class implementations. If your derived algorithm performs feature projection, please register this here. If it needs training for the projector or the enroller, please set this here, too.

Parameters:

performs_projection : bool

Set to True if your derived algorithm performs a projection. Also implement the project() function, and the load_projector() if necessary.

requires_projector_training : bool

Only valid, when performs_projection = True. Set this flag to False, when the projection is applied, but the projector does not need to be trained.

split_training_features_by_client : bool

Only valid, when performs_projection = True and requires_projector_training = True. If set to True, the train_projector() function will receive a double list (a list of lists) of data (sorted by identity). Otherwise, the train_projector() function will receive data in a single list.

use_projected_features_for_enrollment : bool

Only valid, when performs_projection = True. If set to false, the enrollment is performed using the original features, otherwise the features projected using the project() function are used for model enrollment.

requires_enroller_training : bool

Set this flag to True, when the enroller requires specialized training. Which kind of features are used for training depends on the use_projected_features_for_enrollment flag.

multiple_model_scoring : str or None

The way, scores are fused when multiple features are stored in a one model. See bob.bio.base.score_fusion_strategy() for possible values.

multiple_probe_scoring : str or None

The way, scores are fused when multiple probes are available. See bob.bio.base.score_fusion_strategy() for possible values.

kwargs : key=value pairs

A list of keyword arguments to be written in the __str__() function.

__str__() → info

This function returns all parameters of this class (and its derived class).

Returns:

info : str

A string containing the full information of all parameters of this (and the derived) class.

enroll(enroll_features) → model

This function will enroll and return the model from the given list of features. It must be overwritten by derived classes.

Parameters:

enroll_features : [object]

A list of features used for the enrollment of one model.

Returns:

model : object

The model enrolled from the enroll_features. Must be writable with the write_model() function and readable with the read_model() function.

load_enroller(enroller_file)

Loads the parameters required for model enrollment from file. This function usually is only useful in combination with the train_enroller() function. This function is always called after calling load_projector(). In this base class implementation, it does nothing.

Parameters:

enroller_file : str

The file to read the enroller from.

load_projector(projector_file)

Loads the parameters required for feature projection from file. This function usually is useful in combination with the train_projector() function. In this base class implementation, it does nothing.

Please register performs_projection = True in the constructor to enable this function.

Parameters:

projector_file : str

The file to read the projector from.

project(feature) → projected

This function will project the given feature. It must be overwritten by derived classes, as soon as performs_projection = True was set in the constructor. It is assured that the load_projector() was called once before the project function is executed.

Parameters:

feature : object

The feature to be projected.

Returns:

projected : object

The projected features. Must be writable with the write_feature() function and readable with the read_feature() function.

read_feature(feature_file) → feature

Reads the projected feature from file. In this base class implementation, it uses bob.io.base.load() to do that. If you have different format, please overwrite this function.

Please register performs_projection = True in the constructor to enable this function.

Parameters:

feature_file : str or bob.io.base.HDF5File

The file open for reading, or the file name to read from.

Returns:

feature : object

The feature that was read from file.

read_model(model_file) → model

Loads the enrolled model from file. In this base class implementation, it uses bob.io.base.load() to do that.

If you have a different format, please overwrite this function.

Parameters:

model_file : str or bob.io.base.HDF5File

The file open for reading, or the file name to read from.

Returns:

model : object

The model that was read from file.

read_probe(probe_file) → probe

Reads the probe feature from file. By default, the probe feature is identical to the projected feature. Hence, this base class implementation simply calls read_feature().

If your algorithm requires different behavior, please overwrite this function.

Parameters:

probe_file : str or bob.io.base.HDF5File

The file open for reading, or the file name to read from.

Returns:

probe : object

The probe that was read from file.

score(model, probe) → score

This function will compute the score between the given model and probe. It must be overwritten by derived classes.

Parameters:

model : object

The model to compare the probe with. The model was read using the read_model() function.

probe : object

The probe object to compare the model with. The probe was read using the read_probe() function.

Returns:

score : float

A similarity between model and probe. Higher values define higher similarities.

score_for_multiple_models(models, probe) → score

This function computes the score between the given model list and the given probe. In this base class implementation, it computes the scores for each model using the score() method, and fuses the scores using the fusion method specified in the constructor of this class. Usually this function is called from derived class score() functions.

Parameters:

models : [object]

A list of model objects.

probe : object

The probe object to compare the models with.

Returns:

score : float

The fused similarity between the given models and the probe.

score_for_multiple_probes(model, probes) → score

This function computes the score between the given model and the given probe files. In this base class implementation, it computes the scores for each probe file using the score() method, and fuses the scores using the fusion method specified in the constructor of this class.

Parameters:

model : object

A model object to compare the probes with.

probes : [object]

The list of probe object to compare the models with.

Returns:

score : float

The fused similarity between the given model and the probes.

train_enroller(training_features, enroller_file)

This function can be overwritten to train the model enroller. If you do this, please also register the function by calling this base class constructor and enabling the training by require_enroller_training = True.

Parameters:

training_features : [object] or [[object]]

A list of extracted features that can be used for training the projector. Features will be split into lists, each of which contains the features of a single (training-)client.

enroller_file : str

The file to write. This file should be readable with the load_enroller() function.

train_projector(training_features, projector_file)

This function can be overwritten to train the feature projector. If you do this, please also register the function by calling this base class constructor and enabling the training by requires_projector_training = True.

Parameters:

training_features : [object] or [[object]]

A list of extracted features that can be used for training the projector. Features will be provided in a single list, if split_training_features_by_client = False was specified in the constructor, otherwise the features will be split into lists, each of which contains the features of a single (training-)client.

projector_file : str

The file to write. This file should be readable with the load_projector() function.

write_feature(feature, feature_file)

Saves the given projected feature to a file with the given name. In this base class implementation:

• If the given feature has a save attribute, it calls feature.save(bob.io.base.HDF5File(feature_file), 'w'). In this case, the given feature_file might be either a file name or a bob.io.base.HDF5File.
• Otherwise, it uses bob.io.base.save() to do that.

If you have a different format, please overwrite this function.

Please register ‘performs_projection = True’ in the constructor to enable this function.

Parameters:

feature : object

A feature as returned by the project() function, which should be written.

feature_file : str or bob.io.base.HDF5File

The file open for writing, or the file name to write to.

write_model(model, model_file)

Writes the enrolled model to the given file. In this base class implementation:

• If the given model has a ‘save’ attribute, it calls model.save(bob.io.base.HDF5File(model_file), 'w'). In this case, the given model_file might be either a file name or a bob.io.base.HDF5File.
• Otherwise, it uses bob.io.base.save() to do that.

If you have a different format, please overwrite this function.

Parameters:

model : object

A model as returned by the enroll() function, which should be written.

model_file : str or bob.io.base.HDF5File

The file open for writing, or the file name to write to.

class bob.bio.base.algorithm.BIC(comparison_function, maximum_training_pair_count=None, subspace_dimensions=None, uses_dffs=False, read_function=<function load>, write_function=<function save>, **kwargs)

Bases: bob.bio.base.algorithm.Algorithm.Algorithm

Computes the Intrapersonal/Extrapersonal classifier using a generic feature type and feature comparison function.

In this generic implementation, any distance or similarity vector that results as a comparison of two feature vectors can be used. Currently two different versions are implemented: One with [MWP98] and one without (a generalization of [GW09]) subspace projection of the features. The implementation of the BIC training is taken from bob.learn.linear.

Parameters:

comparison_function : function

The function to compare the features in the original feature space. For a given pair of features, this function is supposed to compute a vector of similarity (or distance) values. In the easiest case, it just computes the element-wise difference of the feature vectors, but more difficult functions can be applied, and the function might be specialized for the features you put in.

maximum_training_pair_count : int or None

Limit the number of training image pairs to the given value, i.e., to avoid memory issues.

subspace_dimensions : (int, int) or None

A tuple of sizes of the intrapersonal and extrapersonal subspaces. If given, subspace projection is performed (cf. [MWP98]) and the subspace projection matrices are truncated to the given sizes. If omitted, no subspace projection is performed (cf. [GW09]).

uses_dffs : bool

Only valid, when subspace_dimensions are specified. Use the Distance From Feature Space (DFFS) (cf. [MWP98]) during scoring. Use this flag with care!

read_function : function

A function to read a feature from bob.io.base.HDF5File. This function need to be appropriate to read the type of features that you are using. By default, bob.bio.base.load() is used.

write_function : function

A function to write a feature to bob.io.base.HDF5File. This function is used to write the model and need to be appropriate to write the type of features that you are using. By default, bob.bio.base.save() is used.

kwargs : key=value pairs

A list of keyword arguments directly passed to the Algorithm base class constructor.

__str__() → info

This function returns all parameters of this class (and its derived class).

Returns:

info : str

A string containing the full information of all parameters of this (and the derived) class.

enroll(enroll_features) → model

Enrolls the model by storing all given input features. The features must be writable with the write_function defined in the constructor.

Parameters:

enroll_features : [object]

The list of projected features to enroll the model from.

Returns:

model : [object]

The enrolled model (which is identical to the input features).

load_enroller(enroller_file)

Reads the bob.learn.linear.BICMachine from file.

The bob.learn.linear.BICMachine.use_DFFS will be overwritten by the use_dffs value specified in this class’ constructor.

Parameters:

enroller_file : str

An existing file, from which the bob.learn.linear.BICMachine will be read.

read_model(model_file) → model

Reads all features of the model from the given HDF5 file.

To read the features, the read_function specified in the constructor is employed.

Parameters:

model_file : str or bob.io.base.HDF5File

The file (open for reading) or the name of an existing file to read from.

Returns:

model : [object]

The read model, which is a list of features.

read_probe(probe_file) → probe

Reads the probe feature from the given HDF5 file.

To read the feature, the read_function specified in the constructor is employed.

Parameters:

probe_file : str or bob.io.base.HDF5File

The file (open for reading) or the name of an existing file to read from.

Returns:

probe : object

The read probe, which is a feature.

score(model, probe) → float

Computes the BIC score between the model and the probe. First, the comparison_function is used to create the comparison vectors between all model features and the probe feature. Then, a BIC score is computed for each comparison vector, and the BIC scores are fused using the model_fusion_function() defined in the Algorithm base class.

Parameters:

model : [object]

The model storing all model features.

probe : object

The probe feature.

Returns:

score : float

A fused BIC similarity value between model and probe.

score_for_multiple_models(models, probe) → score

This function computes the score between the given model list and the given probe. In this base class implementation, it computes the scores for each model using the score() method, and fuses the scores using the fusion method specified in the constructor of this class. Usually this function is called from derived class score() functions.

Parameters:

models : [object]

A list of model objects.

probe : object

The probe object to compare the models with.

Returns:

score : float

The fused similarity between the given models and the probe.

score_for_multiple_probes(model, probes) → score

This function computes the score between the given model and the given probe files. In this base class implementation, it computes the scores for each probe file using the score() method, and fuses the scores using the fusion method specified in the constructor of this class.

Parameters:

model : object

A model object to compare the probes with.

probes : [object]

The list of probe object to compare the models with.

Returns:

score : float

The fused similarity between the given model and the probes.

train_enroller(train_features, enroller_file)

Trains the BIC by computing intra-personal and extra-personal subspaces.

First, two lists of pairs are computed, which contain intra-personal and extra-personal feature pairs, respectively. Afterward, the comparison vectors are computed using the comparison_function specified in the constructor. Finally, the bob.learn.linear.BICTrainer is used to train a bob.learn.linear.BICMachine.

Parameters:

train_features : [[object]]

A list of lists of feature vectors, which are used to train the BIC. Each sub-list contains the features of one client.

enroller_file : str

A writable file, into which the resulting bob.learn.linear.BICMachine will be written.

write_model(model, model_file)

Writes all features of the model into one HDF5 file.

To write the features, the write_function specified in the constructor is employed.

Parameters:

model : [object]

The model to write, which is a list of features.

model_file : str or bob.io.base.HDF5File

The file (open for writing) or a file name to write into.

class bob.bio.base.algorithm.LDA(lda_subspace_dimension=None, pca_subspace_dimension=None, distance_function=<function euclidean>, is_distance_function=True, uses_variances=False, **kwargs)

Bases: bob.bio.base.algorithm.Algorithm.Algorithm

Computes a linear discriminant analysis (LDA) on the given data, possibly after computing a principal component analysis (PCA).

This algorithm computes a LDA projection (bob.learn.linear.FisherLDATrainer) on the given training features, projects the features to Fisher space and computes the distance of two projected features in Fisher space. For example, the Fisher faces algorithm as proposed by [ZKC+98] can be run with this class.

Additionally, a PCA projection matrix can be computed beforehand, to reduce the dimensionality of the input vectors. In that case, the finally stored projection matrix is the combination of the PCA and LDA projection.

Parameters:

lda_subspace_dimension : int or None

If specified, the LDA subspace will be truncated to the given number of dimensions. By default (None) it is limited to the number of classes in the training set - 1.

pca_subspace_dimentsion : int or float or None

If specified, a combined PCA + LDA projection matrix will be computed. If specified as int, defines the number of eigenvectors used in the PCA projection matrix. If specified as float (between 0 and 1), the number of eigenvectors is calculated such that the given percentage of variance is kept.

distance_function : function

A function taking two parameters and returns a float. If uses_variances is set to True, the function is provided with a third parameter, which is the vector of variances (aka. eigenvalues).

is_distance_function : bool

Set this flag to False if the given distance_function computes a similarity value (i.e., higher values are better)

use_variances : bool

If set to True, the distance_function is provided with a third argument, which is the vector of variances (aka. eigenvalues).

kwargs : key=value pairs

A list of keyword arguments directly passed to the Algorithm base class constructor.

__str__() → info

This function returns all parameters of this class (and its derived class).

Returns:

info : str

A string containing the full information of all parameters of this (and the derived) class.

enroll(enroll_features) → model

Enrolls the model by storing all given input vectors.

Parameters:

enroll_features : [1D numpy.ndarray]

The list of projected features to enroll the model from.

Returns:

model : 2D numpy.ndarray

The enrolled model.

load_projector(projector_file)

Reads the projection matrix and the eigenvalues from file.

Parameters:

projector_file : str

An existing file, from which the PCA or PCA+LDA projection matrix and the eigenvalues are read.

project(feature) → projected

Projects the given feature into Fisher space.

Parameters:

feature : 1D numpy.ndarray

The 1D feature to be projected.

Returns:

projected : 1D numpy.ndarray

The feature projected into Fisher space.

read_feature(feature_file) → feature

Reads the projected feature from file. In this base class implementation, it uses bob.io.base.load() to do that. If you have different format, please overwrite this function.

Please register performs_projection = True in the constructor to enable this function.

Parameters:

feature_file : str or bob.io.base.HDF5File

The file open for reading, or the file name to read from.

Returns:

feature : object

The feature that was read from file.

read_model(model_file) → model

Loads the enrolled model from file. In this base class implementation, it uses bob.io.base.load() to do that.

If you have a different format, please overwrite this function.

Parameters:

model_file : str or bob.io.base.HDF5File

The file open for reading, or the file name to read from.

Returns:

model : object

The model that was read from file.

read_probe(probe_file) → probe

Reads the probe feature from file. By default, the probe feature is identical to the projected feature. Hence, this base class implementation simply calls read_feature().

If your algorithm requires different behavior, please overwrite this function.

Parameters:

probe_file : str or bob.io.base.HDF5File

The file open for reading, or the file name to read from.

Returns:

probe : object

The probe that was read from file.

score(model, probe) → float

Computes the distance of the model to the probe using the distance function specified in the constructor.

Parameters:

model : 2D numpy.ndarray

The model storing all enrollment features.

probe : 1D numpy.ndarray

The probe feature vector in Fisher space.

Returns:

score : float

A similarity value between model and probe

score_for_multiple_models(models, probe) → score

This function computes the score between the given model list and the given probe. In this base class implementation, it computes the scores for each model using the score() method, and fuses the scores using the fusion method specified in the constructor of this class. Usually this function is called from derived class score() functions.

Parameters:

models : [object]

A list of model objects.

probe : object

The probe object to compare the models with.

Returns:

score : float

The fused similarity between the given models and the probe.

score_for_multiple_probes(model, probes) → score

This function computes the score between the given model and the given probe files. In this base class implementation, it computes the scores for each probe file using the score() method, and fuses the scores using the fusion method specified in the constructor of this class.

Parameters:

model : object

A model object to compare the probes with.

probes : [object]

The list of probe object to compare the models with.

Returns:

score : float

The fused similarity between the given model and the probes.

train_projector(training_features, projector_file)

Generates the LDA or PCA+LDA projection matrix from the given features (that are sorted by identity).

Parameters:

training_features : [[1D numpy.ndarray]]

A list of lists of 1D training arrays (vectors) to train the LDA projection matrix with. Each sub-list contains the features of one client.

projector_file : str

A writable file, into which the LDA or PCA+LDA projection matrix (as a bob.learn.linear.Machine) and the eigenvalues will be written.

write_feature(feature, feature_file)

Saves the given projected feature to a file with the given name. In this base class implementation:

• If the given feature has a save attribute, it calls feature.save(bob.io.base.HDF5File(feature_file), 'w'). In this case, the given feature_file might be either a file name or a bob.io.base.HDF5File.
• Otherwise, it uses bob.io.base.save() to do that.

If you have a different format, please overwrite this function.

Please register ‘performs_projection = True’ in the constructor to enable this function.

Parameters:

feature : object

A feature as returned by the project() function, which should be written.

feature_file : str or bob.io.base.HDF5File

The file open for writing, or the file name to write to.

write_model(model, model_file)

Writes the enrolled model to the given file. In this base class implementation:

• If the given model has a ‘save’ attribute, it calls model.save(bob.io.base.HDF5File(model_file), 'w'). In this case, the given model_file might be either a file name or a bob.io.base.HDF5File.
• Otherwise, it uses bob.io.base.save() to do that.

If you have a different format, please overwrite this function.

Parameters:

model : object

A model as returned by the enroll() function, which should be written.

model_file : str or bob.io.base.HDF5File

The file open for writing, or the file name to write to.

class bob.bio.base.algorithm.PCA(subspace_dimension, distance_function=<function euclidean>, is_distance_function=True, uses_variances=False, **kwargs)

Bases: bob.bio.base.algorithm.Algorithm.Algorithm

Performs a principal component analysis (PCA) on the given data.

This algorithm computes a PCA projection (bob.learn.linear.PCATrainer) on the given training features, projects the features to eigenspace and computes the distance of two projected features in eigenspace. For example, the eigenface algorithm as proposed by [TP91] can be run with this class.

Parameters:

subspace_dimension : int or float

If specified as int, defines the number of eigenvectors used in the PCA projection matrix. If specified as float (between 0 and 1), the number of eigenvectors is calculated such that the given percentage of variance is kept.

distance_function : function

A function taking two parameters and returns a float. If uses_variances is set to True, the function is provided with a third parameter, which is the vector of variances (aka. eigenvalues).

is_distance_function : bool

Set this flag to False if the given distance_function computes a similarity value (i.e., higher values are better)

use_variances : bool

If set to True, the distance_function is provided with a third argument, which is the vector of variances (aka. eigenvalues).

kwargs : key=value pairs

A list of keyword arguments directly passed to the Algorithm base class constructor.

__str__() → info

This function returns all parameters of this class (and its derived class).

Returns:

info : str

A string containing the full information of all parameters of this (and the derived) class.

enroll(enroll_features) → model

Enrolls the model by storing all given input vectors.

Parameters:

enroll_features : [1D numpy.ndarray]

The list of projected features to enroll the model from.

Returns:

model : 2D numpy.ndarray

The enrolled model.

load_projector(projector_file)

Reads the PCA projection matrix and the eigenvalues from file.

Parameters:

projector_file : str

An existing file, from which the PCA projection matrix and the eigenvalues are read.

project(feature) → projected

Projects the given feature into eigenspace.

Parameters:

feature : 1D numpy.ndarray

The 1D feature to be projected.

Returns:

projected : 1D numpy.ndarray

The feature projected into eigenspace.

read_feature(feature_file) → feature

Reads the projected feature from file. In this base class implementation, it uses bob.io.base.load() to do that. If you have different format, please overwrite this function.

Please register performs_projection = True in the constructor to enable this function.

Parameters:

feature_file : str or bob.io.base.HDF5File

The file open for reading, or the file name to read from.

Returns:

feature : object

The feature that was read from file.

read_model(model_file) → model

Loads the enrolled model from file. In this base class implementation, it uses bob.io.base.load() to do that.

If you have a different format, please overwrite this function.

Parameters:

model_file : str or bob.io.base.HDF5File

The file open for reading, or the file name to read from.

Returns:

model : object

The model that was read from file.

read_probe(probe_file) → probe

Reads the probe feature from file. By default, the probe feature is identical to the projected feature. Hence, this base class implementation simply calls read_feature().

If your algorithm requires different behavior, please overwrite this function.

Parameters:

probe_file : str or bob.io.base.HDF5File

The file open for reading, or the file name to read from.

Returns:

probe : object

The probe that was read from file.

score(model, probe) → float

Computes the distance of the model to the probe using the distance function specified in the constructor.

Parameters:

model : 2D numpy.ndarray

The model storing all enrollment features.

probe : 1D numpy.ndarray

The probe feature vector in eigenspace.

Returns:

score : float

A similarity value between model and probe

score_for_multiple_models(models, probe) → score

This function computes the score between the given model list and the given probe. In this base class implementation, it computes the scores for each model using the score() method, and fuses the scores using the fusion method specified in the constructor of this class. Usually this function is called from derived class score() functions.

Parameters:

models : [object]

A list of model objects.

probe : object

The probe object to compare the models with.

Returns:

score : float

The fused similarity between the given models and the probe.

score_for_multiple_probes(model, probes) → score

This function computes the score between the given model and the given probe files. In this base class implementation, it computes the scores for each probe file using the score() method, and fuses the scores using the fusion method specified in the constructor of this class.

Parameters:

model : object

A model object to compare the probes with.

probes : [object]

The list of probe object to compare the models with.

Returns:

score : float

The fused similarity between the given model and the probes.

train_projector(training_features, projector_file)

Generates the PCA covariance matrix and writes it into the given projector_file.

Parameters:

training_features : [1D numpy.ndarray]

A list of 1D training arrays (vectors) to train the PCA projection matrix with.

projector_file : str

A writable file, into which the PCA projection matrix (as a bob.learn.linear.Machine) and the eigenvalues will be written.

write_feature(feature, feature_file)

Saves the given projected feature to a file with the given name. In this base class implementation:

• If the given feature has a save attribute, it calls feature.save(bob.io.base.HDF5File(feature_file), 'w'). In this case, the given feature_file might be either a file name or a bob.io.base.HDF5File.
• Otherwise, it uses bob.io.base.save() to do that.

If you have a different format, please overwrite this function.

Please register ‘performs_projection = True’ in the constructor to enable this function.

Parameters:

feature : object

A feature as returned by the project() function, which should be written.

feature_file : str or bob.io.base.HDF5File

The file open for writing, or the file name to write to.

write_model(model, model_file)

Writes the enrolled model to the given file. In this base class implementation:

• If the given model has a ‘save’ attribute, it calls model.save(bob.io.base.HDF5File(model_file), 'w'). In this case, the given model_file might be either a file name or a bob.io.base.HDF5File.
• Otherwise, it uses bob.io.base.save() to do that.

If you have a different format, please overwrite this function.

Parameters:

model : object

A model as returned by the enroll() function, which should be written.

model_file : str or bob.io.base.HDF5File

The file open for writing, or the file name to write to.

class bob.bio.base.algorithm.PLDA(subspace_dimension_of_f, subspace_dimension_of_g, subspace_dimension_pca=None, plda_training_iterations=200, INIT_SEED=5489, INIT_F_METHOD='BETWEEN_SCATTER', INIT_G_METHOD='WITHIN_SCATTER', INIT_S_METHOD='VARIANCE_DATA', multiple_probe_scoring='joint_likelihood')

Bases: bob.bio.base.algorithm.Algorithm.Algorithm

Tool chain for computing PLDA (over PCA-dimensionality reduced) features

Todo

Add more documentation for the PLDA constructor, i.e., by explaining the parameters

Initializes the local (PCA-)PLDA tool chain with the given file selector object

__str__() → info

This function returns all parameters of this class (and its derived class).

Returns:

info : str

A string containing the full information of all parameters of this (and the derived) class.

enroll(enroll_features)

Enrolls the model by computing an average of the given input vectors

load_enroller(projector_file)

Reads the PCA projection matrix and the PLDA model from file

read_model(model_file)

Reads the model, which in this case is a PLDA-Machine

read_probe(probe_file)

Reads the probe using bob.bio.base.load().

score(model, probe)

Computes the PLDA score for the given model and probe

score_for_multiple_models(models, probe) → score

This function computes the score between the given model list and the given probe. In this base class implementation, it computes the scores for each model using the score() method, and fuses the scores using the fusion method specified in the constructor of this class. Usually this function is called from derived class score() functions.

Parameters:

models : [object]

A list of model objects.

probe : object

The probe object to compare the models with.

Returns:

score : float

The fused similarity between the given models and the probe.

score_for_multiple_probes(model, probes)

This function computes the score between the given model and several given probe files. In this base class implementation, it computes the scores for each probe file using the ‘score’ method, and fuses the scores using the fusion method specified in the constructor of this class.

train_enroller(training_features, projector_file)

Generates the PLDA base model from a list of arrays (one per identity), and a set of training parameters. If PCA is requested, it is trained on the same data. Both the trained PLDABase and the PCA machine are written.

write_model(model, model_file)

Writes the enrolled model to the given file. In this base class implementation:

• If the given model has a ‘save’ attribute, it calls model.save(bob.io.base.HDF5File(model_file), 'w'). In this case, the given model_file might be either a file name or a bob.io.base.HDF5File.
• Otherwise, it uses bob.io.base.save() to do that.

If you have a different format, please overwrite this function.

Parameters:

model : object

A model as returned by the enroll() function, which should be written.

model_file : str or bob.io.base.HDF5File

The file open for writing, or the file name to write to.

Databases

class bob.bio.base.database.Database(name, original_directory=None, original_extension=None, annotation_directory=None, annotation_extension='.pos', annotation_type=None, protocol='Default', training_depends_on_protocol=False, models_depend_on_protocol=False, **kwargs)

This class represents the basic API for database access. Please use this class as a base class for your database access classes. Do not forget to call the constructor of this base class in your derived class.

Parameters:

name : str

A unique name for the database.

original_directory : str

The directory where the original data of the database are stored.

original_extension : str

The file name extension of the original data.

annotation_directory : str

The directory where the image annotations of the database are stored, if any.

annotation_extension : str

The file name extension of the annotation files.

annotation_type : str

The type of the annotation file to read, see bob.db.verification.utils.read_annotation_file() for accepted formats.

protocol : str or None

The name of the protocol that defines the default experimental setup for this database.

Todo

Check if the None protocol is supported.

training_depends_on_protocol : bool

Specifies, if the training set used for training the extractor and the projector depend on the protocol. This flag is used to avoid re-computation of data when running on the different protocols of the same database.

models_depend_on_protocol : bool

Specifies, if the models depend on the protocol. This flag is used to avoid re-computation of models when running on the different protocols of the same database.

kwargs

Ignored extra arguments.

__str__() → info

This function returns all parameters of this class.

Returns:

info : str

A string containing the full information of all parameters of this class.

all_files(groups=None) → files

Returns all files of the database. This function needs to be implemented in derived class implementations.

Parameters:

groups : some of ('world', 'dev', 'eval') or None

The groups to get the data for. If None, data for all groups is returned.

Returns:

files : [File]

The sorted and unique list of all files of the database.

annotations(file) → annots

Returns the annotations for the given File object, if available. It uses bob.db.verification.utils.read_annotation_file() to load the annotations.

Parameters:

file : File

The file for which annotations should be returned.

Returns:

annots : dict or None

The annotations for the file, if available.

arrange_by_client(files) → files_by_client

Arranges the given list of files by client id. This function returns a list of lists of File’s.

Parameters:

files : File

A list of files that should be split up by File.client_id.

Returns:

files_by_client : [[File]]

The list of lists of files, where each sub-list groups the files with the same File.client_id

client_id_from_model_id(model_id, group = 'dev') → client_id

In some databases, each client can contain several models. Hence, client and model ids differ. This function converts the given model id into its according the client id. This function needs to be implemented in derived class implementations.

Parameters:

model_id : int or str

A unique ID that identifies the model for the client.

group : one of ('dev', 'eval')

The group to get the client ids for.

Returns:

client_id : [int] or [str]

A unique ID that identifies the client, to which the model belongs.

enroll_files(model_id, group = 'dev') → files

Returns a list of File objects that should be used to enroll the model with the given model id from the given group. This function needs to be implemented in derived class implementations.

Parameters:

model_id : int or str

A unique ID that identifies the model.

group : one of ('dev', 'eval')

The group to get the enrollment files for.

Returns:

files : [File]

The list of files used for to enroll the model with the given model id.

file_names(files, directory, extension) → paths

Returns the full path of the given File objects.

Parameters:

files : [File]

The list of file object to retrieve the file names for.

directory : str

The base directory, where the files can be found.

extension : str

The file name extension to add to all files.

Returns:

paths : [str] or [[str]]

The paths extracted for the files, in the same order. If this database provides file sets, a list of lists of file names is returned, one sub-list for each file set.

model_ids(group = 'dev') → ids

Returns a list of model ids for the given group. This function needs to be implemented in derived class implementations.

Parameters:

group : one of ('dev', 'eval')

The group to get the model ids for.

Returns:

ids : [int] or [str]

The list of (unique) model ids for the given group.

original_file_names(files) → paths

Returns the full path of the original data of the given File objects.

Parameters:

files : [File]

The list of file object to retrieve the original data file names for.

Returns:

paths : [str] or [[str]]

The paths extracted for the files, in the same order. If this database provides file sets, a list of lists of file names is returned, one sub-list for each file set.

probe_file_sets(model_id = None, group = 'dev') → files

Returns a list of probe FileSet objects. If a model_id is specified, only the probe files that should be compared with the given model id are returned (for most databases, these are all probe files of the given group). Otherwise, all probe files of the given group are returned. This function needs to be implemented in derived class implementations, if the uses_probe_file_sets() returns True.

Parameters:

model_id : int or str or None

A unique ID that identifies the model.

group : one of ('dev', 'eval')

The group to get the enrollment files for.

Returns:

files : [FileSet]

The list of file sets used to probe the model with the given model id.

probe_files(model_id = None, group = 'dev') → files

Returns a list of probe File objects. If a model_id is specified, only the probe files that should be compared with the given model id are returned (for most databases, these are all probe files of the given group). Otherwise, all probe files of the given group are returned. This function needs to be implemented in derived class implementations.

Parameters:

model_id : int or str or None

A unique ID that identifies the model.

group : one of ('dev', 'eval')

The group to get the enrollment files for.

Returns:

files : [File]

The list of files used for to probe the model with the given model id.

sort(files) → sorted

Returns a sorted version of the given list of File’s (or other structures that define an ‘id’ data member). The files will be sorted according to their id, and duplicate entries will be removed.

Parameters:

files : [File]

The list of files to be uniquified and sorted.

Returns:

sorted : [File]

The sorted list of files, with duplicate File.ids being removed.

training_files(step = None, arrange_by_client = False) → files

Returns all training File objects for the given step, and arranges them by client, if desired. This function needs to be implemented in derived class implementations.

Parameters:

step : one of ('train_extractor', 'train_projector', 'train_enroller') or None

The step for which the training data should be returned. Might be ignored in derived class implementations.

arrange_by_client : bool

Should the training files be arranged by client?

Note

You can use arrange_by_client() in derived class implementations to arrange the files.

Returns:

files : [File] or [[File]]

The (arranged) list of files used for the training of the given step.

uses_probe_file_sets()

Defines if, for the current protocol, the database uses several probe files to generate a score. By default, False is returned. Overwrite the default if you need different behavior.

class bob.bio.base.database.DatabaseBob(database, all_files_options={}, extractor_training_options={}, projector_training_options={}, enroller_training_options={}, check_original_files_for_existence=False, **kwargs)

Bases: bob.bio.base.database.Database.Database

This class can be used whenever you have a database that follows the Bob verification database interface, which is defined in bob.db.verification.utils.Database

Parameters:

database : derivative of bob.db.verification.utils.Database

The database instance (such as a bob.db.atnt.Database) that provides the actual interface, see Databases that Implement the Database Interface for a list.

all_files_options : dict

Dictionary of options passed to the bob.db.verification.utils.Database.objects() database query when retrieving all data.

extractor_training_options : dict

Dictionary of options passed to the bob.db.verification.utils.Database.objects() database query used to retrieve the files for the extractor training.

projector_training_options : dict

Dictionary of options passed to the bob.db.verification.utils.Database.objects() database query used to retrieve the files for the projector training.

enroller_training_options : dict

Dictionary of options passed to the bob.db.verification.utils.Database.objects() database query used to retrieve the files for the enroller training.

check_original_files_for_existence : bool

Enables to test for the original data files when querying the database.

kwargs : key=value pairs

The arguments of the Database base class constructor.

Note

Usually, the name, protocol, training_depends_on_protocol and models_depend_on_protocol keyword parameters of the base class constructor need to be specified.

__str__() → info

This function returns all parameters of this class (and its derived class).

Returns:

info : str

A string containing the full information of all parameters of this (and the derived) class.

all_files(groups=None) → files

Returns all files of the database, respecting the current protocol. The files can be limited using the all_files_options in the constructor.

Parameters:

groups : some of ('world', 'dev', 'eval') or None

The groups to get the data for. If None, data for all groups is returned.

Returns:

files : [bob.db.verification.utils.File]

The sorted and unique list of all files of the database.

annotations(file) → annots

Returns the annotations for the given File object, if available.

Parameters:

file : bob.db.verification.utils.File

The file for which annotations should be returned.

Returns:

annots : dict or None

The annotations for the file, if available.

arrange_by_client(files) → files_by_client

Arranges the given list of files by client id. This function returns a list of lists of File’s.

Parameters:

files : File

A list of files that should be split up by File.client_id.

Returns:

files_by_client : [[File]]

The list of lists of files, where each sub-list groups the files with the same File.client_id

client_id_from_model_id(model_id, group = 'dev') → client_id

Uses bob.db.verification.utils.Database.get_client_id_from_model_id() to retrieve the client id for the given model id.

Parameters:

model_id : int or str

A unique ID that identifies the model for the client.

group : one of ('dev', 'eval')

The group to get the client ids for.

Returns:

client_id : [int] or [str]

A unique ID that identifies the client, to which the model belongs.

enroll_files(model_id, group = 'dev') → files

Returns a list of File objects that should be used to enroll the model with the given model id from the given group, respecting the current protocol.

Parameters:

model_id : int or str

A unique ID that identifies the model.

group : one of ('dev', 'eval')

The group to get the enrollment files for.

Returns:

files : [bob.db.verification.utils.File]

The list of files used for to enroll the model with the given model id.

file_names(files, directory, extension) → paths

Returns the full path of the given File objects.

Parameters:

files : [File]

The list of file object to retrieve the file names for.

directory : str

The base directory, where the files can be found.

extension : str

The file name extension to add to all files.

Returns:

paths : [str] or [[str]]

The paths extracted for the files, in the same order. If this database provides file sets, a list of lists of file names is returned, one sub-list for each file set.

model_ids(group = 'dev') → ids

Returns a list of model ids for the given group, respecting the current protocol.

Parameters:

group : one of ('dev', 'eval')

The group to get the model ids for.

Returns:

ids : [int] or [str]

The list of (unique) model ids for the given group.

original_file_names(files) → paths

Returns the full path of the original data of the given File objects, as returned by bob.db.verification.utils.Database.original_file_names().

Parameters:

files : [bob.db.verification.utils.File]

The list of file object to retrieve the original data file names for.

Returns:

paths : [str] or [[str]]

The paths extracted for the files, in the same order. If this database provides file sets, a list of lists of file names is returned, one sub-list for each file set.

probe_file_sets(model_id = None, group = 'dev') → files

Returns a list of probe FileSet objects, respecting the current protocol. If a model_id is specified, only the probe files that should be compared with the given model id are returned (for most databases, these are all probe files of the given group). Otherwise, all probe files of the given group are returned.

Parameters:

model_id : int or str or None

A unique ID that identifies the model.

group : one of ('dev', 'eval')

The group to get the enrollment files for.

Returns:

files : [FileSet] or something similar

The list of file sets used to probe the model with the given model id.

probe_files(model_id = None, group = 'dev') → files

Returns a list of probe File objects, respecting the current protocol. If a model_id is specified, only the probe files that should be compared with the given model id are returned (for most databases, these are all probe files of the given group). Otherwise, all probe files of the given group are returned.

Parameters:

model_id : int or str or None

A unique ID that identifies the model.

group : one of ('dev', 'eval')

The group to get the enrollment files for.

Returns:

files : [bob.db.verification.utils.File]

The list of files used for to probe the model with the given model id.

replace_directories(replacements=None)

This helper function replaces the original_directory and the annotation_directory of the database with the directories read from the given replacement file.

This function is provided for convenience, so that the database configuration files do not need to be modified. Instead, this function uses the given dictionary of replacements to change the original directory and the original extension (if given).

The given replacements can be of type dict, including all replacements, or a file name (as a str), in which case the file is read. The structure of the file should be:

# Comments starting with # and empty lines are ignored

[YOUR_..._DATA_DIRECTORY] = /path/to/your/data
[YOUR_..._ANNOTATION_DIRECTORY] = /path/to/your/annotations

If no annotation files are available (e.g. when they are stored inside the database), the annotation directory can be left out.

Parameters:

replacements : dict or str

A dictionary with replacements, or a name of a file to read the dictionary from. If the file name does not exist, no directories are replaced.

sort(files) → sorted

Returns a sorted version of the given list of File’s (or other structures that define an ‘id’ data member). The files will be sorted according to their id, and duplicate entries will be removed.

Parameters:

files : [File]

The list of files to be uniquified and sorted.

Returns:

sorted : [File]

The sorted list of files, with duplicate File.ids being removed.

test_files(groups = ['dev']) → files

Returns all test files (i.e., files used for enrollment and probing) for the given groups, respecting the current protocol. The files for the steps can be limited using the all_files_options defined in the constructor.

Parameters:

groups : some of ('dev', 'eval')

The groups to get the data for.

Returns:

files : [bob.db.verification.utils.File]

The sorted and unique list of test files of the database.

training_files(step = None, arrange_by_client = False) → files

Returns all training files for the given step, and arranges them by client, if desired, respecting the current protocol. The files for the steps can be limited using the ..._training_options defined in the constructor.

Parameters:

step : one of ('train_extractor', 'train_projector', 'train_enroller') or None

The step for which the training data should be returned.

arrange_by_client : bool

Should the training files be arranged by client? If set to True, training files will be returned in [[bob.db.verification.utils.File]], where each sub-list contains the files of a single client. Otherwise, all files will be stored in a simple [bob.db.verification.utils.File].

Returns:

files : [bob.db.verification.utils.File] or [[bob.db.verification.utils.File]]

The (arranged) list of files used for the training of the given step.

uses_probe_file_sets()

Defines if, for the current protocol, the database uses several probe files to generate a score.

class bob.bio.base.database.DatabaseBobZT(database, z_probe_options={}, **kwargs)

Bases: bob.bio.base.database.DatabaseBob.DatabaseBob, bob.bio.base.database.Database.DatabaseZT

This class can be used whenever you have a database that follows the Bob ZT-norm verification database interface, which is defined in bob.db.verification.utils.ZTDatabase.

Parameters:

database : derivative of bob.db.verification.utils.ZTDatabase

The database instance (such as a bob.db.mobio.Database) that provides the actual interface, see Databases that Implement the Database Interface for a list.

z_probe_options : dict

Dictionary of options passed to the bob.db.verification.utils.ZTDatabase.z_probe_files() database query when retrieving files for Z-probing.

kwargs : key=value pairs

The arguments of the DatabaseBob base class constructor.

Note

Usually, the name, protocol, training_depends_on_protocol and models_depend_on_protocol keyword parameters of the Database base class constructor need to be specified.

__str__() → info

This function returns all parameters of this class (and its derived class).

Returns:

info : str

A string containing the full information of all parameters of this (and the derived) class.

all_files(groups=None) → files

Returns all files of the database, including those for ZT norm, respecting the current protocol. The files can be limited using the all_files_options and the the z_probe_options in the constructor.

Parameters:

groups : some of ('world', 'dev', 'eval') or None

The groups to get the data for. If None, data for all groups is returned.

Returns:

files : [bob.db.verification.utils.File]

The sorted and unique list of all files of the database.

annotations(file) → annots

Returns the annotations for the given File object, if available.

Parameters:

file : bob.db.verification.utils.File

The file for which annotations should be returned.

Returns:

annots : dict or None

The annotations for the file, if available.

arrange_by_client(files) → files_by_client

Arranges the given list of files by client id. This function returns a list of lists of File’s.

Parameters:

files : File

A list of files that should be split up by File.client_id.

Returns:

files_by_client : [[File]]

The list of lists of files, where each sub-list groups the files with the same File.client_id

client_id_from_model_id(model_id, group = 'dev') → client_id

Uses bob.db.verification.utils.Database.get_client_id_from_model_id() to retrieve the client id for the given model id.

Parameters:

model_id : int or str

A unique ID that identifies the model for the client.

group : one of ('dev', 'eval')

The group to get the client ids for.

Returns:

client_id : [int] or [str]

A unique ID that identifies the client, to which the model belongs.

client_id_from_t_model_id(t_model_id, group = 'dev') → client_id

Returns the client id for the given T-Norm model id. In this base class implementation, we just use the client_id_from_model_id() function. Overload this function if you need another behavior.

Parameters:

t_model_id : int or str

A unique ID that identifies the T-Norm model.

group : one of ('dev', 'eval')

The group to get the client ids for.

Returns:

client_id : [int] or [str]

A unique ID that identifies the client, to which the T-Norm model belongs.

enroll_files(model_id, group = 'dev') → files

Returns a list of File objects that should be used to enroll the model with the given model id from the given group, respecting the current protocol.

Parameters:

model_id : int or str

A unique ID that identifies the model.

group : one of ('dev', 'eval')

The group to get the enrollment files for.

Returns:

files : [bob.db.verification.utils.File]

The list of files used for to enroll the model with the given model id.

file_names(files, directory, extension) → paths

Returns the full path of the given File objects.

Parameters:

files : [File]

The list of file object to retrieve the file names for.

directory : str

The base directory, where the files can be found.

extension : str

The file name extension to add to all files.

Returns:

paths : [str] or [[str]]

The paths extracted for the files, in the same order. If this database provides file sets, a list of lists of file names is returned, one sub-list for each file set.

model_ids(group = 'dev') → ids

Returns a list of model ids for the given group, respecting the current protocol.

Parameters:

group : one of ('dev', 'eval')

The group to get the model ids for.

Returns:

ids : [int] or [str]

The list of (unique) model ids for the given group.

original_file_names(files) → paths

Returns the full path of the original data of the given File objects, as returned by bob.db.verification.utils.Database.original_file_names().

Parameters:

files : [bob.db.verification.utils.File]

The list of file object to retrieve the original data file names for.

Returns:

paths : [str] or [[str]]

The paths extracted for the files, in the same order. If this database provides file sets, a list of lists of file names is returned, one sub-list for each file set.

probe_file_sets(model_id = None, group = 'dev') → files

Returns a list of probe FileSet objects, respecting the current protocol. If a model_id is specified, only the probe files that should be compared with the given model id are returned (for most databases, these are all probe files of the given group). Otherwise, all probe files of the given group are returned.

Parameters:

model_id : int or str or None

A unique ID that identifies the model.

group : one of ('dev', 'eval')

The group to get the enrollment files for.

Returns:

files : [FileSet] or something similar

The list of file sets used to probe the model with the given model id.

probe_files(model_id = None, group = 'dev') → files

Returns a list of probe File objects, respecting the current protocol. If a model_id is specified, only the probe files that should be compared with the given model id are returned (for most databases, these are all probe files of the given group). Otherwise, all probe files of the given group are returned.

Parameters:

model_id : int or str or None

A unique ID that identifies the model.

group : one of ('dev', 'eval')

The group to get the enrollment files for.

Returns:

files : [bob.db.verification.utils.File]

The list of files used for to probe the model with the given model id.

replace_directories(replacements=None)

This helper function replaces the original_directory and the annotation_directory of the database with the directories read from the given replacement file.

This function is provided for convenience, so that the database configuration files do not need to be modified. Instead, this function uses the given dictionary of replacements to change the original directory and the original extension (if given).

The given replacements can be of type dict, including all replacements, or a file name (as a str), in which case the file is read. The structure of the file should be:

# Comments starting with # and empty lines are ignored

[YOUR_..._DATA_DIRECTORY] = /path/to/your/data
[YOUR_..._ANNOTATION_DIRECTORY] = /path/to/your/annotations

If no annotation files are available (e.g. when they are stored inside the database), the annotation directory can be left out.

Parameters:

replacements : dict or str

A dictionary with replacements, or a name of a file to read the dictionary from. If the file name does not exist, no directories are replaced.

sort(files) → sorted

Returns a sorted version of the given list of File’s (or other structures that define an ‘id’ data member). The files will be sorted according to their id, and duplicate entries will be removed.

Parameters:

files : [File]

The list of files to be uniquified and sorted.

Returns:

sorted : [File]

The sorted list of files, with duplicate File.ids being removed.

t_enroll_files(t_model_id, group = 'dev') → files

Returns a list of File objects that should be used to enroll the T-Norm model with the given model id from the given group, respecting the current protocol.

Parameters:

t_model_id : int or str

A unique ID that identifies the model.

group : one of ('dev', 'eval')

The group to get the enrollment files for.

Returns:

files : [bob.db.verification.utils.File]

The sorted list of files used for to enroll the model with the given model id.

t_model_ids(group = 'dev') → ids

Returns a list of model ids of T-Norm models for the given group, respecting the current protocol.

Parameters:

group : one of ('dev', 'eval')

The group to get the model ids for.

Returns:

ids : [int] or [str]

The list of (unique) model ids for T-Norm models of the given group.

test_files(groups = ['dev']) → files

Returns all test files (i.e., files used for enrollment and probing) for the given groups, respecting the current protocol. The files for the steps can be limited using the all_files_options defined in the constructor.

Parameters:

groups : some of ('dev', 'eval')

The groups to get the data for.

Returns:

files : [bob.db.verification.utils.File]

The sorted and unique list of test files of the database.

training_files(step = None, arrange_by_client = False) → files

Returns all training files for the given step, and arranges them by client, if desired, respecting the current protocol. The files for the steps can be limited using the ..._training_options defined in the constructor.

Parameters:

step : one of ('train_extractor', 'train_projector', 'train_enroller') or None

The step for which the training data should be returned.

arrange_by_client : bool

Should the training files be arranged by client? If set to True, training files will be returned in [[bob.db.verification.utils.File]], where each sub-list contains the files of a single client. Otherwise, all files will be stored in a simple [bob.db.verification.utils.File].

Returns:

files : [bob.db.verification.utils.File] or [[bob.db.verification.utils.File]]

The (arranged) list of files used for the training of the given step.

uses_probe_file_sets()

Defines if, for the current protocol, the database uses several probe files to generate a score.

z_probe_file_sets(group = 'dev') → files

Returns a list of probe FileSet objects used to compute the Z-Norm. The Z-probe files can be limited using the z_probe_options in the query to

Parameters:

group : one of ('dev', 'eval')

The group to get the Z-norm probe files for.

Returns:

files : [FileSet] or similar

The unique list of file sets used to compute the Z-norm.

z_probe_files(group = 'dev') → files

Returns a list of probe files used to compute the Z-Norm, respecting the current protocol. The Z-probe files can be limited using the z_probe_options in the query to bob.db.verification.utils.ZTDatabase.z_probe_files()

Parameters:

group : one of ('dev', 'eval')

The group to get the Z-norm probe files for.

Returns:

files : [bob.db.verification.utils.File]

The unique list of files used to compute the Z-norm.

class bob.bio.base.database.DatabaseFileList(database, **kwargs)

Bases: bob.bio.base.database.DatabaseBob.DatabaseBobZT

This class can be used whenever you have a database that uses the Bob filelist database interface, which is defined in bob.db.verification.filelist.Database

Parameters:

database : a bob.db.verification.filelist.Database

The database instance that provides the actual interface.

kwargs : key=value pairs

The arguments of the DatabaseBobZT or DatabaseBob base class constructors.

Note

Usually, the name, protocol, training_depends_on_protocol and models_depend_on_protocol keyword parameters of the base class constructor need to be specified.

__str__() → info

This function returns all parameters of this class (and its derived class).

Returns:

info : str

A string containing the full information of all parameters of this (and the derived) class.

all_files(groups=None) → files

Returns all files of the database, respecting the current protocol. If the current protocol is 'None', None will be used instead. When the underlying file list database provides files for ZT score normalization, these files are returned as well. The files can be limited using the all_files_options in the constructor.

Parameters:

groups : some of ('world', 'dev', 'eval') or None

The groups to get the data for. If None, data for all groups is returned.

Returns:

files : [bob.db.verification.filelist.File]

The sorted and unique list of all files of the database.

annotations(file) → annots

Returns the annotations for the given File object, if available.

Parameters:

file : bob.db.verification.utils.File

The file for which annotations should be returned.

Returns:

annots : dict or None

The annotations for the file, if available.

arrange_by_client(files) → files_by_client

Arranges the given list of files by client id. This function returns a list of lists of File’s.

Parameters:

files : File

A list of files that should be split up by File.client_id.

Returns:

files_by_client : [[File]]

The list of lists of files, where each sub-list groups the files with the same File.client_id

client_id_from_model_id(model_id, group = 'dev') → client_id

Uses bob.db.verification.filelist.Database.get_client_id_from_model_id() to retrieve the client id for the given model id. If the current protocol is 'None', None will be used instead.

Parameters:

model_id : str

A unique ID that identifies the model for the client.

group : one of ('dev', 'eval')

The group to get the client ids for.

Returns:

client_id : str

A unique ID that identifies the client, to which the model belongs.

client_id_from_t_model_id(t_model_id, group='dev')

client_id_from_t_model_idt_(model_id, group = ‘dev’) -> client_id

Uses bob.db.verification.filelist.Database.get_client_id_from_t_model_id() to retrieve the client id for the T-norm given model id. If the current protocol is 'None', None will be used instead.

Parameters:

t_model_id : str

A unique ID that identifies the T-Norm model.

group : one of ('dev', 'eval')

The group to get the client ids for.

Returns:

client_id : str

A unique ID that identifies the client, to which the T-Norm model belongs.

enroll_files(model_id, group = 'dev') → files

Returns a list of File objects that should be used to enroll the model with the given model id from the given group, respecting the current protocol.

Parameters:

model_id : int or str

A unique ID that identifies the model.

group : one of ('dev', 'eval')

The group to get the enrollment files for.

Returns:

files : [bob.db.verification.utils.File]

The list of files used for to enroll the model with the given model id.

file_names(files, directory, extension) → paths

Returns the full path of the given File objects.

Parameters:

files : [File]

The list of file object to retrieve the file names for.

directory : str

The base directory, where the files can be found.

extension : str

The file name extension to add to all files.

Returns:

paths : [str] or [[str]]

The paths extracted for the files, in the same order. If this database provides file sets, a list of lists of file names is returned, one sub-list for each file set.

model_ids(group = 'dev') → ids

Returns a list of model ids for the given group, respecting the current protocol. If the current protocol is 'None', None will be used instead.

Parameters:

group : one of ('dev', 'eval')

The group to get the model ids for.

Returns:

ids : [str]

The list of (unique) model ids for the given group.

original_file_names(files) → paths

Returns the full path of the original data of the given File objects, as returned by bob.db.verification.utils.Database.original_file_names().

Parameters:

files : [bob.db.verification.utils.File]

The list of file object to retrieve the original data file names for.

Returns:

paths : [str] or [[str]]

The paths extracted for the files, in the same order. If this database provides file sets, a list of lists of file names is returned, one sub-list for each file set.

probe_file_sets(model_id = None, group = 'dev') → files

Returns a list of probe FileSet objects, respecting the current protocol. If a model_id is specified, only the probe files that should be compared with the given model id are returned (for most databases, these are all probe files of the given group). Otherwise, all probe files of the given group are returned.

Parameters:

model_id : int or str or None

A unique ID that identifies the model.

group : one of ('dev', 'eval')

The group to get the enrollment files for.

Returns:

files : [FileSet] or something similar

The list of file sets used to probe the model with the given model id.

probe_files(model_id = None, group = 'dev') → files

Returns a list of probe File objects, respecting the current protocol. If a model_id is specified, only the probe files that should be compared with the given model id are returned (for most databases, these are all probe files of the given group). Otherwise, all probe files of the given group are returned.

Parameters:

model_id : int or str or None

A unique ID that identifies the model.

group : one of ('dev', 'eval')

The group to get the enrollment files for.

Returns:

files : [bob.db.verification.utils.File]

The list of files used for to probe the model with the given model id.

replace_directories(replacements=None)

This helper function replaces the original_directory and the annotation_directory of the database with the directories read from the given replacement file.

This function is provided for convenience, so that the database configuration files do not need to be modified. Instead, this function uses the given dictionary of replacements to change the original directory and the original extension (if given).

The given replacements can be of type dict, including all replacements, or a file name (as a str), in which case the file is read. The structure of the file should be:

# Comments starting with # and empty lines are ignored

[YOUR_..._DATA_DIRECTORY] = /path/to/your/data
[YOUR_..._ANNOTATION_DIRECTORY] = /path/to/your/annotations

If no annotation files are available (e.g. when they are stored inside the database), the annotation directory can be left out.

Parameters:

replacements : dict or str

A dictionary with replacements, or a name of a file to read the dictionary from. If the file name does not exist, no directories are replaced.

sort(files) → sorted

Returns a sorted version of the given list of File’s (or other structures that define an ‘id’ data member). The files will be sorted according to their id, and duplicate entries will be removed.

Parameters:

files : [File]

The list of files to be uniquified and sorted.

Returns:

sorted : [File]

The sorted list of files, with duplicate File.ids being removed.

t_enroll_files(t_model_id, group = 'dev') → files

Returns a list of File objects that should be used to enroll the T-Norm model with the given model id from the given group, respecting the current protocol.

Parameters:

t_model_id : int or str

A unique ID that identifies the model.

group : one of ('dev', 'eval')

The group to get the enrollment files for.

Returns:

files : [bob.db.verification.utils.File]

The sorted list of files used for to enroll the model with the given model id.

t_model_ids(group = 'dev') → ids

Returns a list of model ids of T-Norm models for the given group, respecting the current protocol. If the current protocol is 'None', None will be used instead.

Parameters:

group : one of ('dev', 'eval')

The group to get the model ids for.

Returns:

ids : [int] or [str]

The list of (unique) model ids for T-Norm models of the given group.

test_files(groups = ['dev']) → files

Returns all test files (i.e., files used for enrollment and probing) for the given groups, respecting the current protocol. The files for the steps can be limited using the all_files_options defined in the constructor.

Parameters:

groups : some of ('dev', 'eval')

The groups to get the data for.

Returns:

files : [bob.db.verification.utils.File]

The sorted and unique list of test files of the database.

training_files(step = None, arrange_by_client = False) → files

Returns all training files for the given step, and arranges them by client, if desired, respecting the current protocol. The files for the steps can be limited using the ..._training_options defined in the constructor.

Parameters:

step : one of ('train_extractor', 'train_projector', 'train_enroller') or None

The step for which the training data should be returned.

arrange_by_client : bool

Should the training files be arranged by client? If set to True, training files will be returned in [[bob.db.verification.utils.File]], where each sub-list contains the files of a single client. Otherwise, all files will be stored in a simple [bob.db.verification.utils.File].

Returns:

files : [bob.db.verification.utils.File] or [[bob.db.verification.utils.File]]

The (arranged) list of files used for the training of the given step.

uses_probe_file_sets()

File sets are not (yet) supported in the bob.db.verification.filelist.Database, so this function returns False throughout.

z_probe_file_sets(group = 'dev') → files

Returns a list of probe FileSet objects used to compute the Z-Norm. The Z-probe files can be limited using the z_probe_options in the query to

Parameters:

group : one of ('dev', 'eval')

The group to get the Z-norm probe files for.

Returns:

files : [FileSet] or similar

The unique list of file sets used to compute the Z-norm.

z_probe_files(group = 'dev') → files

Returns a list of probe files used to compute the Z-Norm, respecting the current protocol. The Z-probe files can be limited using the z_probe_options in the query to bob.db.verification.utils.ZTDatabase.z_probe_files()

Parameters:

group : one of ('dev', 'eval')

The group to get the Z-norm probe files for.

Returns:

files : [bob.db.verification.utils.File]

The unique list of files used to compute the Z-norm.

class bob.bio.base.database.DatabaseZT(name, original_directory=None, original_extension=None, annotation_directory=None, annotation_extension='.pos', annotation_type=None, protocol='Default', training_depends_on_protocol=False, models_depend_on_protocol=False, **kwargs)

Bases: bob.bio.base.database.Database.Database

This class defines additional API functions that are required to compute ZT score normalization. This class does not define a constructor. During construction of a derived class, please call the constructor of the base class Database directly.

__str__() → info

This function returns all parameters of this class.

Returns:

info : str

A string containing the full information of all parameters of this class.

all_files(groups=None) → files

Returns all files of the database. This function needs to be implemented in derived class implementations.

Parameters:

groups : some of ('world', 'dev', 'eval') or None

The groups to get the data for. If None, data for all groups is returned.

Returns:

files : [File]

The sorted and unique list of all files of the database.

annotations(file) → annots

Returns the annotations for the given File object, if available. It uses bob.db.verification.utils.read_annotation_file() to load the annotations.

Parameters:

file : File

The file for which annotations should be returned.

Returns:

annots : dict or None

The annotations for the file, if available.

arrange_by_client(files) → files_by_client

Arranges the given list of files by client id. This function returns a list of lists of File’s.

Parameters:

files : File

A list of files that should be split up by File.client_id.

Returns:

files_by_client : [[File]]

The list of lists of files, where each sub-list groups the files with the same File.client_id

client_id_from_model_id(model_id, group = 'dev') → client_id

In some databases, each client can contain several models. Hence, client and model ids differ. This function converts the given model id into its according the client id. This function needs to be implemented in derived class implementations.

Parameters:

model_id : int or str

A unique ID that identifies the model for the client.

group : one of ('dev', 'eval')

The group to get the client ids for.

Returns:

client_id : [int] or [str]

A unique ID that identifies the client, to which the model belongs.

client_id_from_t_model_id(t_model_id, group = 'dev') → client_id

Returns the client id for the given T-Norm model id. In this base class implementation, we just use the client_id_from_model_id() function. Overload this function if you need another behavior.

Parameters:

t_model_id : int or str

A unique ID that identifies the T-Norm model.

group : one of ('dev', 'eval')

The group to get the client ids for.

Returns:

client_id : [int] or [str]

A unique ID that identifies the client, to which the T-Norm model belongs.

enroll_files(model_id, group = 'dev') → files

Returns a list of File objects that should be used to enroll the model with the given model id from the given group. This function needs to be implemented in derived class implementations.

Parameters:

model_id : int or str

A unique ID that identifies the model.

group : one of ('dev', 'eval')

The group to get the enrollment files for.

Returns:

files : [File]

The list of files used for to enroll the model with the given model id.

file_names(files, directory, extension) → paths

Returns the full path of the given File objects.

Parameters:

files : [File]

The list of file object to retrieve the file names for.

directory : str

The base directory, where the files can be found.

extension : str

The file name extension to add to all files.

Returns:

paths : [str] or [[str]]

The paths extracted for the files, in the same order. If this database provides file sets, a list of lists of file names is returned, one sub-list for each file set.

model_ids(group = 'dev') → ids

Returns a list of model ids for the given group. This function needs to be implemented in derived class implementations.

Parameters:

group : one of ('dev', 'eval')

The group to get the model ids for.

Returns:

ids : [int] or [str]

The list of (unique) model ids for the given group.

original_file_names(files) → paths

Returns the full path of the original data of the given File objects.

Parameters:

files : [File]

The list of file object to retrieve the original data file names for.

Returns:

paths : [str] or [[str]]

The paths extracted for the files, in the same order. If this database provides file sets, a list of lists of file names is returned, one sub-list for each file set.

probe_file_sets(model_id = None, group = 'dev') → files

Returns a list of probe FileSet objects. If a model_id is specified, only the probe files that should be compared with the given model id are returned (for most databases, these are all probe files of the given group). Otherwise, all probe files of the given group are returned. This function needs to be implemented in derived class implementations, if the uses_probe_file_sets() returns True.

Parameters:

model_id : int or str or None

A unique ID that identifies the model.

group : one of ('dev', 'eval')

The group to get the enrollment files for.

Returns:

files : [FileSet]

The list of file sets used to probe the model with the given model id.

probe_files(model_id = None, group = 'dev') → files

Returns a list of probe File objects. If a model_id is specified, only the probe files that should be compared with the given model id are returned (for most databases, these are all probe files of the given group). Otherwise, all probe files of the given group are returned. This function needs to be implemented in derived class implementations.

Parameters:

model_id : int or str or None

A unique ID that identifies the model.

group : one of ('dev', 'eval')

The group to get the enrollment files for.

Returns:

files : [File]

The list of files used for to probe the model with the given model id.

sort(files) → sorted

Returns a sorted version of the given list of File’s (or other structures that define an ‘id’ data member). The files will be sorted according to their id, and duplicate entries will be removed.

Parameters:

files : [File]

The list of files to be uniquified and sorted.

Returns:

sorted : [File]

The sorted list of files, with duplicate File.ids being removed.

t_enroll_files(t_model_id, group = 'dev') → files

Returns a list of File objects that should be used to enroll the T-Norm model with the given model id from the given group. This function needs to be implemented in derived class implementations.

Parameters:

t_model_id : int or str

A unique ID that identifies the model.

group : one of ('dev', 'eval')

The group to get the enrollment files for.

Returns:

files : [File]

The list of files used for to enroll the model with the given model id.

t_model_ids(group = 'dev') → ids

Returns a list of model ids of T-Norm models for the given group. This function needs to be implemented in derived class implementations.

Parameters:

group : one of ('dev', 'eval')

The group to get the model ids for.

Returns:

ids : [int] or [str]

The list of (unique) model ids for T-Norm models of the given group.

training_files(step = None, arrange_by_client = False) → files

Returns all training File objects for the given step, and arranges them by client, if desired. This function needs to be implemented in derived class implementations.

Parameters:

step : one of ('train_extractor', 'train_projector', 'train_enroller') or None

The step for which the training data should be returned. Might be ignored in derived class implementations.

arrange_by_client : bool

Should the training files be arranged by client?

Note

You can use arrange_by_client() in derived class implementations to arrange the files.

Returns:

files : [File] or [[File]]

The (arranged) list of files used for the training of the given step.

uses_probe_file_sets()

Defines if, for the current protocol, the database uses several probe files to generate a score. By default, False is returned. Overwrite the default if you need different behavior.

z_probe_file_sets(group = 'dev') → files

Returns a list of probe FileSet objects used to compute the Z-Norm. This function needs to be implemented in derived class implementations.

Parameters:

group : one of ('dev', 'eval')

The group to get the Z-norm probe files for.

Returns:

files : [FileSet]

The unique list of file sets used to compute the Z-norm.

z_probe_files(group = 'dev') → files

Returns a list of probe File objects used to compute the Z-Norm. This function needs to be implemented in derived class implementations.

Parameters:

group : one of ('dev', 'eval')

The group to get the Z-norm probe files for.

Returns:

files : [File]

The unique list of files used to compute the Z-norm.

class bob.bio.base.database.File(file_id, client_id, path)

This class defines the minimum interface of a database file that needs to be exported.

Each file has a path, an id and an associated client (aka. identity, person, user). Usually, this file is part of a database, which has a common directory for all files. The path of this file is usually relative to that common directory, and it is usually stored without filename extension. The file id can be anything hashable, but needs to be unique all over the database. The client id can be anything hashable, but needs to be identical for different files of the same client, and different between clients.

Parameters:

file_id : str or int

A unique ID that identifies the file. This ID might be identical to the path, though integral IDs perform faster.

client_id : str or int

A unique ID that identifies the client (user) to which this file belongs. This ID might be the name of the person, though integral IDs perform faster.

path : str

The file path of the file, which is relative to the common database directory, and without filename extension.

make_path(directory = None, extension = None) → path

Generates the full path using the given directory and filename extension.

Parameters:

directory : str or None

The directory to prepend. If None, no directory will be preprended.

extension : str or None

The filename extension to append. If None, no file name extension will be appended.

Returns:

path : str

The full path including directory and extension.

class bob.bio.base.database.FileSet(file_set_id, files, path=None)

This class defines the minimum interface of a set of database files that needs to be exported.

Use this class, whenever the database provides several files that belong to the same probe.

Each file set has an id, and a list of associated files, which are of type File of the same client. The file set id can be anything hashable, but needs to be unique all over the database.

Parameters:

file_set_id : str or int

A unique ID that identifies the file set.

files : [File]

A list of File objects that should be stored inside this file. All files of that list need to have the same client ID.

Grid Configuration

class bob.bio.base.grid.Grid(grid_type='sge', number_of_preprocessing_jobs=32, number_of_extraction_jobs=32, number_of_projection_jobs=32, number_of_enrollment_jobs=32, number_of_scoring_jobs=32, training_queue='8G', preprocessing_queue='default', extraction_queue='default', projection_queue='default', enrollment_queue='default', scoring_queue='default', number_of_parallel_processes=1, scheduler_sleep_time=1.0)

This class is defining the options that are required to submit parallel jobs to the SGE grid, or jobs to the local queue.

If the given grid_type is 'sge' (the default), this configuration is set up to submit algorithms to the SGE grid. In this setup, specific SGE queues can be specified for different steps of the tool chain, and different numbers of parallel processes can be specified for each step. Currently, only the SGE at Idiap is tested and supported, for other SGE’s we do not assure compatibility.

If the given grid_type is 'local', this configuration is set up to run using a local scheduler on a single machine. In this case, only the number_of_parallel_processes and scheduler_sleep_time options will be taken into account.

Parameters:

grid_type : one of ('sge', 'local')

The type of submission system, which should be used. Currently, only sge and local submissions are supported.

number_of_preprocessing_jobs, number_of_extraction_jobs, number_of_projection_jobs, number_of_enrollment_jobs, number_of_scoring_jobs : int

Only valid if grid_type = 'sge'. The number of parallel processes that should be executed for preprocessing, extraction, projection, enrollment or scoring.

training_queue, preprocessing_queue, extraction_queue, projection_queue, enrollment_queue, scoring_queue : str or dict

Only valid if grid_type = 'sge'. SGE queues that should be used for training, preprocessing, extraction, projection, enrollment or scoring. The queue can be defined using a dictionary of keywords that will directly passed to the gridtk.tools.qsub() function, or one of our PREDEFINED_QUEUES, which are adapted for Idiap.

number_of_parallel_processes : int

Only valid if grid_type = 'local'. The number of parallel processes, with which the preprocessing, extraction, projection, enrollment and scoring should be executed.

scheduler_sleep_time : float

The time (in seconds) that the local scheduler will sleep between its iterations.

queue(params) → dict

This helper function translates the given queue parameters to grid options. When the given params are a dictionary already, they are simply returned. If params is a string, the PREDEFINED_QUEUES are indexed with them. If params is None, or the grid_type is 'local', an empty dictionary is returned.

is_local()

Returns whether this grid setup should use the local submission or the SGE grid.

bob.bio.base.grid.PREDEFINED_QUEUES

A dictionary of predefined queue keywords, which are adapted to the Idiap SGE.

PREDEFINED_QUEUES = {
  "16G": {
    "hvmem": "8G",
    "memfree": "16G",
    "pe_opt": "pe_mth 2",
    "queue": "q1dm"
  },
  "16G-io-big": {
    "hvmem": "8G",
    "io_big": true,
    "memfree": "16G",
    "pe_opt": "pe_mth 2",
    "queue": "q1dm"
  },
  "2G": {
    "memfree": "2G",
    "queue": "all.q"
  },
  "32G": {
    "hvmem": "8G",
    "io_big": true,
    "memfree": "32G",
    "pe_opt": "pe_mth 4",
    "queue": "q1dm"
  },
  "4G": {
    "memfree": "4G",
    "queue": "all.q"
  },
  "4G-io-big": {
    "io_big": true,
    "memfree": "4G",
    "queue": "q1d"
  },
  "64G": {
    "hvmem": "8G",
    "io_big": true,
    "memfree": "64G",
    "pe_opt": "pe_mth 8",
    "queue": "q1dm"
  },
  "8G": {
    "memfree": "8G",
    "queue": "q1d"
  },
  "8G-io-big": {
    "io_big": true,
    "memfree": "8G",
    "queue": "q1d"
  },
  "Week": {
    "hvmem": "8G",
    "memfree": "32G",
    "pe_opt": "pe_mth 4",
    "queue": "q1wm"
  },
  "default": {}
}