Tools implemented in bob.bio.video

Summary

bob.bio.video.select_frames(count[, …])	Returns indices of the frames to be selected given the parameters.
bob.bio.video.VideoAsArray(path[, …])	A memory efficient class to load only select video frames.
bob.bio.video.VideoLikeContainer(data, …)
bob.bio.video.transformer.VideoWrapper(…)	Wrapper class to run image preprocessing algorithms on video data.
bob.bio.video.annotator.Base()	The base class for video annotators.
bob.bio.video.annotator.Wrapper(annotator[, …])	Annotates video files using the provided image annotator.
bob.bio.video.annotator.FailSafeVideo(annotators)	A fail-safe video annotator.

Databases

bob.bio.video.database.YoutubeBioDatabase([…])

YouTube Faces database implementation of bob.bio.base.database.ZTBioDatabase interface.

Details

class bob.bio.video.VideoAsArray(path, selection_style=None, max_number_of_frames=None, step_size=None, transform=None, **kwargs)

Bases: object

A memory efficient class to load only select video frames. It also supports efficient conversion to dask arrays.

class bob.bio.video.VideoLikeContainer(data, indices, **kwargs)

Bases: object

property dtype

classmethod load(file)

property ndim

save(file)

classmethod save_function(other, file)

property shape

bob.bio.video.get_config()

Returns a string containing the configuration information.

bob.bio.video.select_frames(count, max_number_of_frames=None, selection_style=None, step_size=None)

Returns indices of the frames to be selected given the parameters.

Different selection styles are supported:

• first : The first frames are selected

• spread : Frames are selected to be taken from the whole video with equal spaces in between.

• step : Frames are selected every step_size indices, starting at step_size/2 Think twice if you want to have that when giving FrameContainer data!

• all : All frames are selected unconditionally.

Parameters

• count (int) – Total number of frames that are available

• max_number_of_frames (int) – The maximum number of frames to be selected. Ignored when selection_style is “all”.

• selection_style (str) – One of (first, spread, step, all). See above.

• step_size (int) – Only useful when selection_style is step.

Returns

A range of frames to be selected.

Return type

range

Raises

ValueError – If selection_style is not one of the supported ones.

bob.bio.video.annotator.normalize_annotations(annotations, validator, max_age=- 1)

Normalizes the annotations of one video sequence. It fills the annotations for frames from previous ones if the annotation for the current frame is not valid.

Parameters

• annotations (collections.OrderedDict) – A dict of dict where the keys to the first dict are frame indices as strings (starting from 0). The inside dicts contain annotations for that frame. The dictionary needs to be an ordered dict in order for this to work.

• validator (callable) – Takes a dict (annotations) and returns True if the annotations are valid. This can be a check based on minimal face size for example: see bob.bio.face.annotator.min_face_size_validator.

• max_age (int, optional) – An integer indicating for a how many frames a detected face is valid if no detection occurs after such frame. A value of -1 == forever

Yields

• str – The index of frame.

• dict – The corrected annotations of the frame.

class bob.bio.video.annotator.Base

Bases: bob.bio.base.annotator.Annotator

The base class for video annotators.

static frame_ids_and_frames(frames)

Takes the frames and yields frame_ids and frames.

Parameters

frames (bob.bio.video.VideoLikeContainer or bob.bio.video.VideoAsArray or numpy.array) – The frames of the video file.

Yields

• frame_id (str) – A string that represents the frame id.

• frame (numpy.array) – The frame of the video file as an array.

annotate(frames, **kwargs)

Annotates videos.

Parameters

• frames (bob.bio.video.VideoLikeContainer or bob.bio.video.VideoAsArray or numpy.array) – The frames of the video file.

• **kwargs – Extra arguments that annotators may need.

Returns

A dictionary where its key is the frame id as a string and its value is a dictionary that are the annotations for that frame.

Return type

collections.OrderedDict

Note

You can use the Base.frame_ids_and_frames functions to normalize the input in your implementation.

transform(samples)

Takes a batch of data and annotates them.

Each kwargs value is a list of parameters, with each element of those lists corresponding to each element of samples (for example: with [s1, s2, ...] as samples, kwargs['annotations'] should contain [{<s1_annotations>}, {<s2_annotations>}, ...]).

class bob.bio.video.annotator.FailSafeVideo(annotators, max_age=15, validator=None, **kwargs)

Bases: bob.bio.video.annotator.Base

A fail-safe video annotator. It tries several annotators in order and tries the next one if the previous one fails. However, the difference between this annotator and bob.bio.base.annotator.FailSafe is that this one tries to use annotations from older frames (if valid) before trying the next annotator.

Warning

You must be careful in using this annotator since different annotators could have different results. For example the bounding box of one annotator be totally different from another annotator.

Parameters

• annotators (list) – A list of annotators to try.

• max_age (int) – The maximum number of frames that an annotation is valid for next frames. This value should be positive. If you want to set max_age to infinite, then you can use the bob.bio.video.annotator.Wrapper instead.

• validator (callable) – A function that takes the annotations of a frame and validates it.

Please see Base for more accepted parameters.

annotate(frames)

See Base.annotate

class bob.bio.video.annotator.Wrapper(annotator, normalize=False, validator=None, max_age=- 1, **kwargs)

Bases: bob.bio.video.annotator.Base

Annotates video files using the provided image annotator. See the documentation of Base too.

Parameters

• annotator (bob.bio.base.annotator.Annotator or str) – The image annotator to be used. The annotator could also be the name of a bob.bio.annotator resource which will be loaded.

• max_age (int) – see normalize_annotations.

• normalize (bool) – If True, it will normalize annotations using normalize_annotations

• validator (object) – See normalize_annotations and bob.bio.face.annotator.min_face_size_validator for one example.

Please see Base for more accepted parameters.

Warning

You should only set normalize to True only if you are annotating all frames of the video file.

annotate(frames)

See Base.annotate

class bob.bio.video.transformer.VideoWrapper(estimator, **kwargs)

Bases: sklearn.base.TransformerMixin, sklearn.base.BaseEstimator

Wrapper class to run image preprocessing algorithms on video data.

Parameters:

estimatorstr or sklearn.base.BaseEstimator instance

The transformer to be used to preprocess the frames.

transform(videos, **kwargs)

fit(X, y=None, **fit_params)

Does nothing

class bob.bio.video.database.VideoBioFile(client_id, path, file_id, original_directory=None, original_extension='.avi', annotation_directory=None, annotation_extension=None, annotation_type=None, selection_style=None, max_number_of_frames=None, step_size=None, **kwargs)

Bases: bob.bio.base.database.BioFile

load()

Loads the data at the specified location and using the given extension. Override it if you need to load differently.

Parameters

• original_directory (str (optional)) – The path to the root of the dataset structure. If None, will try to use self.original_directory.

• original_extension (str (optional)) – The filename extension of every files in the dataset. If None, will try to use self.original_extension.

Returns

The loaded data (normally numpy.ndarray).

Return type

object

class bob.bio.video.database.YoutubeBioDatabase(original_directory=None, original_extension='.jpg', annotation_extension='.labeled_faces.txt', **kwargs)

Bases: bob.bio.base.database.ZTBioDatabase

YouTube Faces database implementation of bob.bio.base.database.ZTBioDatabase interface. It is an extension of an SQL-based database interface, which directly talks to bob.db.youtube.Database database, for verification experiments (good to use in bob.bio framework).

annotations(myfile)

Returns the annotations for the given File object, if available. You need to override this method in your high-level implementation. If your database does not have annotations, it should return None.

Parameters:

filebob.bio.base.database.BioFile

The file for which annotations should be returned.

Returns:

annotsdict or None

The annotations for the file, if available.

client_id_from_model_id(model_id, group='dev')

Return the client id associated with the given model id. In this base class implementation, it is assumed that only one model is enrolled for each client and, thus, client id and model id are identical. All key word arguments are ignored. Please override this function in derived class implementations to change this behavior.

model_ids_with_protocol(groups=None, protocol=None, **kwargs) → ids

Returns a list of model ids for the given groups and given protocol.

Parameters:

groupsone or more of ('world', 'dev', 'eval')

The groups to get the model ids for.

protocol: a protocol name

Returns:

ids[int] or [str]

The list of (unique) model ids for the given groups.

objects(groups=None, protocol=None, purposes=None, model_ids=None, **kwargs)

This function returns a list of bob.bio.base.database.BioFile objects or the list of objects which inherit from this class. Returned files fulfill the given restrictions.

Keyword parameters:

groupsstr or [str]

The groups of which the clients should be returned. Usually, groups are one or more elements of (‘world’, ‘dev’, ‘eval’)

protocol

The protocol for which the clients should be retrieved. The protocol is dependent on your database. If you do not have protocols defined, just ignore this field.

purposesstr or [str]

The purposes for which File objects should be retrieved. Usually, purposes are one of (‘enroll’, ‘probe’).

model_ids[various type]

The model ids for which the File objects should be retrieved. What defines a ‘model id’ is dependent on the database. In cases, where there is only one model per client, model ids and client ids are identical. In cases, where there is one model per file, model ids and file ids are identical. But, there might also be other cases.

property original_directory

tmodel_ids_with_protocol(protocol=None, groups=None, **kwargs)

This function returns the ids of the T-Norm models of the given groups for the given protocol.

Keyword parameters:

groupsstr or [str]

The groups of which the model ids should be returned. Usually, groups are one or more elements of (‘dev’, ‘eval’)

protocolstr

The protocol for which the model ids should be retrieved. The protocol is dependent on your database. If you do not have protocols defined, just ignore this field.

tobjects(groups=None, protocol=None, model_ids=None, **kwargs)

This function returns the File objects of the T-Norm models of the given groups for the given protocol and the given model ids.

Keyword parameters:

groupsstr or [str]

The groups of which the model ids should be returned. Usually, groups are one or more elements of (‘dev’, ‘eval’)

protocolstr

The protocol for which the model ids should be retrieved. The protocol is dependent on your database. If you do not have protocols defined, just ignore this field.

model_ids[various type]

The model ids for which the File objects should be retrieved. What defines a ‘model id’ is dependent on the database. In cases, where there is only one model per client, model ids and client ids are identical. In cases, where there is one model per file, model ids and file ids are identical. But, there might also be other cases.

zobjects(groups=None, protocol=None, **kwargs)

This function returns the File objects of the Z-Norm impostor files of the given groups for the given protocol.

Keyword parameters:

groupsstr or [str]

The groups of which the model ids should be returned. Usually, groups are one or more elements of (‘dev’, ‘eval’)

protocolstr

The protocol for which the model ids should be retrieved. The protocol is dependent on your database. If you do not have protocols defined, just ignore this field.