Tools implemented in bob.bio.video

Summary

bob.bio.video.FrameSelector([...])	A class for selecting frames from videos.
bob.bio.video.FrameContainer([hdf5, ...])	A class for reading, manipulating and saving video content.
bob.bio.video.preprocessor.Wrapper([...])	Wrapper class to run image preprocessing algorithms on video data.
bob.bio.video.extractor.Wrapper(extractor[, ...])	Wrapper class to run feature extraction algorithms on frame containers.
bob.bio.video.algorithm.Wrapper(algorithm[, ...])	Wrapper class to run face recognition algorithms on video data.

Databases

bob.bio.video.database.MobioBioDatabase([...])	MOBIO database implementation of bob.bio.base.database.ZTDatabase interface.
bob.bio.video.database.YoutubeBioDatabase([...])	YouTube Faces database implementation of bob.bio.base.database.ZTBioDatabase interface.

Details

bob.bio.video.get_config()

Returns a string containing the configuration information.

class bob.bio.video.FrameContainer(hdf5=None, load_function=<function load>)

A class for reading, manipulating and saving video content.

add(frame_id, frame, quality=None)

Adds the frame with the given id and the given quality.

is_similar_to(other)

load(hdf5, load_function=<function load>)

save(hdf5, save_function=<function save>)

Save the content to the given HDF5 File. The contained data will be written using the given save_function.

class bob.bio.video.FrameSelector(max_number_of_frames=20, selection_style='spread', step_size=10)

A class for selecting frames from videos. In total, up to max_number_of_frames is selected (unless selection style is all

Different selection styles are supported:

• first : The first frames are selected
• spread : Frames are selected to be taken from the whole video
• step : Frames are selected every step_size indices, starting at step_size/2 Think twice if you want to have that when giving FrameContainer data!
• all : All frames are stored unconditionally
• quality (only valid for FrameContainer data) : Select the frames based on the highest internally stored quality value

bob.bio.video.load_compressed(filename, load_function)

bob.bio.video.save_compressed(frame_container, filename, save_function, create_link=True)

class bob.bio.video.preprocessor.Wrapper(preprocessor='landmark-detect', frame_selector=<bob.bio.video.utils.FrameSelector.FrameSelector instance>, quality_function=None, compressed_io=False, read_original_data=None)

Bases: bob.bio.base.preprocessor.Preprocessor

Wrapper class to run image preprocessing algorithms on video data.

This class provides functionality to read original video data from several databases. So far, the video content from bob.db.mobio and the image list content from bob.db.youtube are supported.

Furthermore, frames are extracted from these video data, and a preprocessor algorithm is applied on all selected frames. The preprocessor can either be provided as a registered resource, i.e., one of Preprocessors, or an instance of a preprocessing class. Since most of the databases do not provide annotations for all frames of the videos, commonly the preprocessor needs to apply face detection.

The frame_selector can be chosen to select some frames from the video. By default, a few frames spread over the whole video sequence are selected.

The quality_function is used to assess the quality of the frame. If no quality_function is given, the quality is based on the face detector, or simply left as None. So far, the quality of the frames are not used, but it is foreseen to select frames based on quality.

Parameters:

preprocessor : str or bob.bio.base.preprocessor.Preprocessor instance

The preprocessor to be used to preprocess the frames.

frame_selector : bob.bio.video.FrameSelector

A frame selector class to define, which frames of the video to use.

quality_function : function or None

A function assessing the quality of the preprocessed image. If None, no quality assessment is performed. If the preprocessor contains a quality attribute, this is taken instead.

compressed_io : bool

Use compression to write the resulting preprocessed HDF5 files. This is experimental and might cause trouble. Use this flag with care.

read_original_data: callable or None

Function that loads the raw data. If not explicitly defined the raw data will be loaded by bob.bio.video.database.VideoBioFile.load() using the specified frame_selector

read_data(filename) → frames

Reads the preprocessed data from file and returns them in a frame container. The preprocessors read_data function is used to read the data for each frame.

Parameters:

filename : str

The name of the preprocessed data file.

Returns:

frames : bob.bio.video.FrameContainer

The read frames, stored in a frame container.

write_data(frames, filename)

Writes the preprocessed data to file.

The preprocessors write_data function is used to write the data for each frame.

Parameters:

frames : bob.bio.video.FrameContainer

The preprocessed frames, as returned by the __call__ function.

filename : str

The name of the preprocessed data file to write.

class bob.bio.video.extractor.Wrapper(extractor, frame_selector=<bob.bio.video.utils.FrameSelector.FrameSelector instance>, compressed_io=False)

Bases: bob.bio.base.extractor.Extractor

Wrapper class to run feature extraction algorithms on frame containers.

Features are extracted for all frames in the frame container using the provided extractor. The extractor can either be provided as a registered resource, i.e., one of Feature extractors, or an instance of an extractor class.

The frame_selector can be chosen to select some frames from the frame container. By default, all frames from the previous preprocessing step are kept, but fewer frames might be selected in this stage.

Parameters:

extractor : str or bob.bio.base.extractor.Extractor instance

The extractor to be used to extract features from the frames.

frame_selector : bob.bio.video.FrameSelector

A frame selector class to define, which frames of the preprocessed frame container to use.

compressed_io : bool

Use compression to write the resulting features to HDF5 files. This is experimental and might cause trouble. Use this flag with care.

load(extractor_file)

Loads the trained extractor from file.

This function calls the wrapped classes load function.

extractor_file : str

The name of the extractor that should be loaded.

read_feature(filename) → frames

Reads the extracted data from file and returns them in a frame container. The extractors read_feature function is used to read the data for each frame.

Parameters:

filename : str

The name of the extracted data file.

Returns:

frames : bob.bio.video.FrameContainer

The read frames, stored in a frame container.

train(training_frames, extractor_file)

Trains the feature extractor with the preprocessed data of the given frames.

Note

This function is not called, when the given extractor does not require training.

This function will train the feature extractor using all data from the selected frames of the training data. The training_frames must be aligned by client if the given extractor requires that.

Parameters:

training_frames : [bob.bio.video.FrameContainer] or [[bob.bio.video.FrameContainer]]

The set of training frames, which will be used to train the extractor.

extractor_file : str

The name of the extractor that should be written.

write_feature(frames, filename)

Writes the extracted features to file.

The extractors write_features function is used to write the features for each frame.

Parameters:

frames : bob.bio.video.FrameContainer

The extracted features for the selected frames, as returned by the __call__ function.

filename : str

The file name to write the extracted feature into.

class bob.bio.video.algorithm.Wrapper(algorithm, frame_selector=<bob.bio.video.utils.FrameSelector.FrameSelector instance>, compressed_io=False)

Bases: bob.bio.base.algorithm.Algorithm

Wrapper class to run face recognition algorithms on video data.

This class provides a generic interface for all face recognition algorithms to use several frames of a video. The algorithm can either be provided as a registered resource, or an instance of an extractor class. Already in previous stages, features were extracted from only some selected frames of the image. This algorithm now uses these features to perform face recognition, i.e., by enrolling a model from several frames (possibly of several videos), and fusing scores from several model frames and several probe frames. Since the functionality to handle several images for enrollment and probing is already implemented in the wrapped class, here we only care about providing the right data at the right time.

Parameters:

algorithm : str or bob.bio.base.algorithm.Algorithm instance

The algorithm to be used.

frame_selector : bob.bio.video.FrameSelector

A frame selector class to define, which frames of the extracted features of the frame container to use. By default, all features are selected.

compressed_io : bool

Use compression to write the projected features to HDF5 files. This is experimental and might cause trouble. Use this flag with care.

enroll(enroll_frames) → model

Enrolls the model from features of all selected frames of all enrollment videos for the current client.

This function collects all desired frames from all enrollment videos and enrolls a model with that, using the algorithms enroll function.

Parameters:

enroll_frames : [bob.bio.video.FrameContainer]

Extracted or projected features from one or several videos of the same client.

Returns:

model : object

The model as created by the algorithms enroll function.

load_enroller(enroller_file)

Loads the trained enroller from file.

This function calls the wrapped classes load_enroller function.

enroller_file : str

The name of the enroller that should be loaded.

load_projector(projector_file)

Loads the trained extractor from file.

This function calls the wrapped classes load_projector function.

projector_file : str

The name of the projector that should be loaded.

project(frames) → projected

Projects the frames from the extracted frames and returns a frame container.

This function is used to project features using the desired algorithm for all frames that are selected by the frame_selector specified in the constructor of this class.

Parameters:

frames : bob.bio.video.FrameContainer

The frame container containing extracted feature frames.

Returns:

projected : bob.bio.video.FrameContainer

A frame container containing projected features.

read_feature(projected_file) → frames

Reads the projected data from file and returns them in a frame container. The algorithms read_feature function is used to read the data for each frame.

Parameters:

filename : str

The name of the projected data file.

Returns:

frames : bob.bio.video.FrameContainer

The read frames, stored in a frame container.

read_model(filename)

Reads the model using the algorithms read_model function.

Parameters:

filename : str

The file name to read the model from.

Returns:

model : object

The model read from file.

score(model, probe) → score

Computes the score between the given model and the probe.

As the probe is a frame container, several scores are computed, one for each frame of the probe. This is achieved by using the algorithms score_for_multiple_probes function. The final result is, hence, a fusion of several scores.

Parameters:

model : object

The model in the type desired by the wrapped algorithm.

probe : bob.bio.video.FrameContainer

The selected frames from the probe objects, which contains the probes are desired by the wrapped algorithm.

Returns:

score : float

A fused score between the given model and all probe frames.

score_for_multiple_models(*args, **kwargs)

score_for_multiple_probes(model, probes) → score

Computes the score between the given model and the given list of probes.

As each probe is a frame container, several scores are computed, one for each frame of each probe. This is achieved by using the algorithms score_for_multiple_probes function. The final result is, hence, a fusion of several scores.

Parameters:

model : object

The model in the type desired by the wrapped algorithm.

probes : [bob.bio.video.FrameContainer]

The selected frames from the probe objects, which contains the probes are desired by the wrapped algorithm.

Returns:

score : float

A fused score between the given model and all probe frames.

train_enroller(training_frames, enroller_file)

Trains the enroller with the features of the given frames.

Note

This function is not called, when the given algorithm does not require enroller training.

This function will train the enroller using all data from the selected frames of the training data.

Parameters:

training_frames : [[bob.bio.video.FrameContainer]]

The set of training frames aligned by client, which will be used to perform enroller training of the algorithm.

enroller_file : str

The name of the enroller that should be written.

train_projector(training_frames, projector_file)

Trains the projector with the features of the given frames.

Note

This function is not called, when the given algorithm does not require projector training.

This function will train the projector using all data from the selected frames of the training data. The training_frames must be aligned by client if the given algorithm requires that.

Parameters:

training_frames : [bob.bio.video.FrameContainer] or [[bob.bio.video.FrameContainer]]

The set of training frames, which will be used to perform projector training of the algorithm.

extractor_file : str

The name of the projector that should be written.

write_feature(frames, projected_file)

Writes the projected features to file.

The extractors write_features function is used to write the features for each frame.

Parameters:

frames : bob.bio.video.FrameContainer

The projected features for the selected frames, as returned by the project() function.

projected_file : str

The file name to write the projetced feature into.

write_model(model, filename)

Writes the model using the algorithm’s write_model function.

Parameters:

model : object

The model returned by the enroll() function.

filename : str

The file name of the model to write.

class bob.bio.video.database.MobioBioDatabase(original_directory=None, original_extension=None, annotation_directory=None, annotation_extension='.pos', **kwargs)

Bases: bob.bio.base.database.ZTBioDatabase

MOBIO database implementation of bob.bio.base.database.ZTDatabase interface. It is an extension of an SQL-based database interface, which directly talks to Mobio database, for verification experiments (good to use in bob.bio.base framework).

annotations(myfile)

Annotations are not available when using videos

model_ids_with_protocol(groups=None, protocol=None, gender=None)

objects(groups=None, protocol=None, purposes=None, model_ids=None, **kwargs)

tmodel_ids_with_protocol(protocol=None, groups=None, **kwargs)

tobjects(groups=None, protocol=None, model_ids=None, **kwargs)

zobjects(groups=None, protocol=None, **kwargs)

class bob.bio.video.database.VideoBioFile(client_id, path, file_id)

Bases: bob.bio.base.database.BioFile

load(directory=None, extension='.avi', frame_selector=<bob.bio.video.utils.FrameSelector.FrameSelector instance>)

class bob.bio.video.database.YoutubeBioDatabase(original_directory=None, original_extension='.jpg', annotation_extension='.labeled_faces.txt', **kwargs)

Bases: bob.bio.base.database.ZTBioDatabase

YouTube Faces database implementation of bob.bio.base.database.ZTBioDatabase interface. It is an extension of an SQL-based database interface, which directly talks to bob.db.youtube.Database database, for verification experiments (good to use in bob.bio framework).

annotations(myfile)

client_id_from_model_id(model_id, group='dev')

model_ids_with_protocol(groups=None, protocol=None, **kwargs)

objects(groups=None, protocol=None, purposes=None, model_ids=None, **kwargs)

tmodel_ids_with_protocol(protocol=None, groups=None, **kwargs)

tobjects(groups=None, protocol=None, model_ids=None, **kwargs)

zobjects(groups=None, protocol=None, **kwargs)