COHFACE Database

Development

This package can, optionally, automatically annotate the following key aspects of the COHFACE dataset:

• Average heart-rate in beats-per-minute (BPM), using a custom peak detector

• Face bounding boxes, as detected by the default detector on Bob’s Face Detection Routines

Warning

Note this procedure is outdated by current metadata which is already shipped with the COHFACE dataset and this package. Only use it in case you know what you’re doing and/or want to modify/re-evaluate this package’s metadata.

For it to work properly, you’ll need to modify the method bob.db.cohface.File.load_face_detection() to take it into account. As of today, it is set to load face detections from the files distributed with the COHFACE dataset.

The annotation procedure can be launched with the following command:

$ bob_dbmanage.py cohface mkmeta

Each video, which is composed of a significant number of frames (hundreds), takes about 5 minutes to get completely processed. If are at Idiap, you can launch the job on the SGE queue using the following command-line:

$ jman sub -q q1d --io-big -t 160 `which bob_dbmanage.py` cohface mkmeta

API

A very simple API to query and load data from the COHFACE database.

class bob.db.cohface.Database(dbdir='/idiap/project/cohface/ddp')

Bases: object

objects(protocol='all', subset=None)

Returns a list of unique File objects for the specific query by the user.

Parameters

• protocol (str, optional) – If set, it should be either clean, natural or all. All, which is the default, considers all the illumination conditions whereas clean retrieves only sequences where the spot in on and natural the ones with daylight illumination.

• subset (str, optional) – If set, it could be either train, dev or test or a combination of them (i.e. a list). If not set (default), the files from all these sets are retrieved, according to the protocol.

Returns

A list of File objects.

Return type

list

class bob.db.cohface.File(path)

Bases: bob.db.base.File

Generic file container for COHFACE files

Parameters

path (str) – The stem of the files for a particular session

default_extension()

estimate_heartrate_in_bpm(directory)

Estimates the person’s heart rate using the contact PPG sensor data

Parameters

directory (str) – A directory name that leads to the location the database is installed on the local disk

load(directory=None, extension='.avi')

Loads the video for this file entry

Parameters

directory (str) – The path to the root of the database installation. This is the path leading to directories named D where D’s correspond to digits.

Returns

A 4D array of 8-bit unsigned integers corresponding to the input video for this file in (frame,channel,y,x) notation (Bob-style).

Return type

numpy.ndarray

load_drmf_keypoints()

Loads the 66-keypoints coming from the Discriminative Response Map Fitting (DRMF) landmark detector.

Reference: http://ibug.doc.ic.ac.uk/resources/drmf-matlab-code-cvpr-2013/.

The code was written for Matlab. Data for the first frame of the colour video of this object was loaded on a compatible Matlab framework and the keypoints extracted taking as basis the currently available face bounding box, enlarged by 7% (so the key-point detector performs reasonably well). The extracted keypoints were then merged into this database access package so they are easy to load from python.

The points are in the form (y, x), as it is standard on Bob-based packages.

load_face_detection()

Load bounding boxes for this file

This function loads bounding boxes for each frame of a video sequence. Bounding boxes are loaded from the package base directory and are the ones provided with it. These bounding boxes were generated from run_face_detector() over the whole dataset.

Returns

A dictionary where the key is the frame number and the values are instances of bob.ip.facedetect.BoundingBox.

Return type

dict

load_hdf5(directory)

Loads the hdf5 file containing the sensor data

Parameters:

directory (str): A directory name that will be prefixed to the returned

result.

Returns

bob.io.base.HDF5File

load_heart_rate_in_bpm()

Loads the heart-rate from locally stored files if they exist, fails gracefully otherwise, returning None

load_video(directory)

Loads the colored video file associated to this object

Parameters

directory (str) – A directory name that will be prefixed to the returned result.

Returns

bob.io.video.reader: Preloaded and ready to be iterated by your code.

make_path(directory=None, extension=None)

Wraps this files’ filename so that a complete path is formed

Parameters

• directory (str) – An optional directory name that will be prefixed to the returned result.

• extension (str) – An optional extension that will be suffixed to the returned filename. The extension normally includes the leading . character as in .png or .bmp. If not specified the default extension for the original file in the database will be used.

Returns a string containing the newly generated file path.

metadata(directory)

Returns a dictionary with metadata about this session:

Parameters:

directory (str): A directory name that will be prefixed to the returned

result.

Returns

Containing the following fields

• birth-date: format: %d.%m.%Y

• client-id: integer

• illumination: str (lamp | natural)

• sample-rate-hz: integer - always 256 (Hz)

• scale: str - always uV

• session: integer

Return type

dict

These values are extracted from the HDF5 attributes

run_face_detector(directory, max_frames=0)

Runs bob.ip.facedetect stock detector on the selected frames.

Warning

This method is deprecated and serves only as a development basis to clean-up the load_face_detection(), which for now relies on text files shipped with the database. Technically, the output of this method and the detected faces shipped should be the same as of today, 13 december 2016.

Parameters

• directory (str) – A directory name that leads to the location the database is installed on the local disk

• max_frames (int) – If set, delimits the maximum number of frames to treat from the associated video file.

Returns

A dictionary where the key is the frame number and the values are instances of bob.ip.facedetect.BoundingBox.

Return type

dict

save(data, directory=None, extension='.hdf5')

Saves the input data at the specified location and using the given extension.

Parameters:

data (

The data blob to be saved (normally a numpy.ndarray).

directory

If not empty or None, this directory is prefixed to the final file destination

extension

The extension of the filename - this will control the type of output and the codec for saving the input blob.

class bob.db.cohface.Interface

Bases: bob.db.base.driver.Interface

name()

The name of this database

Returns

a Python-conforming name for this database. This must match the package name. If the package is named bob.db.foo, then this function must return foo.

Return type

str

files()

List of meta-data files for the package to be downloaded/uploaded

This function should normally return an empty list, except in case the database being implemented requires download/upload of metadata files that are not kept in its (git) repository.

Returns

A python iterable with all metadata files needed. The paths listed by this method should correspond to full paths (not relative ones) w.r.t. the database package implementing it. This is normally achieved by using pkg_resources.resource_filename().

Return type

list

version()

The version of this package

Returns

The current version number defined in setup.py

Return type

str

type()

The type of auxiliary files you have for this database

Returns

A string defining the type of database implemented. You can return only two values on this function, either sqlite or text. If you return sqlite, then we append special actions such as dbshell on bob_dbmanage automatically for you. Otherwise, we don’t.

Return type

str

add_commands(parser)

Add specific subcommands that the action “dumplist” can use

bob.db.cohface.resource_filename(package_or_requirement, resource_name)

Return a true filesystem path for specified resource