Command-Line Interface (CLI)

This package provides a single entry point for all of its applications using Bob’s unified CLI mechanism. A list of available applications can be retrieved using:

$ bob tb --help
Error: mkl-service + Intel(R) MKL: MKL_THREADING_LAYER=INTEL is incompatible with libgomp.so.1 library.
	Try to import numpy first or set the threading layer accordingly. Set MKL_SERVICE_FORCE_INTEL to force it.

Setup

A CLI application to list and check installed (raw) datasets.

$ bob tb dataset --help
Error: mkl-service + Intel(R) MKL: MKL_THREADING_LAYER=INTEL is incompatible with libgomp.so.1 library.
	Try to import numpy first or set the threading layer accordingly. Set MKL_SERVICE_FORCE_INTEL to force it.

List available datasets

Lists supported and configured raw datasets.

$ bob tb dataset list --help
Error: mkl-service + Intel(R) MKL: MKL_THREADING_LAYER=INTEL is incompatible with libgomp.so.1 library.
	Try to import numpy first or set the threading layer accordingly. Set MKL_SERVICE_FORCE_INTEL to force it.

Check available datasets

Checks if we can load all files listed for a given dataset (all subsets in all protocols).

$ bob tb dataset check --help
Error: mkl-service + Intel(R) MKL: MKL_THREADING_LAYER=INTEL is incompatible with libgomp.so.1 library.
	Try to import numpy first or set the threading layer accordingly. Set MKL_SERVICE_FORCE_INTEL to force it.

Preset Configuration Resources

A CLI application allows one to list, inspect and copy available configuration resources exported by this package.

$ bob tb config --help
Error: mkl-service + Intel(R) MKL: MKL_THREADING_LAYER=INTEL is incompatible with libgomp.so.1 library.
	Try to import numpy first or set the threading layer accordingly. Set MKL_SERVICE_FORCE_INTEL to force it.

Listing Resources

$ bob tb config list --help
Error: mkl-service + Intel(R) MKL: MKL_THREADING_LAYER=INTEL is incompatible with libgomp.so.1 library.
	Try to import numpy first or set the threading layer accordingly. Set MKL_SERVICE_FORCE_INTEL to force it.

Available Resources

Here is a list of all resources currently exported.

$ bob tb config list -v
Error: mkl-service + Intel(R) MKL: MKL_THREADING_LAYER=INTEL is incompatible with libgomp.so.1 library.
	Try to import numpy first or set the threading layer accordingly. Set MKL_SERVICE_FORCE_INTEL to force it.

Describing a Resource

$ bob tb config describe --help
Error: mkl-service + Intel(R) MKL: MKL_THREADING_LAYER=INTEL is incompatible with libgomp.so.1 library.
	Try to import numpy first or set the threading layer accordingly. Set MKL_SERVICE_FORCE_INTEL to force it.

Applications for experiments

These applications allow to run every step of the experiment cycle. They also work well with our preset configuration resources.

Training CNNs or shallow networks

Training creates of a new PyTorch model. This model can be used for inference.

$ bob tb train --help
Error: mkl-service + Intel(R) MKL: MKL_THREADING_LAYER=INTEL is incompatible with libgomp.so.1 library.
	Try to import numpy first or set the threading layer accordingly. Set MKL_SERVICE_FORCE_INTEL to force it.

Prediction with CNNs or shallow networks

Inference takes as input a PyTorch model and generates output probabilities. The generated csv file indicates from 0 to 1 (floating-point number), the probability of TB presence on a chest X-ray, from less probable (0.0) to more probable (1.0).

$ bob tb predict --help
Error: mkl-service + Intel(R) MKL: MKL_THREADING_LAYER=INTEL is incompatible with libgomp.so.1 library.
	Try to import numpy first or set the threading layer accordingly. Set MKL_SERVICE_FORCE_INTEL to force it.

CNN Performance Evaluation

Evaluation takes inference results and compares it to ground-truth, generating measure files and score tables which are useful to understand model performance.

$ bob tb evaluate --help
Error: mkl-service + Intel(R) MKL: MKL_THREADING_LAYER=INTEL is incompatible with libgomp.so.1 library.
	Try to import numpy first or set the threading layer accordingly. Set MKL_SERVICE_FORCE_INTEL to force it.

Performance Comparison

Performance comparison takes the prediction results and generate combined figures and tables that compare results of multiple systems.

$ bob tb compare --help
Error: mkl-service + Intel(R) MKL: MKL_THREADING_LAYER=INTEL is incompatible with libgomp.so.1 library.
	Try to import numpy first or set the threading layer accordingly. Set MKL_SERVICE_FORCE_INTEL to force it.

Converting predictions to JSON dataset

This script takes radiological signs predicted on a TB dataset and generate a new JSON dataset from them.

$ bob tb predtojson --help
Error: mkl-service + Intel(R) MKL: MKL_THREADING_LAYER=INTEL is incompatible with libgomp.so.1 library.
	Try to import numpy first or set the threading layer accordingly. Set MKL_SERVICE_FORCE_INTEL to force it.

Aggregate multiple prediction files together

This script takes a list of prediction files and aggregate them into a single file. This is particularly useful for cross-validation.

$ bob tb aggregpred --help
Error: mkl-service + Intel(R) MKL: MKL_THREADING_LAYER=INTEL is incompatible with libgomp.so.1 library.
	Try to import numpy first or set the threading layer accordingly. Set MKL_SERVICE_FORCE_INTEL to force it.