Python API

bob.devtools.log	Logging utilities.
bob.devtools.ci	Tools to help CI-based builds and artifact deployment.
bob.devtools.constants	Constants used for building and more.
bob.devtools.release	Utilities to needed to release packages.
bob.devtools.changelog	Utilities for retrieving, parsing and auto-generating changelogs.
bob.devtools.bootstrap	Bootstraps a new miniconda installation and prepares it for development.
bob.devtools.build	Tools for self-building and other utilities.
bob.devtools.mirror	Mirroring functionality for conda channels
bob.devtools.deploy	Deployment utilities for conda packages and documentation via webDAV.
bob.devtools.graph	Utilities for calculating package dependencies and drawing graphs

Detailed Information

Logging utilities.

bob.devtools.log.COLORMAP = {'critical': {'attrs': ['bold'], 'color': 'red'}, 'debug': {}, 'error': {'color': 'red'}, 'exception': {'attrs': ['bold'], 'color': 'red'}, 'info': {'attrs': ['bold']}, 'warn': {'attrs': ['bold'], 'color': 'yellow'}, 'warning': {'attrs': ['bold'], 'color': 'yellow'}}

Default color map for homogenized color display

class bob.devtools.log.ColorLog(logger)

Bases: object

Colorizes logging colors.

bob.devtools.log.get_logger(name)

Returns the default logger as setup by this module.

bob.devtools.log.echo_normal(text)

Color preset for normal text output for click.echo()

bob.devtools.log.echo_info(text)

Color preset for normal text output for click.echo()

bob.devtools.log.echo_warning(text)

Color preset for normal warning output for click.echo()

bob.devtools.log.setup(logger_name, format='%(levelname)s:%(name)s@%(asctime)s: %(message)s')

This function returns a logger object that is set up to perform logging using Bob loggers.

Parameters:

• logger_name (str) – The name of the module to generate logs for

• format (str, optional) – The format of the logs, see logging.LogRecord for more details. By default, the log contains the logger name, the log time, the log level and the massage.

Returns:

logger – The logger configured for logging. The same logger can be retrieved using the logging.getLogger() function.

Return type:

logging.Logger

bob.devtools.log.set_verbosity_level(logger, level)

Sets the log level for the given logger.

Parameters:

• logger (logging.Logger or str) – The logger to generate logs for, or the name of the module to generate logs for.

• level (int) – Possible log levels are: 0: Error; 1: Warning; 2: Info; 3: Debug.

Raises:

ValueError – If the level is not in range(0, 4).

bob.devtools.log.verbosity_option(**kwargs)

Adds a -v/–verbose option to a click command.

Parameters:

**kwargs – All kwargs are passed to click.option.

Returns:

A decorator to be used for adding this option.

Return type:

callable

Tools to help CI-based builds and artifact deployment.

bob.devtools.ci.is_master(refname, tag, repodir)

Tells if we’re on the master branch via ref_name or tag.

This function checks if the name of the branch being built is “master”. If a tag is set, then it checks if the tag is on the master branch. If so, then also returns True, otherwise, False.

Parameters:

• refname – The value of the environment variable CI_COMMIT_REF_NAME

• tag – The value of the environment variable CI_COMMIT_TAG - (may be None)

Returns: a boolean, indicating we’re building the master branch or that the tag being built was issued on the master branch.

bob.devtools.ci.is_private(baseurl, package)

Tells if a given package (with a namespace) is public or private

This function checks if a fully qualified package in the format <namespace>/<name> is publicly accessible. It does this by trying to access info/refs?service=git-upload-pack from the package in question.

This method does not rely on the fact the user has access to Gitlab.

Warning

This method only works for fully qualified package names (i.e., containing at least one forward-slash /).

Parameters:

• baseurl – The base URL for the gitlab service to consult

• package – Fully qualified (i.e., with a namespace) package name. For example: bob/bob.extension.

Returns: a boolean, indicating if the package is private (True) or not (False).

bob.devtools.ci.is_stable(package, refname, tag, repodir)

Determines if the package being published is stable.

This is done by checking if a tag was set for the package. If that is the case, we still cross-check the tag is on the “master” branch. If everything checks out, we return True. Else, False.

Parameters:

• package – Package name in the format “group/name”

• refname – The current value of the environment CI_COMMIT_REF_NAME

• tag – The current value of the enviroment CI_COMMIT_TAG (may be None)

• repodir – The directory that contains the clone of the git repository

Returns: a boolean, indicating if the current build is for a stable release

bob.devtools.ci.read_packages(filename)

Return a python list of tuples (repository, branch), given a file containing one package (and branch) per line.

Comments are excluded

bob.devtools.ci.select_build_file(basename, paths, branch)

Selects the file to use for a build.

This method will return the name of the most adequate build-accessory file (conda_build_config.yaml, recipe_append.yaml) for a given build, in this order of priority:

1. The first file found is returned

2. We first search for a specific file if branch is set

3. If that file does not exist, returns the unbranded filename if that exists in one of the paths

4. If no candidates exists, returns None

The candidate filename is built using os.path.splitext(os.path.basename(basename))[0].

Parameters:

• basename – Name of the file to use for the search

• paths (list) – A list of paths leading to the location of the variants file to use. Priority is given to paths that come first

• branch (str) – Optional key to be set when searching for the variants file to use. This is typically the git-branch name of the current branch of the repo being built.

Returns:

A string containing the full, resolved path of the file to use. Returns None, if no candidate is found

Return type:

str

bob.devtools.ci.select_conda_build_config(paths, branch)

Selects the default conda_build_config.yaml.

See select_build_file() for implementation details. If no build config file is found by select_build_file(), then returns the default conda_build_config.yaml shipped with this package.

bob.devtools.ci.select_conda_recipe_append(paths, branch)

Selects the default recipe_append.yaml.

See select_build_file() for implementation details. If no recipe append file is found by select_build_file(), then returns the default recipe_append.yaml shipped with this package.

bob.devtools.ci.select_user_condarc(paths, branch)

Selects the user condarc file to read (if any)

See select_build_file() for implementation details. If no recipe condarc is found by select_build_file(), then returns None.

bob.devtools.ci.cleanup(dry_run, username, password, includes)

Cleans-up WebDAV resources. Executes if dry_run==False only.

Parameters:

• dry_run (bool) – If set, then does not execute any action, just print what it would do instead.

• username (str) – The user to use for interacting with the WebDAV service

• password (str) – Password the the above user

• includes (re.SRE_Pattern) – A regular expression that matches the names of packages that should be considered for clean-up. For example: for Bob and BATL packages, you may use ^(bob|batl|gridtk).* For BEAT packages you may use ^beat.*

bob.devtools.ci.temporary_cwd(path)

Temporarily changes the working directory to a given path

A context manager that temporarily changes the path to the given directory. The working directory changes the working directory back to the starting directory when it exits.

Parameters:

path – The directory to temporarily change to

Constants used for building and more.

bob.devtools.constants.BASE_CONDARC = "add_pip_as_python_dependency: false #!final\nalways_yes: true #!final\nanaconda_upload: false #!final\nchannel_priority: strict #!final\nconda_build: #!final\n  pkg_format: '2'\ndefault_channels: #!final\n  - https://repo.anaconda.com/pkgs/main\nchannels:\n  - conda-forge\nchannel_alias: https://conda.anaconda.org #!final\nquiet: true #!final\nremote_connect_timeout_secs: 120.0 #!final\nremote_max_retries: 50 #!final\nremote_read_timeout_secs: 180.0 #!final\nshow_channel_urls: true #!final\nssl_verify: false #!final\n"

Default setup for conda builds

bob.devtools.constants.CONDA_BUILD_CONFIG = '/scratch/builds/bob/bob.devtools/miniconda/conda-bld/bob.devtools_1678185904023/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pla/lib/python3.10/site-packages/bob/devtools/data/conda_build_config.yaml'

Configuration variants we like building

bob.devtools.constants.CONDA_RECIPE_APPEND = '/scratch/builds/bob/bob.devtools/miniconda/conda-bld/bob.devtools_1678185904023/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pla/lib/python3.10/site-packages/bob/devtools/data/recipe_append.yaml'

Extra information to be appended to every recipe upon building

bob.devtools.constants.WEBDAV_PATHS = {False: {False: {'conda': '/conda/label/beta', 'docs': '/docs', 'root': '/private-upload'}, True: {'conda': '/conda/label/beta', 'docs': '/docs', 'root': '/public-upload'}}, True: {False: {'conda': '/conda', 'docs': '/docs', 'root': '/private-upload'}, True: {'conda': '/conda', 'docs': '/docs', 'root': '/public-upload'}}}

Default locations of our webdav upload paths

bob.devtools.constants.CACERT_URL = 'https://curl.haxx.se/ca/cacert.pem'

Location of the most up-to-date CA certificate bundle

bob.devtools.constants.CACERT = '/scratch/builds/bob/bob.devtools/miniconda/conda-bld/bob.devtools_1678185904023/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pla/lib/python3.10/site-packages/bob/devtools/data/cacert.pem'

We keep a copy of the CA certificates we trust here

To update this file use: curl --remote-name --time-cond cacert.pem https://curl.haxx.se/ca/cacert.pem

More information here: https://curl.haxx.se/docs/caextract.html

bob.devtools.constants.MATPLOTLIB_RCDIR = '/scratch/builds/bob/bob.devtools/miniconda/conda-bld/bob.devtools_1678185904023/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pla/lib/python3.10/site-packages/bob/devtools/data'

Base directory where the file matplotlibrc lives

It is required for certain builds that use matplotlib functionality.

bob.devtools.constants.BOBRC_PATH = '/scratch/builds/bob/bob.devtools/miniconda/conda-bld/bob.devtools_1678185904023/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pla/lib/python3.10/site-packages/bob/devtools/data/bobrc'

The path to custom Bob configuration file to be used during the CI

Utilities to needed to release packages.

bob.devtools.release.download_path(package, path, output=None, ref='master')

Downloads paths from gitlab, with an optional recurse.

This method will download an archive of the repository from chosen reference, and then it will search inside the zip blob for the path to be copied into output. It uses zipfile.ZipFile to do this search. This method will not be very efficient for larger repository references, but works recursively by default.

Parameters:

• package – the gitlab package object to use (should be pre-fetched)

• path – the path on the project to download

• output – where to place the path to be downloaded - if not provided, use the basename of path as storage point with respect to the current directory

• ref – the name of the git reference (branch, tag or commit hash) to use

bob.devtools.release.get_gitlab_instance()

Returns an instance of the gitlab object for remote operations.

bob.devtools.release.get_latest_tag_name(gitpkg)

Find the name of the latest tag for a given package in the format ‘#.#.#’.

Parameters:

gitpkg – gitlab package object

Returns: The name of the latest tag in format ‘#.#.#’. None if no tags for the package were found.

bob.devtools.release.get_parsed_tag(gitpkg, tag)

An older tag is formatted as ‘v2.1.3 (Sep 22, 2017 10:37)’, from which we need only v2.1.3.

The latest tag is either patch, minor, major, or none

bob.devtools.release.update_tag_comments(gitpkg, tag_name, tag_comments_list, dry_run=False)

Write annotations inside the provided tag of a given package.

Parameters:

• gitpkg – gitlab package object

• tag_name – The name of the tag to update

• tag_comments_list – New annotations for this tag in a form of list

• dry_run – If True, nothing will be committed or pushed to GitLab

Returns: The gitlab object for the tag that was updated

bob.devtools.release.update_files_with_mr(gitpkg, files_dict, message, branch, automerge, dry_run, user_id)

Update (via a commit) files of a given gitlab package, through an MR.

This function can update a file in a gitlab package, but will do this through a formal merge request. You can auto-merge this request

Parameters:

• gitpkg – gitlab package object

• files_dict – Dictionary of file names and their contents (as text)

• message – Commit message

• branch – The branch name to use for the merge request

• automerge – If we should set the “merge if build succeeds” flag on the created MR

• dry_run – If True, nothing will be pushed to gitlab

• user_id – The integer which numbers the user to attribute this MR to

bob.devtools.release.update_files_at_master(gitpkg, files_dict, message, dry_run)

Update (via a commit) files of a given gitlab package, directly on the master branch.

Parameters:

• gitpkg – gitlab package object

• files_dict – Dictionary of file names and their contents (as text)

• message – Commit message

• dry_run – If True, nothing will be committed or pushed to GitLab

bob.devtools.release.get_last_pipeline(gitpkg)

Returns the last pipeline of the project.

Parameters:

gitpkg – gitlab package object

Returns: The gitlab object of the pipeline

bob.devtools.release.just_build_package(gitpkg, dry_run=False)

Creates the pipeline with the latest tag and starts it.

Parameters:

• gitpkg – gitlab package object

• dry_run – If True, the pipeline will not be created on GitLab

Returns:

bob.devtools.release.wait_for_pipeline_to_finish(gitpkg, pipeline_id, dry_run=False)

Using sleep function, wait for the latest pipeline to finish building.

This function pauses the script until pipeline completes either successfully or with error.

Parameters:

• gitpkg – gitlab package object

• pipeline_id – id of the pipeline for which we are waiting to finish

• dry_run – If True, outputs log message and exit. There wil be no waiting.

bob.devtools.release.cancel_last_pipeline(gitpkg)

Cancel the last started pipeline of a package.

Parameters:

gitpkg – gitlab package object

bob.devtools.release.release_package(gitpkg, tag_name, tag_comments_list, dry_run=False)

Release package.

The provided tag will be annotated with a given list of comments. README.rst and version.txt files will also be updated according to the release procedures.

Parameters:

• gitpkg – gitlab package object

• tag_name – The name of the release tag

• tag_comments_list – New annotations for this tag in a form of list

• dry_run – If True, nothing will be committed or pushed to GitLab

bob.devtools.release.parse_and_process_package_changelog(gl, gitpkg, package_changelog, dry_run)

Process the changelog of a single package.

Parse the log following specific format. Update annotations of the provided older tags and release the package by following the last tag description.

Parameters:

• gl – Gitlab API object

• gitpkg – gitlab package object

• package_changelog – the changelog corresponding to the provided package

• dry_run – If True, nothing will be committed or pushed to GitLab

Returns: the name of the latest tag, and tag’s comments

Utilities for retrieving, parsing and auto-generating changelogs.

bob.devtools.changelog.parse_date(d)

Parses any date supported by dateutil.parser.parse()

bob.devtools.changelog.get_file_from_gitlab(gitpkg, path, ref='master')

Retrieves a file from a Gitlab repository, returns a (StringIO) file.

bob.devtools.changelog.get_last_tag(package)

Returns the last (gitlab object) tag for the given package.

Parameters:

package – The gitlab project object from where to fetch the last release date information

Returns: a tag object

bob.devtools.changelog.get_last_tag_date(package)

Returns the last release date for the given package.

Falls back to the first commit date if the package has not yet been tagged

Parameters:

package – The gitlab project object from where to fetch the last release date information

Returns: a datetime object that refers to the last date the package was

released. If the package was never released, then returns the date just before the first commit.

bob.devtools.changelog.get_changes_since(gitpkg, since)

Gets the list of MRs, tags, and commits since the provided date

Parameters:

• gitpkg (object) – A gitlab package object

• since (object) – A parsed date

Returns:

mrs, tags, commits

Return type:

tuple

bob.devtools.changelog.write_tags_with_commits(f, gitpkg, since, mode)

Writes all tags and commits of a given package to the output file.

Parameters:

• f – A File ready to be written at

• gitpkg – A pointer to the gitlab package object

• since – Starting date (as a datetime object)

• mode – One of mrs (merge-requests), commits or tags indicating how to list entries in the changelog for this package

bob.devtools.changelog.write_tags(f, gitpkg, since)

Writes all tags of a given package to the output file.

Parameters:

• f – A File ready to be written at

• gitpkg – A pointer to the gitlab package object

• since – Starting date as a datetime object

Bootstraps a new miniconda installation and prepares it for development.

bob.devtools.bootstrap.set_environment(name, value, env=environ({'DIRTY': '', 'NOSE_EVAL_ATTR': '', 'CI_PROJECT_NAME': 'bob.devtools', 'CI_API_V4_URL': 'https://gitlab.idiap.ch/api/v4', 'CI_COMMIT_SHORT_SHA': 'e2831e84', 'CI_DEPENDENCY_PROXY_USER': 'gitlab-ci-token', 'CI_COMMIT_REF_PROTECTED': 'false', 'CPU_COUNT': '40', 'CI_REGISTRY': 'docker.idiap.ch', 'supkg': 'su-exec', 'HOSTNAME': 'runner-h1nkcwbs-project-2853-concurrent-4', 'CI_BUILD_TOKEN': '64_qbG71Su3Z8Q_G7Ehysyi', 'CI_DEPENDENCY_PROXY_PASSWORD': '64_qbG71Su3Z8Q_G7Ehysyi', 'CI_PROJECT_URL': 'https://gitlab.idiap.ch/bob/bob.devtools', 'CI_PIPELINE_CREATED_AT': '2023-03-07T10:40:15Z', 'CI_PROJECT_VISIBILITY': 'public', 'CI_COMMIT_TAG_MESSAGE': '', 'FF_NETWORK_PER_BUILD': 'false', 'FF_USE_NEW_SHELL_ESCAPE': 'false', 'CI_JOB_URL': 'https://gitlab.idiap.ch/bob/bob.devtools/-/jobs/316534', 'pin_run_as_build': "OrderedDict([('python', {'min_pin': 'x.x', 'max_pin': 'x.x'}), ('r-base', {'min_pin': 'x.x', 'max_pin': 'x.x'})])", 'CONDA_PERL': '5.26.2', 'PYTHONNOUSERSITE': '1', 'HISTSIZE': '1000', 'CI_REGISTRY_USER': 'gitlab-ci-token', 'CI_DEPENDENCY_PROXY_DIRECT_GROUP_IMAGE_PREFIX': 'gitlab.idiap.ch:443/bob/dependency_proxy/containers', 'c_compiler': 'gcc', 'CI_SERVER_PROTOCOL': 'https', 'RUNNER_TEMP_PROJECT_DIR': '/scratch/builds/bob/bob.devtools.tmp', 'CONDA_SHLVL': '3', 'FF_USE_LEGACY_KUBERNETES_EXECUTION_STRATEGY': 'false', 'CI_BUILD_BEFORE_SHA': '0000000000000000000000000000000000000000', 'PYTHONUNBUFFERED': '1', 'CONDA_PROMPT_MODIFIER': '(/scratch/builds/bob/bob.devtools/miniconda/conda-bld/bob.devtools_1678185904023/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pla) ', 'GSETTINGS_SCHEMA_DIR_CONDA_BACKUP': '', 'CI_BUILD_ID': '316534', 'CI_SERVER_VERSION': '15.8.4', 'GITLAB_USER_LOGIN': 'ydayer', 'PYPIPASS': 'pwd4Biometrics', 'BOB_BUILD_NUMBER': '0', 'GITLAB_USER_EMAIL': 'yannick.dayer@idiap.ch', 'ignore_build_only_deps': "['numpy', 'python']", 'FF_SCRIPT_SECTIONS': 'false', 'CI_PROJECT_ROOT_NAMESPACE': 'bob', 'DISTRO_NAME': 'centos', 'CI_COMMIT_AUTHOR': 'Yannick DAYER <yannick.dayer@idiap.ch>', 'CONDA_OVERRIDE_CUDA': '11.6', 'LC_ALL': 'en_US.UTF-8', 'CI_COMMIT_TITLE': 'Increased stable version to 5.4.2', 'CI_DISPOSABLE_ENVIRONMENT': 'true', 'CI_RUNNER_EXECUTABLE_ARCH': 'linux/amd64', 'CI_SERVER_TLS_CA_FILE': '/scratch/builds/bob/bob.devtools.tmp/CI_SERVER_TLS_CA_FILE', 'CI_JOB_NAME_SLUG': 'build-linux-bob-devtools', 'CI_SERVER_VERSION_PATCH': '4', 'CI_COMMIT_REF_NAME': 'v5.4.2', 'PERL_VER': '5.26', 'CI_SERVER_VERSION_MINOR': '8', 'USER': 'conda', 'CI_JOB_IMAGE': 'quay.io/condaforge/linux-anvil-cos7-x86_64', 'CI_JOB_TOKEN': '64_qbG71Su3Z8Q_G7Ehysyi', 'CI_PROJECT_ID': '2853', 'CMAKE_GENERATOR': 'Unix Makefiles', 'PY_VER': '3.10', 'CI_RUNNER_ID': '81', 'CI_RUNNER_REVISION': 'd540b510', 'DISTRO_VER': '7', 'cran_mirror': 'https://cran.r-project.org', 'REQUESTS_CA_BUNDLE': '', 'FF_DISABLE_UMASK_FOR_DOCKER_EXECUTOR': 'false', 'CONDA_EXE': '/scratch/builds/bob/bob.devtools/miniconda/bin/conda', 'NPY_VER': '1.16', 'CI_SERVER_VERSION_MAJOR': '15', 'CI_RELEASE_DESCRIPTION': "- bob/bob.devtools!331 [doc] Moves CI installation information to bob/private>\n- bob/bob.devtools!332 Adding xgboost library: As per request from @smichel, adding xgboost library\n- bob/bob.devtools!334 Fix changed runner tags: This MR fixes the clause matching `${CI_RUNNER_TAGS}` in the CI of multiple build files to better match GitLab's runner variables.  Apparently this stopped working after the last upgrade.\n- bob/bob.devtools!333 [bootstrap,deploy] Fix bootstrap and deployment base servers for (internal) CI builds\n- bob/bob.devtools!335 Adding openpyxl/shap deps: As requested by @smichel:  adding openpyxl and shap libaru\n- bob/bob.devtools!316 Use XDG_CACHE_HOME for defining cache directory for torch, pip and pre-commit: This MR just replaces the various cache variables by a central `XDG_CACHE_HOME`, that is respected (at least) by pip, pre-commit and torch.  Cache paths were updated accordingly preserving the current behaviour.\n- bob/bob.devtools!336 Fix reference of tags creating nested lists: The `!...", 'CONDA_BUILD_STATE': 'TEST', 'CONDA_R': '3.5', 'FF_ENABLE_BASH_EXIT_CODE_CHECK': 'false', 'CI_COMMIT_DESCRIPTION': '', 'CI_SERVER_URL': 'https://gitlab.idiap.ch', 'CI_COMMIT_MESSAGE': 'Increased stable version to 5.4.2', 'PKG_BUILDNUM': '0', 'CI_PROJECT_TITLE': 'bob.devtools', 'CI_PROJECT_REPOSITORY_LANGUAGES': 'python', 'CI_DEPENDENCY_PROXY_SERVER': 'gitlab.idiap.ch:443', 'PYPIUSER': 'biometrics', 'CI_PIPELINE_ID': '70826', 'CI_BUILD_REF': 'e2831e84bc465ed9b5df45e7be4f9c923f34d479', 'CI_BUILD_REF_NAME': 'v5.4.2', '_CE_CONDA': '', 'CONDA_BUILD': '1', 'FF_DISABLE_POWERSHELL_STDIN': 'false', 'CI_PAGES_DOMAIN': 'example.com', 'CI_COMMIT_REF_SLUG': 'v5-4-2', 'CI_BUILD_TAG': 'v5.4.2', 'CONDA_ROOT': '/scratch/builds/bob/bob.devtools/miniconda', 'CONDA_PREFIX_1': '/opt/conda', 'PATH': '/scratch/builds/bob/bob.devtools/miniconda/conda-bld/bob.devtools_1678185904023/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pla:/scratch/builds/bob/bob.devtools/miniconda/conda-bld/bob.devtools_1678185904023/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pla/bin:/scratch/builds/bob/bob.devtools/miniconda/conda-bld/bob.devtools_1678185904023/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pla/bin:/opt/conda/condabin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/home/conda/bin', 'CI_REPOSITORY_URL': 'https://gitlab-ci-token:64_qbG71Su3Z8Q_G7Ehysyi@gitlab.idiap.ch/bob/bob.devtools.git', 'GITLAB_USER_NAME': 'Yannick DAYER', 'MAIL': '/var/spool/mail/conda', 'GSETTINGS_SCHEMA_DIR': '/scratch/builds/bob/bob.devtools/miniconda/conda-bld/bob.devtools_1678185904023/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pla/share/glib-2.0/schemas', 'fortran_compiler': 'gfortran', 'SRC_DIR': '/scratch/builds/bob/bob.devtools/miniconda/conda-bld/bob.devtools_1678185904023/test_tmp', 'CI_PROJECT_DIR': '/scratch/builds/bob/bob.devtools', 'CI_REGISTRY_PASSWORD': '64_qbG71Su3Z8Q_G7Ehysyi', 'CI_BUILD_STAGE': 'build', '_': '/scratch/builds/bob/bob.devtools/miniconda/conda-bld/bob.devtools_1678185904023/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pla/bin/sphinx-build', 'CONDA_PREFIX_2': '/scratch/builds/bob/bob.devtools/miniconda', 'FF_USE_DIRECT_DOWNLOAD': 'true', 'CI_RUNNER_TAGS': '["bob", "linux", "docker", "intel"]', 'BUILD': 'x86_64-conda_cos6-linux-gnu', 'LD_RUN_PATH': '/scratch/builds/bob/bob.devtools/miniconda/conda-bld/bob.devtools_1678185904023/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pla/lib', 'CI_NODE_TOTAL': '1', 'CONDA_PREFIX': '/scratch/builds/bob/bob.devtools/miniconda/conda-bld/bob.devtools_1678185904023/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pla', 'PWD': '/scratch/builds/bob/bob.devtools/miniconda/conda-bld/bob.devtools_1678185904023/test_tmp', 'cpu_optimization_target': 'nocona', 'CI_PROJECT_NAMESPACE_ID': '373', 'CI_TEMPLATE_REGISTRY_HOST': 'registry.gitlab.com', 'target_platform': 'linux-64', 'PKG_VERSION': '5.4.2', 'PY3K': '1', 'FF_POSIXLY_CORRECT_ESCAPES': 'false', 'CI_JOB_STAGE': 'build', 'CI_SERVER_HOST': 'gitlab.idiap.ch', 'CI_PAGES_URL': 'http://bob.example.com/bob.devtools', 'CI_PIPELINE_SOURCE': 'push', 'LUA_VER': '5', 'STDLIB_DIR': '/scratch/builds/bob/bob.devtools/miniconda/conda-bld/bob.devtools_1678185904023/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pla/lib/python3.10', 'CI_SERVER_NAME': 'GitLab', 'CI_PROJECT_PATH': 'bob/bob.devtools', 'LANG': 'en_US.UTF-8', 'CONDA_PY': '310', 'GITLAB_FEATURES': '', 'CI_PROJECT_DESCRIPTION': 'Tools for development and CI integration of Bob/BEAT packages', 'CI_COMMIT_BEFORE_SHA': '0000000000000000000000000000000000000000', 'DOCPASS': 'zXmol5PPQz', 'SHLIB_EXT': '.so', 'CI_PIPELINE_IID': '1891', 'extend_keys': "['pin_run_as_build', 'ignore_build_only_deps', 'ignore_version', 'extend_keys']", 'FF_ENABLE_JOB_CLEANUP': 'false', 'CI_PIPELINE_URL': 'https://gitlab.idiap.ch/bob/bob.devtools/-/pipelines/70826', 'ROOT': '/scratch/builds/bob/bob.devtools/miniconda', 'GITLAB_CI': 'true', 'CI_SERVER_PORT': '443', 'CI_RUNNER_VERSION': '15.9.1', 'HTTPS_PROXY': '', 'CI_SERVER_REVISION': '54f2a13176e', 'BUILD_EGG': 'true', 'BUILD_PREFIX': '/scratch/builds/bob/bob.devtools/miniconda/conda-bld/bob.devtools_1678185904023/_build_env', 'PYTEST_ADDOPTS': "-m ''", 'FF_RESOLVE_FULL_TLS_CHAIN': 'true', 'CI_COMMIT_SHA': 'e2831e84bc465ed9b5df45e7be4f9c923f34d479', 'HISTCONTROL': 'ignoredups', '_CE_M': '', 'CI_BUILDS_DIR': '/scratch/builds', 'PKG_BUILD_STRING': 'py310_0', 'FF_CMD_DISABLE_DELAYED_ERROR_LEVEL_EXPANSION': 'false', 'CI_BUILD_NAME': 'build_linux_bob_devtools', 'CI_CONFIG_PATH': '.gitlab-ci.yml', 'HOME': '/home/conda', 'SHLVL': '3', 'build_platform': 'linux-64', 'FF_SKIP_NOOP_BUILD_STAGES': 'true', 'CI_SERVER': 'yes', 'CI_PROJECT_PATH_SLUG': 'bob-bob-devtools', 'FF_USE_POWERSHELL_PATH_RESOLVER': 'false', 'FF_USE_IMPROVED_URL_MASKING': 'false', 'CI_RUNNER_SHORT_TOKEN': 'h1nkcWBs', 'CI': 'true', 'cxx_compiler': 'gxx', 'SP_DIR': '/scratch/builds/bob/bob.devtools/miniconda/conda-bld/bob.devtools_1678185904023/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pla/lib/python3.10/site-packages', 'HTTP_PROXY': '', 'CONDA_ALLOW_SOFTLINKS': 'false', 'CI_JOB_STATUS': 'running', 'CI_DEFAULT_BRANCH': 'master', 'r_base': '3.5', 'SYS_PREFIX': '/scratch/builds/bob/bob.devtools/miniconda', 'CI_PROJECT_NAMESPACE': 'bob', 'XDG_CACHE_HOME': '/scratch/builds/bob/bob.devtools/cache', 'CONDA_NPY': '1.16', 'SUBDIR': 'linux-64', 'PYPI_PACKAGE_REGISTRY_TOKEN': 'glpat-Q161VQfr-1qn1_1ssVHi', 'LOGNAME': 'conda', 'CONDA_PYTHON_EXE': '/scratch/builds/bob/bob.devtools/miniconda/bin/python', 'R_VER': '3.5', 'NPY_DISTUTILS_APPEND_FLAGS': '1', 'PREFIX': '/scratch/builds/bob/bob.devtools/miniconda/conda-bld/bob.devtools_1678185904023/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pla', 'CI_CONCURRENT_ID': '4', 'CI_JOB_TIMEOUT': '10800', 'CI_COMMIT_TIMESTAMP': '2023-03-07T10:40:03+00:00', 'RECIPE_DIR': '/tmp/tmp083_p6kn/info/recipe', 'CONDA_LUA': '5', 'CI_JOB_STARTED_AT': '2023-03-07T10:41:28Z', 'CI_PROJECT_CLASSIFICATION_LABEL': '', 'PKG_CONFIG_PATH': '/scratch/builds/bob/bob.devtools/miniconda/conda-bld/bob.devtools_1678185904023/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pla/lib/pkgconfig', 'FF_USE_NEW_BASH_EVAL_STRATEGY': 'false', 'CI_BUILD_REF_SLUG': 'v5-4-2', 'CONDA_DEFAULT_ENV': '/scratch/builds/bob/bob.devtools/miniconda/conda-bld/bob.devtools_1678185904023/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pla', 'ARCH': '64', 'BOB_PACKAGE_VERSION': '5.4.2', 'FF_USE_FASTZIP': 'false', 'CI_DEPENDENCY_PROXY_GROUP_IMAGE_PREFIX': 'gitlab.idiap.ch:443/bob/dependency_proxy/containers', 'CI_RUNNER_DESCRIPTION': 'linux-docker', 'FF_KUBERNETES_HONOR_ENTRYPOINT': 'false', 'GITLAB_USER_ID': '609', 'DOCUSER': 'uploader', 'PKG_NAME': 'bob.devtools', 'SYS_PYTHON': '/scratch/builds/bob/bob.devtools/miniconda/bin/python3', 'CI_JOB_ID': '316534', 'PKG_HASH': '', 'CI_COMMIT_TAG': 'v5.4.2', 'PYTHON': '/scratch/builds/bob/bob.devtools/miniconda/conda-bld/bob.devtools_1678185904023/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pla/bin/python', 'FF_USE_WINDOWS_LEGACY_PROCESS_STRATEGY': 'true', 'CI_CONCURRENT_PROJECT_ID': '4', 'CI_JOB_NAME': 'build_linux_bob_devtools', 'FF_USE_DYNAMIC_TRACE_FORCE_SEND_INTERVAL': 'false', 'DOCUTILSCONFIG': '/scratch/builds/bob/bob.devtools/miniconda/conda-bld/bob.devtools_1678185904023/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pla/share/doc/bob.devtools/doc/docutils.conf'}))

Function to setup the environment variable and print debug message.

Parameters:

• name – The name of the environment variable to set

• value – The value to set the environment variable to

• env – Optional environment (dictionary) where to set the variable at

bob.devtools.bootstrap.human_time(seconds, granularity=2)

Returns a human readable time string like “1 day, 2 hours”.

bob.devtools.bootstrap.run_cmdline(cmd, env=None, **kwargs)

Runs a command on a environment, logs output and reports status.

Parameters:

• cmd (list) – The command to run, with parameters separated on a list

• env (dict, Optional) – Environment to use for running the program on. If not set, use os.environ.

bob.devtools.bootstrap.touch(path)

Python-implementation of the “touch” command-line application.

bob.devtools.bootstrap.merge_conda_cache(cache, prefix, name)

Merges conda pkg caches and conda-bld folders.

Parameters:

• cache – The cached directory (from previous builds)

• prefix – The current prefix (root of conda installation)

• name – The name of the current package

bob.devtools.bootstrap.ensure_miniconda_sh()

Retrieves the miniconda3 installer for the current system.

Checks the hash of the miniconda3 installer against the expected version, if that does not match, erase existing installer and re-downloads new installer.

bob.devtools.bootstrap.install_miniconda(prefix, name)

Creates a new miniconda installation.

Parameters:

• prefix – The path leading to the (new) root of the miniconda installation

• name – The name of this package

bob.devtools.bootstrap.get_channels(public, stable, server, intranet, group, add_dependent_channels=False)

Returns the relevant conda channels to consider if building project.

The subset of channels to be returned depends on the visibility and stability of the package being built. Here are the rules:

• public and stable: returns the public stable channel

• public and not stable: returns the public beta channel

• not public and stable: returns both public and private stable channels

• not public and not stable: returns both public and private beta channels

Public channels have priority over private channles, if turned.

Parameters:

• public – Boolean indicating if we’re supposed to include only public channels

• stable – Boolean indicating if we’re supposed to include stable channels

• server – The base address of the server containing our conda channels

• intranet – Boolean indicating if we should add “private”/”public” prefixes on the conda paths

• group – The group of packages (gitlab namespace) the package we’re compiling is part of. Values should match URL namespaces currently available on our internal webserver. Currently, only “bob” or “beat” will work.

• add_dependent_channels – If True, will add the conda-forge channel to the list

Returns: a list of channels that need to be considered.

bob.devtools.bootstrap.setup_logger(logger, level)

Sets-up the logging for this command at level INFO

Tools for self-building and other utilities.

bob.devtools.build.root_logger_protection()

Protects the root logger against spurious (conda) manipulation

bob.devtools.build.comment_cleanup(lines)

Cleans-up comments and empty lines from textual data read from files.

bob.devtools.build.load_order_file(path)

Loads an order.txt style file, removes empty lines and comments.

bob.devtools.build.conda_arch()

Returns the current OS name and architecture as recognized by conda.

bob.devtools.build.should_skip_build(metadata_tuples)

Takes the output of render_recipe as input and evaluates if this recipe’s build should be skipped.

bob.devtools.build.next_build_number(channel_url, basename)

Calculates the next build number of a package given the channel.

This function returns the next build number (integer) for a package given its resulting tarball base filename (can be obtained with get_output_path()).

Parameters:

• channel_url – The URL where to look for packages clashes (normally a beta channel)

• basename – The tarball basename to check on the channel

Returns: The next build number with the current configuration. Zero (0) is returned if no match is found. Also returns the URLs of the packages it finds with matches on the name, version and python-version, ordered by (reversed) build-number.

bob.devtools.build.make_conda_config(config, python, append_file, condarc_options)

Creates a conda configuration for a build merging various sources.

This function will use the conda-build API to construct a configuration by merging different sources of information.

Parameters:

• config – Path leading to the conda_build_config.yaml to use

• python – The version of python to use for the build as x.y (e.g. 3.6)

• append_file – Path leading to the recipe_append.yaml file to use

• condarc_options – A dictionary (typically read from a condarc YAML file) that contains build and channel options

Returns: A dictionary containing the merged configuration, as produced by conda-build API’s get_or_merge_config() function.

bob.devtools.build.get_output_path(metadata, config)

Renders the recipe and returns the name of the output file.

bob.devtools.build.use_mambabuild()

Will inject mamba solver to conda build API to speed up resolves

bob.devtools.build.get_rendered_metadata(recipe_dir, config)

Renders the recipe and returns the interpreted YAML file.

bob.devtools.build.get_parsed_recipe(metadata)

Renders the recipe and returns the interpreted YAML file.

bob.devtools.build.exists_on_channel(channel_url, basename)

Checks on the given channel if a package with the specs exist.

This procedure always ignores the package hash code, if one is set. It differentiates between .conda and .tar.bz2 packages.

Parameters:

• channel_url – The URL where to look for packages clashes (normally a beta channel)

• basename – The basename of the tarball to search for

Returns: A complete package url, if the package already exists in the channel or None otherwise.

bob.devtools.build.remove_pins(deps)

bob.devtools.build.uniq(seq, idfun=None)

Very fast, order preserving uniq function.

bob.devtools.build.parse_dependencies(recipe_dir, config)

bob.devtools.build.get_env_directory(conda, name)

Get the directory of a particular conda environment or fail silently.

bob.devtools.build.conda_create(conda, name, overwrite, condarc, packages, dry_run, use_local)

Creates a new conda environment following package specifications.

This command can create a new conda environment following the list of input packages. It will overwrite an existing environment if indicated.

Parameters:

• conda – path to the main conda executable of the installation

• name – the name of the environment to create or overwrite

• overwrite – if set to `True, overwrite potentially existing environments with the same name

• condarc – a dictionary of options for conda, including channel urls

• packages – the package list specification

• dry_run – if set, then don’t execute anything, just print stuff

• use_local – include the local conda-bld directory as a possible installation channel (useful for testing multiple interdependent recipes that are built locally)

bob.devtools.build.get_docserver_setup(public, stable, server, intranet, group)

Returns a setup for BOB_DOCUMENTATION_SERVER.

BOB_DOCUMENTATION_SERVER is used in bob.extension.utils.link_documentation to find the links of the documentation of Bob packages. The initial servers in the output servers list have higher priority.

What is available to build the documentation depends on the setup of public and stable:

• public and stable: only returns the public stable folder

• public and not stable: only returns the public beta folder

• not public and stable: returns public and private stable folders

• not public and not stable: returns public and private beta folders

Will also look for a sphinx directory.

If returned, Private channels have lower priority over public channels because, private packages can enventually become public.

Parameters:

• public – Boolean indicating if we’re supposed to include only public channels

• stable – Boolean indicating if we’re supposed to include only stable channels

• server – The base address of the server containing our conda channels

• intranet – Boolean indicating if we should add “private”/”public” prefixes on the returned paths

• group – The group of packages (gitlab namespace) the package we’re

• compiling – is part of. Values should match URL namespaces currently available on our internal webserver. Currently, only “bob” or “beat” will work.

Returns: a string to be used by bob.extension to find dependent documentation projects.

bob.devtools.build.check_version(workdir, envtag)

Checks if the version being built and the value reported match.

This method will read the contents of the file version.txt and compare it to the potentially set envtag (may be None). If the value of envtag is different than None, ensure it matches the value in version.txt or raises an exception.

Parameters:

• workdir – The work directory where the repo of the package being built was checked-out

• envtag – (optional) tag provided by the environment

Returns: A tuple with the version of the package that we’re currently building and a boolean flag indicating if the version number represents a pre-release or a stable release.

bob.devtools.build.git_clean_build(runner, verbose)

Runs git-clean to clean-up build products.

Parameters:

• runner – A pointer to the run_cmdline() function

• verbose – A boolean flag indicating if the git command should report erased files or not

bob.devtools.build.base_build(bootstrap, server, intranet, group, recipe_dir, conda_build_config, condarc_options)

Builds a non-beat/non-bob software dependence that doesn’t exist on conda-forge.

This function will build a software dependence that is required for our software stack, but does not (yet) exist on the conda-forge channels. It first check if the build should run for the current architecture, checks if the package is not already built on our public channel and, if that is true, then proceeds with the build of the dependence.

Parameters:

• bootstrap – Module that should be pre-loaded so this function can be used in a pre-bdt build

• server – The base address of the server containing our conda channels

• intranet – Boolean indicating if we should add “private”/”public” prefixes on the returned paths

• group – The group of packages (gitlab namespace) the package we’re compiling is part of. Values should match URL namespaces currently available on our internal webserver. Currently, only “bob” or “beat” will work.

• recipe_dir – The directory containing the recipe’s meta.yaml file

• conda_build_config – Path to the conda_build_config.yaml file to use

• condarc_options – Pre-parsed condarc options loaded from the respective YAML file

Returns:

The list of built packages, as returned by conda_build.api.build()

Return type:

list

bob.devtools.build.load_packages_from_conda_build_config(conda_build_config, condarc_options, with_pins=False)

bob.devtools.build.bob_devel(bootstrap, server, intranet, group, conda_build_config, condarc_options, work_dir)

Tests that all packages listed in bob/devtools/data/conda_build_config.yaml can be installed in one environment.

Mirroring functionality for conda channels

Some constructs are bluntly copied from https://github.com/valassis-digital-media/conda-mirror

bob.devtools.mirror.get_json(channel, platform, name)

Get a JSON file for a channel/platform combo on conda channel

Parameters:

• channel (str) – Complete channel URL

• platform ({'linux-64', 'linux-aarch64', 'osx-64', 'osx-arm64', 'noarch'}) – The platform of interest

• name (str) – The name of the file to retrieve. If the name ends in ‘.bz2’, then it is auto-decompressed

Returns:

repodata – contents of repodata.json

Return type:

dict

Raises:

RuntimeError : – If the URL cannot be reached

bob.devtools.mirror.get_local_contents(path, arch)

Returns the local package contents as a set

bob.devtools.mirror.load_glob_list(path)

Loads a list of globs from a configuration file

Excludes comments and empty lines

bob.devtools.mirror.blacklist_filter(packages, globs)

Filters out the input package set with the glob list

bob.devtools.mirror.whitelist_filter(packages, globs)

Filters in the input package set with the glob list

bob.devtools.mirror.download_packages(packages, repodata, channel_url, dest_dir, arch, dry_run)

Downloads remote packages to a download directory

Packages are downloaded first to a temporary directory, then validated according to the expected sha256/md5 sum and then moved, one by one, to the destination directory. An error is raised if the package cannot be correctly downloaded.

Parameters:

• packages (list of str) – List of packages to download from the remote channel

• repodata (dict) – A dictionary containing the remote repodata.json contents

• channel_url (str) – The complete channel URL

• dest_dir (str) – The local directory where the channel is being mirrored

• arch (str) – The current architecture which we are mirroring

• dry_run (bool) – A boolean flag indicating if this is just a dry-run (simulation), flagging so we don’t really do anything (set to True).

bob.devtools.mirror.remove_packages(packages, dest_dir, arch, dry_run)

Removes local packages that no longer matter

bob.devtools.mirror.copy_and_clean_json(url, dest_dir, arch, name, dry_run)

Copies and cleans conda JSON file

bob.devtools.mirror.copy_and_clean_patch(url, dest_dir, arch, name, dry_run)

Copies and cleans conda patch_instructions JSON file

bob.devtools.mirror.checksum_packages(repodata, dest_dir, arch, packages)

Checksums packages on the local mirror and compare to remote repository

Parameters:

• repodata (dict) – Data loaded from repodata.json on the remote repository

• dest_dir (str) – Path leading to local mirror

• arch (str) – Current architecture being considered (e.g. noarch, linux-64, linux-aarch64, osx-64, osx-arm64)

• packages (list) – List of packages that are available locally, by name

Returns:

issues – List of matching errors

Return type:

list

Deployment utilities for conda packages and documentation via webDAV.

bob.devtools.deploy.deploy_conda_package(package, arch, stable, public, username, password, overwrite, dry_run)

Deploys a single conda package on the appropriate path.

Parameters:

• package (str) – Path leading to the conda package to be deployed

• arch (str) – The conda architecture to deploy to (linux-64, linux-aarch64, osx-64, osx-arm64, noarch, or None - in which case the architecture is going to be guessed from the directory where the package sits)

• stable (bool) – Indicates if the package should be deployed on a stable (True) or beta (False) channel

• public (bool) – Indicates if the package is supposed to be distributed publicly or privatly (within Idiap network)

• username (str) – The name of the user on the webDAV server to use for uploading the package

• password (str) – The password of the user on the webDAV server to use for uploading the package

• overwrite (bool) – If we should overwrite a package with equal name existing on the destination directory. Otherwise, an exception is raised.

• dry_run (bool) – If we’re supposed to really do the actions, or just log messages.

bob.devtools.deploy.deploy_documentation(path, package, stable, latest, public, branch, tag, username, password, dry_run)

Deploys sphinx documentation to the appropriate webdav locations.

Parameters:

• path (str) – Path leading to the root of the documentation to be deployed

• package (str) – Full name (with namespace) of the package being treated

• stable (bool) – Indicates if the documentation corresponds to the latest stable build

• latest (bool) – Indicates if the documentation being deployed correspond to the latest stable for the package or not. In case the documentation comes from a patch release which is not on the master branch, please set this flag to False, which will make us avoid deployment of the documentation to master and stable sub-directories.

• public (bool) – Indicates if the documentation is supposed to be distributed publicly or privatly (within Idiap network)

• branch (str) – The name of the branch for the current build

• tag (str) – The name of the tag currently built (may be None)

• username (str) – The name of the user on the webDAV server to use for uploading the package

• password (str) – The password of the user on the webDAV server to use for uploading the package

• dry_run (bool) – If we’re supposed to really do the actions, or just log messages.

Utilities for calculating package dependencies and drawing graphs

bob.devtools.graph.compute_adjencence_matrix(gl, package, conda_config, main_channel, recurse_regexp='^(bob|beat|batl|gridtk)(\\.)?(?!-).*$', current={}, ref='master', deptypes=[])

Given a target package, returns an adjacence matrix with its dependencies returned via the conda-build API

Parameters:

• gl (object) – Pre-instantiated instance of the gitlab server API to use, of type gitlab.Gitlab.

• package (str) – Name of the package, including its group in the format group/package

• conda_config (dict) – Dictionary of conda configuration options loaded from command-line and read from defaults available.

• main_channel (str) – Main channel to consider when looking for the next build number of the target package

• recurse_regexp (str) – Regular expression to use, for determining where to recurse for resolving dependencies. Typically, this should be set to a list of packages which exists in gitlab. If it includes more than that, then we may not be able to reach the package repository and an error will be raised. The default expression avoids recursing over bob/beat-devel packages.

• current (dict) – Current list of packages already inspected - useful for recurrent calls to this function, to avoid potential cyclic dependencies. Corresponds to the current return value of this function.

• ref (str) – Name of the git reference (branch, tag or commit hash) to use

• deptypes (list) – A list of dependence types to preserve when building the graph. If empty, then preserve all. You may set values “build”, “host”, “run” and “test”, in any combination

Returns:

adjacence_matrix – A dictionary that contains the dependencies of all packages considered in the recursion. The keys are the name of the packages, the values, correspond to the dependencies (host, build, run and test) as a list of strings.

Return type:

dict

bob.devtools.graph.generate_graph(adjacence_matrix, deptypes, whitelist)

Computes a graphviz/dot representation of the build graph

Parameters:

• adjacence_matrix (dict) – A dictionary containing the adjacence matrix, that states the dependencies for each package in the build, to other packages

• deptypes (list) – A list of dependence types to preserve when building the graph. If empty, then preserve all. You may set values “build”, “host”, “run” and “test”, in any combination

• whitelist (str) – Regular expression for matching strings to preserve while building the graph

Returns:

graph – The generated graph

Return type:

graphviz.Digraph