CI framework internals

The _CI/ directory is the vendored Invoke-based workflow framework behind ./workflow.cmd. This page is the lookup reference for its building blocks — for the task catalog see Invoke task catalog, for the design rationale see The _CI tasks architecture.

Shared utilities (_CI/tasks/shared.py)

Utility Purpose
execute(ctx, cmd) Run a shell command; raise SystemExit(1) on failure
@run(cmd) Decorator — replace function body with a shell command
@logged(name) Decorator — print pass/fail status after execution
run_steps(*fns) Run all steps, accumulate failures, raise once at the end

Configuration (_CI/tasks/configuration.py)

Centralized constants shared across task modules:

Constant Purpose
PATHS Standard directories targeted by linters/formatters: src/ _CI/tasks/ tests/
SECURITY_OVERRIDE_ENV Environment variable name for security audit overrides
IGNORE_PATTERN Regex for parsing vulnerability IDs with optional expiry dates
LOCAL_MODULE Path to _CI/tasks/local.py, the project-owned task module (in __init__.py)
IMAGE_NAME Container image name for the deps cache
DOCKERFILE_DEPS Dockerfile for the deps image (also a deps-image tag input)
DEPS_IMAGE_FILE Where container.publish records the resolved image reference
PYSCN_REPORTS_DIR Directory for pyscn HTML reports
SENTINEL Bootstrap sentinel file path

The deps image and its cache

It runs as a non-root user

Dockerfile.deps ends with USER 1001, so nothing in CI runs as root. Two things have to line up for that to work, and both are in the Dockerfile:

  • /app is chowned to 1001. The image is built with --no-install-project, so uv run installs the project into UV_PROJECT_ENVIRONMENT (/app/.venv) at job time. Left root-owned, every task would fail on the first write.
  • The user has a home directory. uv caches under $HOME.

The uid is not arbitrary. 1001 is the uid GitHub's hosted runners own the mounted workspace with. If your runner uses a different uid, the workspace arrives unwritable and jobs fail with permission errors on paths under the checkout — change the uid in Dockerfile.deps to match. That edit changes the deps-image tag automatically, because the Dockerfile is one of the tag's inputs, so you will not be served a stale image built for the old uid.

CI jobs run inside a prebuilt image holding the project's dependencies, so each job starts with a warm environment instead of resolving from scratch. container.publish builds and publishes it; .deps-image records the reference downstream jobs consume.

The tag is content-addressed. deps_image_tag() hashes the lockfile, Dockerfile.deps and the resolved base-image string together (length-prefixed, so no combination of inputs can collide with another). Change any one of them and you get a new tag — which is what stops CI from silently reusing an image built from a stale Dockerfile or base.

The reference is digest-pinned. container.publish resolves the published image to repository@sha256:… (via the local repo digest) rather than handing downstream jobs a mutable tag. A tag repointed between the build job and a consumer job therefore cannot change the container that consumer runs in.

Residual risk — reuse of an existing tag. When the tag already exists in the registry, the build is skipped and that image is reused. This keeps CI fast, but it trusts whoever pushed the tag first: an actor able to pre-seed <repo>-deps:<tag> would have later jobs run inside their image, including the publish job that holds PyPI OIDC. Digest pinning does not remove this — you would pin their digest.

Because the mitigation is an access-control one, keep it that way:

  • Restrict write access to the <repo>-deps package to this pipeline. Only build-deps-image needs packages: write; every other job runs at packages: read.
  • Audit who else can publish packages in the org, and treat that set as trusted to the same degree as the release credentials.
  • To remove the trade-off entirely, drop the manifest inspect skip in publish_deps_image() so every run rebuilds from the repository. The cost is a full image build per gated run.

Bootstrap framework (_CI/tasks/bootstrap.py)

A sentinel file (_CI/.bootstrapped) ensures first-time setup runs exactly once; every task inherits bootstrap as an Invoke pre task. New setup steps are added by appending to the STEPS list:

STEPS: list[BootstrapStep] = [
    BootstrapStep(
        name='pre-commit hooks',
        action=install_pre_commit,
        prompt='Install pre-commit hooks? [y/N] ',
        ci_behavior='skip',  # 'run' or 'skip' in CI
    ),
    # Add more steps here
]

Each step has a prompt for local interactive use and a ci_behavior for unattended execution — when the CI environment variable is set, prompts are suppressed and steps execute or skip accordingly. See Customize the bootstrap for the recipe.

Directory layout

_CI/
  tasks/
    __init__.py        Namespace aggregation + bootstrap wiring
    configuration.py   Centralized constants
    shared.py          Core decorators and utilities
    bootstrap.py       One-time setup framework
    build.py           Package build
    container.py       Docker image
    develop.py         Pre-commit management
    document.py        ProperDocs documentation
    format_.py         Ruff formatting + import sorting
    lint.py            Ruff, pylint, ty, complexipy, commitizen
    quality.py         pyscn analysis
    release.py         Version bump, changelog, publish
    secure.py          pip-audit, CycloneDX SBOM
    test.py            pytest
    github.py          Host-specific helpers (registry, release PR/MR)
    local.py           YOUR tasks — never overwritten by `copier update`
  lib/
    vendor/            Vendored Invoke + dependencies (committed — don't hand-edit)
    vendor.txt         Pinned dependency list

Everything in that tree is template-owned and replaced on copier update — except local.py. It is listed in the template's _skip_if_exists, so it is seeded once and then left alone. Put project-specific tasks there and updates stay conflict-free; put them in any other module and that hunk needs resolving on every update. __init__.py includes it automatically when present, with the same bootstrap pre-task the built-in namespaces get, so there is no registration step. See Add a workflow task.

See also