blockrun.dask_config¶

Dask cluster config: bundled templates, layered overrides, key validation.

Bundled templates: --config dask-local resolves through importlib.resources, so it works from any directory, including for an installed command.
Layering: --config is repeatable and deep-merged left to right, so a site config holds only the keys that differ from a template:
```
--config dask-slurm-example --config configs/my-site.yaml
```
Validation: an unknown key would otherwise merge silently and do nothing, so each file is checked against dask’s own config tree (unknown_keys()).

Attributes¶

SUFFIX

Exceptions¶

ConfigKeyError

An unrecognised key in a dask config — almost always a typo.

Functions¶

`bundled_configs`() → list[str]	Names of the dask configs shipped with this package.
`deep_merge`(base, overlay) → dict	`overlay` over `base`: recurse into dicts, REPLACE everything else.
`load_config`(sources, *[, validate]) → tuple[dict, list[str]]	Resolve, load, validate and merge one or more `--config` values.
`resolve_config`(value) → str	One `--config` value -> a filesystem path.
`unknown_keys`(doc[, reference, _prefix]) → list[str]	Dotted paths in `doc` that dask's config tree does not define.
`write_effective_config`(dest, merged, sources) → str	Record the config dask actually received. Returns the path written.

Module Contents¶

exception blockrun.dask_config.ConfigKeyError¶

Bases: ValueError

An unrecognised key in a dask config — almost always a typo.

Initialize self. See help(type(self)) for accurate signature.

blockrun.dask_config.bundled_configs() → list[str]¶: Names of the dask configs shipped with this package.

blockrun.dask_config.deep_merge(base: Mapping[str, Any], overlay: Mapping[str, Any]) → dict¶

overlay over base: recurse into dicts, REPLACE everything else.

Lists are replaced, not concatenated — job-extra-directives and worker-extra-args are complete argument vectors, and appending to them produces a command line nobody wrote. Scalars replace so that spill: false can be overridden back to a number.

blockrun.dask_config.load_config(sources: str | Sequence[str], *, validate: bool = True) → tuple[dict, list[str]]¶

Resolve, load, validate and merge one or more --config values.

Returns (merged, resolved_paths). Each file is validated on its own before merging, so an error names the file that introduced the key.

blockrun.dask_config.resolve_config(value: str) → str¶

One --config value -> a filesystem path.

An existing path wins over a bundled name, so a local file can shadow a template and every previously-working invocation keeps working.

blockrun.dask_config.unknown_keys(doc: Mapping[str, Any], reference: Mapping[str, Any] | None = None, _prefix: tuple[str, ...] = ()) → list[str]¶

Dotted paths in doc that dask’s config tree does not define.

Names are canonicalised as dask does (log_directory is log-directory). Checking stops below an empty-dict default, which dask uses for free-form mappings such as distributed.worker.resources, and below scalar or list defaults.

blockrun.dask_config.write_effective_config(dest: str, merged: Mapping[str, Any], sources: Iterable[str]) → str¶

Record the config dask actually received. Returns the path written.

dest is a directory (the file is named dask-config.effective.yaml) or a full path. Kept with a run, it records what layered overrides produced.

blockrun.dask_config.SUFFIX = '.yaml'¶