blockrun.engine¶

Block-map engine.

A job is a map over output chunks: tile the output grid into blocks (Block), and run a per-block function that reads the input it needs, transforms it, and writes its chunk. Tasks should be idempotent, so a relaunch resumes, and should return small status values rather than arrays: outputs go to the store, not through the scheduler.

Attributes¶

Region

Classes¶

Block

One output chunk: its grid index and its region in the output volume.

Functions¶

`block_map`(blocks, fn, *[, client, npartitions, on_result]) → list[Any]	Apply `fn` to each block, returning the per-block results.
`idempotent`(fn, is_done, *[, skipped]) → Callable[[Block], Any]	Wrap `fn` so blocks that are already done are skipped.
`iter_blocks`(shape, chunks) → Iterator[Block]	Tile `shape` into blocks of `chunks` (edge blocks are clipped).

Module Contents¶

class blockrun.engine.Block¶

One output chunk: its grid index and its region in the output volume.

index: tuple[int, ...]¶

region: Region¶

property shape: tuple[int, ...]¶

blockrun.engine.block_map(blocks: Sequence[Block], fn: Callable[[Block], Any], *, client: Any | None = None, npartitions: int | None = None, on_result: Callable[[list[Any]], None] | None = None) → list[Any]¶

Apply fn to each block, returning the per-block results.

fn must be top-level / picklable and should return a small status value (write big outputs to the store, not back through the scheduler).

on_result, if given, is called with each batch’s results as they arrive, e.g. to record them in a Manifest so a run that dies midway leaves an accurate record.

With client=None it runs serially in-process. Otherwise blocks are split round-robin into npartitions batches (default min(len(blocks), 512)), one task each, and results stream back through as_completed.

blockrun.engine.idempotent(fn: Callable[[Block], Any], is_done: Callable[[Block], bool], *, skipped: Any = 'skipped') → Callable[[Block], Any]¶

Wrap fn so blocks that are already done are skipped.

is_done runs on the worker, so keep it cheap (a store existence or metadata check).

blockrun.engine.iter_blocks(shape: Sequence[int], chunks: Sequence[int]) → Iterator[Block]¶

Tile shape into blocks of chunks (edge blocks are clipped).

Axis order is whatever shape/chunks use (canonical (c,) z, y, x).

blockrun.engine.Region¶