blockrun.manifest¶

A completion log for resumable runs, written only by the driver.

As task results stream back, the driver appends one JSON line per task; on resume, callers read it and skip finished tasks before dispatch, with no per-object existence scan. Each line has:

group: a JSON-serialisable group id (a pyramid level, or 0 if there is one group);

key: the task’s identity, a scalar (a segment id) or a sequence (a block index);

status: any string (neu-vol writes written, empty, skipped);

t: when the driver recorded it, in Unix seconds, so a reader can measure a rate.

A line with meta and no status records facts about a group, most usefully its task total (Manifest.record_meta()), since only the driver knows it. Anything reading the file directly must skip lines without a status.

Classes¶

Manifest

Append-only JSONL record of completed tasks, keyed by (group, key).

Module Contents¶

class blockrun.manifest.Manifest(path: str | None)¶

Append-only JSONL record of completed tasks, keyed by (group, key).

close() → None¶

counts(group: Any = _ALL) → dict[str, int]¶: {status: count} over every group, or over one group if given.

done_keys(group: Any) → set¶: All recorded keys for group.

is_done(group: Any, key: Any) → bool¶: Whether any status is recorded for the task, including "failed". To retry failures, filter on status() instead.

load() → Manifest¶

meta(group: Any) → dict¶: Facts recorded about group (e.g. its task total); {} if none.

record(group: Any, results: Iterable[tuple[Any, str]]) → None¶: Record (key, status) results for a group; durable (flush+fsync).

record_meta(group: Any, **fields: Any) → None¶: Record facts about group, most usefully its task total. Call it before dispatching the group, so progress readers have the total from the start.

reset() → Manifest¶: Truncate the manifest (fresh run): clear in-memory + on-disk records.

status(group: Any, key: Any) → str | None¶

path¶