neu_morpho.ops.manifest¶

Manifest status policy shared by the ops.

is_complete(): resume skips successful tasks. blockrun’s Manifest.is_done tests key presence, so a recorded failed would read as done and never be retried.
guarded(): per-task fault isolation, for stage-2 workers only.
check_selection(): a resume may not ask stage 1 for bodies it already ran without.
check_grid(): nor resume stage 1 on a different block grid.

Stage-2 tasks are one body each and independent, so a bad one costs that body and is recorded. Stage-1 tasks are blocks, and stage 2 aggregates across them: a skipped block would truncate every body passing through it while the output looks complete. So stage 1 fails fast, and resume makes the relaunch cheap.

Attributes¶

FAILED

Exceptions¶

`GridChanged`	A resume would reuse completed stage-1 blocks on a different block grid.
`SelectionChanged`	A resume asks for bodies that stage 1's completed blocks were not run for.
`StageAborted`	Stage stopped early by the failure breaker (not a per-task failure).
`StaleManifest`	A manifest records completed work whose output is no longer there.

Classes¶

FailureBreaker

Stops a stage when failures stop looking like isolated bad bodies.

Functions¶

`check_grid`(manifest, group, grid, *, stage, ...) → None	Refuse a resume whose block grid or stage-1 settings (`grid_record`) differ from
`check_manifest_matches_output`(manifest, out_dir, *, ...) → None	Refuse to resume a manifest that has outlived the data it describes.
`check_selection`(manifest, group, allow, *, stage, ...) → None	Refuse a resume for bodies the completed stage-1 blocks were not run for.
`grid_changes`(before, now) → dict	`{field: (before, now)}` over the fields both records have: a record from before a
`grid_mismatch`(stage, where, complete, changes) → str
`grid_record`(block_shape, voxel_size, **settings) → dict	The block grid stage 1 runs on, and any `settings` its blocks' output depends on
`group_counts`(manifest, group) → dict[str, int]	Per-group status tallies.
`guarded`(worker, key) → tuple	Run a stage-2 worker, converting an exception into a `failed` result.
`is_complete`(manifest, group, key) → bool	Has this task finished successfully? (`failed` records must be retried.)
`is_systemic`(exc) → bool	Would this error recur for every remaining task?
`write_failures`(path, failures) → str \| None	Dump per-task failures as JSONL (tracebacks would bloat the run summary).

Module Contents¶

exception neu_morpho.ops.manifest.GridChanged¶

Bases: RuntimeError

A resume would reuse completed stage-1 blocks on a different block grid.

Initialize self. See help(type(self)) for accurate signature.

exception neu_morpho.ops.manifest.SelectionChanged¶

Bases: RuntimeError

A resume asks for bodies that stage 1’s completed blocks were not run for.

Initialize self. See help(type(self)) for accurate signature.

exception neu_morpho.ops.manifest.StageAborted¶

Bases: RuntimeError

Stage stopped early by the failure breaker (not a per-task failure).

Initialize self. See help(type(self)) for accurate signature.

exception neu_morpho.ops.manifest.StaleManifest¶

Bases: RuntimeError

A manifest records completed work whose output is no longer there.

Initialize self. See help(type(self)) for accurate signature.

class neu_morpho.ops.manifest.FailureBreaker(max_consecutive: int = 10)¶

Stops a stage when failures stop looking like isolated bad bodies.

Triggers: a systemic exception (is_systemic()), at once, or max_consecutive failures in a row (in completion order); a success resets the streak. It records a reason and check() raises, so a batch’s successes are applied first.

check() → None¶: Raise StageAborted if a trigger fired.

failure(key: Any, info: dict | None) → None¶

success() → None¶

consecutive = 0¶

max_consecutive = 10¶

reason: str | None = None¶

total = 0¶

neu_morpho.ops.manifest.check_grid(manifest: Any, group: str, grid: dict, *, stage: str, progress_path: str, record: bool) → None¶: Refuse a resume whose block grid or stage-1 settings (grid_record) differ from those its completed blocks ran with; record (this run processes stage-1 blocks) notes them in the manifest. A manifest from before they were recorded is not checked.

neu_morpho.ops.manifest.check_manifest_matches_output(manifest: Any, out_dir: str, *, stage: str, progress_path: str, resume: bool) → None¶

Refuse to resume a manifest that has outlived the data it describes.

The manifest lives in the work dir and the data possibly in an object store, so clearing the destination would leave a manifest saying every task is done: the run would write nothing and report success. The probe is the stage’s own info, not the segmentation’s, so meshing into a volume whose labels were never exported is not mistaken for a cleared destination.

Only checked when resuming; resume=False is an explicit fresh start.

neu_morpho.ops.manifest.check_selection(manifest: Any, group: str, allow: set[int] | None, *, stage: str, progress_path: str, record: bool) → None¶

Refuse a resume for bodies the completed stage-1 blocks were not run for.

A completed block is never revisited, so a body outside the selection it ran with gets no fragments: stage 2 finds nothing for it and the run reports success. A later run may ask for that selection or a subset of it. record is whether this run processes stage-1 blocks: its selection is then recorded in the manifest, since the remaining blocks see only that (None for every label).

neu_morpho.ops.manifest.grid_changes(before: dict, now: dict) → dict¶: {field: (before, now)} over the fields both records have: a record from before a field was recorded is not refused for it.

neu_morpho.ops.manifest.grid_mismatch(stage: str, where: str, complete: int, changes: dict) → str¶

neu_morpho.ops.manifest.grid_record(block_shape, voxel_size, **settings) → dict¶: The block grid stage 1 runs on, and any settings its blocks’ output depends on (a mesh’s smoothing and decimation), as recorded and compared.

neu_morpho.ops.manifest.group_counts(manifest: Any, group: str) → dict[str, int]¶

Per-group status tallies.

Manifest.counts() aggregates every group, conflating blocks with bodies.

neu_morpho.ops.manifest.guarded(worker: Callable[[Any], tuple], key: Any) → tuple¶

Run a stage-2 worker, converting an exception into a failed result.

Workers return (key, status, metrics, info); on failure info carries the traceback and a systemic flag for the breaker. KeyboardInterrupt and SystemExit are not caught.

neu_morpho.ops.manifest.is_complete(manifest: Any, group: str, key: Any) → bool¶: Has this task finished successfully? (failed records must be retried.)

neu_morpho.ops.manifest.is_systemic(exc: BaseException) → bool¶

Would this error recur for every remaining task?

MemoryError counts as systemic even though one enormous body can cause it: a process that has hit OOM is not worth trusting for the remaining tasks.

neu_morpho.ops.manifest.write_failures(path: str, failures: list[dict]) → str | None¶: Dump per-task failures as JSONL (tracebacks would bloat the run summary).

neu_morpho.ops.manifest.FAILED = 'failed'¶