neu_eval.adjudicate¶

Recording that a disagreement was the reference’s fault, and rescoring without it.

A manual annotation can be wrong, and scoring against it there charges the segmentation for being right. So a disagreement table round-trips: it goes out with an empty verdict column, a reviewer fills it in, and it comes back as the basis for a second, adjudicated score.

Report both scores, and how much was excluded: an adjudicated score alone cannot be checked. apply() therefore moves excluded voxels into n_ignored rather than dropping them, and frac_scored appears in every summary. Verdicts say correct, not right, so they cannot be confused with left/right.

Attributes¶

`VERDICTS`
`VERDICT_COLUMN`

Functions¶

`apply`(c, verdicts, *[, exclude, labels]) → neu_eval.overlap.Contingency	`c` with the adjudicated pairs' voxels moved out of the denominator.
`check_verdicts`(verdicts, *[, labels]) → None	Raise on any verdict outside the vocabulary (blank is allowed). Otherwise a typo
`default_exclude`([labels]) → tuple[str, ...]	The verdicts whose voxels leave the denominator: only "the reference is wrong here",
`summarize`(verdicts, *[, labels]) → dict[str, int]	How many of each verdict, including how many rows are still unreviewed.
`verdict_names`([labels]) → tuple[str, ...]	The verdict vocabulary for a pair of side names; `VERDICTS` is the

Module Contents¶

neu_eval.adjudicate.apply(c: neu_eval.overlap.Contingency, verdicts: Mapping[str, str], *, exclude: Iterable[str] | None = None, labels: Sequence[str] | None = None) → neu_eval.overlap.Contingency¶

c with the adjudicated pairs’ voxels moved out of the denominator.

verdicts maps pair_key ("<a_id>:<b_id>") to a verdict. Pairs whose verdict is in exclude (by default, “the segmentation was right”) are dropped and their voxels added to n_ignored, so n_scored + n_ignored still totals the region.

This declines to score where the reference is wrong; it does not substitute the segmentation’s answer as a corrected reference, which would need the voxels. Unknown pair_key values are ignored, since a verdict file can outlive the table it came from (a different min_frac changes which pairs appear).

neu_eval.adjudicate.check_verdicts(verdicts: Mapping[str, str], *, labels: Sequence[str] = ('gt', 'seg')) → None¶: Raise on any verdict outside the vocabulary (blank is allowed). Otherwise a typo in a hand-edited column would silently count as “not excluded”.

neu_eval.adjudicate.default_exclude(labels: Sequence[str] = ('gt', 'seg')) → tuple[str, ...]¶: The verdicts whose voxels leave the denominator: only “the reference is wrong here”, spelled for the given side names (--labels v1,v2 gives v2_correct).

neu_eval.adjudicate.summarize(verdicts: Mapping[str, str], *, labels: Sequence[str] = ('gt', 'seg')) → dict[str, int]¶: How many of each verdict, including how many rows are still unreviewed.

neu_eval.adjudicate.verdict_names(labels: Sequence[str] = ('gt', 'seg')) → tuple[str, ...]¶: The verdict vocabulary for a pair of side names; VERDICTS is the ("gt", "seg") case.

neu_eval.adjudicate.VERDICTS = ('gt_correct', 'seg_correct', 'ambiguous', 'skip')¶

neu_eval.adjudicate.VERDICT_COLUMN = 'verdict'¶