`neu-eval`¶

From neu-eval. Comparison metrics between a segmentation and a reference labeling, and a ranked list of where the two disagree.

python -m neu_eval.cli is equivalent.

compare reads both sides as two registered pieces — the segmentation is read with crop=<the reference piece>, so the reference’s physical box is converted using the segmentation level’s own voxel size and origin rather than an assumed 2**level factor — scores them, and writes four things:

file
`summary.parquet`	the metrics, one row per piece (plus a pooled row under `--all-datasets`)
`per_label.parquet`	each reference body’s share of the merge error and each segment’s share of the split error
`disagreements.csv`	ranked worst-first, with a `verdict` column to fill in, and a `.README.txt` legend beside it
`annotations.csv`	the same points as a coordinate list for `neu-glance annotate`

The disagreement table round-trips. A reference is not always right: a manual annotation and an algorithm disagree both because the algorithm is wrong and because the annotation is. So you fill in verdict while looking at the voxels and pass the file back with --adjudicated, which reports a second score excluding the pairs you judged to be reference defects — alongside the raw one, together with how much of the region each was computed over. Only <segmentation>_correct leaves the denominator; ambiguous is still scored, because “I could not tell” is not evidence the reference was wrong.

The points become local annotations, and that is deliberate. A ranked worst-first list is something you step through, and a precomputed annotation source renders in the viewport but puts zero rows in the Annotations tab — so [ and ] do not step. neu-glance annotate inlines them in the state instead. See the viewing guide.

Run neu-vol relabel on a multi-crop reference first. Every annotated region numbers its bodies from 1, so the same integer names a different cell in each crop; untreated, a low-numbered “body” is a chimera of unrelated cells and every metric is meaningless with nothing to show it. compare warns when reference labels are scattered far beyond their own size, which is what that looks like. For the same reason --all-datasets pools rather than sums — each crop’s labels are renumbered into their own range first, so pooled ids are positions in a concatenation and name no bodies.

Compare a segmentation against a reference labeling.

Produces the standard metrics (variation of information with its split/merge decomposition, adapted Rand, IoU matching, fragmentation, completeness, covering) and a ranked, located list of the disagreements behind them.

Because a reference is not always right, the disagreement table round-trips: it goes out with a blank verdict column, you fill it in while looking at the voxels, and passing it back with –adjudicated reports a second score that excludes the pairs you judged to be reference defects — alongside the raw one, never instead of it.

usage: neu-eval [-h] [--version] [--store-logs] COMMAND ...

Positional Arguments¶

COMMAND: Possible choices: compare, help

Named Arguments¶

--version

show program’s version number and exit

--store-logs

do not filter the store’s benign credential-provider chatter

Default: False

Sub-commands¶

compare¶

Read two labelings of the same physical box and score them.

The segmentation is read with crop=<the reference piece>, so the two are registered by construction — the reference’s physical box is converted using the segmentation level’s own voxel size and origin, never an assumed 2**level factor, because real pyramids are anisotropic.

neu-eval compare [-h] --reference SRC --segmentation SRC [--level LEVEL]
                 [--reference-level REFERENCE_LEVEL] [--all-datasets]
                 [--out DIR] [--labels A,B] [--ignore-reference IDS]
                 [--ignore-segmentation IDS] [--top TOP] [--min-frac MIN_FRAC]
                 [--min-voxels MIN_VOXELS] [--rank {severity,fraction}]
                 [--point-at {seam,overlap}] [--no-locate] [--adjudicated CSV]
                 [--voxel-size Z,Y,X] [--dtype DTYPE] [--no-scatter-check]

Named Arguments¶

--reference

the reference labeling: PATH or PATH:/DATASET, or a source URL

--segmentation

the labeling being assessed, read registered to the reference

--level

level of the segmentation to read (default: 0)

Default: 0

--reference-level

level of the reference, for a multiscale reference (default: 0)

Default: 0

--all-datasets

score every array in the reference container, and pool them

Default: False

--out

where to write the results (default: the working directory)

Default: '.'

--labels

names for the two sides, used throughout the output (default: gt,seg)

Default: 'gt,seg'

--ignore-reference

comma-separated label values to drop on the reference side; 0 is unannotated, so it is the default. ‘’ keeps everything

Default: '0'

--ignore-segmentation

label values to drop on the segmentation side. Empty by default: a 0 there means ‘assigned to no body’ inside a region somebody did annotate, which is an error and not an absence

Default: ''

--top

how many disagreements to report (default: 200; 0 for all)

Default: 200

--min-frac

a pair must hold this fraction of either side to count as structural (default: 0.05). Below it is speck overlap, and these volumes have a great deal of that

Default: 0.05

--min-voxels

a pair must hold at least this many voxels (default: 1)

Default: 1

--rank

Possible choices: severity, fraction

order the disagreements by VOI contribution (default) or by how much of a body or segment the pair accounts for. Severity inherits VOI’s size weighting, so a large body losing a sliver outranks a small one cut in half; –rank fraction is the one to use when the list matters more than the metric

Default: 'severity'

--point-at

Possible choices: seam, overlap

put each point on the surface where the two labelings part company (default) or in the middle of what they agree about. A split’s seam is the false cut, a merge’s is the false join

Default: 'seam'

--no-locate

skip the per-pair distance transform, so no coordinates. Faster when only the scalars are wanted

Default: False

--adjudicated

a disagreement table with its verdict column filled in; adds the adjudicated score beside the raw one

--voxel-size

voxel size in nm for a reference that records none

--dtype

cast both sides to this on read (default: uint64), so crops exported by different tools compare

Default: 'uint64'

--no-scatter-check

skip the check for reference labels scattered far beyond their own size, which is what an un-relabelled multi-crop reference looks like

Default: False

examples:: # one annotated crop against a segmentation, read registered to it neu-eval compare –reference gt.h5:/crop_01 –segmentation SEG_URL –level 0 –out cmp/

# every crop in the container, plus a pooled score over all of them
neu-eval compare --reference gt.h5 --all-datasets --segmentation SEG_URL --out cmp/

# after filling in the verdict column, rescore without the reference's own errors
neu-eval compare --reference gt.h5:/crop_01 --segmentation SEG_URL \
    --adjudicated cmp/disagreements.csv --out cmp/

# step through the worst disagreements in a viewer. The coordinates are nanometres,
# so `annotate` needs a frame: --volume is better than --voxel-size, since it also
# bounds-checks. Then compose the layer with the volumes.
neu-glance annotate --points cmp/annotations.csv --nm --volume REF_URL \
    --name disagreements --format layer --out layer.json
neu-glance gen --image IMAGE_URL --seg REF_URL --seg SEG_URL --layer layer.json \
    --format url

help¶

show help for a command

neu-eval help [-h] [topic]

Positional Arguments¶

topic

examples:

one annotated crop against a segmentation, read registered to it

neu-eval compare –reference gt.h5:/crop_01 –segmentation SEG_URL –level 0 –out cmp/

every crop in the container, plus a pooled score over all of them

neu-eval compare –reference gt.h5 –all-datasets –segmentation SEG_URL –out cmp/

after filling in the verdict column, rescore without the reference’s own errors

neu-eval compare –reference gt.h5:/crop_01 –segmentation SEG_URL
–adjudicated cmp/disagreements.csv –out cmp/

step through the worst disagreements in a viewer. The coordinates are nanometres,

so annotate needs a frame: –volume is better than –voxel-size, since it also

bounds-checks. Then compose the layer with the volumes.

neu-glance annotate –points cmp/annotations.csv –nm –volume REF_URL
–name disagreements –format layer –out layer.json neu-glance gen –image IMAGE_URL –seg REF_URL –seg SEG_URL –layer layer.json
–format url

neu-eval¶

Positional Arguments¶

Named Arguments¶

Sub-commands¶

compare¶

Named Arguments¶

help¶

Positional Arguments¶

`neu-eval`¶