`neu-vol`¶

From neu-vol. Everything to do with the volumes themselves.

python -m neu_vol is equivalent, which is occasionally useful when you want to be certain which installation you are running.

neu-vol: convert, downsample and inspect large 3D volumes, locally or on SLURM.

Every write is block-mapped over a dask cluster — local or SLURM, chosen by –config — and resumable, so an interrupted run continues rather than starting over. –config takes a bundled template name or a path and is repeatable, deep-merged left to right, so a site config carries only the keys that differ from a template.

convert and copy are one operation under two defaulting policies: convert states the output it wants, copy takes the source’s own format, chunking, voxel size and image/segmentation type. Either takes the whole volume or one –crop-bbox.

create and write are the small-pieces path and are not block-mapped: create lays out an empty volume, optionally copying a reference’s frame exactly, then each write places one image stack, HDF5 file or array into one level of it at a voxel offset. Single-scale by design — run downsample afterwards if the result needs a pyramid.

Anything a viewer consumes lives in neu-glance: neu-glance gen for a link or a state, bboxes for a layer of boxes over a sparse volume’s data, annotate for a layer of your own coordinates. This package supplies the occupancy analysis those build on (ops.occupancy.labeled_regions) and knows nothing about neuroglancer beyond it.

python -m neu_vol is equivalent. Run neu-vol <subcommand> --help for its arguments.

usage: neu-vol [-h]
               {info,convert,copy,downsample,create,write,progress,mask-by-value,to-hdf5,align-bbox,relabel,wk-mappings,wk-ls,help}
               ...

Positional Arguments¶

command: Possible choices: info, convert, copy, downsample, create, write, progress, mask-by-value, to-hdf5, align-bbox, relabel, wk-mappings, wk-ls, help

Sub-commands¶

info¶

Report a volume’s format, coordinate metadata and the levels present. Reads only.

Works on anything this package can read: a zarr or neuroglancer-precomputed volume, a dvid:// instance, an HDF5 file, or a directory or glob of 2D slices. The last two are a single array, so they report one level; an HDF5 file also reports whatever frame it records about itself, which is what neu-vol to-hdf5 writes.

neu-vol info [-h] [--dataset DATASET] [--provenance] [--store-logs] volume

Positional Arguments¶

volume

Named Arguments¶

--dataset

which array to describe inside an HDF5 container. Only needed when the file holds more than one 3D+ dataset — with one, it is found; with several, the error lists them. Ignored for every other format

--provenance

print the whole provenance.json record if the volume has one (a one-line summary is shown without this). Written by convert; it names the source a volume was made from — for a DVID export, the resolved node, which is the only way to say afterwards which proofreading snapshot this is

Default: False

--store-logs

Default: False

convert¶

Convert a source volume into a multiscale zarr and/or neuroglancer-precomputed volume, whole or cropped to a box. Resumable.

To copy a volume as it is — same format, chunking, voxel size and image/segmentation type — use neu-vol copy, which reads those from the source instead of defaulting them.

neu-vol convert [-h] [--src-format SRC_FORMAT] --src SRC [--src-level N]
                [--dvid-supervoxels] [--dvid-locked] [--wk-mapping NAME]
                [--wk-full-extent] --dst DST
                [--format {precomputed,zarr,both}]
                [--kind {image,probability,segmentation}] [--dtype TYPE]
                [--voxel-size VOXEL_SIZE] [--chunk CHUNK] [--shard SHARD]
                [--crop-bbox Z0,Y0,X0,Z1,Y1,X1]
                [--mask-bbox Z0,Y0,X0,Z1,Y1,X1] [--mask-value MASK_VALUE]
                [--background V[,V...]] [--bbox-scale N]
                [--bbox-order {zyx,xyz}] [--profile PROFILE]
                [--factors FACTORS] [--max-levels MAX_LEVELS]
                [--min-dim MIN_DIM] [--single-level] [--sparse] [--fresh]
                [--no-validate] [--config NAME_OR_PATH] [--workers WORKERS]
                [--serial] [--store-logs]

Named Arguments¶

--src-format

backend for –src (default: auto-detect). A dvid:// URL is detected by its scheme, a zarr or precomputed volume by its metadata document, and an HDF5 file or a directory/glob of ordered 2D slices (PNG/TIFF) by name — pass this to override, or to name a format whose files are called something unusual. An image stack always needs –voxel-size, since image files carry no physical scale

--src

path, s3://…, precomputed, .zarr, or a DVID labelmap as dvid://SERVER/UUID/INSTANCE — three segments, always: SERVER may carry a port (emdata3:8900) and defaults to http, UUID may be abbreviated or carry a branch (93fdbc:main), INSTANCE is the labelmap name. Use dvid+https:// for a TLS server. e.g. dvid://dvid.example.org/93fdbc:main/labels. Or a webknossos layer as wk://DATASET/LAYER (dataset name or id), with the server and token from ~/.config/webknossos/credentials.toml

--src-level

which scale of a multiscale source becomes the output’s level 0 (default: the finest one that actually stores data — an info may declare scales that were never written, and those read as the fill value everywhere). The output’s voxel size is that scale’s own, never an assumed 2**N

--dvid-supervoxels

read SUPERVOXELS rather than agglomerated bodies (DVID sources only). Default is bodies, i.e. the proofread segmentation

Default: False

--dvid-locked

pull the newest LOCKED node at or before the given ref, rather than the ref itself (DVID sources only). A branch ref such as 93fdbc:main resolves to the branch HEAD, which in a lock-and-spawn repo is the open, still-mutable node — legal to read, but its data can change while the run is going and the pull is not reproducible. Needs the repo:branch form, since walking ancestors uses DVID’s ref~N syntax. Either way the resolved node id is reported: convert and copy record it in provenance.json

Default: False

--wk-mapping

apply this mapping, e.g. an agglomerate view, on the server as the layer is read (webknossos sources only). Default: none, i.e. SUPERVOXELS — the raw ids — even when the viewer shows the layer through a default mapping. neu-vol info wk://… lists the mappings

--wk-full-extent

keep the layer’s full origin-anchored extent (webknossos sources only). Default: crop to the layer’s bounding box, keeping its frame, since everything outside it reads as zeros and would only add empty tasks. –crop-bbox overrides either

Default: False

--dst

destination base (path or s3://bucket/prefix). With –format both, ‘.precomputed’ and ‘.zarr’ are appended. For a DVID source it may contain placeholders resolved from the node actually exported: {uuid} (8 hex chars; {uuid:N} for N, {uuid:full} for all 32), {branch}, {instance} — so ‘seg_{uuid}’ names the export after the version it came from, which a branch ref alone does not

--format

Possible choices: precomputed, zarr, both

output format (default: precomputed)

Default: 'precomputed'

--kind

Possible choices: image, probability, segmentation

reducer for the pyramid: mean for image/probability, label-preserving mode for segmentation. Getting this wrong is silent — averaging label ids invents ids. Default: from the source where it records one, else image

--dtype

cast to this dtype on the way out (default: the source’s). Usually needed with –kind segmentation, whose compressed_segmentation encoding takes uint32/uint64 only. Narrowing is not checked

--voxel-size

z,y,x nm (default: from source)

--chunk

z,y,x chunk (default: 128,128,128)

Default: '128,128,128'

--shard

z,y,x shard (zarr only)

--crop-bbox

copy only this box instead of the whole volume, in voxels at –bbox-scale. Half-open, clipped to the volume. The output keeps the source’s frame — its physical offset shifts by the crop origin — so the two overlay in a viewer

--mask-bbox

EXCLUDE this box: copy everything else, and write –mask-value inside it. Repeatable. The hole is inherited by every pyramid level, since each is derived from the one below. Coordinates are the SOURCE’s, so a mask means the same box whether or not –crop-bbox is also given. A box that misses the volume is an error, not a no-op

--mask-value

what to write inside –mask-bbox (default: 0, which is background for a segmentation and is elided rather than stored)

Default: 0

--background

value(s) the SOURCE uses for background, replaced with 0 as it is read. Manual segmentation numbered from 0 makes background 1, and an all-background block of 1s is not all-fill, so without this every such block is stored and the volume stops answering ‘where is the data’ by which chunks exist

--bbox-scale

the scale EVERY bbox argument is given in (default: 0, level-0 voxels). Converted using the source’s own per-level voxel sizes, never an assumed 2**N, since real pyramids are anisotropic. The same six values name a different box at every scale, so check the resolved level-0 box that is logged

Default: 0

--bbox-order

Possible choices: zyx, xyz

the axis order of EVERY bbox argument (default: zyx, as everywhere else here). Pass xyz for numbers copied straight out of neuroglancer, which displays xyz; each corner is reversed, and the resolved zyx box is logged

Default: 'zyx'

--profile

storage profile name (default: chosen from –format and whether –dst is remote)

--factors

explicit per-level factors, e.g. ‘1,2,2;1,2,2’ (default auto). An explicit schedule is used verbatim: –max-levels and –min-dim do not apply to it

--max-levels

at most this many levels, COUNTING LEVEL 0 (default: 8, i.e. levels 0-7). Usually –min-dim stops the pyramid first

Default: 8

--min-dim

stop when the largest spatial dim is <= this

Default: 128

--single-level

write level 0 only, no pyramid

Default: False

--sparse

skip PYRAMID tasks whose input holds no stored chunk. On a sparse volume that is nearly all of them, and it is exact rather than a guess: an all-fill chunk is never stored, so a task with no stored input would write nothing anyway. One listing per level replaces the reads. It cannot skip any of the level-0 copy, whose source is foreign

Default: False

--fresh

delete and restart instead of resuming

Default: False

--no-validate

skip OME metadata validation

Default: False

--config

dask config: a bundled template (dask-local, dask-slurm-example) or a path to your own YAML. Repeatable, deep-merged left to right, so an overlay need only carry what differs. Default: dask-local

--workers

Default: 4

--serial

no dask at all — run in this process (smallest smoke test)

Default: False

--store-logs

keep TensorStore’s benign S3 credential-chain logging (suppressed by default; real errors are never suppressed)

Default: False

copy¶

Copy a volume, or a box out of it, keeping the source’s own format, chunking, voxel size and image/segmentation type.

The same operation as convert under a different defaulting policy: each of those parameters is read from the source rather than defaulted, and one the source does not record is an error rather than a guess. Pass any of them explicitly to override.

–crop-bbox copies one box instead of the whole volume; the output keeps the source’s coordinate frame, its physical offset shifted by the crop origin. –mask-bbox is the complement, copying everything except a box. The two compose, and coordinates for both are always the source’s.

The pyramid is rebuilt from the copied level 0; the source’s coarse levels are never read, so a whole-volume copy recomputes levels the source already has.

Resumable: re-run to continue.

neu-vol copy [-h] --src SRC [--src-level N] [--dvid-supervoxels]
             [--dvid-locked] [--wk-mapping NAME] [--wk-full-extent] --dst DST
             [--format {precomputed,zarr,both}]
             [--kind {image,probability,segmentation}] [--dtype TYPE]
             [--voxel-size VOXEL_SIZE] [--chunk CHUNK] [--shard SHARD]
             [--crop-bbox Z0,Y0,X0,Z1,Y1,X1] [--mask-bbox Z0,Y0,X0,Z1,Y1,X1]
             [--mask-value MASK_VALUE] [--background V[,V...]]
             [--bbox-scale N] [--bbox-order {zyx,xyz}] [--profile PROFILE]
             [--factors FACTORS] [--max-levels MAX_LEVELS] [--min-dim MIN_DIM]
             [--single-level] [--sparse] [--fresh] [--no-validate]
             [--config NAME_OR_PATH] [--workers WORKERS] [--serial]
             [--store-logs] [--dry-run]

Named Arguments¶

--src

--src-level

--dvid-supervoxels

read SUPERVOXELS rather than agglomerated bodies (DVID sources only). Default is bodies, i.e. the proofread segmentation

Default: False

--dvid-locked

Default: False

--wk-mapping

--wk-full-extent

Default: False

--dst

--format

Possible choices: precomputed, zarr, both

output format (default: from the source)

--kind

Possible choices: image, probability, segmentation

--dtype

cast to this dtype on the way out (default: the source’s). Usually needed with –kind segmentation, whose compressed_segmentation encoding takes uint32/uint64 only. Narrowing is not checked

--voxel-size

z,y,x nm (default: from source)

--chunk

z,y,x chunk (default: from the source)

--shard

z,y,x shard (zarr only) (default: from the source)

--crop-bbox

--mask-bbox

--mask-value

what to write inside –mask-bbox (default: 0, which is background for a segmentation and is elided rather than stored)

Default: 0

--background

--bbox-scale

Default: 0

--bbox-order

Possible choices: zyx, xyz

Default: 'zyx'

--profile

storage profile name (default: chosen from –format and whether –dst is remote)

--factors

explicit per-level factors, e.g. ‘1,2,2;1,2,2’ (default auto). An explicit schedule is used verbatim: –max-levels and –min-dim do not apply to it

--max-levels

at most this many levels, COUNTING LEVEL 0 (default: 8, i.e. levels 0-7). Usually –min-dim stops the pyramid first

Default: 8

--min-dim

stop when the largest spatial dim is <= this

Default: 128

--single-level

write level 0 only, no pyramid

Default: False

--sparse

Default: False

--fresh

delete and restart instead of resuming

Default: False

--no-validate

skip OME metadata validation

Default: False

--config

dask config: a bundled template (dask-local, dask-slurm-example) or a path to your own YAML. Repeatable, deep-merged left to right, so an overlay need only carry what differs. Default: dask-local

--workers

Default: 4

--serial

no dask at all — run in this process (smallest smoke test)

Default: False

--store-logs

keep TensorStore’s benign S3 credential-chain logging (suppressed by default; real errors are never suppressed)

Default: False

--dry-run

report what would be copied — resolved parameters, the box, the level shapes — and write nothing

Default: False

downsample¶

Regenerate a multiscale pyramid IN PLACE, from a level you trust.

Downsampling is cascaded (level N derives from N-1), so a bad level poisons every level above it. –start-level is read, never written; everything above it is rebuilt. This does not create a new volume — use convert for that.

neu-vol downsample [-h] [--start-level START_LEVEL]
                   [--kind {image,segmentation}] [--min-dim MIN_DIM]
                   [--max-levels MAX_LEVELS] [--factors FACTORS]
                   [--chunk CHUNK] [--voxel-size VOXEL_SIZE]
                   [--encoding ENCODING] [--profile PROFILE] [--resume]
                   [--sparse] [--progress-path PROGRESS_PATH] [--dry-run]
                   [--config NAME_OR_PATH] [--workers WORKERS] [--serial]
                   [--store-logs]
                   volume

Positional Arguments¶

volume: the multiscale volume to repair (path or s3://…)

Named Arguments¶

--start-level

level to derive from; it is read, never written (default: 0)

Default: 0

--kind

Possible choices: image, segmentation

reducer (default: read from the volume)

--min-dim

Default: 128

--max-levels

at most this many levels, COUNTING LEVEL 0. Defaults to as many as the volume records, since that is a property of the pyramid being repaired rather than a preference — pass it only to EXTEND a pyramid. –dry-run compares the schedule against the levels on disk and refuses if they disagree

--factors

explicit per-level factors, e.g. ‘1,2,2;1,2,2’. MUST match the original conversion, as must –min-dim/–max-levels

--chunk

z,y,x (default: level 0’s chunking)

--voxel-size

z,y,x nm; overrides the volume’s own metadata

--encoding

precomputed encoding (e.g. raw)

--profile

storage profile

--resume

continue an interrupted rebuild

Default: False

--sparse

skip tasks whose input holds no stored chunk. On a sparse volume — ground truth, an ROI export — that is nearly every task, and it is EXACT rather than a guess: an all-fill chunk is never stored, so a task whose input objects are all absent would write nothing anyway. Each level costs one listing instead of a read per task. Refuses if the level it seeds from has no stored chunks at all, rather than writing nothing and reporting success. Not usable on a SHARDED level, which hides which of its chunks exist

Default: False

--progress-path

--dry-run

report the plan and exit, touching nothing

Default: False

--config

dask config: a bundled template (dask-local, dask-slurm-example) or a path to your own YAML. Repeatable, deep-merged left to right, so an overlay need only carry what differs. Default: dask-local

--workers

Default: 4

--serial

no dask at all — run in this process (smallest smoke test)

Default: False

--store-logs

keep TensorStore’s benign S3 credential-chain logging (suppressed by default; real errors are never suppressed)

Default: False

create¶

Create an empty zarr v3 or neuroglancer-precomputed volume — every level exists, no chunk data.

Use it when several small pieces (image stacks, HDF5 files) belong at known positions inside one frame: create the frame, then neu-vol write each piece into it.

–like copies a reference volume’s geometry (level shapes, per-level voxel sizes and chunking, dtype, origin), so a voxel index means the same thing in both. Its level shapes are copied verbatim rather than recomputed, unless –shape or –factors overrides them. Anything you pass explicitly wins over the reference.

neu-vol create [-h] [--like VOLUME] [--shape SHAPE] [--dtype DTYPE]
               [--voxel-size VOXEL_SIZE] [--offset-nm OFFSET_NM]
               [--units UNITS] [--kind {image,segmentation}]
               [--format {zarr,precomputed}] [--encoding ENCODING]
               [--chunk CHUNK] [--shard SHARD] [--levels LEVELS]
               [--factors FACTORS] [--max-levels MAX_LEVELS]
               [--min-dim MIN_DIM] [--profile PROFILE] [--name NAME]
               [--overwrite] [--no-validate] [--dry-run] [--store-logs]
               dst

Positional Arguments¶

dst: where to create it (path or s3://bucket/prefix)

Named Arguments¶

--like

reference volume to copy the frame from (zarr or precomputed, local or remote)

--shape

z,y,x level-0 shape (default: –like’s). Overriding it means the pyramid is recomputed rather than copied

--dtype

e.g. uint8, uint64 (default: –like’s). Worth setting when the frame comes from an image volume but the pieces are labels

--voxel-size

z,y,x nm (default: –like’s)

--offset-nm

z,y,x physical origin of the volume (default: –like’s, else 0)

--units

default: –like’s, else nm

--kind

Possible choices: image, segmentation

recorded in the metadata and read back by downsample to pick its reducer (default: –like’s, else image). For precomputed it also picks the default encoding

--format

Possible choices: zarr, precomputed

target format (default: –like’s format, else zarr). precomputed keeps its scales in one intrinsic info; zarr gets OME-NGFF 0.5 group metadata

--encoding

precomputed chunk encoding: raw, jpeg, compressed_segmentation (default: compressed_segmentation for –kind segmentation, raw otherwise)

--chunk

z,y,x, all levels (default: –like’s per-level chunking)

--shard

z,y,x shard

--levels

how many levels to create, counting level 0 (default: as many as –like has, else the auto schedule). –levels 1 for a single-scale volume

--factors

explicit per-level factors, e.g. ‘1,2,2;1,2,2’; forces the pyramid to be computed rather than copied

--max-levels

at most this many levels, COUNTING LEVEL 0. Truncates a mirrored –like pyramid too (each kept level stays verbatim). Default: no cap when mirroring a reference, 8 when the pyramid is computed

--min-dim

stop when the largest spatial dim is <= this. Applies only when the pyramid is COMPUTED — a mirrored –like pyramid keeps every level the reference has, since dropping its small levels by default would silently break the shared frame

Default: 128

--profile

storage profile name, overriding –format and its chunk/compressor defaults (local, ceph, local-neuroglancer, s3-neuroglancer)

--name

OME multiscales name (zarr only)

Default: 'image'

--overwrite

replace an existing volume at –dst (DESTROYS its data)

Default: False

--no-validate

skip OME metadata validation

Default: False

--dry-run

print the layout and exit, creating nothing

Default: False

--store-logs

Default: False

write¶

Place a subvolume — an image stack, an HDF5 dataset, a region of another volume — into an existing volume at a voxel offset.

SINGLE-SCALE on purpose: it writes –level and touches no other, because how a patch should look when coarsened is a separate decision (averaging label ids invents ids). Run neu-vol downsample afterwards if the result needs a pyramid.

Runs in this process — no dask. For anything big enough to need a cluster, use neu-vol convert.

neu-vol write [-h] --src SRC [--offset OFFSET] [--offset-field NAME]
              [--background V[,V...]] [--voxel-size-field NAME]
              [--offset-order {zyx,xyz}] [--level LEVEL]
              [--offset-level OFFSET_LEVEL] [--src-format SRC_FORMAT]
              [--dataset DATASET] [--all-datasets [GLOB]] [--cast] [--dry-run]
              [--store-logs]
              volume

Positional Arguments¶

volume: the destination volume (path or s3://…)

Named Arguments¶

--src

an .h5/.hdf5 file, a directory or glob of ordered 2D slices, or another volume. REPEATABLE: pass several to write a batch in one go. All of them are checked — offsets, bounds, dtype — before any is written, so a mistake in the last one does not leave the earlier ones half-applied

--offset

z,y,x voxel offset of the subvolume’s origin, in –level’s voxels unless –offset-level says otherwise. OPTIONAL: with no –offset the source is asked for one, which an HDF5 file often records (see –offset-field). With several –src, give either no –offset at all or one per –src, in order

--offset-field

what the source calls its stored offset, used when –offset is omitted. In an HDF5 file this is looked for in the dataset’s attributes, the root group’s attributes, and a top-level dataset of that name (default: voxel_offset)

Default: 'voxel_offset'

--background

value(s) the source uses for background, replaced with 0 as it is read. Manually segmented pieces numbered from 0 have background 1, and an all-background block of 1s is not all-fill, so without this it is stored as data

--voxel-size-field

what the source calls its recorded voxel size, if any. Used only to CHECK it against the level being written into — a piece extracted from another level fits and places cleanly while being at the wrong resolution, which nothing else here would notice (default: voxel_size)

Default: 'voxel_size'

--offset-order

Possible choices: zyx, xyz

axis order of the offset, whether typed or read from the source. Default: whatever the source RECORDS (an axes attribute, as neu-vol to-hdf5 writes), else zyx. Worth checking on a stored offset from elsewhere: ‘voxel_offset’ is precomputed’s field name and precomputed means XYZ, while everything in this package is zyx — reversed, the piece lands mirrored through the z=x diagonal. Whichever applied is echoed

--level

which level to write into

Default: 0

--offset-level

the level whose voxels –offset is expressed in, when that is not –level (e.g. coordinates read off level 0 while writing level 2). Converted with the recorded per-level voxel sizes; a non-integral result is an error

--src-format

backend for –src (default: guessed from the path — .h5 is hdf5, a directory/glob/image file is image_stack, anything with an info or zarr.json is detected properly)

--dataset

HDF5 dataset path (default: the file’s only 3D+ dataset)

--all-datasets

write EVERY volumetric dataset of an HDF5 –src, each at its own recorded voxel_offset — a bag of ground-truth crops in one file goes back into the volume in one command. Give a GLOB to narrow it (–all-datasets ‘z*’), matched against the dataset path and its basename. All of them are still planned before any is written. Cannot be combined with –dataset or –offset

--cast

allow a lossy dtype conversion into the destination

Default: False

--dry-run

report the region, tiles and alignment; write nothing

Default: False

--store-logs

Default: False

progress¶

Point-in-time counts for a multiscale volume being written, zarr v3 or precomputed. Reads only; re-run to refresh.

On sparse data the manifest runs ahead of –storage and that gap is real: an all-fill block is elided, so it writes no object. Use the manifest to answer ‘how far along is my run’.

neu-vol progress [-h] [--progress-path PROGRESS_PATH] [--storage]
                 [--store-logs]
                 volume

Positional Arguments¶

volume: the volume (path or s3://…)

Named Arguments¶

--progress-path

the run’s progress manifest (default: the same path the conversion would have used — note a REBUILD names its own ‘<name>.regen-from-<n>.progress.jsonl’). Counting from it is far cheaper than listing an object store

--storage

count what is actually stored instead of what the run recorded — authoritative, but lists every chunk

Default: False

--store-logs

Default: False

mask-by-value¶

Replace one or more label values with 0 in a volume that already holds them.

Manual segmentation does not always call background 0 — a tool numbering labels from 0 makes it 1 — and that one fact breaks more than it looks. Background becomes a body when meshed, and, worse, an all-background block of 1s is NOT all-fill, so it gets stored: the volume ends up with a chunk object everywhere data was written, and ‘which chunks exist’ stops answering ‘where is the data’. That is the question bboxes-json, relabel, downsample –sparse and neu-morpho’s occupancy filter all ask.

PREFER FIXING IT AT INGEST: neu-vol write, to-hdf5 and convert all take –background, and there the correction happens before the storage decision. This command is for data that has already landed.

Either destination restores the sparsity — writing zeros over a stored chunk removes the object, on both formats — so –out is preferred for the ordinary reason instead: a sparse copy is cheap, and the original stays as the record of what was annotated.

SINGLE-SCALE, like write and relabel: run neu-vol downsample –start-level <level> afterwards.

neu-vol mask-by-value [-h] --values V[,V...] [--to TO]
                      (--out VOLUME | --in-place) [--level LEVEL]
                      [--overwrite] [--dry-run] [--store-logs]
                      volume

Positional Arguments¶

volume: the volume to read (path or s3://…)

Named Arguments¶

--values

the value(s) to replace, e.g. 1

--to

what to replace them with (default: 0, the only value the storage layer treats as empty)

Default: 0

--out

write the corrected volume here instead, created with –like <volume> so a voxel index means the same thing in both. Preferred: cheap on sparse data, and the original stays as the record of what was annotated

--in-place

overwrite the values in the volume itself. Restores the sparsity too, but the original values are then unrecoverable

Default: False

--level

which level to correct (default: 0)

Default: 0

--overwrite

allow –out to replace an existing volume

Default: False

--dry-run

report and count what would change, writing nothing. Still reads, since the counts come from the voxels

Default: False

--store-logs

Default: False

to-hdf5¶

Pack an image stack, an HDF5 file or any readable volume into a single HDF5 file, recording where it belongs and at what scale.

The inverse of neu-vol write: that places a piece into a large volume, this produces the piece. An image stack off a microscope or an annotation tool has no coordinates attached, so this attaches them — and then neu-vol write <volume> –src piece.h5 needs no –offset at all.

What it records: voxel_offset (whole voxels, on the dataset — the field write already looks for), and voxel_size / offset / units / axes in this package’s own vocabulary, on the root and the dataset both. axes matters most: precomputed means xyz, this package means zyx, and reversed, a piece lands mirrored through the z=x diagonal. A file written here says which.

The dataset defaults to /data, which is also what the reader assumes, so a file packed with no arguments reads with none. An existing file is ADDED TO when its recorded frame matches — several pieces of one volume in one file is a legitimate arrangement, each with its own voxel_offset — and refused when it does not. A name already in use needs –dataset or –overwrite.

An HDF5 –src is repacked dataset by dataset — one –src-dataset, several, or –all-datasets — each keeping its name and its recorded voxel size, offset, axes and kind: the way to change a file’s –dtype, –chunk or –compression.

–src (when HDF5) and –out may be s3:// or gs:// URLs. h5py reads and writes only filesystems, so each is a temporary local copy: downloaded first (–out only if it exists, so it can be added to), –out uploaded once everything is written. S3 transfers go through boto3 when it is installed (streamed, multipart); otherwise the whole file passes through memory and S3 uploads stop at 5 GB.

Reads are blocked, so a volume larger than advertised streams rather than filling memory. Serial and in-process, like create/write.

neu-vol to-hdf5 [-h] --src SRC [--dvid-supervoxels] [--dvid-locked]
                [--wk-mapping NAME] --out FILE.h5 [--src-format SRC_FORMAT]
                [--voxel-size Z,Y,X] [--offset Z,Y,X] [--level LEVEL]
                [--crop-bbox Z0,Y0,X0,Z1,Y1,X1] [--src-dataset NAME]
                [--all-datasets] [--voxel-size-field NAME]
                [--offset-field NAME] [--units UNITS] [--axes AXES]
                [--dataset NAME] [--dtype DTYPE] [--truncate] [--chunk Z,Y,X]
                [--compression {gzip,lzf,none}] [--background V[,V...]]
                [--overwrite] [--dry-run] [--store-logs]

Named Arguments¶

--src

image stack (directory or glob of ordered 2D slices), an HDF5 file, local or s3:// / gs:// (see –src-dataset), or any readable volume, including dvid://SERVER/UUID/INSTANCE and wk://DATASET/LAYER (give those a –crop-bbox)

--dvid-supervoxels

read SUPERVOXELS rather than agglomerated bodies (DVID sources only). Default is bodies, i.e. the proofread segmentation

Default: False

--dvid-locked

Default: False

--wk-mapping

--out

the HDF5 file to write, local or s3:// / gs://. Created, or added to if it already records the same frame

--src-format

backend for –src (default: auto-detect — a zarr or precomputed volume by its metadata document, an HDF5 file or a glob/file/directory of 2D slices by name)

--voxel-size

physical size of one voxel, e.g. 8,8,8. Required for a source that records none (an image stack, a bare array, an HDF5 file without –voxel-size-field); read from the volume, at –level, when the source is one

--offset

whole-voxel position of this piece’s (0,0,0) corner in the volume it belongs to, in –axes order. Default: what an HDF5 –src already records (–offset-field, searched in the dataset’s attributes, the root’s, and a top-level dataset of that name), plus the –crop-bbox origin if one is given; else 0,0,0

--level

which level of a multiscale –src to read (default: 0). The level’s own recorded voxel size becomes the default frame, and –crop-bbox is in that level’s voxels, so neu-vol write –level N puts the piece back where it came from

Default: 0

--crop-bbox

pack only this box, in –level’s voxels. Half-open, clipped to the source. Its origin becomes the recorded voxel_offset unless –offset says otherwise

--src-dataset

dataset to read, when –src is itself an HDF5 file with more than one. Repeatable: with several, each is written under its own name, and –dataset/–offset/–crop-bbox are refused

--all-datasets

repack every volumetric dataset of an HDF5 –src, each under its own name; a remote –src or –out is transferred once for all

Default: False

--voxel-size-field

attribute to record the voxel size under, and to READ it from when –src is an HDF5 file that carries one (default: voxel_size, this package’s own name). Set it to match files another tool wrote, so a file stays readable by whatever wrote its siblings

Default: 'voxel_size'

--offset-field

attribute to record the voxel offset under (default: voxel_offset, which is what neu-vol write looks for; change both together or write will not find it)

Default: 'voxel_offset'

--units

unit for –voxel-size (default: nm)

Default: 'nm'

--axes

axis order of the array AND of –voxel-size/–offset, recorded in the file so no reader has to guess (default: zyx)

Default: 'zyx'

--dataset

dataset to write (default: /data, which is what a reader assumes when it is not told)

--dtype

cast to this dtype (default: the source’s). A narrower dtype is refused if any value falls outside its range, naming the block; see –truncate

--truncate

cast values that do not fit –dtype anyway, as numpy’s astype does: an integer keeps its low-order bits, so 65537 becomes 1 in uint16 and two labels can merge. Default: refuse

Default: False

--chunk

HDF5 storage chunk (default: up to 64 per axis). This is what governs partial reads when the piece is written back

--compression

Possible choices: gzip, lzf, none

dataset compression (default: gzip)

Default: 'gzip'

--background

value(s) the source uses for background, replaced with 0 while packing, so the packed file is already correct and nothing downstream needs to know the source had this quirk

--overwrite

replace the dataset if that name is already used

Default: False

--dry-run

report what would be written, and write nothing

Default: False

--store-logs

Default: False

align-bbox¶

Align a bounding box to a block grid and print it back, ready to paste into –crop-bbox or –roi.

WHICH GRID is the real question, and there are three, each with a different cost when a box straddles it:

write-unit  the chunk, or the SHARD where the level is sharded. A partial write is a read-modify-write: it keeps the object's existing data, but two concurrent partial writes into one object lose one of them, silently. Aligning to the inner chunk of a sharded level protects against nothing.
pyramid     the cumulative factor of the deepest level. A crop that misses it has coarse levels on their own grid, each level's voxel_offset rounding to it — level 0 stays exact, so nothing looks wrong.
both        the per-axis LCM of those two. What a cropped copy of a multiscale volume wants.
read-chunk  the inner chunk. Read amplification, not write safety.

Boxes are half-open, so a bound already on a boundary stays put. Everything is per axis: real grids are anisotropic, and a level’s chunk is converted to level-0 voxels through its own recorded voxel size, never an assumed 2**N.

Reads a volume’s metadata and nothing else. Writes nothing.

neu-vol align-bbox [-h] --bbox Z0,Y0,X0,Z1,Y1,X1 [--volume PATH_OR_URL]
                   [--block Z,Y,X] [--to {write-unit,pyramid,both,read-chunk}]
                   [--level LEVEL] [--mode {outer,inner,nearest,origin}]
                   [--scale N] [-q] [--store-logs]

Named Arguments¶

--bbox

the box to align, half-open, in voxels at –scale. Repeatable

--volume

take the grid and the extent from this volume. Required unless –block gives the grid outright

--block

align to this grid instead of one read from a volume, in level-0 voxels. Works with no –volume at all

--to

Possible choices: write-unit, pyramid, both, read-chunk

which of the volume’s grids to align to (default: write-unit). See above — they answer different questions

Default: 'write-unit'

--level

take the chunk/shard from this level rather than 0. Its blocks are converted to level-0 voxels, so a 128-chunk at a level coarsened 4x is a 512-voxel grid (default: 0)

Default: 0

--mode

Possible choices: outer, inner, nearest, origin

outer: grow to cover the box (never loses a voxel, cannot fail). inner: shrink to fit inside it. nearest: round both ends. origin: align the origin and keep the EXTENT exactly, for a fixed-size crop — its far edge then stays off the grid (default: outer)

Default: 'outer'

--scale

the box is in scale-N voxels; convert to level 0 with the volume’s real per-level voxel sizes. The aligned box is also reported back at scale N when it is exactly representable there

Default: 0

-q, --quiet

print only the aligned box, one line per –bbox, so it can be substituted: –crop-bbox $(neu-vol align-bbox … -q)

Default: False

--store-logs

Default: False

relabel¶

Renumber a SPARSE segmentation volume so each occupied region gets a disjoint range of ids.

Ground truth annotated chunk by chunk usually numbers every chunk from 1, so one integer means a different cell in each — meshed, that becomes a single body with components scattered across the volume. This walks the regions in order and gives each its own range, so an id identifies one cell in one region.

Regions come from stored-chunk occupancy (as neu-vol bboxes-json), so they are disjoint and chunk-aligned: no write is a partial-chunk update. Serial by construction — each range starts where the last ended — so it runs in this process, no dask.

SINGLE-SCALE, like neu-vol write: run neu-vol downsample –start-level <level> afterwards, then re-mesh.

neu-vol relabel [-h] (--out VOLUME | --in-place) [--level LEVEL]
                [--block-size N] [--map PATH_OR_URL] [--overwrite] [--dry-run]
                [--store-logs]
                volume

Positional Arguments¶

volume: the volume to read (path or s3://…)

Named Arguments¶

--out

write the renumbered volume here instead, created empty with –like <volume> so a voxel index means the same thing in both. Preferred: a sparse copy is nearly free and the original stays as the record of the raw annotation

--in-place

overwrite the ids in the volume itself. Recoverable only through the mapping file

Default: False

--level

which level to renumber (default: 0)

Default: 0

--block-size

number region k from N*k+1 instead of consecutively, so the region a label came from is readable off the id. Refuses if a region holds more than N labels

--map

where to write the old->new mapping — a local path or an object store location (s3://…), so it can sit beside a remote volume (default: ‘<destination>.relabel-<level>.json’ here). It is the only way back from a new id to the region and label it came from

--overwrite

allow –out to replace an existing volume

Default: False

--dry-run

report the regions and the ranges they would get; write nothing. Still reads them, since the ids come from the voxels

Default: False

--store-logs

Default: False

wk-mappings¶

Fetch the agglomerate views of a webknossos segmentation layer as lookup tables, table[supervoxel] = agglomerate, one .npy per view.

With –volume (the layer’s supervoxels, converted with no –wk-mapping), also writes a descriptor <view>.json per view, which reads that volume through the view ON THE FLY wherever a volume is accepted: neu-vol info, read_piece / neu-eval compare –segmentation, and neu-vol convert –src, which turns a chosen view into a real volume. Level 0 of a view is exact; coarser levels map the source’s downsampled supervoxels and are approximate. A converted view rebuilds its pyramid from level 0, so it is exact.

Resumable: a view whose table exists is skipped unless –overwrite.

neu-vol wk-mappings [-h] --out DIR [--volume VOLUME] [--mapping NAME]
                    [--max-id MAX_ID] [--overwrite] [--threads N]
                    src

Positional Arguments¶

src: the layer, as wk://DATASET/LAYER

Named Arguments¶

--out

where the tables, descriptors and mappings.json go: a local directory or an object-store prefix (s3://…)

--volume

the supervoxel volume the descriptors should read

--mapping

a view to fetch (repeatable; default: every agglomerate view)

--max-id

highest supervoxel id (default: the layer’s largestSegmentId)

--overwrite

refetch existing tables

Default: False

--threads

views fetched at once, on threads (default: 4). The time is the server’s, so this is the only lever; keep it modest on a shared public server

Default: 4

wk-ls¶

List the webknossos datasets your token can see, with their layers, dates and size – one request. The server’s list includes other organizations’ public datasets, so it is filtered to the organization in the credentials file unless –org or –all-orgs says otherwise.

–details adds each matching dataset’s description, uploader, voxel size, and per layer its bounding box, mags and agglomerate views (about three requests per dataset, so it is capped by –max-details).

neu-vol wk-ls [-h] [--org ID] [--all-orgs] [--details] [--max-details N]
              [--json]
              [pattern]

Positional Arguments¶

pattern: dataset names to show: a substring, or a glob if it contains ‘*’, ‘?’ or ‘[’ (case-insensitive)

Named Arguments¶

--org

list this organization’s datasets (default: the credentials file’s organization)

--all-orgs

list every visible dataset, whatever its organization

Default: False

--details

fetch each matching dataset’s full record and layer views

Default: False

--max-details

with –details: refuse to fetch more than N datasets (default: 10)

Default: 10

--json

print JSON instead of a table

Default: False

help¶

Print the help for the tool, or for one of its subcommands.

neu-vol help write and neu-vol write –help are the same thing.

neu-vol help [-h] [COMMAND ...]

Positional Arguments¶

COMMAND: the subcommand to describe (default: the whole tool)

neu-vol¶

Positional Arguments¶

Sub-commands¶

info¶

Positional Arguments¶

Named Arguments¶

convert¶

Named Arguments¶

copy¶

Named Arguments¶

downsample¶

Positional Arguments¶

Named Arguments¶

create¶

Positional Arguments¶

Named Arguments¶

write¶

Positional Arguments¶

Named Arguments¶

progress¶

Positional Arguments¶

Named Arguments¶

mask-by-value¶

Positional Arguments¶

Named Arguments¶

to-hdf5¶

Named Arguments¶

align-bbox¶

Named Arguments¶

relabel¶

Positional Arguments¶

Named Arguments¶

wk-mappings¶

Positional Arguments¶

Named Arguments¶

wk-ls¶

Positional Arguments¶

Named Arguments¶

help¶

Positional Arguments¶

`neu-vol`¶