neu_vol.backends.hdf5¶

HDF5-backed reader (h5py).

Reads one dataset of an HDF5 file as a volume, in canonical (z, y, x) / (c, z, y, x) order. Reading only; HDF5Backend.write_region() says where HDF5 output is written instead. Each instance holds an open file, reopened from its spec on each dask worker.

HDF5 is a container, so a path alone is ambiguous: sole_dataset() picks the dataset when there is exactly one. The optional stored_offset / stored_voxel_size / stored_axes / stored_units / stored_kind readers return the frame a writer recorded beside the array, which source_metadata composes into the same metadata a precomputed info yields. An axis order the file states but nobody can interpret is an error, never a guess: reversed, the data lands mirrored through the z=x diagonal.

Attributes¶

`DESCRIBED_ATTRIBUTES`
`FRAME_ATTRIBUTES`
`HDF5_EXTENSIONS`
`TAG`

Classes¶

HDF5Backend

View over a single dataset in an HDF5 file.

Functions¶

`axes_string`(value) → str	An axis-order attribute as `"zyx"`, however h5py handed it over.
`datasets`(path, *[, min_ndim]) → list[str]	Every dataset in the file with at least `min_ndim` dimensions, by path.
`describe_datasets`(path, *[, min_ndim]) → dict[str, dict]	Every volumetric dataset in the file, with its geometry and recorded frame.
`local_copy`(location) → Iterator[str]	`location` downloaded into a temporary directory, removed when the block exits.
`names_remote_hdf5`(src[, src_format]) → bool	True if `src` is an HDF5 file on an object store, by `src_format` or extension.
`open_hdf5`(path[, dataset]) → HDF5Backend	One dataset of an HDF5 file, ready to read regions from.
`require_local_path`(path[, what]) → str	`path` if it is an ordinary filesystem path; a useful error if it is not.
`sole_dataset`(path) → str	The file's one volumetric dataset, for callers that did not name one.
`staged_output`(location, *[, upload]) → Iterator[str]	A local file to write in place of `location`, uploaded when the block exits cleanly.

Module Contents¶

class neu_vol.backends.hdf5.HDF5Backend(spec: Mapping[str, Any])¶

View over a single dataset in an HDF5 file.

close() → None¶

Release the open h5py.File.

Called by release_backends(): HDF5 refuses to open a file for writing while any handle holds it read-only. Idempotent.

classmethod open(spec: Mapping[str, Any]) → HDF5Backend¶

read_region(region: neu_vol.backends.base.Region) → numpy.ndarray¶

stored_axes(name: str = 'axes')¶

The axis order the writer recorded, as (order, where), or None.

What makes stored_offset()’s numbers unambiguous; neu-vol to-hdf5 writes it. Dataset attributes first, then the root group’s.

stored_kind(name: str = 'kind')¶

What the writer said the voxels MEAN, as (kind, where), or None.

Attributes only, dataset before root. Written by neu_vol.write_piece and to-hdf5 (ops/pack.KIND_ATTR). The caller trusts only values in neu_lib.KINDS, since another tool may use the word for something else.

stored_offset(name: str = 'voxel_offset')¶

A voxel offset recorded in the file as (value, where_it_was_found).

None if the file records none. Searched in the dataset’s attributes, then the root group’s, then a top-level dataset of that name.

The axis order is not knowable from here: the name is precomputed’s, where it means xyz. The numbers come back as stored and the caller decides, using stored_axes() when the file has it.

stored_units(name: str = 'units')¶

The physical unit the writer recorded, as (unit, where), or None.

Attributes only, dataset before root. neu-vol to-hdf5 records it beside voxel_size; the suite works in nm.

stored_voxel_size(name: str = 'voxel_size')¶

A physical voxel size recorded in the file, as (value, where), or None.

Searched as stored_offset() is. name is a parameter because another writer may call it something else.

to_spec() → dict[str, Any]¶

abstractmethod write_region(region: neu_vol.backends.base.Region, data: numpy.ndarray) → None¶

Never: HDF5 output is written whole, by a different path.

The block engine writes into a volume; one array in a container has no pyramid and no chunk objects, and HDF5 has no concurrent writers. A piece is written in one call by neu_vol.write_piece() (an array in memory) or neu_vol.pack_hdf5() (neu-vol to-hdf5, from another source).

property chunks: tuple[int, ...]¶

property dtype: numpy.dtype¶

property shape: tuple[int, ...]¶

neu_vol.backends.hdf5.axes_string(value: Any) → str¶

An axis-order attribute as "zyx", however h5py handed it over.

"zyx", b"zyx", ["z","y","x"] and [b"z",b"y",b"x"] are all accepted.

neu_vol.backends.hdf5.datasets(path: str, *, min_ndim: int = 3) → list[str]¶: Every dataset in the file with at least min_ndim dimensions, by path.

neu_vol.backends.hdf5.describe_datasets(path: str, *, min_ndim: int = 3) → dict[str, dict]¶

Every volumetric dataset in the file, with its geometry and recorded frame.

{name: {"shape", "dtype", "chunks", "ndim", <frame attributes>}}, in file order, from one file open, attributes only. What neu-vol info lists for a container of several arrays, such as a file of ground-truth crops.

Frame values come back as recorded, each with its own axes, not reoriented to zyx (source_metadata._read_hdf5 does that, strictly, for one array). Values fall back to the root group’s.

neu_vol.backends.hdf5.local_copy(location: str) → Iterator[str]¶

location downloaded into a temporary directory, removed when the block exits.

h5py reads only filesystems (neu_vol.transfer.download() says how the bytes move). Cached readers of the copy are closed before it is removed.

neu_vol.backends.hdf5.names_remote_hdf5(src: Any, src_format: str | None = None) → bool¶: True if src is an HDF5 file on an object store, by src_format or extension.

neu_vol.backends.hdf5.open_hdf5(path: str, dataset: str | None = None) → HDF5Backend¶

One dataset of an HDF5 file, ready to read regions from.

The front door for a file you are pointing at by hand:

be = open_hdf5("piece.h5")                      # one dataset: found
be = open_hdf5("gt_v1_eval.h5", "/z07901")      # a container: name it
be.read_region((slice(0, 64), slice(0, 64), slice(0, 64)))

dataset is optional only when the file holds exactly one 3D+ array; with several it is required, and the error lists them (sole_dataset()).

Not a general “open anything” helper: the format is in the name, so nothing is detected. To find out what something is, use describe(). Goes through open_backend(), so the open file joins the per-process cache.

neu_vol.backends.hdf5.require_local_path(path: str, what: str = 'an HDF5 file') → str¶

path if it is an ordinary filesystem path; a useful error if it is not.

h5py has no object-store driver; without this check it tries to create a local file called s3://bucket/piece.h5 and reports errno = 2.

neu_vol.backends.hdf5.sole_dataset(path: str) → str¶

The file’s one volumetric dataset, for callers that did not name one.

Answers only when there is exactly one 3D+ dataset; otherwise raises listing them and saying how to choose one.

neu_vol.backends.hdf5.staged_output(location: str, *, upload: bool = True) → Iterator[str]¶

A local file to write in place of location, uploaded when the block exits cleanly.

Starts as a copy of the object if there is one, so a file can be added to. Nothing is uploaded if the block raises, or with upload=False (a dry run).

neu_vol.backends.hdf5.DESCRIBED_ATTRIBUTES¶

neu_vol.backends.hdf5.FRAME_ATTRIBUTES¶

neu_vol.backends.hdf5.HDF5_EXTENSIONS = ('.h5', '.hdf5', '.hdf', '.he5')¶

neu_vol.backends.hdf5.TAG = 'hdf5'¶