ZarrConvert

Hard-link conversion between FilesDaf and ZarrDaf directories.

The two on-disk formats differ in their per-property metadata encoding ( FilesDaf uses one JSON sidecar per array, ZarrDaf uses one .zarray per array plus a consolidated .zmetadata ), but both store every numeric blob — dense array chunks and the colptr / rowval / nzind / nzval components of sparse arrays — as the same raw little-endian bytes without headers. The functions in this module exploit that equivalence to convert one tree into the other by hard-linking every numeric blob and re-serializing only the metadata and the string-valued properties, so the on-disk cost of a conversion is close to zero.

Hard-linking requires the source and destination to live on the same filesystem; each conversion verifies this up front with an actual hard-link probe and refuses otherwise. Each conversion also refuses if the destination path already exists — it never overwrites pre-existing data. On any error the partially-populated destination is removed, so the destination is always either fully-populated or fully-absent.

Only the directory-tree ZarrDaf backend is supported (paths ending with .daf.zarr ). The ZIP archive backend ( .daf.zarr.zip , .dafs.zarr.zip#/group ) and the HTTP backend ( http(s)://… ) are rejected — neither provides the per-chunk on-disk file that hard-linking needs.

zarr_to_files(;
    zarr_path::AbstractString,
    files_path::AbstractString,
)::Nothing

Convert a ZarrDaf directory at zarr_path into an equivalent FilesDaf directory at files_path . Every numeric blob (dense chunks and the colptr / rowval / nzind / nzval components of sparse arrays) is hard-linked from the source into the destination; scalars, axes, string vectors and string matrices are re-serialized through the respective readers/writers. zarr_path must be a directory whose name ends with .daf.zarr (the ZIP and HTTP backends are rejected); files_path must not already exist, and the two paths must live on the same filesystem.

files_to_zarr(;
    files_path::AbstractString,
    zarr_path::AbstractString,
)::Nothing

Convert a FilesDaf directory at files_path into an equivalent ZarrDaf directory at zarr_path . Every numeric blob (dense chunks and the colptr / rowval / nzind / nzval components of sparse arrays) is hard-linked from the source into the destination; scalars, axes, string vectors and string matrices are re-serialized through the respective readers/writers. files_path must be an existing FilesDaf directory; zarr_path must not already exist, its name must end with .daf.zarr (the ZIP and HTTP backends are rejected), and the two paths must live on the same filesystem.