Complete Repositories

The common idiom in Daf is to have multiple repositories in a chain; actually, they typically form a tree where different leaf repositories are based on common ancestor repositories. For example, a base cells repository can be used by multiple alternative metacells repositories.

Tracking this tree manually is possible, by using naming conventions (which is always a good idea). However, this gets tedious. The code here automates this by using an additional convention - each repository contains a scalar property called base_daf_repository which identifies its parent repository (if any). This path is relative to the directory containing the child repository. See open_daf for details.

In addition, it is possible to have another scalar property, base_daf_view . If specified, this should contain JSON serialization of the parameters of a DafView to apply to the base repository. This allows the child repository to be based on a subset of the base data, and/or rename the base data.

Since the same base repository can be used by multiple other repositories, we use the GlobalWeakCache to avoid needlessly re-opening the same repository more than once.

complete_daf(leaf::AbstractString, mode::AbstractString = "r"; name::Maybe{AbstractString} = nothing)::Union{DafReader, DafWriter}

Open a complete chain of Daf repositories by tracing back through the base_daf_repository and the optional base_daf_view . Valid modes are only "r" and "r+"; if the latter, only the first (leaf) repository is opened in write mode.

A convenient way to create persistent complete chains is using complete_chain! .

TODO: Properly indent the log messages of the created leaf repositories. Generic mechanism for indenting all hierarchical log messages?

open_daf(
    path::AbstractString,
    mode::AbstractString = "r";
    name::Maybe{AbstractString} = nothing
)::Union{DafReader, DafWriter}

Open a Daf data set, dispatching to the appropriate backend based on path :

• If path ends with .daf.zarr , ends with .daf.zarr.zip , or contains .dafs.zarr.zip# (followed by a sub-daf group path), open a ZarrDaf .
• Otherwise, if path starts with http:// or https:// , open an HttpDaf . Only mode = "r" is supported for the HTTP backend; any other mode raises an error.
• Otherwise, if path ends with .h5df or contains .h5dfs# (followed by a group path), open an H5df file (or a group in one).
• Otherwise, open a FilesDaf .