Contracts

Enforce input and output contracts of computations using Daf data.

@kwdef struct Contract
    name::Maybe{AbstractString} = nothing
    is_relaxed::Bool = false
    axes::Maybe{ContractAxes} = nothing
    data::Maybe{ContractData} = nothing
end

The contract of a computational tool, specifing the axes and and data . If is_relaxed , this allows for additional inputs and/or outputs; this is typically used when the computation has query parameters, which may need to access such additional data, or when the computation generates a variable set of data.

If name is specified, then the parameter for the daf repository should be so named. Otherwise, the parameter should be the first unnamed parameter (there can be only one such unnamed parameter per function).

A pair where the key is the axis name and the value is a tuple of the ContractExpectation and a description of the axis (for documentation). We also allow specifying a tuple instead of a pairs to make it easy to invoke the API from other languages such as Python which do not have the concept of a Pair .

Specify all the axes for a contract. We would have liked to specify this as AbstractVector{<:ContractAxis} but Julia in its infinite wisdom considers ["a" => "b", ("c", "d")] to be a Vector{Any} , which would require literals to be annotated with the type.

The specification of an axis in a Contract , which is the ContractExpectation for enforcement and a string description for the generated documentation.

A vector of pairs where the key is a DataKey identifying some data property, and the value is a tuple of the ContractExpectation , the expected data type, and a description (for documentation). We also allow specifying a tuple instead of a pairs to make it easy to invoke the API from other languages such as Python which do not have the concept of a Pair .

Specify all the data for a contract. We would have liked to specify this as AbstractVector{<:ContractDatum} but Julia in its infinite wisdom considers ["a" => "b", ("c", "d") => "e"] to be a Vector{Any} , which would require literals to be annotated with the type.

The specification of some property in a Contract , which is the ContractExpectation and the type for enforcement, and a string description for the generated documentation.

The expectation from a specific property for a computation on Daf data.

Input data:

RequiredInput - data that must exist in the data when invoking the computation, will be used as input.

OptionalInput - data that, if existing in the data when invoking the computation, will be used as an input.

Output data:

CreatedOutput - data that is always created by the computation.

GuaranteedOutput - data that will be created by the computation unless it already exists.

OptionalOutput - data that may be created when the computation is done, depending on some condition, which may include the existence of optional input and/or the value of parameters to the computation, and/or the content of the data.

Whether to enforce contracts. By defaults, contracts are not enforced, as this imposes a run-time overhead on computational pipelines. You can set this manually to true , or set the environment variable DAF_ENFORCE_CONTRACTS to a "truthy" value.

contractor(
    computation::AbstractString,
    contract::Contract,
    daf::DafReader;
    name::Maybe{AbstractString} = nothing,
    overwrite::Bool = false,
)::DafReader

Wrap a daf data set to enforce a contract for some computation , possibly allowing for overwrite of existing outputs. If DAF_ENFORCE_CONTRACTS is not set, this just returns the original daf .

struct ContractDaf <: DafWriter ... end

A DafWriter wrapper which restricts access only to the properties listed in some Contract . This also tracks which properties are accessed, so when a computation is done, we can verify that all required inputs were actually accessed. If they weren't, then they weren't really required (should have been marked as optional instead).

This isn't exported and isn't created manually; instead call contractor , or, better yet, use the @computation macro.

verify_input(contract_daf::ContractDaf)::Nothing
verify_input(contract_daf::DafReader)::Nothing

Verify the contract_daf data before a computation is invoked. This verifies that all the required data exists and is of the appropriate type, and that if any of the optional data exists, it has the appropriate type. This is a no-op if the contract_daf is just a DafReader (that is, if DAF_ENFORCE_CONTRACTS was not set).

verify_output(contract_daf::ContractDaf)::Nothing
verify_output(contract_daf::DafWriter)::Nothing

Verify the contract_daf data when a computation is complete. This verifies that all the guaranteed output data exists and is of the appropriate type, and that if any of the optional output data exists, it has the appropriate type. It also verifies that all the required inputs were accessed by the computation. This is a no-op if the contract_daf is just a DafReader (that is, if DAF_ENFORCE_CONTRACTS was not set).