Heatmaps

Graphs for showing a 2D matrix.

A graph showing a heatmap. See HeatmapGraphData and HeatmapGraphConfiguration .

function heatmap_graph(;
    [figure_title::Maybe{AbstractString} = nothing,
    entries::MatrixValuesData = MatrixValuesData(),
    cells::MatrixEntitiesData = MatrixEntitiesData(),
    rows::HeatmapAxisData = HeatmapAxisData(),
    columns::HeatmapAxisData = HeatmapAxisData(),
    configuration::HeatmapGraphConfiguration = HeatmapGraphConfiguration()]
)::HeatmapGraph

Create a HeatmapGraph by initializing only the HeatmapGraphData fields (with an optional HeatmapGraphConfiguration ).

@kwdef mutable struct HeatmapGraphData <: AbstractGraphData
    figure_title::Maybe{AbstractString} = nothing
    entries::MatrixValuesData = MatrixValuesData()
    cells::MatrixEntitiesData = MatrixEntitiesData()
    rows::HeatmapAxisData = HeatmapAxisData()
    columns::HeatmapAxisData = HeatmapAxisData()
end

The data for a graph showing a heatmap (matrix) of entries.

This is shown as a 2D image where each matrix entry is a small rectangle with some color. Due to Plotly limitation, colors must be continuous. The entries values are required; their title is the title of the colors scale. The cells hold the hovers of the entries. The hover for each rectangle is a combination of the hovers of the cell, of its row and of its column.

The rows and columns hold the data of each axis (see HeatmapAxisData ).

Valid combinations of the fields controlling order and clustering are:

data order	data arrange_by	data groups	config reorder	config dendogram_size	config linkage	config metric	result tree	result order	notes
nothing	nothing	ignored	nothing	nothing	nothing	nothing	Not computed	Original data order	Do not cluster, use the original data order (default)
nothing	nothing / AbstractMatrix{<:Real}	ignored	nothing	Any	nothing / Any	nothing / Any	ehclust of original order with linkage or WardLinkage	Original data order	Cluster, preserving the original order
nothing	nothing	ignored	SameOrder	nothing / Any	nothing	nothing	Same as other axis	Same as other axis	Square matrices only
nothing	nothing / AbstractMatrix{<:Real}	nothing	OptimalHclust / RCompatibleHclust	nothing / Any	nothing / Any	nothing / Any	hclust with linkage or WardLinkage	hclust with reorder	Cluster using linkage and branch reorder
nothing	nothing / AbstractMatrix{<:Real}	Any	OptimalHclust / RCompatibleHclust	nothing / Any	nothing / Any	nothing / Any	ehclust with groups and linkage or WardLinkage	hclust with groups and reorder	Cluster using groups , linkage and branch reorder
nothing	nothing / AbstractMatrix{<:Real}	nothing	SlantedHclust / SlantedPreSquaredHclust	nothing / Any	nothing / Any	nothing / Any	hclust with linkage or WardLinkage , then reorder_hclust by slanted_orders	reorder_hclust by slanted_orders	Cluster, then slant preserving the tree
nothing	nothing / AbstractMatrix{<:Real}	Any	SlantedHclust / SlantedPreSquaredHclust	nothing / Any	nothing / Any	nothing / Any	hclust with groups and linkage or WardLinkage , then reorder_hclust by slanted_orders	reorder_hclust by slanted_orders	Cluster using groups , then slant preserving the tree
nothing	nothing / AbstractMatrix{<:Real}	ignored	SlantedOrder / SlantedPreSquaredOrder	nothing / Any	nothing / Any	nothing / Any	ehclust of slanted_orders with linkage or WardLinkage	slanted_orders	Slant, then cluster preserving the slanted order
Hclust	nothing	ignored	nothing	nothing / Any	nothing	nothing	Hclust tree	Hclust order	Force a specific tree and order on the data
Hclust	nothing / AbstractMatrix{<:Real}	ignored	SlantedHclust / SlantedPreSquaredHclust	nothing / Any	nothing	nothing	reorder_hclust by slanted_orders	reorder_hclust by slanted_orders	Slant, preserving a given tree
AbstractVector{<:Integer}	nothing	ignored	nothing	nothing	nothing	nothing	Not computed	order permutation	Do not cluster, use the specified order
AbstractVector{<:Integer}	nothing	ignored	nothing	Any	nothing / Any	nothing / Any	ehclust of order with linkage or WardLinkage	order permutation	Cluster, preserving the specified order
AbstractVector{<:Integer}	nothing	nothing	ReorderHclust	nothing / Any	nothing / Any	nothing / Any	hclust with linkage or WardLinkage	reorder_hclust by data order	Cluster, then reorder branches to be close to order
AbstractVector{<:Integer}	nothing	Any	ReorderHclust	nothing / Any	nothing / Any	nothing / Any	ehclust with groups and linkage or WardLinkage	reorder_hclust by data order	Cluster, then reorder branches to be close to order

All other combinations are invalid. Note:

• When calling hclust and/or ehclust and/or slanted_orders , then specifying arrange_by will use it instead of the displayed data matrix.

• When calling hclust and/or ehclust , then specifying a metric will be used instead of Euclidean to compute the distances matrix.

• Specifying groups only impacts the tree and order when computing a new clustering without other order constraints. They can still be specified to denote gaps in the heatmap, even when they do not impact the tree and/or order.

@kwdef mutable struct HeatmapAxisData
    names::VectorValuesData = VectorValuesData()
    entities::VectorEntitiesData = VectorEntitiesData()
    order::Maybe{Union{Hclust, AbstractVector{<:Integer}}} = nothing
    groups::VectorValuesData = VectorValuesData()
    subgroups::VectorValuesData = VectorValuesData()
    arrange_by::Maybe{AbstractMatrix{<:Real}} = nothing
    annotations::AbstractVector{AnnotationData} = AnnotationData[]
    annotations_order::Maybe{AbstractVector{<:Integer}} = nothing
end

The data of one axis (the rows or the columns) of a HeatmapGraphData . The names are strings, one per entry, shown as the tick labels; their title is the axis title. The entities hold the hovers and mask of the entries. The annotations are shown to the side of the axis. If annotations_order is specified, they are shown in that order; it describes all the annotations, including the ones that are not is_shown .

By default, if reordering the entries, this is based on the entries.matrix of the graph. You can override this by specifying an arrange_by matrix. Only the reordered dimension needs to match the entries.matrix (the rows arrange_by must have the same number of rows, and the columns arrange_by the same number of columns); the other dimension holds whatever features you want to cluster by, and need not match. For efficiency the rows arrange_by matrix should be in row-major layout, but that's not critical.

Alternatively you can force the order of the entries by specifying the order permutation. You can also specify an Hclust object as the order. If you ask for a dendogram and did not specify such a clustering, one will be computed.

If groups values (numbers or strings, one per entry) are specified, then a gap can be added between entries of different groups. Groups can also be used to constrain the computed clustering. The subgroups are a second, finer level of grouping nested in the groups. Neither has a title.

Hidden entries (see the mask of VectorEntitiesData ) are not drawn, but they are still part of the data: the clustering sees them, and the order (a permutation or a tree) always describes all the entries, hidden ones included. This way the order computed for one graph (see heatmap_order ) can be given to another graph of the same data, whether or not the two hide the same entries. At least one entry must be shown.

@kwdef mutable struct HeatmapGraphConfiguration <: AbstractGraphConfiguration
    figure::FigureConfiguration = FigureConfiguration()
    entries::EntriesConfiguration = EntriesConfiguration()
    rows::HeatmapAxisConfiguration = HeatmapAxisConfiguration()
    columns::HeatmapAxisConfiguration = HeatmapAxisConfiguration()
    origin::HeatmapOrigin = HeatmapBottomLeft
    final_order::Maybe{HeatmapGraphOrder} = nothing
end

Configure a graph showing a heatmap.

This displays a matrix of values using a rectangle at each position. Due to Plotly's limitations, you still to manually tweak the graph size for best results; there's no way to directly control the width and height of the rectangles.

The entries configure the entries (see EntriesConfiguration ); the rows and columns configure each axis (see HeatmapAxisConfiguration ).

The final_order caches the computed order of the rows and the columns; access it through the graph's order (e.g., for generating other graphs in an identical order). It is computed once, whether the graph's figure is generated or its order is asked for first.

@kwdef mutable struct EntriesConfiguration <: Validated
    colors::ColorsConfiguration = ColorsConfiguration()
end

Configure the entries of a heatmap. The colors map the values of the entries to colors. Due to Plotly's limitations, only continuous color palettes are supported.

@kwdef mutable struct HeatmapAxisConfiguration <: Validated
    title::Maybe{AbstractString} = nothing
    annotations::AnnotationSize = AnnotationSize()
    reorder::Maybe{HeatmapReorder} = nothing
    linkage::Maybe{HeatmapLinkage} = nothing
    metric::Maybe{PreMetric} = nothing
    include_hidden::Bool = true
    groups_gap::Maybe{Integer} = 1
    subgroups_gap::Maybe{Integer} = nothing
    dendogram_size::Maybe{Real} = nothing
    dendogram_line::LineConfiguration = LineConfiguration()
end

Configure one axis (the rows or the columns) of a heatmap. The title is the title of the axis. The annotations are the sizes of the annotations shown to the side of the axis.

You can use reorder to reorder the entries of the axis. When specifying linkage , by default, the clustering uses the Euclidean distance metric. You can override this by specifying the metric .

By default, a computed clustering sees all the entries of the axis, hidden ones included, so hiding some entries does not move the rest. Set include_hidden to false to cluster the shown entries only. Either way the resulting order and tree (see heatmap_order ) describe all the entries; when the hidden ones were left out of the clustering, they come last, joined to the root of the tree. This has no effect on an Hclust given in the data, which is used as is.

If groups are specified for the entries in the HeatmapAxisData , they can be used to constrain the clustering, and/or to create visible gaps in the heatmap (between entries of different groups). The groups_gap is the number of fake entries to added between the separated entries. That is, the default gap of 1 will add a blank gap of one entry between adjacent entries of different groups. A gap of nothing will not be shown.

If subgroups are also specified, they are a second, finer level of grouping nested in the groups; each group is contiguous, and within it each subgroup is contiguous. Their subgroups_gap works the same way, and defaults to nothing because the usual reason to specify subgroups is to constrain the clustering rather than to show gaps.

Each level is placed independently: a level specified by numbers is laid out in the order of these numbers, and a level specified by names is laid out by the clustering. Numbering both levels therefore lays the entries out in the order of their (group, subgroup) pair, and numbering just the groups keeps the groups in a fixed order while clustering the subgroups inside each of them.

If you specify dendogram_size , then you should either specify linkage (for computing a clustering) or must specify Hclust order in the data. The dendogram tree will be shown to the side of the data. The size is specified in the usual inconvenient units (fractions of the total graph size) because Plotly.

If a dendogram tree is shown, the dendogram_line can be used to control it. The default color is black. The is_filled field shouldn't be set as it has no meaning here.

Specify how to reorder the rows and/or columns.

• OptimalHclust orders hclust branches using the (better) Bar-Joseph method.
• RCompatibleHclust orders hclust branches in the same (bad) way that R does.
• ReorderHclust reorders hclust branches to be as close as possible to a given order (using reorder_hclust ).
• SlantedHclust and SlantedPreSquaredHclust orders hclust branches using Slanter (using slanted_orders and reorder_hclust ).
• SlantedOrder and SlantedPreSquaredOrder uses slanted_orders (if a tree is needed, uses ehclust to create a tree preserving this order).
• SameOrder orders the rows/columns in the same way as the other axis. This can only be applied to square matrices and can't be specified for both axes.

struct HeatmapGraphOrder
    rows_order::AbstractVector{<:Integer}
    rows_hclust::Maybe{Hclust}
    columns_order::AbstractVector{<:Integer}
    columns_hclust::Maybe{Hclust}
end

The computed final order and clustering of the rows and the columns of a heatmap graph, as returned by heatmap_order .

• rows_order is the order of the rows of the data, that is, the index of the original row shown at each position. This is always a permutation of 1:n_rows , which for an axis that isn't reordered at all is the identity.
• rows_hclust is the tree the rows were clustered by, or nothing if they weren't clustered (they were left alone, given an explicit order, or slanted without a tree).
• columns_order and columns_hclust are the same for the columns.

These describe the order of the data, not the order it is displayed in; applying the origin is up to whoever shows the graph, as is skipping the hidden rows and columns (the order and the tree include them).

heatmap_order(graph::HeatmapGraph)::HeatmapGraphOrder

Return the HeatmapGraphOrder of a heatmap graph , that is, the final order of its rows and columns and the trees they were clustered by, without rendering it.

You can just write graph.order instead of heatmap_order(graph) . Either way the order is only computed once; showing the graph will reuse it, and vice versa.

Use this to list the entries in the order they are shown:

ordered_rows_names = graph.data.rows.names.vector[graph.order.rows_order]

Use it to show several graphs in the same order, so they can be compared. Cluster one of them, then give the rest its order (and, if they use the same groups, they will also have the same gaps):

graph.configuration.columns.reorder = OptimalHclust
other_graph.data.columns.order = graph.order.columns_order

If the graphs also show a dendogram, give them the tree instead of the order. This arranges them in the same order and draws the same tree above each of them (this only makes sense if the graphs share the same columns, as the tree refers to the original column indices):

graph.configuration.columns.reorder = OptimalHclust
graph.configuration.columns.dendogram_size = 0.1
other_graph.data.columns.order = graph.order.columns_hclust
other_graph.configuration.columns.dendogram_size = 0.1

reset_order!(graph::HeatmapGraph)::Nothing

Forget the HeatmapGraphOrder cached in the graph's final_order , so that asking for the graph's order (or showing it) will compute it again. Call this after changing anything the order was computed from.

Specify the linkage to use when performing hierarchical clustering ( hclust / ehclust ). The default is WardLinkage .

Specify where the origin (row 1 column 1) should be displayed. The Plotly default is HeatmapBottomLeft .