Heatmaps

Graphs for showing a 2D matrix.

A graph showing a heatmap. See HeatmapGraphData and HeatmapGraphConfiguration .

function heatmap_graph(;
    [figure_title::Maybe{AbstractString} = nothing,
    entries::MatrixValuesData = MatrixValuesData(),
    cells::MatrixEntitiesData = MatrixEntitiesData(),
    rows::HeatmapSideData = HeatmapSideData(),
    columns::HeatmapSideData = HeatmapSideData(),
    configuration::HeatmapGraphConfiguration = HeatmapGraphConfiguration()]
)::HeatmapGraph

Create a HeatmapGraph by initializing only the HeatmapGraphData fields (with an optional HeatmapGraphConfiguration ).

@kwdef mutable struct HeatmapGraphData <: AbstractGraphData
    figure_title::Maybe{AbstractString} = nothing
    entries::MatrixValuesData = MatrixValuesData()
    cells::MatrixEntitiesData = MatrixEntitiesData()
    rows::HeatmapSideData = HeatmapSideData()
    columns::HeatmapSideData = HeatmapSideData()
end

The data for a graph showing a heatmap (matrix) of entries.

This is shown as a 2D image where each matrix entry is a small rectangle with some color. Due to Plotly limitation, colors must be continuous. The entries values are required; their title is the title of the colors scale. The cells hold the hovers of the entries. The hover for each rectangle is a combination of the hovers of the cell, of its row and of its column.

The rows and columns hold the data of each side (see HeatmapSideData ). A cell is shown if both its row and its column are shown; there is no mask of its own. The values of the hidden cells still take part in the range of the colors scale, unless include_hidden is disabled in the entries.colors.scale of the configuration.

The order of the entities of each side and the hclust of its arrangement (see ArrangementData ) take part in the layout of the side as described in HeatmapSideConfiguration . The groups and subgroups of the arrangement constrain a ClusteredTree . They can still be specified to denote gaps in the heatmap, even when they do not impact the tree and/or order.

@kwdef mutable struct HeatmapSideData
    entities::VectorEntitiesData = VectorEntitiesData()
    arrangement::ArrangementData = ArrangementData()
    annotations::AbstractVector{AnnotationData} = AnnotationData[]
    annotations_order::Maybe{AbstractVector{<:Integer}} = nothing
end

The data of one side (the rows or the columns) of a HeatmapGraphData . The entities hold the names, hovers, mask and order of the entries; their names are shown as the tick labels, titled by the title of the side configuration. The arrangement holds what else the entries are arranged by. The annotations are shown next to the entries. If annotations_order is specified, they are shown in that order; it describes all the annotations, including the ones that are not is_shown .

Hidden entries (see the mask of VectorEntitiesData ) are not drawn, but they are still part of the data: the clustering sees them (unless the include_hidden of the side configuration is false ), and the order (a permutation or a tree) always describes all the entries, hidden ones included. This way the order computed for one graph (see heatmap_placement ) can be given to another graph of the same data, whether or not the two hide the same entries. At least one entry must be shown.

@kwdef mutable struct HeatmapGraphConfiguration <: AbstractGraphConfiguration
    figure::FigureConfiguration = FigureConfiguration()
    entries::EntriesConfiguration = EntriesConfiguration()
    rows::HeatmapSideConfiguration = HeatmapSideConfiguration()
    columns::HeatmapSideConfiguration = HeatmapSideConfiguration()
    origin::HeatmapOrigin = HeatmapBottomLeft
    final_placement::Maybe{HeatmapGraphPlacement} = nothing
end

Configure a graph showing a heatmap.

This displays a matrix of values using a rectangle at each position. Due to Plotly's limitations, you still to manually tweak the graph size for best results; there's no way to directly control the width and height of the rectangles.

The entries configure the entries (see EntriesConfiguration ); the rows and columns configure each side (see HeatmapSideConfiguration ).

The final_placement caches the computed placement of the rows and the columns; access it through the graph's placement (e.g., for generating other graphs in an identical placement). It is computed once, whether the graph's figure is generated or its placement is asked for first.

@kwdef mutable struct EntriesConfiguration <: Validated
    colors::ColorsConfiguration = ColorsConfiguration()
end

Configure the entries of a heatmap. The colors map the values of the entries to colors. Due to Plotly's limitations, only continuous color palettes are supported, and a fixed color can't be specified.

@kwdef mutable struct HeatmapSideConfiguration <: Validated
    title::Maybe{AbstractString} = nothing
    show_ticks::Bool = true
    ticks_angle::Maybe{Real} = nothing
    annotations::AnnotationSize = AnnotationSize()
    tree_source::Maybe{TreeSource} = nothing
    order_source::Maybe{OrderSource} = nothing
    linkage::Maybe{HeatmapLinkage} = nothing
    metric::Maybe{PreMetric} = nothing
    include_hidden::Bool = true
    groups_gap::Maybe{Integer} = 1
    subgroups_gap::Maybe{Integer} = nothing
    total_gaps_fraction::Maybe{Real} = 1 / 20
    dendogram_size::Maybe{Integer} = nothing
    dendogram_line::LineConfiguration = LineConfiguration()
end

Configure one side (the rows or the columns) of a heatmap. The title is shown on the axis of the side. The annotations are the sizes of the annotations shown next to the entries.

The entries are labelled by their names from the HeatmapSideData , if they have any. Set show_ticks to false to leave them unlabelled, which is what a side holding thousands of entries wants; the names are still the first line of each hover. By default the labels are shown horizontally, and ticks_angle rotates them (relative to the axis, as in AxisConfiguration ). The titles of the other side's annotations line up with these labels, so they follow a set ticks_angle too. By default, a title below an annotation stands up, so the titles of neighbouring annotations don't overlap.

The layout of the side is an order, and optionally a tree for the dendogram. The tree_source says where the tree comes from (see TreeSource ) and the order_source says where the order comes from (see OrderSource ). A tree is needed when any field depends on one: the dendogram is shown, a tree_source is specified, the order_source is GivenTreeOrder or a tree-reorder, or an hclust is given in the arrangement. Otherwise no tree is built, and specifying linkage or metric is an error. Nothing that is specified is ever ignored.

"Toward the target" means the branches of the tree are reordered to be as close as possible to the target order, so the final order is the leaves and only approximates the target. "Around the target" means the tree is built so that its leaves are exactly the target.

tree source	hclust	order source	order	final tree, when needed	final order
Given	required	GivenTreeOrder	forbidden	given	leaves, as is
Given	required	tree-reorder	forbidden	INVALID: Clustering.jl has no public way to reorder the branches of an existing tree
Given	required	target-order	GivenOrder	given	leaves, toward the target
Same	forbidden	GivenTreeOrder	forbidden	the other side's	leaves, as is
Same	forbidden	tree-reorder	forbidden	INVALID, as for Given
Same	forbidden	target-order	GivenOrder	the other side's	leaves, toward the target
Clustered	forbidden	tree-reorder	forbidden	clustered	leaves, reordered
Clustered	forbidden	target-order	GivenOrder	clustered	leaves, toward the target
Order	forbidden	target-order	GivenOrder	around the target	the target
(none)	forbidden	target-order	GivenOrder		the target
anything else				INVALID

The order column says which order source requires the order data; the other sources forbid it. The invalid remainder is GivenTreeOrder without a given or same tree, OrderTree with GivenTreeOrder or a tree-reorder, which is circular, and a tree-reorder of a given or same tree, which is not implemented.

SameOrder is a plain target-order. With SameTree it copies the layout of the other side; with OrderTree it builds this side's tree around the other's order; with ClusteredTree it clusters toward it.

Both sources default to nothing , which infers them from what is given:

order given	hclust given	tree needed	tree source	order source
no	no	no	(none)	EntryOrder
no	no	yes	ClusteredTree	OptimalTreeReorder
yes	no	no	(none)	GivenOrder
yes	no	yes	OrderTree	GivenOrder
no	yes	always	GivenTree	GivenTreeOrder
yes	yes	always	GivenTree	GivenOrder

One source may be inferred while the other is explicit. An order with an explicit ClusteredTree infers GivenOrder (cluster, then toward the given order). An explicit SlantedOrder with a dendogram infers OrderTree . An explicit SameOrder with a tree needed and nothing given infers SameTree . An explicit SameTree infers SameOrder .

A ClusteredTree or an OrderTree is built using the linkage (by default, WardLinkage ) and the metric (by default, Euclidean ). Neither applies to a GivenTree or a SameTree , so specifying them for one is an error.

By default, a computed clustering sees all the entries of the side, hidden ones included, so hiding some entries does not move the rest. Set include_hidden to false to cluster the shown entries only. Either way the resulting order and tree (see heatmap_placement ) describe all the entries; when the hidden ones were left out of the clustering, they come last, joined to the root of the tree. This has no effect on an Hclust given in the data, which is used as is.

If groups are specified for the entries in the HeatmapSideData , they can be used to constrain the clustering, and/or to create visible gaps in the heatmap (between entries of different groups). The groups_gap is the number of fake entries to add between the separated entries. That is, the default gap of 1 will add a blank gap of one entry between adjacent entries of different groups. A gap of nothing will not be shown.

If subgroups are also specified, they are a second, finer level of grouping nested in the groups; each group is contiguous, and within it each subgroup is contiguous. Their subgroups_gap works the same way, and defaults to nothing because the usual reason to specify subgroups is to constrain the clustering rather than to show gaps. A subgroups_gap requires subgroups.

Groups (or subgroups) which have no effect are rejected: if the side is not clustered, they must be shown with a gap. Likewise, an arrange_by matrix (in the HeatmapSideData ) must have an effect.

A gap of a fixed number of entries is invisible in a side holding thousands of them, and is most of a side holding a few dozen. The total_gaps_fraction fixes this. The gaps are widened together, keeping their ratio, until they take this fraction of the axis. They are never narrowed, so groups_gap and subgroups_gap are minimums. Set it to nothing to use the gaps as given.

Each level is placed independently: a level specified by numbers is laid out in the order of these numbers, and a level specified by names is laid out by the clustering. Numbering both levels therefore lays the entries out in the order of their (group, subgroup) pair, and numbering just the groups keeps the groups in a fixed order while clustering the subgroups inside each of them.

If you specify dendogram_size , the tree (see above) is shown to the side of the data. The size is in pixels.

If a dendogram tree is shown, the dendogram_line can be used to control it. The default color is black. The is_filled field shouldn't be set as it has no meaning here.

Specify where the tree of a heatmap side comes from.

• GivenTree uses the hclust of the arrangement.
• ClusteredTree clusters the data (using the linkage and metric , and the groups and arrange_by of the arrangement).
• OrderTree builds a tree around the target order (using ehclust ), so the leaves are exactly the target order.
• SameTree uses the tree of the other side. This can only be applied to square matrices and can't be specified for both sides.

A tree is needed when any field depends on one: the dendogram is shown, a tree_source is specified, the order_source is GivenTreeOrder or a tree-reorder, or an hclust is given. Otherwise no tree is built.

Specify where the order of a heatmap side comes from.

• GivenTreeOrder uses the leaves of the given tree, as they are.
• OptimalTreeReorder and RCompatibleTreeReorder are tree-reorders. They reorder the branches of the tree (using the (better) Bar-Joseph method, or in the same (bad) way that R does) and use its leaves.
• GivenOrder uses the order of the entities.
• EntryOrder uses the entries as they are.
• SlantedOrder and SlantedPreSquaredOrder slant the data (using slanted_orders ).
• SameOrder uses the order of the other side. This can only be applied to square matrices and can't be specified for both sides.

The last five name a target order. When a tree is needed, the branches of a given, same or clustered tree are reordered toward the target (using reorder_hclust ), so the final order only approximates the target. An OrderTree is built around the target, so the final order is the target.

struct HeatmapGraphPlacement
    rows::SidePlacement
    columns::SidePlacement
end

The computed SidePlacement of the rows and of the columns of a heatmap graph, as returned by heatmap_placement .

heatmap_placement(graph::HeatmapGraph)::HeatmapGraphPlacement

Return the HeatmapGraphPlacement of a heatmap graph , that is, the final order of its rows and columns and the trees they were placed by, without rendering it.

You can just write graph.placement instead of heatmap_placement(graph) . Either way the placement is only computed once; showing the graph will reuse it, and vice versa.

Use this to list the entries in the order they are shown:

ordered_rows_names = graph.data.rows.entities.names[graph.placement.rows.order]

Use it to show several graphs in the same order, so they can be compared. Cluster one of them, then give the rest its order (and, if they use the same groups, they will also have the same gaps):

graph.configuration.columns.order_source = OptimalTreeReorder
other_graph.data.columns.entities.order = graph.placement.columns.order

If the graphs also show a dendogram, give them the tree instead of the order. This arranges them in the same order and draws the same tree above each of them (this only makes sense if the graphs share the same columns, as the tree refers to the original column indices):

graph.configuration.columns.dendogram_size = 0.1
other_graph.data.columns.arrangement.hclust = graph.placement.columns.hclust
other_graph.configuration.columns.dendogram_size = 0.1

reset_placement!(graph::HeatmapGraph)::Nothing

Forget the HeatmapGraphPlacement cached in the graph's final_placement , so that asking for the graph's placement (or showing it) will compute it again. Call this after changing anything the placement was computed from.

Specify the linkage to use when performing hierarchical clustering ( hclust / ehclust ). The default is WardLinkage .

Specify where the origin (row 1 column 1) should be displayed. The Plotly default is HeatmapBottomLeft .