How to visualize missing data in Image?

I would like to plot a heatmap while accounting for gaps in the data.
I have 2-dimensional data, where on one axis there are gaps in the data.
I don’t want to ignore the “irregular sampling” and just skip the gaps, like hv.Image would do by default.
I also don’t want to increase some bins like an irregularly sampled GridMesh would do.
Instead I would like to fill the gaps with the background color (or something similar).
Is there an easier way than manually filling gaps in the input data or splitting it into subplots?

Hi @nritsche

Great questions.

I believe there can be many answers depending on your situation.

Could you provide a minimum reproducible example that show cases what you are trying to do and how. If possible add one or more pictures illustrating the question.

That will make it much easier to provide a useful answer. And for the community and developers to learn.

Thanks

Right, in this particular case even a description of the kind of data you have and what shape it is would be helpful? For example has it already been aggregated? Just as a random guess I’d suggest you give an hv.HeatMap a try.

Thanks for the responses. I tried hv.HeatMap, but firefox keeps crashing on that. If I find the time for that I can try to build a minimal example, until then, I’m going to post the code I now use to display 2D data with gaps in either axis in hv.Images using overlays:

import holoviews as hv
import numpy as np


def hv_image_with_gaps(x, y, z, gap_scale=0.1, opts=None, *args, **kwargs):
    """Produce an overlay with images accounting for gaps in the data.

    Parameters
    ----------
    x, y : np.ndarray[:]
        Location of pixel centres in each direction
    z : np.ndarray[:, :]
        Pixel (z-)values
    gap_scale : float, optional
        If there is an extra gap between pixels of this amount times the nominal
        separation, consider this a gap in the data.
    opts : dict


    Returns
    -------
    overlay : list
        holoviews Overlay with an Image for each uninterruptedly sampled section.
    """

    def _find_splits(ax):
        d = np.diff(ax)
        md = np.median(d)

        ranges = []

        last_cut = 0
        for ii, di in enumerate(d):
            if np.abs(di - md) > np.abs(gap_scale * md):
                ranges.append((last_cut, ii + 1))
                last_cut = ii + 1

        ranges.append((last_cut, len(ax)))
        return ranges

    if opts is None:
        opts = {}
    overlay = None
    x_splits = _find_splits(x)
    y_splits = _find_splits(y)
    for xs, xe in x_splits:
        for ys, ye in y_splits:
            xa = x[xs:xe]
            ya = y[ys:ye]
            ca = z[ys:ye, xs:xe]

            img = hv.Image(
                (xa, ya, ca),
                datatype=["image", "grid"],
                *args, **kwargs,
            ).opts(**opts)

            # Create overlay
            if overlay is None:
                overlay = img
            else:
                overlay *= img

    return overlay

When testing hv.HeatMap, be careful not to call your notebook HeatMap.ipynb, because then various spam filters suppress the code it’s running (apparently because they think HeatMap can only refer to click-tracking code!). If that’s your trouble, just rename the notebook to literally anything else before running it.

In any case, I’m confused by the question, because hv.Image doesn’t “skip the gaps”; it should automatically make those pixels transparent if the data is NaN. You can then set the background color to something that will show up well given your colormap. This should all be the default behavior already for hv.Image with floating-point arrays (as integer arrays can’t have NaN).

1 Like