How to have a combined hover tooltip for multiple images on a tile map?

I have a tile map and two hd.regrid()-ded images overlaid on it. I can get a hover tooltip for both images if that image is overlaid alone on the map, but I have no idea how to do this with both images at the same time.

Colab Notebook example with 2x2 test images at slightly different positions:
https://colab.research.google.com/drive/1nACPvXNk-gnygTyzwUaNlnn-O1Y9i_7U?usp=sharing

One by one they work fine:
(tiles * image_population) + (tiles * image_elevation)
jmZdDmupF1

But if I overlay them on the same map (having alpha=0.5 you an see both images at the same time), I get two separate tooltips at the same position which is pretty useless:
tiles * image_population * image_elevation
AbWx8xVG5q

I can try to create a custom tooltip, but I still don’t have access to any more information:

custom_tooltips = [ 
    ('name', "$name"), # ???
    ('index', "$index"), # 0
    ('@image', '@image{0.00}'), # this is the correct value
    ('@{population_density}', '@{population_density}{0.00} person/km2'), # ???
    ('@{elevation}', '@{elevation}m') # ???
]
custom_hover = bk.models.HoverTool(tooltips=custom_tooltips)
tiles * image_population.options(tools=[custom_hover]) * image_elevation.options(tools=[custom_hover])

As this time even the size and content of the tooltips are the same, you can only see if 1 or 2 tooltips are open if you pay attention to the opacity of the tooltip.
HpeuEhiv1B

I get that the tooltips are attached to the images, so technically this is the expected functionality, but to be honest this is not very useful. I would like to have a way to access both data values in a single tooltip, similarly to the last code example.

Is there any workaround that would allow displaying multiple datasets in one tooltip? Right now I’m open to any hacky solutions, as this should be the single most important core functionality and at the same time the only missing piece of the project I’m working on. I know Bokeh and HV can display multiple values in a single tooltip, as there are many examples showcasing it for basic charts. I even know that Bokeh is capable of showing multiple values for images as there is a pure Bokeh example for that. I just have no idea how to implement that when my data is in two hd.regrid()-ded hv.Image().

Hover tool should work as long as you pass in the hover tool columns to the vdims

e.g.

import holoviews as hv

custom_tooltips = [ 
    ('b', "@b")
]
custom_hover = bk.models.HoverTool(tooltips=custom_tooltips)
hv.Image(ds, kdims=['x', 'y'], vdims=['a', 'b', 'c'])).opts(tools=[custom_hover])

reproducible example:

import xarray as xr
import holoviews as hv
import bokeh as bk

ds = xr.tutorial.open_dataset("air_temperature").isel(time=0)
ds["airx2"] = ds["air"] * 2

hover = bk.models.HoverTool(tooltips=[
    ('air 1', "@air"),
    ('air 2', "@airx2"),
])

hv.Image(ds, ["lon", "lat"], ["air", "air", "airx2"]).opts(tools=[hover])

image

If you don’t want to repeat vdim,

import xarray as xr
import holoviews as hv
import bokeh as bk

ds = xr.tutorial.open_dataset("air_temperature").isel(time=0)
ds["airx2"] = ds["air"] * 2

hover = bk.models.HoverTool(tooltips=[
    ('air 1', "@image"),
    ('air 2', "@airx2"),
])

hv.Image(ds, ["lon", "lat"], ["air", "airx2"]).opts(tools=[hover])

Thanks for the quick reply! This is not exactly what I’m looking for, but it’s getting closer!

Your example shows one dataset with 2 value dimensions. However, as you can see, I have to display 2 datasets, each with 1 value dimension. One of the main differences is that in your case both air and airx2 use the same coordinates. In my case, the 2 datasets have slightly different coordinates, so I can not simply merge their data. Also, a big difference that by having two separate images, I can display both of them with different colormaps and alphas as you can see in the examples above. But this is not a dealbreaker, as I could display the two images separately without tooltips for the visual experience and on top of them a merged image with alpha=0, tools=[hover] for the tooltip.

So if we would go in this direction, the new question would be: how to merge two images with different coordinates? You can try this with the air_temperature and rasm tutorial datasets for example. Tbh I’m pretty sure it’s impossible (but I’m open for it if doable), so most probably we’re back to the original question of somehow accessing the data for two images…

Hover tool should work as long as you pass in the hover tool columns to the vdims

e.g.

import holoviews as hv

custom_tooltips = [ 
    ('b', "@b")
]
custom_hover = bk.models.HoverTool(tooltips=custom_tooltips)
hv.Image(ds, kdims=['x', 'y'], vdims=['a', 'b', 'c'])).opts(tools=[custom_hover])

Now I learned something. I was pretty baffled why referring to value dimensions by key would work in your example and not in mine, but after that, I’ve seen that you entered the air dimension multiple times. And it turned out, that the first dimension will be assigned to @image and all the other dimension will be assigned to @their_own_key, so if you want to access the first value dimension by its own key, you have to enter it two times. This doesn’t solve my problem in itself, but it was definitely useful, thanks.

It’s not impossible, you could theoretically combine them into a single dataset by xesmf.regrid() then merge the datasets and plot it as a single dataset

You can also try using
hover_fill_alpha

Ok, so I did a quick mockup and I can confirm: if my images have the same coordinates, as a workaround I can do the following:

1: get all of their respective .data['variable01'][0] values and put into a single dataset, then create a hv.Image() from that dataset by defining all value dimensions as vdims=['variable01', 'variable01', 'variable02'].
2: overlay my original images and my new combined image somehow like this:

tilemap * image01.options(tools=[]) * image02.options(tools=[]) * combined.options(tools=['hover'], alpha=0)

The downside is that the exact same data would be processed, transferred and displayed twice, but theoretically, I could live with that.

My bigger problem is that I don’t see how this regrid should or could work. The main issue is not that I don’t know how to do that (theoretically I do, although never tried), but that it would cause data degradation. If you imagine a checkboard pattern where every second pixel has a value of 0 or 1 and you regrid that to another coordinate system that would have the same pixel size just half a pixel off to one direction, it would cut your original pixels into half and you would end up with a bunch of grey pixels as every new pixel would have the average value of a black and a white pixel. In real life, this wouldn’t be an issue when you’re zoomed out and you could only see aggregated data, but you would lose a lot of detail when being zoomed in to the actual resolution of your datasets.

And this loss of data is unnecessary. HV was already capable of displaying the two images even with different coordinate systems, and it was capable of sampling both of them for two separate tooltips at the same time. So we know the data is there, it is already available in the browser without degrading it, and it even can be displayed in two separate tooltips, so it seems much more reasonable to somehow display the exact same data in one tooltip than try to regrid all the images (in real life there are dozens of them, not just 2).

You could separate the hover from the image and put it on the side like

(Instead of a tap stream, you can make it a continuous follow Pointerxy)
http://holoviews.org/user_guide/Custom_Interactivity.html

But if you do want a single hover tooltip, it’s an underlying bokeh issue I think, or you can try writing your own javascript (no idea how though :D, https://docs.bokeh.org/en/latest/docs/user_guide/tools.html#custom-tooltip)

1 Like

Now, these are very interesting, thanks!

I can let go of the “tooltip is following the mouse” functionality if I can get a separate box updating with all the necessary information, and PointerXY could be helping me doing exactly that, so that’s a good direction to pursue.

Also, the JS recommendation gave me an idea. As I already mentioned, the tooltips themselves do get access to the data I want to display, just each to its own image, and as far as I know, there is no way of accessing the data from one image from the tooltip of another image. But Bokeh allows us to define custom JS formatters for each of our values. And in those functions, I should be able to do anything not just formatting, like exporting the data into a global variable or importing it from a global variable. So theoretically I could create a system where the tooltip of every image except the first one uses a formatter that saves the current value into a global variable (and I also would make these tooltips fully invisible) and the tooltip of the first image gets and displays these values on top of its own value. It sounds incredibly hacky, and there is a good chance it wouldn’t work for some reason (like it will be an issue to guarantee every other value got updated before displaying them), but it’s worth a try, and fortunately, I have some experience with JS.

I see what I can hack together. Thanks for the ideas, I really appreciate the effort!

This was a ride.

Note for anybody finding this thread later while looking for a way to something similar I’ve described: This is not an official solution, this is an extremely hacky workaround using some functionality for purposes they weren’t designed for, so it comes without any warranty, and also with some caveats, described at the end. However, it creates the intended result, so there’s that.

  • Yes, you can use a custom formatter to export every variable to a global pool.
  • Yes, the tooltips get generated in the same order they have been defined, so by the time the last tooltip gets generated, all the previous tooltips ran, so their variables can be already exported.
  • Yes, you can use CSS to hide every tooltip except the last one.

The biggest roadblock was for me that you can not use not-existing variables with custom formatters in your final tooltip. Imagine my original example above: we have two images, Population named population_density and Elevation named elevation. So my first strategy was to save the population value in a custom formatter of the Population image tooltip, then define the custom tooltips for the Elevation image like this:

custom_tooltips = [ 
    ('Population', '@population_density{custom} p/km2'),
    ('Elevation', '@elevation{0.00}m')
]

import_population_density = """
// Some JS code returning the previously saved population value
"""

custom_formatters = {
    '@population_density': bk.models.CustomJSHover(code=import_population_density), 
}

The problem is that since the population_density variable is not available in the Elevation image, the Bokeh engine doesn’t even run the import_population_density custom formatter I could use to import the value. There could be a workaround for two images if you highjack some other special variables like $x and force their custom formatter to give back the value of the other data, but then you would have a strong limit on the number of images you can overlay, and I needed this to work with any number of images.

Fortunately, instead of defining a list of tuples for the tooltip, you can define the exact HTML code you want to use. This means, that in the Elevation image I can use the always available @image variable (that holds the current value of elevation) to run a custom formatter that takes all the variables the other tooltips exported, imports them, and creates the full HTML code in Javascript with the added benefit of being able to customize everything. This is a way more involved process, but here it is:

# Step 1: The Population tooltip
# This tooltip will be invisible
# We only use this to export the current population data
# JS custom formatter code that takes the current value at the tooltip location
# and saves it into a global variable
export_tooltip_value = """
    var label = "%s";
    if (typeof(window['saved_tooltips'])=="undefined")
    {
        window['saved_tooltips'] = {};
    }
    window['saved_tooltips'][label] = value;
    console.log('export', label, window['saved_tooltips']);
    return value;
"""

# Filling in the JS label variable by replacing %s to 'Population'
# So the value will be saved to window['saved_tooltips']['Population']
export_population = export_tooltip_value % 'Population'

# Making sure the default dataset of the image will be displayed in this tooltip
# with a custom formatter
custom_population_tooltip = [ 
    ('Population', '@image{custom}'),
]

# Letting Bokeh know to run the custom formatter JS code when displaying this 
# tooltip
custom_population_formatters = {
    '@image': bk.models.CustomJSHover(code=export_population), 
}

# creating a custom hover tool from all the ingredients above
custom_population_hover = bk.models.HoverTool(
    tooltips=custom_population_tooltip, 
    formatters=custom_population_formatters
)

# Step 2: The Elevation tooltip
# This is the tooltip that will be visible in the end.
# So in this we have to import the previously exported values and display them.

elevation_tooltip_formatter = """

// The actual label of the value this tooltip has access to. 
// Percent-s gets replaced to the value of the python variable last_label, 
// that will be "Elevation" in this case.
var last_label = "%s";
// The actual value this tooltip has access to, that is the value of the 
// elevation data in this case.
var last_value = value;
// get a copy of the previously saved tooltips
var tooltips = {...window['saved_tooltips']};
// add the current label:value pair to the tooltips copy (elevation in this case)
tooltips[last_label] = last_value;

// The html code is copied straight from the tooltip html code normally generated by Bokeh
var ret = '<div class="bk" style="display: table; border-spacing: 2px;">';

for (const [label, value] of Object.entries(tooltips)) 
{
    // formatting each value however we want
    var display_value='';
    switch (label)
    {
    case 'Population':
        display_value = value+' p/km<sup>2</sup>';
    break;

    case 'Elevation':
        display_value = value+' m';
    break;
    }

    // displaying the labels and values
    ret += '\
    <div class="bk" style="display: table-row;">\
        <div class="bk bk-tooltip-row-label" style="display: table-cell;">'+label+': </div>\
        <div class="bk bk-tooltip-row-value" style="display: table-cell;"><span class="bk" data-value="">'+display_value+'</span><span class="bk bk-tooltip-color-block" data-swatch="" style="display: none;"> </span></div>\
    </div>\
    ';
}
ret += '</div>'; // display: table
return ret;
"""

elevation_tooltip_formatter = elevation_tooltip_formatter % 'Elevation'

custom_elevation_tooltip = """
<div class="bk bk-custom">
@image{custom}
</div>
"""

custom_elevation_formatters = {
    '@image': bk.models.CustomJSHover(code=elevation_tooltip_formatter), 
}

custom_elevation_hover = bk.models.HoverTool(tooltips=custom_elevation_tooltip, formatters=custom_elevation_formatters)

map = (tiles * image_population.options(tools=[custom_population_hover]) * image_elevation.options(tools=[custom_elevation_hover])).options(title='Custom Hover Tool')

css = pn.pane.HTML("""
<style>
.bk-tooltip.bk-tooltip-arrow:not(:last-child)
{
display: none!important;
}
</style>
""")
pn.Column(
    map,
    css
)

I updated the original Colab with this code:
https://colab.research.google.com/drive/1nACPvXNk-gnygTyzwUaNlnn-O1Y9i_7U?usp=sharing

Note that as you can see on the gif, the workaround is not perfect. If the coverage of the images is not exactly the same:

  • The tooltip only shows up when the pointer hovers the area covered by the last image. Areas where the last image is not present but previous images are don’t have a tooltip. This could be mitigated by enlarging the last image with NaN values to cover all other images.
  • When hovering an area where only the last image is present, the opposite problem appears: the tooltip shows the last known data of the previous images instead of not displaying those rows or showing NaN. This could be solved in multiple ways. Either by saving the special_vars.x and special_vars.y values while exporting and on import only using the saved values if the current special_vars.x and special_vars.y are close enough to the saved x and y values. Or the images could be enlarged and filled with NaN values.

However, these are not a problem for me as my images have only minor differences in their covered areas, so I’m not going to bother with this for now.

This solution has the benefit of being able to handle multiple images: you just have to use the same JS formatter to export the data for all the images except for the last one, and the custom formatter of the last one will be able to display them without any change.

Thanks to @ahuang11 for the tips.

Until a better solution comes up, I mark this comment as the solution.

1 Like

Wow thanks for the detailed write-up. I’m sure I’ll need it someday