DataShader Georeferenced Points with Base Map

Datashader
1

I am looking to go from a multi million Pandas dataframe with lon, lat and 4 other numerical columns to a Datashaded image with ideally hover tool but definitely a base map. I managed to simply use the basic example from the Datashader quickstart page but am not able to add a base map to it. By the way I if it is possible to use datashade with hvplot with this ammount of data and a hover tool that would be ideal but so far I see it have reprojection isses and the hover tool does not work even with index, hover_cols, rasterize and other options permutations.

Here is a simple example where I create a Datashader image and then try to make an interactive hvplot with this example data.

import pandas as pd
import sqlite3
import datashader as ds
import colorcet
import hvplot.pandas
import geopandas
from holoviews.operation import apply_when
from holoviews.operation.datashader import rasterize

df_pq = pd.read_parquet('much_better.pq')
df_pq =df_pq.rename(columns={'HLAT': 'lat', 'HLON': 'lon'})

# Tried with GeoPandas but it takes impossibly long
# gdf = geopandas.GeoDataFrame(
#    df_pq, geometry=geopandas.points_from_xy(df_pq.lon, df_pq.lat)
# )
# gdf.hvplot(rasterize=True, geo=True, tiles=True)

cvs = ds.Canvas(plot_width=850, plot_height=500)
agg = cvs.points(df_pq, 'lon', 'lat')
img = ds.tf.shade(agg, cmap=colorcet.fire, how='log')
img

points = df_pq.hvplot.points(
    x='lon', 
    y='lat',
    width=1200, 
    height=700, 
    coastline=True,
    rasterize=True,
    # datashade=True,
    # hover=True,
    # cmap='hot', 
    geo=True,
    # tiles='OSM',
    # hover_cols='all',
    # hover_cols=['OBSVALUE','TSONDE','OBS_ERROR','FLAG'], 
    # use_index=False
)

apply_when(
    points,
    operation=rasterize,
    predicate=lambda x: len(x) > 100
)
1 Like
2

Hi @StuckDuckF

It is really hard to try to understand the problem and help without a minimal, reproducible code example with data to start from.

Would it be possible to provide?

1 Like
3

Yes Mr. @Marc sorry for the delay had to clear it with my superior. I have added my code and a modified example data set.

1 Like
4

Mr. @jbednar and the good people behind ar.py have given me an avenue for achieving what I want using apply_when with their suggestion, but I keep running into exotic and novel errors (Xarray DataError at the moment). Mr. @Marc and Mr. @jbednar apart from not using coastlines are there any other particularities I should be aware of in order to rasterize but keep the hover functionality for a very large Pandas.DataFrame that is fed into a HvPlot.Points ?

5

I don’t think that’s enough detail to be able to say much more. You should be able to use coastlines if you project the data into Web Mercator before you plot them, so that everything is in the same coordinate system without having to reproject interactively. I don’t think that affects apply_when in any specific way, but if you’re seeing otherwise, we’d have to have a concrete example to debug it.

6

There are example files and the script that generates the hvplot object Mr. @jbednar . I do not think I am missing anything and I still can not get multiple columns to show on hover and the apply_when errors out as described sir.

7

I don’t see apply_when in the example above, so I don’t see how to replicate the issue.

1 Like
8

I’m sorry about that. I just edited the question to add the few lines of code.