I am in process of creating a FastAPP-based Panel app serving, among others, a visualization app which consists of the following components:
- 1 DateRangeSlider widget,
- 1 Float slider widget,
- 1 scatter hvplot of around 1 million points, and
- 3 histagrams ploting the count of the ranges in ‘Speed’, ‘Altitude’ and ‘Accuracy’ fo rth eall GPS coordinates.
- some data informative displays showing ‘Colories burned’, ‘Avrage CO2 sparement’, ‘Trafic Rate vs Time’, etc etc (code not presented).
My GPS data points are collected over a period of time which means that I can do selection based on timestamp, hence having a DateRangeSlider.
So far the original data file consists of around 10 million records (GPS poins). Here I am providing around around 2 million points only, related to one of the cities in the project. The data that I am providing is not “cleaned” and contains only 16 needed columns for this topic.
I am facing two scenarios:
A. If I want to show all the data by rasterizing the scatter hvplot then I can keep the “link selection” between all 4 plots but the sliders can only be connected to the 3 histograms (by using ‘.interactive()’). If I try to connect the sliders to the scatter plot then I get the “Exception: Nesting a DynamicMap inside a DynamicMap is not supported” error message. Disadvantage: I can not see the “date” effect on the visualization.
B. If I want to have/use the ‘link selection’ between the 3 plots and at the same time keep the interactivity between the plots and the sliders then i have to reduce the number of GPS data points by a factor of 0.03 and disable rasterize/datashader function. Disadvantage: only small part of the data is visualized due to the limitation of the browser. Also the plotted points are very grainy.
Now: I want the best of the both worlds! I want to have all the GPS data points in the plot and in rasterized format while keeping the interactivity between all the plots and the slider widgets. Is it possible? What/how should I do?
I feel that I have missed something very obvious but I do not know what!
A video about the “problem”:
# General modules import pandas as pd import panel as pn from panel.theme import Material import colorcet as cc import datashader as ds import hvplot.pandas import pyarrow.parquet # Holoviews import holoviews as hv from holoviews.operation.datashader import datashade, rasterize from holoviews.selection import link_selections hv.extension('bokeh') pn.extension('tabulator') pn.config.design = Material # Read the parquet file into a data frame taken from Glaciers project def load_data(): gps = pd.read_parquet('Karlskrona_public.parquet', engine='pyarrow') gps['latdeg'] = gps.northing gps['x'], gps['y'] = (gps.easting, gps.northing) return gps # Cache the data gps = pn.state.as_cached('glaciers', load_data) # Create a sample data frame (used if not datashader is applied) rgps = gps.sample(frac=0.3) rgps.head() # Make the dataset interactive (https://holoviz.org/tutorial/Interactive_Pipelines.html) gpsi = gps.interactive() # Make 'Date' range sliders (https://panel.holoviz.org/reference/widgets/DateRangeSlider.html) date_slider = pn.widgets.DateRangeSlider( name='Date', start=gps.time.iloc, end=gps.time.iloc[-1]) # Filter the dataframe interactively using the time slider # Interactive Time Filtered Data = itfd itfd = gpsi[['time', 'speed', 'accuracy', 'altitude', 'easting', 'northing']][ (gpsi['time'] >= date_slider.param.value_start) & (gpsi['time'] <= date_slider.param.value_end) ] # Define link selection between components ls = hv.link_selections.instance(unselected_alpha=0.02) # Plot the GPS points # Alternative plot: using hv.Points # USING 'gpsi' INSTEAD OF 'gps' WILL GIVE RAISE TO "NESTED ..." ERROR gps_plot = gps.hvplot('easting', 'northing', kind='scatter', rasterize=True, cmap=cc.fire, cnorm='eq_hist', xlabel='Longitude', ylabel='Latitude', responsive=True, height=600 ) # link selection on scatter plot lsgp = ls(gps_plot) # Group the plot with the background map geo_plot = hv.element.tiles.OSM() * lsgp # Plot the Speed histogram 'sph' # NOTE: if 'gps' is used as the data for the plot, the connection to the date range slider breaks. sph = itfd.hvplot( y='speed', xlabel='Speed', ylabel='Count', kind='hist', responsive=True, min_height=200, fill_color='#88d8b0', alpha=1.0) # groupby='activity.activity_type', ????? #f1948a # link selection on speed hist (lssh) lssph = ls(sph.holoviews()) # .holoviews() # Plot the Accuracy histogram 'ach' ach = itfd.hvplot( y='accuracy', xlabel='Accuracy', ylabel='Count', kind='hist', responsive=True, min_height=200, fill_color='#85c1e9', alpha=1.0) # link selection on accuracy hist (lsah) lsach = ls(ach.holoviews()) # .holoviews() # Plot the Altitude histogram 'alh' alh = itfd.hvplot( # IF 'gps' IS used INSTEAD OF 'itfd' THE GRAPHS WILL BE EMPTY y='altitude', xlabel='Altitude', ylabel='Count', kind='hist', responsive=True, min_height=200, fill_color='#34d1e2', alpha=1.0) # link selection on accuracy hist (lsalh) lsalh = ls(alh.holoviews()) # .holoviews() # Group the histograms slshists = pn.Row(lssph, lsach, lsalh) # Create a linked data-points table table = (itfd.pipe(ls.filter, selection_expr=ls.param.selection_expr) .drop(columns=['easting', 'northing']) .pipe(pn.widgets.Tabulator, pagination='remote', page_size=15, theme='fast') ) # Creating tabs to use in the layout tabs = pn.Tabs( ('Plots', pn.Row(pn.Column(geo_plot, slshists))), ('Table', table.panel()), sizing_mode='stretch_width' ) # ---------------------------------------------------------------------------------- # Creating Dashboard: Define layout dynamic_layout = pn.template.FastListTemplate( title='Project - Visualization', sidebar=[ pn.Row(pn.Column(pn.pane.Markdown("### Some text about the project"), pn.pane.Markdown("Select date:"), pn.Column(date_slider), pn.pane.Markdown("Some other things down here"))) ], main=pn.Row(tabs), theme_toggle=False) dynamic_layout.servable();
The data file:
Send me a “Hi” if you have questions