Dynamic filtering with .apply.select not working. Bug?

[Jan]

I am trying to understand how to use dynamic selection expressions for holoviews plots. I know that .select takes a parameter called selection_expr. This works fine with a static dim expression. But I can’t get it to work with a dynamic dim expression with a slider. I know that I have to change .select to .apply.select, but even then it doesn’t work. It gets stuck at the first value that it sees from the slider.

Here is a self contained, minimal code example:

# this code generates a scatter plot and a slider 
# so that only points > slider value will appear

import numpy as np
import holoviews as hv
from holoviews import dim
import panel as pn

hv.extension('bokeh')


threshold = pn.widgets.IntSlider(start=10, end=90, value=50, 
                                 name='minimum x value')

scatter = hv.Scatter((np.arange(100), np.random.randn(100)), 
                     kdims=['x'], vdims=['y'])

sel = (dim('x') >= threshold)       

pn.Column(threshold, scatter.apply.select(selection_expr=sel))

[carl]

Hi @Jan,

Does this maybe need to be a dynamic map thing, there’s a similar idea here in the manual that is applying outlier type stuff dynamically Data Processing Pipelines — HoloViews v1.15.0

Thanks, Carl

[Jan]

Hi @carl

I think .apply.select should generate a dynamic map out of the scatter plot when you supply a slider as an argument. It works for .apply.transform.

There is:
.apply.aggregate
.apply.reduce
.apply.opts
.apply.select
.apply.sample
.apply.transform

I was under the impression that they all should be able to handle dynamic dim expressions.

Best,
Jan

[Marc]

Hi @Jan

I was trying to find some documentation or examples supporting your assumption to figure out how to get it working. But I was not able to find any useful documentation for apply.select.

Can you share a link?

[Marc]

An alternative would be something like

import numpy as np
import holoviews as hv
from holoviews import dim
import panel as pn

hv.extension('bokeh')


threshold = pn.widgets.IntSlider(start=10, end=90, value=50, 
                                 name='minimum x value')

scatter = hv.Scatter((np.arange(100), np.random.randn(100)), 
                     kdims=['x'], vdims=['y'])

def get_plot(threshold):
    return scatter.select(dim('x') >= threshold)
plot = hv.DynamicMap(pn.bind(get_plot, threshold=threshold))

pn.Column(threshold, plot).servable()
scatter.select()

When you change the slider the plot updates.

[Jan]

Hi @Marc!

I got it from here. Below is the relevant section of the code.

Data pipelines - HackMD

window = pn.widgets.IntSlider(start=0, end=10)
sigma = pn.widgets.FloatSlider(start=0, end=3)

avg = dim('y').pd.rolling(window=window).mean()
std = dim('y').pd.rolling(window=window).std()
outliers = (np.abs(dim('y')-avg) > std * sigma)

curve = hv.Curve(some_timeseries)
curve.apply.transform(y=avg) * curve.apply.select(outliers).apply(hv.Scatter)

Unfortunately that example does not work. It gives an error message if the outlier parameter is not given explicitly using selection_expr=outlier to .select.apply()

Otherwise I wasn’t able to find an example in the official documentation. Just generally I wished the documentation would be a bit better. Going through the API, there seems to be a lot of (relatively abstract) functionality that is not really described or used anywhere. I think even if just the API had more text, that would already help a lot.

I really like holoviews and hvplot a lot, especially in combination with Bokeh. I just wished that sometimes the documentation would be clearer or more examples given.

Like I remember this one feature where you can add numbers on top of your heatmap plot, and it’s using this trick with hv.Labels(heatmap). Or that you can add a linear fit to a scatter plot with hv.Slope.from_scatter(scatter). Or that you get markers on your line plots by overlaying a scatter plot. Stuff like this is nifty, but not very intuitive. I get the feeling that there might be more “tricks” like this, but not described in the documentation.

[Jan]

Hi @Marc,

The code you show makes sense. This is probably anyway what holoviews would roughly do in the background if the example would be working.

[Jan]

Actually that link that I shared is an article from Philipp Rudiger who works on the project. So I am somewhat surprised that it’s not working. Maybe the API has changed.

Another thing: I think your workaround isn’t exactly the same. The .apply methods are supposed to also take NdLayouts, Holomaps and DynamicMaps as input and apply the function to every element individually. Maybe using

return scatter.apply.select(selection_expr=….)

would work.

[marcbernot]

Hi @Jan, it looks like a bug. The code below shows that apply.sample, apply.transform, apply.opts all lead to a live link between the widget and the output. But for apply.select, the live link seems broken(but the link is not broken since as soon as the display is refreshed through the use of the other widgets, we can see that the value of the threshold widget is correctly taken into account).

import numpy as np
import holoviews as hv
import panel as pn

hv.extension('bokeh')

ksample = 5
ds = hv.Dataset((np.arange(ksample),np.random.rand(ksample)), 'x', 'y')

size = pn.widgets.IntSlider(start=5,end=15,name='size')
yoffset = pn.widgets.FloatSlider(value=0,start=-1,end=1,step=0.05,name='yoffset')
samples = pn.widgets.CheckBoxGroup(options = list(ds['x']),inline=True,value=list(ds['x']))
threshold= pn.widgets.FloatSlider(value=1,start=0,end=1,step=0.1,name='threshold')

dynscatter = ds.apply.sample(x=samples).\
                apply.transform(y=hv.dim('y')+yoffset).\
                apply.select(selection_expr=(hv.dim('y')<=threshold)).\
                apply(hv.Scatter).\
                apply.opts(size=size,xlim=(-1,ksample),ylim=(-0.1,1.1))
        

pn.Row(pn.Column(yoffset,size,samples,threshold), dynscatter)

Would you file a bug report here?

[Jan]

Hi @marcbernot

Okay, I just submitted the issue.

Dynamic filtering with .apply.select not working