How can I replace part of a panel via a selection change in a Bokeh figure?

Normally, I am able to replace parts of a panel via pop and insert, which updates any existing panels automatically. However, if these are triggered from a bokeh selected.on_change callback,
existing panels do not update.

For example, running the following in JupyterLab works

from bokeh.plotting import figure
from bokeh.sampledata.iris import flowers
from bokeh.models import ColumnDataSource
import panel as pn


pn.extension()

def create_figure():
    src = ColumnDataSource(flowers)
    p = figure(height=200, width=200, tools='box_select')
    p.circle("petal_length", "petal_width", source=src)
    return p

pnl = pn.panel(pn.Row(create_figure, create_figure))
pnl

The displayed panel will update as expected when I run the following in the next cell:

pnl.pop(0)
pnl.insert(0, figure)

However, if I do the same thing via a callback when the selection of the column data source changes, the panel does not update as I select data points in the plot:

def replace_plot(attr, old, new):
    pnl.objects.pop(0)
    pnl.objects.insert(0, figure)

def create_figure():
    src = ColumnDataSource(flowers)
    p = figure(height=200, width=200, tools='box_select')
    p.circle("petal_length", "petal_width", source=src)
    src.selected.on_change('indices', replace_plot)
    return p

pnl = pn.panel(pn.Row(create_figure, create_figure))
pnl

What does work is to replace the entire pnl.objects with a new list:

def replace_plot(attr, old, new):
    pnl.objects = [figure]

Strangely, this only work when I call pnl.show() to display the panel in a new browser tab, in the notebook I need to display the panel again in a new cell to see the update. I tried replacing individual items in the objects list via indexing, but this worked the same as pop and insert, the panel did not update automatically.

Is there a way to replace parts of a panel vi a selected.on_change callback and have it refresh automatically (preferably inside the notebook but via show also works)?

Versions:

-----
bokeh       2.0.1
pandas      1.0.3
panel       0.9.5
sinfo       0.3.1+0.gd7037be.dirty
-----
IPython             6.5.0
jupyter_client      5.2.3
jupyter_core        4.6.3
jupyterlab          2.1.0
notebook            5.6.0
-----
Python 3.7.6 | packaged by conda-forge | (default, Mar 23 2020, 23:03:20) [GCC 7.3.0]
Linux-5.6.13-arch1-1-x86_64-with-arch
4 logical CPU cores
-----
Session information updated at 2020-05-21 18:38

I have found a workaround for this problem by wrapping two panels inside a third parent panel and using app() to run the bokeh server in the notebook:

def replace_plot(attr, old, new):
    pnl2.objects = [pn.Row(figure)]

def create_figure():
    src = ColumnDataSource(flowers)
    p = figure(height=200, width=200, tools='box_select')
    p.circle("petal_length", "petal_width", source=src)
    src.selected.on_change('indices', replace_plot)
    return p

pnl = pn.panel(pn.Row(create_figure, create_figure))
pnl2 = pn.panel(create_figure)
pn.panel(pn.Column(pnl, pnl2)).app('localhost:8888')

I don’t know if there is any downsides to wrapping two panels like this, but I would prefer to not run the bokeh server inside the notebook for this particular scenario and instead use the same approach as I do for other panels which is just displaying them without app(). So I am still interesting to a solution that can replace part of a panel and update existing panels upon selection in bokeh.

So there’s definitely some weirdness here. One thing before we get into it is that pn.panel on an existing Panel object is a no-op, so your example really just reduces to:

pnl = pn.Row(create_figure, create_figure)
pn.Column(pnl, create_figure).app('localhost:8888')

The other issue in your initial example is that it modifies .objects inplace, which doesn’t trigger any events. Panel layout objects are list like so you can call the methods directly on it, and this ensures events are generated, e.g. this:

def replace_plot(attr, old, new):
    pnl.objects.pop(0)
    pnl.objects.insert(0, figure)

reduces to:

def replace_plot(attr, old, new):
    pnl[0] = figure

But now we’re left with the remaining issue, which is that it seems like events triggered in a regular bokeh on_change callback do not themselves trigger events which send the changes back to the frontend. This is why this app does actually modify the contents of pnl but you don’t see that change appear:

def replace_plot(attr, old, new):
    pnl[0] = 1
    
def create_figure():
    src = ColumnDataSource(flowers)
    p = figure(height=200, width=200, tools='box_select')
    p.circle("petal_length", "petal_width", source=src)
    src.selected.on_change('indices', replace_plot)
    return p

pnl = pn.Row(create_figure, create_figure)
pnl

That to me seems like a bug and should be addressed. We can see what the problem is by using this hack:

from panel.io.notebook import push

def replace_plot(attr, old, new):
    for ref in pnl._models:
        _, _, doc, comm = pn.state._views[ref]
        doc.hold()
        pnl[0] = 1
        push(doc, comm)
    for ref in pnl._models:
        _, _, doc, comm = pn.state._views[ref]
        doc.unhold()

The problem seems to be that it doesn’t expect these events so the events aren’t captured and sent to the frontend. I think all we need to make sure that a Document.hold() is active at all times in the notebook. An issue would be appreciated.

Thanks for the detailed reply @philippjfr! I opened https://github.com/holoviz/panel/issues/1368 for tracking this issue. For now, I am running with your workaround and it is working well.