Best pattern for reactive scope, watchers etc in panel?

A common set up is to have a scope with filters and other things in it that interact with various components.

Is there a better way to do this than to create a hidden placeholder with a watch? This example changes the filter on click and then the logic for how to update things is in the watch callback.

import panel as pn
import pandas as pd
import numpy as np
from string import ascii_uppercase

np.random.seed(0)

height = 300
pn.extension('tabulator')

filters = dict()

@pn.cache
def apply_filters(df, filters):
    for k, v in filters.items():
        df = df[df[k] == v]
    return df

df = pd.DataFrame(np.random.randn(100, 3), columns=['a', 'b', 'c'])
df['d'] = [ascii_uppercase[i] for i in np.random.randint(0, len(ascii_uppercase), df.shape[0])]

agg = df.groupby('d').size().to_frame('count')

df_filtered = apply_filters(df, filters)

agg_tabulator = pn.widgets.Tabulator(
    agg,
    width=400,
    height=height,
    hidden_columns=['index'],
    columns=[dict(field=k, editable=False) for k in agg.columns.tolist()],
    disabled=True,
    selectable=False,
)

df_tabulator = pn.widgets.Tabulator(
    df_filtered,
    width=400,
    height=height,
    hidden_columns=['index'],
    columns=[dict(field=k, editable=False) for k in df_filtered.columns.tolist()],
    disabled=True,
    selectable=False,
)
text = pn.pane.Markdown(f'## what {filters}')

def click(event):
    if filters.get(event.column, None) == event.value:
        filters.pop(event.column)
    else:
        filters[event.column] = event.value
    print('click event', filters)
    hidden_target.object = filters
    # text.object = f'filters: {filters}'
    # df_tabulator.value = apply_filters(df, filters)
    print('aaa', hidden_target.object)

agg_tabulator.on_click(click, column='d') 

hidden_target = pn.pane.PaneBase('init')

def watch_callback(*events):
    # RERENDER EVERYTHING AS NEEDED I GUESS, ANY OTHER WAY?
    # WE ARE STORING THE DAG LOGIC IN HERE AND IN THE CODE?
    print(f'watch callback {events}')
    for event in events:
        print(event.name, event.new, event.old)
    text.object = f'what? {filters}'
    df_tabulator.value = apply_filters(df, filters)

watcher = hidden_target.param.watch(watch_callback, ['object'], onlychanged=False)

app = pn.Column(
    text,
    agg_tabulator,
    df_tabulator,
)


app.servable()

cottrell,

Not sure if I fully grasp what you’re after in that example.
If it’s just about filtering down the 2nd table based on the row selected in the first table, it works for me even with removing the “hidden” workaround.

import panel as pn
import pandas as pd
import numpy as np
from string import ascii_uppercase

np.random.seed(0)

height = 300
pn.extension('tabulator')

filters = dict()

@pn.cache
def apply_filters(df, filters):
    for k, v in filters.items():
        df = df[df[k] == v]
    return df

df = pd.DataFrame(np.random.randn(100, 3), columns=['a', 'b', 'c'])
df['d'] = [ascii_uppercase[i] for i in np.random.randint(0, len(ascii_uppercase), df.shape[0])]

agg = df.groupby('d').size().to_frame('count')

df_filtered = apply_filters(df, filters)

agg_tabulator = pn.widgets.Tabulator(
    agg,
    width=400,
    height=height,
    hidden_columns=['index'],
    #columns=[dict(field=k, editable=False) for k in agg.columns.tolist()],
    disabled=True,
    selectable=False,
)

df_tabulator = pn.widgets.Tabulator(
    df_filtered,
    width=400,
    height=height,
    hidden_columns=['index'],
    #columns=[dict(field=k, editable=False) for k in df_filtered.columns.tolist()],
    disabled=True,
    selectable=False,
)
text = pn.pane.Markdown(f'## what {filters}')

def click(event):
    if filters.get(event.column, None) == event.value:
        filters.pop(event.column)
    else:
        filters[event.column] = event.value
    print('click event', filters)
    text.object = f'filters: {filters}'
    df_tabulator.value = apply_filters(df, filters)

agg_tabulator.on_click(click, column='d') 

app = pn.Column(
    text,
    agg_tabulator,
    df_tabulator,
)

app.servable()

At least in my test in jupyter-lab. Having said that, sometimes jupyter-lab gets messed up over time on my side (or is it MS-Edge), so if widget/plot updates don’t work I sometimes just replace the
“app.servable()” with an “app.show()” to have jupyter-lab display the UI on a standalone Web-browser without jupyter-lab involved and see if it works there (it basically kicks of a panel server)

A few other comments:

  1. I commented out the following parameters in your Tabulator widget creation.
columns=[dict(field=k, editable=False) for k in df_filtered.columns.tolist()],

“columns” is not a valid parameter for that widget and it throws an error. You don’t need it anyway.

  1. For your generic question how to handle a typical scenario where your UI app has a “Config/Control” section for INPUT and some OUTPUT ui (Plots, other Widgets) and how to link them together, that really depends on your use-case. Panel supports a variety of options, but it’s mostly one of 2 I use

a) directly link an input and output together - usually works fine if there as a 1:1 relationship (eg. Some INPUT Widget values being plotted as Output, or some INPUT Widgets directly driving some OUTPUT panel Indicator/Widget value/setting parameters ).

b) use a callback function/method to bridge between handle the INPUT changes and drive the OUTPUT changes. This is the one I use most often because it gives flexibility, easy to expand and evolve over time.

Ideas from my learning curve:

  • Need to ensure the callback procedure has access to all Input, Output and other data as needed to perform it’s function.
    So if it needs input from multiple Widgets and/or other data and needs knowledge of the OUTPUT, it makes sense to wrap all 3 things (input, output and extra data) into a class and use a method as a callback. By doing that, the method has access to everything it needs.
    I usually use a param.Parameterized as as a base, because it allows to simplify/automate a lot of the widget generation and linking/watching with Panel.

  • Another key decision I usually need to make with this method is if I just redo the OUTPUT from scratch, or if I just push the updated “values” to the existing OUTPUT. I use both variants often. If I have lots of plots (an example was a Column-layout with 15 rows, each one Button, a HTML and 2 plots with 2 Boxplot) on a page I do find the redo from scratch being faster (eg in my example 6sec vs. 30+sec for redrawing). If the Output is lots of simple stuff, or only a few plots, pushing updates is as good and more transparent to the user.

  • If you do OUTPUT from scratch and have multiple TABS etc or similar complex layouts and need to just redo a part, eg. one of several TABs, the key is to have everything in a container (column or row, or …) and just clear this container out and re-add the sub-elements. This way a parent of the container sees the update (if you just make another container, the parent (eg. a pn.Tab or something) would still refer to the old container.

Regards,
Johann