Param.depends(... watch=True) does not trigger for widgets?

I’ve thoroughly documented the following file, and have added some questions for parts that I don’t understand. Have I made any mistakes?

In particular, does anyone know why @param.depends('_start_date.value', '_end_date.value', 'percent', watch=True) only works for the parameter percent and if there is anyway to make it work for widgets?

"""
Alluding to the React paradigm of "lifting state" this file demonstrates how to pass a widget into
a child class that will then respond to change in the parent widgets
"""
import panel as pn
import param


class Child(param.Parameterized):
    """The child class will only keep track of percentile
    But should respond to changes in the parent classes start/end dates
    """
    percent = param.Number()

    def __init__(self, start_date_widget, end_date_widet):
        """Call super to initialize percent as a parameter and store start and end date values"""
        super().__init__()
        self._start_date = start_date_widget
        self._end_date = end_date_widet

    # @param.depends('_start_date.value', '_end_date.value', 'percent', watch=True)
    # This is interesting and I don't really understand it...
    # although below, when start_date and end_date change, the view updates, for this function, 
    # using the above depends decorator only causes an update for percent. Changes to _start_date
    # and _end_date have no effect
    # @pn.depends('_start_date.value', '_end_date.value', 'percent') has absolutely no effect
    def side_effect(self):
        """Functions that are "side effects" as in they are not directly rendered by param must
        include @param.depends(..., watch=True) in order to update when the parameters change
        """
        print(f'Values changed: {self._start_date.value}, {self._end_date.value}, {self.percent}')

    @param.depends('_start_date.value', '_end_date.value', 'percent')
    def view(self):
        """
        @param.depends(...) ONLY generates "a hint to Panel and other Param-based libraries 
        (e.g. HoloViews) that the method should be re-evaluated when a parameter changes". The
        important thing to note here is that the function view must be nested in a panel object w/o
        being called in order to update. 
        """
        return pn.Column(
            # pn.Param(...) takes in a parameter and renders a widget. 
            # pn.Param(self.percent) doesn't work 
            # pn.Param(self.param.percent) DOES work.
            pn.Param(self.param.percent),

            # notice we use "self.percent", but "self._start_date.value". 
            # perent is a param, and start date is a widget. They are different types!
            pn.panel(f'## {self.percent}, {self._start_date.value}, {self._end_date.value}'),
        )

# Here we create our "super class" widgets
start_date = pn.widgets.DatePicker()
end_date = pn.widgets.DatePicker()

# widgets are passed into child class
child = Child(start_date, end_date)

# widgets are rendered. Note that child.view is not called, the function is passed indicating to
# panel that it should be re run every time the hinted parameters in @param.depends() are changed
pn.Row(start_date, end_date, child.view).servable()

# this doesn't work, because child.view is CALLED:
"""" pn.Row(start_date, end_date, child.view()).servable() """

You need to define the widgets before super().__init__()

class Child(param.Parameterized):
    """The child class will only keep track of percentile
    But should respond to changes in the parent classes start/end dates
    """
    percent = param.Number()

    def __init__(self, start_date_widget, end_date_widet, **params):
        """Call super to initialize percent as a parameter and store start and end date values"""
        self._start_date = start_date_widget
        self._end_date = end_date_widet
        super().__init__(**params)

    # @param.depends('_start_date.value', '_end_date.value', 'percent', watch=True)
    # This is interesting and I don't really understand it...
    # although below, when start_date and end_date change, the view updates, for this function,
    # using the above depends decorator only causes an update for percent. Changes to _start_date
    # and _end_date have no effect
    @param.depends('_start_date.value', '_end_date.value', 'percent', watch=True)  
    def side_effect(self):
        """Functions that are "side effects" as in they are not directly rendered by param must
        include @param.depends(..., watch=True) in order to update when the parameters change
        """
        print(f'Values changed: {self._start_date.value}, {self._end_date.value}, {self.percent}')

    @param.depends('_start_date.value', '_end_date.value', 'percent')
    def view(self):
        """
        @param.depends(...) ONLY generates "a hint to Panel and other Param-based libraries
        (e.g. HoloViews) that the method should be re-evaluated when a parameter changes". The
        important thing to note here is that the function view must be nested in a panel object w/o
        being called in order to update.
        """
        return pn.Column(
            # pn.Param(...) takes in a parameter and renders a widget.
            # pn.Param(self.percent) doesn't work
            # pn.Param(self.param.percent) DOES work.
            pn.Param(self.param.percent),

            # notice we use "self.percent", but "self._start_date.value".
            # perent is a param, and start date is a widget. They are different types!
            pn.panel(f'## {self.percent}, {self._start_date.value}, {self._end_date.value}'),
        )

   ...
1 Like

A typical use case of Param, that is certainly why it was created in the first place, is to be able to declare upfront the types of your class attributes/fields - i.e. Parameters in Param - and their properties. This is very useful even outside of the context of a Panel app, e.g.:

import param

class Algo(param.Parameterized):
    timestep = param.Integer(0, bounds=(0, Non), doc="Time step in seconds")
    alpha = param.Number(0.5, bounds=(0, 1), doc="...")

On top of that Panel allows you to register callbacks that are executed when a Parameter value changes:

  • by decorating a method/function with e.g. @param.depends('timestamp')
  • or with the more lower-level API, e.g. .param.watch(callback, 'timestamp')

Registering callbacks is super useful for GUI programming, it’s used extensively internally by Panel and HoloViews.

When Param and Panel are combined to create a Panel app as you did, this becomes a very powerful approach to maintain your code:

  • Parameters (like param.Integer, param.String, etc.) are automatically mapped to widgets. For instance, an Integer Parameter is mapped to an IntSlider widget. If you’re unhappy with the default mapping or want to further customize the widgets you obtained, Panel provides APIs to do achieve that easily. So how that works then is that when you ask Panel to render a Parameter, it will actually be displayed as a widget (e.g. pn.panel(algo.param.timestamp) will render as an IntSlider widget). This allows to separate your code logic from GUI programming, you just deal with normal instance attributes as you normally do in a Python class, not directly with widgets.
  • When Panel is given a method decorated with e.g. @param.depends('timestamp') it will automatically re-execute it and re-render its output when one of the parameters watched changes.

Given all that, you could change your code in two ways:

  • You decide to use the Param/Panel approach to its best in which case you don’t need to pass any widget to the Child class. Instead to declare two additional param.CalendarDate Parameters, which when displayed will render as DatePickers. With this approach your callback methods just need to depend on the class Parameters with e.g. @param.depends('start_date', 'end_date').
  • If you still prefer to pass widgets to the class, you can actually declare that this class has two widgets Parameters/fields with e.g. start_date_widget = param.ClassSelector(class_=pn.widgets.DatePicker, is_instance=True) and instantiate the class with these parameters.

The solution offered by @Hoxbro is also a valid approach!

1 Like

Thanks @Hoxbro for the quick and dirty fix, it totally worked. I guess that’s because on init param registers all parameters of a class with the dependency system?

Thanks @maximlt for the detailed suggestions and context. I deeply appreciate the response, and have some questions:

I think I understand the separation of GUI and Logic concept. My situation involves some “global filters” that are instantiated on the template.sidebar and effect many of these Child classes that I’ve asked about. Thus, my situation (I think) requires me to pass in existing widgets such that Child_1, …, Child_n all update when the template.sidebar widgets update. In this case I don’t think that I want each Child to create it’s own widgets. Would you agree?

Given this context, it seems as though option 2 might be the best path forward (because it is indeed ugly that some parameters I access transparently, and some are access with .value. Unfortuntely, I must be misunderstanding your instructions. Here is how I interpreted:
you can actually declare that this class has two widgets Parameters/fields with e.g. start_date_widget = param.ClassSelector(class_=pn.widgets.DatePicker, is_instance=True) and instantiate the class with these parameters

class PanelLinePlot(param.Parameterized):
    """
    start_date / end_date are filters in the sidebar
    percentile is a filter that is part of the component
    df_filter is a global filter that doesn't cause a presto refetch
    """

    table = param.Selector(allow_None=False)
    percentile = param.Number(softbounds=[.1,.9])
    df = param.DataFrame(pd.DataFrame())

    start_date = param.ClassSelector(class_=pn.widgets.DatePicker)
    end_date = param.ClassSelector(class_=pn.widgets.DatePicker)

     # **params = {"start_date"=start_date_sidebar_widtet, "end_date"=end_date_sidebar_widget}
     # this is convenient becuase I can create a dict of all the sidebar widets and pass them in together
    def __init__(self, percentile, filter_col, filter_vals, connection, **params):
        self.percentile = percentile
        self.filter_vals = filter_vals
        self.filter_col = filter_col
        self.conn = connection

        # parameters above this call will be registered in the param dependency system
        super().__init__(**params)
        self.get_data()

    # global widgets use .value, local widgets don't
    @param.depends('percentile', 'start_date', 'end_date', 'filter_col.value', watch=True)
    def get_data(self):
        """fetch data from presto with the latest parameters"""
        print('initiating data fetch')
        df = self.conn.sql(sql_pct_lineplot(
            group_col=self.filter_col.value,
            value_col='capacityImbalancePct_max',
            start_date=self.start_date.value,
            end_date=self.end_date.value,
            percentile=self.percentile,
            table='bms_developers.cac_v3',
        ))
        df['event_date'] = pd.to_datetime(df['event_date'])
        self.df = df

And this also works, but I wanted to check to ensure that this is what you had in mind, as I still have to use .value to access the widget values.

2 Likes

Could you give me a more concrete example of what these global filters are and what effects and UI you expect them to have?

1 Like

Sure,

Below is my current progress on the project. The sidebar contains start/end date, the “grouping column”, and the groups that are filtered. Suppose there are N plots similar to the first CAC imbalance plot which are also managed by the same start/end date and grouping column.

When the start/end date values change, we fetch a new DataFrame from presto. The same thing happens when the grouping column changes.

Why not fetch all the data in one go?

It’s certainly something that I’m considering with this suite of tools. Currently the lineplot only shows the nth percentile of the fleet’s data. This is because there are over a million vehicles in the fleet, and thus we would need 1 million x N days of data which adds up to be quite a lot. I’m very open to modifying things to use datashading and actually plot a line for each vehicle, a-la: Timeseries — Datashader v0.14.0
However it would be important to maintain other information about the vehicle such as firmware version per day, and cell type etc. Thus we (I think) cannot have a dataframe where each row represents a fixed-length curve (from above link).

1 Like

Here’s I think a way to achieve what you want, with two global filters that are injected in two views, each view having its own set of widgets and visual representation. I’ve adapted some of my existing code but I’m sure it’s possible to reduce some of the boiler plate code in the views, if I find some time I’ll have a look at it later.

import datetime

import hvplot.pandas  # noqa
import param
import numpy as np
import pandas as pd

import panel as pn
pn.extension()

START = datetime.date(2020, 1, 1)
END = datetime.date(2020, 12, 31)
TS = pd.date_range(start=START, end=END)
DATA = pd.DataFrame(data=np.random.rand(len(TS), 2), columns=['A', 'B'], index=TS)

def db_fetch(query):
    return DATA.loc[query["start"] : query["end"]]

class Global(param.Parameterized):
    start = param.CalendarDate(datetime.date(2020, 1, 2), bounds=(START, END))
    end = param.CalendarDate(datetime.date(2020, 6, 1), bounds=(START, END))
    data = param.DataFrame()

    @param.depends('start', 'end', watch=True, on_init=True)
    def _fetch_data(self):
        print('fetch data')
        query = {'start': self.start, 'end': self.end}
        self.data = db_fetch(query)

class ViewA(pn.viewable.Viewer):
    glob = param.Parameter()
    a = param.Range(default=(0.3, 0.6), bounds=(0, 1))

    def __init__(self, **params):
        super().__init__(**params)
        self._plot_pane = pn.pane.HoloViews()
        self._layout = pn.Column(self.param.a, self._plot_pane)
        self._update_plot()

    @param.depends('glob.data', 'a', watch=True)
    def _update_plot(self):
        amin, amax = self.a
        df = self.glob.data.query('@amin < A < @amax')
        self._plot_pane.object = df.hvplot.scatter(y='A', height=100)

    def __panel__(self):
        return self._layout

class ViewB(pn.viewable.Viewer):
    glob = param.Parameter()
    b = param.Range(default=(0.1, 0.9), bounds=(0, 1))

    def __init__(self, **params):
        super().__init__(**params)
        self._plot_pane = pn.pane.HoloViews()
        self._layout = pn.Column(self.param.b, self._plot_pane)
        self._update_plot()

    @param.depends('glob.data', 'b', watch=True)
    def _update_plot(self):
        bmin, bmax = self.b
        df = self.glob.data.query('@bmin < B < @bmax')
        self._plot_pane.object = df.hvplot.scatter(y='B', height=100)

    def __panel__(self):
        return self._layout

glob = Global()

va = ViewA(glob=glob)
vb = ViewB(glob=glob)

pn.Row(pn.Column(glob.param.start, glob.param.end), pn.Column(va, vb)).servable()

As a side note the Lumen project aims to simplify this sort of app, by declaring in a YAML file the data sources of your app, its filters (global or per view) and its views. We haven’t really found yet the time to document and release it properly!

3 Likes

Wow thanks this is great stuff! This is exactly what I was looking for in this post, so I’ll add a link for posterity :slight_smile:

2 Likes