Replacing options dictionary doesn't replace keys/choices as expcected

I’ve set up a series of inter-dependent parameters, based on my data. (In-situ water quality measurements in local streams.) The data is parameterized by

  • measurement type,
  • a sub-set of locations where that was measured for cross-site comparison,
  • a single location from out of that subset for time-series plotting,
  • and a single storm event from the list of those available at the single location to view in a detailed plot.

Roughly, the workflow goes like this: first the user picks a measurement type (“conductivity”, “turbidity”, etc). A @depends linked function then slices my summary dataframe to sites where that parameter was measured and changes the “options” for the second parameter. The data from the sliced dataframe is used to populate a map view of the locations. A multi-select widget is also shown for the second parameter. The user then selects a subset of those locations either with said widget or using the plot selection tools on the map - either of which would updates the values of the second parameter (via a 1-D stream in the case of the map). Another @depends/watched function updates the options for the 3rd single-site parameter based on the sites selected via the map or the multi-select. All of those steps work as I expect. In each case the “options” are simple lists and I use a create a new temporary list as suggested in several posts and then repopulate the new options from it. Took me a while to figure out, but it’s working now.

But… in the next step of the workflow, the user gets to pick one single event from the one single location they’ve picked for a zoomed in plot. But my event data is in a separate dataframe keyed by event number and I want to show the user the starting event date rather than the event number in a discrete slider widget. So I create a dictionary as the options for the single storm event parameter (key is the date as a text string, value is the event number). When a new site is selected, the options dictionary is replaced. BUT… it doesn’t work properly, when I pick a date, then parameter value is set to that text string instead of to the event number the string is a key for. If I pick a static dictionary and do not ever change it, it works as expected, selecting the date key updates the parameter value to the event number. But when the options dictionary is changed, it no longer looks up the key in the option and returns the key itself instead of the value of the key. I assume this is not intended behavior. Am I coding wrong (likey) or is it really a bug I should report?

My code’s not very neat, but here are the relevant chunks:

class ParameterizedStorm(param.Parameterized):

    measured_parameter = param.ObjectSelector(
        objects=measured_parameter_options,
        default="turbidity",
        label="Measured parameter",
        doc="The parameter measured in-situ in the stream.",
    )
    locations_selected_on_map = param.ListSelector(
        label="Locations for cross-site comparison",
        doc="The locations to view and compare.",
    )
    single_location = param.ObjectSelector(
        label="Location for time-series plot",
        doc="A single location to plot time-series data from.",
    )
    # placeholder
    no_storm_selected_dict = {
        "Select a location!": 0,
        "Select any location!": 1,
    }
    single_storm_number = param.ObjectSelector(
        label="Storm for time-series plot",
        doc="A single storm to plot time-series data from.",
        objects=no_storm_selected_dict,
        default=0,
    )


    def __init__(self):
        super().__init__()
        # other stuff not related to the date parameter happens

    def get_discrete_storm_opts(
        self, the_param: str, the_location: str
    ) -> Tuple[Dict, int]:
        pk_data = self.by_storm_summary.loc[
            (self.by_storm_summary["measured_param"] == the_param)
            & (self.by_storm_summary["Location"] == the_location)
        ].reset_index()
        df2 = (
            pk_data[["date_text", "storm_number"]]
            .drop_duplicates()
            .set_index("date_text")
        )
        date_opts_dict = df2.to_dict()["storm_number"]
        if len(df2.index) > 0:
            init_selected_date = df2.index[0]
            return date_opts_dict, init_selected_date
        else:
            error_dict = {
                "No storms recorded for this param and location": 0,
                "Pick another location": 1,
            }
            return (
                error_dict,
                next(iter(error_dict)),
            )

    # This updates the date list available on the date slider based on the selected location
    @param.depends("single_location", watch=True)
    def _update_dates(self):
        print(
            strftime("%H:%M:%S  ", gmtime())
            + "_update_dates: Updating the possible storm dates based on {} at {}".format(
                self.measured_parameter, self.single_location
            )
        )
        one_loc = self.single_location
        cur_selected_storm_no = self.single_storm_number

        # If there is a single location selected, we get the possible storms
        # based on that location and the current parameter value
        if one_loc is not None:
            # given the current location and parameter, find the possible dates
            new_date_opts, new_selected_date = self.get_discrete_storm_opts(
                self.measured_parameter, self.single_location
            )
            # set the options to the full list of dates
            self.param["single_storm_number"].objects = new_date_opts
            new_selected_storm_no = new_selected_date
        # if there is not a single location selected
        else:
            # put in dummy values asking for a location to be selected
            self.param["single_storm_number"].objects = no_storm_selected_dict
            new_selected_storm_no = next(iter(no_storm_selected_dict))

        # if the value isn't changed (ie, 0 to 0) no trigger will happen for the
        # single_storm_number event, so we manually trigger it
        self.single_storm_number = new_selected_storm_no
        if new_selected_storm_no == cur_selected_storm_no:
            print(
                strftime("%H:%M:%S  ", gmtime())
                + "Manually triggering a single_storm_number event"
            )
            self.param.trigger("single_storm_number")
        print(strftime("%H:%M:%S  ", gmtime()) + "finished _update_dates")
1 Like

Hi @SRGDamia1

Welcome to the community.

It would be really helpful if you could provide a Minimum Reproducible Example. I started looking at the code and since it is long and not runnable it will simply take me too long time to understand and try to provide a solution to.

And if you could add one or more screenshots or a video of the issue then again it is much easier to provide help.

Thanks.

I’m sorry.

Here’s something that runs:

#%%
from typing import Tuple, Dict
from time import gmtime, strftime

import pandas as pd
import panel as pn
import param

pn.extension()

#%%
class ParameterizedStorm(param.Parameterized):
    by_storm_summary = pd.DataFrame(
        {
            "Location": [
                "site1",
                "site2",
                "site1",
                "site2",
                "site1",
                "site2",
                "site1",
                "site2",
                "site1",
                "site2",
                "site1",
                "site2",
                "site3",
                "site3",
            ],
            "storm_number": [1, 1, 2, 2, 3, 3, 4, 4, 3, 3, 4, 4, 1, 2],
            "date_text": [
                "01Jan16",
                "07Jan16",
                "01Feb16",
                "04Feb16",
                "24Feb16",
                "17Feb16",
                "26Feb16",
                "02Mar16",
                "24Feb16",
                "17Feb16",
                "26Feb16",
                "02Mar16",
                "26Feb16",
                "02Mar16",
            ],
        }
    )
    single_location = param.ObjectSelector(
        label="Location for time-series plot",
        doc="A single location to plot time-series data from.",
        objects=by_storm_summary["Location"].unique().tolist(),
        default="site1",
    )
    # placeholder
    no_storm_selected_dict = {
        "Select a location!": 0,
        "Select any location!": 1,
    }
    single_storm_number = param.ObjectSelector(
        label="Storm for time-series plot",
        doc="A single storm to plot time-series data from.",
        objects=no_storm_selected_dict,
        default=0,
    )

    def __init__(self):
        super().__init__()
        # other stuff not related to the date parameter happens

    def get_discrete_storm_opts(self, the_location: str) -> Tuple[Dict, int]:
        pk_data = self.by_storm_summary.loc[
            (self.by_storm_summary["Location"] == the_location)
        ].reset_index()
        df2 = (
            pk_data[["date_text", "storm_number"]]
            .drop_duplicates()
            .set_index("date_text")
        )
        date_opts_dict = df2.to_dict()["storm_number"]
        return date_opts_dict

    # This updates the date list available on the date slider based on the selected location
    @param.depends("single_location", watch=True)
    def _update_dates(self):
        print(
            strftime("%H:%M:%S  ", gmtime())
            + "_update_dates: Updating the possible storm dates at {}".format(
                self.single_location
            )
        )
        new_date_opts = self.get_discrete_storm_opts(self.single_location)
        # set the options to the full list of dates
        self.param["single_storm_number"].objects = new_date_opts

    @param.depends(
        "single_location", "single_storm_number",
    )
    def view_states(self):
        text = (
            "Single Location {}".format(self.single_location)
            + "\nPossible single storm options: {}".format(
                self.param["single_storm_number"].objects
            )
            + "\nSingle Storm: {}".format(self.single_storm_number)
        )
        text_pane: pn.pane.Str = pn.pane.Str(object=text)
        print(strftime("%H:%M:%S  ", gmtime()) + "view_states ran")
        return pn.Row(text_pane)


# %%
if __name__ == "__main__":
    the_storm_parmer = ParameterizedStorm()
    widget_pane = pn.Param(
        the_storm_parmer.param,
        widgets={
            "single_location": pn.widgets.Select,
            "single_storm_number": pn.widgets.DiscreteSlider,
        },
    )

    p = pn.Row(widget_pane, the_storm_parmer.view_states)
    p.show()

Basically, when I initially supply the object selector with a text:int dictionary as objects, the parameter’s widget displays the text but the parameter’s value is set to the integer as expected. But, after the parameter’s object dictionary is changed, the widget still displays the text, but when the widget is used, the parameter’s value is set to the key in the dictionary instead of the value referred to by that key.

Hi @SRGDamia1

I am currently looking at it. I believe the key is really understanding.

  • What would you expect to be the key and the value of your single_storm objects? The date or the integer?
  • And how the Param.ObjectSelector works.
    • What should be the keys and values of the objects.
    • What is the value of a Param.ObjectSelector? Is it the key or the value.

I expect the key to be a string date, I expect the value to be the integer. When I initially create the parameter and supply an object dictionary keyed (keys=string date, values=integers) that is the result. The same happens when I supply a different parameter with a dictionary with keys=long helpful text, values=short code. The value of the parameter is set as the short code when I pick the long helpful text from a drop down selector. When I use a function to change the options dictionary, the same no longer applies.

Hi @SRGDamia1

We hit this issue https://github.com/holoviz/param/issues/398.

The param.ObjectSelector simply works in a way that I simply have a hard time of grasping. I believe that is your problem as well. Please put a comment in that issue if you think so too. That will help change things.

I will continue working on a solution though. Will be a bit of a workaround.

Yup, definitely my issue.

I believe this should work

I had to refactor your code a bit in order to get an understanding of it. I cannot just read code unfortunately :slight_smile:

from typing import Tuple, Dict
from time import gmtime, strftime

import pandas as pd
import panel as pn
import param

pn.extension()

BY_STORM_SUMMARY = pd.DataFrame(
    {
        "Location": [
            "site1",
            "site2",
            "site1",
            "site2",
            "site1",
            "site2",
            "site1",
            "site2",
            "site1",
            "site2",
            "site1",
            "site2",
            "site3",
            "site3",
        ],
        "storm_number": [1, 1, 2, 2, 3, 3, 4, 4, 3, 3, 4, 4, 1, 2],
        "date_text": [
            "01Jan16",
            "07Jan16",
            "01Feb16",
            "04Feb16",
            "24Feb16",
            "17Feb16",
            "26Feb16",
            "02Mar16",
            "24Feb16",
            "17Feb16",
            "26Feb16",
            "02Mar16",
            "26Feb16",
            "02Mar16",
        ],
    }
)

NO_STORM_SELECTED_DICT = {
    "Select a location!": 0,
    "Select any location!": 1,
}


def get_discrete_storm_opts(by_storm_summary: pd.DataFrame, the_location: str) -> Tuple[Dict, int]:
    pk_data = by_storm_summary.loc[(by_storm_summary["Location"] == the_location)].reset_index()
    df2 = pk_data[["date_text", "storm_number"]].drop_duplicates().set_index("date_text")
    date_opts_dict = df2.to_dict()["storm_number"]
    return date_opts_dict


class ParameterizedStorm(param.Parameterized):
    by_storm_summary = param.DataFrame()
    single_location = param.ObjectSelector(
        label="Location for time-series plot",
        doc="A single location to plot time-series data from.",
    )
    single_storm_number = param.ObjectSelector(
        label="Storm for time-series plot",
        doc="A single storm to plot time-series data from.",
        objects={},
    )
    view = param.Parameter()

    def __init__(self, by_storm_summary: pd.DataFrame):
        super().__init__(by_storm_summary=by_storm_summary)

        self._handle_by_storm_summary_change()
        self._handle_single_location_change()

        self._widget_pane = pn.Param(
            self,
            parameters=["single_location", "single_storm_number"],
            widgets={
                "single_location": pn.widgets.Select,
                "single_storm_number": pn.widgets.DiscreteSlider,
            },
            show_name=False,
            width=400,
        )
        self._text_pane = pn.pane.Str()
        self.view = pn.Row(
            pn.WidgetBox(self._widget_pane),
            self._text_pane,
            )



    @param.depends("by_storm_summary", watch=True)
    def _handle_by_storm_summary_change(self):
        unique_locations = self.by_storm_summary["Location"].unique().tolist()
        print("by_storm_summary changed", unique_locations)
        self.param.single_location.objects = unique_locations
        if unique_locations:
            default_location = unique_locations[0]
            self.param.single_location.default = default_location
            if self.single_location not in unique_locations:
                self.single_location = default_location

    @param.depends("single_location", watch=True)
    def _handle_single_location_change(self):
        new_date_opts = get_discrete_storm_opts(
            by_storm_summary=self.by_storm_summary, the_location=self.single_location
        )

        # See https://github.com/holoviz/param/issues/398
        self.param["single_storm_number"].names = new_date_opts
        self.param["single_storm_number"].objects = list(new_date_opts.values())

        if new_date_opts:
            values = list(new_date_opts.values())
            default_single_storm_number = values[0]
            self.param["single_storm_number"].default = default_single_storm_number
            if not self.single_storm_number in values:
                self.single_storm_number = default_single_storm_number

    @param.depends(
        "single_location",
        "single_storm_number",
        watch=True
    )
    def view_states(self):
        text = (
            "Single Location {}".format(self.single_location)
            + "\nPossible single storm options: {}".format(
                self.param["single_storm_number"].objects
            )
            + "\nSingle Storm: {}".format(self.single_storm_number)
        )
        self._text_pane.object = text


if __name__ == "__main__":
    the_storm_parmer = ParameterizedStorm(by_storm_summary=BY_STORM_SUMMARY)
    the_storm_parmer.view.show(port=5006)

I think this is the correct fix. Param doesn’t use the names internally, though it was later extended to extract and save them for Panel if you do supply them using a dictionary.

We should definitely make this more obvious. I think we could write a _set_objects method, then make self.objects be a property, so that both the constructor and setting objects would both extract the names and store it in this way. Would that make the behavior match what you expect?

It might be tough to maintain backwards compatibility with the above code, though!

Thank you.

Yes, that is exactly the behavior I expected.

Ok, I opened a PR: https://github.com/holoviz/param/pull/440

1 Like

And I opened a different PR, which addresses more of the underlying issues but does have backwards compatibility implications (for anyone who may have been iterating over the objects list): https://github.com/holoviz/param/pull/441