Overwrite Altair/Vega selection parameters with custom function

Hi all,

first of all I would like to thank all people working on Panel. The library is amazing.

I have one problem though which I could not resolve after several hours of debugging and reading the docs. I have a layered Altair/Vega chart, consisting of five subfigures, in my code, where a hover over a subfigures updates the selection for the corresponding values of the subfigures. This selection is then used to update a LaTeX pane, where the currently selected values are displayed and a simple additive computation is performed and displayed (markdown_view()). This part actually works like a charm.

My problem is that I need to have a custom function which can manually set the selection so that the LaTeX display and hover on each subfigure is automatically updated. I try to do this in the update_pane() function. Here I simply generate new charts for the five subfigures and assign them as new objects.
When setting a new chart I get the an error (ValueError: List parameter 'Selection.encoding_bitrate_avg' must be a list, not an object of <class 'NoneType'>.), which is why I catch the error here. I know that it is not a good practice, but I do not know how to avoid this error. I also do not fully understand why the error appears, because I provide an initial value. Even though I catch this error, the figures are correctly updated. When I update the markdown manually it also works. The actual issue appears when I execute this custom function and then hover again over the subfigures. As the selection params are never properly overwritten, the LaTeX pane resets to the values active before executing the custom function. The figures however remain the same.

I tried to keep a global param, which is either updated by the selection param or by my custom function, but I couldn’t manage to make it work. I also did not find anything which helps me to clear the previous selection.

Can somebody please help me with this problem? What is the Panel way to solve this problem?

Thanks!

#!/usr/bin/env python3
# -*- coding: utf-8 -*-

import pathlib
import numpy as np
import pandas as pd
import pickle
import altair as alt
import panel as pn

class Visualization():
    
    def __init__(self, path: pathlib.Path):

        self._feature_labels = {
            "A": "A",
            "B": "B",
            "C": "C",
            "D": "D",
            "E": "E",
        }
        self._feature_limits = {
            "B": [0, 10],
            "A": [0, 10],
            "C": [0, 8],
            "D": [0, 40],
            "E":[0, 20],
        }
        
        self._feature_order = ['A', 
                              'B',
                              'C',
                              'D', 
                              'E']
        
        self._features_discrete = ["C", "E"]
        
        self._model = pickle.load(open(path / "model.pkl", "rb"))
        self._data = pd.DataFrame()
        step_size = 0.1
        for feature, value_range in self._feature_limits.items():
            start, end = value_range
            if feature in self._features_discrete:
                x = np.arange(start, end+1, 1)
            else:
                x = np.arange(start, end+step_size, step_size)
            y = self._model.predict(x, feature)
            self._data = pd.concat([self._data, pd.DataFrame({"x": x, "y": y, "feature": feature})], axis=0)
        
    def get_model(self):
        return self._model

    def get_data(self):
        return self._data

    def _get_chart(self, data: pd.DataFrame, feature: str, initial_value: float = None):
        """Compute the graph of the plot."""
        data.loc[:, "mark_x_start"] = 0
        data.loc[:, "mark_y_start"] = -5

        lines = (
            alt.Chart(data)
            .mark_line(clip=True)
            .encode(
                x="x:Q",
                y="y:Q",
                color=alt.value('darkblue'),
                strokeWidth=alt.value(2),
            )
        )
        
        # Draw points on the line, and highlight based on selection
        selection = alt.selection_point(
            value=[{'x': initial_value if initial_value!=None else 0.0}],
            fields=["x"],
            nearest=True,
            on="mouseover",
            empty=False,
            name=feature,
        )

        points = lines.transform_filter(selection).mark_circle(size=100, clip=True).encode(color=alt.value('red'))
        hline = lines.transform_filter(selection).mark_rule(clip=True).encode(
            x="mark_x_start:Q",
            x2="x:Q",
            y="y:Q",
            color=alt.value('red'),
            opacity=alt.condition(selection, alt.value(0), alt.value(0)),
            strokeWidth=alt.value(3),
        )
        
        # Draw a rule at the location of the selection
        tooltips = (
            alt.Chart(data)
            .mark_rule(clip=True)
            .encode(
                x="x:Q",
                y="mark_y_start:Q",
                y2="y:Q",
                color=alt.value('red'),
                opacity=alt.condition(selection, alt.value(1), alt.value(0)),
                strokeWidth=alt.value(3),
                tooltip=[
                    alt.Tooltip("x", title=self._feature_labels[feature], format=".2f" if not feature in self._features_discrete else "d"),
                    alt.Tooltip("y", title="Effect [MOS]", format=".2f"),
                ],
            )
            .add_params(selection)
        )
        
        return (lines + hline + points + tooltips)
    
    def _load(self, initial_values: dict = None):
        """Compute all plots and return them once as large plots and once as small ones."""
        charts = []
        for i, feature in enumerate(self._feature_order, start=0):
            # Get the graph data
            df_graph = self._data[self._data.feature==feature]
            
            initial_value = initial_values[feature] if initial_values!=None else None
            base = self._get_chart(df_graph, feature, initial_value=initial_value)
            base = base.encode(alt.X("x",
                                     scale=alt.Scale(nice=False, domain=self._feature_limits[feature]),
                                     title=self._feature_labels[feature]),
                               alt.Y("y", scale=alt.Scale(domain=[-2.5, 2.5]),
                                     title=''))
           
            charts.append(base.properties(width=200, height=150))
            
        return charts
    
    def markdown_view(self, a, b, c, d, e):
        result = np.clip(np.round(3.42 + a + b + c + d + e, 2), 1, 5)
        output = "$ 3.42"
        output += f" + {a:.2f}" if np.sign(a)>=0 else f" - {np.abs(a):.2f}"
        output += f" + {b:.2f}" if np.sign(b)>=0 else f" - {np.abs(b):.2f}"
        output += f" + {c:.2f}" if np.sign(c)>=0 else f" - {np.abs(c):.2f}"
        output += f" + {d:.2f}" if np.sign(d)>=0 else f" - {np.abs(d):.2f}"
        output += f" + {e:.2f}" if np.sign(e)>=0 else f" - {np.abs(e):.2f}"
        output += f" = {result:.2f} $"
        return {
            'object': output,
        }
    
    def value_lookup(self, x, feature):
        return self._data.loc[(self._data.feature==feature) & (np.isclose(self._data.loc[:,"x"], x)), "y"].item()

    def update_panes(self, initial_values: dict = None):
        charts = self._load(initial_values)
        
        try:
            self.a_pane.object = charts[0]
        except ValueError:
            pass

        try:
            self.b_pane.object = charts[1]
        except ValueError:
            pass
        
        try:
            self.c_pane.object = charts[2]
        except ValueError:
            pass
        
        try:
            self.d_pane.object = charts[3]
        except ValueError:
            pass
        
        try:
            self.e_pane.object = charts[4]
        except ValueError:
            pass
        
        a = self.value_lookup(initial_values["A"], "A")
        b = self.value_lookup(initial_values["B"], "B")
        c = self.value_lookup(initial_values["C"], "C")
        d = self.value_lookup(initial_values["D"], "D")
        e = self.value_lookup(initial_values["E"], "E")
        self.latex_pane.object = self.markdown_view(a, b, c, d, e)["object"]
        
    def update_selection(self, selection, feature):
        if not selection:
            return 0
        if isinstance(selection, list):
            x = np.round(selection[0]["x"],1)
        else:
            x = selection
        return self.value_lookup(x, feature)

    def get_pane(self):
        charts = self._load()
        self.a_pane = pn.pane.Vega(charts[0], debounce=10)
        self.b_pane = pn.pane.Vega(charts[1], debounce=10)
        self.c_pane = pn.pane.Vega(charts[2], debounce=10)
        self.d_pane = pn.pane.Vega(charts[3], debounce=10)
        self.e_pane = pn.pane.Vega(charts[4], debounce=10)
        
        a = pn.bind(lambda x: self.update_selection(x, "A"), self.a_pane.selection.param["A"])
        b = pn.bind(lambda x: self.update_selection(x, "B"), self.b_pane.selection.param["B"])
        c = pn.bind(lambda x: self.update_selection(x, "C"), self.c_pane.selection.param["C"])
        d = pn.bind(lambda x: self.update_selection(x, "D"), self.d_pane.selection.param["D"])
        e = pn.bind(lambda x: self.update_selection(x, "E"), self.e_pane.selection.param["E"])
        irefs = pn.bind(self.markdown_view, a, b, c, d, e)
        self.latex_pane = pn.pane.LaTeX(refs=irefs)
        
        return pn.Column(
            pn.Row(pn.Spacer(margin=(50,0,0,0))),
            pn.pane.LaTeX("$ bias + f_{a} + f_{b} + f_{c} + f_{d} + f_{e} =  X $"), 
            self.latex_pane,
            pn.Row(pn.Spacer(margin=(50,0,0,50))),
            pn.Row(self.a_pane, self.b_pane, self.c_pane, styles={"margin": "auto"}),
            pn.Row(self.d_pane, self.e_pane, styles={"margin": "auto"}),
            sizing_mode="stretch_width",
            width_policy="max"
        )

Hi @QMon

Welcome to the community and thanks for the question.

Could you make the example reproducible? Right now I cannot run the example because 1) I need to provide a path and 2) Because I do not know how to serve it (Visualization(path=...).get_pane())?

Hi Marc,

thanks for answering. I attached a self-contained piece of code. The problem happens when you hover first across the figures, setting some values in the LaTeX pane, and then clicking on the button in the top left. The button sets manually new values in the LaTeX pane. The problem is that as soon as I hover again across one of the figures the original set (via hover, the param selection) values are immediately shown in the LaTeX pane. I would like to overwrite them so that the LaTeX pane is not reset to the state before the click.

#!/usr/bin/env python3
# -*- coding: utf-8 -*-

import numpy as np
import pandas as pd
import altair as alt
import panel as pn

class Visualization():
    
    def __init__(self):

        self._feature_labels = {
            "A": "A",
            "B": "B",
            "C": "C",
            "D": "D",
            "E": "E",
        }
        self._feature_limits = {
            "B": [0, 10],
            "A": [0, 10],
            "C": [0, 8],
            "D": [0, 40],
            "E":[0, 20],
        }
        
        self._feature_order = ['A', 
                              'B',
                              'C',
                              'D', 
                              'E']
        
        self._features_discrete = ["C", "E"]
        
        self._data = pd.DataFrame()
        step_size = 0.1
        for feature, value_range in self._feature_limits.items():
            start, end = value_range
            if feature in self._features_discrete:
                x = np.arange(start, end+1, 1)
            else:
                x = np.arange(start, end+step_size, step_size)
            y = np.random.uniform(low=-2, high=2, size=len(x))
            self._data = pd.concat([self._data, pd.DataFrame({"x": x, "y": y, "feature": feature})], axis=0)
        
    def get_model(self):
        return self._model

    def get_data(self):
        return self._data

    def get_density(self):
        return self._density
    
    def _get_chart(self, data: pd.DataFrame, feature: str, initial_value: float = None):
        """Compute the graph of the plot."""
        data.loc[:, "mark_x_start"] = 0
        data.loc[:, "mark_y_start"] = -5

        lines = (
            alt.Chart(data)
            .mark_line(clip=True)
            .encode(
                x="x:Q",
                y="y:Q",
                color=alt.value('darkblue'),
                strokeWidth=alt.value(2),
            )
        )
        
        # Draw points on the line, and highlight based on selection
        selection = alt.selection_point(
            value=[{'x': initial_value if initial_value!=None else 0.0}],
            fields=["x"],
            nearest=True,
            on="mouseover",
            empty=False,
            name=feature,
        )

        points = lines.transform_filter(selection).mark_circle(size=100, clip=True).encode(color=alt.value('red'))
        hline = lines.transform_filter(selection).mark_rule(clip=True).encode(
            x="mark_x_start:Q",
            x2="x:Q",
            y="y:Q",
            color=alt.value('red'),
            opacity=alt.condition(selection, alt.value(0), alt.value(0)),
            strokeWidth=alt.value(3),
        )
        
        # Draw a rule at the location of the selection
        tooltips = (
            alt.Chart(data)
            .mark_rule(clip=True)
            .encode(
                x="x:Q",
                y="mark_y_start:Q",
                y2="y:Q",
                color=alt.value('red'),
                opacity=alt.condition(selection, alt.value(1), alt.value(0)),
                strokeWidth=alt.value(3),
                tooltip=[
                    alt.Tooltip("x", title=self._feature_labels[feature], format=".2f" if not feature in self._features_discrete else "d"),
                    alt.Tooltip("y", title="Effect", format=".2f"),
                ],
            )
            .add_params(selection)
        )
        
        return (lines + hline + points + tooltips)
    
    def _load(self, initial_values: dict = None):
        """Compute all plots and return them once as large plots and once as small ones."""
        charts = []
        for i, feature in enumerate(self._feature_order, start=0):
            # Get the graph data
            df_graph = self._data[self._data.feature==feature]
            # Get the density data
            
            initial_value = initial_values[feature] if initial_values!=None else None
            base = self._get_chart(df_graph, feature, initial_value=initial_value)
            base = base.encode(alt.X("x",
                                     scale=alt.Scale(nice=False, domain=self._feature_limits[feature]),
                                     title=self._feature_labels[feature]),
                               alt.Y("y", scale=alt.Scale(domain=[-2.5, 2.5]),
                                     title=''))
            
            charts.append(base.properties(width=200, height=150))
            
        return charts
    
    def markdown_view(self, a, b, c, d, e):
        result = np.round(3.42 + a + b + c + d + e, 2)
        output = "$ 3.42"
        output += f" + {a:.2f}" if np.sign(a)>=0 else f" - {np.abs(a):.2f}"
        output += f" + {b:.2f}" if np.sign(b)>=0 else f" - {np.abs(b):.2f}"
        output += f" + {c:.2f}" if np.sign(c)>=0 else f" - {np.abs(c):.2f}"
        output += f" + {d:.2f}" if np.sign(d)>=0 else f" - {np.abs(d):.2f}"
        output += f" + {e:.2f}" if np.sign(e)>=0 else f" - {np.abs(e):.2f}"
        output += f" = {result:.2f} $"
        return {
            'object': output,
        }
    
    def value_lookup(self, x, feature):
        return self._data.loc[(self._data.feature==feature) & (np.isclose(self._data.loc[:,"x"], x)), "y"].item()

    def update_panes(self, initial_values: dict = None):
        charts = self._load(initial_values)
        
        try:
            self.a_pane.object = charts[0]
        except ValueError:
            pass

        try:
            self.b_pane.object = charts[1]
        except ValueError:
            pass
        
        try:
            self.c_pane.object = charts[2]
        except ValueError:
            pass
        
        try:
            self.d_pane.object = charts[3]
        except ValueError:
            pass
        
        try:
            self.e_pane.object = charts[4]
        except ValueError:
            pass
        
        a = self.value_lookup(initial_values["A"], "A")
        b = self.value_lookup(initial_values["B"], "B")
        c = self.value_lookup(initial_values["C"], "C")
        d = self.value_lookup(initial_values["D"], "D")
        e = self.value_lookup(initial_values["E"], "E")
        self.latex_pane.object = self.markdown_view(a, b, c, d, e)["object"]
        
    def update_selection(self, selection, feature):
        if not selection:
            return 0
        if isinstance(selection, list):
            x = np.round(selection[0]["x"],1)
        else:
            x = selection
        return self.value_lookup(x, feature)

    def get_pane(self):
        charts = self._load()
        self.a_pane = pn.pane.Vega(charts[0], debounce=10)
        self.b_pane = pn.pane.Vega(charts[1], debounce=10)
        self.c_pane = pn.pane.Vega(charts[2], debounce=10)
        self.d_pane = pn.pane.Vega(charts[3], debounce=10)
        self.e_pane = pn.pane.Vega(charts[4], debounce=10)
        
        a = pn.bind(lambda x: self.update_selection(x, "A"), self.a_pane.selection.param["A"])
        b = pn.bind(lambda x: self.update_selection(x, "B"), self.b_pane.selection.param["B"])
        c = pn.bind(lambda x: self.update_selection(x, "C"), self.c_pane.selection.param["C"])
        d = pn.bind(lambda x: self.update_selection(x, "D"), self.d_pane.selection.param["D"])
        e = pn.bind(lambda x: self.update_selection(x, "E"), self.e_pane.selection.param["E"])
        irefs = pn.bind(self.markdown_view, a, b, c, d, e)
        self.latex_pane = pn.pane.LaTeX(refs=irefs)
        
        return pn.Column(
            pn.Row(pn.Spacer(margin=(50,0,0,0))),
            pn.pane.LaTeX("$ bias + f_{a} + f_{b} + f_{c} + f_{d} + f_{e} =  X $"), 
            self.latex_pane,
            pn.Row(pn.Spacer(margin=(50,0,0,50))),
            pn.Row(self.a_pane, self.b_pane, self.c_pane, styles={"margin": "auto"}),
            pn.Row(self.d_pane, self.e_pane, styles={"margin": "auto"}),
            sizing_mode="stretch_width",
            width_policy="max"
        )
    

pn.extension("vega")
pn.extension("katex")
pn.extension("tabulator")

# Load data
vis = Visualization()

tabs = pn.Tabs(
    ("Model", vis.get_pane()),
    styles={"font-size": "16px"}
)

button = pn.widgets.Button(name="Manually select values")
def update_indicator(event):
    if not event:
        return
    initial_values = {
        "A": 4.5,
        "B": 2.5,
        "C": 3.0,
        "D": 4.5,
        "E": 1.0
    }
    vis.update_panes(initial_values)

pn.bind(update_indicator, button, watch=True)
sidebar = pn.Column(button)

template = pn.template.VanillaTemplate(
    title="Dashboard",
    sidebar=sidebar,
    main=tabs,
    sidebar_width=420,
)

template.servable();

Getting this too. Can’t seem to figure out why a vega pane doesn’t like to be receive a new object. Same error message triggers

Hi,

I solved it now. I guess there may be a bug in the _update_selections method of panel/pane/vega.py.

def _update_selections(self, *args):
        params = {
            e: param.Dict(allow_refs=False) if stype == 'interval' else param.List(allow_refs=False)
            for e, stype in self._selections.items()
        }
        if self.selection and (set(self.selection.param) - {'name'}) == set(params):
            self.selection.param.update({p: None for p in params})
            return
        self.selection = type('Selection', (param.Parameterized,), params)()

As soo as I update the object of my pn.pane.Vega(), the line self.selection.param.update({p: None for p in params}) is executed. However, setting None always raises the error discussed above (at least for point selections). I also do not get why one should set the selection to None there. Probably to reset the selection?
Anyway, if I remove the line containing the param update, I can update the pn.pane.Vega object without any error and then update the selection with a custom value using for example self.a_pane.selection.param.update({“A”: [{“x”: 1}]}).

Best regards