New user questions around custom data table implementation

CmpCtrl · April 21, 2022, 7:26pm

First up some perspective from a new user… I dont mean for this to be overly critical, i appreciate the complexity of the challenge, i just mean to remind the old pro’s of the perspective of a new user.

I’m looking for a way to do a browser based user interface for a python app with interactive plots, tables, etc. I started off with Dash, but abandoned it since the stateless-ness became such a hassle for something i intend to host locally for a single user. Panel seems like a great alternative with much of the same functionality but a much more intuitive callback/functional structure.

Now, i’ve spent a few days trying to move to Panel and i find it extremely confusing. There’s so much magic in this library that i’m struggling to figure out how to do anything even slightly different from the examples. I’m sure i’m not the target use case, but for me it missed the balance between making the easy case easy and making the slightly different case understandable.

As for some specific comments.
Holy cow is it hard to search google for help on ‘panel’. google has no idea that i mean the proper name Panel. Couldnt it have been panelx or something unique? i can use ‘holoviz panel’ but plenty of help info out there doesnt reference holoviz. (same goes for dash tho google seems to associate it with plotly better than panel with holoviz).

Getting the layout to look the way i want is much more difficult than with Dash, i actually like its style for the layout. Again that’s my preference for control vs magic. It sounds like some improvements are coming in version 1.

Early on i struggled to figure out how to initialize empty components( and im still a bit unclear on the differences between panes and panels and params and widgets). My ui starts nearly empty and populates as the user makes selections and open files, etc. I eventually found the following example which helped a lot, it might be worth some mention in the user guide since it is quite simple.
https://panel.holoviz.org/gallery/dynamic/dynamic_plot_layout.html

However, i still don’t really want placeholders (i usually make the component invisible to start), so i get the following warning often. Is there a more elegant way to avoid the warning without using a placeholder that will be hidden?
WARNING:bokeh.core.validation.check:W-1002 (EMPTY_LAYOUT): Layout has no children: Row(id='1013', ...)

Param (the proper name) is also very confusing, it feels like its fighting python. How can i define a param (proper name) as a class attribute, then pass the class a kwarg with the same key at instantiation, and then have it behave as an instance variable? I agree those behaviors are handy, but they’re very confusing since that’s not how python works normally. Oh, and it is magically a widget? I’m sure that much of my misunderstanding of Panel stems from misunderstanding Param. Also, it makes the documentation quite confusing to tell whats a proper param and whats a parameter.

Now if anyone has stuck around this long, i have some real questions.

First, i want a widget for editing table data, but my table data is not in a pandas dataframe and it is often 2d. I would like to have an active cell, keyboard navigation, and for the cell values to be editable. Heres a simple example of what i’d like it to look like.

I spent some time looking into tabulator but couldnt see a good path to get what im looking for (surely there is, it seems quite powerful but i couldnt see it at first glance). So i headed down the path of a reactiveHTML component to build a simple table. Again, this seems like it can be very powerful but im stuck at not understanding the magic thats happening in the background.

Is there a way to use a param of a custom class? I have custom classes for working with my table data that contain all the info you might want in a plot, so it would be most handy to pass that to this reactiveHTML. I tried using param.ClassSelector but then i wasnt able to access the attributes in the class from the template. I guess would also need to be able to work with the data in a function to make this really useful, i suppose i could do an def __init__(self,**kwargs): but clearly thats not intended.

So, i broke the params up.

class CalData1d(pn.reactive.ReactiveHTML):
    """Class for custom table data control."""

    name = param.String(doc="Param Name.")
    axis1_name = param.String(doc="Axis1 Name.")
    axis1 = param.Array(doc="Axis1 Breakpoints.")
    data = param.Array(doc="Param Data.")

    _child_config = {
        "name": "literal",
        # "axis1_name": "literal",
        # "axis1": "literal",
        # "data": "literal",
    }

    _template = """
    <table id="cal-data" class="cal-data" style="border:1px solid;border-radius:8px;">
        <tbody>
        <tr id="cal-data-row-0">
            <th id="cal-data-0-0" class="cal-data-name" colspan="${axis1.size}">${pardata.name}</th>
        </tr>
        <tr id="cal-data-row-1">
            <th id="cal-data-1-0" class="cal-data-axis1name" colspan="${axis1.size}"">${axis1_name}</th>
        </tr>
        <tr id="cal-data-row-2">
            {% for val in axis1 %}
            <th id="cal-data-2-{{loop.index0}}" class="cal-data-axis1" >{{val}}</th>
            {% endfor %}
        </tr>
        <tr id="cal-data-row-3">
            {% for val in data %}
            <th id="cal-data-3-{{loop.index0}}" class="cal-data-data" >{{val}}</th>
            {% endfor %}
        </tr>
        </tbody>
    </table>
    """

But, now im running into ValueError's when instantiating this reactiveHTML class in a callback. for the simplified example below i am getting the following error ValueError: String parameter 'axis1_name' only takes a string value, not value of type <class 'panel.pane.markup.Markdown'>.

Its very confusing that it magics my string into a pane but only if its a child.

So if i modify this code to mark all of the params as literal (which is confusing, these aren’t literals, but i think i get what this means) then i get the following error: AttributeError: 'numpy.ndarray' object has no attribute 'on_change'.

This line in the User Guide seems like a clue, but i really need another clue or two to put it together.
“The difference between Jinja2 literal templating and the JS templating syntax is important to note. While literal values are inserted during the initial rendering step they are not dynamically linked.”

So, what does literal templating mean? I presume that simply means passing the value of the param into the template, not a magic pane.
Is that still different from marking the param as literal in the _child_config?

What does dynamically linked mean precisely? Is that linking callbacks? And if so which?

Also, im using the jinja syntax here which sounds like it is not linked, so what is looking for an on_change attribute (which sounds like a link) in the np.ndarray?

Anyway, sorry for the whinging. I’m clearly trying to do something that this wasnt intended for, and i dont understand it well enough to accomplish that. I can see it has a lot of potential and is very powerful, but as a new user all the magic makes it very hard to move from the example cases to something more specialized. I would appreciate some tips either of specifics, or of more general suggestions on a better approach, or pointers to more documentation and help.
Thanks for the help.

CmpCtrl · April 21, 2022, 9:10pm

I made a bit of progress this afternoon. There were some mistakes in the template that caused some of the errors. Updating everything to use the jinja syntax worked at least for this simplest of examples.

Is the js templating syntax only meant for a param.pane?

here’s the corrected template for reference.

    _template = """
    <table id="cal-data"  class="cal-data" style="border:1px solid;border-radius:8px;">
        <tbody>
        <tr id="cal-data-row-0">
            <th id="cal-data-0-0" class="cal-data-name" colspan="{{axis1.size}}">${name}</th>
        </tr>
        <tr id="cal-data-row-1">
            <th id="cal-data-1-0" class="cal-data-axis1name" colspan="{{axis1.size}}">${axis1_name}</th>
        </tr>
        <tr id="cal-data-row-2">
            {% for val in axis1 %}
            <th id="cal-data-2-{{loop.index0}}" class="cal-data-axis1" >{{val}}</th>
            {% endfor %}
        </tr>
            <tr id="cal-data-row-3">
            {% for val in data %}
            <th id="cal-data-3-{{loop.index0}}" class="cal-data-data" >{{val}}</th>
            {% endfor %}
        </tr>

        </tbody>
    </table>
    """

Next up i need to sort out callbacks.

Marc · April 22, 2022, 6:06am

HI @CmpCtrl

Thanks for taking the time to put together your feedback.

Panel name

I think nobody thought about search engine optimization when the name was found. Most people I know in the community would have made another choice in hindsight, but nobody seems willing to change the name now. It would probably require lots of work and a lot of confusion for (potential) users.

But we all feel the pain.

Terminology

Yes there is some terminology to get into and also too much. It’s on the roadmap for Panel 1.0 to clean up the api and improve the documentation.

For Panel I would say the most important terminology to know about is

pane: Wraps some python object to enable rendering it. For example the Matplotlib or Plotly pane. For a lot of objects you don’t have to use the specific Pane, Panel will automatically recognize your object and wrap it in the right Pane if you add the object to a layout. Alternatively you can provide your object to pn.panel and the right Pane will be returned with a high probability.
widget: Provides input values from the users. For example the IntSlider or the Select dropdown.
layouts: Helps put panes, widgets and layouts together. For example the Row or Column layout.
template: An easy to use template like the FastListTemplate can be used to get a nice looking result quickly.
pn.bind: You can bind your function to widgets such that it gets rerun when a widget value changes.

You can get a long way using with the above for building smaller dashboards and creating interactive data exploration tools.

An example is show below. I believe its close to how we would like to introduce the Panel api if we rewrote the documentation today.

import panel as pn

pn.extension(sizing_mode="stretch_width", template="fast")

layout = pn.Column().servable()

append_button = pn.widgets.Button(name="Append", button_type="primary").servable(area="sidebar")
pop_button = pn.widgets.Button(name="Pop").servable(area="sidebar")


def append(_):
    layout.append(
        f"I'm component {len(layout)}"
    )  # Here you can append many types of python objects


def pop(_):
    if len(layout) > 0:
        layout.pop(len(layout) - 1)


pn.bind(append, append_button, watch=True)
pn.bind(pop, pop_button, watch=True)

pn.state.template.param.update(title="Dynamic Layouts")

If you are a more advanced app builder who wants to build reusable, custom components and complex apps you can test the next step is to dive into more advanced parts of Panel

Custom Template: A way to make custom templates built using HTML, CSS and javascript. This might be worth investing in for organisations and larger use cases.
ReactiveHTML: A way to make new Panel widgets, panes and layouts with bidirectional communition.

You would also dive into the more advanced parts of Param, which provides the reactivity to changes in values/ parameters that Panel and the HoloViz ecosystem is built on top of. Here you would be learning about

Parameterized classes with parameters. They are a bit like DataClasses or Pydantic models but provide additional functionality for creating graphical user interfaces
param.depends/ pn.depends. An annotator which does almost the same as pn.bind.
pn.Param which can turn a Parameterized class into a list of widgets.

This is an example

import panel as pn
import param

pn.extension(sizing_mode="stretch_width", template="fast")

class DynamicLayout(pn.viewable.Viewer):
    append = param.Event()
    pop = param.Event()

    def __init__(self, **params):
        super().__init__(**params)

        self.layout = pn.Column()
        self.widgets = pn.Param(self, parameters=["append", "pop"], widgets={"append": {"button_type": "primary"}})

    def __panel__(self):
        return pn.Column(self.widgets, self.layout)

    @param.depends("append", watch=True)
    def _append(self):
        self.layout.append(
            f"I'm component {len(self.layout)}"
        )  # Here you can append many types of python objects


    @param.depends("pop", watch=True)
    def _pop(self):
        if len(self.layout) > 0:
            self.layout.pop(len(self.layout) - 1)

layout=DynamicLayout()
pn.panel(layout).servable()

pn.state.template.param.update(title="Dynamic Layouts")

`EMPTY_LAYOUT`

I don’t see this error in my code, but I believe you Feel free to post a bug report with a minimum reproducible code example.

Interactive table

If I can find the time later I will try to give it a shot.

Additional resources

Check out my site https://awesome-panel.org.

maximlt · April 22, 2022, 7:17am

Hi @CmpCtrl !

I feel that the root cause of your issues starting off with Panel come down to its documentation, which is planned to be reworked for Panel 1.0.

There’s certainly a lot of magics in Panel, and to be honest in Python in general. This is a tradeoff though, with Panel (or other dashboarding libraries) you can build a web app in a few lines of Python code. This is just an insane amount of power! But maybe what you meant by magic is just that things are confusing, and in that regard the documentation work should improve things.

Yep googling Panel doesn’t return great results unfortunately. I would say though that the best and most comprehensive content you can find on Panel is in three different sites so you don’t really need to Google anything:

Param (the proper name) is also very confusing, it feels like its fighting python. How can i define a param (proper name) as a class attribute, then pass the class a kwarg with the same key at instantiation, and then have it behave as an instance variable? I agree those behaviors are handy, but they’re very confusing since that’s not how python works normally.

Oh this is very much how Python works, see dataclasses for a reference from the standard library itself. I will say though that the first time I used the Param library I was a tiny bit surprised by that too, but I wasn’t aware at the time of dataclasses or other frameworks like pydantic. It feels quite natural now.

First, i want a widget for editing table data, but my table data is not in a pandas dataframe and it is often 2d.

Isn’t a table a 2D object? And why can’t you have it as a dataframe?

I would like to have an active cell, keyboard navigation, and for the cell values to be editable.

The Tabulator widget allows users to edit a cell, and you can register callbacks with the on_edit method to do whatever you want to do with the edited value (and its location in the table, returned by the callback event), if you want to do anything with that. Users can also navigate through the table. The only thing you won’t get (I think) is the active cell highlighting.

Panel already provides lots of components, including tons of widgets, multiple layouts, and a great deal of panes to display all sorts of stuff. This covers most of what people need, but sometimes you just need that extra functionality. I see the ReactiveHTML component as a way to unlock Panel developers in those situations, the tradeoff being that they have to be comfortable with front-end technologies (HTML, JavaScript, CSS). What I wish to see and what would unlock even more Panel developers would be a nice way to share these custom components, transferring the knowledge and output of power devs to more regular devs.

nghenzi · April 22, 2022, 8:43am

Welcome !!!

I have a PoC of an editable table to improve. I use editablecontent=true in the td elements and a callback in an oninput event with ${script(‘some_script’)} given by the reactive HTML scripts properties.

I do not know how to get the position of the edited cell, then I read all the data and asign to data.data. In this way, you have this data available in the python server.
edit_table

Here is a 1st version of the code

import panel as pn, numpy as np
import param 
from panel.reactive import ReactiveHTML

css = """ 
table, tr, th, td {
  border: 1px  solid black;
  border-collapse: collapse;
  padding: 7px;}
td {
    border: 2px  solid black;   
    padding: 5px;
}
"""

pn.extension(raw_css = [css])

class CustomTable(ReactiveHTML):
    
    nameY = param.String(doc="Param Name.")
    nameX= param.String(doc="Axis1 Name.")
    axis1 = param.Array(doc="Axis1 Breakpokints.")
    data = param.Array(doc="Param Data.")

    _template = """
        <table id='table'> 
            <tr>
                <th rowspan="2">${nameY}</th>
                <th colspan='${axis1.length}'>${nameX}</td>
            </tr>
            <tr>
                {% for obj in axis1 %}
                    <th id="bkt-{{loop.index0}}"> 
                        {{obj}} 
                    </th>
                {% endfor %}
            </tr>
            {% for row in data %}
                {% set outer_loop = loop %}
                <tr>
                    {% for col in row %}
            <td id="col-{{loop.index0}}-{{outer_loop.index0}}" 
                        contenteditable="true" 
                    oninput="${script('some_script')}"
                        >
                            {{col}}
                        </td>
                    {% endfor %}             
                </tr>
            {% endfor %}

        </table>
    """

    _scripts = {
        'render': """
            console.log(data.data)
        """,
        'some_script': """
            const rows = Array.from(table.rows).slice(2,);
            const arr = [];
            for (const idx in rows ){
                cols = Array.from(rows[idx].children)
                const arr2 = [];
                for (const col in cols){
                // console.log(cols[col].innerText)
                    arr2.push(cols[col].innerText)
                } 
                arr.push(arr2)
            }
            data.data = arr; 
        """,
    }


ct = CustomTable(
    axis1 = np.arange(1000,7001,2000),
    data = np.random.randint(100000, size=(5, 5)),
    # data2 = data2, 
    nameX = 'EngineSpeed [rpm]',
    nameY = 'TPS [%]'
    )

btn = pn.widgets.Button(name='Save edited data')

def save(event):
    print (ct.data)

btn.on_click(save)

pn.Column(ct,btn).servable()

nghenzi · April 22, 2022, 9:35am

with the same logic of the script and using the onfocus and onblur events you can select the active cell.
it is possible now to put the content you want, but in the some_script function you can validate the data. blur

import panel as pn, numpy as np
import param 
from panel.reactive import ReactiveHTML

css = """ 
table, tr, th, td {
  border: 1px  solid black;
  border-collapse: collapse;
  padding: 7px;}
td {
    border: 2px  solid black;   
    padding: 5px;
}
"""

pn.extension(raw_css = [css])

class CustomTable(ReactiveHTML):
    
    nameY = param.String(doc="Param Name.")
    nameX= param.String(doc="Axis1 Name.")
    axis1 = param.Array(doc="Axis1 Breakpokints.")
    data = param.Array(doc="Param Data.")

    _template = """
        <table id='table'> 
            <tr>
                <th rowspan="2">${nameY}</th>
                <th colspan='${axis1.length}'>${nameX}</td>
            </tr>
            <tr>
                {% for obj in axis1 %}
                    <th id="bkt-{{loop.index0}}"> 
                        {{obj}} 
                    </th>
                {% endfor %}
            </tr>
            {% for row in data %}
                {% set outer_loop = loop %}
                <tr>
                    {% for col in row %}
            <td id="col-{{loop.index0}}-{{outer_loop.index0}}" 
                        contenteditable="true" 
                    onfocus="${script('some_focus')}"
                    onblur="${script('some_blur')}"
                    oninput="${script('some_script')}"
                        >
                            {{col}}
                        </td>
                    {% endfor %}             
                </tr>
            {% endfor %}

        </table>
    """

    _scripts = {
        'render': """
            console.log(data.data);
 
        """,
         'some_focus': """
            console.log('focus',event.target);
            event.target.style.backgroundColor = 'yellow';
         """,
         'some_blur': """
            console.log('focus',event.target);
            event.target.style.backgroundColor = 'white';
         """,
        'some_script': """
            console.log(window.getSelection())
            const rows = Array.from(table.rows).slice(2,);
            const arr = [];
            for (const idx in rows ){
                cols = Array.from(rows[idx].children)
                const arr2 = [];
                for (const col in cols){
                // console.log(cols[col].innerText)
                    arr2.push(parseFloat(cols[col].innerText))
                } 
                arr.push(arr2)
            }
            data.data = arr; 
        """,
    }


ct = CustomTable(
    axis1 = np.arange(1000,7001,2000),
    data = np.random.randint(100000, size=(5, 5)),
    # data2 = data2, 
    nameX = 'EngineSpeed [rpm]',
    nameY = 'TPS [%]'
    )

btn = pn.widgets.Button(name='Save edited data')

def save(event):
    print (type(ct.data[0][0]))

btn.on_click(save)

pn.Column(ct,btn).servable()

CmpCtrl · April 22, 2022, 2:57pm

First off, thank you very much for the replies. I appreciate the effort you put into the responses and they’re helpful. Also, i understand my complaints stem from my ignorance, and that many of them wont, cant, or shouldnt change, i just mean to share them as a reminder of what is hard for this particular newbie.

Thank you for the examples, i’ll need to take a min to digest them a little.

@Marc, the first example is great for showing the dynamic layout. One confusing thing for me tho was understanding how the servable() method works, and i found it more straightforward to instantiate a template object template = pn.template.FastListTemplate() and then append to the main or sidebar attributes. i had tried the .servable(area='main') at one point but must have had something else wrong in defining the template and couldn’t get it to work.

I need to learn more about the Viewer and @param.depends() decorator.

Your website is great as well, thanks for sharing so many examples.

@maximlt, Copy that on the power of building a ui with so little code, and for sure what i mean by magic is the stuff that is doing a lot of work in the background and is therefore confusing to me. It is certainly a tradeoff, but personally i would lean towards a little less magic in exchange for making it a little more intuitive. I see that the focus is on quick prototyping but i’m trying to use it for a little more customized tool, and that next step has been a steep one on the learning curve.

Yea, i think making it clear that a Param is like a dataclass would have helped me initially (and would still help me to understand how it differs). I’m still confused about how you can instantiate a seemingly mutable object in a Param class. I thought that was not allowed, and also a bad idea. So i guess the param.List() for instance is not a mutable object, i’ve got more to learn here clearly…

I certainly could use dataframes but they’re really inconvenient for this type of data. They’re great for many different series of data against a single axis (columns vs index) like maybe accounting data or time series data, but im more interested in matrix algebra which is doable, but not easy. The tabulator is good at working with dataframe style of data. But i’m working with data that has a 2d matrix with 2x 1d arrays defining breakpoints. Storing that sort of data in a dataframe is inconvenient. You could use column names for 1 axis, but they behave a little differently from the index which itself behaves differently from the column data. So, the best bet is to store a column for each axis and a column for the values, but that means some work reshaping before displaying or doing math. Anyway, im really not a fan of pandas, and ive got my own class to store that data in a nice convenient way, which means i now need to do a more specialized table widget.

@nghenzi this is great, and is the approach i plan to pursue. Thank you much for giving me a head start. I’ll be sure to share what i come up with.

Again, thanks for all the help.

Marc · April 22, 2022, 3:03pm

Hi @CmpCtrl

Again thanks for taking the time. This can help shape the api and doc clean up towards Panel 1.0.

Regarding the .servable versus explicit pn.template.FastListTemplate(main=..., sidebar=...).

I actually also like the second api, but the first api is a little simpler, more like the Streamlit api and works a bit better in a notebook. There are pros and cons.

CmpCtrl · April 26, 2022, 3:11pm

Alright getting back to this, i finally made some progress. I’m no good at .js but i think i have most of the basics sorted out now. it is quite messy, im sure it could be cleaned up a lot.

Again thanks for the help.

ezgif-4-907ed0e82d

import panel as pn
import param

CSS = """
table {
    padding: 8px;
    text-align: center;
    border-collapse: collapse;
}
td {
    padding: 8px;
}
tr {border-top: 1px solid}

.cal-data {
    border:2px solid;
    border-radius:8px;
    background-color:#7c7e85;
}
.cal-data-name {
    background-color:#676d81;
}

.cal-data-axis0name, .cal-data-axis0{
    background-color:#678174;
}
.cal-data-axis0name:first-child ,
.cal-data-axis0:first-child{
    border-left:2px solid;
}
.cal-data-axis0name:last-child ,
.cal-data-axis0:last-child{
    border-right:2px solid;
}
th.cal-data-axis0name {
    border-top:2px solid;
}
td.cal-data-axis0 {
    border-bottom:2px solid;
}

.cal-data-axis1name, .cal-data-axis1{
    background-color:#716781;
}
.cal-data-axis1name:first-child{
    border-top:2px solid;
}
.cal-data-axis1:last-child{
    border-bottom:2px solid;
}
th.cal-data-axis1name {
    border-left:2px solid;
    border-right:2px solid;
}
td.cal-data-axis1 {
    border-left:2px solid;
    border-right:2px solid;
}

.active-cell{
    background-color:#6a4897;
    border:1px solid #6f00ff
}
"""

pn.extension(
    raw_css=[CSS],
    sizing_mode="stretch_width",
)


class CalData2d(pn.reactive.ReactiveHTML):
    """Class for custom table data control."""

    table_name = param.String(doc="Param Name.")
    axis0_name = param.String(doc="Axis0 Name.")
    axis0 = param.Array(doc="Axis0 Breakpoints.")
    axis1_name = param.String(doc="Axis1 Name.")
    axis1 = param.Array(doc="Axis1 Breakpoints.")
    data = param.Array(doc="Param Data.")
    axis0_precision = param.Integer(doc="Axis0 Precision")
    axis1_precision = param.Integer(doc="Axis1 Precision")
    data_precision = param.Integer(doc="Data Precision")

    _template = """
<table id="cal-data" class="cal-data">
    <tbody>
        <tr id="cal-data-row-0">
            <th id="cal-data-0-0" class="cal-data-name"
                colspan="{{axis0.size+1}}">{{table_name}}</th>
        </tr>
        <tr id="cal-data-row-1">
            <th id="cal-data-1-0" class="cal-data-axis1name"
                rowspan="2">{{axis1_name}}</th>
            <th id="cal-data-1-1" class="cal-data-axis0name"
                colspan="{{axis0.size}}">{{axis0_name}}</th>
        </tr>
        <tr id="cal-data-row-2">
            {% for val in axis0 %}
            <td id="cal-data-2-{{loop.index0+1}}" class="cal-data-axis0"
                contenteditable="true"
                onfocus="${script('focus')}"
                onblur="${script('blur')}"
                onkeydown="${script('axis0_keydown')}"
                onkeypress="${script('keypress')}"
            >
                {{val}}
            </td>
            {% endfor %}
        </tr>
        {% for row in range(data.shape[1]) %}
        {% set outer_loop=loop %}
        <tr>
            <td id="cal-data-{{outer_loop.index0+3}}-0"
                contenteditable="true" class="cal-data-axis1"
                onfocus="${script('focus')}"
                onblur="${script('blur')}"
                onkeydown="${script('axis1_keydown')}"
                onkeypress="${script('keypress')}"
            >
                {{axis1[loop.index0]}}
            </td>
            {% for col in range(data.shape[0]) %}
            <td id="cal-data-{{outer_loop.index0+3}}-{{loop.index0+1}}"
                contenteditable="true" class="cal-data-data"
                onfocus="${script('focus')}"
                onblur="${script('blur')}"
                onkeydown="${script('data_keydown')}"
                onkeypress="${script('keypress')}"
            >
                {{data[loop.index0,outer_loop.index0]}}
            </td>
            {% endfor %}
        </tr>
        {% endfor %}
    </tbody>
</table>
    """

    _scripts = {
        "render": """
            //console.log(data.data);
            // console.log(data.data[0]);
            state.maxrow = data.data[0].length+2;
            state.maxcol = data.data.length;

        """,
        "block_def": """
            event.preventDefault();
        """,
        "getRowColID": """
            let regex = new RegExp(/cal-data-(?<row>[0-9]*)-(?<col>[0-9]*)-(?<id>[0-9]*)/);
            let match = regex.exec(event.target.id);
            return [parseInt(match[1]), parseInt(match[2]),parseInt(match[3])];
        """,
        "focus": """
            event.target.classList.add("active-cell");
            var cell=event.target;
            console.log('focus event, target',cell)
            var range,selection;
            if (document.body.createTextRange) {
                range = document.body.createTextRange();
                range.moveToElementText(cell);
                range.select();
            } else if (window.getSelection) {
                selection = window.getSelection();
                range = document.createRange();
                range.selectNodeContents(cell);
                selection.removeAllRanges();
                selection.addRange(range);
            }
            console.log('range',range)
            event.stopImmediatePropagation()
        """,
        "blur": """
            event.target.classList.remove("active-cell");
            console.log('blur',event.target);
        """,
        "keypress": """
            //console.log('keypress',event)
            switch(event.key) {
                case 'Enter':
                case 'ArrowUp':
                case 'ArrowDown':
                case 'ArrowLeft':
                case 'ArrowRight':
                    event.preventDefault();
                    break;
                }
        """,
        "data_keydown": """
            console.log('data keydown',event)
            var [row, col, id] = self.getRowColID(event);
            let new_row
            let new_col
            let new_val
            let orig = parseFloat(data.data[col-1][row-3]).toFixed(data.data_precision);
            console.log('loc',[row, col, id])
            switch(event.key) {
                case 'Enter':
                    event.stopImmediatePropagation();
                    console.log('txt',event.target.innerHTML)
                    new_val = parseFloat(event.target.innerHTML).toFixed(data.data_precision);
                    console.log('new_val',new_val)
                    if (isNaN(new_val)||new_val==null) {
                        new_val = orig;
                    } else {
                        data.data[col-1][row-3] = new_val;
                        console.log('updated data to',data.data[col-1][row-3])
                    }
                    event.target.innerHTML=new_val;
                    event.target.blur();
                    event.target.focus();

                    break;
                case 'ArrowUp':
                    // set data back to orig
                    event.preventDefault();
                    event.target.innerHTML = orig;
                    new_row = Math.min(Math.max(3,row-1),state.maxrow);
                    console.log('target_id',`cal-data-${new_row}-${col}-${id}`)
                    event.target.blur(); // in case of being at an edge
                    document.getElementById(`cal-data-${new_row}-${col}-${id}`).focus()
                    break;
                case 'ArrowDown':
                    // set data back to orig
                    event.preventDefault();
                    event.target.innerHTML = orig;
                    new_row = Math.min(Math.max(3,row+1),state.maxrow);
                    console.log('target_id',`cal-data-${new_row}-${col}-${id}`)
                    event.target.blur(); // in case of being at an edge
                    document.getElementById(`cal-data-${new_row}-${col}-${id}`).focus()
                    break;
                case 'ArrowLeft':
                    // set data back to orig
                    event.preventDefault();
                    event.target.innerHTML = orig;
                    new_col = Math.min(Math.max(1,col-1),state.maxcol);
                    console.log('target_id',`cal-data-${row}-${new_col}-${id}`)
                    event.target.blur(); // in case of being at an edge
                    document.getElementById(`cal-data-${row}-${new_col}-${id}`).focus()
                    break;
                case 'ArrowRight':
                    // set data back to orig
                    event.preventDefault();
                    event.target.innerHTML = orig;
                    new_col = Math.min(Math.max(1,col+1),state.maxcol);
                    console.log('target_id',`cal-data-${row}-${new_col}-${id}`)
                    event.target.blur(); // in case of being at an edge
                    document.getElementById(`cal-data-${row}-${new_col}-${id}`).focus()
                    break;
                case 'Escape':
                    event.target.innerHTML = orig;
                    event.target.blur();
            }
        """,
        "axis0_keydown": """
            console.log('axis0 keydown',event)
            var [row, col, id] = self.getRowColID(event);
            let new_row
            let new_col
            let new_val
            let orig = parseFloat(data.axis0[col-1]).toFixed(data.axis0_precision);
            console.log('loc',[row, col, id])
            switch(event.key) {
                case 'Enter':
                    event.stopImmediatePropagation();
                    //console.log('txt',event.target.innerHTML)
                    new_val = parseFloat(event.target.innerHTML).toFixed(data.axis0_precision);
                    console.log('new_val',new_val)
                    if (isNaN(new_val)||new_val==null) {
                        new_val = orig;
                    } else {
                        data.axis0[col-1] = new_val;
                        console.log('updated data to',data.axis0[col-1])
                    }
                    event.target.innerHTML=new_val;
                    event.target.blur();
                    event.target.focus();

                    break;
                case 'ArrowUp':
                    // set data back to orig
                    event.preventDefault();
                    event.target.innerHTML = orig;
                    new_row = 2;
                    console.log('target_id',`cal-data-${new_row}-${col}-${id}`)
                    event.target.blur(); // in case of being at an edge
                    document.getElementById(`cal-data-${new_row}-${col}-${id}`).focus()
                    break;
                case 'ArrowDown':
                    // set data back to orig
                    event.preventDefault();
                    event.target.innerHTML = orig;
                    new_row = 2;
                    console.log('target_id',`cal-data-${new_row}-${col}-${id}`)
                    event.target.blur(); // in case of being at an edge
                    document.getElementById(`cal-data-${new_row}-${col}-${id}`).focus()
                    break;
                case 'ArrowLeft':
                    // set data back to orig
                    event.preventDefault();
                    event.target.innerHTML = orig;
                    new_col = Math.min(Math.max(1,col-1),state.maxcol);
                    console.log('target_id',`cal-data-${row}-${new_col}-${id}`)
                    event.target.blur(); // in case of being at an edge
                    document.getElementById(`cal-data-${row}-${new_col}-${id}`).focus()
                    break;
                case 'ArrowRight':
                    // set data back to orig
                    event.preventDefault();
                    event.target.innerHTML = orig;
                    new_col = Math.min(Math.max(1,col+1),state.maxcol);
                    console.log('target_id',`cal-data-${row}-${new_col}-${id}`)
                    event.target.blur(); // in case of being at an edge
                    document.getElementById(`cal-data-${row}-${new_col}-${id}`).focus()
                    break;
                case 'Escape':
                    event.target.innerHTML = orig;
                    event.target.blur();
            }
        """,
        "axis1_keydown": """
            console.log('axis1 keydown',event)
            var [row, col, id] = self.getRowColID(event);
            let new_row
            let new_col
            let new_val
            let orig = parseFloat(data.axis1[row-3]).toFixed(data.axis1_precision);
            console.log('loc',[row, col, id])
            switch(event.key) {
                case 'Enter':
                    event.stopImmediatePropagation();
                    console.log('txt',event.target.innerHTML)
                    new_val = parseFloat(event.target.innerHTML).toFixed(data.axis1_precision);
                    console.log('new_val',new_val)
                    if (isNaN(new_val)||new_val==null) {
                        new_val = orig;
                    } else {
                        data.axis1[row-3] = new_val;
                        console.log('updated data to',data.axis1[row-3])
                    }
                    event.target.innerHTML=new_val;
                    event.target.blur();
                    event.target.focus();

                    break;
                case 'ArrowUp':
                    // set data back to orig
                    event.preventDefault();
                    event.target.innerHTML = orig;
                    new_row = Math.min(Math.max(3,row-1),state.maxrow);
                    console.log('target_id',`cal-data-${new_row}-${col}-${id}`)
                    event.target.blur(); // in case of being at an edge
                    document.getElementById(`cal-data-${new_row}-${col}-${id}`).focus()
                    break;
                case 'ArrowDown':
                    // set data back to orig
                    event.preventDefault();
                    event.target.innerHTML = orig;
                    new_row = Math.min(Math.max(3,row+1),state.maxrow);
                    console.log('target_id',`cal-data-${new_row}-${col}-${id}`)
                    event.target.blur(); // in case of being at an edge
                    document.getElementById(`cal-data-${new_row}-${col}-${id}`).focus()
                    break;
                case 'ArrowLeft':
                    // set data back to orig
                    event.preventDefault();
                    event.target.innerHTML = orig;
                    new_col = 0;
                    console.log('target_id',`cal-data-${row}-${new_col}-${id}`)
                    event.target.blur(); // in case of being at an edge
                    document.getElementById(`cal-data-${row}-${new_col}-${id}`).focus()
                    break;
                case 'ArrowRight':
                    // set data back to orig
                    event.preventDefault();
                    event.target.innerHTML = orig;
                    new_col = 0;
                    console.log('target_id',`cal-data-${row}-${new_col}-${id}`)
                    event.target.blur(); // in case of being at an edge
                    document.getElementById(`cal-data-${row}-${new_col}-${id}`).focus()
                    break;
                case 'Escape':
                    event.target.innerHTML = orig;
                    event.target.blur();
            }
        """,
    }

CmpCtrl · April 26, 2022, 6:13pm

Well it didnt take me long to get stuck again.

I have the js callbacks working reasonably well for the behavior of the table for editing. But, then struggled to figure out how to get the updated data back into python.

@nghenzi’s example is very straightforward with updates in js to data.data available in python via ct.data. That’s wasn’t working for my case and i couldn’t find a good explanation of the magic that makes that work normally to help me figure out where to look.

I’m doing a few things differently. For one, I’m using a different template structure, i need to transpose the data to be consistent with x-y indexing (vs i-j matrix indexing) so i’m using indexing ( {% for col in range(data.shape[0]) %} {{data[loop.index0,outer_loop.index0]}} {% endfor %} instead of a generator {% for col in row %} {{col}} {% endfor %} in the template. Does that matter?

Also, im using a different event callback to edit the data.data in js. Surely that doesn’t matter?

Im also updating specific values in the data.data array instead of replacing the entire array, but it seems that’s ok on the js side, is there an issue with that on the python side?

Turns out this was the issue.

Replacing the entire array in data.data seems to be very important to make it work on the python side.

new_array = data.data
new_array[col-1][row-3] = new_val;
data.data = new_array

nghenzi · April 26, 2022, 7:34pm

You explained better i could.

I believe the data from js client is only sent to the python server only when you have a new object. If you only update one element of the array, the same reference of the object is kept, then bokeh and panel js machinery can not know a different data is obtained.

CmpCtrl · April 27, 2022, 4:01pm

I’ve run into trouble again in getting data back into js. This feels like a real bug specifically, but im still worried that fixing the bug wont get all the functionality i need since to update the table from python.

More details and an example here.

github.com/holoviz/panel

ReactiveHTML table issue with param.Array() and callbacks

opened 03:23PM - 27 Apr 22 UTC

CmpCtrl

I've been working on a reactiveHTML table implementation and have gotten a lot o…f help on discord [here](https://discourse.holoviz.org/t/new-user-questions-around-custom-data-table-implementation/3604/9). But i think i've now run into some real bugs. It seems like there are some problems with callbacks using `param.Array()` parameters. I want to be able to move data between python and js, i was able to get data out of js once i realized that i must assign a new object to a param in the `data.` object, modifying one in place didnt seem to trigger the magic that updates the param on the python side (some more documentation on how this works would be great). Now, going the other way, trying to assign a new value to a param.Array() object from python triggers the following error. ``` File "...\.venv310\lib\site-packages\bokeh\document\events.py", line 443, in generate if self.model != value: ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all() ``` This feels like an issue with the implementation of param.Array(), but i really dont know enough about whats going on under the hood to have a real sense. if i am able to get around this issue, will updating the `data.` object in js magically update my table? it seems like probably not since i am using jinja templating instead of the `${}` syntax. The user guide is very vague on what is actually different in that case, but says "The difference between Jinja2 literal templating and the JS templating syntax is important to note. While literal values are inserted during the initial rendering step they are not dynamically linked.". I have had no luck using the `${}` syntax with a `param.Array` it causes the following `ValueError: Array parameter 'axis0' value must be an instance of ndarray, not Str(ndarray, sizing_mode='stretch_width').` Is there a way to trigger a js callback from python explicitly without all the magic around params? #### ALL software version info ``` panel 0.13.0 param 1.12.1 python 3.10.2 ``` #### Description of expected behavior and the observed behavior I expect to be able to assign a new array to a `param.Array()` object in python to pass the new data to the js `data.` object. #### Complete, minimal, self-contained example code that reproduces the issue This example is not as minimal as id like, but should demonstrate. If you edit a value in the top row, it should trigger the `update_axes` callback the first several lines of which work, returning ``` axis0 edited to :[ 0. 100. 200. 300. 400. 500. 0. 700. 800. 900. 1000.] data edited in place to: [1000. 1. 2. 3. 4. 5. 6. 7. 8. 9. 10.] data updated to :[1000. 1. 2. 3. 4. 5. 6. 7. 8. 9. 10.] ``` I think this shows that editing the `data` param in place does not trigger the `update_data` callback, but then does call it when re-assigning the whole parameter. That does update the `data` param on the python side, but is not passed to js. Then, trying to assign random data to the `data` param appears to actually trigger updating the js which fails with the value error. ``` ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all() data update to :[0.37946985 0.69968089 0.25976111 0.74877305 0.38919826 0.82425161 0.92448139 0.70760514 0.1614588 0.37518864 0.17476377] ``` That does however trigger the `update_data` callback again, but again hasn't passed any data to .js. ```python """Minimal Example""" import panel as pn import param import numpy as np CSS = """ table { padding: 8px; text-align: center; border-collapse: collapse; } td { padding: 8px; } tr {border-top: 1px solid} .cal-data { border:2px solid; border-radius:8px; background-color:#7c7e85; } .cal-data-name { background-color:#676d81; } .cal-data-axis0name, .cal-data-axis0{ background-color:#678174; } .cal-data-axis0name:first-child , .cal-data-axis0:first-child{ border-left:2px solid; } .cal-data-axis0name:last-child , .cal-data-axis0:last-child{ border-right:2px solid; } th.cal-data-axis0name { border-top:2px solid; } td.cal-data-axis0 { border-bottom:2px solid; } .active-cell{ background-color:#6a4897; border:1px solid #6f00ff } """ pn.extension( raw_css=[CSS], sizing_mode="stretch_width", ) class CalData1d(pn.reactive.ReactiveHTML): """Class for custom table data control.""" table_name = param.String(doc="Param Name.") axis0_name = param.String(doc="Axis0 Name.") axis0 = param.Array(doc="Axis0 Breakpoints.") data = param.Array(doc="Param Data.") _template = """ <table id="cal-data" class="cal-data"> <tbody> <tr id="cal-data-row-0"> <th id="cal-data-0-0" class="cal-data-name" colspan="{{axis0.size}}">{{table_name}}</th> </tr> <tr id="cal-data-row-1"> <th id="cal-data-1-1" class="cal-data-axis0name" colspan="{{axis0.size}}">{{axis0_name}}</th> </tr> <tr id="cal-data-row-2"> {% for val in axis0 %} <td id="cal-data-2-{{loop.index0}}" class="cal-data-axis0" contenteditable="true" onfocus="${script('focus')}" onblur="${script('blur')}" onkeydown="${script('axis0_keydown')}" onkeypress="${script('keypress')}" > {{val}} </td> {% endfor %} </tr> <tr> {% for val in data %} <td id="cal-data-3-{{loop.index0}}" contenteditable="true" class="cal-data-data" onfocus="${script('focus')}" onblur="${script('blur')}" onkeydown="${script('data_keydown')}" onkeypress="${script('keypress')}" > {{val}} </td> {% endfor %} </tr> </tbody> </table> """ _scripts = { "render": """ //console.log(data.data); // console.log(data.data[0]); state.maxrow = 2; state.maxcol = data.data.length; """, "block_def": """ event.preventDefault(); """, "getRowColID": """ let regex = new RegExp(/cal-data-(?<row>[0-9]*)-(?<col>[0-9]*)-(?<id>[0-9]*)/); let match = regex.exec(event.target.id); return [parseInt(match[1]), parseInt(match[2]),parseInt(match[3])]; """, "focus": """ event.target.classList.add("active-cell"); var cell=event.target; console.log('focus event, target',cell) var range,selection; if (document.body.createTextRange) { range = document.body.createTextRange(); range.moveToElementText(cell); range.select(); } else if (window.getSelection) { selection = window.getSelection(); range = document.createRange(); range.selectNodeContents(cell); selection.removeAllRanges(); selection.addRange(range); } console.log('range',range) event.stopImmediatePropagation() """, "blur": """ event.target.classList.remove("active-cell"); console.log('blur',event.target); """, "keypress": """ //console.log('keypress',event) switch(event.key) { case 'Enter': case 'ArrowUp': case 'ArrowDown': case 'ArrowLeft': case 'ArrowRight': event.preventDefault(); break; } """, "data_keydown": """ console.log('data keydown',event) var [row, col, id] = self.getRowColID(event); let new_row let new_col let new_val let new_array let orig = parseFloat(data.data[col]); const clone = (items) => items.map(item => Array.isArray(item) ? clone(item) : item); console.log('loc',[row, col, id]) switch(event.key) { case 'Enter': event.stopImmediatePropagation(); console.log('txt',event.target.innerHTML) new_val = parseFloat(event.target.innerHTML); console.log('new_val',new_val) if (isNaN(new_val)||new_val==null) { new_val = orig; } else { new_array = clone(data.data) new_array[col] = new_val; data.data = new_array console.log('updated data to',data.data[col]) } event.target.innerHTML=new_val; event.target.blur(); event.target.focus(); break; case 'ArrowUp': // set data back to orig event.preventDefault(); event.target.innerHTML = orig; new_row = 3; console.log('target_id',`cal-data-${new_row}-${col}-${id}`) event.target.blur(); // in case of being at an edge document.getElementById(`cal-data-${new_row}-${col}-${id}`).focus() break; case 'ArrowDown': // set data back to orig event.preventDefault(); event.target.innerHTML = orig; new_row = 3; console.log('target_id',`cal-data-${new_row}-${col}-${id}`) event.target.blur(); // in case of being at an edge document.getElementById(`cal-data-${new_row}-${col}-${id}`).focus() break; case 'ArrowLeft': // set data back to orig event.preventDefault(); event.target.innerHTML = orig; new_col = Math.min(Math.max(0,col-1),state.maxcol); console.log('target_id',`cal-data-${row}-${new_col}-${id}`) event.target.blur(); // in case of being at an edge document.getElementById(`cal-data-${row}-${new_col}-${id}`).focus() break; case 'ArrowRight': // set data back to orig event.preventDefault(); event.target.innerHTML = orig; new_col = Math.min(Math.max(0,col+1),state.maxcol); console.log('target_id',`cal-data-${row}-${new_col}-${id}`) event.target.blur(); // in case of being at an edge document.getElementById(`cal-data-${row}-${new_col}-${id}`).focus() break; case 'Escape': event.target.innerHTML = orig; event.target.blur(); } """, "axis0_keydown": """ console.log('axis0 keydown',event) var [row, col, id] = self.getRowColID(event); let new_row let new_col let new_val let new_array let orig = parseFloat(data.axis0[col]); const clone = (items) => items.map(item => Array.isArray(item) ? clone(item) : item); console.log('loc',[row, col, id]) switch(event.key) { case 'Enter': event.stopImmediatePropagation(); //console.log('txt',event.target.innerHTML) new_val = parseFloat(event.target.innerHTML); console.log('new_val',new_val) if (isNaN(new_val)||new_val==null) { new_val = orig; } else { new_array = clone(data.axis0) new_array[col] = new_val; state.is_monoton_arg = new_array data.axis0 = new_array console.log('updated data to',data.axis0[col]) } event.target.innerHTML=new_val; event.target.blur(); event.target.focus(); break; case 'ArrowUp': // set data back to orig event.preventDefault(); event.target.innerHTML = orig; new_row = 2; console.log('target_id',`cal-data-${new_row}-${col}-${id}`) event.target.blur(); // in case of being at an edge document.getElementById(`cal-data-${new_row}-${col}-${id}`).focus() break; case 'ArrowDown': // set data back to orig event.preventDefault(); event.target.innerHTML = orig; new_row = 2; console.log('target_id',`cal-data-${new_row}-${col}-${id}`) event.target.blur(); // in case of being at an edge document.getElementById(`cal-data-${new_row}-${col}-${id}`).focus() break; case 'ArrowLeft': // set data back to orig event.preventDefault(); event.target.innerHTML = orig; new_col = Math.min(Math.max(0,col-1),state.maxcol); console.log('target_id',`cal-data-${row}-${new_col}-${id}`) event.target.blur(); // in case of being at an edge document.getElementById(`cal-data-${row}-${new_col}-${id}`).focus() break; case 'ArrowRight': // set data back to orig event.preventDefault(); event.target.innerHTML = orig; new_col = Math.min(Math.max(0,col+1),state.maxcol); console.log('target_id',`cal-data-${row}-${new_col}-${id}`) event.target.blur(); // in case of being at an edge document.getElementById(`cal-data-${row}-${new_col}-${id}`).focus() break; case 'Escape': event.target.innerHTML = orig; event.target.blur(); } """, } def update_axes(event: param.parameterized.Event): """handle updates to axes""" print(f"axis0 edited to :{event.new.astype(float)}") event.obj.data[0] = 1000 print(f"data edited in place to: {event.obj.data}") event.obj.data = event.obj.data event.obj.data = np.random.random((11,)) def update_data(event: param.parameterized.Event): """handle data update.""" new = event.new.astype(float) print(f"data updated to :{new}") def build_panel_widget() -> CalData1d: """Build CalData widget for param.""" data = np.linspace(0, 10, 11) axis0 = np.linspace(0, 1000, 11) cdata = CalData1d( table_name="Test Table", axis0_name=("test axis"), axis0=axis0, data=data ) watcher = cdata.param.watch(update_axes, ["axis0"]) watcher2 = cdata.param.watch(update_data, ["data"]) return cdata if __name__ == "__main__": pn.serve( {"demo": build_panel_widget}, show=True, autoreload=False, ) ```