How to get a JavaScript event to trigger a Python event

I would like to propagate mouse selection from vtk.js back to the vtk Pane. I am able to capture clicks in the vtk.js window, but I am having trouble getting the selection information from JavaScript back into Python. Looking through the examples, I haven’t been able to identify any bidirectional connections where the JavaScript code updates a field and it is received on the Python side. Are there any examples of a Pane with such a bidirectional connection between JavaScript and Python?

EDIT: this work is in conjunction with ERDC.

There are some hacky ways to achieve this right now but in future this will be supported more easily. I’d recommend looking at this issue to see both the current workarounds and discussions about a cleaner approach that will become possible once bokeh 2.0 is released.

@philippjfr Thank you for your fast response! I will take a look at the issue you mentioned as a way to proceed.

However in the issue it’s a hacky solution to send data from python to javascript but I suppose you want to do the opposite

Ah, @xavArtley is right. I do want to go in the opposite direction. @philippjfr can I modify a ColumnDataSource in javascript and be notified of its changes in python?

Oh right sorry. Yes, the opposite direction is also possible but potentially more difficult particularly where a ColumnDataSource is concerned. I’ll try to put together an example.

1 Like

Thanks @philippjfr! Please let me know if there is anything I can do to help.

Here is my hacky way to do it in a notebook:

import panel as pn
pn.extension('vtk')

# create a vtk pane
dragon = pn.pane.VTK('https://raw.githubusercontent.com/Kitware/vtk-js/master/Data/StanfordDragon.vtkjs',
                     sizing_mode='stretch_width', height=400, enable_keybindings=True, orientation_widget=True,
                     serialize_on_instantiation=True)

# comm to add an event listener on the vtk pan
comm_py_js = pn.widgets.StaticText(style={'visibility': 'hidden', 'width': 0, 'height': 0, 'overflow': 'hidden'}, margin=0)
# comm to return informations from javascript to python
comm_js_py = pn.widgets.StaticText(style={'visibility': 'hidden', 'width': 0, 'height': 0, 'overflow': 'hidden'}, margin=0)

comm_py_js.jscallback(args={'vtkpan':dragon, "comm_js_py":comm_js_py}, value="""
vtkpan.renderer_el.getContainer().addEventListener("click", (evt) => {
const mouse_pos = {
    screenX: evt.screenX,
    screenY: evt.screenY,
    clientX: evt.clientX,
    clientY: evt.clientY,
}
comm_js_py.setv({text: JSON.stringify(mouse_pos)}, {silent: true})
comm_js_py.properties.text.change.emit()
})
""")

# ouput div to write some information retrieve on python side
output_div = pn.widgets.StaticText(width=500, value='')
def update_output(*events):
    output_div.value += events[0].new + '\n'
comm_js_py.param.watch(update_output, ['value'], onlychanged=False)

display(pn.Column(comm_js_py, comm_py_js, pn.Row(dragon, output_div), sizing_mode="stretch_width"))
comm_py_js.param.trigger("value") #set the mouse click event (must happen after the panel display)

ezgif.com-video-to-gif (7)

Thanks @xavArtley! That’s roughly what I had in mind. Hopefully the Data model will make this a lot cleaner.

I notice a bug when I tried to deploy in a bokeh server this solution.
this comm_js_py.properties.text.change.emit() does not trigger update on the python side
comm_js_py.text = JSON.stringify(mouse_pos) works but only if the mouse click is not on the same position
I sligtly modify the example to add the click listener with a toggle button :

import panel as pn
pn.extension('vtk')

# create a vtk pane
dragon = pn.pane.VTK('https://raw.githubusercontent.com/Kitware/vtk-js/master/Data/StanfordDragon.vtkjs',
                     sizing_mode='stretch_width', height=400, enable_keybindings=True, orientation_widget=True,
                     serialize_on_instantiation=True)

# comm to add an event listener on the vtk pan
btn = pn.widgets.Toggle(name='custom click event', value=False)
# comm to return informations from javascript to python
comm_js_py = pn.widgets.StaticText(style={'visibility': 'hidden', 'width': 0, 'height': 0, 'overflow': 'hidden'}, margin=0)

btn.jscallback(args={'vtkpan':dragon, "comm_js_py":comm_js_py}, value="""
if (!cb_obj.listener_function){
    cb_obj.listener_function = (evt) => {
    const mouse_pos = {
        x: evt.offsetX,
        y: evt.offsetY,
    }
    /*comm_js_py.text = JSON.stringify(mouse_pos)*/
    console.log('event')
    comm_js_py.setv({text: JSON.stringify(mouse_pos)}, {silent: true})
    comm_js_py.properties.text.change.emit()
    }
}
if(cb_obj.active){
    vtkpan.renderer_el.getContainer().addEventListener("click", cb_obj.listener_function)
}else{
    vtkpan.renderer_el.getContainer().removeEventListener("click", cb_obj.listener_function)
}


""")

# ouput div to write some information retrieve on python side
output_div = pn.widgets.StaticText(width=500, value='')
def update_output(*events):
    output_div.value += events[0].new + '<br>'
comm_js_py.param.watch(update_output, ['value'], onlychanged=False)

pn.Column(comm_js_py, btn, pn.Row(dragon, output_div), sizing_mode="stretch_width").show()

Thanks for the example! This is just what I need to iterate on my idea to register a selection function.

1 Like

Thank you @philippjfr and @xavArtley for your help thus far! Unfortunately, I am still stuck. I am currently trying to construct a watchable element that can be passed to the vtk pane’s _link_props() method. I have found an example of how bokeh’s ColumnDataSource is passed into javascript, but it doesn’t appear “watchable” (i.e. it doesn’t have an instance of param.parameterized.Parameters). The above snippet shows how a pn.widgets.StaticText is “watchable”, but I don’t know how to pass it into javascript (I have tried declaring it as Any and as String in panel/panel/models/vtk.(py,ts), but both resulted in nothing on the javascript side).

FWIW, when I ran either of the above code snippets they also did not return selection values. It appears as though the jscallback is never called (simply putting a console.log('hello world') in the callback resulted in no calls). For this test, I was running a vanilla panel instance from a docker container.

@xavArtley What tag or SHA of Panel are you running? Perhaps my issue is that I am using Panel v0.7.0?

@tjcorona @xavArtley @philippjfr
Thanks for this really useful post - I am trying to do something similar to your example, however rather than return the screen pointer position to my python script, I really want to be able to select a node or cell of the mesh, or alternatively hover over a node and it returns the scalar value and the node ID. Do you think this is possible with the current vtk.js and panel? Any pointers / help on this would be much appreciated.