Update DataFrame with Button Click

Hello,

I am working on a small panel app to show me the status of a supercomputer scheduling queue. I’d like to allow the user to push a button to get new information. So far, I have this:

import panel as pn
import pandas as pd
import fabric
pn.extension()

header = pn.Pane("# Monitoring for `LIG-T`")

def get_queue():
    print("Updating queue!")
    stan = fabric.Connection("stan1.awi.de")
    result = stan.run("qstat -l", hide=True)
    titles = result.stdout.split()[:14]
    values = result.stdout.split()[14+14:]
    queue_dict = {title: value for title, value in zip(titles, values)}
    return pn.widgets.DataFrame(pd.DataFrame.from_dict(queue_dict, orient="index").transpose())

def get_progressbar():
    print("Updating progressbar!")
    stan = fabric.Connection("stan1.awi.de")
    with stan.cd("/ace/user/pgierz/cosmos-aso-wiso/LIG-T/scripts"):
        result = stan.run("cat *date", hide=True)
        date, run_number = result.stdout.split()
        start_year = 2698
        year = int(date[:4]) - start_year
        progress = year / (6699 - start_year) * 100
        return pn.Row(str(round(progress, 2)) + "%", pn.widgets.Progress(value=round(progress)))

def run_both(event):
    queue = get_queue()
    progress_bar = get_progressbar()

button = pn.widgets.Button(name='Update', button_type='primary')
button.on_click(run_both)

progress_bar = get_progressbar()
queue = get_queue()

gspec = pn.GridSpec(sizing_mode='stretch_both', max_width=1000)#, max_height=800)
gspec[0, 0] = header
gspec[1, 0] = button
gspec[2, 0] = progress_bar
gspec[3, 0] = queue
gspec.servable()
#gspec

So far, so good. I get a panel app that displays what I want: the progress of my simulation, and the current supercomputer queue. I can then serve this with:

$ panel server <notebook_name>

Pushing the button also gives me a printout in my terminal, so I know the functions are running. However, the data frame and progress bar don’t change – for the progress bar I can understand, this simulation will take several weeks to finish, so I won’t see a percent jump every time I click. However, one part of the queue data frame is the elapsed time of the job, in seconds. This should definitely change when I push the button.

What am I doing wrong?

The problem is that your run_both function calls the other functions but then doesn’t do anything with those variables. You probably want something like:

def run_both(event):
    gspec[2, 0] = get_progressbar()
    gspec[3, 0] = get_queue()

I had a sneaky suspicion that the updated objects were landing in memory nirvana :wink: However, that change yields the following:

Updating queue!
WARNING:param.GridSpec00013: Specified region overlaps with the following existing object(s) in the grid:

    (3, 0): DataFrame(sizing_mode='stretch_both', value=              ...)

The following shows a view of the grid (empty: 0, occupied: 1, overlapping: 2):

[[1]
 [1]
 [1]
 [2]]
2020-04-17 13:20:04,811 Specified region overlaps with the following existing object(s) in the grid:

    (3, 0): DataFrame(sizing_mode='stretch_both', value=              ...)

The following shows a view of the grid (empty: 0, occupied: 1, overlapping: 2):

[[1]
 [1]
 [1]
 [2]]
2020-04-17 13:20:04,812 Exception in callback functools.partial(<bound method IOLoop._discard_future_result of <tornado.platform.asyncio.AsyncIOMainLoop object at 0x118022eb8>>, <Task finished coro=<_needs_document_lock.<locals>._needs_document_lock_wrapper() done, defined at /Users/pgierz/opt/miniconda3/envs/panel_apps/lib/python3.6/site-packages/bokeh/server/session.py:51> exception=KeyError('1001',)>)
Traceback (most recent call last):
  File "/Users/pgierz/opt/miniconda3/envs/panel_apps/lib/python3.6/site-packages/tornado/ioloop.py", line 743, in _run_callback
    ret = callback()
  File "/Users/pgierz/opt/miniconda3/envs/panel_apps/lib/python3.6/site-packages/tornado/ioloop.py", line 767, in _discard_future_result
    future.result()
  File "/Users/pgierz/opt/miniconda3/envs/panel_apps/lib/python3.6/site-packages/bokeh/server/session.py", line 67, in _needs_document_lock_wrapper
    result = func(self, *args, **kwargs)
  File "/Users/pgierz/opt/miniconda3/envs/panel_apps/lib/python3.6/site-packages/bokeh/server/session.py", line 195, in with_document_locked
    return func(*args, **kwargs)
  File "/Users/pgierz/opt/miniconda3/envs/panel_apps/lib/python3.6/site-packages/bokeh/document/document.py", line 1164, in wrapper
    return doc._with_self_as_curdoc(invoke)
  File "/Users/pgierz/opt/miniconda3/envs/panel_apps/lib/python3.6/site-packages/bokeh/document/document.py", line 1150, in _with_self_as_curdoc
    return f()
  File "/Users/pgierz/opt/miniconda3/envs/panel_apps/lib/python3.6/site-packages/bokeh/document/document.py", line 1163, in invoke
    return f(*args, **kwargs)
  File "/Users/pgierz/opt/miniconda3/envs/panel_apps/lib/python3.6/site-packages/bokeh/document/document.py", line 953, in remove_then_invoke
    return callback(*args, **kwargs)
  File "/Users/pgierz/opt/miniconda3/envs/panel_apps/lib/python3.6/site-packages/panel/viewable.py", line 525, in _change_event
    self._process_events(events)
  File "/Users/pgierz/opt/miniconda3/envs/panel_apps/lib/python3.6/site-packages/panel/viewable.py", line 511, in _process_events
    self.param.set_param(**self._process_property_change(events))
  File "/Users/pgierz/opt/miniconda3/envs/panel_apps/lib/python3.6/site-packages/param/parameterized.py", line 1365, in set_param
    self_._batch_call_watchers()
  File "/Users/pgierz/opt/miniconda3/envs/panel_apps/lib/python3.6/site-packages/param/parameterized.py", line 1480, in _batch_call_watchers
    watcher.fn(*events)
  File "/Users/pgierz/Documents/Code/panel_apps/monitoring_ligt/LIG-T_monitoring.ipynb", line 35, in run_both
    "    } finally {\n",
  File "/Users/pgierz/opt/miniconda3/envs/panel_apps/lib/python3.6/site-packages/panel/layout.py", line 1180, in __setitem__
    self.param.trigger('objects')
  File "/Users/pgierz/opt/miniconda3/envs/panel_apps/lib/python3.6/site-packages/param/parameterized.py", line 1422, in trigger
    self_.set_param(**params)
  File "/Users/pgierz/opt/miniconda3/envs/panel_apps/lib/python3.6/site-packages/param/parameterized.py", line 1365, in set_param
    self_._batch_call_watchers()
  File "/Users/pgierz/opt/miniconda3/envs/panel_apps/lib/python3.6/site-packages/param/parameterized.py", line 1480, in _batch_call_watchers
    watcher.fn(*events)
  File "/Users/pgierz/opt/miniconda3/envs/panel_apps/lib/python3.6/site-packages/panel/viewable.py", line 789, in param_change
    self._update_model(events, msg, root, model, doc, comm)
  File "/Users/pgierz/opt/miniconda3/envs/panel_apps/lib/python3.6/site-packages/panel/layout.py", line 70, in _update_model
    msg[self._rename['objects']] = self._get_objects(model, old, doc, root, comm)
  File "/Users/pgierz/opt/miniconda3/envs/panel_apps/lib/python3.6/site-packages/panel/layout.py", line 983, in _get_objects
    child, _ = obj._models[root.ref['id']]
KeyError: '1001'

is that a bug?

When you construct the GridSpec can you set mode='override'. That said this part does seem like an error:

  File "/Users/pgierz/opt/miniconda3/envs/panel_apps/lib/python3.6/site-packages/panel/layout.py", line 983, in _get_objects
    child, _ = obj._models[root.ref['id']]
KeyError: '1001'

I’m experiencing something similar with the simple snippet below:


import panel as pn


gspec = pn.GridSpec(mode='override')
gspec[0,0]=pn.Spacer(background='blue')

def update():
    gspec[0,1] = pn.Spacer(background='Orange')
    cb.stop()


cb = gspec.add_periodic_callback(update, 300)

gspec.servable()

It yields almost the same error as described above:

 /python3.7/site-packages/panel/layout.py", line 990, in _get_objects
    child, _ = obj._models[root.ref['id']]
KeyError: '1002'

I get the same after setting mode to ‘override’

@jannathundal Which version of Panel are you using?

0.9.7