Layout Problem with DynamicMaps and hv.Buffer

Panel
,
1

Getting really rusty: I wanted to use an hv.Buffer to update two Graphs and a table.
For the life of me, I can’t get the layout pn.Column( pn.Row( h_xy, h_err), h_tbl )

I’d appreciate help getting this to work, or a better solution?!

from IPython.core.interactiveshell import InteractiveShell
InteractiveShell.ast_node_interactivity = "all"

import time
import numpy as np
import pandas as pd

from bokeh.models.widgets import HTMLTemplateFormatter
import param
import holoviews as hv; hv.extension("bokeh", logo=False)
import panel as pn;     pn.extension()

class GraphicalMonitorA():
    """monitor the error evolution of an iterative scheme"""
    def __init__(self, sz=10,use_log=False):
        data_def     = {'step':int, 'x': float, 'y': float, 'error': float }
        data         = pd.DataFrame( {i:[] for i in data_def.keys()}, columns=data_def.keys())\
                            .astype(dtype=data_def)

        self.buffer  = hv.streams.Buffer(data, length=sz, index=False)
        self.use_log = use_log

        def monitor_data(dat):
            self.buffer .send( pd.DataFrame( [[dat[0], dat[1], dat[2], dat[3] ]],
                                             columns=data_def .keys() ).astype(dtype=data_def ))    
        def plot(data):
            if data.empty:
                return ((hv.Scatter([],'x','y')*hv.Curve([],'x','y')*\
                         hv.Scatter([],'x','y')).opts(width=500,xticks=4,yticks=4,  tools=['hover'], show_grid=True,
                                                      title='Solution Estimate')+\
                         hv.Curve([],'step','error').opts(logy=self.use_log, yticks=4, show_grid=True,
                                                          title='Error') +\
                         hv.Table( data ).opts(height=450)
                       )

            h_last_point = hv.Scatter( (data['x'].iloc[-1], data['y'].iloc[-1]), 'x', 'y' ).opts( size=10,color='red')
            h_points     = hv.Curve(   (data['x'],data['y']), 'x', 'y' )*\
                           hv.Scatter( (data['x'],data['y']), 'x', 'y' ).opts(color='darkblue',padding=.05,size=8, tools=['hover'], show_grid=True)
            h_error      = hv.Curve(   (data['step'],data['error']), 'step', 'error' )
            h_tbl        = hv.Table( data )

            h            = (h_points.opts(xticks=4,yticks=4) * h_last_point).opts(width=500) +\
                           h_error.opts(padding=.05,xticks=4,logy=self.use_log) +\
                           h_tbl

            return h.cols(2)

        self.monitor  = monitor_data
        self.plot     = plot

        self.dmap     = hv.DynamicMap( self.plot,  streams=[self.buffer ])

    def reset_plots(self):
        self.buffer .clear()

plots = GraphicalMonitorA(5,use_log=False)
plots.dmap
for i in range(5):
    plots.monitor( [i, 0.1 * i, 0.2 *i, 1e-12 *i] )
2

Try this:

import pandas as pd
import panel as pn
import holoviews as hv

hv.extension("bokeh")


class GraphicalMonitorA:
    """monitor the error evolution of an iterative scheme"""

    def __init__(self, sz=10, use_log=False):
        self.buffer = hv.streams.Buffer(self._to_dataframe(), length=sz, index=False)
        self.use_log = use_log

        self.dmap = hv.DynamicMap(self.plot, streams=[self.buffer])

    def _to_dataframe(self, data=None):
        data_def = {"step": int, "x": float, "y": float, "error": float}
        if data is None:
            return pd.DataFrame(
                {i: [] for i in data_def},
            ).astype(dtype=data_def)

        return pd.DataFrame([data], columns=data_def).astype(dtype=data_def)

    def reset_plots(self):
        self.buffer.clear()

    def monitor(self, data):
        self.buffer.send(self._to_dataframe(data))

    def plot(self, data):
        if data.empty:
            return (
                (
                    hv.Scatter([], "x", "y")
                    * hv.Curve([], "x", "y")
                    * hv.Scatter([], "x", "y")
                ).opts(
                    width=500,
                    xticks=4,
                    yticks=4,
                    tools=["hover"],
                    show_grid=True,
                    title="Solution Estimate",
                )
                + hv.Curve([], "step", "error").opts(
                    logy=self.use_log, yticks=4, show_grid=True, title="Error"
                )
                + hv.Table(data).opts(height=450)
            ).cols(2)

        h_last_point = hv.Scatter(
            (data["x"].iloc[-1], data["y"].iloc[-1]), "x", "y"
        ).opts(size=10, color="red")

        h_points = hv.Curve((data["x"], data["y"]), "x", "y") * hv.Scatter(
            (data["x"], data["y"]), "x", "y"
        ).opts(color="darkblue", padding=0.05, size=8, tools=["hover"], show_grid=True)
        h_error = hv.Curve((data["step"], data["error"]), "step", "error")
        h_tbl = hv.Table(data)

        h = (
            (h_points.opts(xticks=4, yticks=4) * h_last_point).opts(width=500)
            + h_error.opts(padding=0.05, xticks=4, logy=self.use_log)
            + h_tbl
        )

        return h.cols(2)




plots = GraphicalMonitorA(5, use_log=False)


# This is to make it update the plot  :)
i = 1

def add_point():
    global i
    i += 1
    plots.monitor([i, 0.1 * i, 0.2 * i, 1e-12 * i])

pn.state.add_periodic_callback(add_point, 1000)

# Could be replaced with these line if needed
# for i in range(5):
#     plots.monitor([i, 0.1 * i, 0.2 * i, 1e-12 * i])

pn.panel(plots.dmap).servable()

2 Likes
3

Thank you! I forgot to return h.cols(2) in both the empty data and the non empty data case.
I like your improvements to my code too!

I ended up with separate plot functions for each of the display elements,
wrapped in DynamicMaps and displayed using panel instead.

Still wondering if there might be a solution using newer capabilities of Panel, Param, holoviews :slight_smile:

4

Hmmm, now the table formatting does not work.
In the following, the error column does not display according to the specified
format, yet, when I display the table separately, I get what I expected:

table
table922×786 66.9 KB 

from IPython.core.interactiveshell import InteractiveShell
InteractiveShell.ast_node_interactivity = "all"

import time
import numpy as np
import pandas as pd

from bokeh.models.widgets import HTMLTemplateFormatter
import param
import holoviews as hv; hv.extension("bokeh", logo=False)
import panel as pn;     pn.extension()
class GraphicalMonitorA:
    """monitor the error evolution of an iterative scheme"""

    def __init__(self, sz=10, use_log=False):
        self.buffer  = hv.streams.Buffer(self._to_dataframe(), length=sz, index=False)
        self.use_log = use_log

        self.plots = pn.Column(
            pn.Row( hv.DynamicMap(self.display_xy,  streams=[self.buffer]),
                    hv.DynamicMap(self.display_err, streams=[self.buffer])
                  ),
             hv.DynamicMap(self.display_tbl, streams=[self.buffer])
        )

    def _to_dataframe(self, data=None):
        data_def = {"step": int, "x": float, "y": float, "error": float}
        if data is None:
            return pd.DataFrame(
                {i: [] for i in data_def},
            ).astype(dtype=data_def)

        return pd.DataFrame([data], columns=data_def).astype(dtype=data_def)

    def reset_plots(self):
        self.buffer.clear()

    def monitor(self, data):
        self.buffer.send(self._to_dataframe(data))

    def plot(self, data):
        return ( self.display_xy(data) + self.display_err(data) + self.display_tbl(data)).cols(2)

    def display_xy(self, data):
        if data.empty:
            return ( hv.Scatter([], "x", "y") * hv.Curve([], "x", "y")  * hv.Scatter([], "x", "y")
                    ).opts( width=500,  xticks=4, yticks=4, tools=["hover"], show_grid=True, title="Solution Estimate" )
        h_last_point = hv.Scatter( (data["x"].iloc[-1], data["y"].iloc[-1]), "x", "y" ).opts(size=10, color="red")

        h_points = hv.Curve((data["x"], data["y"]), "x", "y") *\
                   hv.Scatter( (data["x"], data["y"]), "x", "y" )\
                     .opts(color="darkblue", padding=0.05, size=8, tools=["hover"], show_grid=True)

        return (h_points.opts(xticks=4, yticks=4) * h_last_point).opts(width=500)

    def display_err( self, data):
        if data.empty:
            return hv.Curve([], "step", "error").opts(  logy=self.use_log, yticks=4, show_grid=True, title="Error" )
        return hv.Curve((data["step"], data["error"]), "step", "error").opts(padding=0.05, xticks=4, logy=self.use_log)

    def display_tbl( self, data ):
        if data.empty:
            return hv.Table(data).opts(height=450,width=500, hooks=[GraphicalMonitorA._table_formatter])
        return hv.Table(data).opts(hooks=[GraphicalMonitorA._table_formatter])

    def _table_formatter(plot, element):
        plot.handles['table'].columns[3].formatter = HTMLTemplateFormatter(
            template = """<div style="color:red;"><%= value ? value.toExponential(4) : value %></div>""")


plots = GraphicalMonitorA(15,use_log=False)
plots.plots
for i in range(20):
    plots.monitor( [i, 0.1 * i, 0.2 *i, 1e-8 *i] )
5

I think you should either remove _table_formatter outside the class or add self to it.

6

I tried to move it outside the class. Same issue :frowning:

7

If you add your imports to the example, I will take a look at it.

8

I just did. I really appreciate the help!

9

Happy to help :slight_smile:

You needs to move the hook outside the calss and rename the function in display_tbl

...
    def display_tbl( self, data ):
        if data.empty:
            return hv.Table(data).opts(height=450,width=500, hooks=[_table_formatter])
        return hv.Table(data).opts(hooks=[_table_formatter])


def _table_formatter(plot, element):
    plot.handles['table'].columns[3].formatter = HTMLTemplateFormatter(
        template = """<div style="color:red;"><%= value ? value.toExponential(4) : value %></div>""")

...

image
image907×634 52 KB

10

Weird. This does not work in my environment:
I get the table updating without formating,
yet displaying the table with plots.plots[1] following execution shows it
with the format applied. Oh well…

numpy : 1.21.5
pandas : 1.3.5
bokeh : 2.4.2
holoviews : 1.14.8
panel : 0.12.7

11

I could recreate your problem. It seems to be related to the IPython imports at the top.

If I add plots.plots to the bottom of the cell I get the following (somewhat weird) result:

12

yup! that’s what I see. I’d be really interested in a workaround?!

A related question: I tried to add an np.array to the hv.Buffer,
rather than explicitely adding vector entries.
Any way to do that?
My use case consists of vectors with dimension <= 5.

I hacked to_data_frame to a flattened list using array.tolist()…