Streamz + hvplot seems fairly broken (any alternatives?)

dharhas · November 21, 2022, 11:15pm

We are trying to build some streaming plots and we are experiencing a lot of jankiness. In trying to get a reproducible sample, I found that in a new environment I can’t even do simple streaming, i.e.

import hvplot.streamz  # noqa
from streamz.dataframe import Random
df = Random(interval='200ms', freq='50ms')
df.hvplot()

errors out with

File ~/mamba/envs/test-streaming/lib/python3.11/site-packages/holoviews/core/spaces.py:514, in Callable.noargs(self)
    511 @property
    512 def noargs(self):
    513     "Returns True if the callable takes no arguments"
--> 514     noargs = inspect.ArgSpec(args=[], varargs=None, keywords=None, defaults=None)
    515     return self.argspec == noargs

AttributeError: module 'inspect' has no attribute 'ArgSpec'

So a couple of questions

should I raise this on hvplot or the streamz github repo. I think it might be an hvplot issue because the streaming dataframe table seems to display correctly.
Does anyone know of a combination of hvplot and streamz that does work.
streamz looks like it isn’t very active nowadays so are there other alternatives that straightforward to use with hvplot?

versions → streamz = 0.6.4, hvplot 0.8.1, ipywidgets 7.7.2 (since streamz is not compatible with ipywidgets > 8)

Hoxbro · November 22, 2022, 9:10am

The error you are seeing is related to python 3.11. Try to downgrade to python 3.10.

Though even when you downgrade, there could still be problems with streamz. See here.

I have had some success using holoviews directly.

import holoviews as hv
import streamz
import streamz.dataframe
from holoviews.streams import Buffer

hv.extension("bokeh")

hist_source = streamz.dataframe.Random(freq="5ms", interval="100ms")
sdf = (hist_source - 0.5).cumsum()

# Example 1
dmap = hv.DynamicMap(hv.Dataset, streams=[Buffer(sdf.x, length=100)])
hv.operation.histogram(dmap, dimension="x", bins=None)

# Example 2
hv.DynamicMap(hv.Curve, streams=[Buffer(sdf.x, length=100)])

dharhas · November 22, 2022, 9:24pm

Hmmm. Seems like there is more going on. I’m getting a different error on Python=3.10 & 3.9

import hvplot.streamz  # noqa

Traceback.

Unexpected exception formatting exception. Falling back to standard exception
Traceback (most recent call last):
  File "/Users/dharhas/mamba/envs/streaming-conda/lib/python3.9/site-packages/IPython/core/interactiveshell.py", line 3433, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "/var/folders/g3/w0xtq3td3_xcjbb4gxx3j4k00000gn/T/ipykernel_64153/2584773187.py", line 1, in <module>
    import hvplot.streamz  # noqa
  File "/Users/dharhas/mamba/envs/streaming-conda/lib/python3.9/site-packages/hvplot/streamz.py", line 19, in <module>
    patch()
  File "/Users/dharhas/mamba/envs/streaming-conda/lib/python3.9/site-packages/hvplot/streamz.py", line 17, in patch
    post_patch(extension, logo)
  File "/Users/dharhas/mamba/envs/streaming-conda/lib/python3.9/site-packages/hvplot/__init__.py", line 167, in post_patch
    hvplot_extension(extension, logo=logo)
  File "/Users/dharhas/mamba/envs/streaming-conda/lib/python3.9/site-packages/pyviz_comms/__init__.py", line 64, in __new__
    return param.ParameterizedFunction.__new__(cls, *args, **kwargs)
  File "/Users/dharhas/mamba/envs/streaming-conda/lib/python3.9/site-packages/param/parameterized.py", line 3654, in __new__
    return inst.__call__(*args,**params)
  File "/Users/dharhas/mamba/envs/streaming-conda/lib/python3.9/site-packages/hvplot/utilities.py", line 45, in __call__
    super().__call__(*args, **params)
  File "/Users/dharhas/mamba/envs/streaming-conda/lib/python3.9/site-packages/holoviews/ipython/__init__.py", line 149, in __call__
    ip.run_line_magic('config', 'IPCompleter.use_jedi = False')
  File "/Users/dharhas/mamba/envs/streaming-conda/lib/python3.9/site-packages/IPython/core/interactiveshell.py", line 2364, in run_line_magic
    result = fn(*args, **kwargs)
  File "/Users/dharhas/mamba/envs/streaming-conda/lib/python3.9/site-packages/IPython/core/magics/config.py", line 163, in config
    configurables = sorted(set([ c for c in self.shell.configurables
  File "/Users/dharhas/mamba/envs/streaming-conda/lib/python3.9/site-packages/IPython/core/magics/config.py", line 164, in <listcomp>
    if c.__class__.class_traits(config=True)
AttributeError: type object 'CommManager' has no attribute 'class_traits'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/dharhas/mamba/envs/streaming-conda/lib/python3.9/site-packages/IPython/core/interactiveshell.py", line 2052, in showtraceback
    stb = self.InteractiveTB.structured_traceback(
  File "/Users/dharhas/mamba/envs/streaming-conda/lib/python3.9/site-packages/IPython/core/ultratb.py", line 1112, in structured_traceback
    return FormattedTB.structured_traceback(
  File "/Users/dharhas/mamba/envs/streaming-conda/lib/python3.9/site-packages/IPython/core/ultratb.py", line 1006, in structured_traceback
    return VerboseTB.structured_traceback(
  File "/Users/dharhas/mamba/envs/streaming-conda/lib/python3.9/site-packages/IPython/core/ultratb.py", line 859, in structured_traceback
    formatted_exception = self.format_exception_as_a_whole(etype, evalue, etb, number_of_lines_of_context,
  File "/Users/dharhas/mamba/envs/streaming-conda/lib/python3.9/site-packages/IPython/core/ultratb.py", line 812, in format_exception_as_a_whole
    frames.append(self.format_record(r))
  File "/Users/dharhas/mamba/envs/streaming-conda/lib/python3.9/site-packages/IPython/core/ultratb.py", line 730, in format_record
    result += ''.join(_format_traceback_lines(frame_info.lines, Colors, self.has_colors, lvals))
  File "/Users/dharhas/mamba/envs/streaming-conda/lib/python3.9/site-packages/stack_data/utils.py", line 144, in cached_property_wrapper
    value = obj.__dict__[self.func.__name__] = self.func(obj)
  File "/Users/dharhas/mamba/envs/streaming-conda/lib/python3.9/site-packages/stack_data/core.py", line 720, in lines
    pieces = self.included_pieces
  File "/Users/dharhas/mamba/envs/streaming-conda/lib/python3.9/site-packages/stack_data/utils.py", line 144, in cached_property_wrapper
    value = obj.__dict__[self.func.__name__] = self.func(obj)
  File "/Users/dharhas/mamba/envs/streaming-conda/lib/python3.9/site-packages/stack_data/core.py", line 663, in included_pieces
    scope_pieces = self.scope_pieces
  File "/Users/dharhas/mamba/envs/streaming-conda/lib/python3.9/site-packages/stack_data/utils.py", line 144, in cached_property_wrapper
    value = obj.__dict__[self.func.__name__] = self.func(obj)
  File "/Users/dharhas/mamba/envs/streaming-conda/lib/python3.9/site-packages/stack_data/core.py", line 603, in scope_pieces
    for piece in self.source.pieces
  File "/Users/dharhas/mamba/envs/streaming-conda/lib/python3.9/site-packages/stack_data/utils.py", line 144, in cached_property_wrapper
    value = obj.__dict__[self.func.__name__] = self.func(obj)
  File "/Users/dharhas/mamba/envs/streaming-conda/lib/python3.9/site-packages/stack_data/core.py", line 101, in pieces
    return list(self._clean_pieces())
  File "/Users/dharhas/mamba/envs/streaming-conda/lib/python3.9/site-packages/stack_data/core.py", line 125, in _clean_pieces
    raise AssertionError("Pieces mismatches: %s" % mismatches)
AssertionError: Pieces mismatches: [{680, 681}, {696, 695}, {708, 709}, {714, 715}]

Hoxbro · November 22, 2022, 9:26pm

There has been a new release of ipykernel yesterday, which breaks a lot, see here. Downgrade it to 6.17.

The pieces mismatch is nothing to worry about.

dharhas · November 22, 2022, 10:27pm

Ok I have hvplot + streamz working together with:

- python<3.11
- ipykernel<6.18
- ipywidgets<8.0

I’ll open a new discussion once I have a reproducible example with the jankiness I originally wanted to report.

maximlt · November 24, 2022, 12:32pm

Hey @dharhas!

Nice to see you’re trying out the streaming capabilities of hvPlot. I’m not surprised there’s some issue though, I think it’s not the most tested part of our system, but it’s something we could improve with some feedback. And it’s good to know people are using it, it helps up prioritize work!

On the issues that you hit, just to complement what Simon already said:

Support for Python 3.11: it’s coming, we’ve made some changes to Param (available in a dev release) and to HoloViews (not yet released). The ecosystem on which is not yet fully compatible with Python 3.11. Most notably when I tried to run HoloViews’ test suite on Python 3.11 Numba wasn’t yet available.
The latest release of ipykernel broke more projects than us. It was quickly yanked on PyPI but (yet?) removed from conda-forge.
The Pieces mismatches error comes from a bug in IPython. We’ve made a change in Param to work around that bug (not yet released)

RNarayan73 · November 6, 2023, 9:33pm

@maximlt I believe the above mentioned errors around numba and ipykernel have been resolved for 3.11.
Is there an eta for this?

maximlt · November 7, 2023, 8:40am

Hi @RNarayan73, an ETA for what exactly?

RNarayan73 · November 7, 2023, 3:09pm

@maximlt for Python 3.11 support.

maximlt · November 7, 2023, 3:28pm

hvPlot supports 3.11 and 3.12 hvplot · PyPI

RNarayan73 · November 10, 2023, 8:06pm

My bad. The module I’m using uses older versions of holoviews - 1.14.9 and hvplot - 0.7.3 and is therefore stuck at Python 3.10