Panel meets D3.js and WebSocket

Showcase
#1

Finally figured out enough of the pieces. The code is in two files:

The producer is a python script that serves random data.
The notebook has a panel HTML pane that displays a canvas.
Notebook code opens a websocket to the server.
As data comes in, d3.js is used to draw it in the canvas.
Nothing fancy, just a proof of concept.

Comments would be very, very welcome!
Also, if anybody notices why pulling the commands into a class object
does not work, I’d love to know!

WebSocketPanelD3js

3 Likes
#2

@ea42gh

This is a great example and very nice description in the video.

This is close to perfect :slight_smile: It would be perfect if there was not noice/ echo on the recording. Do you know where that comes from?

#3

Thank you @Marc.

I had not taken the time to set up a better environment:
normally I have the microphone in a box with acoustic foam
and only one side open.

I was surprised at the really poor quality of the recording though:
a very high level of background noise…
I had taken about 4 min of recording, edited it down,
and processed the heck out of the audio (noise removal,
compression, deessing, …)

The file was bigger than discourse would accept,
so I ran it through a transcoder to cut it down…

1 Like
#4

Hi @ea42gh

  • What tool do you use for the recording? I use snagit and it does not highlight any areas for me like in your recording. That is a nice feature?

  • Why do you need d3.js? And why do you setup seperate web socket connection instead of using for example the periodic callback supported by Panel? (Just curious).

#5

Hi @Marc,
re tools: Covid forced me to do video productions…
(look at the youtube channel I ended up with!)

Currently, I record videos using OBS,
do some editing with Camtasia (the snagit folks),
and finish with DaVinci Resolve…
Let me know if you want to get into something along those lines…

  • d3.js : I have a lot of custom displays I want to use
  • websocket: I need to be able to get data from various sensors
    and other data sources that reside somewhere else on the net.
    I do not control them, so I have to adapt to whatever service they offer…
1 Like
#6

The Python version WebSocket_Py.ipynb in the same gitlab repository is now working as well.

#7

Thanks @ea42gh

Regarding the sensors and websockets. I don’t have any experience in that area with Panel, so I’m just wondering what the best practice would be?

Panel is running on a web socket, it supports executing javascript and it supports asynchronous. So my question is really if you need separate web sockets or if Panel already could do the job???

#8

@Marc,
I never really explored any of this before.

Sockets are connections to some address and port.
Having a socket running on some connection
does not mean I can use it to monitor a different connection
as far as I can tell? The documentation I find is woefully confusing…

1 Like
#9

Love this. I certainly would like to provide lower-level access to the main Websocket that Bokeh itself is running on. I’ll try to play with setting up a custom Bokeh message, which can then be leveraged by some Panel API, which acts like a low level websocket. An issue to remind me to explore this would be appreciated.

#10

Thank you @philippjfr
An issue where, in holoviews?

I am currently reading up on crossbar.io
As far as I can tell currently, starting crossbar running
sets it up as a router. Servers/clients can register
on the same port, the router handles the traffic.

I am hoping to set up a little toy:

  1. a sensor
  2. a sensor and an actuator
  3. a Panel Gui with a control to turn the actuator on and off…

Apologies to the folks that actually know this topic:
I am new to this, sot the lingo still escapes me!
Any comments, hints,help would be very welcome!

#11

Let’s go with Panel for the issue for now.

#12

Figured out quite a bit more using crossbar.io:
I have two python processes generating measurement data,
a router monitor that logs websocket measurements to the console
A notebook that subscribes to the sensors.

  • crossbar start starts the system running (open the console at localhost:8060)
  • running the notebook will create a dashboard plotting the sensor outputs.
  • a button on the dashboard propagates a command back to the sensor_1 code, causing the trend of the measurements to reverse.

The code is in the files
CROSSBAR subdirectory https://gitlab.com/ea42gh/awesomepanelplayground/-/tree/master/WebSocket/CROSSBAR/sensors
(you will need all the files in the subtree, and a working crossbar.io installation)

StatsPlotCard PR