How To: GUI for Hardware Automation / Scientific Instrument Control

I have been using Holoviews, Bokeh, and Panel to build GUI interfaces for instrument control, and I’ve been really happy with the results. These packages are fun to work with.

I wanted to share a simple example of a fully-functioning app (minus actual hardware communication…) for collecting data from an instrument over time. I’ve used this same basic app to get started with several different instruments, so probably others would also find it useful.

Also, if anyone has any constructive criticism, that would be wonderful.

The plotting part of this is more or less straight from the Holoviews docs on Working with Streaming Data. But here the data is “streaming” from an instrument, and can optionally be saved to a csv file.

To run this example, you need to make two files: fake_instrument.py and timeseries_plot_single_variable.py. (This code, as well as a version showing communication with two different instruments, is also posted on github here.)

fake_instrument.py

"""
fake_instrument.py --

A mock instrument that returns a random value when queried.
"""
import numpy as np


class FakeInstrument(object):
    def __init__(self, offset=0.0):
        self.offset = offset

    def set_offset(self, value):
        self.offset = value

    def read_data(self):
        return np.random.random() + self.offset

Obviously, replace fake_instrument.py with whatever module you use to communicate with your instrument.

The main script is here:
timeseries_plot_single_variable.py

# Run with 
# bokeh serve --show timeseries_plot_single_variable.py
import pandas as pd
import panel as pn
import holoviews as hv
from holoviews.streams import Buffer
from bokeh.models import Button, Slider, Spinner
import time
import asyncio
from fake_instrument import FakeInstrument  # replace this with whatever you're trying to communicate with

#
# Initialize your instrument
#
instrument = FakeInstrument()

#
# Make a dataframe to hold the buffered data. You could get by without making a Pandas Dataframe, but it does a good
# job of writing to a csv file.
#
def make_df(time_sec=0.0, temperature_degC=0.0):
    return pd.DataFrame({'Time (s)': time_sec, 'Temperature (°C)': temperature_degC}, index=[0])

#
# Initialize the buffer and the plot.
# Holoviews handles all of the plotting and makes some guesses about what columns to plot.
#
example_df = pd.DataFrame(columns=make_df().columns)
buffer = Buffer(example_df, length=1000, index=False)
plot = hv.DynamicMap(hv.Curve, streams=[buffer]).opts(padding=0.1, width=600, xlim=(0, None))

#
# Define any other GUI components
#
LABEL_START = 'Start'
LABEL_STOP = 'Stop'
LABEL_CSV_START = "Save to csv"
LABEL_CSV_STOP = "Stop save"
CSV_FILENAME = 'tmp.csv'

button_startstop = Button(label=LABEL_START)
button_csv = Button(label=LABEL_CSV_START)
offset = Slider(title='Offset', start=-10.0, end=10.0, value=0.0, step=0.1)
interval = Spinner(title="Interval (sec)", value=0.1, step=0.01)


#
# Define behavior
#
acquisition_task = None
save_to_csv = False


#
# Here we're using a coroutine to handle getting the data from the instrument and plotting it without blocking
# the GUI. In my experience, this works fine if you are only trying to get data from your instrument once every
# ~50 ms or so. The buffering and plotting take on the order of 10-20 ms. If you need to communicate with your
# instrument on millisecond timescales, then you'll want a separate thread and maybe even separate hardware. And you
# won't want to update the plot with every single data point.
#
async def acquire_data(interval_sec=0.1):
    global save_to_csv
    t0 = time.time()
    while True:
        instrument.set_offset(offset.value)
        time_elapsed = time.time() - t0
        value = instrument.read_data()
        b = make_df(time_elapsed, value)
        buffer.send(b)

        if save_to_csv:
            b.to_csv(CSV_FILENAME, header=False, index=False, mode='a')

        time_spent_buffering = time.time() - t0 - time_elapsed
        if interval_sec > time_spent_buffering:
            await asyncio.sleep(interval_sec - time_spent_buffering)


def toggle_csv():
    global save_to_csv
    if button_csv.label == LABEL_CSV_START:
        button_csv.label = LABEL_CSV_STOP
        example_df.to_csv(CSV_FILENAME, index=False)  # example_df is empty, so this just writes the header
        save_to_csv = True
    else:
        save_to_csv = False
        button_csv.label = LABEL_CSV_START


def start_stop():
    global acquisition_task, save_to_csv
    if button_startstop.label == LABEL_START:
        button_startstop.label = LABEL_STOP
        buffer.clear()
        acquisition_task = asyncio.get_running_loop().create_task(acquire_data(interval_sec=interval.value))
    else:
        acquisition_task.cancel()
        button_startstop.label = LABEL_START
        if save_to_csv:
            toggle_csv()


button_startstop.on_click(start_stop)
button_csv.on_click(toggle_csv)

hv.extension('bokeh')
hv.renderer('bokeh').theme = 'caliber'
controls = pn.WidgetBox('# Controls',
                        interval,
                        button_startstop,
                        button_csv,
                        offset)

pn.Row(plot, controls).servable()

Run the GUI from a command prompt:

> bokeh serve --show timeseries_plot_single_variable.py 

It should look like this gif:

1 Like

@kasey

This a really nice example. I currently try to learn about Streaming and would like to provide a few examples at awesome-panel.org. Would it be ok to create an example close to the one you have provided?

I think the Panel Gallery could also use nice streaming examples. If you have time feel free to do a PR.

Thanks for sharing again. By the way we have a new show case category where this would fit nicely. So I have moved it to that category. Let me know if it is not OK.

@Marc

Thanks! Yes, please move it to wherever is most appropriate and feel free to use it as a base to make an example on awesome-panel.org. I hadn’t seen that site before; looks useful!

I would need more guidance on what you’re looking for in terms of a PR for the Panel Gallery. You mean something to file under “Simple Apps” on https://panel.holoviz.org/gallery/index.html?

1 Like

Yes simple apps. Or maybe a new category called Streaming Apps

What you would do is to add a your notebook example to the relevant folder at https://github.com/holoviz/panel/tree/master/examples/gallery and submit the PR.

The notebook should be cleaned out when submitting the PR. I’ve forgot to do this several times :slight_smile:

1 Like

Hey, this is just what I needed :slight_smile:
While trying to adapt this for my needs, I realized that I don’t quite understand how asyncio works I think.
How would I have to adapt the example, if the experiment were to terminate on it’s own (instead of canceling it with the button)?.
Right now, in that case, to get the experiment to start again, one has to press the button 2 times. Once to return to the initial state, and then to start it.
If I add (trivially simple) code to the start_stop() function, then the button resets early (and I can start several at once, which I don’t want).
I guess, what I want is that the button resets only after the experiment finishes. But If I add code to the end of the acquire_data() function, then for some reason the button doesn’t work (I guess this time the problem is related to the panel events)
Could you maybe point me to some examples where asyncio interacts with panel?
Thanks

I am quite interested in what you come up with!
When I struggled with the problem, I used crossbar.io
to allow sensor code residing on different hardware communicating over the web
and providing a command interface.

For many applications this would be overkill: I am looking forward to more
info in this thread!

Nevermind, I figured it out.
A button.clicks +=1 does the job at the right place.
My problem was that I imported Buttons both from panel.widgets.Button as well as bokeh.models.Button.
The latter is different from the former, and does not work.

1 Like

I’ve continued to fool around with this template, and really like it, thank you.
My project has grown, and I suddenly realized that I don’t understand some very basic things regarding the
DynamicMap.
The way it is constructed here, is different from the way it is used in the examples (for example here or here)
My very simple problem is the following: Using the above example, I want to save a dataframe of more than 2 columns. Then I want to visualize the data using several plots, each time plotting different columns.
The above examples never explicitly defines which columns are key dimensions, or which columns are plotted. I think it just defaults to column 1 and 2.
How do I have to modify the example to achieve this? Even better, are there any particular examples where this is explained in a simple, step by step manner. Most examples i looked at just jump into the deep end it seems.

1 Like

Hi @Feargus

Please create a new post. You can link to this one. But you need to help the community by making your questions very specific :+1: And if possible please include a minimum, reproducible example. This post is already very big and complicated to relate to.

Thanks.