Panel - Starting a Stream of Data

Motivated by a few posts by @fishing

I would like to provide some examples on how to start streams in Panel.

  1. Simple “Streaming” using meta_refresh
  2. Streaming using pn.state.add_periodic_callback
  3. Streaming from a separate thread.
1 Like

Automatic Reload using meta_refresh

This is not really streaming but a quick and dirty way to have the browser reload the page periodically.

You can achieve this using the Fast Templates meta_refresh argument. See FastListTemplate.

For example if you set the meta_refresh argument to '30' then the app will refresh automatically every 30 seconds.

This will work better the less updates you have, the less interactive your apps is, the less users you have and the less compute costly your “refresh” is.

You can serve the code below via panel serve name_of_script.py.

import datetime

import panel as pn

pn.extension(sizing_mode="stretch_both")

ACCENT_BASE_COLOR = "#DAA520"

text = f"""# Last Update: {datetime.datetime.utcnow().strftime('%H:%M:%S')}"""
streaming_component = pn.pane.Markdown(text)

panel = pn.Row(streaming_component, height=90)

template = pn.template.FastListTemplate(
    site="Awesome Panel",
    title="Streaming using meta_refresh",
    logo="https://panel.holoviz.org/_static/logo_stacked.png",
    header_background=ACCENT_BASE_COLOR,
    accent_base_color=ACCENT_BASE_COLOR,
    meta_refresh="3",
    main=[panel],
).servable()

Streaming using a PeriodicCallback

If you need better performance or user experience than meta_refresh can provide you can look into adding a PeriodicCallback using pn.state.add_periodic_callback.

There is also a nice gallery example here which is running live here.

The updated example looks like

and the code is

import datetime

import panel as pn

pn.extension(sizing_mode="stretch_both")

ACCENT_BASE_COLOR = "#DAA520"

streaming_component = pn.pane.Markdown()

panel = pn.Row(streaming_component, height=90)

def update():
    text = f"""# Last Update: {datetime.datetime.utcnow().strftime('%H:%M:%S.%f')}"""
    print(text)
    streaming_component.object = text

pn.state.add_periodic_callback(callback=update, period=500)

template = pn.template.FastListTemplate(
    site="Awesome Panel",
    title="Streaming using meta_refresh",
    logo="https://panel.holoviz.org/_static/logo_stacked.png",
    header_background=ACCENT_BASE_COLOR,
    accent_base_color=ACCENT_BASE_COLOR,
    main=[panel],
).servable()

Sharing the PeriodicCallback across users and session.

UPDATE: I’ve later learned that the callback added using pn.state.add_periodic_callback is not global and will stop when the browser tab is closed. A PR is on the way to add this functionality. #2661. In the mean time you can use threading instead as shown in the posts below. Alternatively you can use tornados add_timeout, call_at or call_later


If you have many users it might not be feasible to run it individually for all your users. Instead you should run it once for your entire application.

You can use pn.state.cache (a globally shared dictionary) to share the PeriodicCallback and the streaming_component across users.

You can serve the below code via panel serve name_of_script.py (dont use --autoreload flag).

import datetime

import panel as pn

pn.extension(sizing_mode="stretch_both")

ACCENT_BASE_COLOR = "#DAA520"

if not "streaming_component" in pn.state.cache:
    streaming_component = pn.pane.Markdown("hello")
    def update():
        text = f"""# Last Update: {datetime.datetime.utcnow().strftime('%H:%M:%S.%f')}"""
        print(text)
        streaming_component.object = text
    pn.state.cache["streaming_component"]=streaming_component
    periodic_callback = pn.state.add_periodic_callback(callback=update, period=500)

streaming_component=pn.state.cache["streaming_component"]


panel = pn.Row(streaming_component, height=90)

template = pn.template.FastListTemplate(
    site="Awesome Panel",
    title="add_periodic_callback",
    logo="https://panel.holoviz.org/_static/logo_stacked.png",
    header_background=ACCENT_BASE_COLOR,
    accent_base_color=ACCENT_BASE_COLOR,
    main=[panel],
).servable()
1 Like

Streaming from a Separate Thread

If your periodic callback is long running it might freeze your application. Then its better to look into Async and Concurrency.

Here we will provide a small example using threading.

Note how the application keeps responsive even though a python script is continuously running in separate thread.

You can serve the app via panel serve name_of_script.py.

import datetime
import threading
import panel as pn
import time

pn.extension(sizing_mode="stretch_both")

ACCENT_BASE_COLOR = "#DAA520"



if not "streaming_component" in pn.state.cache:
    streaming_component = pn.pane.Markdown("hello")
    pn.state.cache["streaming_component"]=streaming_component

    def update():
        while True:
            text = f"""# Last Update: {datetime.datetime.utcnow().strftime('%H:%M:%S.%f')}"""
            print(text)
            streaming_component.object = text
            time.sleep(0.5)

    thread=threading.Thread(target=update)
    thread.daemon = True
    thread.start()

streaming_component=pn.state.cache["streaming_component"]
slider = pn.widgets.IntSlider(value=0, start=0, end=10, margin=25)
panel = pn.Column(streaming_component, slider, slider.param.value, height=225)

template = pn.template.FastListTemplate(
    site="Awesome Panel",
    title="add_periodic_callback",
    logo="https://panel.holoviz.org/_static/logo_stacked.png",
    header_background=ACCENT_BASE_COLOR,
    accent_base_color=ACCENT_BASE_COLOR,
    main=[panel],
).servable()
3 Likes

Personalization

If you want to personalize what is shown to the user, for example based on inputs from sliders etc, that is also possible.

To enable personalization you can update some shared parameter value from another thread. Then each user session can bind to or depend on the updates to the shared parameter value.

import datetime
import threading
import panel as pn
import time
import param

pn.extension(sizing_mode="stretch_both")

ACCENT_BASE_COLOR = "#DAA520"

class App(param.Parameterized):
    utcnow = param.Date()

    def update_utcnow(self):
        self.utcnow = datetime.datetime.utcnow()


if not "app" in pn.state.cache:
    app = App()
    pn.state.cache["app"]=app
    def update():
        while True:
            app.utcnow = datetime.datetime.utcnow()
            print("update app.utcnow: ", app.utcnow)
            time.sleep(0.5)

    thread=threading.Thread(target=update)
    thread.daemon = True
    thread.start()

app = pn.state.cache["app"]
streaming_component = pn.pane.Markdown("hello")
slider = pn.widgets.IntSlider(value=0, start=0, end=10, margin=25)

@param.depends(utcnow=app.param.utcnow, slider_value=slider.param.value, watch=True)
def update_streaming_component(utcnow, slider_value):
    print("update streaming_component", streaming_component.name)
    utcnow += datetime.timedelta(seconds=slider_value*60*60)
    streaming_component.object = f"""# Last Update: {utcnow.strftime('%H:%M:%S.%f')}"""

panel = pn.Column(streaming_component, slider, slider.param.value, height=225)

template = pn.template.FastListTemplate(
    site="Awesome Panel",
    title="Streaming from Another Thread",
    logo="https://panel.holoviz.org/_static/logo_stacked.png",
    header_background=ACCENT_BASE_COLOR,
    accent_base_color=ACCENT_BASE_COLOR,
    main=[panel],
).servable()

Panel and the HoloViz ecosystem provides a lot of features and tools for streaming because there are so many different use cases and requirements. For more check out.