Best practice for displaying high resolution camera images captured on server

Hi,

I’m trying to expose some camera functionallity on a panel dashboard.
At the moment, a PC has an industrial camera connected and serves panel. On that same PC I open up the dashboard in a browser, click a “Snapshot” button, which captures a frame and updates a param.Parameter. I have this code that depends on that param:


class Camera(param.Parameterized):
    latest = param.Parameter(default=(np.ones(shape=(50, 50), dtype=np.uint16) * 2000)) # Just some initial content


class InstrumentState(param.Parameterized):
    camera = Camera()

........

@pn.depends(image=instrumentState.camera.param.latest, watch=True)
def img0(image):
    return hv.Image(image, bounds=bounds).opts(*options)

layout = pn.Row(
    pn.Column(
        hv.DynamicMap(img0)
    )
)

I felt like it took ages to take a snapshot, so I made some very primitive profiling and found that it took me 1.3s to do:

instrumentState.camera.latest = nparray_reshaped # A 3208x2200 numpy 8bit array.

I can see in the developer console that there is a 7.1MB binary sent on the websocket when .latest is updated. It should take no time at all to send 7.1 from one application to a browser on the same PC, right?

Maybe I’m not doing this the right way at all. If anyone has previous experience of updating large images in panel i would be very interested to hear what kind of results you have achieved.

I will probably setup a minimal reproducable example tomorrow.

1 Like

Hi @Wiggan

One thing I notice is that you have watch=True on img0. This will make the Dynamic map update twice, so you can safely remove it.

Hi @Wiggan

I also tried building a minimum, reproducible example and I also experience that the it takes a very long time to transfer the high resolution data. The server also raises an exception for me before I reach the dimensions of your example.

I have raised it as a bug on Github here Transfer of high resolution image takes way too long and even raises WebSocketClosedError · Issue #3874 · holoviz/panel (github.com).

My hypothesis is that the issue is that the data is transfered as .json which is not efficient for this use case. But I don’t know.

import param
import panel as pn
import numpy as np
import holoviews as hv
import time

pn.extension()

bounds=(-1,-1,1,1)

X=2200
Y=3208

class Camera(param.Parameterized):
    
    xresolution = param.Integer(100, bounds=(100, 2200), step=100)
    yresolution = param.Integer(100, bounds=(100, 3200), step=100)
    
    take = param.Event()
    count = param.Integer()
    duration = param.Number()

    value = param.Parameter() # Just some initial content


    @pn.depends("take", watch=True, on_init=True)
    def _take_a_picture(self):

        xls = np.linspace(0, 10, self.xresolution)
        yls = np.linspace(0, 10, self.yresolution)
        xx, yy = np.meshgrid(yls, xls)
        
        start = time.time()
        self.value = np.sin(xx+self.count)*np.cos(yy)
        end = time.time()
        
        self.count += 1
        self.duration = end-start

camera = Camera()

@pn.depends(image=camera.param.value)
def img0(image):
    return hv.Image(image,bounds=bounds)
layout = pn.Column(pn.Param(camera, parameters=["xresolution", "yresolution", "take", "duration"]), hv.DynamicMap(img0), ).servable()
2022-09-22 15:51:15,374 Failed sending message as connection was closed
2022-09-22 15:51:15,374 Task exception was never retrieved
future: <Task finished name='Task-2604' coro=<WebSocketProtocol13.write_message.<locals>.wrapper() done, defined at /opt/conda/lib/python3.9/site-packages/tornado/websocket.py:1100> exception=WebSocketClosedError()>
Traceback (most recent call last):
  File "/opt/conda/lib/python3.9/site-packages/tornado/websocket.py", line 1102, in wrapper
    await fut
tornado.iostream.StreamClosedError: Stream is closed

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/conda/lib/python3.9/site-packages/tornado/websocket.py", line 1104, in wrapper
    raise WebSocketClosedError()
tornado.websocket.WebSocketClosedError
2022-09-22 15:51:15,375 WebSocket connection closed: code=None, reason=None

Hi @Wiggan. We should have a new HoloViews out very soon which speeds this up considerably.

3 Likes

Hi, and thank you so much for fast response and even making a reprex! Wow!
I tried setting watch=False, but don’t see an impact on time spent in the assignment. It does however still work, so I will keep it that way.

1 Like

Great! I’ll keep an eye on the PR and upcoming release.
Meanwhile, is the method for live updating images above reasonable, or would you have solved the problem with some other method? I was thinking about WebRTC, but that would probably not be as well integrated with zoom tools and such. That would take me much more time to implement as well.