Hi,
It’s very interesting to hear about another experience, thanks! (Tbh, just hearing that other than the memory leak, it is feasible to host these apps on Heroku in practice is a valuable information me. I did not dare to try that seeing my initial memory consumption, so right now I’m on a full blown ubuntu AWS EC2 server, but in case this issue gets resolved, I will try Heroku too.)
Relating to your question: I assume we talk about the same thing, but not sure. I did not dive into the Bokeh server code yet, so so I don’t know anything about sessions being created or not. All I do is start my script with panel serve, refresh the localhost:5006/appname url in a browser multiple times and then I can see that on every browser reload the memory consumption keeps growing. Most probably a new session is created and that’s where something goes pear-shaped.
May I ask what kind of elements and application logic do you use?
For example I use in my code:
- Holoviews Points() and through
holoviews.operation.datashaderhd.datashadeandhd.aggregateto display points; - Holoviews Image() to display Tiff files loaded by rioxarray, the images being put through
hd.regrid(); - these elements being overlaid on a tilemap in a
combined = (tilemap * image * points)way; - while all of these are controlled by
pn.widgets.widgets used either in@pn.depends(widget_value = widget.param.value)decorated functions or throughsome_element.apply(function_name, widget_value = widget)way; - the backend for everything is Bokeh.
Maybe if we would have a few summaries like these (assuming if we already know of 3 cases like this, there could be other people affected who could chip in), we could see some common patterns that could help us pinpoint the specific area.
My experience so far is not necessarily helpful in this, as there is only one memory leak I successfully eliminated by commenting out the hooks= option, but assuming that feature is rarely used and reaches into the internal workings of the Bokeh backend itself, I’m not surprised it had issues. And that still left me with some other leak, most probably rooted in a more common functionality. I have a hunch that it could be related to either the Panel widgets/panes or to the underlying Params functionality (that I don’t use directly anywhere) because enabling-disabling areas that are almost entirely built out of Panel elements make it appear-disappear. But I’ve had many false hunches during this bughunt, and seeing how the logic of Holoviews is that “the developer just creates a recipe that will be implemented if anything request that element”, it’s pretty hard be sure in whether the leak is in the part you actually commented out, or if it is still in the live code it’s just not getting executed as nothing made that code run anymore… Anyways, that’s why I think it would be interesting to pool our usecases and look for common elements. Like if you are using Panel and Params at all, those can be the culprits, but if you don’t then either we have multiple similar issues, or it may worth to look into some common elements…
And also an update, just as an experiment I’ve kept my script running with automatic reload and in about 2h it reached 48GB memory consumption on my local machine, so yeah, it does not seem to be stopping. And btw, previously I’ve tried spreading some gc.collect() all over my code and even manually del-ing every variable, and of course, those did not do anything, as I didn’t expect them, but had to try.