I would like to use the on_server_loaded-method for a panel app. My understanding is that I need to use the directory format and specify the lifecycle hooks in the app_hooks.py file as described in the bokeh documentation… This works fine if I use the panel serve command, however I would like to serve my app inside a python file using the panel.serve method, as I need to perform some initialization first… And this does not trigger the hooks what I can see, should this be working?
If so, I can definitely create a small example. If not, do I have any other options?
If you need to perform some initialization before starting the server I believe you can just run that code before pn.serve. If you need to perform some initialization before each session starts I believe you can take a look at pn.state.on_session_created.
Hope that helps. Otherwise try to add some more specific information on what you are trying to achieve.
I am well aware that the architecture of this system is not the best but the issue is that I would like to download a relatively large data set from an FTP server on startup, it takes 5-10min, and I would like to have this data available for the users on the first session request. I host the app on azure as a container so when running the code before pn.serve I needed to increase the start-up time out limit, and it cannot be set to more than 15min. Because of this I wanted to perform the download after the app is loaded, as I am able to with the lifecycle hooks…(at least I assume that this is a solution)
I am also planning to have the app restart once a day to download any updated data and as a temporary work around for some memory leakage issues…
I assume one workaround could be to use the command line argument panel serve, from within a python script, to launch the app in directory format?..
I use lifecyle hooks in my panel and/or native bokeh server apps, but only use the command-line mechanism to run the server following the directory structure and naming requirements outlined in the bokeh documentation.
With that caveat, I think the following example might work using panel.serve. I did not arrive at this solution from any documentation, but rather by poking around in the panel.state information and trial-and-error experimentation with the syntax to register the hooks.
My understanding of the onload-callback, though, is that it is triggered on session start and not on server start (ref. Deploy and Export — Panel 0.11.3 documentation). However, I while working on this I also realized that I was chasing the wrong solution with the lifecycle hooks. I needed to have the server returning something while I was downloading the data, but the on_server_loaded-method is run before the server is available to accept requests, so I will need to find some other solutions (which includes not performed the full data download on server start…)
However experimentation makes it seem like the onload method behaves like the on_server_loaded lifecycle hook in the core bokeh solution.
With the trivial example included above in the thread, the sequence of printouts confirms this and upon a user revisiting the URL from a different browser shows the distinction between when the onload method is called and when on_session_created hook is executed.
Just mentioning this for future reference in case someone needs to use these hooks but has constraints on either the command-line approach and/or the bokeh server directory format.
I don’t use Azure and don’t have a complete feel for your actual use case, so don’t have any helpful concrete additional recommendations. However, I will say that I use Redis in memory stores to communicate large data files for analysis to my apps hosted on a platform-as-a-service (Heroku).
If you need to have the server communicating something, perhaps the bokeh/panel server keep-alive pings in combination with some threading (or bokeh periodic callbacks or whatever) to handle the time-intensive downloads could work.
Man this is really frustrating. I spent a ton of time re-factoring code to try to make use of pn.state.onload() to then realize it doesn’t work as expected when using pn.serve(). The docs are really unclear about the differences of the methods of serving an app. There’s a one liner about “Launching from the CLI is usually better…” but how is that helpful if there’s no explanation of why? As i’ve complained about many times before, i can’t figure out all the magic, so its very hard for me to get beyond the functionality of the examples.
So, in my case i think i want to launch the app with pn.serve() since i like the functionality of using a function that returns a template and lets me instantiate a new instance of my app class on each re-load. I’m using this as a locally hosted app for a single user. Is there a better way to defer some long running tasks until after the server has started? Or do i need to learn how to use the command line panel serve style of launching?
Sorry to hear; can you provide a minimal example of it not working with pn.serve?
I don’t use pn.serve much, but my understanding is that to use panel serve CLI, you simply need an object with servable(). And if you want to use a template, you can do pn.template.FastListTemplate(main=[your_pn_serve_obj]) or pn.extension(design="fast") with servable()
I’ve just removed the pandas and plotting and added some logging. This in the pn.serve() config, but switching it to .servable() and using the command line launch behaves very differently. with pn.serve() it does not seem to defer the loading as expected.
import logging
import time
import panel as pn
import param
_LOGGER = logging.getLogger(__name__)
logging.basicConfig(
format="%(asctime)s.%(msecs)03d %(levelname)s {%(module)s} [%(funcName)s] %(message)s",
datefmt="%Y-%m-%d %H:%M:%S",
level=logging.DEBUG,
)
pn.extension(sizing_mode="stretch_width", template="bootstrap", theme="dark")
class AppState(param.Parameterized):
data = param.Dict()
def update(self):
_LOGGER.debug("Updating data")
time.sleep(2)
self.data = {"x": [1, 2, 3, 4], "y": [1, 3, 2, 4]}
_LOGGER.debug("Data updated")
def loading_indicator(label):
return pn.indicators.LoadingSpinner(value=True, name=label, size=25, align="center")
def short_running_task():
return "# I'm shown on load"
def table(data):
if data is None:
return loading_indicator("Loading data")
return pn.pane.JSON(data)
def plot(data):
_LOGGER.debug("Plotting start")
if data is None:
_LOGGER.debug("Plotting yield wait for data")
yield loading_indicator("Waiting for data")
return
_LOGGER.debug("Plotting yield loading indicator")
yield loading_indicator("Transforming data")
time.sleep(2) # Some long running transformation
_LOGGER.debug("Plotting done")
yield "# plot"
state = AppState()
pn.state.onload(state.update)
_LOGGER.debug("serving")
pn.serve(
pn.Column(
short_running_task,
pn.bind(table, data=state.param.data),
pn.bind(plot, data=state.param.data),
)
)
Below is your example using a function to work correctly with the loading indicators.
You can start this with both python3 file.py and python3 -m panel serve file.py and it will work in both cases showing the loading indictors for the 2 second time.sleep.
I still have questions and am concerned about the state of the application, if I look at the /admin it shows multiple sessions being created when using pn.serve–I believe it is one when it first launches and one when I go to it in the browser. I’m still trying to learn this, because I am trying to convert a large application that has been using the cli -m panel serve to use pn.serve so we can create another application endpoint that opens in a new window.
import logging
import time
import panel as pn
import param
def get_template():
_LOGGER = logging.getLogger(__name__)
logging.basicConfig(
format="%(asctime)s.%(msecs)03d %(levelname)s {%(module)s} [%(funcName)s] %(message)s",
datefmt="%Y-%m-%d %H:%M:%S",
level=logging.DEBUG,
)
#pn.extension(sizing_mode="stretch_width", template="bootstrap", theme="dark")
class AppState(param.Parameterized):
data = param.Dict()
def update(self):
_LOGGER.debug("Updating data")
time.sleep(2)
self.data = {"x": [1, 2, 3, 4], "y": [1, 3, 2, 4]}
_LOGGER.debug("Data updated")
def loading_indicator(label):
return pn.indicators.LoadingSpinner(value=True, name=label, size=25, align="center")
def short_running_task():
return "# I'm shown on load"
def table(data):
if data is None:
return loading_indicator("Loading data")
return pn.pane.JSON(data)
def plot(data):
print("start... plot")
_LOGGER.debug("Plotting start")
if data is None:
_LOGGER.debug("Plotting yield wait for data")
yield loading_indicator("Waiting for data")
print("wait... plot")
return
_LOGGER.debug("Plotting yield loading indicator")
yield loading_indicator("Transforming data")
time.sleep(2) # Some long running transformation
_LOGGER.debug("Plotting done")
yield "# plot"
print("end... plot")
state = AppState()
pn.state.onload(state.update)
_LOGGER.debug("serving")
print("start layout...")
layout = pn.Column(
short_running_task,
pn.bind(table, data=state.param.data),
pn.bind(plot, data=state.param.data),
)
print("end layout.")
template = pn.template.BootstrapTemplate(title='Bootstrap Template', theme='dark')
template.main.append(layout)
template.servable()
return template
if __name__ == "__main__":
pn.serve(
get_template,
admin=True,
dev=True
)
else:
get_template()
