I am using panel/holoviews/hvplot for dashboards for last two years in my company but it’s loading/rendering time in browser is always slow and it took to 2 to 3 minutes to load depending on the type of holoviews i used .
I tried every options explained here and in bokeh disclourse but it always took 2 to 3 minutes as it’s understandable that bokeh rendering always tooks time.
As the same time i am also expert in tableau and it’s dashboard always loaded in the quick time and it’s understandable it’s a paid and industry leader in it.
Always in comparison (python ETL + panel app dashboard) vs (python ETL + Tableau dashboard)
To add to @Marc’s answer, bokeh rendering happens entirely on the browser side so it has zero effect on the time it takes to render the initial page. I remember going over this with you a long time ago but without some example it was always impossible to figure out what exactly was taking up so much time. What I might recommend is that you install panel 0.13.0dev6 and pyinstrument and then launch your application with the following options panel serve your_app.py --admin --profiler pyinstrument and then navigate to http://localhost:/admin and specifically the Launch Profiler tab and then provide us with that output.
It is in pandas dataframe max memory in 50 MB when processed.
Notebook contains function that return holoviews/hvplot/Matplot graphs for every tabs
Kindly review the script and suggest
Why template and tabs take time to load on other LAN nodes browser
But when loaded and when clicked on any tab menu items its bokeh graphs displays quickly
What I meant, you can fabricate fake data, for example numpy.random.random(matrix.shape) so the people who have free time to check this problem can help you. It is a nice gesture to provide a minimun reproducible example so people can check it fast. The last thing, there is several exec commands with some missing py file which I do not know if that command is doing something.
In short, people will be better able to provide help if you provide code that they can easily understand and use to reproduce the problem.
I second @nghenzi . Please take the time to make a minimum, reproducible example. Without it, it can take a very long time to understand the issue or even make it impossible. Thanks.
I was running your example (with panel 0.13.0a5.post19+ga2e4c848 and bokeh 2.4) and effectively it has some problem, but I could not see it. Sometimes the bokeh messages hass this problem of the bokeh referrers
and the app keeps increasing the memory each time 1 new session is launched.
It begins with rendering time in 20 seconds and after 5 or 6 sessions it renders now in 110 seconds. Each time you run it, it increases 10 seconds. The memory begins in 600 Mb and it continues increasing till 1300 Mb now.
The app is really big and messy in order to debug, beyond my time capabilities. The only advice I have is related to avoid nested layout as much as you can. In some case you have inside a Tab something like pn.Column(pn.FlexBox(pn.indicators.Trend(w=200, h=200), which is completely unnecesary. In any case, the problem is not related to this, but with the memory leak. If you delete the column and the flexbox you reduce the rendering time by 1 or 2 seconds.
I copied the code below with several things deleted, but the improve is marginal as compared with the increase produced in each session. The data is the csv which was shared previously.
you can try to save the csv data in pn.state.cache ir order to avoid to run it . There is several problems where you can improve, but pandas.read_csv takes 1 or 2 seconds more each time you call it
if 'data' not in pn.state.cache.keys():
cus_compl = pd.read_csv('cust_compl.csv')
pn.state.cache['data'] = cus_compl.copy()
else:
cus_compl = pn.state.cache['data']
it improves the rendering time. After 10 sessions, it keeps in 20 - 30 seconds of rendering time. However you need to improve your nwd_trends() function. I do not understand what it does, it performs a lot of pandas operations, one class and several panel objects altogether, which it makes difficult to see where the bottleneck is. I would try to divide the pandas operations and the (panel, holoviews, bokeh) operations and watch the profiler in order to define exactly where the problem is. I am inclined to believe that there is some problem with pandas, more than with bokeh, but it is hard to see it with this code structure.
Before nothing, I love the profiler and the admin site. I spent like one hour playing with it…
If you check the profiler, there is something happening with the legends outside of the plot I think. You have a lot of plots and stuff, so it is difficult to improve your code. You need to review it carefully and decrease the complexity. I think the difference you see with tableau is in that case you are an expert, then the code is really polished.
I stopped of using the code from @khannaum and I tried the following code
You can check that begin rendering with 1-2 seconds, and after several tries it takes 14 seconds. This is in big app generate the 100 seconds that @khannaum gets it.
In general avoid nesting layouts. The Bokeh layout engine is not capable of handling this well and will slow down the app. Use one of the built in templates or build your own.
In Bokeh 3.0 the bokeh layout engine should be fixed/ removed which should solve a lot of layout performance issues.
From my code kindly identify nesting layouts so that i make it better and optimal
Previous year I had similar issues with another dashboard that uses data in GB
then i had reverted to vaex dataframe groupby instead of pandas.
and there is significant performances in reducing rendering time from 4-5 minutes to 2 minutes
In next few days i will share the same example with vaex dataframe groupby and we discuss on it.
but the problem is that we have to compete with same tableau dashboards which have no such issues with the same data
Hello! Have you used pyinstrument here or is it another package? Can I get more information/documentation on this as I need to do some diagnosis on my dashboard please?
You can attach it here I think.
