How to scale a DB and CPU intensive Panel app for multiple users?

Panel
1

We have developed a nice Panel app and would now like to improve our deployment to support multiple users.

Currently we are running single-thread in K8S with 1 CPU core in AKS (Azure Kubernetes Service), and the problem we have is that the main thread and app blocks for all users every time we do a database query (typically takes a second, but sometimes takes up to a minute for us) or a CPU-intensive computation (typically takes a second, but sometimes takes up to a minute).

We are currently investigating using the existing threading / async / multi-processing / caching options in the hope that we can easily scale up or out without too much effort. To scale up we ran into Multiprocessing crashes on macOS when a page makes an HTTP request · Issue #5780 · holoviz/panel · GitHub with multiprocessing on MacOS which is annoying for devs, it might work on Linux.

However, I was wondering if anyone has experience or advice or working examples how to scale Panel to a performant deployment? What architecture or tools (FastAPI, Litestar, DuckDB, Prefect, Celery) have you used or would try with Panel?

2

Some random thoughts:

  1. Depending on how unique the database queries are, have you tried to use pn.cache? More ideas here on caching Discussion on best practices for advanced caching - please join - #14 by jbednar

  2. DuckDB is a great replacement for SQLite, but I assume your database isn’t SQLite

  3. Async is probably a good way forward, and with Panel magic, you don’t have to convert your entire app to become async, just your database queries. Just link it to a widget, like I did in ChatInterface here, here, and here.

  4. Maybe you could involve dask? Asynchronous Operation — Dask.distributed 2023.10.1 documentation

3

Lowest effort way to scale is simply replicating the app and doing load balancing with a reverse proxy.

4

We more and more try to prepare the data outside the Panel context. If the query to the db is slow we set up a cronjob to load the data from db to azure blob storage. Then we load it from there on demand.

5

One thing I would like to try one day is using Dask as the compute engine powering panel. All calculations and data loads can be done in Dask orchestrated via async function calls from Panel.

When I’ve explored this, It has been amazing. I’ve been working on a Dask how to guide but got distracted. Pull Request #4234