How to analyze usage and resource consumption for Panel Dashboards?

Panel
1

My team and I would love to be able to monitor the usages (mainly visits) and resource consumption (primarily memory) on a per dashboard level for all of our Panel Dashboards. Our vision is to have some application that could collect this information from the log files and display the results in another Panel dashboard as a sort of metrics dashboard.

As it stands, we are currently making use of –log-level trace but we have experimented with each of the available log levels. After manually inspecting the log information it seems as though we could gather the number of visitors per dashboard using the following section of the server log

2020-06-26 11:57:45,143 [pid 24525] 4 clients connected
2020-06-26 11:57:45,143 [pid 24525] /TTF has 1 sessions with 0 unused
2020-06-26 11:57:45,143 [pid 24525] /hello_world has 3 sessions with 0 unused

Ideally, we would like a way to see a more detailed output so we could determine the number of unique users per day, week, etc. However, this information is a good starting point.

Where we are struggling is finding a way to monitor the resource consumption per dashboard on a time-based interval. We are making use of the --mem-log-frequency, but the only information that this seems to provide is the total memory consumption of all the Panel dashboards. Example:

2020-06-26 11:57:40,142 [pid 24525] Memory usage: 277.00 MB (RSS), 561.00 MB (VMS)

I’m curious if anyone else in the community has set up a monitoring system for their Panel application. If so, how does your output look? How was it achieved?

Thanks!

4 Likes
2

I have not had time to look into this yet. But i am very interested and would eventually need this.

2 Likes
3

@Marc , Any progress on this?

4

Nope.

  • It is not on a dashboard level. But now the admin can be used for monitoring. See Enable the admin panel
  • At work I also have access to professional monitoring of kubernetes where we deploy Panel apps.
5

We (Ploomber team) built something like this already, I can share some of the technical details.

Since we wanted to provide a flexible solution that worked with other frameworks (Flask, FastAPI, Streamlit, etc.), we separated the analytics layer from the Panel app.

We run a reverse proxy (NGINX) in front of the Panel application, and configured the reverse proxy to collect usage metrics (requests, number of users, IPs, etc). These metrics are collected and aggregated in a separate dashboard. We haven’t implemented resource consumption yet but since we run Panel apps as containers, we can easily monitor resources. If you don’t run Panel apps as containers, monitoring resources will be a lot harder since you’ll have to implement a solution at the operating system level.

1 Like
6

@edu Would you have any good resources for deploying Panel applications out of containers?

7

I don’t, since the deployment process largely depends on your infrastructure (Kubernetes, ECS, etc), but here’s a sample app you can use.

8

I hadn’t seen that before so thank you for the resource!