Discussion on best practices for advanced caching - please join

Panel
12

Thanks for the investigation here. I would like to see two issues come out of this:

  1. Improvements for the in-memory cache based around pn.state.cache and pn.state.as_cached and memoize decorators.

  2. An issue discussing integration with an out-of-memory (i.e. disk cache). My main concern here is that disk caches are quite dependent on the kind of data you are meaning to persist, however since DataFrames are going to be the dominant use case I’d be quite happy to ship something in Panel to improve the caching experience there. At the same time though, a savy user will be able to achieve quite good performance by persisting columnar data to something like parquet and nD-arrays to something like Zarr. Beyond that an persistent in-memory solution like Redis will surely provide some additional speedup. In either case I’d love to see an issue discussing these options.

1 Like