I’m using Panel Pipelines to build a complex app with many pages (parameterized classes). From the Panel docs:
Pipeline stages may be either Parameterized instances or Parameterized classes. With classes, the class will not be instantiated until that stage is reached, which lets you postpone allocating memory, reading files, querying databases, and other expensive actions that might be in the constructor until the parameters for that stage have been collected from previous stages
Classes aren’t instantiated until you get to the page. But if I click on a previous network node to jump BACK in the pipeline, the state has been kept. So as far as I can tell, if I’m running a Panel Pipeline, the further I go in the pipeline, the more memory I’ll collect since I’ll have instantiated classes for all the pages by the end. Is that correct?
I believe you are right. And I also think it’s good that the previous state is kept as this is what a user would expect.
I would not expect there to be any memory problems. I would not worry so much about it until I can see clearly that there is a problem. Then I would identify the cause and solve the problem.
Thanks Marc. There isn’t a “problem” with handling things this way. In fact, it enables the previous pages to be reloaded in their previous state, which is useful.
I asked because my use case involves pipeline pages which contain a lot of data in memory. So for me, I need to be aware that all those previous pages are still in memory and consider that in my design. I don’t think this is explicitly stated on the docs, but perhaps it should be.
Right, this should definitely be documented. I’d also be happy for there to be an option to destroy the state, however in a complex pipeline this would generally mean you’d have to jump all the way back to the beginning since you can’t restore the state of an intermediate stage.