This topic is a constant struggle. I think we should document this better.
You can understand a little bit about what is going on and how to control the behavior here:
https://holoviews.org/FAQ.html (look at the questions about auto-ranging and the one that comes after that)
The likely working solution is to use options framewise=False, dynamic=False. I would also try the solution that @ahuang11 proposed, I have not tried it.
However, the downside with dynamic=False is that it switches your elements from Dynamicmap to Holomap, which might make things quite a bit slower for some cases.
Older related questions:
- Inconsistent behavior of colormap for Adjoint layout with hist (some of the issue raised here is answered in the FAQ link I post above).