Thanks @Marc. I’m currently using hv.Layout for the subplots and hvplot.bar and hv.Overlay for my barplots.
Do you think holomaps make a difference? I’m not too familiar with them. 
The remaining issue is that I can’t seem to update the HoloViews elements AND update the display of these elements without having a method returning the full layout (which is the slow part).
In Updating matplotlib figure in Parameterized class, I did see @philippjfr suggest a trigger functionality on a matplotlib object.
self.mpl.param.trigger('object')
I didn’t get it working for the hv.Layout object, but even then I’m not sure if a similar approach on a HoloViews object would resolve the speed issue?
does anyone know of documentation / examples of trigger on HoloViews?