It is my understanding that hv.Labels uses Bokeh’s Text model under the hood.
As a consequence, the range of options you can set is quite reduced in comparison with what you can do with LabelSet. Most notably, I was surprised to see that you cannot have a background behind the label (background_fill_color parameter in LabelSet). This is definitely useful when you want to enhance text readability when it is displayed above some data or image.
So far I’ve had some luck using hooks to add a LabelSet glyph corresponding to my need. However, it can only go so far, as it seems that you can’t make it work within a DynamicMap.
I guess changing the corresponding backend model used would entail significant work. Still, I think there are only benefits to using LabelSet instead of Text. There may be reasons I’m not aware of as to why the latter was chosen over the former though. Maybe LabelSet did not exist at the time hv.Labels was created.