In the docs, I see it next to the labels
For Inputs you should be able to do it with
pn.widgets.IntInput(value=0, name="Test", description="Test").
Though it is not all widgets that are supported right now, see here.
Work to implement a standalone tooltip is being worked on here. And work on Bokeh side is also being done.