I don’t know where is set the font, but if you use \U0002F441 instead of \1F441 it would works.
Check out this page
With this site you would be able to find the codes for python
You can create this button with this line :
pn.widgets.Button(name='\U0001F441')
In the notebook, the eye is rendered correctly