Catch legend hide clicks in plotly pane

Hi there,
I am looking for a way to hide/show annotations together with the corresponding traces in plotly.

Let’s say I have a simple scatter plot in a panel and added an annotation to one of the points:

p = pn.pane.Plotly(px.scatter(x=[1,2], y=[1,2], color=['red','blue']))
p.object.layout.annotations = [dict(x=1,y=1,text='red')]

My goal is to hide the annotation ‘red’ together with the trace.

The question is twofold: 1. How can I tell which traces are hidden? 2. Is there a parameter or event I can watch to react to it?

I know it is not possible to link the annotation directly to a legendgroup. Using the text parameter of scatter instead is not feasible for me, since my real life plots are several thousand points and the annotations are added from a callback. Accessing the data instead of the layout slows everything down considerably.

I already checked and none of the exposed parameters currently react to the legend, if I bind them like this:

c = pn.widgets.IntInput(value=0)
def count(event):
     c.value +=1
pn.bind(count, p.param.relayout_data, watch=True)

Any help would be highly appreciated.