Relate data point with Altair's selection_point after transform_aggregation

Regarding this issue: Support point selections in VegaLite/Altair · Issue #3398 · holoviz/panel · GitHub

After clicking a point in the chart, I see vega_multi.selection collects index number of the data point.
But if data points are plotted after “transform_aggregate” in Altair, then the re-indexed number of the aggregation are collected in the vega_multi.selection. But I don’t know how I can relate the re-indexed number to my dataset because aggregation is performed by altair internally.
Do you have any idea?