# Mapping Irregular Point Measurements with Xarray

**URL:** <https://discourse.holoviz.org/t/mapping-irregular-point-measurements-with-xarray/1280>\
**Category:** hvPlot\
**Tags:** holoviews\
**Created:** [October 6, 2020, 5:21pm UTC](https://discourse.holoviz.org/t/mapping-irregular-point-measurements-with-xarray/1280 "2020-10-06T17:21:38Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![kleavor](https://discourse.holoviz.org/letter_avatar_proxy/v4/letter/k/e47c2d/32.png) [@kleavor](https://discourse.holoviz.org/u/kleavor)\
**Post date:** [October 6, 2020, 5:21pm UTC](https://discourse.holoviz.org/t/mapping-irregular-point-measurements-with-xarray/1280/1 "2020-10-06T17:21:38Z")

</div>

I primarily work with satellite data that generates vertical profiles such that the data are collected in time and altitude with an associated latitude and longitude. I process the data into xarray with dimensions of `time` and `Altitude`. `latitude` and `longitude` are `data_vars` specified as coordinates along with other measurement-descriptive parameters such that I end up with a DataArray in the DataSet that looks something like

```auto
<xarray.DataArray 'Ozone_ao3' (time: 27691, Altitude: 200)>
dask.array<where, shape=(27691, 200), dtype=float32, chunksize=(938, 200), chunktype=numpy.ndarray>
Coordinates:
  * Altitude (Altitude) float32 0.5 1.0 1.5 2.0 ... 99.0 99.5 100.0
    event_id (time) int32 dask.array<chunksize=(617,), meta=np.ndarray>
    latitude (time) float32 dask.array<chunksize=(617,), meta=np.ndarray>
    longitude (time) float32 dask.array<chunksize=(617,), meta=np.ndarray>
    GRAM95_Version (time) float32 dask.array<chunksize=(617,), meta=np.ndarray>
    Met_Version (time) float32 dask.array<chunksize=(617,), meta=np.ndarray>
    sc_evt_type (time) int32 dask.array<chunksize=(617,), meta=np.ndarray>
    gnd_evt_type (time) int32 dask.array<chunksize=(617,), meta=np.ndarray>
    BetaAngle_Solar (time) float32 dask.array<chunksize=(617,), meta=np.ndarray>
    Ephemeris_Source (time) int32 dask.array<chunksize=(617,), meta=np.ndarray>
    met_source (time) int32 dask.array<chunksize=(617,), meta=np.ndarray>
  * time (time) datetime64[ns] 2017-06-07T16:03:42 ... 2020-08-3...

```

My ultimate goal is to create an interactive map where regions can be selected and the altitude profiles summarized over the selected time period and geographic area. The basic map is already easy to make by taking the `DataArray` object and requesting `.plot.scatter('longitude', 'latitude')` to generate something like this with a simulated selection box and summary profile like this

 ![image](https://discourse.holoviz.org/uploads/default/original/2X/a/aa200cb66db24cb762a71a08e9cf3e62084ae08b.jpeg)

where the right plot is now the average profile of atmospheric ozone concentration in the selected region.

The problem I am encountering is that, if I am understanding hvplot/geoviews correctly, the structure is expected to have **dimensions** of the coordinates to be plotted, and it is insufficient that they exist as variables. For example, if I attempt `.hvplot.points('longitude', 'latitude', geo=True)` on my `DataArray` I am met with an extremely long error sequence

```auto
---------------------------------------------------------------------------
IndexError Traceback (most recent call last)
~/.conda/envs/analysis/lib/python3.8/site-packages/IPython/core/formatters.py in __call__ (self, obj, include, exclude)
    968 
    969 if method is not None:
--> 970 return method(include=include, exclude=exclude)
    971 return None
    972 else:

~/.conda/envs/analysis/lib/python3.8/site-packages/holoviews/core/dimension.py in _repr_mimebundle_(self, include, exclude)
   1310 combined and returned.
   1311 """
-> 1312 return Store.render(self)
   1313 
   1314 

~/.conda/envs/analysis/lib/python3.8/site-packages/holoviews/core/options.py in render(cls, obj)
   1393 data, metadata = {}, {}
   1394 for hook in hooks:
-> 1395 ret = hook(obj)
   1396 if ret is None:
   1397 continue

~/.conda/envs/analysis/lib/python3.8/site-packages/holoviews/ipython/display_hooks.py in pprint_display(obj)
    280 if not ip.display_formatter.formatters['text/plain'].pprint:
    281 return None
--> 282 return display(obj, raw_output=True)
    283 
    284 

~/.conda/envs/analysis/lib/python3.8/site-packages/holoviews/ipython/display_hooks.py in display(obj, raw_output, **kwargs)
    250 elif isinstance(obj, (CompositeOverlay, ViewableElement)):
    251 with option_state(obj):
--> 252 output = element_display(obj)
    253 elif isinstance(obj, (Layout, NdLayout, AdjointLayout)):
    254 with option_state(obj):

~/.conda/envs/analysis/lib/python3.8/site-packages/holoviews/ipython/display_hooks.py in wrapped(element)
    144 try:
    145 max_frames = OutputSettings.options['max_frames']
--> 146 mimebundle = fn(element, max_frames=max_frames)
    147 if mimebundle is None:
    148 return {}, {}

~/.conda/envs/analysis/lib/python3.8/site-packages/holoviews/ipython/display_hooks.py in element_display(element, max_frames)
    190 return None
    191 
--> 192 return render(element)
    193 
    194 

~/.conda/envs/analysis/lib/python3.8/site-packages/holoviews/ipython/display_hooks.py in render(obj, **kwargs)
     66 renderer = renderer.instance(fig='png')
     67 
---> 68 return renderer.components(obj, **kwargs)
     69 
     70 

~/.conda/envs/analysis/lib/python3.8/site-packages/holoviews/plotting/renderer.py in components(self, obj, fmt, comm, **kwargs)
    386 doc = Document()
    387 with config.set(embed=embed):
--> 388 model = plot.layout._render_model(doc, comm)
    389 if embed:
    390 return render_model(model, comm)

~/.conda/envs/analysis/lib/python3.8/site-packages/panel/viewable.py in _render_model(self, doc, comm)
    416 if comm is None:
    417 comm = state._comm_manager.get_server_comm()
--> 418 model = self.get_root(doc, comm)
    419 
    420 if config.embed:

~/.conda/envs/analysis/lib/python3.8/site-packages/panel/viewable.py in get_root(self, doc, comm)
    645 """
    646 doc = doc or _curdoc()
--> 647 root = self._get_model(doc, comm=comm)
    648 self._preprocess(root)
    649 ref = root.ref['id']

~/.conda/envs/analysis/lib/python3.8/site-packages/panel/layout.py in _get_model(self, doc, root, parent, comm)
    118 if root is None:
    119 root = model
--> 120 objects = self._get_objects(model, [], doc, root, comm)
    121 props = dict(self._init_properties(), objects=objects)
    122 model.update(**self._process_param_change(props))

~/.conda/envs/analysis/lib/python3.8/site-packages/panel/layout.py in _get_objects(self, model, old_objects, doc, root, comm)
    108 else:
    109 try:
--> 110 child = pane._get_model(doc, root, model, comm)
    111 except RerenderError:
    112 return self._get_objects(model, current_objects[:i], doc, root, comm)

~/.conda/envs/analysis/lib/python3.8/site-packages/panel/pane/holoviews.py in _get_model(self, doc, root, parent, comm)
    225 plot = self.object
    226 else:
--> 227 plot = self._render(doc, comm, root)
    228 
    229 plot.pane = self

~/.conda/envs/analysis/lib/python3.8/site-packages/panel/pane/holoviews.py in _render(self, doc, comm, root)
    284 kwargs = {}
    285 
--> 286 return renderer.get_plot(self.object, **kwargs)
    287 
    288 def _cleanup(self, root):

~/.conda/envs/analysis/lib/python3.8/site-packages/holoviews/plotting/bokeh/renderer.py in get_plot(self_or_cls, obj, doc, renderer, **kwargs)
     71 combining the bokeh model with another plot.
     72 """
---> 73 plot = super(BokehRenderer, self_or_cls).get_plot(obj, doc, renderer, **kwargs)
     74 if plot.document is None:
     75 plot.document = Document() if self_or_cls.notebook_context else curdoc()

~/.conda/envs/analysis/lib/python3.8/site-packages/holoviews/plotting/renderer.py in get_plot(self_or_cls, obj, doc, renderer, comm, **kwargs)
    235 init_key = tuple(v if d is None else d for v, d in
    236 zip(plot.keys[0], defaults))
--> 237 plot.update(init_key)
    238 else:
    239 plot = obj

~/.conda/envs/analysis/lib/python3.8/site-packages/holoviews/plotting/plot.py in update(self, key)
    898 def update(self, key):
    899 if len(self) == 1 and ((key == 0) or (key == self.keys[0])) and not self.drawn:
--> 900 return self.initialize_plot()
    901 item = self. __getitem__ (key)
    902 self.traverse(lambda x: setattr(x, '_updated', True))

~/.conda/envs/analysis/lib/python3.8/site-packages/geoviews/plotting/bokeh/plot.py in initialize_plot(self, ranges, plot, plots, source)
    107 def initialize_plot(self, ranges=None, plot=None, plots=None, source=None):
    108 opts = {} if isinstance(self, HvOverlayPlot) else {'source': source}
--> 109 fig = super(GeoPlot, self).initialize_plot(ranges, plot, plots, **opts)
    110 if self.geographic and self.show_bounds and not self.overlaid:
    111 from . import GeoShapePlot

~/.conda/envs/analysis/lib/python3.8/site-packages/holoviews/plotting/bokeh/element.py in initialize_plot(self, ranges, plot, plots, source)
   1337 self.handles['plot'] = plot
   1338 
-> 1339 self._init_glyphs(plot, element, ranges, source)
   1340 if not self.overlaid:
   1341 self._update_plot(key, plot, style_element)

~/.conda/envs/analysis/lib/python3.8/site-packages/holoviews/plotting/bokeh/element.py in _init_glyphs(self, plot, element, ranges, source)
   1281 else:
   1282 style = self.style[self.cyclic_index]
-> 1283 data, mapping, style = self.get_data(element, ranges, style)
   1284 current_id = element._plot_id
   1285 

~/.conda/envs/analysis/lib/python3.8/site-packages/geoviews/plotting/bokeh/plot.py in get_data(self, element, ranges, style)
    162 def get_data(self, element, ranges, style):
    163 if self._project_operation and self.geographic:
--> 164 element = self._project_operation(element, projection=self.projection)
    165 return super(GeoPlot, self).get_data(element, ranges, style)
    166 

~/.conda/envs/analysis/lib/python3.8/site-packages/param/parameterized.py in __new__ (class_, *args, **params)
   2810 inst = class_.instance()
   2811 inst.param._set_name(class_. __name__ )
-> 2812 return inst. __call__ (*args,**params)
   2813 
   2814 def __call__ (self,*args,**kw):

~/.conda/envs/analysis/lib/python3.8/site-packages/holoviews/core/operation.py in __call__ (self, element, **kwargs)
    197 kwargs['per_element'] = self._per_element
    198 kwargs['link_dataset'] = self._propagate_dataset
--> 199 return element.apply(self, **kwargs)
    200 
    201 

~/.conda/envs/analysis/lib/python3.8/site-packages/holoviews/core/accessors.py in pipelined_call(*args, **kwargs)
     43 
     44 try:
---> 45 result = __call__ (*args, **kwargs)
     46 
     47 if not in_method:

~/.conda/envs/analysis/lib/python3.8/site-packages/holoviews/core/accessors.py in __call__ (self, apply_function, streams, link_inputs, link_dataset, dynamic, per_element, **kwargs)
    202 if hasattr(apply_function, 'dynamic'):
    203 inner_kwargs['dynamic'] = False
--> 204 new_obj = apply_function(self._obj, **inner_kwargs)
    205 if (link_dataset and isinstance(self._obj, Dataset) and
    206 isinstance(new_obj, Dataset) and new_obj._dataset is None):

~/.conda/envs/analysis/lib/python3.8/site-packages/holoviews/core/operation.py in __call__ (self, element, **kwargs)
    192 elif ((self._per_element and isinstance(element, Element)) or
    193 (not self._per_element and isinstance(element, ViewableElement))):
--> 194 return self._apply(element)
    195 elif 'streams' not in kwargs:
    196 kwargs['streams'] = self.p.streams

~/.conda/envs/analysis/lib/python3.8/site-packages/holoviews/core/operation.py in _apply(self, element, key)
    130 
    131 element_pipeline = getattr(element, '_pipeline', None)
--> 132 ret = self._process(element, key)
    133 
    134 for hook in self._postprocess_hooks:

~/.conda/envs/analysis/lib/python3.8/site-packages/geoviews/operation/projection.py in _process(self, element, key)
     38 
     39 def _process(self, element, key=None):
---> 40 return element.map(self._process_element, self.supported_types)
     41 
     42 

~/.conda/envs/analysis/lib/python3.8/site-packages/holoviews/core/data/ __init__.py in pipelined_fn(*args, **kwargs)
    214 
    215 try:
--> 216 result = method_fn(*args, **kwargs)
    217 
    218 op = method_op.instance(

~/.conda/envs/analysis/lib/python3.8/site-packages/holoviews/core/data/ __init__.py in map(self, *args, **kwargs)
   1181 
   1182 def map(self, *args, **kwargs):
-> 1183 return super(Dataset, self).map(*args, **kwargs)
   1184 map. __doc__ = LabelledData.map. __doc__
   1185 

~/.conda/envs/analysis/lib/python3.8/site-packages/holoviews/core/dimension.py in map(self, map_fn, specs, clone)
    703 return deep_mapped
    704 else:
--> 705 return map_fn(self) if applies else self
    706 
    707 

~/.conda/envs/analysis/lib/python3.8/site-packages/geoviews/operation/projection.py in _process_element(self, element)
    161 return element.clone(crs=self.p.projection)
    162 xdim, ydim = element.dimensions()[:2]
--> 163 xs, ys = (element.dimension_values(i) for i in range(2))
    164 coordinates = self.p.projection.transform_points(element.crs, xs, ys)
    165 mask = np.isfinite(coordinates[:, 0])

~/.conda/envs/analysis/lib/python3.8/site-packages/geoviews/operation/projection.py in <genexpr>(.0)
    161 return element.clone(crs=self.p.projection)
    162 xdim, ydim = element.dimensions()[:2]
--> 163 xs, ys = (element.dimension_values(i) for i in range(2))
    164 coordinates = self.p.projection.transform_points(element.crs, xs, ys)
    165 mask = np.isfinite(coordinates[:, 0])

~/.conda/envs/analysis/lib/python3.8/site-packages/holoviews/core/data/ __init__.py in pipelined_fn(*args, **kwargs)
    214 
    215 try:
--> 216 result = method_fn(*args, **kwargs)
    217 
    218 op = method_op.instance(

~/.conda/envs/analysis/lib/python3.8/site-packages/holoviews/core/data/ __init__.py in dimension_values(self, dimension, expanded, flat)
   1069 """
   1070 dim = self.get_dimension(dimension, strict=True)
-> 1071 return self.interface.values(self, dim, expanded, flat)
   1072 
   1073 

~/.conda/envs/analysis/lib/python3.8/site-packages/holoviews/core/data/xarray.py in values(cls, dataset, dim, expanded, flat, compute, keep_index)
    379 return data.T.flatten() if flat and not keep_index else data
    380 elif expanded:
--> 381 data = cls.coords(dataset, dim.name, expanded=True)
    382 return data.T.flatten() if flat else data
    383 else:

~/.conda/envs/analysis/lib/python3.8/site-packages/holoviews/core/data/xarray.py in coords(cls, dataset, dimension, ordered, expanded, edges)
    322 data = dataset.data[dim]
    323 else:
--> 324 data = util.expand_grid_coords(dataset, dim)
    325 if edges:
    326 data = cls._infer_interval_breaks(data, axis=1)

~/.conda/envs/analysis/lib/python3.8/site-packages/holoviews/core/util.py in expand_grid_coords(dataset, dim)
   1942 return array * np.ones_like(example)
   1943 else:
-> 1944 arrays = [dataset.interface.coords(dataset, d.name, True)
   1945 for d in dataset.kdims]
   1946 idx = dataset.get_dimension_index(dim)

~/.conda/envs/analysis/lib/python3.8/site-packages/holoviews/core/util.py in <listcomp>(.0)
   1942 return array * np.ones_like(example)
   1943 else:
-> 1944 arrays = [dataset.interface.coords(dataset, d.name, True)
   1945 for d in dataset.kdims]
   1946 idx = dataset.get_dimension_index(dim)

~/.conda/envs/analysis/lib/python3.8/site-packages/holoviews/core/data/xarray.py in coords(cls, dataset, dimension, ordered, expanded, edges)
    332 if ordered and data.shape and np.all(data[1:] < data[:-1]):
    333 data = data[::-1]
--> 334 shape = cls.shape(dataset, True)
    335 
    336 if dim in dataset.kdims:

~/.conda/envs/analysis/lib/python3.8/site-packages/holoviews/core/data/xarray.py in shape(cls, dataset, gridded)
     64 array = dataset.data[..., 0]
     65 else:
---> 66 array = dataset.data[dataset.vdims[0].name]
     67 if not gridded:
     68 return (np.product(array.shape, dtype=np.intp), len(dataset.dimensions()))

IndexError: list index out of range

```

of which I am noticing that the function is walking over `dims`.

I can generate scatter plots using `.hvplot.scatter('longitude', 'latitude')` just fine, though that doesn’t work with geographic axes. I’d also prefer not to translate the `latitude` and `longitude` coordinates into dimensions, because then I would have incredibly sparse matrices filled with empty values.

The hvPlot and GeoViews examples all illustrate data that are spatially gridded, so any insight on this use case would be appreciated.

My package versions for xarray, geoviews, and hvplot are

| Package | Version |
| --- | --- |
| xarray | 0.16.0 |
| GeoViews | 1.8.1 |
| hvPlot | 0.6.0 |

---

<div class="post-metadata">

**Author:** ![rsignell](https://discourse.holoviz.org/user_avatar/discourse.holoviz.org/rsignell/32/5855_2.png) [@rsignell](https://discourse.holoviz.org/u/rsignell)\
**Post date:** [June 5, 2023, 1:13pm UTC](https://discourse.holoviz.org/t/mapping-irregular-point-measurements-with-xarray/1280/2 "2023-06-05T13:13:33Z")

</div>

2.5 years later, I’m hitting this same problem. My workaround is to convert the xarray to a pandas dataframe, and _then_ use points. But I’ll raise this as an issue on hvplot.

```auto
import numpy as np
import xarray as xr
import hvplot.xarray
import hvplot.pandas
 
ds = xr.Dataset({
    'lat': ('trajectory', np.linspace(30, 35, 15)),
    'lon': ('trajectory', np.linspace(16, 20, 15))
})

```

This doesn’t work, giving: `KeyError: 'lon'`

```auto
ds.hvplot.points(x='lon', y='lat', geo=True, tiles='OSM')  

```

But this _does_ work:

```auto
ds.to_dataframe().hvplot.points(x='lon', y='lat', geo=True, tiles='OSM')

```

I just [raised an issue](https://github.com/holoviz/hvplot/issues/1080)