I’m trying to launch an app running under Panel in a docker container, it loads a netcdf file and make some plots, it is supposed to look like this:
The application code is available here: https://github.com/Paleoclim-CNRS/netcdf_editor_app/blob/main/Single_Page_WebApp/app.ipynb
Issue
When starting the container, the app launches and allow to choose a netcdf file but once the file is selected, nothing happens and an error is triggered:
b'\x00\x00\x00\x00\xc0i\x00\x00\x00\x00\x00\x00\xc0i\x00\x00'
...
b'\x00\x00\x00\x00\xc0\xaa\xe0\x00\x00\x00\x00\x00\xc0\xaa\xe0\x00'
...
b'\x00\x00\x00\x00\xc0\xaa\xe0\x00\x00\x00\x00\x00\xc0\xaa\xe0\x00'
b'\x80\x00\x00\x00\xc0\xaaF4\x80\x00\x00\x00\xc0\xaaF4'
...
...
b'\x80\x00\x00\x00\xc0\xaaF4\x80\x00\x00\x00\xc0\xaaF4'
b'\x80\x00\x00\x00'}
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/xarray/core/dataset.py", line 1348, in _construct_dataarray
variable = self._variables[name]
KeyError: None
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/panel/reactive.py", line 381, in _process_events
self.param.update(**self_events)
File "/usr/local/lib/python3.8/site-packages/param/parameterized.py", line 1902, in update
self_._batch_call_watchers()
File "/usr/local/lib/python3.8/site-packages/param/parameterized.py", line 2063, in _batch_call_watchers
self_._execute_watcher(watcher, events)
File "/usr/local/lib/python3.8/site-packages/param/parameterized.py", line 2025, in _execute_watcher
watcher.fn(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/param/parameterized.py", line 669, in caller
return function()
File "/usr/local/lib/python3.8/site-packages/param/parameterized.py", line 407, in _depends
return func(*args, **kw)
File "/usr/src/app/app.ipynb", line 209, in _parse_file_input
" else:\n",
File "/usr/local/lib/python3.8/site-packages/param/parameterized.py", line 369, in _f
return f(self, obj, val)
File "/usr/local/lib/python3.8/site-packages/param/parameterized.py", line 1252, in __set__
obj.param._call_watcher(watcher, event)
File "/usr/local/lib/python3.8/site-packages/param/parameterized.py", line 2043, in _call_watcher
self_._execute_watcher(watcher, (event,))
File "/usr/local/lib/python3.8/site-packages/param/parameterized.py", line 2025, in _execute_watcher
watcher.fn(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/param/parameterized.py", line 669, in caller
return function()
File "/usr/local/lib/python3.8/site-packages/param/parameterized.py", line 407, in _depends
return func(*args, **kw)
File "/usr/src/app/app.ipynb", line 651, in get_plots
" passage_problems = hv.DynamicMap(self.load_passage_problems).opts(\n",
File "/usr/local/lib/python3.8/site-packages/xarray/core/dataset.py", line 1439, in __getitem__
return self._construct_dataarray(key)
File "/usr/local/lib/python3.8/site-packages/xarray/core/dataset.py", line 1350, in _construct_dataarray
_, name, variable = _get_virtual_variable(self._variables, name, self.dims)
File "/usr/local/lib/python3.8/site-packages/xarray/core/dataset.py", line 182, in _get_virtual_variable
raise KeyError(key)
KeyError: None
I don’t have much experience with all of theses packages and I have no idea where the issue could come from. The combinations of versions for Panel/bokeh/holoviews/xarray I tried were not successful so far.
I get a bunch of binary strings at the top of the error b'\x80\x00\x00\x00\xc0\xaaF4\x80\x00\x00\x00\xc0\xaaF4', so I was wondering if the netcdf file was read properly (h5netcdf/nettcdf4 packages ?) hence the error with xarray (?)
Or it could be a package version which could break up everything…
Do you have any leads on what I should look at to solve this ?
packages used
The python packages used for this:
matplotlib 3.7.1
xarray 2023.1.0
panel 1.2.0
bokeh 2.4.3
holoviews 1.16.2
datashader 0.15.0
hvplot 0.8.4
shapely 2.0.1
h5netcdf 1.1.0
netcdf4 1.6.3
scipy 1.10.1
requests 2.31.0
spatialpandas 0.4.8
scikit-image 0.21.0
jinja2 3.1.2
jupyter 1.0.0
Code
I’m not sure about what part of the script is getting into trouble but the app is defined through a class with the __init__ method:
def __init__(self, **params):
self.param.file.default = pn.widgets.FileInput(max_width=200)
self.param.ds.default = xr.Dataset()
self.param.loaded.default = False
super().__init__(**params)
self.apply.on_click(self._apply_values)
self.undo_button.on_click(self.undo)
self.redo_button.on_click(self.redo)
self.download_netcdf.callback = self._download_netcdf
self.download_script.callback = self._download_script
self.fill_depressions_button.on_click(self._fill_depressions_callback)
self.file_pane.append(self.file)
self._auto_update_cmap_min = True
self._auto_update_cmap_max = True
self.curvilinear_coordinates = None
self._undo_list = []
self._redo_list = []
self.colormap_min.param.watch(self._colormap_callback, 'value')
self.colormap_max.param.watch(self._colormap_callback, 'value')
self.colormap_range_slider.param.watch(self._colormap_callback, 'value')
And the loaded file is processed with this method:
@pn.depends("file.value", watch=True)
def _parse_file_input(self):
self.loaded = False
value = self.file.value
# We are dealing with a h5netcdf file ->
# The reader can't read bytes so we need to write it to a file like object
if value.startswith(b"\211HDF\r\n\032\n"):
value = io.BytesIO(value)
ds = xr.open_dataset(value)
self.curvilinear_coordinates = None
number_coordinates_in_system = len(list(ds.coords.variables.values())[0].dims)
# Standard Grid
if number_coordinates_in_system == 1:
pass
# Curvilinear coordinates
elif number_coordinates_in_system == 2:
dims = list(ds[list(ds.coords)[0]].dims)
# Store the true coordinates for export
self.curvilinear_coordinates = list(ds.coords)
# Add the dimension into the coordinates this results in an ij indexing
ds.coords[dims[0]] = ds[dims[0]]
ds.coords[dims[1]] = ds[dims[1]]
# Remove the curvilinear coordinates from the original coordinates
ds = ds.reset_coords()
else:
raise ValueError("Unknown number of Coordinates")
self.ds = ds
self.attribute.options = list(ds.keys())
self._original_ds = ds.copy(deep=True)
self.loaded = True
return True
Thank you for your help !