From SO: https://stackoverflow.com/questions/60864238/streaming-for-xarray-netcdf-files
I was wondering if there was a way to stream data directly from a NetCDF file as it’s being written with xarray.
I think I can “create” a non-buffered file like this?
import io
ts_file_stream = io.open("/some/file/being/written/to.nc", mode='rb', buffering=-1)
I also know that I can open this with Xarray:
import xarray as xr
ds = xr.open_dataset(ts_file_stream)
However, I am unsure if the arrays will then continually be updated? The purpose of this whole thing is as follows: I have a numerical model producing output, and I’d like to visualize some variables as the model runs to get a feeling for the current state. I know this is supported by the Holoviews people: https://hvplot.holoviz.org/user_guide/Streaming.html
Would this require me to make my own stream with the streamz library? https://streamz.readthedocs.io/en/latest/index.html
Any hints on how to get that to work for netcdf would be wonderful!
Cheers,
Paul