I was wondering if there was a way to stream data directly from a NetCDF file as it’s being written with
I think I can “create” a non-buffered file like this?
import io ts_file_stream = io.open("/some/file/being/written/to.nc", mode='rb', buffering=-1)
I also know that I can open this with Xarray:
import xarray as xr ds = xr.open_dataset(ts_file_stream)
However, I am unsure if the arrays will then continually be updated? The purpose of this whole thing is as follows: I have a numerical model producing output, and I’d like to visualize some variables as the model runs to get a feeling for the current state. I know this is supported by the Holoviews people: https://hvplot.holoviz.org/user_guide/Streaming.html
Would this require me to make my own stream with the
streamz library? https://streamz.readthedocs.io/en/latest/index.html
Any hints on how to get that to work for netcdf would be wonderful!