Panel FileDownload a netCDF file using Xarray to_netcdf() function

Panel
1

Hello, I’d like to use the FileDownload widget to download my hvplot() as a netCDF dataset using xarray to_netcdf() function. I’ve tried a few different ways but seem to be missing something.

# Python 3.7, anaconda dist
from datetime import date, datetime, timedelta
import panel as pn #version 0.9.5
import xarray as xr #version 0.11.3
import hvplot.xarray #version 0.5.2
import hvplot.pandas #version 0.5.2
import geoviews as gv #version 1.8.1
gv.extension('bokeh')

# read in the refET dataset
dsRefET = xr.open_dataset("http://basin.ceoe.udel.edu/thredds/dodsC/DEOSAG.nc")

# declare dataset list
datasets = ['refET']

# generate panel widgets
dataset = pn.widgets.Select(name='Dataset', options=datasets, value=datasets[0])
dateVal = pn.widgets.DatePicker(name='Date Picker', value=(date.today()))

# Define the function
def make_plot(dataset, dateVal):
    # Convert Date object to Datetime to work-around Date object / timedelta error panel was showing
    sDate = datetime(dateVal.year, dateVal.month, dateVal.day)

    # select data
    df = dsRefET.sel(time=[sDate], method='nearest')         

    # create quadmesh plot
    chart = df.refET.hvplot.quadmesh(x='longitude', y='latitude',project=True,geo=True,
                               rasterize=True, dynamic=False)
    
    return chart, df

# create button to update plot
def update(event):
    dashboard[1].object = make_plot(dataset.value, dateVal.value)[0]

generate_button = pn.widgets.Button(name='Plot', button_type='primary')
generate_button.on_click(update)

# create filepath widget
ncpath = pn.widgets.TextInput(name='File Download Path + Name', placeholder='~/Downloads/DEOS_AgWx.nc')

# create download widget
fd = pn.widgets.FileDownload(
    file=make_plot(dataset.value, dateVal.value)[1].to_netcdf(path=ncpath.value),
    filename='AgWx.nc')
    
# Create the dashboard
dashboard = pn.Row(pn.Column(dataset, dateVal, generate_button),
                   make_plot(dataset.value, dateVal.value)[0], ncpath, fd)

# show the dashboard
dashboard.show()
2

Maybe give the callback option a try.

from io import BytesIO
def callback():
    file_obj = BytesIO()
    make_plot(dataset.value, dateVal.value)[1].to_netcdf(file_obj)
    file_obj.seek(0)
    return file_obj

fd = pn.widgets.FileDownload(
    callback=callback,
    filename='AgWx.nc')

I havent really tested this code but I use something similar which works for me.
An update to FileDownload should be release soon: https://github.com/holoviz/panel/pull/1306
See also the reference gallery for a callback example: https://panel.holoviz.org/reference/widgets/FileDownload.html#widgets-gallery-filedownload

3

Thanks for the response, I think we’re almost there! I integrated that code, but am receiving the following error:

ValueError: I/O operation on closed file.

Here’s the reproducible code with your callback suggestion:

# Python 3.7, anaconda dist
from datetime import date, datetime, timedelta
import panel as pn #version 0.9.5
import xarray as xr #version 0.11.3
import hvplot.xarray #version 0.5.2
import hvplot.pandas #version 0.5.2
import geoviews as gv #version 1.8.1
from io import BytesIO
gv.extension('bokeh')

# read in the refET dataset
dsRefET = xr.open_dataset("http://basin.ceoe.udel.edu/thredds/dodsC/DEOSAG.nc")

# declare dataset list
datasets = ['refET']

# generate panel widgets
dataset = pn.widgets.Select(name='Dataset', options=datasets, value=datasets[0])
dateVal = pn.widgets.DatePicker(name='Date Picker', value=(date.today()))

# Define the function
def make_plot(dataset, dateVal):
    # Convert Date object to Datetime to work-around Date object / timedelta error panel was showing
    sDate = datetime(dateVal.year, dateVal.month, dateVal.day)

    # select data
    df = dsRefET.sel(time=[sDate], method='nearest')         

    # create quadmesh plot
    chart = df.refET.hvplot.quadmesh(x='longitude', y='latitude',project=True,geo=True,
                               rasterize=True, dynamic=False)
    
    return chart, df

# create button to update plot
def update(event):
    dashboard[1].object = make_plot(dataset.value, dateVal.value)[0]

generate_button = pn.widgets.Button(name='Plot', button_type='primary')
generate_button.on_click(update)

# create filepath widget
def callback():
    file_obj = BytesIO()
    make_plot(dataset.value, dateVal.value)[1].to_netcdf(path=file_obj, mode='w')
    file_obj.seek(0)
    return file_obj

fd = pn.widgets.FileDownload(
    callback=callback,
    filename='AgWx.nc')

# Create the dashboard
dashboard = pn.Row(pn.Column(dataset, dateVal, generate_button),
                   make_plot(dataset.value, dateVal.value)[0],fd)

# show the dashboard
dashboard.show()

Just as an FYI, I played with the xarray to_netcdf() arguments to see if that could help. No luck thus far. Thank you!

4

I suspect the to_netcdf closes the BytesIO object in which case this trick won’t work because if you close those objects the contents are gone.

I think the callback needs to return an open file object, so you could try:

def callback():
    file_obj = open('tempfile.dat', 'wb')
    make_plot(dataset.value, dateVal.value)[1].to_netcdf(path=file_obj)
    return open('tempfile.dat', 'r')
5

Unfortunately this causes an encoding error -

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf8 in position 689: invalid start byte

I tried declaring the encoding in the to_netcdf function without any luck. Any other ideas or workarounds? Thanks so much!

6

Try this callback:

from io import BytesIO

@pn.depends(dataset, dateVal)
def download_cb(ds, date):
    bout = make_plot(ds, date)[1].to_netcdf()
    bio = BytesIO()
    bio.write(bout)
    bio.seek(0)
    return bio

If you also want to make an issue/PR that allows callbacks to return bytes directly so that could be simplified to the following that would be appreciated:

@pn.depends(dataset, dateVal)
def download_cb(ds, date):
    return make_plot(ds, date)[1].to_netcdf()
1 Like
7

Hi @philippjfr @jsimkins2 ! This thread is a couple years old, but I’m wondering if anyone has figured out a solution to this. I’ve used @philippjfr 's method. It works, except the resulting netCDF that is downloaded is missing all the attributes.

from io import StringIO, BytesIO
def callback():
    netcdf = ds.to_netcdf() 
    data = BytesIO()
    data.write(netcdf)
    data.seek(0)
    return data

pn.widgets.FileDownload(callback=callback,
                         label={'en':'Download netCDF','fr':'Télécharger netCDF'}[LOCALE],
                         embed=True, 
                         filename='subset_data.nc')

Any insight would be much appreciated!

8

Perhaps try a different engine or directly pass the BytesIO buf to_netcdf as the “path” since it accepts file-like objects.

9

Thank you! I will try this.