Cannot execute script with Panel

[Nobody]

Hi,

this is my script:

import time
import datetime
from datetime import date
import panel as pn

import numpy as np
import pandas as pd
import seaborn
from panel.widgets import Tqdm
from awesome_panel_extensions.widgets.progress_ext import ProgressExt
pn.extension()


radio_group = pn.widgets.RadioButtonGroup(
    name='Select', options=['A', 'B'], button_type='success')

radio_group


text = pn.widgets.TextInput(value='Ready')
x = 1
y = 2
button = pn.widgets.Button(name='Run!', button_type='primary')


def b(event):

    if radio_group.value == 'A':
        text.value ='Calculating A'
        exec(open('A.py').read())
        text.value = 'Finished with A'

    elif radio_group.value == 'B':
        text.value = 'Calculating B'
        exec(open('B.py').read())
        text.value = 'Finished with B'
        
button.on_click(b)
pn.Row(button, text)


template = pn.template.FastListTemplate(
    title='XX!', 
    sidebar=[pn.pane.Markdown("Choose which Calculation you want to run"), 
             radio_group,
             pn.Row(button, text),
            ],
)
template.servable();

A.py and B.py create some folders, read some input files, do some calculations on them and store the results in the outputfolder, e.g. a folder name “output/B/corrected_raw_data/” and stores files in there.
When I click the button within Jupyter or execute “B.py” in the cli, everything works fine.
However, when I serve it with panel and then hit the button I’ll get this error:

IsADirectoryError: [Errno 21] Is a directory: ‘/home/user/output/B/corrected_raw_data/’

It seems that panel cannot read the input data. Probably permission related?

Any ideas?

edit:

inserting

try:
    open('/home/user/input/2023.06.02/230306 002_Blk.TXT','r')
    print('Succes')
except IOError:
    print("The file cannot be opened")

in the “elif”-part I get as result:

Succes

when running in jupyterlab.

Pressing the button on panel I get this error message in my cli:

The file cannot be opened

So I think this is the problem:

User authentication hooks NOT provided (default user enabled)

How can I start panel using my default linux user?

[Marc]

Hi @Nobody

The message User authentication hooks NOT provided (default user enabled) means that no authentication like OAuth has been setup to protect your application. I.e. users of the application are not required to login in the browser. Its not a problem and very normal.

If you are starting your server with your normal linux user then the server will have the permissions of that user.

Regarding the the file cannot be opened its hard to say exactly what the problem is as I cannot reproduce it because I don’t have the exact same folder and file structure available.

One hypothesis could be that when you are in the notebook your current working directory is the directory where the notebook is located. When you run the panel server your current working directory is the directory where you started your server from. I.e. the current working directory can be different for the two use cases. So if you are referencing any files relatively, then you might experience the issues you do.

On another note, a suggestion would be to try to avoid exec. Normally you would import the python module A and B and then run a function from those modules instead.

If you can provide more information like a video showing exactly how your file structure looks and what you do, or a fully minimum, reproducible example then it would be easier to figure out the cause.

[Nobody]

The jupyter lab notebook is located here:

/home/user/

The python script file is located here:
/home/user/

The input files which should be processed are located here:

/home/user/input/

The processed files should be stored here:

/home/user/output/

The location where I execute/start the panel server:
/home/user/

It doesn’t matter if I use exec(open).read() or import it as a module. I get the same error:
The file cannot be opened

If I try the exact same code within a python shell or alone in a notebook without panel it works: “Success”

[carl]

Hi @Nobody,

Try replacing the decimals used in the date with underscores, it’s normal to have the file extension after first decimal? Maybe that’s more a windows thing… If it is permission related going on you’ll want to make sure all the scripts /files / directories have the appropriate chmod in place. That’s all I have for suggestions here.

Thanks, Carl

[Nobody]

why should I replace the decimals in the date? Within Python it works like a charm. Only when I try to do it with

panel serve

it does not work.

I tried to start panel as root, but that does not work either.

[carl]

It might be an issue it might not but might be worth considering changing to see something to rule out? Though reading more about it, the last decimal seems to be where the OS will interpret the extension though in windows it can’t be at the end of directory name

[Nobody]

I’m running panel on Linux and changing it from decimal to “_” would result in changing many lines in several files, which I would like to avoid.

[carl]

Hi @Nobody,

Try this code, it works for myself, so A works B fails does it do the same for you? For me it’s the dot notation at the start of the directory path that does it, without it, it fails

import panel as pn
pn.extension()


radio_group = pn.widgets.RadioButtonGroup(
    name='Select', options=['A', 'B'], button_type='success')

radio_group


text = pn.widgets.TextInput(value='Ready')

button = pn.widgets.Button(name='Run!', button_type='primary')


def b(event):

    if radio_group.value == 'A':
        text.value ='Calculating A'
        print(text.value)
        #exec(open('A.py').read())
        try:
            open('./data/test.txt','r')
            #df = pd.read_csv('./data/test.txt')
            print('Succes with open()')
        except IOError:
            print("Failed with open()")
            text.value = 'Finished with A'
        print(text.value)

    elif radio_group.value == 'B':
        text.value = 'Calculating B'
        print(text.value)
        #exec(open('B.py').read())
        try:
            open('/data/test1.txt','r')
            print('Succes')
        except IOError:
            print("failed with open")
        text.value = 'Finished with B'
        print(text.value)
        
button.on_click(b)
pn.Row(button, text)


template = pn.template.FastListTemplate(
    title='XX!', 
    sidebar=[pn.pane.Markdown("Choose which Calculation you want to run"), 
             radio_group,
             pn.Row(button, text),
            ],
)
template.servable();

[Nobody]

Yes, I get the same result as you.

First Success with open for A

and failed with open for B.

But changing B into:

open('data/test.txt','r')

I get a Success as well

[Nobody]

I removed the date from the folder structure but still the same problem:

error: FileNotFoundError(2, ‘No such file or directory’)

But again: only when I use panel serve

If I execute the jupyter lab notebook, it works like a charm.

Edit: I inserted the code from A.py into the

def b(event):

    if radio_group.value == 'A':
...

part, but, however, it does not work either. It does not find the input folder or files, respectively.

[carl]

So I’m now on my fedora linux machine both data and ./data methods work in the environment and tornado server for myself. I’m serving using

panel serve test.ipynb

[Nobody]

Really really strange.

Running the .py script from shell:
Works

Running the jupyterlab Notebook:
Works

Running a similar code with streamlit:
Works

Running panel serve:
Does not work.

I’ve tried running it from several locations and even tried different servers, even windows. And even run it as root. But nothing.

Thus, the Problem musst be within panel, but I want to use Panel since it is the only python dashboard supporting a file selector on the server itself.

[Hoxbro]

I will use absolut paths like this:

from pathlib import Path

import panel as pn

pn.extension()

PATH = Path(__file__).parent

radio_group = pn.widgets.RadioButtonGroup(
    name='Select', options=['A', 'B'], button_type='success')



text = pn.widgets.TextInput(value='Ready')
button = pn.widgets.Button(name='Run!', button_type='primary')


def b(event):

    if radio_group.value == 'A':
        text.value ='Calculating A'
        with open(PATH / 'A.py') as f:
            exec(f.read())
        text.value = 'Finished with A'

    elif radio_group.value == 'B':
        text.value = 'Calculating B'
        with open(PATH / 'B.py') as f:
            exec(f.read())
        text.value = 'Finished with B'

button.on_click(b)
pn.Row(button, text)


template = pn.template.FastListTemplate(
    title='XX!',
    sidebar=[pn.pane.Markdown("Choose which Calculation you want to run"),
             radio_group,
             pn.Row(button, text),
            ],
)
template.servable()

[Nobody]

Thank you very much, but faceing the same problem:

Within Jupyter Lab it works like a charm. But running panel serve I get again the “File not found” error.

edit: Just to make it clear: all of the methods mentioned can open “A.py”, but not the processes files within “/home/user/input”. However, the files within “/home/user/output” seems to be accessible.

[Hoxbro]

Then do the following change: PATH = Path("/home/user/output")

[Nobody]

As I said, the Input folder can’t be read

[Nobody]

The only way to solve the problem is sadly to move back to streamlit.

However, thank you very much for your help