Serving panel with plotly extension offline

[Huggies23]

I want to serve a panel entirely offline using the plotly extension.

The local bokeh server instance calls on two js sources from URLs when pn.extension('plotly') is included in the script.

According to https://github.com/holoviz/panel/blob/master/panel/models/plotly.py these scripts are specified in the PlotlyPlot class:

__javascript__ = ['https://cdn.plot.ly/plotly-latest.min.js',
                  'https://cdnjs.cloudflare.com/ajax/libs/lodash.js/4.17.15/lodash.min.js']

__js_require__ = {'paths': {'plotly': 'https://cdn.plot.ly/plotly-latest.min',
                            'lodash': 'https://cdnjs.cloudflare.com/ajax/libs/lodash.js/4.17.15/lodash.min'},
                  'exports': {'plotly': 'Plotly',
                              'lodash': '_'}}

I was hoping that by simply saving the two source .js files locally (as plotly.js and lodash.js) and specifying these file locations in the panel.config object, I would be able to serve the panel app with full functionality without the need for a internet connection. I,e.:

panel.config.js_files  = {'plotly': 'C:\path\plotly.js',
                          'lodash': 'C:\path\lodash.js'}

However, a blank output is shown in a new browser tab when the .show() method is run on the panel object when no internet connection is available. Inspecting the html of the blank app in the browser window the following script elements are present:

<script type="text/javascript" src="https://cdn.plot.ly/plotly-latest.min.js"></script>
<style id="plotly.js-style-global"></style>
<script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/lodash.js/4.17.15/lodash.min.js"></script>
<script type="text/javascript" src="static/js/bokeh.min.js?v=547e7d2591695b654def5914bdd697fa"></script>
<script type="text/javascript" src="static/js/bokeh-widgets.min.js?v=423bf6bb32b8def8b6c9df74817506e4"></script>
<script type="text/javascript" src="static/js/bokeh-tables.min.js?v=5f778b8a005d8538b5b13998ec45fc16"></script>
<script type="text/javascript" src="static/js/bokeh-gl.min.js?v=be19384f76795da42f51580e7b5fd473"></script>
<script type="text/javascript" src="C:\path\plotly.js"></script>
<script type="text/javascript" src="C:\path\lodash.js"></script>

Any ideas on how I can achieve completely offline functionality by using local copies of the js?

Thanks!

Dave

[philippjfr]

Sorry I never replied here, if you’re still struggling with this I’d love to see an issue for it. Once we have static file handling I think we should bundle all the JS dependencies and allow serving inline and via the server.

[Huggies23]

Thanks @philippjfr. Sorry I missed this. I’ll have another crack at it and if unsuccessful I’ll create an issue and update here asap.

[hyamanieu]

@philippjfr I would like to restart this discussion. It seems since 0.11 there is this new Bokeh model called “tabulator” which gets loaded even if it isn’t used. It is declared exactly like the plotly model, so I believe we could restart this discussion in general about panel “custom” bokeh models working offline.

Using solutions as mentioned here to have it inline doesn’t work: Serving panel with plotly extension offline Panel regression? (embedding panel in gunicorn+Flask)

pn.config.js_files  = {'tabulator': r'/static/js/tabulator.js',
                          'moment': r'/static/js/moment.js'}
pn.config.inline = True
settings.resources = 'inline'

The two js files and the css files are systematically loaded from the external resource and I don’t see how this PR: Add support for serving static files by philippjfr · Pull Request #1319 · holoviz/panel · GitHub will help me with it.

I also don’t understand why tabulator JS code gets loaded although I don’t have a tabulator object. In one case I don’t even have any pn.widgets object but it still requires to load these three files.

Can we change the path to which __javascript__, __css__ and __js_require__ point to without changing Panel’s code?

Thanks

[philippjfr]

So we already bundle most models these days (including plotly). The way that works is that during the build step all models with a __javascript_raw__ declaration are bundled into panel/dist and in a server scenario they are indeed loaded from there. When serving an application the resource will then be loaded from the local directory. I suppose the missing piece is that I’m not sure whether they are also loaded from there in the inline case, which may mean it is not currently working in a notebook but that should actually be easy to fix.

[philippjfr]

Could you expand on your usecase? Because I just tested it and by default Plotly does in fact get served from the local dist directory and therefore works offline. So this issue is solved from my perspective.

As an aside, Tabulator gets loaded by default because panel/models/tabulator.py is imported by default while most other models are only imported when they are actually used.

[hyamanieu]

I meant to link that topic and not loop-link this current topic:

Panel regression? (embedding panel in gunicorn+Flask)

Hi Philipp, I encounter the same issue with Panel 0.11.3, I had to downgrade to 0.11.1 for my project to work. The problem appears starting 0.11.2. The bug is still the same : the URL of the CSS are relative, not absolute. Thus, if I serve bokeh documents (with port 5006) via Flask (port 8080), Panel CSS are loading with port 8080 instead of 5006. # from flask app, when reaching 127.0.0.1:8080 script = server_document('http://127.0.0.1:5006/bokeh_page') return render_template("embed.html", s…

so we have a flask server serving an application to several LANs, and one of them has no internet access. This is problematic only since tabulator was added, it was working flawlessly before as only bokeh models were loaded.

These three guys get always loaded in the apps (except one where I am only serving a pn.pane.Markdown)

JS_SRC = "https://unpkg.com/tabulator-tables@4.9.3/dist/js/tabulator.js"
MOMENT_SRC = "https://unpkg.com/moment@2.27.0/moment.js"
THEME_URL = "https://unpkg.com/tabulator-tables@4.9.3/dist/css/"

If I understand correctly your answer, we actually need to modify the panel source code by adding a __javascript_raw__ attribute? I’ll try that.

[pierrotsmnrd]

Hi everyone,

@philippjfr for the context : I work with @hyamanieu on his project.

You mentionned that all models with a __javascript_raw__ declaration were bundled into panel/dist.

But with 0.11.3, I don’t see the bundle for Tabulator :

root@b2fc575a6576:/app# ls -la /usr/local/lib/python3.8/site-packages/panel/dist/bundled/
total 92
drwxr-xr-x 23 root root 4096 Apr 29 11:10 .
drwxr-xr-x  7 root root 4096 Apr 29 11:10 ..
drwxr-xr-x  3 root root 4096 Apr 29 11:10 abstractvtkplot
drwxr-xr-x  3 root root 4096 Apr 29 11:10 aceplot
drwxr-xr-x  2 root root 4096 Apr 29 11:10 bootstraptemplate
drwxr-xr-x  6 root root 4096 Apr 29 11:10 css
drwxr-xr-x  2 root root 4096 Apr 29 11:10 darktheme
drwxr-xr-x  4 root root 4096 Apr 29 11:10 deckglplot
drwxr-xr-x  2 root root 4096 Apr 29 11:10 defaulttheme
drwxr-xr-x  3 root root 4096 Apr 29 11:10 echarts
drwxr-xr-x  2 root root 4096 Apr 29 11:10 fastbasetemplate
drwxr-xr-x  2 root root 4096 Apr 29 11:10 fastdarktheme
drwxr-xr-x  2 root root 4096 Apr 29 11:10 fastdefaulttheme
drwxr-xr-x  2 root root 4096 Apr 29 11:10 fastgridtemplate
drwxr-xr-x  2 root root 4096 Apr 29 11:10 fastlisttemplate
drwxr-xr-x  2 root root 4096 Apr 29 11:10 goldentemplate
drwxr-xr-x  9 root root 4096 Apr 29 11:10 js
drwxr-xr-x  2 root root 4096 Apr 29 11:10 materialtemplate
drwxr-xr-x  2 root root 4096 Apr 29 11:10 plotlyplot
drwxr-xr-x  2 root root 4096 Apr 29 11:10 reacttemplate
drwxr-xr-x  2 root root 4096 Apr 29 11:10 theme
drwxr-xr-x  2 root root 4096 Apr 29 11:10 vanillatemplate
drwxr-xr-x  3 root root 4096 Apr 29 11:10 vegaplot

If I understood properly, we should see a datatabulator directory here.

Is it because 'panel.models.tabulator' is missing from file config.py:419 in class panel_extension.__imports__ ?

[pierrotsmnrd]

• I have cloned panel, checked out tag 0.11.3,
• added 'datatabulator':'panel.models.tabulator' to config.py:419 in class panel_extension.__imports__
• python setup.py install
  → then I see datatabulator files under /usr/local/lib/python3.8/site-packages/panel/dist/bundled/

Unfortunately it doesn’t fix the problem yet. I guess I miss something else, related to how bokeh loads ressources from extensions.

I hope this helps.

[hyamanieu]

I have partially succeeded.

If bokeh.settings.settings.resources = 'inline', I can see it’s getting tabulator from unpkg.com, so that didn’t work in this case.

If not setting inline, both accessing the flask app (on port 8080) and bokeh’s (on port 5006) make the browser trying to get the local “bundled” js files for tabulator rather than from unpkg.com .

This is what I can see in the page source

<script type="text/javascript" src="[static/extensions/panel/bundled/datatabulator/tabulator-tables@4.9.3/dist/js/tabulator.js](view-source:http://127.0.0.1:5006/static/extensions/panel/bundled/datatabulator/tabulator-tables@4.9.3/dist/js/tabulator.js)"></script> <script type="text/javascript" src="[static/extensions/panel/bundled/datatabulator/moment@2.27.0/moment.js](view-source:http://127.0.0.1:5006/static/extensions/panel/bundled/datatabulator/moment@2.27.0/moment.js)"></script>

The flask app of course doesn’t work due to the relative link with the flask port rather than bokeh’s.

So we have two solutions:

• look for the way the inline process works (looks tough)
• or try to fix back the issue with relative links for Flask. It got fixed in panel 0.11.1 and came back in 0.11.3, so it shouldn’t be too difficult.

@pierrotsmnrd on top of the modifications you did, I have modified how __javascript__ and __js_skip__ class properties are made by copying what was done for plotly:

git diff ./panel/models/tabulator.py

@@ -9,6 +9,8 @@ from bokeh.core.properties import (
 from bokeh.models import ColumnDataSource
 from bokeh.models.layouts import HTMLBox
 from bokeh.models.widgets.tables import TableColumn
+from ..util import classproperty
+from ..io.resources import bundled_files
 
 
 JS_SRC = "https://unpkg.com/tabulator-tables@4.9.3/dist/js/tabulator.js"
@@ -69,11 +71,16 @@ class DataTabulator(HTMLBox):
 
     __css__ = [THEME_URL+'tabulator_simple.min.css']
 
-    __javascript__ = [
+    @classproperty
+    def __javascript__(cls):
+        return bundled_files(cls)
+
+    __javascript_raw__ = [
         JS_SRC,
         MOMENT_SRC
     ]
 
+
     __js_require__ = {
         'paths': {
             'tabulator': JS_SRC[:-3]
@@ -81,4 +88,6 @@ class DataTabulator(HTMLBox):
         'exports': {'tabulator': 'Tabulator'}
     }
 
-    __js_skip__ = {'tabulator': __javascript__}
+    @classproperty
+    def __js_skip__(cls):
+        return {'tabulator': cls.__javascript__}

[philippjfr]

You are right there’s a few problems:

1. Tabulator gets imported by default. This is an easy fix, we simply put the panel.models.tabulator import behind panel.util.lazy_load on the Tabulator widget.
2. We do not currently bundle Tabulator. This is a harder problem to fix because there’s a lot of plugins and other auxiliary files which I did not find straightforward ways to bundle into Panel.
3. Relative paths won’t work for pages embedded pages so we need to provide some way to configure set up absolute URLs.
4. I haven’t tested this but we should also make sure that inline loads the files from the bundled paths instead of the remote URLs.

Would you one of you open issues about these items?

[hyamanieu]

We will do that and try to fix the first one during the week if it’s easy.
It seems you fixed 3 in a another version, so I’ll try to look into it while doing 1.

As for 2, is this due to the many css themes?

Thanks!

[philippjfr]

Thanks.

As for 2, is this due to the many css themes?

That and many other plugins. For some other libraries we simply point it to a tar.gz bundle containing everything so we could explore that for Tabulator as well.

@hyamanieu

It seems you fixed 3 in a another version, so I’ll try to look into it while doing 1

No, in fact the opposite, I switched to relative paths quite deliberately to ensure that the paths are correct in a proxy scenario. But I guess we also need to be able to turn that off. What would be nice is if we could automatically determine whether to use relative or absolute paths based on source of the request. Could you elaborate on the approach you’re taking to embed the Panel app in Flask?

[hyamanieu]

@philippjfr

Could you elaborate on the approach you’re taking to embed the Panel app in Flask?

Some thing like that, I readapted by using only localhost ip addresses and an example from panel’s doc.

in app.py:

from bokeh.embed import server_document
from flask import Flask, render_template

def create_app():
    app = Flask(__name__)

    @app.route('/bokeh_app')
    def bokeh_app():
        script = server_document(f'http://127.0.0.1:5006/bokeh_app',
                                 )
        return render_template("embed.html", script=script)

   return app
if __name__ == "__main__":
    app = create_app()
    app.run(debug=True, host="0.0.0.0", port=8080)

in bokeh_app.py (using the sine example from the doc):

from app import create_app
import numpy as np
import param
import panel as pn
from tornado.ioloop import IOLoop

from bokeh.settings import settings
import panel as pn

settings.resources = 'inline'

#not sure if below are usefull
pn.config.js_files  = {'tabulator': r'/static/js/tabulator.js',
                          'moment': r'/static/js/moment.js'}
pn.config.inline = True


class Sine(param.Parameterized):

    phase = param.Number(default=0, bounds=(0, np.pi))

    frequency = param.Number(default=1, bounds=(0.1, 2))

    @param.depends('phase', 'frequency')
    def view(self):
        y = np.sin(np.linspace(0, np.pi * 3, 40) * self.frequency + self.phase)
        y = ((y - y.min()) / y.ptp()) * 20
        array = np.array(
            [list((' ' * (int(round(d)) - 1) + '*').ljust(20)) for d in y])
        return pn.pane.Str('\n'.join([''.join(r) for r in array.T]), height=380, width=500)





def bokehapp_doc(doc):
    app = create_app()
    with app.app_context():
        sine = Sine(name='ASCII Sine Wave')

        r = pn.Row(sine.param, sine.view).get_root()
                          
        doc.add_root(r)

if __name__ == "__main__":
    server = Server({'/bokeh_app': bokehapp_doc,
                     },
                    io_loop=IOLoop(),
                    allow_websocket_origin=['127.0.0.1:5006',
                                            '127.0.0.1:8080'],
                    )

    server.start()
    server.io_loop.start()

Then we start each file in a separate process

>> python app.py
>> python bokeh_app.py

Over the week-end I was having a slightly different approach, where app.py was similar but I was just simply serving some panel example using panel serve rather than building and editing a bokeh doc as in bokeh_app.py.

[hyamanieu]

@philippjfr

4. I haven’t tested this but we should also make sure that inline loads the files from the bundled paths instead of the remote URLs.

I have posted 3 issues for the three first points, but as for point 4 I first need to try on another model. Perhaps I did something wrong while trying to bundle the tabulator files which lead to the strange behavior of the inline setting.

[hyamanieu]

@philippjfr

1. Tabulator gets imported by default. This is an easy fix, we simply put the panel.models.tabulator import behind panel.util.lazy_load on the Tabulator widget.

I will need more help on that one.
I noticed that MathJax is not bundled like tabulator, and that Ace is bundled unlike the other guys and has uses the lazy_load utility function.

From there, I have modified the _get_model from the DataTabulator class in tables.py as follows:

    def _get_model(self, doc, root=None, parent=None, comm=None):
        print(f"#!# {doc}")
        
        if self._widget_type is None:
            self._widget_type = lazy_load(
                'panel.models.tabulator', 'DataTabulator', isinstance(comm, JupyterComm)
            )
        model = super()._get_model(doc, root, parent, comm)
        if root is None:
            root = model
        self._link_props(model, ['page', 'sorters'], doc, root, comm)
        return model

and left off the _widget_type in the class declaration as done for Ace. It’s declared as None in a parent class.

It doesn’t change the result: the three remote javascript & css files are still being loaded and mentioned in js_urls and css_urls lists.

Now I have modified the panel application to include a MathJax pane and an Ace pane to see how it behaves. Now their remote files get referenced as well in the inline setting:

  var js_urls = ["https://cdnjs.cloudflare.com/ajax/libs/ace/1.4.11/ace.js", "https://cdnjs.cloudflare.com/ajax/libs/ace/1.4.11/ext-language_tools.js", "https://cdnjs.cloudflare.com/ajax/libs/ace/1.4.11/ext-modelist.js", "https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-MML-AM_CHTML", "https://unpkg.com/tabulator-tables@4.9.3/dist/js/tabulator.js", "https://unpkg.com/moment@2.27.0/moment.js"];
  var css_urls = ["https://unpkg.com/tabulator-tables@4.9.3/dist/css/tabulator_simple.min.css"];

Which is the bug n°4 (I would have expected Ace not to be there).

I still don’t understand how come the js and css files from tabulator gets to be loaded while MathJax is not loaded when unused. I included some prints here and there around the tabulator widgets and models: it never reaches stdout.

I guess something is happening outside python that I can’t understand.