Panel template doesn't work when served from bokeh server document

lastmeta · January 9, 2021, 7:29pm

So we serve a bokeh app using bokeh server document, the servable object was a gpsec, but I converted it to a panel Template kinda like this:

        template = """
            {%% extends base %%}
                {%% block postamble %%}
                <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/materialize/1.0.0/css/materialize.min.css">
                <script src="https://ajax.googleapis.com/ajax/libs/jquery/3.3.1/jquery.min.js"></script>
                <script src="https://kit.fontawesome.com/2304e4818b.js"></script>
                <script src="https://cdnjs.cloudflare.com/ajax/libs/materialize/1.0.0/js/materialize.min.js"></script>
                {%% endblock %%}
                {%% block contents %%}
                {%% set context = '%s' %%}
                    <div class="container" style="margin-left: 10px;">
                        <div class="row">
                              <div class="col">
                                {{ embed(roots.title) }}
                              </div>
                              <div class="col">
                                {{ embed(roots.actions) }}
                              </div>
                          </div>
                          {{ embed(roots.separator) }}
                          <div class="row">
                            <div class="col">
                              {{ embed(roots.widgets) }}
                            </div>
                            <div class="col">
                              {{ embed(roots.report) }}
                            </div>
                          </div>
                    </div>
                {%% endblock %%}
        """
    self.template = pn.Template(
        template=(template % 'server'),
        nb_template=(template % 'notebook'))
    self.template.add_panel('fav', pn.Column(width=220, css_classes=['fav_place'], margin=0))
    self.template.add_panel('title', self.info)
    self.template.add_panel('actions', pn.Column(pn.Row(self.actions, align='center', margin=0), align='end', margin=0))
    self.template.add_panel('separator', pn.Row(self.widget_box_show, pn.layout.Divider(margin=(0, 0, 0, 0), sizing_mode='stretch_width')))
    self.template.add_panel('widgets', self.widget_box)
    self.template.add_panel('report', report_area)
    return self.template.servable()

The template has basically 3 rows of elements:

[title, button]
[button, separator]
[widget box, table of data or plots or whatever]

But that’s now how it renders. It renders each element down the left side of the screen:

It seems I’m unable to get the Template().servable() object to render correctly when serving the bokeh app (through flask) using the server_document from bokeh:

def route...
...
return render_template(
    'base.html',
    script=server_document(url=url, arguments=kwargs))

Should I be able to? If this is supposed to work, there must be something wrong on my end, maybe I have old bokeh js files or something, idk. Or perhaps panel Templates aren’t servable with bokeh server_document?

Any advice would be greatly appreciated. I did find a workaround using an iframe to serve it, but it requires me to rebuild a bunch of functionality which I’d like to avoid.

Thanks!

cisaacstern · January 9, 2021, 9:04pm

@lastmeta, I believe any jinja2 variable references in your base.html template will need to be passed to Flask’s render_template() as keyword arguments. My (as-yet-incomplete) approach includes copying both bokeh’s _templates directory and bokeh’s templates.py into my flask app’s root directory. At the bottom of templates.py I add the following dictionary:

jinja_templates = {
    'js_resources': JS_RESOURCES,
    'css_resources': CSS_RESOURCES,
    'script_tag': SCRIPT_TAG,
    'plot_div': PLOT_DIV,
    'root_div': ROOT_DIV,
    'doc_js': DOC_JS,
    'doc_nb_js': DOC_NB_JS,
    'file': FILE,
    'macros': MACROS,
    'notebook_load': NOTEBOOK_LOAD,
    'autoload_js': AUTOLOAD_JS,
    'autoload_nb_js': AUTOLOAD_NB_JS,
    'autoload_tag': AUTOLOAD_TAG,
    'autoload_request_tag': AUTOLOAD_REQUEST_TAG}

Then, in my flask_app.py file, I pass these jinja2 variables to Flask’s render_template() as follows:

from templates import jinja_templates 

@app.route(...)
def page():
...
return render_template('react.html', base=base, docs=docs, roots=_render_items, 
                        local_port=local_port, **jinja_templates, **context)

You’ll notice that jinja_templates is not the only additional kwarg I am passing into jinja2’s namespace. There’s a lot going on there! Once this is all working, I will write it up formally, but will postpone describing some of those other arguments now, as not all of them will necessarily make it into the final approach.

Does that general approach make sense? And is it relevant to your situation? I should have I working example by sometime next week.

lastmeta · January 10, 2021, 1:07am

I think that does make sense, thank you for the detail, especially the dictionary. Maybe I’ll be able to use the same approach.

cisaacstern · January 10, 2021, 6:31pm

Sure thing. What I’ve learned from this foray is that panel serve does a lot of work in the background to present the templates. Replicating that process with render_template() entails a lot of components, of which jinja2 context is only one. More to come.

cisaacstern · January 12, 2021, 10:20pm

I am here to admit defeat. This issue is considerably more challenging—and, by extension, more fascinating—than I’d anticipated. I’ll record some of my findings below, in hopes that someone else interested in this topic (perhaps even my future self) will find this trail of breadcrumbs useful.

A brief note on motivation. I have a Flask homepage, call it example.com, to which I plan to associate various Panel applications. Initially, I’d thought I’d achieve this by making these apps routes of the root domain, such as example.com/panel-app etc. This is relatively trivial for non-templated apps, as described, e.g., here.

This approach is not plug-and-play for Templated apps for a reason which, in retrospect, seems obvious: when panel serve serves a Templated app, it iterates over the roots attribute of a bokeh.document. This attribute is a list of various bokeh.models, which are embedded into the correct part of the selected jinja2 template based on their tags attribute. Incidentally, this is what happens when we append() a component of our Panel app to a specific region of a Panel Template: we’re actually setting the model’s tag attribute in such a way that it can be recognized in by jinja2’s control flow. For example, here’s an excerpt from Panel’s react template:

{% for doc in docs %}
{% for root in doc.roots %}
{% if "header" in root.tags %}
{{ embed(root) | indent(8) }}
{% endif %}
{% endfor %}
{% endfor %}

We can see that when the jinja2 variable doc is list of bokeh.document class instances, the correct attributes will exist to insert the app’s components into the designated part of the template.

But here’s the kicker, despite also using the word “document” in its name, the method bokeh.embed.server_document does not return bokeh.document class instances, but rather a <script> which (if I understand correctly) is populated by the AutoloadJsHandler.

As such, I now see that the work I described above to provide the proper context for Flask’s render_template() was pointless (at least as regards solving this problem): a <script> is not a bokeh.document.

In terms of my particular use case: I’m planning to just put the panel app on a subdomain such as panel-app.example.com. This should make it relatively straightforward to run the app on its own server (thereby sidestepping this problem entirely), but still have it thematically associated with the root domain, which was my goal all along.

My guess is that a solution (rather than a side-step) for the original question would probably require some sort of custom subclass of SessionHandler which modifies the behavior of AutoloadJsHandler to provide a script (or scripts) to expose session-linked components to insert into a specific <div> of the Flask template. This seems … really hard! But then again, I don’t know a lot of Javascript.

Eagerly welcome any and all follow-up questions and/or corrections from those more knowledgeable than me.

Marc · January 13, 2021, 2:18pm

Hi @cisaacstern

Thanks for sharing the information. My use case is serving a site of templated panel apps via the bokeh/ panel server.

So much looking forward to hear how it goes for you with panel-app.example.com.

My template for a site of apps is located here https://github.com/MarcSkovMadsen/awesome-analytics-apps-template

jmurray6 · March 31, 2021, 9:14pm

Has anyone found a solution for this? I’m deploying a beautiful templated Panel app for my team, and I’m securing it with Google auth through a Flask app. But the display is all wonky! The sidebar is not on the side!

@cisaacstern when reading through this thread I was getting so excited until I saw that you admitted defeat! So sad! Would love to know if anyone has been able to get past this or if any updates make it any easier.

Marc · April 1, 2021, 5:43am

Hi @jmurray6

I don’t know about Flask. But would it be possible for you to deploy it as a Tornado app using panel serve or panel.serve? And then use the integrated auth in Panel?

jmurray6 · April 1, 2021, 1:21pm

Hey @Marc

In this case Flask is the better option in terms of integrating with the rest of the tools we’re setting up for our team. So for now, I have it working with Flask and with Google auth, but it just doesn’t look as nice as it would with a template.

nghenzi · April 1, 2021, 2:54pm

to use the templates in flask I embed the panel app with an iframe and I use flask login for authentication. Here you have a photo how it looks

lastmeta · April 11, 2021, 11:41pm

this is be best solution we were able to come to so far as well. Seems to work pretty well, we are loving panel!