Developing custom models in panel help

I am trying to create a custom bokeh model so I can work towards creating a PR to add jstree to panel now that panel 1.0 rc is out. I have my previous code implemented using with the typescript code being a string in the python code and then having __implementation__ = TypeScript(TS_CODE) in the python model class. Which works for now, but has had me run into some limitations, and I know I would need to change it eventually. As I tried to change it over, I could not get it to resolve it on the js side. I’ve been getting things like "could not resolve type ‘panel.models.tree.jsTreePlot’, which could be due to a widget or a custom model not being registered before first usage’.

I tried simplifying by taking an already existing model and seeing if I could get that working in a new file with no changes to the implementation. So I took the Trend indicator from the trend files in models because it was a pretty short implementation. Same problem. I ran python setup.py develop after changing things in both cases. Here is what I have for the custom select code (called tree since I was working on adding jstree).

I know it looks like a lot of code, but 99% of it is copy and paste from panel’s code base, and I outline the few changes before each code block.

panel/models/tree.ts (same code as trend.ts except static __module__ = "panel.models.tree" and classes renamed Tree for namespace reasons)

import {HTMLBox, HTMLBoxView} from "./layout"
import {build_view} from "@bokehjs/core/build_views"
import {Plot} from "@bokehjs/models/plots"
import {Line, Step, VArea, VBar} from "@bokehjs/models/glyphs"
import * as p from "@bokehjs/core/properties"
import {div} from "@bokehjs/core/dom"
import {ColumnDataSource} from "@bokehjs/models/sources/column_data_source"
import {BasicTickFormatter, NumeralTickFormatter, TickFormatter} from "@bokehjs/models/formatters"

const red: string="#d9534f";
const green: string="#5cb85c";
const blue: string="#428bca";

export class TreeIndicatorView extends HTMLBoxView {
  model: TreeIndicator
  containerDiv: HTMLDivElement
  textDiv: HTMLDivElement
  titleDiv: HTMLDivElement
  valueDiv: HTMLDivElement
  value2Div: HTMLDivElement
  changeDiv: HTMLElement
  plotDiv: HTMLDivElement
  plot: Plot
  _value_format: string
  _value_change_format: string

  initialize(): void {
    super.initialize()
    this.containerDiv = div({style: "height:100%; width:100%;"})
    this.titleDiv = div({style: "font-size: 1em; word-wrap: break-word;"})
    this.valueDiv = div({style: "font-size: 2em"})
    this.value2Div = div({style: "font-size: 1em; opacity: 0.5; display: inline"})
    this.changeDiv = div({style: "font-size: 1em; opacity: 0.5; display: inline"})
    this.textDiv = div({}, this.titleDiv, this.valueDiv, div({}, this.changeDiv, this.value2Div))

    this.updateTitle()
    this.updateValue()
    this.updateValue2()
    this.updateValueChange()
    this.updateTextFontSize()

    this.plotDiv = div({})
    this.containerDiv = div({style: "height:100%; width:100%"}, this.textDiv, this.plotDiv)
    this.updateLayout()
  }

  connect_signals(): void {
    super.connect_signals()

    const {pos_color, neg_color} = this.model.properties
    this.on_change([pos_color, neg_color], () => this.updateValueChange())
    const {plot_color, plot_type, width, height, sizing_mode} = this.model.properties
    this.on_change([plot_color, plot_type, width, height, sizing_mode], () => this.render())

    this.connect(this.model.properties.title.change, () => this.updateTitle(true))
    this.connect(this.model.properties.value.change, () => this.updateValue(true))
    this.connect(this.model.properties.value_change.change, () => this.updateValue2(true))
    this.connect(this.model.properties.layout.change, () => this.updateLayout())
  }

  async render(): Promise<void> {
    super.render()
    this.shadow_el.appendChild(this.containerDiv)
    await this.setPlot()
  }

  private async setPlot() {
    this.plot = new Plot({
      background_fill_color: null,
      border_fill_color: null,
      outline_line_color: null,
      min_border: 0,
      sizing_mode: "stretch_both",
      toolbar_location: null,
    });

    var source = this.model.source
    if (this.model.plot_type === "line"){
      var line = new Line({
        x: { field: this.model.plot_x },
        y: { field: this.model.plot_y },
        line_width: 4,
        line_color: this.model.plot_color,
      })
      this.plot.add_glyph(line, source)
    } else if (this.model.plot_type === "step"){
      var step = new Step({
        x: { field: this.model.plot_x },
        y: { field: this.model.plot_y },
        line_width: 3,
        line_color: this.model.plot_color,
      })
      this.plot.add_glyph(step, source)
    } else if (this.model.plot_type === "area") {
      var varea = new VArea({
        x: { field: this.model.plot_x },
        y1: { field: this.model.plot_y },
        y2: 0,
        fill_color: this.model.plot_color,
        fill_alpha: 0.5,
      })
      this.plot.add_glyph(varea, source)
      var line = new Line({
        x: { field: this.model.plot_x },
        y: { field: this.model.plot_y },
        line_width: 3,
        line_color: this.model.plot_color,
      })
      this.plot.add_glyph(line, source)
    } else {
      var vbar = new VBar({
        x: { field: this.model.plot_x },
        top: { field: this.model.plot_y },
        width: 0.9,
        line_color: null,
        fill_color: this.model.plot_color
      })
      this.plot.add_glyph(vbar, source)
    }

    const view = await build_view(this.plot)
    this.plotDiv.innerHTML = ""
    view.render_to(this.plotDiv)
  }

  after_layout(): void {
    super.after_layout()
    this.updateTextFontSize()
  }

  updateTextFontSize(): void {
    this.updateTextFontSizeColumn();
  }

  updateTextFontSizeColumn(): void {
    let elWidth = this.containerDiv.clientWidth;
    let elHeight = this.containerDiv.clientHeight;
    if (this.model.layout === "column")
      elHeight = Math.round(elHeight/2)
    else
      elWidth = Math.round(elWidth/2)

    const widthTitle = this.model.title.length
    const widthValue = 2*this._value_format.length
    const widthValue2 = this._value_change_format.length+1

    const widthConstraint1 = elWidth/widthTitle*2.0
    const widthConstraint2 = elWidth/widthValue*1.8
    const widthConstraint3 = elWidth/widthValue2*2.0
    const heightConstraint = elHeight/6

    const fontSize = Math.min(widthConstraint1, widthConstraint2, widthConstraint3, heightConstraint)
    this.textDiv.style.fontSize = Math.trunc(fontSize) + "px";
    this.textDiv.style.lineHeight = "1.3";
  }

  updateTitle(update_fontsize: boolean = false): void {
    this.titleDiv.innerText = this.model.title
    if (update_fontsize)
      this.updateTextFontSize()
  }

  updateValue(update_fontsize: boolean = false): void {
    this._value_format = this.model.formatter.doFormat([this.model.value], {loc: 0})[0]
    this.valueDiv.innerText = this._value_format
    if (update_fontsize)
      this.updateTextFontSize()
  }

  updateValue2(update_fontsize: boolean = false): void {
    this._value_change_format = this.model.change_formatter.doFormat([this.model.value_change], {loc: 0})[0]
    this.value2Div.innerText = this._value_change_format
    this.updateValueChange()
    if (update_fontsize)
      this.updateTextFontSize()
  }

  updateValueChange(): void {
    if (this.model.value_change > 0) {
      this.changeDiv.innerHTML = "&#9650;"
      this.changeDiv.style.color = this.model.pos_color
    }
    else if (this.model.value_change < 0) {
      this.changeDiv.innerHTML = "&#9660;"
      this.changeDiv.style.color = this.model.neg_color
    }
    else {
      this.changeDiv.innerHTML = "&nbsp;"
      this.changeDiv.style.color = "inherit"
    }
  }

  updateLayout(): void {
    if (this.model.layout === "column"){
      this.containerDiv.style.display = "block"
      this.textDiv.style.height = "50%";
      this.textDiv.style.width = "100%";
      this.plotDiv.style.height = "50%";
      this.plotDiv.style.width = "100%";
    } else {
      this.containerDiv.style.display = "flex"
      this.textDiv.style.height = "100%";
      this.textDiv.style.width = "";
      this.plotDiv.style.height = "100%";
      this.plotDiv.style.width = "";
      this.textDiv.style.flex = "1"
      this.plotDiv.style.flex = "1"
    }
    if (this._has_finished)
      this.invalidate_layout()
  }
}

export namespace TreeIndicator {
  export type Attrs = p.AttrsOf<Props>

  export type Props = HTMLBox.Props & {
    change_formatter: p.Property<TickFormatter>
    description: p.Property<string>
    formatter: p.Property<TickFormatter>
    layout: p.Property<string>
    source: p.Property<any>
    plot_x: p.Property<string>
    plot_y: p.Property<string>
    plot_color: p.Property<string>
    plot_type: p.Property<string>
    pos_color: p.Property<string>
    neg_color: p.Property<string>
    title: p.Property<string>
    value: p.Property<number>
    value_change: p.Property<number>
  }
}

export interface TreeIndicator extends TreeIndicator.Attrs { }

export class TreeIndicator extends HTMLBox {
  properties: TreeIndicator.Props

  constructor(attrs?: Partial<TreeIndicator.Attrs>) {
    super(attrs)
  }

  static __module__ = "panel.models.tree"

  static {
    this.prototype.default_view = TreeIndicatorView;

    this.define<TreeIndicator.Props>(({Number, String, Ref}) => ({
      description:  [ String,              "" ],
      formatter:        [ Ref(TickFormatter), () => new BasicTickFormatter() ],
      change_formatter: [ Ref(TickFormatter), () => new NumeralTickFormatter() ],
      layout:       [ String,        "column" ],
      source:       [ Ref(ColumnDataSource)   ],
      plot_x:       [ String,             "x" ],
      plot_y:       [ String,             "y" ],
      plot_color:   [ String,            blue ],
      plot_type:    [ String,           "bar" ],
      pos_color:    [ String,           green ],
      neg_color:    [ String,             red ],
      title:        [ String,              "" ],
      value:        [ Number,               0 ],
      value_change: [ Number,               0 ],
    }))
  }
}

panel/models/tree.py (same code as panel/models/trend.py)

"""
A Bokeh model indicating trends.
"""
from bokeh.core.properties import Float, Instance, String
from bokeh.models import (
    BasicTickFormatter, NumeralTickFormatter, TickFormatter,
)
from bokeh.models.sources import ColumnDataSource

from .layout import HTMLBox


class TreeIndicator(HTMLBox):
    """
    A Bokeh model indicating trends.
    """

    description = String()
    change_formatter = Instance(TickFormatter, default=lambda: NumeralTickFormatter(format='0.00%'))
    formatter = Instance(TickFormatter, default=lambda: BasicTickFormatter())
    layout = String()
    source = Instance(ColumnDataSource)
    plot_x = String()
    plot_y = String()
    plot_color = String()
    plot_type = String()
    pos_color = String()
    neg_color = String()
    title = String()
    value = Float()
    value_change = Float()

added export {TreeIndicator} from "./tree" to panel/models/index.ts

panel/widgets/tree.py (same code as the Trend class in panel/widgets/indicators.py except the _widget_type points to my TreeInidicator from above)

from __future__ import annotations

from typing import (
    TYPE_CHECKING, ClassVar, List, Mapping, Type,
)

import numpy as np
import param

from bokeh.models import ColumnDataSource

from ..models.tree import TreeIndicator as _BkTrendIndicator
from ..reactive import SyncableData
from ..util import updating
from .indicators import Indicator

if TYPE_CHECKING:
    from bokeh.model import Model

RED   = "#d9534f"
GREEN = "#5cb85c"
BLUE  = "#428bca"

class Custom(SyncableData, Indicator):
    data = param.Parameter(doc="""
      The plot data declared as a dictionary of arrays or a DataFrame.""")

    layout = param.ObjectSelector(default="column", objects=["column", "row"])

    plot_x = param.String(default="x", doc="""
      The name of the key in the plot_data to use on the x-axis.""")

    plot_y = param.String(default="y", doc="""
      The name of the key in the plot_data to use on the y-axis.""")

    plot_color = param.String(default=BLUE, doc="""
      The color to use in the plot.""")

    plot_type = param.ObjectSelector(default="bar", objects=["line", "step", "area", "bar"], doc="""
      The plot type to render the plot data as.""")

    pos_color = param.String(GREEN, doc="""
      The color used to indicate a positive change.""")

    neg_color = param.String(RED, doc="""
      The color used to indicate a negative change.""")

    sizing_mode = param.ObjectSelector(default=None, objects=[
        'fixed', 'stretch_width', 'stretch_height', 'stretch_both',
        'scale_width', 'scale_height', 'scale_both', None])

    title = param.String(doc="""The title or a short description of the card""")

    value = param.Parameter(default='auto', doc="""
      The primary value to be displayed.""")

    value_change = param.Parameter(default='auto', doc="""
      A secondary value. For example the change in percent.""")

    _data_params: ClassVar[List[str]] = ['data']

    _manual_params: ClassVar[List[str]] = ['data']

    _rename: ClassVar[Mapping[str, str | None]] = {
        'data': None, 'name': 'name', 'selection': None
    }

    _widget_type: ClassVar[Type[Model]] = _BkTrendIndicator

    def _get_data(self):
        if self.data is None:
            return None, {self.plot_x: [], self.plot_y: []}
        elif isinstance(self.data, dict):
            return self.data, self.data
        return self.data, ColumnDataSource.from_df(self.data)

    def _init_params(self):
        props = super()._init_params()
        self._processed, self._data = self._get_data()
        props['source'] = ColumnDataSource(data=self._data)
        return props

    def _trigger_auto_values(self):
        trigger = []
        if self.value == 'auto':
            trigger.append('value')
        if self.value_change == 'auto':
            trigger.append('value_change')
        if trigger:
            self.param.trigger(*trigger)

    @updating
    def _stream(self, stream, rollover=None):
        self._trigger_auto_values()
        super()._stream(stream, rollover)

    def _update_cds(self, *events):
        super()._update_cds(*events)
        self._trigger_auto_values()

    def _update_data(self, data):
        if isinstance(data, _BkTrendIndicator):
            return
        super()._update_data(data)

    def _process_param_change(self, msg):
        msg = super()._process_param_change(msg)
        ys = self._data.get(self.plot_y, [])
        if 'value' in msg and msg['value'] == 'auto':
            if len(ys):
                msg['value'] = ys[-1]
            else:
                msg['value'] = 0
        if 'value_change' in msg and msg['value_change'] == 'auto':
            if len(ys) > 1:
                y1, y2 = self._data.get(self.plot_y)[-2:]
                msg['value_change'] = 0 if y1 == 0 else (y2/y1 - 1)
            else:
                msg['value_change'] = 0
        return msg

and finally app.py

import panel as pn

pn.Column(pn.widgets.Custom(name='Name')).servable()

When I run panel serve on this after running python setup.pu develop, the broswer console gives me “error: could not resolve type ‘panel.models.tree.TreeIndicator’, which could be due to a widget or a custom model not being registered before first usage”

What am I missing?

Maybe it would be better to just jump to working with what I have with jstree. I figured some stuff out. I got the jstree panel model to work properly (with the typescript code in a seperate file and running panel build src/panel_jstree in the panel_jstree repo), and the I can run an app with jstree, blah blah, but only on panel<1. On panel>1 and bokeh>3 (just tested with panel==1.0.0rc9 and bokeh==3.1.1) I get the same error as above “could not resolve type ‘panel_jstree.bokeh_extensions.jstree.jsTreePlot’, which could be due to a widget or a custom model not being registered before first usage”

Any ideas why this isn’t working on panel>1?

edit: I figured out that I hadn’t updated the bokehjs and the paneljs versions in the package.json file. It is closer to working now. I am getting past the previous error, but now the properties on my model are each raising an error when used, which wasn’t happening in panel<1/bokeh<3. I will get errors like “error: unknown property panel_jstree.bokeh_extensions.jstree.jsTreePlot._new_nodes”. Even though that is defined in the model .ts file, the model .py and the widget .py file, (and it was working in panel<1). I will investigate further and push my changes if anyone wants to help. I would like to add this to panel at some point.

This is one of the more advanced parts of Panel. A guide about it can be found here: panel/Developing_Custom_Models.ipynb at main · holoviz/panel · GitHub, though it is a bit outdated, so try to follow the thoughts and take a look at the code in Panel repo itself.

The panel_jstree.bokeh_extensions.jstree.jsTreePlot’, which could be due to a widget or a custom model not being registered before first usage is because the model hasn’t been registered and can be done by panel build . or python -m pip install -e ., where the later is only needed the first time. In Panel 1.0 you also need to run this panel bundle --all --verbose once.

@Hoxbro I’ve ran those commands and I still can’t get it to work (and had already read through that page). I was wondering if it would be helpful if I created a draft PR into panel as I work on this to get help.

Also if I follow the developers guide linked above (and change some of the imports for panel>1 like HTMLBox now is in panel/model/layouts instead of from bokeh), even on the simplest example, I get the exact same error (after running panel bundle --all --verbose and panel build panel). Even on the simplest example that is just a button, and it isn’t clear why it doens’t work. Like a lot of the panel models don’t have `javascript attributes, and are as simple, and I can’t tell what from the source code would make the ones in panel bundle right but not this.

Two things you could try:

  1. Have you pip installed with export SETUPTOOLS_ENABLE_FEATURES=legacy-editable?
  2. Try to use export BOKEH_RESOURCES=server when running panel serve.

I think I got that part resolved. I didn’t have caching disabled. Now I learned that lesson. It is not throwing errors now, but it isn’t working still, where it workings in panel<1. I posted a question about it in gitter and discord, because I think it will be a more debugging, but I will ask here too.

What is happening is that it won’t create the jstree object at all anymore, and therefore it doesn’t show up.

The way jstree works is that you give a div an id and then the constructor with that div id like

$('#tree').jstree({
'core' : {
  'data' : {
    'url' : function (node) {
      return node.id === '#' ?
        'ajax_roots.json' :
        'ajax_children.json';
    },
    'data' : function (node) {
      return { 'id' : node.id };
    }
  }
});

But in panel>1/bokeh>3 that isn’t working with the divs created by bokeh’s div constructor. I don’t know why. I don’t do that much front end stuff. It works if you are in the browser console and you use the id of one of the outermost divs. It also works in panel<1. It seems like the elements from bokeh now are shadow elements? It seems like maybe those are the ones that jstree won’t attach too maybe?

I have a repo for it if anyone wants to look (current branch I am working on to get to panel 1.0): GitHub - madeline-scyphers/panel-jstree at feature/panel-1.0

and an issue explaining all of this: panel>1 jstree not doing anything when called on div id · Issue #2 · madeline-scyphers/panel-jstree · GitHub