App load time is 7s with Panel servable(), how to start optimizing it?

[SteveAKopias]

My first Panel/Holoviews/Datashader app is almost ready to deploy. Except, that it takes about 8s for the site to load. Chrome showed it is a 6.9kB file, so it was transferred in 3ms, but indeed it was waiting for the server for 7.7s. So I jumped into the code and quickly printed out timelogs. Turns out the whole code is executed in about a half second, and then the final servable() takes about 7s. As a next step I started commenting out bits and pieces and I found a block that is responsible for about 6.8s, namely this:

combined = (
        hv.DynamicMap(get_tiles)

        * hd.regrid(hv.DynamicMap(get_image_01)).apply.opts(cmap=pn_01_cmap, alpha=pn_01_alpha) 
        * hd.regrid(hv.DynamicMap(get_image_02)).apply.opts(cmap=pn_02_cmap, alpha=pn_02_alpha) 
        * hd.regrid(hv.DynamicMap(get_image_03)).apply.opts(cmap=pn_03_cmap, alpha=pn_03_alpha) 
        * hd.regrid(hv.DynamicMap(get_image_04)).apply.opts(cmap=pn_04_cmap, alpha=pn_04_alpha)
        * hd.regrid(hv.DynamicMap(get_image_05)).apply.opts(cmap=pn_05_cmap, alpha=pn_05_alpha)
        * hd.regrid(hv.DynamicMap(get_image_06)).apply.opts(cmap=pn_06_cmap, alpha=pn_06_alpha) 
        * hd.regrid(hv.DynamicMap(get_image_07)).apply.opts(cmap=pn_07_cmap, alpha=pn_07_alpha)
        * hd.regrid(hv.DynamicMap(get_image_08)).apply.opts(cmap=pn_08_cmap, alpha=pn_08_alpha)

        * hd.regrid(hv.DynamicMap(get_white_bg_for_points)).options(cmap=['white']).apply.opts(alpha=pn_white_bg_for_points_alpha)
        * points_aggregated_categorical
        * points_aggregated_for_hover
        * hv_points
        * hd.regrid(hv.DynamicMap(get_image_for_hover_tooltip))
)

Now the first interesting thing is that this is not even the most difficult part of the code. The whole application is practically a map-viewer app, where there is a widgetbar with copious amount of widgets the user can use to turn on/off various layers and customize their alpha and opacity and there is the map with the overlaid layers. Now the widgetbar with all the widgets is processed in no time, and this is the map part itself that’s causing the trouble.

The second interesting thing is that if I comment out all the elements inside combined and uncomment them one by one, each of them needs about 0.2s processing on average. Not ideal bar far, but it gets worse, if I enable more of them at the same time, they need about 0.3s each. So two layers that need 0.2s each alone together need 0.6s.

I can’t provide a fully working example to tweak it as it is a pretty big project with pretty big dataset.

But even without taking any code into consideration the biggest help for me would be any kind of answers (tips, pointers, directions) to these questions:

• What does actually servable() do? I assume it processes the data to be able to be displayed later, but where can I find any info on what its method is to be able to figure out what can it be that occupied with?
• Is it possible to somehow save or cache the result of servable? Assuming the code rarely changes, whatever happens in servable() for 7s, it seems weird that it would happen on every single page load as it should create the exact same result from processing the exact same code every time.
• Is there any kind of documentation on how to optimize the speed of Panel applications?

[SteveAKopias]

I don’t expect anybody to be able to debug this without a working code, but maybe something obvious jumps out from this for somebody. Most layers above use a similar way to load the image based on a few widget values, like this:

@pn.depends(
    pn_layer_is_active=pn_is_active['01'],
    pn_resolution=pn_resolutions.param.value,
    pn_date=pn_dates.param.value_throttled,
    pn_01_type=pn_01_types.param.value,
)
def get_image_01(pn_layer_is_active, pn_resolution, pn_date, pn_01_type):
    ret = get_image_generic('01', pn_layer_is_active, pn_resolution, pn_date, pn_01_type)
    return ret

def get_image_generic(category_name, pn_layer_is_active, pn_resolution, pn_date=None, subcategory_name='-'):
    layer_is_active = int(pn_layer_is_active)
    customn_js_hover_tool = get_export_hover_tool(category_name)

    if layer_is_active:
        subcategory_name = subcategory_name.lower()
        if pn_date:
            this_image = load_image(pn_resolution=pn_resolution, category_name=category_name, subcategory_name=subcategory_name, pn_date=pn_date)
        else:
            this_image = load_image(pn_resolution=pn_resolution, category_name=category_name, subcategory_name=subcategory_name)

        used_logz = source_data[category_name][subcategory_name]['logz'] if 'logz' in source_data[category_name][subcategory_name] else False
        used_min_val = source_data[category_name][subcategory_name]['min']
        used_max_val = source_data[category_name][subcategory_name]['max']

        this_image.opts(
            logz=used_logz,
            clim=(used_min_val, used_max_val),
            tools=[customn_js_hover_tool]
        )
    else:
        this_image = image_filled_with_nan.options(alpha=0, colorbar=False, tools=[customn_js_hover_tool])
    return this_image

def get_export_hover_tool(label):
    custom_tooltips = [
        ('Export Tooltip [' + str(label) + ']', '@image{custom}'),
    ]

    code = """
        var label = "**label**";
        if (typeof(window['saved_tooltips'])=="undefined")
        {
        window['saved_tooltips'] = {};
        }
        window['saved_tooltips'][label] = value;
        return value;
        """
    code = code.replace('**label**', label)

    custom_formatters = {
        '@image': bk.models.CustomJSHover(code=code),
    }
    this_hover_export = bk.models.HoverTool(tooltips=custom_tooltips, formatters=custom_formatters)
    return this_hover_export

[philippjfr]

I suspect a lot of time is spent on the numba JIT compilation of the calls into datashader but can’t be entirely sure. You could run a profiler in a notebook to see what takes so long. In a notebook you could do:

%%prun
hv.render(combined)

There may be some way to cache the compiled numba code (see Notes on Caching — Numba 0.50.1 documentation). Alternatively I can only suggest rendering the app without that component, adding a loading indicator and then using pn.state.onload to schedule a callback which renders the map once the page itself has rendered.

[SteveAKopias]

Thanks, definitely will try to profile it today and look up the numba caching.

Meanwile, how should I start with pn.state.onload? The only examples I found are these:

# https://panel.holoviz.org/user_guide/Deploy_and_Export.html
def app():
    widget = pn.widgets.Select()
    def on_load():
        time.sleep(1) # Emulate some long running process
        widget.options = ['A', 'B', 'C']
    pn.state.onload(on_load)
    return widget

# https://panel.holoviz.org/gallery/simple/defer_data_load.html
select_ticker = pn.widgets.Select(name='Stock Ticker')

def load_data():
    if 'stocks' not in pn.state.cache: 
        pn.state.cache['stocks'] = df = pd.read_csv(
        stocks_url, parse_dates=['date']).set_index('symbol')
    else:
        df = pn.state.cache['stocks']
    symbols = list(df.index.unique())
    select_ticker.options = symbols
    select_ticker.value = symbols[0]
    
pn.state.onload(load_data)

Both of them only change some parameters of an existing widget. Meanwhile, what I need is manually update the list of elements in an Overlay (combined). I’ve tried generating a generating an initial combined with 2 elements only and then a new combined on load, but the onload either runs immediately if I call it before displaying combined, or if called after combined is already displayed it does not refresh it.
I created a simplified example here:
https://colab.research.google.com/drive/1roR6vAfXC6tTwkbp1Umx-vWOkz7rzWgJ?usp=sharing

So I assume I misunderstood how on_load works and I would instead need something like this mockup code where #? and all caps shows fake code that does not exist in this form :

combined = (hv.DynamicMap(get_tiles)  * hd.regrid(initial_content))

def load_data():
    IF 'intitial_content' IN combined:                                   # ?
        combined.REMOVE_LAYER('initial_content')                         # ??

    if 'image01' not in pn.state.cache: 
        pn.state.cache['image01'] = get_image_01()
    image01 = pn.state.cache['image01']
    combined.ADD_LAYER(                                                  # ???
        hd.regrid(hv.DynamicMap(image01)).apply.opts(
            cmap=pn_01_cmap, alpha=pn_01_alpha)
   )

    if 'image02' not in pn.state.cache: 
        pn.state.cache['image02'] = get_image_02()
    image02 = pn.state.cache['image02']
    combined.ADD_LAYER(                                                  # ???
        hd.regrid(hv.DynamicMap(image02)).apply.opts(
            cmap=pn_02_cmap, alpha=pn_02_alpha)
   )
    
pn.state.onload(load_data)

So the biggest question here, I think, is how to add or remove elements to and from an Overlay composite to be able to update it onload? Or maybe I misunderstood how onload works completely, in that case what would be the recommended way to do this?

[Marc]

Hi @SteveAKopias

The key is to update a Panel component onload. For example something like

from time import sleep

import holoviews as hv
import holoviews.operation.datashader as hd
import numpy as np
import panel as pn


pn.extension(sizing_mode="stretch_width")
hv.extension("bokeh", logo=False)

left, right, top, bottom = 3490000, 3510000, 3490000, 3510000

pn_tiles = pn.widgets.Select(name="pn_tiles", options=["OSM", "CartoDark"], value="OSM")


@pn.depends(pn_tile=pn_tiles.param.value)
def get_tiles(pn_tile):
    return getattr(hv.element.tiles, pn_tile)().options(
        xaxis=None, yaxis=None, border=0, margin=0, responsive=True, height=600
    )


# Image content depends on Select, cmap depends on another Select
pn_image01_contents = pn.widgets.Select(
    name="pn_image01_contents", options=["1000", "2000"], value="1000"
)
pn_image01_cmaps = pn.widgets.Select(
    name="pn_image01_cmaps", options=["blues", "reds", "greens"], value="blues"
)


@pn.depends(pn_image01_content=pn_image01_contents.param.value)
def get_image01(pn_image01_content):
    return hv.Image(
        np.ones((100, 100)),
        bounds=(
            left + 2000 + int(pn_image01_content),
            bottom,
            right,
            top + 2000 + int(pn_image01_content),
        ),
    ).opts(alpha=0.6)


# Image content depends on Select, cmap depends on another Select
pn_image02_contents = pn.widgets.Select(
    name="pn_image02_contents", options=["1000", "2000"], value="1000"
)
pn_image02_cmaps = pn.widgets.Select(
    name="pn_image02_cmaps", options=["blues", "reds", "greens"], value="reds"
)


@pn.depends(pn_image02_content=pn_image02_contents.param.value)
def get_image02(pn_image02_content):
    return hv.Image(
        np.ones((100, 100)),
        bounds=(
            left,
            bottom - 2000 - int(pn_image02_content),
            right - 2000 - int(pn_image02_content),
            top,
        ),
    )


def get_combined():
    sleep(3)
    return (
        hv.DynamicMap(get_tiles)
        * hd.regrid(hv.DynamicMap(get_image01)).apply.opts(cmap=pn_image01_cmaps)
        * hd.regrid(hv.DynamicMap(get_image02)).apply.opts(cmap=pn_image02_cmaps)
    ).opts(responsive=True)


plot_container = pn.pane.HoloViews(sizing_mode="stretch_both")


def update_plot_container():
    settings.loading = True
    plot_container.object = get_combined()
    settings.loading = False


pn.state.onload(update_plot_container)

settings = pn.Column(
    pn_tiles,
    pn_image01_contents,
    pn_image01_cmaps,
    pn_image02_contents,
    pn_image02_cmaps,
)
settings.loading = True

pn.template.FastListTemplate(
    site="Panel", title="Deferred loading", sidebar=[settings], main=[plot_container]
).servable()

ps. Also to get Colab working you need to run pn.extension(comms='colab'). C.f. GitHub - holoviz/panel: A high-level app and dashboarding solution for Python.

[Marc]

Regarding caching you can use pn.state.cache. Its simple dictionary that you can store things in. If you want to persist the cache I would suggest you take a look at Python DiskCache. It works really well with Panel.

[SteveAKopias]

@philippjfr

%%prun
hv.render(combined)

Finally, I’ve managed to try this out, first 200 line:

ncalls  tottime  percall  cumtime  percall filename:lineno(function)
   948028   10.792    0.000   57.044    0.000 parameterized.py:1277(_setup_params)
   603265   10.706    0.000   10.706    0.000 {method 'reduce' of 'numpy.ufunc' objects}
  3431699   10.326    0.000   19.883    0.000 parameterized.py:859(__set__)
 16858798    8.962    0.000   11.676    0.000 parameterized.py:837(__get__)
41508293/41458385    8.725    0.000   10.159    0.000 {built-in method builtins.isinstance}
 10091811    8.496    0.000   11.513    0.000 parameterized.py:2544(param)
   948028    7.126    0.000   90.773    0.000 parameterized.py:2506(__init__)
 33310420    5.577    0.000    5.577    0.000 {method 'get' of 'dict' objects}
18704698/15794852    5.539    0.000   42.819    0.000 {built-in method builtins.getattr}
1560937/1467108    5.120    0.000    9.164    0.000 copy.py:128(deepcopy)
3441379/3431699    4.360    0.000   25.471    0.000 parameterized.py:313(_f)
  2413554    3.951    0.000    7.488    0.000 parameterized.py:2278(get_param_descriptor)
  1569222    3.906    0.000   17.021    0.000 bases.py:328(prepare_value)
2849141/1901113    3.832    0.000   77.730    0.000 parameterized.py:1060(override_initialization)
   492002    3.690    0.000   10.406    0.000 util.py:874(isfinite)
   496897    3.633    0.000   64.395    0.000 options.py:466(__init__)
   512166    3.612    0.000    9.323    0.000 dimension.py:606(matches)
  4364549    3.373    0.000    4.330    0.000 parameterized.py:160(classlist)
   948028    3.337    0.000   16.867    0.000 parameterized.py:1271(_generate_name)
  1467108    3.103    0.000   12.784    0.000 parameterized.py:1346(_instantiate_param)
1384402/1320516    3.040    0.000    7.782    0.000 tree.py:216(__setattr__)
 10091811    3.015    0.000    3.015    0.000 parameterized.py:1142(__init__)
 11584572    2.806    0.000    2.847    0.000 {built-in method builtins.issubclass}
  1581411    2.798    0.000    3.671    0.000 parameterized.py:1480(objects)
  1099948    2.772    0.000    8.139    0.000 parameterized.py:1745(get_value_generator)
13167827/13167294    2.697    0.000    3.362    0.000 {built-in method builtins.hasattr}
2417128/2082328    2.676    0.000    4.510    0.000 model.py:824(_visit_value_and_its_immediate_references)
  2806210    2.600    0.000    3.735    0.000 util.py:374(tree_attribute)
  2240461    2.374    0.000    5.459    0.000 util.py:737(__call__)
   305405    2.298    0.000   13.118    0.000 parameterized.py:1694(get_param_values)
1513795/1491575    2.236    0.000   28.916    0.000 bases.py:182(themed_default)
5295758/5295682    2.228    0.000   24.014    0.000 {built-in method builtins.setattr}
1491402/1469182    2.194    0.000   33.908    0.000 descriptors.py:704(_get_default)
322200/80550    2.106    0.000   37.498    0.000 options.py:772(options)
   832218    2.044    0.000    3.055    0.000 container.py:178(validate)
   563850    2.035    0.000    3.866    0.000 options.py:745(<genexpr>)
2275396/2253176    1.977    0.000   36.326    0.000 descriptors.py:676(_get)
1608012/1590189    1.945    0.000   12.129    0.000 {built-in method builtins.any}
  9115003    1.883    0.000    1.883    0.000 {method 'endswith' of 'str' objects}
   341962    1.785    0.000   43.215    0.000 model.py:808(_visit_immediate_value_references)
   422440    1.683    0.000    2.870    0.000 functools.py:35(update_wrapper)
   396388    1.681    0.000    2.744    0.000 __init__.py:1398(_check_type)
  2015339    1.678    0.000    5.860    0.000 {built-in method builtins.sorted}
  1512075    1.620    0.000    3.875    0.000 copy.py:66(copy)
1491402/1469182    1.597    0.000   30.865    0.000 descriptors.py:584(instance_default)
  3625810    1.399    0.000    1.399    0.000 util.py:690(<genexpr>)
   145204    1.348    0.000   11.893    0.000 util.py:965(max_range)
2403531/2359227    1.335    0.000   37.331    0.000 descriptors.py:464(__get__)
  7481724    1.302    0.000    1.302    0.000 {method 'items' of 'dict' objects}
   771978    1.270    0.000    1.514    0.000 copy.py:242(_keep_alive)
1513795/1491575    1.250    0.000    8.360    0.000 bases.py:161(_copy_default)
   161100    1.200    0.000    7.717    0.000 options.py:733(find)
  1894502    1.197    0.000    1.393    0.000 parameterized.py:1020(_validate)
   409525    1.149    0.000    3.091    0.000 options.py:548(__getitem__)
247504/236040    1.145    0.000    1.233    0.000 {built-in method numpy.array}
   953085    1.140    0.000    9.215    0.000 parameterized.py:1265(_set_name)
    75033    1.096    0.000   46.657    0.001 options.py:82(lookup_options)
344844/48153    1.092    0.000   32.915    0.001 dimension.py:637(traverse)
  1149436    1.087    0.000    1.087    0.000 wrappers.py:138(__init__)
  1643709    1.069    0.000    1.069    0.000 has_props.py:228(accumulate_dict_from_superclasses)
  5663596    1.066    0.000    1.448    0.000 parameterized.py:1229(__iter__)
5118740/4744552    1.032    0.000    1.329    0.000 {built-in method builtins.len}
    80880    1.029    0.000    1.996    0.000 dataset.py:1300(_construct_dataarray)
   140490    1.021    0.000    6.909    0.000 nanfunctions.py:228(nanmin)
  3058732    1.021    0.000    1.497    0.000 has_props.py:664(themed_values)
  4750747    0.962    0.000    0.962    0.000 parameterized.py:2148(_get_param)
  4364549    0.956    0.000    0.956    0.000 inspect.py:488(getmro)
   751468    0.935    0.000    2.514    0.000 container.py:189(wrap)
   422440    0.930    0.000    4.091    0.000 parameterized.py:273(__get__)
  1666596    0.926    0.000    3.480    0.000 dimension.py:632(<genexpr>)
   109200    0.920    0.000   31.792    0.000 options.py:639(_merge_options)
1478897/1465907    0.909    0.000    9.200    0.000 bases.py:172(_raw_default)
   417291    0.907    0.000    1.762    0.000 container.py:74(validate)
     1844    0.901    0.000   44.685    0.024 model.py:55(collect_filtered_models)
   327210    0.898    0.000   27.711    0.000 options.py:261(__init__)
1782669/1781100    0.897    0.000    2.563    0.000 {built-in method builtins.all}
   461446    0.887    0.000    4.312    0.000 streams.py:795(contents)
   267036    0.877    0.000    1.948    0.000 version.py:309(parse)
   492002    0.874    0.000    0.977    0.000 util.py:1513(is_dask_array)
318591/20537    0.870    0.000    1.443    0.000 spaces.py:759(get_nested_dmaps)
     1872    0.862    0.000   54.575    0.029 plot.py:717(_compute_group_range)
   505318    0.861    0.000    1.021    0.000 __init__.py:964(_validate)
   625597    0.860    0.000    3.561    0.000 dimension.py:359(__eq__)
    44265    0.859    0.000   17.105    0.000 util.py:1708(stream_parameters)
   190785    0.833    0.000    2.593    0.000 warnings.py:130(filterwarnings)
45689/34643    0.826    0.000   67.946    0.002 __init__.py:196(pipelined_fn)
   140490    0.818    0.000    6.911    0.000 nanfunctions.py:343(nanmax)
   362581    0.799    0.000    3.496    0.000 util.py:681(allowable)
   292148    0.792    0.000   14.013    0.000 streams.py:400(contents)
   396847    0.791    0.000    3.681    0.000 __init__.py:1372(_validate)
587136/586286    0.791    0.000    3.960    0.000 has_props.py:273(__setattr__)
   267038    0.790    0.000    0.790    0.000 {method 'split' of 're.Pattern' objects}
   612493    0.769    0.000    1.046    0.000 __init__.py:1316(_validate)
  1513795    0.748    0.000    1.731    0.000 has_props.py:593(_overridden_defaults)
  3948188    0.748    0.000    0.748    0.000 {built-in method builtins.id}
  1031884    0.746    0.000    1.977    0.000 dimension.py:302(spec)
   177468    0.744    0.000    0.744    0.000 {built-in method _warnings.warn}
    63920    0.725    0.000   17.705    0.000 dimension.py:491(__init__)
  2543132    0.711    0.000    0.711    0.000 {method 'get' of 'mappingproxy' objects}
    80220    0.709    0.000   12.993    0.000 tree.py:231(__getattr__)
  5612420    0.679    0.000    0.679    0.000 {method 'isupper' of 'str' objects}
   461446    0.676    0.000    0.934    0.000 streams.py:800(<dictcomp>)
   751492    0.666    0.000    1.376    0.000 wrappers.py:289(__init__)
    80550    0.654    0.000   41.306    0.001 options.py:755(closest)
581840/573650    0.654    0.000    1.063    0.000 copy.py:200(_deepcopy_list)
  2043741    0.648    0.000    0.972    0.000 options.py:546(<genexpr>)
   109755    0.623    0.000   11.532    0.000 options.py:516(__call__)
160847/87021    0.601    0.000   35.473    0.000 tree.py:106(set_path)
   137793    0.589    0.000    1.542    0.000 dimension.py:901(dimensions)
   402506    0.582    0.000    0.888    0.000 options.py:561(<dictcomp>)
62976/28308    0.565    0.000   29.536    0.001 dimension.py:1350(__init__)
   292148    0.563    0.000    0.743    0.000 streams.py:403(<dictcomp>)
   145204    0.554    0.000    0.772    0.000 util.py:981(<listcomp>)
  2838602    0.550    0.000    0.550    0.000 {method 'append' of 'list' objects}
   466812    0.548    0.000    0.892    0.000 model.py:82(queue_one)
  3431699    0.533    0.000    0.533    0.000 parameterized.py:945(_post_setter)
    63920    0.527    0.000   19.419    0.000 dimension.py:849(__init__)
    76528    0.525    0.000    0.535    0.000 util.py:1124(int_to_roman)
104381/56518    0.508    0.000   34.599    0.001 tree.py:44(__init__)
428397/365660    0.501    0.000    1.864    0.000 bases.py:304(is_valid)
   518221    0.496    0.000    0.957    0.000 numeric.py:1816(isscalar)
    27300    0.496    0.000   33.330    0.001 options.py:697(__setattr__)
   402506    0.496    0.000    0.808    0.000 options.py:556(<dictcomp>)
  2951310    0.495    0.000    0.495    0.000 {method 'upper' of 'str' objects}
   419945    0.494    0.000    0.961    0.000 bases.py:502(validate)
   399439    0.491    0.000    1.328    0.000 container.py:121(wrap)
   461446    0.488    0.000    2.088    0.000 streams.py:799(<dictcomp>)
   281276    0.477    0.000    1.364    0.000 {method 'any' of 'numpy.generic' objects}
   190787    0.457    0.000    0.907    0.000 warnings.py:181(_add_filter)
374652/322281    0.451    0.000   14.993    0.000 {built-in method numpy.core._multiarray_umath.implement_array_function}
170930/159972    0.447    0.000    2.678    0.000 either.py:104(validate)
   109200    0.438    0.000    9.688    0.000 options.py:269(__add__)
   905538    0.435    0.000    0.476    0.000 util.py:1856(<genexpr>)
    61752    0.432    0.000    6.894    0.000 options.py:504(filtered)
   982908    0.432    0.000    0.583    0.000 generic.py:30(_check)
   264270    0.428    0.000    2.561    0.000 version.py:333(_cmp)
   588753    0.427    0.000    0.538    0.000 {built-in method _abc._abc_instancecheck}
   408515    0.424    0.000    1.903    0.000 options.py:543(cyclic)
    25704    0.423    0.000   31.461    0.001 plot.py:1416(get_extents)
   160847    0.418    0.000    4.721    0.000 tree.py:113(<listcomp>)
  1997423    0.397    0.000    0.397    0.000 bases.py:534(validation_on)
    51047    0.394    0.000   14.339    0.000 util.py:1043(dimension_range)
  1240700    0.390    0.000    0.390    0.000 model.py:245(id)
   165195    0.389    0.000    0.501    0.000 tree.py:177(__getitem__)
   205888    0.377    0.000    1.227    0.000 __init__.py:772(__get__)
    90406    0.363    0.000    1.041    0.000 overlay.py:221(group)
   193986    0.363    0.000    0.611    0.000 re.py:289(_compile)
   408515    0.360    0.000    5.345    0.000 options.py:565(options)
   190787    0.355    0.000    0.395    0.000 warnings.py:458(__enter__)
   397944    0.350    0.000    0.728    0.000 wrappers.py:197(__init__)
  1433003    0.343    0.000    0.343    0.000 parameterized.py:1182(self_or_cls)
    21182    0.340    0.000   10.916    0.001 variable.py:1685(reduce)
   341962    0.338    0.000    0.541    0.000 query.py:90(match)
   192381    0.332    0.000    0.332    0.000 {method 'remove' of 'list' objects}
   399229    0.331    0.000    0.331    0.000 has_props.py:205(accumulate_from_superclasses)
   496897    0.330    0.000    0.330    0.000 options.py:488(<listcomp>)
    86669    0.329    0.000    3.524    0.000 dimension.py:950(get_dimension)
    61752    0.320    0.000    0.320    0.000 options.py:510(<dictcomp>)
    53901    0.317    0.000    0.548    0.000 parameterized.py:2570(__setstate__)
    80550    0.315    0.000   41.699    0.001 options.py:1255(lookup_options)
    59854    0.310    0.000   13.043    0.000 parameterized.py:2995(instance)
   226468    0.310    0.000    1.201    0.000 util.py:836(asarray)
  1945636    0.307    0.000    0.307    0.000 bases.py:281(validate)
  1100137    0.302    0.000    0.302    0.000 {method 'startswith' of 'str' objects}
574591/533531    0.300    0.000    2.066    0.000 either.py:107(<genexpr>)
    59288    0.300    0.000    1.013    0.000 settings.py:290(__call__)
   178159    0.292    0.000    0.793    0.000 options.py:669(__getitem__)
   422440    0.291    0.000    0.291    0.000 functools.py:65(wraps)
174064/87032    0.290    0.000    0.396    0.000 tree.py:70(path)
   267036    0.281    0.000    0.281    0.000 version.py:314(<listcomp>)
   718734    0.275    0.000    0.471    0.000 util.py:982(<genexpr>)
14560/455    0.271    0.000  124.768    0.274 spaces.py:1087(_execute_callback)
   545465    0.270    0.000    0.274    0.000 common.py:262(__setattr__)
   190787    0.267    0.000    0.302    0.000 warnings.py:477(__exit__)
   108632    0.266    0.000    0.549    0.000 dataarray.py:370(__init__)
   588753    0.265    0.000    0.803    0.000 abc.py:96(__instancecheck__)
    59276    0.262    0.000    1.305    0.000 serialization.py:239(make_id)
   260590    0.259    0.000    2.778    0.000 version.py:69(__ge__)
    67428    0.257    0.000    1.447    0.000 util.py:1850(make_path_unique)
   283617    0.257    0.000    0.360    0.000 <frozen importlib._bootstrap>:398(parent)
   205888    0.257    0.000    0.478    0.000 __init__.py:600(__get__)
119769/118859    0.256    0.000    1.171    0.000 copy.py:258(_reconstruct)
    90436    0.256    0.000    0.494    0.000 overlay.py:250(<setcomp>)
   264612    0.250    0.000    0.675    0.000 missing.py:133(_isna)
   108540    0.249    0.000    0.249    0.000 util.py:2014(is_nan)
14560/455    0.248    0.000  126.349    0.278 spaces.py:1280(__getitem__)
   303104    0.245    0.000    0.245    0.000 util.py:1080(<genexpr>)
    59252    0.243    0.000    0.357    0.000 has_props.py:676(apply_theme)
   246548    0.239    0.000    0.333    0.000 {method 'join' of 'str' objects}
   226544    0.238    0.000    0.348    0.000 tree.py:269(__iter__)
  1479380    0.234    0.000    0.234    0.000 bases.py:269(transform)
     5532    0.229    0.000    0.229    0.000 {built-in method nt.stat}
174952/101126    0.228    0.000    0.228    0.000 tree.py:143(_propagate)
62976/28308    0.227    0.000   29.644    0.001 overlay.py:139(__init__)
    36431    0.226    0.000    0.467    0.000 parameterized.py:1237(__contains__)
  1211634    0.225    0.000    0.225    0.000 {method 'keys' of 'dict' objects}
   137338    0.225    0.000    0.761    0.000 dimension.py:933(<listcomp>)
   316922    0.223    0.000    0.433    0.000 options.py:293(__bool__)
   292148    0.223    0.000    0.223    0.000 streams.py:402(<dictcomp>)
   119769    0.221    0.000    1.022    0.000 {method '__reduce_ex__' of 'object' objects}

Will experiment with this, but at first glance these ncalls values themselves look a bit suspicious…

[Marc]

Thanks for staying on this @SteveAKopias

[philippjfr]

@SteveAKopias That indeed looks quite suspicious. It seems like it’s creating a lot of Parameterized objects, e.g. spending 8.725 seconds solely on isinstance calls seems crazy. Not quite sure where to go from here, but I’ll think about it.

[SteveAKopias]

@Marc : Well, I kinda have to if I want to make it work :D, but thank you both for being with me on this.

@philippjfr : Yeah, that’s what I figured. Finally, I can afford to spend some time on this, so I will experiment with it and also try to create a dumbed down reproducible version.

[SteveAKopias]

I’ve just cleaned and restructured the code that was in pretty “rapid development” shape, a little bit hoping it would solve the issue in itself, no such luck, so continuing with reducing the script to a simpler form.

Meanwhile, I realized that the runtime grows every time I rerun the display(combined) cell or anything similar:

# After full kernel restart and running all the cells above once, then only running this cell:
# %%timeit -n 1 -r 1
# display(pn.Row(content, widgets))
# 1st run: 8.41 s
# 2nd run: 8.94 s # tested multiple times, the 2nd run does not increase by much
# 3rd run: 12.3 s # all the consecutive runs do
# 4th run: 16.2 s
# 5th run: 20.8 s
# 6th run: 23.9 s

# After full kernel restart and running all the cells above once, then only running this cell:
# %%timeit -n 1 -r 1
# hv.render(combined)
# 1st run: 15.6 s
# 2nd run: 24.5 s
# 3rd run: 38.4 s
# 4th run: 50.7 s

So I’m pretty sure it affected the prun result above too. Now I every restart the kernel every time before running the measured cell, here are the new results. Not that they are good, but at least now I know they are precise:

# %%prun -l 200
# display(pn.Row(content, widgets))

# After full kernel restart and running all the cells above once, then only running this cell:

output = """
         19735172 function calls (19043317 primitive calls) in 14.769 seconds

   Ordered by: internal time
   List reduced from 6403 to 200 due to restriction <200>

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
     8452    0.842    0.000    1.009    0.000 ffi.py:149(__call__)
2441815/2438121    0.399    0.000    0.481    0.000 {built-in method builtins.isinstance}
    32709    0.337    0.000    1.702    0.000 parameterized.py:1277(_setup_params)
   121773    0.325    0.000    0.548    0.000 parameterized.py:859(__set__)
   606066    0.291    0.000    0.375    0.000 parameterized.py:837(__get__)
    23370    0.291    0.000    0.291    0.000 {method 'reduce' of 'numpy.ufunc' objects}
198716/53562    0.284    0.000    0.612    0.000 copy.py:128(deepcopy)
   359538    0.242    0.000    0.337    0.000 parameterized.py:2544(param)
  1515900    0.217    0.000    0.223    0.000 {method 'get' of 'dict' objects}
    32709    0.214    0.000    2.673    0.000 parameterized.py:2506(__init__)
182805/33235    0.208    0.000    0.298    0.000 ir.py:313(_rec_list_vars)
     2386    0.204    0.000    0.204    0.000 win32.py:92(_winapi_test)
776930/657838    0.202    0.000    1.545    0.000 {built-in method builtins.getattr}
    68423    0.152    0.000    0.717    0.000 bases.py:328(prepare_value)
122581/121773    0.140    0.000    0.728    0.000 parameterized.py:313(_f)
      631    0.127    0.000    0.128    0.000 pprint.py:99(_safe_tuple)
    86570    0.126    0.000    0.239    0.000 parameterized.py:2278(get_param_descriptor)
98297/65588    0.120    0.000    2.285    0.000 parameterized.py:1060(override_initialization)
      265    0.117    0.000    0.120    0.000 encoder.py:204(iterencode)
    16118    0.110    0.000    0.306    0.000 util.py:874(isfinite)
    16640    0.104    0.000    1.830    0.000 options.py:466(__init__)
   155135    0.104    0.000    0.135    0.000 parameterized.py:160(classlist)
      917    0.103    0.000    0.121    0.000 analysis.py:91(liveness)
    16881    0.103    0.000    0.279    0.000 dimension.py:606(matches)
549374/549285    0.102    0.000    0.135    0.000 {built-in method builtins.hasattr}
    53236    0.097    0.000    0.388    0.000 parameterized.py:1346(_instantiate_param)
   359539    0.095    0.000    0.095    0.000 parameterized.py:1142(__init__)
97163/83396    0.093    0.000    0.155    0.000 model.py:824(_visit_value_and_its_immediate_references)
   441645    0.092    0.000    0.097    0.000 {built-in method builtins.issubclass}
    40745    0.092    0.000    0.270    0.000 parameterized.py:1745(get_value_generator)
    61040    0.091    0.000    0.120    0.000 parameterized.py:1480(objects)
64730/63636    0.087    0.000    1.093    0.000 bases.py:182(themed_default)
64009/62915    0.083    0.000    1.288    0.000 descriptors.py:704(_get_default)
    32709    0.082    0.000    0.469    0.000 parameterized.py:1271(_generate_name)
    35019    0.078    0.000    0.125    0.000 container.py:178(validate)
42710/40742    0.077    0.000    0.207    0.000 tree.py:216(__setattr__)
    10220    0.076    0.000    0.432    0.000 parameterized.py:1694(get_param_values)
204462/204344    0.074    0.000    0.856    0.000 {built-in method builtins.setattr}
96810/95716    0.074    0.000    1.378    0.000 descriptors.py:676(_get)
    77114    0.073    0.000    0.174    0.000 util.py:737(__call__)
    86572    0.072    0.000    0.103    0.000 util.py:374(tree_attribute)
67159/65841    0.068    0.000    0.715    0.000 {built-in method builtins.any}
85393/45878    0.068    0.000    0.215    0.000 {method 'format' of 'str' objects}
     5450    0.065    0.000    0.129    0.000 dataset.py:1300(_construct_dataarray)
81951/79642    0.064    0.000    0.386    0.000 {built-in method builtins.sorted}
    66746    0.064    0.000    0.186    0.000 copy.py:66(copy)
    13958    0.063    0.000    1.537    0.000 model.py:808(_visit_immediate_value_references)
    19369    0.061    0.000    0.114    0.000 options.py:745(<genexpr>)
64009/62915    0.060    0.000    1.170    0.000 descriptors.py:584(instance_default)
11068/2767    0.059    0.000    1.072    0.000 options.py:772(options)
   319933    0.056    0.000    0.056    0.000 {method 'endswith' of 'str' objects}
      527    0.054    0.000    0.061    0.000 analysis.py:81(def_reach)
    14471    0.051    0.000    0.086    0.000 functools.py:35(update_wrapper)
104772/102660    0.051    0.000    1.383    0.000 descriptors.py:464(__get__)
   312304    0.050    0.000    0.050    0.000 {method 'items' of 'dict' objects}
    39825    0.048    0.000    0.058    0.000 copy.py:242(_keep_alive)
295879/276115    0.048    0.000    0.059    0.000 {built-in method builtins.len}
   352097    0.047    0.000    0.047    0.000 {built-in method builtins.id}
64730/63636    0.047    0.000    0.316    0.000 bases.py:161(_copy_default)
    33725    0.045    0.000    0.169    0.000 copy.py:209(_deepcopy_tuple)
   276765    0.044    0.000    0.044    0.000 {method 'append' of 'list' objects}
        3    0.043    0.014    0.045    0.015 crs.py:138(__init__)
    72787    0.043    0.000    0.044    0.000 has_props.py:228(accumulate_dict_from_superclasses)
11795/10774    0.041    0.000    0.049    0.000 {built-in method numpy.array}
   132074    0.040    0.000    0.060    0.000 has_props.py:664(themed_values)
   115920    0.040    0.000    0.040    0.000 util.py:690(<genexpr>)
27075/26089    0.040    0.000    0.323    0.000 copy.py:200(_deepcopy_list)
    48159    0.039    0.000    0.039    0.000 wrappers.py:138(__init__)
    65575    0.038    0.000    0.045    0.000 parameterized.py:1020(_validate)
   249337    0.038    0.000    0.052    0.000 parameterized.py:1229(__iter__)
     4832    0.038    0.000    0.342    0.000 util.py:965(max_range)
  172/167    0.037    0.000    0.226    0.001 analysis.py:23(compute_use_defs)
72709/72414    0.036    0.000    0.157    0.000 {built-in method builtins.all}
22082/18800    0.035    0.000    0.464    0.000 {built-in method numpy.core._multiarray_umath.implement_array_function}
    17742    0.035    0.000    0.125    0.000 container.py:74(validate)
    32879    0.035    0.000    0.278    0.000 parameterized.py:1265(_set_name)
     5534    0.034    0.000    0.230    0.000 options.py:733(find)
    31410    0.034    0.000    0.092    0.000 container.py:189(wrap)
29369/28846    0.034    0.000    0.284    0.000 has_props.py:273(__setattr__)
63300/62646    0.033    0.000    0.344    0.000 bases.py:172(_raw_default)
     2583    0.032    0.000    1.379    0.001 options.py:82(lookup_options)
       79    0.032    0.000    1.586    0.020 model.py:55(collect_filtered_models)
    12818    0.032    0.000    0.087    0.000 options.py:548(__getitem__)
    12724    0.032    0.000    0.106    0.000 instructions.py:13(__init__)
     5096    0.032    0.000    0.185    0.000 nanfunctions.py:228(nanmin)
35211/20111    0.031    0.000    0.137    0.000 bases.py:304(is_valid)
   155139    0.031    0.000    0.031    0.000 inspect.py:488(getmro)
11385/1808    0.030    0.000    0.955    0.001 dimension.py:637(traverse)
    65938    0.030    0.000    0.038    0.000 {built-in method _abc._abc_instancecheck}
   168671    0.030    0.000    0.030    0.000 parameterized.py:2148(_get_param)
    31277    0.029    0.000    0.049    0.000 bases.py:502(validate)
    21209    0.028    0.000    0.114    0.000 dimension.py:359(__eq__)
    13120    0.028    0.000    0.164    0.000 values.py:219(_to_string)
    14458    0.028    0.000    0.122    0.000 parameterized.py:273(__get__)
43633/19213    0.028    0.000    0.189    0.000 _utils.py:44(__str__)
    64714    0.028    0.000    0.066    0.000 has_props.py:593(_overridden_defaults)
11016/747    0.027    0.000    0.043    0.000 spaces.py:759(get_nested_dmaps)
    17474    0.027    0.000    0.032    0.000 __init__.py:964(_validate)
    53392    0.026    0.000    0.099    0.000 dimension.py:632(<genexpr>)
     8956    0.026    0.000    0.058    0.000 version.py:309(parse)
    11064    0.026    0.000    0.762    0.000 options.py:261(__init__)
     5092    0.026    0.000    0.185    0.000 nanfunctions.py:343(nanmax)
    31134    0.026    0.000    0.068    0.000 event.py:227(notify)
     6813    0.026    0.000    0.082    0.000 warnings.py:130(filterwarnings)
    14482    0.026    0.000    0.120    0.000 streams.py:795(contents)
     3620    0.026    0.000    0.872    0.000 options.py:639(_merge_options)
      725    0.025    0.000    0.025    0.000 {built-in method nt.stat}
    31422    0.025    0.000    0.050    0.000 wrappers.py:289(__init__)
     1582    0.025    0.000    0.483    0.000 util.py:1708(stream_parameters)
    16118    0.024    0.000    0.027    0.000 util.py:1513(is_dask_array)
    14486    0.024    0.000    0.035    0.000 __init__.py:1372(_validate)
    22126    0.024    0.000    0.034    0.000 __init__.py:1316(_validate)
11250/10331    0.024    0.000    0.164    0.000 either.py:104(validate)
     8958    0.024    0.000    0.024    0.000 {method 'split' of 're.Pattern' objects}
    97625    0.023    0.000    0.023    0.000 {method 'get' of 'mappingproxy' objects}
    65938    0.023    0.000    0.061    0.000 abc.py:96(__instancecheck__)
     9357    0.023    0.000    0.402    0.000 streams.py:400(contents)
    11592    0.023    0.000    0.101    0.000 util.py:681(allowable)
       79    0.023    0.000    1.368    0.017 plot.py:717(_compute_group_range)
32612/32594    0.023    0.000    0.048    0.000 _utils.py:54(get_reference)
    33725    0.022    0.000    0.121    0.000 copy.py:210(<listcomp>)
    17237    0.022    0.000    0.038    0.000 _utils.py:24(deduplicate)
1798/1363    0.022    0.000    3.132    0.002 __init__.py:196(pipelined_fn)
    31727    0.021    0.000    0.056    0.000 dimension.py:302(spec)
       22    0.021    0.001    0.021    0.001 {built-in method binascii.b2a_base64}
     2134    0.021    0.000    0.550    0.000 dimension.py:491(__init__)
     6046    0.021    0.000    0.021    0.000 {built-in method _warnings.warn}
   159324    0.021    0.000    0.021    0.000 {method 'extend' of 'list' objects}
40445/40336    0.021    0.000    0.056    0.000 {method 'join' of 'str' objects}
    83751    0.020    0.000    0.020    0.000 {method 'startswith' of 'str' objects}
"""

The same thing ordered by cumulative time:

# %%prun -l 200 -s cumulative
# display(pn.Row(content, widgets))

# After full kernel restart and running all the cells above once, then only running this cell:
output = """
         19692178 function calls (19000246 primitive calls) in 15.751 seconds

   Ordered by: cumulative time
   List reduced from 6403 to 200 due to restriction <200>

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
     32/1    0.001    0.000   15.757   15.757 {built-in method builtins.exec}
      3/1    0.000    0.000   15.755   15.755 display.py:131(display)
        1    0.000    0.000   15.689   15.689 formatters.py:89(format)
       11    0.000    0.000   15.689    1.426 decorator.py:228(fun)
       11    0.000    0.000   15.689    1.426 formatters.py:220(catch_format_error)
        1    0.000    0.000   15.642   15.642 formatters.py:949(__call__)
        1    0.000    0.000   15.642   15.642 viewable.py:529(_repr_mimebundle_)
        1    0.000    0.000   15.298   15.298 viewable.py:422(_render_model)
        1    0.000    0.000   15.232   15.232 viewable.py:466(get_root)
     20/1    0.000    0.000   15.192   15.192 base.py:109(_get_model)
     20/1    0.000    0.000   15.191   15.191 base.py:82(_get_objects)
        2    0.000    0.000   14.999    7.500 holoviews.py:223(_get_model)
        2    0.000    0.000   14.936    7.468 holoviews.py:273(_render)
        2    0.000    0.000   14.936    7.468 renderer.py:66(get_plot)
        2    0.000    0.000   14.936    7.468 renderer.py:206(get_plot)
   501/14    0.008    0.000    9.265    0.662 spaces.py:1280(__getitem__)
   501/14    0.008    0.000    9.216    0.658 spaces.py:1087(_execute_callback)
   501/14    0.004    0.000    9.216    0.658 spaces.py:667(__call__)
   124/14    0.000    0.000    9.215    0.658 __init__.py:1041(dynamic_operation)
       15    0.000    0.000    9.166    0.611 plot.py:980(update)
   124/14    0.001    0.000    8.398    0.600 __init__.py:1026(resolve)
       13    0.000    0.000    8.266    0.636 streams.py:146(trigger)
       13    0.000    0.000    8.264    0.636 plot.py:208(refresh)
       13    0.000    0.000    8.224    0.633 plot.py:252(_trigger_refresh)
       13    0.000    0.000    8.223    0.633 plot.py:437(__getitem__)
       13    0.002    0.000    8.223    0.633 element.py:2391(update_frame)
      124    0.001    0.000    7.832    0.063 __init__.py:1033(apply)
      124    0.001    0.000    7.826    0.063 __init__.py:1012(_process)
   182/14    0.002    0.000    6.700    0.479 spaces.py:206(dynamic_mul)
       33    0.000    0.000    5.462    0.166 util.py:242(initialize_dynamic)
  181/179    0.002    0.000    5.015    0.028 operation.py:126(_apply)
       24    0.000    0.000    5.012    0.209 operation.py:176(process_element)
        6    0.000    0.000    4.346    0.724 dispatcher.py:340(_compile_for_args)
     28/6    0.001    0.000    4.345    0.724 dispatcher.py:864(compile)
     12/6    0.000    0.000    4.343    0.724 dispatcher.py:78(compile)
     12/6    0.000    0.000    4.343    0.724 dispatcher.py:85(_compile_cached)
     12/6    0.001    0.000    4.343    0.724 dispatcher.py:100(_compile_core)
     12/6    0.000    0.000    4.343    0.724 compiler.py:580(compile_extra)
     12/6    0.000    0.000    4.338    0.723 compiler.py:347(compile_extra)
     12/6    0.000    0.000    4.335    0.722 compiler.py:410(_compile_bytecode)
     13/6    0.000    0.000    4.335    0.722 compiler.py:370(_compile_core)
     13/6    0.001    0.000    4.332    0.722 compiler_machinery.py:316(run)
  289/134    0.001    0.000    4.331    0.032 compiler_lock.py:32(_acquire_compile_lock)
  288/134    0.005    0.000    4.327    0.032 compiler_machinery.py:257(_runPass)
  864/402    0.001    0.000    4.320    0.011 compiler_machinery.py:261(check)
      102    0.001    0.000    3.806    0.037 plot.py:1265(_get_frame)
       28    0.000    0.000    3.805    0.136 util.py:255(get_plot_frame)
1798/1363    0.026    0.000    3.213    0.002 __init__.py:196(pipelined_fn)
       20    0.001    0.000    2.989    0.149 datashader.py:885(_process)
    32709    0.257    0.000    2.961    0.000 parameterized.py:2506(__init__)
    13/12    0.001    0.000    2.819    0.235 typed_passes.py:451(run_pass)
    13/12    0.001    0.000    2.814    0.235 typed_passes.py:360(run_pass)
       20    0.001    0.000    2.759    0.138 core.py:896(raster)
       20    0.000    0.000    2.730    0.137 resampling.py:277(resample_2d)
       20    0.000    0.000    2.724    0.136 resampling.py:482(_resample_2d)
98297/65588    0.147    0.000    2.518    0.000 parameterized.py:1060(override_initialization)
    16640    0.112    0.000    2.067    0.000 options.py:466(__init__)
   316/87    0.002    0.000    2.020    0.023 dimension.py:677(map)
   150/83    0.000    0.000    1.894    0.023 accessors.py:575(_dispatch_opts)
   150/83    0.001    0.000    1.894    0.023 accessors.py:637(_base_opts)
       83    0.001    0.000    1.886    0.023 dimension.py:1230(options)
       83    0.000    0.000    1.876    0.023 __init__.py:192(apply_groups)
       83    0.000    0.000    1.875    0.023 __init__.py:157(_apply_groups_to_backend)
    32709    0.354    0.000    1.865    0.000 parameterized.py:1277(_setup_params)
      222    0.003    0.000    1.853    0.008 accessors.py:30(pipelined_call)
      277    0.001    0.000    1.853    0.007 __init__.py:1221(map)
    13/12    0.000    0.000    1.821    0.152 lowering.py:131(lower)
       67    0.000    0.000    1.803    0.027 accessors.py:524(__call__)
        4    0.000    0.000    1.772    0.443 datashader.py:432(_process)
       48    0.000    0.000    1.760    0.037 accessors.py:625(<lambda>)
       83    0.003    0.000    1.734    0.021 options.py:1791(set_options)
        4    0.001    0.000    1.697    0.424 core.py:172(points)
        4    0.000    0.000    1.696    0.424 core.py:1161(bypixel)
      8/4    0.000    0.000    1.682    0.421 utils.py:102(__call__)
        4    0.000    0.000    1.682    0.421 pandas.py:15(pandas_pipeline)
        4    0.000    0.000    1.682    0.421 pandas.py:23(default)
       79    0.000    0.000    1.670    0.021 model.py:98(collect_models)
       79    0.035    0.000    1.669    0.021 model.py:55(collect_filtered_models)
    13/12    0.000    0.000    1.646    0.137 lowering.py:182(lower_normal_function)
       78    0.001    0.000    1.645    0.021 model.py:554(references)
776930/657838    0.217    0.000    1.627    0.000 {built-in method builtins.getattr}
    13/12    0.000    0.000    1.624    0.135 lowering.py:196(lower_function_body)
    90/54    0.005    0.000    1.623    0.030 lowering.py:220(lower_block)
    13958    0.067    0.000    1.616    0.000 model.py:808(_visit_immediate_value_references)
 1154/774    0.007    0.000    1.611    0.002 lowering.py:321(lower_inst)
       31    0.002    0.000    1.587    0.051 plot.py:598(compute_ranges)
       79    0.026    0.000    1.548    0.020 plot.py:717(_compute_group_range)
        4    0.000    0.000    1.512    0.378 points.py:187(extend)
       15    0.000    0.000    1.500    0.100 element.py:749(_update_plot)
     2583    0.005    0.000    1.471    0.001 plot.py:290(lookup_options)
       68    0.001    0.000    1.468    0.022 plots.py:90(select)
     2583    0.034    0.000    1.466    0.001 options.py:82(lookup_options)
104772/102660    0.051    0.000    1.451    0.000 descriptors.py:464(__get__)
96810/95716    0.078    0.000    1.449    0.000 descriptors.py:676(_get)
     13/7    0.000    0.000    1.445    0.206 typed_passes.py:97(run_pass)
     13/7    0.000    0.000    1.445    0.206 typed_passes.py:61(type_inference_stage)
       83    0.004    0.000    1.436    0.017 options.py:1623(create_custom_trees)
     13/7    0.000    0.000    1.396    0.199 typeinfer.py:1052(propagate)
    26/14    0.004    0.000    1.395    0.100 typeinfer.py:141(propagate)
       15    0.001    0.000    1.363    0.091 element.py:803(_update_ranges)
64009/62915    0.089    0.000    1.356    0.000 descriptors.py:704(_get_default)
       15    0.000    0.000    1.342    0.089 element.py:759(_update_labels)
  422/300    0.004    0.000    1.287    0.004 typeinfer.py:568(resolve)
  462/338    0.001    0.000    1.279    0.004 context.py:187(resolve_function_type)
  462/338    0.002    0.000    1.276    0.004 context.py:231(_resolve_user_function_type)
     2767    0.008    0.000    1.272    0.000 options.py:1255(lookup_options)
  422/300    0.001    0.000    1.272    0.004 typeinfer.py:1500(resolve_call)
       15    0.000    0.000    1.269    0.085 element.py:763(<dictcomp>)
       30    0.002    0.000    1.269    0.042 element.py:647(_axis_properties)
     2767    0.021    0.000    1.262    0.000 options.py:755(closest)
        1    0.000    0.000    1.249    1.249 parfor_lowering.py:28(_lower_parfor_parallel)
64009/62915    0.064    0.000    1.229    0.000 descriptors.py:584(instance_default)
        2    0.000    0.000    1.212    0.606 datashader.py:1522(_process)
  160/106    0.001    0.000    1.177    0.011 typeinfer.py:558(__call__)
4975/2703    0.017    0.000    1.162    0.000 tree.py:106(set_path)
       15    0.001    0.000    1.156    0.077 plot.py:1838(get_extents)
     22/6    0.000    0.000    1.153    0.192 functions.py:521(get_call_type)
     22/6    0.000    0.000    1.153    0.192 dispatcher.py:300(get_call_template)
64730/63636    0.091    0.000    1.148    0.000 bases.py:182(themed_default)
11068/2767    0.064    0.000    1.143    0.000 options.py:772(options)
3321/1822    0.015    0.000    1.136    0.001 tree.py:44(__init__)
      906    0.016    0.000    1.103    0.001 options.py:697(__setattr__)
        2    0.001    0.001    1.099    0.549 datashader.py:1479(_process)
  312/157    0.004    0.000    1.072    0.007 operation.py:197(__call__)
        1    0.000    0.000    1.060    1.060 parfor_lowering.py:850(_create_gufunc_for_parfor_body)
     3620    0.030    0.000    1.055    0.000 options.py:639(_merge_options)
        1    0.000    0.000    1.043    1.043 compiler.py:609(compile_ir)
        1    0.000    0.000    1.042    1.042 compiler.py:355(compile_ir)
        1    0.000    0.000    1.042    1.042 compiler.py:417(_compile_ir)
     8452    0.852    0.000    1.024    0.000 ffi.py:149(__call__)
11385/1808    0.036    0.000    1.012    0.001 dimension.py:637(traverse)
       15    0.003    0.000    0.977    0.065 plot.py:1798(_get_subplot_extents)
      842    0.013    0.000    0.976    0.001 plot.py:1416(get_extents)
204462/204344    0.078    0.000    0.910    0.000 {built-in method builtins.setattr}
        1    0.000    0.000    0.895    0.895 element.py:2322(initialize_plot)
 1940/872    0.007    0.000    0.865    0.001 overlay.py:139(__init__)
 1940/872    0.016    0.000    0.861    0.001 dimension.py:1350(__init__)
        2    0.000    0.000    0.815    0.408 typed_passes.py:288(run_pass)
        2    0.000    0.000    0.815    0.408 parfor.py:2774(run)
    11064    0.028    0.000    0.813    0.000 options.py:261(__init__)
       83    0.003    0.000    0.803    0.010 options.py:1667(<dictcomp>)
      475    0.003    0.000    0.800    0.002 options.py:1533(apply_customizations)
    83/56    0.000    0.000    0.788    0.014 codegen.py:566(_ensure_finalized)
       15    0.000    0.000    0.788    0.053 codegen.py:673(finalize)
       13    0.000    0.000    0.771    0.059 spaces.py:1443(apply_map)
122581/121773    0.144    0.000    0.766    0.000 parameterized.py:313(_f)
67159/65841    0.072    0.000    0.761    0.000 {built-in method builtins.any}
    68423    0.158    0.000    0.753    0.000 bases.py:328(prepare_value)
       79    0.002    0.000    0.677    0.009 plot.py:909(_traverse_options)
      155    0.000    0.000    0.671    0.004 options.py:983(<lambda>)
      156    0.006    0.000    0.671    0.004 options.py:902(collapse_element)
      890    0.003    0.000    0.668    0.001 plot.py:918(lookup)
       13    0.000    0.000    0.654    0.050 cpu.py:197(get_executable)
       26    0.000    0.000    0.652    0.025 codegen.py:946(get_pointer_to_function)
     2134    0.017    0.000    0.646    0.000 dimension.py:849(__init__)
      802    0.005    0.000    0.646    0.001 raster.py:525(range)
198716/53562    0.296    0.000    0.642    0.000 copy.py:128(deepcopy)
 1381/950    0.007    0.000    0.634    0.001 options.py:616(__init__)
        7    0.000    0.000    0.593    0.085 ir_utils.py:1460(simplify)
     2134    0.024    0.000    0.589    0.000 dimension.py:491(__init__)
   121773    0.343    0.000    0.580    0.000 parameterized.py:859(__set__)
      389    0.004    0.000    0.546    0.001 __init__.py:498(range)
      647    0.003    0.000    0.538    0.001 plot.py:1311(get_padding)
      671    0.002    0.000    0.537    0.001 tree.py:160(__setitem__)
     1582    0.027    0.000    0.515    0.000 util.py:1708(stream_parameters)
2441205/2437511    0.425    0.000    0.514    0.000 {built-in method builtins.isinstance}
22082/18800    0.040    0.000    0.513    0.000 {built-in method numpy.core._multiarray_umath.implement_array_function}
    32709    0.088    0.000    0.510    0.000 parameterized.py:1271(_generate_name)
     1002    0.002    0.000    0.498    0.000 util.py:1748(dimensionless_contents)
      464    0.009    0.000    0.497    0.001 xarray.py:252(range)
"""

On full reruns, there is only a slight change in some of the tottime/cumtime/percall values. What’s more interesting though that for the rows with the enormously high ncalls values (usually with the parameterized.py) there is a slight change in the ncalls values too. Not a lot, like in the first table the 3rd row has 32709 ncalls now, but sometimes it’s 32705 or something like that, and all the other similar values vary too. Which is weird, as I’m pretty sure it shouldn’t change how much something in a script gets called if the inputs did not change and it’s not dependent on time…

What was also surprising for me that the first function written by me to show up in the cumulative list by name is at the 268th line (not included above because of character limit) with a cumtime of 0.209. I wrote a pretty linear code, but I do have a lot of functions and it’s interesting to see that whatever the issue is it happens outside of them. I guess it’s a good idea to start to throw stuff into functions to see which one blows up the cumulative list…

[philippjfr]

Okay, a lot of it is indeed in the numba JIT compiler, which means that you should only incur that cost the first time the plots are rendered, subsequent renders should not incur the same cost. Otherwise I don’t really see any targets for you you to optimize but some possible candidates for internal optimizations in HoloViews and Panel.

I think you’re seeing the render time increase because using display means that Panel cannot correctly clean up the previously rendered models when a cell is rerun so it ends up updating multiple copies of the same objects.

[SteveAKopias]

So far I managed to isolate the parts that load the images (tiffs to be precise) and the waterpoints.

As a recap, this how combined looks like:

combined = (
        # PART 1
        hv.DynamicMap(get_tiles)

        # PART 2
        * hd.regrid(hv.DynamicMap(get_image_01)).apply.opts(cmap=pn_01_cmap, alpha=pn_01_alpha) 
        * hd.regrid(hv.DynamicMap(get_image_02)).apply.opts(cmap=pn_02_cmap, alpha=pn_02_alpha) 
        * hd.regrid(hv.DynamicMap(get_image_03)).apply.opts(cmap=pn_03_cmap, alpha=pn_03_alpha) 
        * hd.regrid(hv.DynamicMap(get_image_04)).apply.opts(cmap=pn_04_cmap, alpha=pn_04_alpha)
        * hd.regrid(hv.DynamicMap(get_image_05)).apply.opts(cmap=pn_05_cmap, alpha=pn_05_alpha)
        * hd.regrid(hv.DynamicMap(get_image_06)).apply.opts(cmap=pn_06_cmap, alpha=pn_06_alpha) 
        * hd.regrid(hv.DynamicMap(get_image_07)).apply.opts(cmap=pn_07_cmap, alpha=pn_07_alpha)
        * hd.regrid(hv.DynamicMap(get_image_08)).apply.opts(cmap=pn_08_cmap, alpha=pn_08_alpha)

        # PART 3
        * hd.regrid(hv.DynamicMap(get_white_bg_for_points)).options(cmap=['white']).apply.opts(alpha=pn_white_bg_for_points_alpha)
        * points_aggregated_categorical
        * points_aggregated_for_hover
        * hv_points

        # PART 4
        * hd.regrid(hv.DynamicMap(get_image_for_hover_tooltip))
)

• Part 1: Just returns a getattr(hv.element.tiles, pn_tile)() based on a Select.
• Part 2: Depending on the selectable layers checkboxlist it either actually loads and returns the actual image from disk based on other widgets, or returns an empty image as a placeholder.
• Part3: Has a basic placeholder as background, a categorical hd.datashade, a default hd.aggregate and a hv.Points based on the same DataFrame loaded from a CSV.
• Part4: Just a placeholder for displaying custom tooltips.

So there are only two possible type of datasources: the images are loaded from the tiffs and the points are loaded from a CSV.

So I added a few variables to control these:

# If dummy_image_scale==True, it gives back generated dummy images instead of 
#     loading the tiffs.
# dummy_image_scale defines the size of the image, 100(%) meaning 100x250
# hv.Image(np.ones((dummy_image_scale, int(dummy_image_scale *2.5))), 
#     bounds=(left,bottom,right,top))
load_dummy_images = True
dummy_image_scale = 10

# If load_dummy_wp==True, it generates a waterpoints DataFrame with the same 
#     columns but with filler data instead of loading the csv
load_dummy_wp = True

# If wp_dummy_limit>0, it limits the length of the waterpoints DataFrame (real or 
#     dummy)
# If wp_dummy_limit==0, it completely disables anything related to waterpoints, 
#     including loading/generating data, manipulating it, generating aggregates 
#     and even removes the whole PART3 from combined
wp_dummy_limit = 130000 # default: 130000, the size of the original CSV

This means that I can now completely disable 1) touching the files 2) dealing with large datasets. I started with gradual changes, and again, restarting the kernel every time.

The code I ran:

%%prun -l 100
display(pn.Row(content, widgets))

(content are widgets are the pn containers, the former containing combined, the latter containing all the widgets, of course.)

And the results with the relevant settings (bold if not default):

Original settings

{‘load_dummy_images’: False, ‘dummy_image_scale’: 100,
‘load_dummy_wp’: False, ‘wp_dummy_limit’: 130000}
19309648 function calls (18620261 primitive calls) in 13.693 seconds

Dummy Images, real WP

{‘load_dummy_images’: True, ‘dummy_image_scale’: 100,
‘load_dummy_wp’: False, ‘wp_dummy_limit’: 130000}
15689963 function calls (15303463 primitive calls) in 11.841 seconds

Dummy Images & Dummy WP

{‘load_dummy_images’: True, ‘dummy_image_scale’: 100,
‘load_dummy_wp’: True, ‘wp_dummy_limit’: 130000}
15691597 function calls (15305097 primitive calls) in 11.399 seconds

Dummy Images & Dummy WP, Dummy Image Scale: 10%

{‘load_dummy_images’: True, ‘dummy_image_scale’: 10,
‘load_dummy_wp’: True, ‘wp_dummy_limit’: 130000}
15728239 function calls (15341594 primitive calls) in 12.093 seconds

Dummy both, Image Scale: 10%, WP Limit: 130

{‘load_dummy_images’: True, ‘dummy_image_scale’: 10,
‘load_dummy_wp’: True, ‘wp_dummy_limit’: 130}
16473182 function calls (16176164 primitive calls) in 11.786 seconds

Dummy Images, Image Scale: 10%, Waterpoints disabled

{‘load_dummy_images’: True, ‘dummy_image_scale’: 10,
‘load_dummy_wp’: True, ‘wp_dummy_limit’: 0}
7451420 function calls (7295576 primitive calls) in 5.589 seconds

I’ve ran the same tests also with hv.render(combined), that produced similar changes, but this time ranging from 31s (original settings) to 12s (dummy images downscaled, waterpoints disabled).

What I have learned from this:

• Disabling all the real image loading helped only about 7%. That’s good, because I was worried that if loading big data files causes the issue, there wouldn’t be any solution. Decreasing the dummy image size didn’t improve the results, although increasing (not logged) worsened it, so that 7% is most probably due to the actual resolution difference between the real and dummy images.
• Replacing the waterpoints DataFrame with a faux version, of course, did not result in any change, I wasn’t expecting it either.
• What’s surprising, that reducing the number of records in the waterpoints DataFrame to 0.1% (or even when reducing them to 1 single record) didn’t change anything either. Which is, again, good news, as it’s not the size of the dataset that is the problem, but this was unexpected.
• Completely eliminating everything that’s related to the waterpoint DataFrame results in an about 60% time reduction, so there is definitely something going on there. However there’s still a significant amount of time remaining.

Also, @philippjfr: what makes you say this is a numba issue? This is what I’ve got for the last two tests (everthing reduced to minimal and waterpoints disabled) above:

Dummy both, Image Scale: 10%, WP Limit: 130

   Ordered by: internal time
   List reduced from 4748 to 100 due to restriction <100>

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
   940822    0.423    0.000    0.556    0.000 parameterized.py:837(__get__)
    33054    0.357    0.000    1.814    0.000 parameterized.py:1277(_setup_params)
   123543    0.347    0.000    0.583    0.000 parameterized.py:859(__set__)
1644391/1642463    0.342    0.000    0.427    0.000 {built-in method builtins.isinstance}
   363350    0.259    0.000    0.355    0.000 parameterized.py:2544(param)
99332/66278    0.255    0.000    2.559    0.000 parameterized.py:1060(override_initialization)
  1564530    0.236    0.000    0.236    0.000 {method 'get' of 'dict' objects}
    33054    0.229    0.000    2.973    0.000 parameterized.py:2506(__init__)
813332/696487    0.220    0.000    1.621    0.000 {built-in method builtins.getattr}
    67396    0.159    0.000    0.774    0.000 bases.py:328(prepare_value)
124347/123543    0.147    0.000    0.773    0.000 parameterized.py:313(_f)
    20523    0.139    0.000    0.392    0.000 util.py:874(isfinite)
65311/55738    0.138    0.000    0.291    0.000 copy.py:128(deepcopy)
    87988    0.132    0.000    0.259    0.000 parameterized.py:2278(get_param_descriptor)
   207750    0.131    0.000    0.346    0.000 dimension.py:302(spec)
    27590    0.116    0.000    0.116    0.000 {method 'reduce' of 'numpy.ufunc' objects}
   157603    0.112    0.000    0.143    0.000 parameterized.py:160(classlist)
    16645    0.111    0.000    2.044    0.000 options.py:466(__init__)
    16455    0.108    0.000    0.277    0.000 dimension.py:606(matches)
540569/540479    0.107    0.000    0.120    0.000 {built-in method builtins.hasattr}
   469554    0.104    0.000    0.105    0.000 {built-in method builtins.issubclass}
    55738    0.102    0.000    0.411    0.000 parameterized.py:1346(_instantiate_param)
96114/81924    0.098    0.000    0.164    0.000 model.py:824(_visit_value_and_its_immediate_references)
   363350    0.097    0.000    0.097    0.000 parameterized.py:1142(__init__)
    60555    0.095    0.000    0.125    0.000 parameterized.py:1480(objects)
    40435    0.095    0.000    0.279    0.000 parameterized.py:1745(get_value_generator)
   106603    0.095    0.000    0.468    0.000 dimension.py:359(__eq__)
63680/62610    0.091    0.000    1.158    0.000 bases.py:182(themed_default)
    33054    0.087    0.000    0.493    0.000 parameterized.py:1271(_generate_name)
62924/61854    0.085    0.000    1.359    0.000 descriptors.py:704(_get_default)
42713/40745    0.083    0.000    0.221    0.000 tree.py:216(__setattr__)
    34497    0.081    0.000    0.136    0.000 container.py:178(validate)
    10179    0.080    0.000    0.447    0.000 parameterized.py:1694(get_param_values)
212929/212850    0.078    0.000    0.920    0.000 {built-in method builtins.setattr}
95351/94281    0.078    0.000    1.454    0.000 descriptors.py:676(_get)
    86578    0.077    0.000    0.110    0.000 util.py:374(tree_attribute)
    74609    0.076    0.000    0.174    0.000 util.py:737(__call__)
    13691    0.068    0.000    1.615    0.000 model.py:808(_visit_immediate_value_references)
    64833    0.068    0.000    0.205    0.000 copy.py:66(copy)
61361/60129    0.067    0.000    0.495    0.000 {built-in method builtins.any}
     7764    0.066    0.000    0.545    0.000 util.py:965(max_range)
19883/18013    0.065    0.000    0.091    0.000 {built-in method numpy.array}
11012/2753    0.063    0.000    1.122    0.000 options.py:772(options)
    19271    0.063    0.000    0.120    0.000 options.py:745(<genexpr>)
71682/71551    0.062    0.000    0.262    0.000 {built-in method builtins.sorted}
62924/61854    0.061    0.000    1.236    0.000 descriptors.py:584(instance_default)
   318373    0.060    0.000    0.060    0.000 {method 'endswith' of 'str' objects}
    14750    0.055    0.000    0.093    0.000 functools.py:35(update_wrapper)
103428/101364    0.053    0.000    1.455    0.000 descriptors.py:464(__get__)
63680/62610    0.049    0.000    0.337    0.000 bases.py:161(_copy_default)
   283289    0.047    0.000    0.047    0.000 {method 'items' of 'dict' objects}
    32230    0.046    0.000    0.055    0.000 copy.py:242(_keep_alive)
    71865    0.045    0.000    0.045    0.000 has_props.py:228(accumulate_dict_from_superclasses)
219785/198812    0.045    0.000    0.059    0.000 {built-in method builtins.len}
   129963    0.043    0.000    0.063    0.000 has_props.py:664(themed_values)
   114840    0.042    0.000    0.042    0.000 util.py:690(<genexpr>)
    47314    0.042    0.000    0.042    0.000 wrappers.py:138(__init__)
     3760    0.041    0.000    0.082    0.000 dataset.py:1300(_construct_dataarray)
       80    0.041    0.001    2.659    0.033 plot.py:717(_compute_group_range)

Dummy Images, Image Scale: 10%, Waterpoints disabled

 7451420 function calls (7295576 primitive calls) in 5.589 seconds

   Ordered by: internal time
   List reduced from 4039 to 100 due to restriction <100>

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
17006    0.194    0.000    0.955    0.000 parameterized.py:1277(_setup_params)
62718    0.188    0.000    0.309    0.000 parameterized.py:859(__set__)
699794/698666    0.142    0.000    0.165    0.000 {built-in method builtins.isinstance}
   262817    0.141    0.000    0.179    0.000 parameterized.py:837(__get__)
   181577    0.139    0.000    0.194    0.000 parameterized.py:2544(param)
17006    0.124    0.000    1.515    0.000 parameterized.py:2506(__init__)
364326/303210    0.108    0.000    0.848    0.000 {built-in method builtins.getattr}
37325    0.091    0.000    0.449    0.000 bases.py:328(prepare_value)
   576076    0.090    0.000    0.091    0.000 {method 'get' of 'dict' objects}
63129/62718    0.078    0.000    0.410    0.000 parameterized.py:313(_f)
44381    0.071    0.000    0.137    0.000 parameterized.py:2278(get_param_descriptor)
30948/26699    0.069    0.000    0.144    0.000 copy.py:128(deepcopy)
51120/34114    0.069    0.000    1.289    0.000 parameterized.py:1060(override_initialization)
 8797    0.061    0.000    1.090    0.000 options.py:466(__init__)
80197    0.061    0.000    0.079    0.000 parameterized.py:160(classlist)
   181578    0.055    0.000    0.055    0.000 parameterized.py:1142(__init__)
48314/42498    0.054    0.000    0.091    0.000 model.py:824(_visit_value_and_its_immediate_references)
283205/283134    0.054    0.000    0.061    0.000 {built-in method builtins.hasattr}
 7371    0.053    0.000    0.152    0.000 util.py:874(isfinite)
26699    0.053    0.000    0.207    0.000 parameterized.py:1346(_instantiate_param)
   223262    0.051    0.000    0.052    0.000 {built-in method builtins.issubclass}
34534/33650    0.050    0.000    0.650    0.000 bases.py:182(themed_default)
34149/33265    0.049    0.000    0.762    0.000 descriptors.py:704(_get_default)
19103    0.048    0.000    0.140    0.000 parameterized.py:1745(get_value_generator)
29078    0.048    0.000    0.063    0.000 parameterized.py:1480(objects)
17006    0.047    0.000    0.267    0.000 parameterized.py:1271(_generate_name)
10543    0.045    0.000    0.045    0.000 {method 'reduce' of 'numpy.ufunc' objects}
 6900    0.044    0.000    0.120    0.000 dimension.py:606(matches)
22482/21434    0.044    0.000    0.118    0.000 tree.py:216(__setattr__)
17035    0.044    0.000    0.070    0.000 container.py:178(validate)
52191/51307    0.043    0.000    0.814    0.000 descriptors.py:676(_get)
105391/105279    0.043    0.000    0.578    0.000 {built-in method builtins.setattr}
45548    0.042    0.000    0.059    0.000 util.py:374(tree_attribute)
38420/38289    0.038    0.000    0.180    0.000 {built-in method builtins.sorted}
 4576    0.038    0.000    0.221    0.000 parameterized.py:1694(get_param_values)
 7600    0.037    0.000    0.865    0.000 model.py:808(_visit_immediate_value_references)
34753    0.037    0.000    0.114    0.000 copy.py:66(copy)
5956/1489    0.036    0.000    0.642    0.000 options.py:772(options)
33029    0.035    0.000    0.082    0.000 util.py:737(__call__)
10423    0.035    0.000    0.066    0.000 options.py:745(<genexpr>)
  458    0.035    0.000    0.044    0.000 ffi.py:149(__call__)
   177366    0.034    0.000    0.034    0.000 {method 'endswith' of 'str' objects}
34149/33265    0.034    0.000    0.693    0.000 descriptors.py:584(instance_default)
27927/27382    0.031    0.000    0.260    0.000 {built-in method builtins.any}
58209/56505    0.029    0.000    0.801    0.000 descriptors.py:464(__get__)
34534/33650    0.028    0.000    0.217    0.000 bases.py:161(_copy_default)
40613    0.027    0.000    0.028    0.000 has_props.py:228(accumulate_dict_from_superclasses)
19023/18560    0.027    0.000    0.259    0.000 has_props.py:273(__setattr__)
 6695    0.026    0.000    0.044    0.000 functools.py:35(update_wrapper)
 2244    0.025    0.000    0.050    0.000 dataset.py:1300(_construct_dataarray)
26061    0.025    0.000    0.025    0.000 wrappers.py:138(__init__)
   145772    0.024    0.000    0.024    0.000 {method 'items' of 'dict' objects}
71076    0.023    0.000    0.035    0.000 has_props.py:664(themed_values)
14894    0.023    0.000    0.027    0.000 copy.py:242(_keep_alive)
131821    0.022    0.000    0.029    0.000 parameterized.py:1229(__iter__)
 9850    0.022    0.000    0.109    0.000 container.py:74(validate)
54740    0.021    0.000    0.021    0.000 util.py:690(<genexpr>)
34083    0.021    0.000    0.025    0.000 parameterized.py:1020(_validate)
 2978    0.020    0.000    0.133    0.000 options.py:733(find)

So even when it runs in 5s, when it does not touch any real data, when it only handles really downsized faux data, it still has tens or hundreds of thousands of calls to parameterized.py functions. I don’t use params directly, but I use a lot of panel widgets, so my best guess that maybe they somehow get tangled together and keep triggering each other?

Now that I removed real data as a requirement, I can move towards creating a reduced example, and also will experiment with removing some widgets and functions being dependent on them.

[SteveAKopias]

https://colab.research.google.com/drive/1j0hrAPS3aCxeZ1OhuG4oiN89BPsLo3ex?usp=sharing

I’m very impressed.

• Reduced the script to about 600 lines from 4000.
• Removed every real data from it, it’s only using dummy data now.
• Removed everything not strictly necessary to display combined.
• Removed every style, JS link, custom hover tooltip, HTML template, not even displaying any of the widgets.

The runtime did not change at all.

I kept, however, the core structure of the script:

• creating widgets
• loading spatial datasets
• loading waterpoints
• displaying combined (spatial datasets + waterpoints)

The next step is to start experimenting with the widgets, checking out what happens if I one by one remove them and we’ll see. Meanwhile, if anybody has the time and mental space to check out the code (even if I know that 600 lines are still not a comfortable size) and anything jumps out as an obvious source of error, I would be very grateful for some pointers.

[philippjfr]

Thank you so much for the detailed analysis, it was extremely valuable as a starting point. I’ve started working on some optimizations in the HoloViews and Param internals which I think will make a reasonable difference. Param is used extensively inside Panel and HoloViews so you shouldn’t be alarmed to see it pop up in the profiler but there do seem to be a good amount of low-level optimizations we can apply to reduce that by what I hope will be ~50% in your case.

[philippjfr]

@SteveAKopias If you could try out Optimize and clean up options system by philippjfr · Pull Request #4954 · holoviz/holoviews · GitHub and see if it makes a difference for your use case that would be massively appreciated.

[SteveAKopias]

Wow, thanks a lot!

There is an 18% improvement when running the dummy version on my laptop with the optimize_options, so this definitely helped, although the runtime is still around 10s with hundreds of thousands of calls. (I’ve also updated the Colab Notebook with this, and that had a 15% improvement goint from 13.78s to 11.76s on average.)

Running with the original holoviews locally:

 15428135 function calls (15085925 primitive calls) in 11.893 seconds

   Ordered by: internal time
   List reduced from 5139 to 100 due to restriction <100>

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
 6129    0.540    0.000    0.695    0.000 ffi.py:149(__call__)
1677076/1674620    0.351    0.000    0.496    0.000 {built-in method builtins.isinstance}
55041    0.317    0.000    0.738    0.000 colors.py:197(_to_rgba_no_colorcycle)
25408    0.287    0.000    1.451    0.000 parameterized.py:1277(_setup_params)
93775    0.269    0.000    0.452    0.000 parameterized.py:859(__set__)
   458655    0.242    0.000    0.308    0.000 parameterized.py:837(__get__)
   294779    0.215    0.000    0.297    0.000 parameterized.py:2544(param)
25408    0.188    0.000    2.290    0.000 parameterized.py:2506(__init__)
628296/536118    0.182    0.000    1.355    0.000 {built-in method builtins.getattr}
   981279    0.152    0.000    0.156    0.000 {method 'get' of 'dict' objects}
21758    0.140    0.000    0.140    0.000 {method 'reduce' of 'numpy.ufunc' objects}
52645    0.127    0.000    0.629    0.000 bases.py:328(prepare_value)
59437/43811    0.123    0.000    0.266    0.000 copy.py:128(deepcopy)
55041    0.122    0.000    0.886    0.000 colors.py:161(to_rgba)
94542/93775    0.115    0.000    0.601    0.000 parameterized.py:313(_f)
114088/112909    0.115    0.000    0.664    0.000 {built-in method builtins.any}
14698    0.112    0.000    0.306    0.000 util.py:874(isfinite)
66735    0.105    0.000    0.199    0.000 parameterized.py:2278(get_param_descriptor)
  214    0.104    0.000    1.034    0.005 colors.py:776(from_list)
76393/50985    0.097    0.000    1.945    0.000 parameterized.py:1060(override_initialization)
36737    0.089    0.000    0.260    0.000 parameterized.py:1745(get_value_generator)
   119645    0.087    0.000    0.112    0.000 parameterized.py:160(classlist)
54375    0.087    0.000    0.115    0.000 parameterized.py:1480(objects)
43805    0.086    0.000    0.350    0.000 parameterized.py:1346(_instantiate_param)
12405    0.083    0.000    1.521    0.000 options.py:466(__init__)
   294779    0.082    0.000    0.082    0.000 parameterized.py:1142(__init__)
421518/421433    0.082    0.000    0.112    0.000 {built-in method builtins.hasattr}
   346629    0.080    0.000    0.084    0.000 {built-in method builtins.issubclass}
76985/68183    0.077    0.000    0.131    0.000 model.py:824(_visit_value_and_its_immediate_references)
42094/40148    0.076    0.000    0.210    0.000 tree.py:216(__setattr__)
 9465    0.076    0.000    0.419    0.000 parameterized.py:1694(get_param_values)
85322    0.075    0.000    0.108    0.000 util.py:374(tree_attribute)
105578/105361    0.074    0.000    0.399    0.000 {built-in method builtins.all}
49765/48831    0.073    0.000    0.952    0.000 bases.py:182(themed_default)
  740    0.073    0.000    0.073    0.000 win32.py:92(_winapi_test)
25408    0.071    0.000    0.393    0.000 parameterized.py:1271(_generate_name)
49079/48145    0.071    0.000    1.115    0.000 descriptors.py:704(_get_default)
78574/77640    0.066    0.000    1.194    0.000 descriptors.py:676(_get)
26228    0.063    0.000    0.100    0.000 container.py:178(validate)
10298    0.063    0.000    0.158    0.000 dimension.py:606(matches)
19145    0.062    0.000    0.118    0.000 options.py:745(<genexpr>)
   198579    0.061    0.000    0.069    0.000 {built-in method _abc._abc_instancecheck}
   198579    0.060    0.000    0.129    0.000 abc.py:96(__instancecheck__)
165246/165153    0.059    0.000    0.718    0.000 {built-in method builtins.setattr}
   220160    0.059    0.000    0.206    0.000 colors.py:266(<genexpr>)
10940/2735    0.059    0.000    1.107    0.000 options.py:772(options)
   312895    0.059    0.000    0.059    0.000 {method 'endswith' of 'str' objects}
54534    0.059    0.000    0.134    0.000 util.py:737(__call__)
   275200    0.057    0.000    0.057    0.000 colors.py:276(<genexpr>)
  265    0.057    0.000    0.060    0.000 encoder.py:204(iterencode)
62845/61088    0.055    0.000    0.189    0.000 {built-in method builtins.sorted}
10593    0.055    0.000    1.335    0.000 model.py:808(_visit_immediate_value_references)
42622/8082    0.055    0.000    0.078    0.000 ir.py:313(_rec_list_vars)
49689    0.053    0.000    0.127    0.000 copy.py:66(copy)
   357684    0.053    0.000    0.053    0.000 {method 'append' of 'list' objects}
306558/289947    0.053    0.000    0.063    0.000 {built-in method builtins.len}
49079/48145    0.052    0.000    1.016    0.000 descriptors.py:584(instance_default)
13822    0.051    0.000    0.086    0.000 functools.py:35(update_wrapper)
55481/29913    0.051    0.000    0.161    0.000 {method 'format' of 'str' objects}
 3936    0.047    0.000    0.094    0.000 dataset.py:1300(_construct_dataarray)
84694/82894    0.044    0.000    1.202    0.000 descriptors.py:464(__get__)
10940/10045    0.042    0.000    0.048    0.000 {built-in method numpy.array}
   112730    0.040    0.000    0.040    0.000 util.py:690(<genexpr>)
 4678    0.040    0.000    0.361    0.000 util.py:965(max_range)
49765/48831    0.039    0.000    0.291    0.000 bases.py:161(_copy_default)
27389    0.039    0.000    0.047    0.000 copy.py:242(_keep_alive)
   234587    0.038    0.000    0.038    0.000 {method 'items' of 'dict' objects}
55885    0.036    0.000    0.036    0.000 has_props.py:228(accumulate_dict_from_superclasses)

Running with optimize_options locally (using conda --create new --clone old and pip install git+https://github.com/holoviz/holoviews.git@optimize_options on the new):

13021956 function calls (12717894 primitive calls) in 9.759 seconds

   Ordered by: internal time
   List reduced from 5138 to 100 due to restriction <100>

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
     6129    0.464    0.000    0.602    0.000 ffi.py:149(__call__)
1511266/1508810    0.307    0.000    0.450    0.000 {built-in method builtins.isinstance}
   416001    0.208    0.000    0.266    0.000 parameterized.py:837(__get__)
    55041    0.186    0.000    0.599    0.000 colors.py:197(_to_rgba_no_colorcycle)
548640/466334    0.155    0.000    1.166    0.000 {built-in method builtins.getattr}
     4678    0.148    0.000    0.154    0.000 util.py:917(<listcomp>)
    21758    0.133    0.000    0.133    0.000 {method 'reduce' of 'numpy.ufunc' objects}
   177794    0.133    0.000    0.133    0.000 {method 'items' of 'dict' objects}
   179166    0.128    0.000    0.176    0.000 parameterized.py:2544(param)
     8900    0.126    0.000    0.769    0.000 parameterized.py:1277(_setup_params)
    55041    0.119    0.000    0.743    0.000 colors.py:161(to_rgba)
   771638    0.115    0.000    0.118    0.000 {method 'get' of 'dict' objects}
    47763    0.111    0.000    0.556    0.000 bases.py:328(prepare_value)
113570/112427    0.109    0.000    0.531    0.000 {built-in method builtins.any}
    14698    0.105    0.000    0.291    0.000 util.py:810(isfinite)
      214    0.100    0.000    0.884    0.004 colors.py:776(from_list)
    31865    0.092    0.000    0.163    0.000 parameterized.py:859(__set__)
    36699    0.085    0.000    0.247    0.000 parameterized.py:1745(get_value_generator)
42948/27322    0.084    0.000    0.299    0.000 copy.py:128(deepcopy)
    54301    0.081    0.000    0.107    0.000 parameterized.py:1480(objects)
     8900    0.076    0.000    1.152    0.000 parameterized.py:2506(__init__)
    85322    0.075    0.000    0.107    0.000 util.py:335(tree_attribute)
42094/40148    0.075    0.000    0.208    0.000 tree.py:223(__setattr__)
101695/101487    0.070    0.000    0.384    0.000 {built-in method builtins.all}
     9465    0.070    0.000    0.396    0.000 parameterized.py:1694(get_param_values)
      740    0.067    0.000    0.068    0.000 win32.py:92(_winapi_test)
66756/60572    0.065    0.000    0.110    0.000 model.py:824(_visit_value_and_its_immediate_references)
44985/44135    0.063    0.000    0.829    0.000 bases.py:182(themed_default)
    10290    0.060    0.000    0.153    0.000 dimension.py:596(matches)
   198250    0.060    0.000    0.127    0.000 abc.py:96(__instancecheck__)
44299/43449    0.059    0.000    0.970    0.000 descriptors.py:704(_get_default)
289225/289140    0.059    0.000    0.087    0.000 {built-in method builtins.hasattr}
   198250    0.059    0.000    0.068    0.000 {built-in method _abc._abc_instancecheck}
   220160    0.057    0.000    0.203    0.000 colors.py:266(<genexpr>)
70781/69931    0.056    0.000    1.037    0.000 descriptors.py:676(_get)
   275200    0.056    0.000    0.056    0.000 colors.py:276(<genexpr>)
    23516    0.055    0.000    0.088    0.000 container.py:178(validate)
      265    0.055    0.000    0.058    0.000 encoder.py:204(iterencode)
42622/8082    0.053    0.000    0.076    0.000 ir.py:313(_rec_list_vars)
   230426    0.051    0.000    0.054    0.000 {built-in method builtins.issubclass}
    13824    0.051    0.000    0.086    0.000 functools.py:35(update_wrapper)
   355614    0.051    0.000    0.051    0.000 {method 'append' of 'list' objects}
287399/270788    0.048    0.000    0.058    0.000 {built-in method builtins.len}
     9343    0.048    0.000    1.141    0.000 model.py:808(_visit_immediate_value_references)
   179166    0.048    0.000    0.048    0.000 parameterized.py:1142(__init__)
    27316    0.047    0.000    0.340    0.000 parameterized.py:1346(_instantiate_param)
55440/29872    0.047    0.000    0.149    0.000 {method 'format' of 'str' objects}
    45147    0.046    0.000    0.118    0.000 copy.py:66(copy)
44299/43449    0.045    0.000    0.884    0.000 descriptors.py:584(instance_default)
    44308    0.045    0.000    0.103    0.000 util.py:682(__call__)
     3936    0.043    0.000    0.086    0.000 dataset.py:1300(_construct_dataarray)
32632/31865    0.042    0.000    0.217    0.000 parameterized.py:313(_f)
     8325    0.041    0.000    0.249    0.000 options.py:453(__init__)
   112740    0.039    0.000    0.039    0.000 util.py:636(<genexpr>)
10940/10045    0.039    0.000    0.044    0.000 {built-in method numpy.array}
     4678    0.039    0.000    0.490    0.000 util.py:901(max_range)
76784/75152    0.038    0.000    1.041    0.000 descriptors.py:464(__get__)
    21333    0.038    0.000    0.072    0.000 parameterized.py:2278(get_param_descriptor)
26869/17969    0.037    0.000    0.992    0.000 parameterized.py:1060(override_initialization)
119760/119670    0.036    0.000    0.398    0.000 {built-in method builtins.setattr}
40306/38549    0.035    0.000    0.078    0.000 {built-in method builtins.sorted}
44985/44135    0.034    0.000    0.253    0.000 bases.py:161(_copy_default)

Based on this I’m even more sure that this is not about param or hv being inefficient. You assumed, there could be a 50% imporovement and we got third of that. I’m pretty sure your assumption was precise, meaning 2/3 of the total runtime is caused by something completely unnecessary stuff. My best guess would be that I used either .apply, @pn.depends or .link incorrectly somewhere and that created a runaway effect. So there are what I’m going to investigate next, although today I will have limited time for this.

[philippjfr]

Could you give me the profiling output sorted by cumulative time as well (%%prun -s cumulative)?

[SteveAKopias]

Sure, here you are:

15427017 function calls (15084816 primitive calls) in 12.494 seconds

   Ordered by: cumulative time
   List reduced from 5139 to 200 due to restriction <200>

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
      5/1    0.000    0.000   12.497   12.497 {built-in method builtins.exec}
      3/1    0.000    0.000   12.496   12.496 display.py:131(display)
        1    0.000    0.000   12.468   12.468 formatters.py:89(format)
       11    0.000    0.000   12.465    1.133 decorator.py:228(fun)
       11    0.000    0.000   12.465    1.133 formatters.py:220(catch_format_error)
        1    0.000    0.000   12.460   12.460 formatters.py:949(__call__)
        1    0.000    0.000   12.460   12.460 dimension.py:1310(_repr_mimebundle_)
        1    0.000    0.000   12.460   12.460 options.py:1389(render)
        1    0.000    0.000   12.460   12.460 display_hooks.py:274(pprint_display)
        1    0.000    0.000   12.451   12.451 display_hooks.py:235(display)
        3    0.000    0.000   12.451    4.150 display_hooks.py:138(wrapped)
        1    0.000    0.000   12.451   12.451 display_hooks.py:195(map_display)
        1    0.000    0.000   12.451   12.451 display_hooks.py:52(render)
        1    0.000    0.000   12.451   12.451 renderer.py:379(components)
        1    0.000    0.000   12.263   12.263 viewable.py:422(_render_model)
        1    0.000    0.000   12.215   12.215 viewable.py:466(get_root)
        1    0.000    0.000   12.181   12.181 base.py:109(_get_model)
      2/1    0.000    0.000   12.180   12.180 base.py:82(_get_objects)
        2    0.000    0.000   12.178    6.089 holoviews.py:223(_get_model)
        1    0.000    0.000   12.127   12.127 holoviews.py:273(_render)
        1    0.000    0.000   12.127   12.127 renderer.py:66(get_plot)
        1    0.000    0.000   12.127   12.127 renderer.py:206(get_plot)
       14    0.000    0.000    8.882    0.634 plot.py:980(update)
       13    0.001    0.000    6.869    0.528 streams.py:146(trigger)
       13    0.000    0.000    6.866    0.528 plot.py:208(refresh)
       13    0.000    0.000    6.825    0.525 plot.py:252(_trigger_refresh)
       13    0.000    0.000    6.823    0.525 plot.py:437(__getitem__)
       13    0.002    0.000    6.823    0.525 element.py:2391(update_frame)
   487/14    0.008    0.000    5.852    0.418 spaces.py:1280(__getitem__)
   487/14    0.009    0.000    5.805    0.415 spaces.py:1087(_execute_callback)
   487/14    0.004    0.000    5.804    0.415 spaces.py:667(__call__)
   110/14    0.000    0.000    5.804    0.415 __init__.py:1041(dynamic_operation)
   110/14    0.000    0.000    4.968    0.355 __init__.py:1026(resolve)
      110    0.001    0.000    4.649    0.042 __init__.py:1033(apply)
      110    0.001    0.000    4.643    0.042 __init__.py:1012(_process)
   182/14    0.002    0.000    4.190    0.299 spaces.py:206(dynamic_mul)
      102    0.001    0.000    2.957    0.029 plot.py:1265(_get_frame)
       28    0.000    0.000    2.955    0.106 util.py:255(get_plot_frame)
1461/1187    0.021    0.000    2.924    0.002 __init__.py:196(pipelined_fn)
       31    0.000    0.000    2.899    0.094 util.py:242(initialize_dynamic)
  180/178    0.002    0.000    2.784    0.016 operation.py:126(_apply)
       24    0.000    0.000    2.782    0.116 operation.py:176(process_element)
    25404    0.190    0.000    2.360    0.000 parameterized.py:2506(__init__)
        4    0.000    0.000    2.183    0.546 datashader.py:432(_process)
        5    0.000    0.000    2.158    0.432 dispatcher.py:340(_compile_for_args)
     25/5    0.001    0.000    2.156    0.431 dispatcher.py:864(compile)
     11/5    0.000    0.000    2.155    0.431 dispatcher.py:78(compile)
     11/5    0.000    0.000    2.155    0.431 dispatcher.py:85(_compile_cached)
     11/5    0.001    0.000    2.155    0.431 dispatcher.py:100(_compile_core)
     11/5    0.000    0.000    2.154    0.431 compiler.py:580(compile_extra)
     11/5    0.000    0.000    2.150    0.430 compiler.py:347(compile_extra)
     11/5    0.000    0.000    2.149    0.430 compiler.py:410(_compile_bytecode)
     11/5    0.000    0.000    2.149    0.430 compiler.py:370(_compile_core)
     11/5    0.003    0.000    2.146    0.429 compiler_machinery.py:316(run)
  242/110    0.001    0.000    2.144    0.019 compiler_lock.py:32(_acquire_compile_lock)
  242/110    0.005    0.000    2.141    0.019 compiler_machinery.py:257(_runPass)
  726/330    0.001    0.000    2.135    0.006 compiler_machinery.py:261(check)
   315/86    0.002    0.000    2.129    0.025 dimension.py:677(map)
        4    0.000    0.000    2.101    0.525 core.py:172(points)
        4    0.000    0.000    2.101    0.525 core.py:1161(bypixel)
      8/4    0.000    0.000    2.088    0.522 utils.py:102(__call__)
        4    0.000    0.000    2.088    0.522 pandas.py:15(pandas_pipeline)
        4    0.000    0.000    2.088    0.522 pandas.py:23(default)
        1    0.000    0.000    2.058    2.058 element.py:2322(initialize_plot)
76381/50977    0.101    0.000    2.011    0.000 parameterized.py:1060(override_initialization)
      276    0.001    0.000    1.960    0.007 __init__.py:1221(map)
        4    0.000    0.000    1.902    0.475 points.py:187(extend)
    12403    0.086    0.000    1.582    0.000 options.py:466(__init__)
       11    0.001    0.000    1.545    0.140 typed_passes.py:451(run_pass)
       11    0.001    0.000    1.539    0.140 typed_passes.py:360(run_pass)
     2565    0.005    0.000    1.516    0.001 plot.py:290(lookup_options)
     2565    0.035    0.000    1.511    0.001 options.py:82(lookup_options)
       14    0.001    0.000    1.504    0.107 element.py:1380(initialize_plot)
    25404    0.296    0.000    1.491    0.000 parameterized.py:1277(_setup_params)
     11/5    0.000    0.000    1.467    0.293 typed_passes.py:97(run_pass)
     11/5    0.000    0.000    1.466    0.293 typed_passes.py:61(type_inference_stage)
     11/5    0.000    0.000    1.459    0.292 typeinfer.py:1052(propagate)
    22/10    0.002    0.000    1.459    0.146 typeinfer.py:141(propagate)
   150/28    0.000    0.000    1.450    0.052 context.py:187(resolve_function_type)
   138/16    0.002    0.000    1.450    0.091 typeinfer.py:568(resolve)
     62/8    0.001    0.000    1.449    0.181 typeinfer.py:558(__call__)
   138/16    0.000    0.000    1.449    0.091 typeinfer.py:1500(resolve_call)
   150/28    0.001    0.000    1.449    0.052 context.py:231(_resolve_user_function_type)
     20/4    0.000    0.000    1.448    0.362 functions.py:521(get_call_type)
     20/4    0.000    0.000    1.447    0.362 dispatcher.py:300(get_call_template)
       74    0.000    0.000    1.431    0.019 model.py:98(collect_models)
       74    0.028    0.000    1.430    0.019 model.py:55(collect_filtered_models)
       73    0.001    0.000    1.412    0.019 model.py:554(references)
       14    0.001    0.000    1.408    0.101 element.py:803(_update_ranges)
628296/536118    0.186    0.000    1.404    0.000 {built-in method builtins.getattr}
       14    0.001    0.000    1.392    0.099 element.py:1342(_init_glyphs)
    10593    0.057    0.000    1.387    0.000 model.py:808(_visit_immediate_value_references)
       14    0.000    0.000    1.359    0.097 element.py:749(_update_plot)
       23    0.001    0.000    1.313    0.057 raster.py:83(get_data)
     2734    0.009    0.000    1.313    0.000 options.py:1255(lookup_options)
       66    0.001    0.000    1.305    0.020 plots.py:90(select)
     2734    0.022    0.000    1.302    0.000 options.py:755(closest)
        2    0.001    0.000    1.295    0.647 datashader.py:1522(_process)
       25    0.001    0.000    1.275    0.051 element.py:1828(_get_colormapper)
84694/82894    0.045    0.000    1.253    0.000 descriptors.py:464(__get__)
78574/77640    0.067    0.000    1.246    0.000 descriptors.py:676(_get)
       15    0.001    0.000    1.212    0.081 plot.py:1838(get_extents)
       14    0.000    0.000    1.197    0.085 element.py:759(_update_labels)
       23    0.001    0.000    1.194    0.052 util.py:887(process_cmap)
        2    0.001    0.001    1.183    0.592 datashader.py:1479(_process)
       55    0.001    0.000    1.181    0.021 <frozen importlib._bootstrap>:1002(_find_and_load)
       52    0.000    0.000    1.178    0.023 <frozen importlib._bootstrap>:967(_find_and_load_unlocked)
10936/2734    0.064    0.000    1.177    0.000 options.py:772(options)
49079/48145    0.076    0.000    1.164    0.000 descriptors.py:704(_get_default)
       30    0.002    0.000    1.161    0.039 plot.py:598(compute_ranges)
  310/156    0.004    0.000    1.158    0.007 operation.py:197(__call__)
       63    0.001    0.000    1.155    0.018 util.py:666(_list_cmaps)
        2    0.000    0.000    1.136    0.568 <frozen importlib._bootstrap>:659(_load_unlocked)
        2    0.000    0.000    1.136    0.568 <frozen importlib._bootstrap_external>:784(exec_module)
        2    0.000    0.000    1.129    0.565 <frozen importlib._bootstrap>:220(_call_with_frames_removed)
        1    0.005    0.005    1.129    1.129 __init__.py:1(<module>)
       14    0.000    0.000    1.121    0.080 element.py:763(<dictcomp>)
       28    0.002    0.000    1.121    0.040 element.py:647(_axis_properties)
       78    0.022    0.000    1.120    0.014 plot.py:717(_compute_group_range)
      214    0.002    0.000    1.092    0.005 __init__.py:72(mpl_cm)
      214    0.108    0.001    1.086    0.005 colors.py:776(from_list)
49079/48145    0.051    0.000    1.057    0.000 descriptors.py:584(instance_default)
   122/69    0.000    0.000    1.042    0.015 accessors.py:575(_dispatch_opts)
   122/69    0.001    0.000    1.042    0.015 accessors.py:637(_base_opts)
       69    0.001    0.000    1.035    0.015 dimension.py:1230(options)
       15    0.004    0.000    1.030    0.069 plot.py:1798(_get_subplot_extents)
       69    0.000    0.000    1.025    0.015 __init__.py:192(apply_groups)
       69    0.000    0.000    1.024    0.015 __init__.py:157(_apply_groups_to_backend)
      840    0.015    0.000    1.022    0.001 plot.py:1416(get_extents)
      207    0.003    0.000    1.006    0.005 accessors.py:30(pipelined_call)
49765/48831    0.076    0.000    0.994    0.000 bases.py:182(themed_default)
       53    0.000    0.000    0.951    0.018 accessors.py:524(__call__)
    55041    0.129    0.000    0.927    0.000 colors.py:161(to_rgba)
       34    0.000    0.000    0.904    0.027 accessors.py:625(<lambda>)
 1932/868    0.007    0.000    0.893    0.001 overlay.py:139(__init__)
 1932/868    0.018    0.000    0.890    0.001 dimension.py:1350(__init__)
       69    0.002    0.000    0.874    0.013 options.py:1791(set_options)
7948/1531    0.026    0.000    0.865    0.001 dimension.py:637(traverse)
4757/2489    0.017    0.000    0.818    0.000 tree.py:106(set_path)
2683/1398    0.013    0.000    0.791    0.001 tree.py:44(__init__)
       13    0.000    0.000    0.774    0.060 spaces.py:1443(apply_map)
    55041    0.327    0.000    0.772    0.000 colors.py:197(_to_rgba_no_colorcycle)
165246/165153    0.064    0.000    0.740    0.000 {built-in method builtins.setattr}
       69    0.003    0.000    0.728    0.011 options.py:1623(create_custom_trees)
114088/112910    0.120    0.000    0.701    0.000 {built-in method builtins.any}
      154    0.000    0.000    0.688    0.004 options.py:983(<lambda>)
      155    0.006    0.000    0.688    0.004 options.py:902(collapse_element)
     6129    0.527    0.000    0.680    0.000 ffi.py:149(__call__)
       73    0.002    0.000    0.677    0.009 plot.py:909(_traverse_options)
      884    0.003    0.000    0.668    0.001 plot.py:918(lookup)
     2089    0.018    0.000    0.657    0.000 dimension.py:849(__init__)
     8464    0.023    0.000    0.654    0.000 options.py:261(__init__)
    52645    0.133    0.000    0.653    0.000 bases.py:328(prepare_value)
94528/93761    0.117    0.000    0.620    0.000 parameterized.py:313(_f)
       11    0.000    0.000    0.611    0.056 lowering.py:131(lower)
     2089    0.024    0.000    0.598    0.000 dimension.py:491(__init__)
      645    0.004    0.000    0.548    0.001 plot.py:1311(get_padding)
      486    0.009    0.000    0.536    0.001 options.py:697(__setattr__)
1677024/1674568    0.375    0.000    0.532    0.000 {built-in method builtins.isinstance}
     1940    0.016    0.000    0.510    0.000 options.py:639(_merge_options)
      272    0.004    0.000    0.507    0.002 dimension.py:539(clone)
     1542    0.026    0.000    0.499    0.000 util.py:1708(stream_parameters)
       11    0.000    0.000    0.483    0.044 cpu.py:197(get_executable)
      974    0.002    0.000    0.481    0.000 util.py:1748(dimensionless_contents)
       22    0.000    0.000    0.481    0.022 codegen.py:946(get_pointer_to_function)
    62/43    0.000    0.000    0.480    0.011 codegen.py:566(_ensure_finalized)
       11    0.000    0.000    0.480    0.044 codegen.py:673(finalize)
    93761    0.279    0.000    0.469    0.000 parameterized.py:859(__set__)
       69    0.001    0.000    0.462    0.007 options.py:1667(<dictcomp>)
      265    0.002    0.000    0.460    0.002 options.py:1533(apply_customizations)
     1623    0.013    0.000    0.452    0.000 util.py:1043(dimension_range)
      630    0.007    0.000    0.442    0.001 plot.py:1345(_get_range_extents)
       11    0.000    0.000    0.440    0.040 lowering.py:182(lower_normal_function)
     9465    0.075    0.000    0.428    0.000 parameterized.py:1694(get_param_values)
       11    0.000    0.000    0.419    0.038 lowering.py:196(lower_function_body)
       47    0.004    0.000    0.417    0.009 lowering.py:220(lower_block)
105572/105355    0.078    0.000    0.416    0.000 {built-in method builtins.all}
     2346    0.012    0.000    0.415    0.000 parameterized.py:2995(instance)
    25404    0.074    0.000    0.414    0.000 parameterized.py:1271(_generate_name)
     8713    0.024    0.000    0.412    0.000 streams.py:400(contents)
       79    0.002    0.000    0.411    0.005 raster.py:266(__init__)
      657    0.006    0.000    0.402    0.001 lowering.py:321(lower_inst)
       33    0.001    0.000    0.395    0.012 codegen.py:654(add_ir_module)
     2450    0.021    0.000    0.389    0.000 tree.py:231(__getattr__)
     4678    0.041    0.000    0.383    0.000 util.py:965(max_range)
       20    0.002    0.000    0.366    0.018 datashader.py:885(_process)
    43801    0.085    0.000    0.355    0.000 parameterized.py:1346(_instantiate_param)
17199/13918    0.029    0.000    0.344    0.000 {built-in method numpy.core._multiarray_umath.implement_array_function}
48463/47904    0.030    0.000    0.333    0.000 bases.py:172(_raw_default)
       85    0.001    0.000    0.333    0.004 __init__.py:1179(clone)
      461    0.001    0.000    0.325    0.001 tree.py:160(__setitem__)
   458593    0.251    0.000    0.324    0.000 parameterized.py:837(__get__)
    14698    0.118    0.000    0.322    0.000 util.py:874(isfinite)
       11    0.000    0.000    0.313    0.028 codegen.py:720(_finalize_final_module)
      640    0.004    0.000    0.310    0.000 raster.py:525(range)
       11    0.000    0.000    0.309    0.028 lowering.py:233(create_cpython_wrapper)
       11    0.000    0.000    0.309    0.028 cpu.py:154(create_cpython_wrapper)
49765/48831    0.043    0.000    0.308    0.000 bases.py:161(_copy_default)
   294751    0.219    0.000    0.304    0.000 parameterized.py:2544(param)
       52    0.000    0.000    0.292    0.006 raster.py:415(clone)

(I will be offline for most of the day, but feel free to play around with the Colab Notebook if you need more info, it works without any additional files or dependencies and provides the same results, just with slightly sower runtimes.)

[SteveAKopias]

To clarify, these are some examples where I’m not sure I used the tools correctly:

.link() and @pn.depends()

def split_checkboxes_to_individual_values(target, event):
    """
    Function used to link a checkboxgroup to individual checkboxes.
    Namely it will be used to enable/disable pn_is_active[category_name] values
    based on the selected values of pn_layers_all
    """
    category_name = target.name.split('__')[0]
    is_active = 1 if category_name in event.new else 0
    if target.value != str(is_active):
        target.value = str(is_active)

pn_layers_options = ['terraclimate', 'geography', 'population']
pn_layers_default_values = ['geography']
pn_layers_all = pn.widgets.CheckBoxGroup(name='Layers', 
    value=pn_layers_default_values, options=pn_layers_options, inline=False)

# These widgets are not displayed, but they are used to be able to avoid triggering 
# changes in every layer when only one layer was changed, as a function dependent 
# on aCheckBoxGroup would be triggered every time, even if the function is 
# interested in the state of only one checkbox.
pn_is_active = {}
for category_label, category_name in pn_layers_options.items():
    default_is_active_value = 1 if category_name in pn_layers_default_values else 0
    pn_is_active[category_name] = pn.widgets.TextInput(
        name=category_name + '__is_active', value=str(default_is_active_value))
    pn_layers_all.link(pn_is_active[category_name], 
        callbacks={'value': split_checkboxes_to_individual_values})

# The pn_is_active[*] widgets then used like this:
@pn.depends(
    pn_layer_is_active=pn_is_active['population'],
    pn_resolution=pn_resolutions.param.value,
    pn_date=pn_dates.param.value_throttled,
    pn_population_gender=pn_population_genders.param.value,
    pn_population_age_limit=pn_population_age_limits.param.value_throttled,
)
def get_image_population(
        pn_layer_is_active,
        pn_resolution,
        pn_date,
        pn_population_gender,
        pn_population_age_limit,
):
    """
    Processes the relevant widget values and loads a population image based on them.
    """
    category_name = 'population'
    layer_is_active = int(pn_layer_is_active)

    if layer_is_active:
        # load_population_image replaced:
        population_image=hv.Image(np.ones((100, 250)), 
            bounds=(mw_left+2000,mw_bottom,mw_right,mw_top+2000))
    # if the layer is turned off, returns an empty image
    else:
        population_image = malawi_nan_image.clone()
        
    return population_image

the .apply()s building on each other:

# setting up streams to use in determining zoom
    stream_rangexy_source_original_wpc_hv_points = hv.streams.RangeXY(source=original_wpc_hv_points)
    streams = [stream_rangexy_source_original_wpc_hv_points]
    
    # only work with the currently visible points
    def get_points_in_viewport(original_points, pn_is_active_waterpoints, x_range, y_range):
        """
        Gives back currently visible waterpoints.
        If the value of the pn_is_active_waterpoints Widget is False, it returns fake_points.
        If it is True, then if range is set, it returns the points in range and every point
        otherwise.
        """
        if pn_is_active_waterpoints:
            if x_range is None or y_range is None:
                return original_points
            else:
                return original_points[x_range, y_range]
        else:
            return fake_points

    original_wpc_hv_points = hv.Points(wp, ['easting', 'northing']).apply(
        get_original_wpc_hv_points,
        pn_date=pn_dates,
        pn_wp_expiry=pn_wp_expiries
    )

    points_in_viewport = original_wpc_hv_points.apply(
       get_points_in_viewport,
       pn_is_active_waterpoints=pn_is_active_waterpoints_chk, streams=streams
    )
    
    def get_wp_hv_points(points_in_viewport, pn_is_active_waterpoints=True, threshold=5000):
        """
        Based on the currently visible (fake or real) points, it either gives back fake 
        points to force displaying the legend, but otherwise nothing or all the points, 
        depending on their number being higher or lower than the threshold.
        Note: for some reason the fake points had to be recreated from data otherwise some 
        weird bug appeared.
        """
        if pn_is_active_waterpoints:
            if len(points_in_viewport) > threshold:
                ret = hv.Points(fake_points_data, ['easting', 'northing'])
                return ret
            else:
                return points_in_viewport.options(color_points_by__cutoff_status_latest_result) \
                    .options(marker=hv.dim('cutoff_status_latest_result').categorize({'Functional': 'plus', 'Needs repair': 'circle', 'Not functional': 'square', 'Unknown': 'triangle'}))
        else:
            return hv.Points(fake_points_data, ['easting', 'northing'])

    # get either fake or real Points to display
    wp_hv_points = points_in_viewport.apply(
        get_wp_hv_points,
        pn_is_active_waterpoints=pn_is_active_waterpoints_chk
    )
    
    def get_wp_points_for_aggregate(points_in_viewport, pn_is_active_waterpoints=True, threshold=5000):
        """
        Opposite logic than the get_wp_hv_points(). Based on the currently visible 
        (fake or real) points, it either returns fake Points or the real points, 
        depending on their number being *lower* or *higher* than the threshold.
        These points then can and will be used to generate aggregate data.
        """
        if pn_is_active_waterpoints:
            if len(points_in_viewport) <= threshold:
                ret = hv.Points(fake_points_data, ['easting', 'northing'])
                return ret
            else:
                return points_in_viewport
        else:
            ret = hv.Points(fake_points_data, ['easting', 'northing'])
            return ret
        
    # get either fake or real Points to aggregate
    wp_points_for_aggregate = points_in_viewport.apply(
        get_wp_points_for_aggregate,
        pn_is_active_waterpoints=pn_is_active_waterpoints_chk)