Hey all, I’m a financial data scientist, and I have been aggressively adopting the holoviz stack over the past two years for my development, visualization, and dashboarding workflow.
It has recently clicked that Param is the foundation that enables many of these powerful tools. I feel like Param makes python a strongly typed, object friendly, scalable systems architecture programming language. I feel like this is a game changing development, and I am really surprised that Param is not more widely known, used, and discussed. I find there are extremely few articles/stack overflow posts about Param, and I think this should change over time.
My question to the community: Do you guys see how profound Param is as a simple foundation for hierarchical object development in python? Or am I getting a little too hyped about all of this?
Param is definitely the foundation that makes it possible for a small group like ours to maintain tools this powerful without going crazy. It lets us set up interfaces that are well defended, ensuring that each specific part of the code (and each member of the team) doesn’t have to reason about all the other parts of the code. The richly declared Parameterized interfaces let us state clearly what each class or function does, and then we can just leave it, secure knowing that we’ll be alerted when we or users do anything not allowed by that contract. Whenever I hear people complaining about Python’s lax typing or Python not being appropriate for large codebases or anything like that, mentally I always say “well, of course, unless you use Param”. Of course, as the main person who made Param come into existence (not by coding it, but by noisily demanding it of my employees and students), the fact that I think that shouldn’t be surprising. But it’s nice to hear it from someone else who really “gets it”.
As for why it’s not widely known or used for that purpose, that’s partly from history, in that for the first 10 years of its life we didn’t make it a separate package from our neuroscience-focused academic research simulator, and even when we did make it a proper standalone package we didn’t document or promote it very well. It still isn’t really documented, but we’ve now gotten funding to document it properly sometime in the next few months, so that’s finally going to happen. You can blame academic research for not rewarding anything but an impactful publication in your own narrow subfield; Param is definitely something that’s never going to get any Journal of Neuroscience citations! Or you can blame me having young children while trying to establish an academic career. Or you can blame Param never quite being what anyone intentionally sets out to look for; instead they want a web toolkit or machine learning library or something with more tangible immediate applications. Probably it’s all these things and more! But I’d love to hear ideas how it could be more widely known and appreciated; I cringe every time I see other large Python codebases fully of dozens of assert statements and other error checking code that still makes things only 1/10th as robust as even the most basic Parameterized objects are.
Glad there is going to be documentation soon! Myself I don’t think I really appreciated param before getting into panel and way after I started digging around in the holoviews codebase. But in hindsight it all makes sense: Just like holoviews is about annotating data and letting the library figure it out from there, param lets you annotate the code and let param do the piecing together.
So yes, lack of documentation has definitely been a hurdle for me, but it also has an approach quite different from most other python (data science) libraries I’ve seen.
Wow jbednar, thanks for this awesome answer! It’s nice to get a glimpse into the lore behind holoviz. Really interesting stuff, and I am glad to hear that you have secured funding to improve the documentation.
It’s kind of cool ‘being on the inside’ of this circle of people who see the potential of the holoviz stack and are wielding it at early stages. At the same time, I am happy to know that it is starting to get the attention it deserves and will spread to the masses over time.
The way you cringe at code full of assert statements is exactly how I cringe at my own code that I wrote before introducing param into my workflow.
Thank you very much to you and your team for such great contributions. I think you will secure the credit you deserve far beyond what a simple impactful publication would do in the long run. I’ll do my best to start contributing where I can since I gain so much value from these libraries. Cheers
I started working with Panel about 2 weeks ago, and very quickly realized that it heavily leverages Param. However, I have not been able to figure out Param, and my sense is that it would be extremely useful if I really understood it. Is there an update on improved documentation as mentioned by @jbednar at the end of March? I would love to go through it!
Also, once I understand it, I think Param would be helpful for many of the software projects in my academic research group, and I would be very happy to reference and promote it within my academic and commercial circles (we spun out a company). Perhaps it would make sense to consider writing and submitting a paper on Param to the Journal of Open Source Software (https://joss.theoj.org)? It seems like a natural fit.
I found the https://github.com/holoviz/panel/blob/master/examples/user_guide/Param.ipynb notebook and went through it, which was helpful. This is the most complete documentation about Param that I have found. It still doesn’t explain why there is a .param attribute on a class subclassed from param.Parameterized, or exactly how you work with it based on what I have seen in other examples like at https://discourse.holoviz.org/t/how-to-trigger-param-depends-only-once-when-updating-multiple-parameters/848.
Sadly there has been no progress on the documentation but it’s a major priority. Not only because it’s long overdue for our users but also because we have a consulting contract which calls this out as a specific task.
I’m glad to hear that there is a consulting contract task for the documentation. For me this has been the biggest hurdle in getting up to speed with Param. However, this is offset by how responsive you and Jim are, and Marc. I really appreciate each of you, particularly since I am sure you have many other demands on your time.
You’re not the first one being after for a more extensive Param documentation (I was too! ) . I guess that this package used to be mostly used by the Holoviz devs to maintain their other packages like Holoviews, Geoviews, etc. Panel has now uncovered Param with the handy use of parameterized classes to create dashboards. It seems that it’ll get a documentation in the coming months, given how good is the documentation over the whole Holoviz ecosystem we can expect something real good, it’s just great! In the meantime the community (us!) could also start documenting how we use Param, writing blog posts for instance. That would also be great! But first you could check https://awesome-panel.org/ where you’ll find plenty of scripts using both Panel and Param.
(Typing sheepishly) Yes! There is now a dev version of it up at Welcome to Param! — param 1.10.2a7 documentation , though it is far from complete. This week I’m focusing on a Datashader release, but Param is next after that. Really!
Glad you liked it! I’ll find time to finish the remaining pages soon. I don’t think there are any academic papers specifically about Param. We developed in in the context of the Topographica project, which we’ve written a few papers about (Google Scholar), but I don’t think we even mentioned it in those. Yet that (along with HoloViews) is the part of Topographica that lives on!
) . I guess that this package used to be mostly used by the Holoviz devs to maintain their other packages like Holoviews, Geoviews, etc. 