I am interested in implementing the Pydantic getting started example in Param. Right now I am stuck on this:
signup_ts: Optional[datetime] = None
I believe that to emulate “Optional” we supply allow_none=True, correct?
But given that only dates are directly supported how do we create an optional datetime field using Param?
Hi @metaperl
Welcome to the community. I’m also interested in the relation between Param and Pydantic.
The Pydantic getting started example would look something like
import param
from typing import List, Optional
from datetime import datetime
import json
class User(param.Parameterized):
id: int = param.Integer(doc="A Unique ID")
name: str = param.String(default='John Doe', doc="""The full name of the user""")
signup_ts: Optional[datetime] = param.Date(allow_None=True, doc="""The sign up time of the user""")
friends: List[int] = param.List(doc="""A list of friends given by User id""")
external_data = {
'id': 123,
'signup_ts': '2019-06-01T12:22:00.000',
'friends': [1, 2, 3],
}
json_data = json.dumps(external_data)
# See https://param.holoviz.org/user_guide/Serialization_and_Persistence.html?highlight=deserialize#serializing-with-json
deserialized_data = User.param.deserialize_parameters(json_data)
user = User(**deserialized_data)
print(user.id)
print(repr(user.signup_ts))
print(user.friends)
print(user.param.serialize_parameters())
Most people would probably not include the types int, str etc. on the parameters. As you can see I had to change the external_data a bit as Param is very strict with respect to serialization and deserialization.
The output looks like
$ python 'script.py'
123
datetime.datetime(2019, 6, 1, 12, 22)
[1, 2, 3]
{"name": "User00002", "id": 123, "signup_ts": "2019-06-01T12:22:00.000000", "friends": [1, 2, 3]}
But. It seems that Date does not work with allow_None · Issue #578
Depending on your use case you can fix it by implementing a custom parameter (or contributing a fix to Param).
import datetime as dt
import json
import param
from param import Date, dt_types
class AllowNoneDate(Date):
"""
Date parameter of datetime or date type.
"""
def _validate_step(self, val, step):
if step is not None and not isinstance(step, dt_types):
raise ValueError("Step parameter can only be None, a datetime or datetime type")
@classmethod
def serialize(cls, value):
if not value:
return None
return super().serialize(value)
@classmethod
def deserialize(cls, value):
if value=="null" or not value:
return None
return super().deserialize(value)
class User(param.Parameterized):
signup_ts = AllowNoneDate(allow_None=True, doc="""The sign up time of the user""")
external_data = {
'signup_ts': None,
}
json_data = json.dumps(external_data)
deserialized_data = User.param.deserialize_parameters(json_data)
user = User(**deserialized_data)
print(user.param.serialize_parameters())
$ python 'script.py'
{"name": "User00002", "signup_ts": null}
Please write a bit about your use case and why it would be interesting to check out Param. Thanks.
Interesting exercise you’re doing @metaperl ! I wonder what Param is missing on this side to make it more useful and user friendly (I assume Pydantic has theses properties, given its popularity, might be wrong there!).
Is this documented in the user guide or tutorial? I wasnt aware that Param had any way to have custom parameters.
I saw the live webinar on Param 1-2 months ago and was impressed. I’m always researching and evaluating python object systems to see which is the best.
Feature-wise, Param is on par with Traits, Traitlets and Nucleic because it has reactive objects, which Pydantic does not.
but the time being, i cannot recommend Param if even a trivial example from pydantic docs takes this much work and exposes trivial misfeatures.
But given that only dates are directly supported (Parameter types — param v2.0.1)
For historical reasons, the param.Date Parameter type supports both date and datetime types, despite its name; we later added CalendarDate for values specific to dates only. So unless I’m misunderstanding, it’s not true that only dates are directly supported in Param.
But it seems that Date does not work with allow_None (Serializing Date parameters does not support allow_None · Issue #578 · holoviz/param · GitHub)
To be more specific, serialization of Dates did not support None; Date and CalendarDate have always supported None. Thanks to Marc’s bug report, a fix for serializing None in Date and various other fancy types has just been merged to master and will be in the next Param release. So far None hasn’t come up for serializing those types, because we use serialization primarily for communicating with a GUI, and (unfortunately) the available GUI widgets do not support None in any case.
Is this documented in the user guide or tutorial? I wasnt aware that Param had any way to have custom parameters.
Yes, Parameters and Parameterized objects — param v2.0.1 covers how to write a custom Parameter class, but it shouldn’t be needed in this case, as None is already allowed for Dates and similar types. Users are definitely encouraged (a) to avoid custom types when there is already a suitable type available, but (b) to add a custom type wherever it makes sense, so that they can maintain tight control over allowable values in their applications.
I cannot recommend Param if even a trivial example from pydantic docs takes this much work and exposes trivial misfeatures.
It’s unlikely that code for any library can trivially be adapted to use another library unless at least one of the libraries was written with the other in mind. Pydantic was written independently of Param, about 14 years later than Param, and so it focuses on the specific use cases of Pydantic’s author (REST APIs and Django initially, I think). Param focused on quite different use cases (batch job configuration + optional native OS GUI support initially), with serialization added relatively late in its lifetime and used in relatively few of its applications. So it’s not surprising that there are holes in Param’s serialization support, just as there are plenty of things implemented in Param but not (yet?) in Pydantic.
For users, it would be great to smooth the transition from Pydantic to Param, so that people could more easily make use of Param’s additional features where they are desired. So it would be amazing if a Pydantic user who is interested in something provided by Param would exercise the parts of Param that correspond to Pydantic, submitting detailed bug reports, PRs, and docs that help smooth that transition. E.g. I’d love to see a user guide page explaining how to use Param with Pydantic classes or for Pydantic users, letting people pick the best library for the task they are doing without getting stuck in one moat or the other.
BTW, here’s how I’d translate the pydantic home page example into Param:
import param
class User(param.Parameterized):
id = param.Integer()
name = param.String()
signup_ts = param.Date(None)
friends = param.List([], item_type=int)
from datetime import datetime
external_data = {
'id': 123,
'name': 'John Doe',
'signup_ts': datetime.strptime('2019-06-01 12:22', '%Y-%m-%d %H:%M'),
'friends': [1, 2, 3],
}
user = User(**external_data)
print(user.id)
#> 123
print(repr(user.signup_ts))
#> datetime.datetime(2019, 6, 1, 12, 22)
print(user.friends)
#> [1, 2, 3]
print(user.param.values())
"""
{
'friends': [1, 2, 3],
'id': 123,
'name': 'John Doe',
'signup_ts': datetime.datetime(2019, 6, 1, 12, 22)
}
"""
This is essentially a direct translation, with some very specific differences:
id is an integer field, Param will raise an exception (by design) for id='123', while Pydantic accepts that and coerces the string to an int.friends= [1, 2, '3'], while Pydantic will coerce it.name is the one Parameter that Parameterized objects always have, and it’s also the only one that has special semantics in Param: If the name is not set on an instance or in its constructor, it has a name autogenerated for it from the class name. The intention of this mechanism is for each object to have some relatively unique name to distinguish it when exploring collections of objects, though that’s not necessarily always useful. In any case, here, if we want it to have the value John Doe as in the Pydantic example, we have to provide that name explicitly on the instance as above. The original Pydantic example doesn’t make a whole lot of sense for name anyway, as it would be a recipe for errors for something like a name to inherit a concrete value like John Doe from the base class.I’d be happy to have this example in the docs or the Comparisons page to clarify how Param and Pydantic relate; PRs welcome!
On of the things you made me aware of @jbednar is the difference between parsing and validation.
Pydantic is focusing on parsing ( Clarify the purpose of pydantic e.g. not strict validation · Issue #578 · samuelcolvin/pydantic (github.com) and Param is focusing on validation.
One more difference as I see it is that Pydantic is heavily used for serialization/ deserialization in a REST API context. Param has not been so far.
Your feedback @metaperl is very much appreciated. You have a broad, outside-in view and can give us eye opening perspectives on how to improve. Or open up to new use cases we did not think of.
So please share as many observations and suggestions as possible.
I would also dream of one day seeing Panel evaluated and categorized in your metaperl/pure-python-web-development: Avoid the CSS/JS/HTML soup - develop web apps entirely in Python (github.com)
Thanks.
How about adding a Datetime parameter type?
I’m happy if you want to rename Date to Datetime and keep Date as an alias for Datetime; seems much clearer!
I don’t think it’s worth it if it’s just renaming.
Look for instance the mapping used by Panel to go from a Parameter to a widget:
param.CalendarDate: DatePicker,
param.Date: DatetimeInput,
I believe this code would be clearer if param.Date were to be replaced by a new Parameter type called param.Datetime that would accept datetime objects only.
It’s fine if Datetime only accepts datetime, sure. But then we couldn’t ever get rid of Date because it’s the only one that accepts both date and datetime objects.
I’ve started working on it. I have no background in Jupyter notebooks, so I manually attempted to get the Getting Started code to work. I’ve run into a bit of a stumbling block.
I hit this recently and came up with the following workaround. Its not a complete implementation but enough to help someone who might be looking for the same.
import param
def get_type_dict(cls):
return {k: v['type'] for k,v in cls.param.schema().items() if 'type' in v}
def coerce_to_type(typename, value):
if typename == 'integer':
return int(value)
elif typename == 'number':
return float(value)
elif typename == 'boolean':
return bool(value)
else:
raise f'Unknown typename {typename}'
def get_value_dict(map, cls):
''' Create a dict from namelist from name: value elements'''
xdict = {key:map[key] for key in map}
tdict = get_type_dict(cls)
for p in cls.param:
if p in xdict:
xdict[p] = coerce_to_type(tdict[p],xdict[p])
return xdict
class MyClass(param.Parameterized):
myint = param.Integer(default=5)
myfloat = param.Number(default=0.23)
params = {'myint': '2', 'myfloat': '0.46'}
my_instance = MyClass(**get_value_dict(params,MyClass))
my_instance
MyClass(myfloat=0.46, myint=2, name=‘MyClass00008’)