Please support static typing

[MarcSkovMadsen]

My Pain

I'm developing my applications in the VS Code editor. Furthermore I use pylint and mypy to help me create code of high quality. And google style docstring for documentation.

Param unfortunately does not work that well with that.

I've tried something like that.

class Scatter2dWithSelectionComponent(param.Parameterized):
    """The Scatter2dWithSelectionComponent enables a user to

    - provide a DataFrame of data
 
    Parameters
    ----------
    data : pd.DataFrame
        A dataframe of data to be shown and selected
    """
    data: pd.DataFrame = param.DataFrame(precedence=0.1)

But

• VS Code cannot provide tab completion for instance.data. or help text on hover of instance.data.
• Pylint complains for example that E1101: Instance of 'DataFrame' has no 'iloc' member (no-member) if I have a line like instance.data.iloc.

Solution

Support static type checking and enable all the help you can get in modern editors.

[jbednar]

We have been focusing on simultaneously supporting Python 2 and Python 3 in the same codebases for the past 5 years, so we haven't been able to make use of any Python3-only syntax like type hinting in our own code. Once we are finally able to leave Python 2 behind, we can become experts in Python 3-only stuff, but for the moment I'm not actually sure what the above type annotation is meant to indicate. At the class level "data" is not a pd.DataFrame; it's a param.DataFrame object, so I'm not sure that's the correct type annotation to supply. But in the instance it will be a pd.DataFrame, which is what matters for instance.data.iloc.

I really don't know if Python 3's type hints are expressive enough to distinguish between type differences between the class and instance levels. So I can't say if this is something that is supported now but with a different syntax, if it can be supported but would require changes to the ParameterizedMetaclass, or if it is simply not possible given the type differences between Parameterized classes and instances (which are inherent to Param's design). Personally, I've found Parameter's type declarations to be vastly more powerful than static types (as they allow specifying ranges and allowed values and not simply type name), but it sounds like the fact that the type is static is something your editor can exploit, and of course the editor doesn't know about Param. Anyway, I wouldn't personally be ready to tackle this issue until we've completely left Python2 behind for good, but if someone else can see a way, be my guest!

[MarcSkovMadsen]

Thanks for the discussion.

I understand that that is a matter of resources and priority. I just wan't to raise it because it's important to the way I work and people working similarly.

I actually like param a lot. That is part of why I like Panel. I have no experiences from python 2 :-)

[MarcSkovMadsen]

I believe this link explains how to add type annotations to descriptors and thus to Param. https://stackoverflow.com/questions/57400478/using-typing-and-mypy-with-descriptors.

[jbednar]

I have no experiences from python 2 :-)

Lucky you! :-)

I'm happy for param 2 to move away from Python2.7 support. Param 2 can happen this summer; it would be the next big thing after we have proper docs (as it basically requires deleting whatever's not in the docs as being no longer supported!) I.e., it's not a big jump from where Param is now to 2.0; it's just an agreement to clean out a bunch of stuff and not look back. So yes, if we can make use of type hints at that point, great!

[jbednar]

https://atom.readthedocs.io/en/latest/basis/typing.html might be helpful for understanding how a similar library moved to start including py3 types. Param 2.0 is very close to release, and that's when we can drop py2 and consider how to move forward. I'd also be interested in seeing if there is some lightweight way to decorate a py3 dataclass to use its types in Param, as a limited but useful entry into Param's functionality using py3 type hints.

[MarcSkovMadsen]

For Param to be really easy to understand and use in a modern IDE lots of type annotations and just getting rid of all the *'params would be need.

Example - Number Parameter

The arguments of the Number.__init__ method would need 1) type annotations 2) docstrings describing the arguments

Example Integer Parameter

The __init__ method would need named arguments instead of the **params and 1) + 2) from above.

Would you be open to accept that kind of PRs @jbednar ?

[philippjfr]

For Param to be really easy to understand and use in a modern IDE lots of type annotations and just getting rid of all the *'params would be need.

Can you expand here, yes it might make declaring parameters easier but that's not really the important case right? Figuring out the types of the parameters themselves (like a dataclass) is what we should optimize for no?

As for this concrete question, I find the idea of repeating all arguments for a parameter whenever you subclass a parameter to be kind of crazy and ugly. Is there really no mechanism for Python typing modules to figure out types from the arguments accepted by the baseclass and the subclass?

[philippjfr]

Specifically why I find this so crazy and ugly is that it introduces a significant risk of argument definitions to become outdated/drift out-of-sync. Adding an argument on the baseclass which is then not also declared on all subclasses is a significant issue that is avoided by passing through **params.

[maximlt]

Would you be open to accept that kind of PRs @jbednar ?

Unfortunately this will have to wait for at least one more release though, i.e. until the moment it is declared that the next release will be Param 2.0.

As for this concrete question, I find the idea of repeating all arguments for a parameter whenever you subclass a parameter to be kind of crazy and ugly. Is there really no mechanism for Python typing modules to figure out types from the arguments accepted by the baseclass and the subclass?

I also find that a bit disappointing, I haven't yet found a solution to this. See this related discussion: github.com/python/typing/discussions/1079

[philippjfr]

Thanks for that reference @maximlt. I would have no particular objection to a @copy_type decorator as outlined in that issue.

[maximlt]

And two other (long) issues on typing for reference:

• Proposal: inheritance + annotations python/typing#269
• Proposal: signature copying for kwargs. python/typing#270