Skip to content

Classes available during type checking only #597

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
srittau opened this issue Dec 5, 2018 · 17 comments
Closed

Classes available during type checking only #597

srittau opened this issue Dec 5, 2018 · 17 comments

Comments

@srittau
Copy link
Collaborator

srittau commented Dec 5, 2018

This came up in srittau/peps#21: Stubs may need to define classes that are not available at runtime. Best examples are the return types of re.compile() and re.match(), although those are handled specially in typing.

One solution we came up with is to add a class decorator to typing, for example @type_check_only. While type checking this is a no-op, during runtime it mangles the name of the class, so that it is not available in the module namespace under its original name.

This is mostly intended for type stubs, but could be used in implementations (starting in Python 3.8). We would add it to typeshed without a version check, so it can be used in stubs immediately. Alternatively, type checkers could special case the import. The documentation should also recommend to make the real class available in own code, instead of using the decorator,

@rchen152 @Solumin

@gvanrossum
Copy link
Member

How do you propose to mangle the name in a decorator at runtime? The decorator has no control over the name to which the result is assigned.

@srittau
Copy link
Collaborator Author

srittau commented Dec 5, 2018

Actually mangling the name is probably not the best option. Having the name importable at runtime makes sense so it can be used in type annotations without if TYPE_CHECKING guards (if available in the implementation). Instead it could return a dummy callable that raises an exception if instantiated.

@JelleZijlstra
Copy link
Member

An alternative way to do this would be that the decorator is only available in stubs. Type checkers would be expected to produce an error for any use of a class decorated with @type_check_only outside of an annotation.

I don't see much use for using this decorator at runtime; if you're exporting the class anyway, you might as well make it visible. Therefore, people using such classes would still have to import them within if TYPE_CHECKING:.

@srittau
Copy link
Collaborator Author

srittau commented Dec 5, 2018

I don't have a really convincing argument why the decorator needs to be available at runtime. The quite weak arguments I have are:

  • Symmetry, it "feels" wrong to have a standard feature only typeshed
  • You never know, we might be overlooking use cases where you need to have it at runtime
  • Giving stubs a capability that implementations don't have also feels wrong

These are the reasons that make me believe (although not very strongly) that having it available at runtime makes some sense.

Another alternative runtime implementation is to immediately raise an error if type_stub_only is called. This means it's not possible to annotate anything at runtime, but the decorator would be available at runtime. Of course that would make it inclusion at runtime even more questionable.

@Solumin
Copy link

Solumin commented Dec 5, 2018

Giving stubs a capability that implementations don't have also feels wrong
I generally agree, but stub-only classes already are available in stubs. It makes sense the decorator for them is, too, especially since we don't have any ideas for how this would be useful at runtime.

@srittau
Copy link
Collaborator Author

srittau commented Dec 8, 2018

Based on this discussion, I would suggest to do the following:

  • Add type_stub_only to typeshed.
  • Document @type-stub-only in Python's typing documentation with a clear warning that it isn't available at runtime.

If it turns out we need a runtime version, we will add it then.

@rpgoldman
Copy link

Let's say we are creating a set of type stubs for a library that we do not control.
In order to capture some aspect of the behavior of the library, we want to introduce a new type name that's exported from the type stubs.
This almost seems like the case you discuss here, except that we can't import the new type name only at type checking time, can we, if we are using it in type hints?
For example:

if TYPE_CHECKING:
   from library import behavior_class

my_var : behavior_class

My example arises from a library that offers a number of data structures that all support a particular kind of lookup behavior, but that are not all of the same class. I do not control this library, so I can't push behavior_class into the code that operates at runtime.

@srittau
Copy link
Collaborator Author

srittau commented Dec 8, 2018

@rpgoldman This is exactly my use case. You can quote the type or use from __future__ import annotations in Python 3.7+ to get around the problem that the type is not importable at runtime.

Edit: All of that is already possible to do in type stubs. This decorator would be more or less a standardized no-op to signal types that are not available at runtime.

@rpgoldman
Copy link

@srittau Thanks! The use of quoting in type hints had slipped my mind! Sorry!

@gvanrossum
Copy link
Member

So it looks like the latest design here has the decorator only available in stubs. Why isn’t a comment (maybe standardized) just as good?

@gvanrossum
Copy link
Member

gvanrossum commented Apr 12, 2019

@srittau Is there still a desire to do this? Or should I just close all the PRs without merging?

@srittau
Copy link
Collaborator Author

srittau commented Apr 12, 2019

Sorry, yes, I still think this has merit. But at this point, I'm unsure on how to proceed. I see several options, but there has been resistance to each of them:

  1. Don't do anything, just use a non-standardized comment. Not desirable, since tools can't automatically recognize this.
  2. Use a standardized comment. Possible, easiest to implement, but makes tooling harder as tools can't just rely on the ast to process this. Also harder for users to recognize this as a standardized idiom. (That's basically my answer to your last question.)
  3. Use a "fake" decorator that's only available during type checking. Easier to document, easier to recognize by tools.
  4. Same as 3, but also have a runtime implementation for symmetry.

Personally, I favor 4 slightly over 3, but the discussion here seems to favor 3. But ultimately, someone has to make the decision with which option to proceed.

@gvanrossum
Copy link
Member

gvanrossum commented Apr 12, 2019 via email

@srittau
Copy link
Collaborator Author

srittau commented Apr 12, 2019

Let's do 3 for now. 4 can always be done later.

@gvanrossum
Copy link
Member

gvanrossum commented Apr 12, 2019 via email

gvanrossum pushed a commit to python/peps that referenced this issue Apr 12, 2019
gvanrossum pushed a commit to python/typeshed that referenced this issue Apr 12, 2019
@gvanrossum
Copy link
Member

The three commits implementing (1) have all been merged, so closing this now. Thanks!

ncoghlan pushed a commit to ncoghlan/peps that referenced this issue May 7, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants