Skip to content

Make asyncio.Task covariant #8781

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Oct 3, 2022
Merged

Make asyncio.Task covariant #8781

merged 7 commits into from
Oct 3, 2022

Conversation

Dreamsorcerer
Copy link
Contributor

@Dreamsorcerer Dreamsorcerer commented Sep 21, 2022

@Dreamsorcerer Dreamsorcerer marked this pull request as ready for review September 21, 2022 18:18
@hauntsaninja
Copy link
Collaborator

See my comment on the mypy issue, I think making Future covariant is unsound

@Dreamsorcerer
Copy link
Contributor Author

Will it still work by making Task covariant and Future not?

@JelleZijlstra
Copy link
Member

Thanks @hauntsaninja, I missed that method. We can't make Task covariant either, because Task.set_result would still be unsafe.

@hauntsaninja
Copy link
Collaborator

Task.set_result always raises, so I think you could get away with that.

@Dreamsorcerer
Copy link
Contributor Author

Indeed, docs says:

Task inherits from Future all of its APIs except Future.set_result() and Future.set_exception().

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@JelleZijlstra
Copy link
Member

I'm still hesitant about having a covariant subclass of an invariant base class. Given B <: A, we'd have Task[A] <: Future[A], Task[B] <: Future[B], and Task[B] <: Task[A], but not Future[B] <: Future[A]. That feels unsound.

@erictraut curious if you have any opinions here: maybe type checkers should not allow covariant subclasses of invariant base classes? And how would this work under PEP 695?

@hauntsaninja
Copy link
Collaborator

hauntsaninja commented Sep 22, 2022

It definitely feels sketchy, but in this particular case as far as I can tell it's not any more unsound than the existing Liskov issue with set_result

@erictraut
Copy link
Contributor

erictraut commented Sep 22, 2022

I agree this is sketchy. Neither mypy nor pyright (I haven't tried the other type checkers) complain about this situation currently, even though it's clearly wrong.

from typing import TypeVar

T = TypeVar("T", covariant=True)

class MyList(list[T]): pass

def append_float(y: MyList[float]):
    y.append(1.0)

x: MyList[int] = MyList([1, 2, 3])
append_float(x)

This is a pretty big hole in type safety, and it should probably be plugged. I'll explore a solution in pyright and see whether it produces a lot of noise for internal Microsoft code bases that use pyright.

In the meantime, I recommend not taking a dependency on this sketchy behavior — especially for classes that are as commonly-used as Task and Future.

@erictraut
Copy link
Contributor

I've implemented support for this missing check in pyright. I didn't see any violations of it in our internal Microsoft source bases, nor were there any existing violations in typeshed.

I'm not sure what I'd recommend for this PR. I understand the reasoning, but I think I'd reject it because it's unsound from a type perspective, and it will no longer pass type checking when using pyright. If you decide to keep it, you can use a # type: ignore or # pyright: ignore comment to suppress the error.

@hauntsaninja
Copy link
Collaborator

hauntsaninja commented Sep 23, 2022

Thanks for checking that!

I'm still in favour of this PR because it solves a real problem, models the runtime better, and there isn't opportunity for unsoundness beyond the pre-existing Liskov violation. Seems worth adding a test case to confirm behaviour, though.

But it is icky and we've managed so far without it, so if we were to reject that would be understandable.

Edit: I added the check to mypy in python/mypy#13714

@Akuli
Copy link
Collaborator

Akuli commented Sep 23, 2022

I don't have a strong opinion on this, but I agree that this can be useful. In one project, I had many collections of tasks that I wanted to cancel when a specific thing happened, so I made it something like list[asyncio.Task[Any]]. But because I never wanted to access the results of any of these tasks, and in fact I would want an error if I assume the results are of any specific type, asyncio.Task[object] would have been better if it worked.

Copy link
Member

@AlexWaygood AlexWaygood left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that doing this is usually unsound, and I'm very happy that we're fixing mypy and pyright to properly detect this unsoundness in the future. But I also agree that this case is weird enough that the usual rules don't really apply here, and that this models the runtime better. I'm happy for this to go in; we can type: ignore it when we upgrade to versions of mypy/pyright that detect this kind of unsoundness.

@github-actions

This comment has been minimized.

1 similar comment
@github-actions
Copy link
Contributor

According to mypy_primer, this change has no effect on the checked open source code. 🤖🎉

@hauntsaninja
Copy link
Collaborator

hauntsaninja commented Oct 3, 2022

This has been open for a bit and while it is a sketchy change, no one seems particularly opposed to it.

So I'm planning on merging. tldr:

  • solves a real problem encountered by multiple people
  • models the runtime better
  • there isn't opportunity for unsoundness beyond the pre-existing Liskov violation
  • two maintainers in favour, no maintainers clearly opposed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants