Type hinting #1920

Bibo-Joshi · 2020-04-25T18:17:55Z

Preparations for type hinting:

Mypy as static type checker is added to pre-commit and make-file
Python 3.5 support is dropped, to allow typing for variables
Two make sure that new PRs include typing, I tried two approaches:
1. Use mypys report functionality together with diff-cover to compute the coverage of the changed lines only and let the test fail, if its < 100%. Unfortunately, mypy will mark any line with an "Any" as uncovered, even if that's allowed during type checking. But especially in telegram.ext we have a lot of stuff that can be "Any" (CH states, entries of user_data and so on). So, even after adding typing for the current code base, we would either
  1. have failing tests all the time or
  2. Allow the test to fail and manually look at the logs. grrr.
2. Use mypys settings to disallow function definitions with no or incomplete typing. Upside compared to the diff-cover approach: its easier to see, if we're satisfied or not. Downside: When a file is changed all functions in that file will be checked and all classes/functions that are imported or inherited from, too. We can either
  1. type the complete base before merging
  2. try to single out the mypy result from the pre-commit test and allow it to fail until the whole base is typed
  3. Just live with the fact the the first few PRs after merging will either fail the pre-commit test or will have to do a lot of type hinting
Personally, I like ii. better that i., which is why this is the current state of this pr. More precisely, I think I'd vote for ii.c)

# Conflicts: # telegram/message.py

Bibo-Joshi · 2020-05-16T11:53:41Z

Okay, I prepared for going for ii.a) :D

TBH, I didn't try to be as specific as possible. I rather tried to annotate most almost everything on a reasonably specific level and made sure that Mypy checks out. Many things can surely be improved, but IMO they can be revisited if

We're touching the code anyway
People have issues with the hints I wrote,

i.e. we can gradually improve the hints.

Some more specific notes:

Updated flake8 so it doesn't complain about # type: ignore[error-code]
Telegram classes that have class methods using the bot, would need a if self.bot is None: raise RuntimeError or something, if we're strict. I found that unnecessary and just told mypy to not check optionals strictly in those classes (see setup.cfg)
Not using Literals just yet as this is the first iteration on typing and they're natively supported only for py>=3.8. They could be useful for things like Poll.type where only specific strings are allowed
No TypedDicts yet as this is the first iteration. These could be used for to_dict, de_json. However the need is not too big, as those are internal methods
reduced code duplication of de_list on the fly
Noticed thet Message.from_user was a positional argument, which is wrong. Fixed. This needs a Release note, as this might be breaking. On the other hand, manually instantiating Messages doesn't happen too often, mostly in tests …
I put some type: ignore where I didn't think doing it 100% correct would be worth the trouble
Request.stop(): Item "AppEngineManager" of "Union[PoolManager, AppEngineManager, SOCKSProxyManager, ProxyManager]" has no attribute "clear" is that a bug?
Passports are scary. I scipped some typing there
Values returned by requests are kind of a trust thingy. I'm ignoring all return value warnings from there right now as we kind of have to expect Telegram to return what they are promising.
Handlers: callback is typed as update, context as old api is deprecated
Similar for Job callbacks
telegram.ext is a bit harder to type for several reasons:
- We're doing some hacky stuff, e.g. calling Filters.filter with Update or Message
- Handlers are a bit tricky, as they usually accept Updates but for advanced handlers, str or even object (for TypeHandler) are okay
- The docs are usually written w.r.t. to handling Updates, partly ignoring the other cases
- The internals of Updater are a black box for me. I just marked a bunch of functions os "no type check" here
That's why I've been a bit more lax on telegram.ext. Big parts of the typing usually isn't relevant for the user most times, anyway …

Comments on CI:

Codacy is complaining about unused imports, which are needed for typing.
Codecov (only gave a quick look) seems to mainly complain about the if TYPE_CHECKING not being covered. We can't do anything against that, i fear …

# Conflicts: # telegram/__main__.py # telegram/bot.py # telegram/callbackquery.py # telegram/chatmember.py # telegram/choseninlineresult.py # telegram/error.py # telegram/ext/callbackqueryhandler.py # telegram/ext/commandhandler.py # telegram/ext/conversationhandler.py # telegram/ext/dictpersistence.py # telegram/ext/filters.py # telegram/ext/inlinequeryhandler.py # telegram/ext/jobqueue.py # telegram/ext/messagequeue.py # telegram/ext/picklepersistence.py # telegram/ext/regexhandler.py # telegram/ext/stringcommandhandler.py # telegram/ext/stringregexhandler.py # telegram/ext/typehandler.py # telegram/files/animation.py # telegram/files/document.py # telegram/files/sticker.py # telegram/files/venue.py # telegram/files/video.py # telegram/files/videonote.py # telegram/files/voice.py # telegram/games/game.py # telegram/games/gamehighscore.py # telegram/inline/inlinekeyboardmarkup.py # telegram/inline/inlinequery.py # telegram/message.py # telegram/messageentity.py # telegram/passport/credentials.py # telegram/passport/encryptedpassportelement.py # telegram/passport/passportdata.py # telegram/passport/passportfile.py # telegram/payment/orderinfo.py # telegram/payment/precheckoutquery.py # telegram/payment/shippingoption.py # telegram/payment/shippingquery.py # telegram/payment/successfulpayment.py # telegram/poll.py # telegram/replykeyboardmarkup.py # telegram/update.py # telegram/user.py # telegram/userprofilephotos.py # telegram/utils/helpers.py # telegram/utils/webhookhandler.py

# Conflicts: # telegram/bot.py # telegram/botcommand.py # telegram/callbackquery.py # telegram/chat.py # telegram/chatpermissions.py # telegram/dice.py # telegram/ext/basepersistence.py # telegram/ext/conversationhandler.py # telegram/ext/dictpersistence.py # telegram/ext/dispatcher.py # telegram/ext/filters.py # telegram/ext/jobqueue.py # telegram/ext/messagequeue.py # telegram/ext/picklepersistence.py # telegram/ext/updater.py # telegram/files/animation.py # telegram/files/audio.py # telegram/files/document.py # telegram/files/photosize.py # telegram/files/sticker.py # telegram/files/video.py # telegram/files/videonote.py # telegram/files/voice.py # telegram/games/game.py # telegram/inline/inlinekeyboardbutton.py # telegram/inline/inlinekeyboardmarkup.py # telegram/inline/inputlocationmessagecontent.py # telegram/message.py # telegram/parsemode.py # telegram/passport/passportfile.py # telegram/payment/invoice.py # telegram/poll.py # telegram/replykeyboardmarkup.py # telegram/update.py # telegram/userprofilephotos.py # telegram/utils/helpers.py # telegram/utils/request.py # telegram/utils/webhookhandler.py # telegram/webhookinfo.py # tests/test_conversationhandler.py # tests/test_message.py # tests/test_updater.py

# Conflicts: # README.rst # telegram/bot.py # telegram/botcommand.py # telegram/chatmember.py # telegram/chatpermissions.py # telegram/dice.py # telegram/ext/basepersistence.py # telegram/ext/defaults.py # telegram/ext/dispatcher.py # telegram/ext/filters.py # telegram/ext/jobqueue.py # telegram/ext/updater.py # telegram/files/animation.py # telegram/files/audio.py # telegram/files/document.py # telegram/files/file.py # telegram/files/photosize.py # telegram/files/sticker.py # telegram/files/video.py # telegram/files/videonote.py # telegram/files/voice.py # telegram/games/game.py # telegram/inline/inlinekeyboardbutton.py # telegram/inline/inlinekeyboardmarkup.py # telegram/inline/inlinequery.py # telegram/message.py # telegram/passport/passportfile.py # telegram/payment/invoice.py # telegram/poll.py # telegram/replykeyboardmarkup.py # telegram/update.py # telegram/userprofilephotos.py # telegram/utils/helpers.py # telegram/utils/request.py # telegram/utils/webhookhandler.py # telegram/webhookinfo.py # tests/test_jobqueue.py # tests/test_message.py

telegram/__main__.py

telegram/base.py

JosXa · 2020-10-02T02:14:18Z

telegram/callbackquery.py

@@ -144,7 +149,8 @@ def edit_message_text(self, text, *args, **kwargs):
            return self.bot.edit_message_text(text, chat_id=self.message.chat_id,
                                              message_id=self.message.message_id, *args, **kwargs)

-    def edit_message_caption(self, caption, *args, **kwargs):
+    def edit_message_caption(self, caption: str, *args: Any,
+                             **kwargs: Any) -> Union[Message, bool]:


strange line break

JosXa · 2020-10-02T02:15:05Z

telegram/callbackquery.py

@@ -172,7 +178,8 @@ def edit_message_caption(self, caption, *args, **kwargs):
                                                 message_id=self.message.message_id,
                                                 *args, **kwargs)

-    def edit_message_reply_markup(self, reply_markup, *args, **kwargs):
+    def edit_message_reply_markup(self, reply_markup: 'InlineKeyboardMarkup', *args: Any,


Tbh I'd just add black to pre-commit.yml aswell if every single file is being changed anyway...

- repo: https://github.com/psf/black rev: 20.8b1 hooks: - id: black

Advantage: Never ever any discussion around formatting again.
Disadvantages: None. Except you don't like PEP8 and prefer your own code style to keep your job with this Arbeitsbeschaffungsmaßnahme. ;D

I think code formatting is worth its own discussion and PR

telegram/chat.py

Poolitzer

Monumental PR! I eagerly look forward to have static typing supported in our library. Just a couple of small discussion points, most of them repeat themself. Then I am all good with it. Thank you so much for putting the time into it.

telegram/bot.py

telegram/chat.py

telegram/chatmember.py

telegram/dice.py

telegram/ext/filters.py

telegram/files/inputmedia.py

telegram/messageentity.py

telegram/poll.py

Poolitzer

Looks good to me now! If there actually are errors, we can fix them in a quick release after 13. I would merge.

Bibo-Joshi · 2020-10-06T16:04:41Z

Codecov is still failing, but that's mostly due to this PR

annotating code that wasn't tested before
adding some if-clauses that are just there to make mypy happy

So we can just up the coverage the time we change some code … Merging :)

(Codacy fails, because the v13 branch is not tracked …)

# Conflicts: # telegram/error.py # telegram/ext/callbackcontext.py # telegram/ext/callbackqueryhandler.py # telegram/ext/commandhandler.py # telegram/ext/conversationhandler.py # telegram/ext/dispatcher.py # telegram/ext/handler.py # telegram/ext/inlinequeryhandler.py # telegram/ext/messagehandler.py # telegram/ext/regexhandler.py # telegram/ext/stringcommandhandler.py # telegram/ext/stringregexhandler.py # telegram/ext/typehandler.py # telegram/utils/promise.py

Bibo-Joshi added enhancement ⚙️ documentation affected functionality: documentation labels Apr 25, 2020

Bibo-Joshi requested a review from Poolitzer April 25, 2020 18:17

Bibo-Joshi changed the title ~~Type hinting master~~ Type hinting Apr 25, 2020

Bibo-Joshi mentioned this pull request Apr 25, 2020

static typing workflow test #1786

Closed

Poolitzer and others added 9 commits May 12, 2020 08:37

adding mypy to the repo

79a76f6

first take on only testing changed files

22a45a0

allowing test branch

f61fecf

improving start condition of workflow

1e94485

Add mypy to pre-commit and make

6a51275

Try computing typing coverage in CI

a89828a

Save coverage report as artifact

656c2bd

Drop py3.5, use mypy options instead of diff-cover

e5f5867

use spaces over tabs in makefile

c856bbb

Bibo-Joshi force-pushed the type_hinting_master branch from b79ada1 to c856bbb Compare May 12, 2020 06:38

Bibo-Joshi added 11 commits May 12, 2020 08:59

Remove deprecated getargspec

783e5d4

Refactor de_json

e13bb33

Annotate all the things! (Except tg.ext)

9160674

Merge branch 'master' into type_hinting_master

d84444b

# Conflicts: # telegram/message.py

Annotate InputMedia, Custom Aliases, improve file typing

7f4410d

Move de_list to TelegramObject

f87fa10

Persistence & Handlers

4b9c010

Filters

1a22799

JobQueue & Job

9a545be

Dispatcher

be15033

MessageQueue

db78e06

Some fixes

b74d0db

Bibo-Joshi added this to the 13.0 milestone Jun 12, 2020

Make context-based callbacks the default setting (#2050)

85c6831

Bibo-Joshi force-pushed the v13 branch from 3993250 to 85c6831 Compare September 13, 2020 09:57

Bibo-Joshi force-pushed the v13 branch 2 times, most recently from 3247477 to b7293a4 Compare September 27, 2020 15:32

JosXa reviewed Oct 2, 2020

View reviewed changes

telegram/__main__.py Show resolved Hide resolved

JosXa reviewed Oct 2, 2020

View reviewed changes

telegram/base.py Outdated Show resolved Hide resolved

JosXa reviewed Oct 2, 2020

View reviewed changes

telegram/chat.py Show resolved Hide resolved

address review

b72ac15

JosXa approved these changes Oct 4, 2020

View reviewed changes

Poolitzer requested changes Oct 4, 2020

View reviewed changes

Bibo-Joshi mentioned this pull request Oct 4, 2020

Account for DST thingies in JobQueue tests #2113

Merged

Bibo-Joshi added 2 commits October 4, 2020 21:07

Merge branch 'v13' into type_hinting_master

0ad6649

Use NoReturn where possible

96b929f

Poolitzer approved these changes Oct 6, 2020

View reviewed changes

Bibo-Joshi added 2 commits October 6, 2020 17:40

Try ignoring the if TYPE_CHECKING lines in coverage report

4398205

minor fix

6edc522

Bibo-Joshi merged commit d2e67d1 into v13 Oct 6, 2020

Bibo-Joshi deleted the type_hinting_master branch October 6, 2020 16:05

Bibo-Joshi mentioned this pull request Oct 6, 2020

Type-hinting for IDEs #1373

Closed

Bibo-Joshi added a commit that referenced this pull request Oct 7, 2020

Type Hinting (#1920)

ee1f347

github-actions bot locked and limited conversation to collaborators Oct 7, 2020

Bibo-Joshi added 🔌 enhancement pr description: enhancement and removed enhancement labels Nov 3, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Type hinting #1920

Type hinting #1920

Uh oh!

Bibo-Joshi commented Apr 25, 2020 •

edited

Loading

Uh oh!

Bibo-Joshi commented May 16, 2020 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

JosXa Oct 2, 2020

Uh oh!

JosXa Oct 2, 2020 •

edited

Loading

Uh oh!

Bibo-Joshi Oct 2, 2020

Uh oh!

Uh oh!

Poolitzer left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Poolitzer left a comment

Uh oh!

Bibo-Joshi commented Oct 6, 2020

Uh oh!

Uh oh!

Type hinting #1920

Type hinting #1920

Uh oh!

Conversation

Bibo-Joshi commented Apr 25, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Bibo-Joshi commented May 16, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

JosXa Oct 2, 2020

Choose a reason for hiding this comment

Uh oh!

JosXa Oct 2, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Bibo-Joshi Oct 2, 2020

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Poolitzer left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Poolitzer left a comment

Choose a reason for hiding this comment

Uh oh!

Bibo-Joshi commented Oct 6, 2020

Uh oh!

Uh oh!

Bibo-Joshi commented Apr 25, 2020 •

edited

Loading

Bibo-Joshi commented May 16, 2020 •

edited

Loading

JosXa Oct 2, 2020 •

edited

Loading