Skip to content

chore: improve type-hinting for managers #1512

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Sep 8, 2021
Merged

chore: improve type-hinting for managers #1512

merged 3 commits into from
Sep 8, 2021

Conversation

JohnVillalovos
Copy link
Member

@JohnVillalovos JohnVillalovos commented Jun 12, 2021

The 'managers' are dynamically created. This unfortunately means that
we don't have any type-hints for them and so editors which understand
type-hints won't know that they are valid attributes.

  • Add the type-hints for the managers we define.
  • Add a unit test that makes sure that the type-hints and the
    '_managers' attribute are kept in sync with each other.
  • Add unit test that makes sure specified managers in '_managers'
    have a name ending in 'Managers' to keep with current convention.
  • Make RESTObject._managers always present with a default value of
    None.

@codecov-commenter
Copy link

codecov-commenter commented Jun 12, 2021

Codecov Report

Merging #1512 (c8611e0) into master (330d69c) will increase coverage by 0.22%.
The diff coverage is 98.66%.

@@            Coverage Diff             @@
##           master    #1512      +/-   ##
==========================================
+ Coverage   91.15%   91.37%   +0.22%     
==========================================
  Files          74       74              
  Lines        4172     4291     +119     
==========================================
+ Hits         3803     3921     +118     
- Misses        369      370       +1     
Flag Coverage Δ
cli_func_v4 81.28% <98.66%> (+0.50%) ⬆️
py_func_v4 80.56% <98.66%> (+0.48%) ⬆️
unit 82.80% <98.66%> (+0.46%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
gitlab/v4/cli.py 81.20% <0.00%> (-0.31%) ⬇️
gitlab/base.py 89.69% <100.00%> (+0.25%) ⬆️
gitlab/v4/objects/boards.py 100.00% <100.00%> (ø)
gitlab/v4/objects/commits.py 94.36% <100.00%> (+0.16%) ⬆️
gitlab/v4/objects/container_registry.py 83.33% <100.00%> (ø)
gitlab/v4/objects/deployments.py 100.00% <100.00%> (ø)
gitlab/v4/objects/discussions.py 100.00% <100.00%> (ø)
gitlab/v4/objects/epics.py 74.35% <100.00%> (+0.67%) ⬆️
gitlab/v4/objects/groups.py 87.20% <100.00%> (+3.36%) ⬆️
gitlab/v4/objects/issues.py 88.00% <100.00%> (+1.04%) ⬆️
... and 10 more

@JohnVillalovos JohnVillalovos requested a review from nejch June 12, 2021 22:11
@JohnVillalovos
Copy link
Member Author

So in my quick check I could just use the annotations to create the managers. As all the annotations which are of type *Manager are also in the _managers attribute.

But I think to be safe would probably be better to do this as a first step and then later switch to using annotations only.

Any thoughts?

@JohnVillalovos JohnVillalovos self-assigned this Jun 13, 2021
Copy link
Member

@nejch nejch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would personally prefer if we could define things in one place and avoid adding too much duplication, and we would also not need to create custom tests to enforce it (and just test _create_managers instead).

Using __annotations__ for this would be convenient although a little magic. But the syntax is so clean it might be less cryptic to new contributors who don't care how managers are created. It could be combined with a simple check that it's a real manager, similar to what we already do for the CLI https://github.com/python-gitlab/python-gitlab/blob/master/gitlab/cli.py#L92-L96

# namespace = gitlab.v4.objects
managers = [man for man in namespace.__dict__ if man.endswith("Manager")]
if manager not in managers:
  continue
...

(something prettier than this I guess, but you get the idea :) )

When I look at this in your PR:

class ProjectDeployment(SaveMixin, RESTObject):
    mergerequests: ProjectDeploymentMergeRequestManager
    _managers = (("mergerequests", "ProjectDeploymentMergeRequestManager"),)

The new syntax is just so nice and reminds me of dataclasses, and the old one starts to look ugly and I'd just remove it 😸 On the other hand I don't want to force something that's too magic, @max-wittig what's your opinion here?

@nejch nejch requested a review from max-wittig June 13, 2021 18:48
@JohnVillalovos
Copy link
Member Author

I would personally prefer if we could define things in one place and avoid adding too much duplication, and we would also not need to create custom tests to enforce it (and just test _create_managers instead).

Agreed.

I am going to update the PR to just use annotations. We can then decide which one we like better.

I do like the annotation only one.

Copy link
Member

@nejch nejch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would personally prefer if we could define things in one place and avoid adding too much duplication, and we would also not need to create custom tests to enforce it (and just test _create_managers instead).

Agreed.

I am going to update the PR to just use annotations. We can then decide which one we like better.

I do like the annotation only one.

I kind of really like this as well, despite the magic so I'll approve it (just needs a rebase and I just have a style question) 😀 When you think about it, the dynamic manager creation has always been a bit magic and considering the stuff we already do in:

mgr_cls_name = cls.__name__ + "Manager"
mgr_cls = getattr(gitlab.v4.objects, mgr_cls_name)

for cls in gitlab.v4.objects.__dict__.values():
if not isinstance(cls, type):
continue
if issubclass(cls, gitlab.base.RESTManager):
if cls._obj_cls is not None:
classes.append(cls._obj_cls)
classes.sort(key=operator.attrgetter("__name__"))

] = getattr(gitlab.v4.objects, self.cls.__name__ + "Manager")

... this is not really outrageous compared to that :) But would like also @max-wittig to take a quick look.

@JohnVillalovos JohnVillalovos requested a review from nejch June 29, 2021 18:09
The 'managers' are dynamically created. This unfortunately means that
we don't have any type-hints for them and so editors which understand
type-hints won't know that they are valid attributes.

 * Add the type-hints for the managers we define.
 * Add a unit test that makes sure that the type-hints and the
   '_managers' attribute are kept in sync with each other.
 * Add unit test that makes sure specified managers in '_managers'
   have a name ending in 'Managers' to keep with current convention.
 * Make RESTObject._managers always present with a default value of
   None.
 * Fix a type-issue revealed now that mypy knows what the type is
@JohnVillalovos
Copy link
Member Author

@nejch Is there anything else that needs to be done for this to get merged?

Convert our manager usage to be done via type annotations.

Now to define a manager to be used in a RESTObject subclass can simply
do:
      class ExampleClass(CRUDMixin, RESTObject):
          my_manager: MyManager

Any type-annotation that annotates it to be of type *Manager (with the
exception of RESTManager) will cause the manager to be created on the
object.
@JohnVillalovos JohnVillalovos marked this pull request as draft September 8, 2021 15:24
Add an additional check to attempt to solve the flakiness of the
test_merge_request_should_remove_source_branch() test.
@nejch
Copy link
Member

nejch commented Sep 8, 2021

@nejch Is there anything else that needs to be done for this to get merged?

I think it'll also be ready @JohnVillalovos just ping me when you're finished with the current rebase

@JohnVillalovos JohnVillalovos marked this pull request as ready for review September 8, 2021 16:29
@JohnVillalovos
Copy link
Member Author

I think it'll also be ready @JohnVillalovos just ping me when you're finished with the current rebase

@nejch It is ready for review. I added a fix for flakiness in one of the functional tests.

@nejch nejch merged commit 5247e8b into python-gitlab:master Sep 8, 2021
@JohnVillalovos JohnVillalovos deleted the jlvillal/type_managers branch January 4, 2022 06:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants