Skip to content

test(functional): start tracking functional test coverage #1440

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
May 6, 2021
Merged

test(functional): start tracking functional test coverage #1440

merged 2 commits into from
May 6, 2021

Conversation

nejch
Copy link
Member

@nejch nejch commented May 1, 2021

Trying out codecov flags to see how much the whole suite covers.

@codecov-commenter
Copy link

codecov-commenter commented May 1, 2021 •
edited
Loading

Codecov Report

Merging #1440 (dfa40c1) into master (1e6305e) will increase coverage by 9.17%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #1440      +/-   ##
==========================================
+ Coverage   80.24%   89.41%   +9.17%     
==========================================
  Files          73       73              
  Lines        4064     4064              
==========================================
+ Hits         3261     3634     +373     
+ Misses        803      430     -373
Flag Coverage Δ
cli_func_v4 78.61% <ø> (?)
py_func_v4 78.32% <ø> (?)
unit 80.24% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
gitlab/base.py 89.24% <0.00%> (+0.63%) ⬆️
gitlab/exceptions.py 99.27% <0.00%> (+1.44%) ⬆️
gitlab/types.py 96.29% <0.00%> (+3.70%) ⬆️
gitlab/v4/objects/jobs.py 68.96% <0.00%> (+5.17%) ⬆️
gitlab/v4/objects/groups.py 82.02% <0.00%> (+5.61%) ⬆️
gitlab/v4/objects/issues.py 86.76% <0.00%> (+5.88%) ⬆️
gitlab/client.py 84.97% <0.00%> (+6.01%) ⬆️
gitlab/mixins.py 83.71% <0.00%> (+6.33%) ⬆️
gitlab/v4/objects/users.py 98.23% <0.00%> (+7.64%) ⬆️
gitlab/v4/objects/deploy_keys.py 100.00% <0.00%> (+8.69%) ⬆️
... and 17 more

@nejch nejch marked this pull request as ready for review May 1, 2021 17:39
@nejch nejch requested review from JohnVillalovos and max-wittig May 1, 2021 17:39
@nejch
Copy link
Member Author

nejch commented May 1, 2021

WDYT? I wanted to get a whole overview of what code covered by tests, including functional. Using flags you can really nicely split the suites in codecov, see https://codecov.io/gh/python-gitlab/python-gitlab/src/f875786ce338b329421f772b181e7183f0fcb333/gitlab/v4/cli.py

I think in docker tests the real bottleneck is the docker container, so I thought it wouldn't hurt adding XML coverage to the runs by default. Most people run unit tests locally I assume.

I also tweaked the codecov commenter a bit as it's not very helpful getting spammed with the comment if there's no change in coverage.

JohnVillalovos
JohnVillalovos approved these changes May 1, 2021
View reviewed changes
Copy link
Member

@JohnVillalovos JohnVillalovos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. I like it.

Thanks.

@JohnVillalovos JohnVillalovos merged commit 0d3b8ae into master May 6, 2021
@nejch nejch deleted the test/functional-test-coverage branch May 6, 2021 19:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@JohnVillalovos JohnVillalovos JohnVillalovos approved these changes

@max-wittig max-wittig Awaiting requested review from max-wittig

Assignees

@JohnVillalovos JohnVillalovos

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@nejch @codecov-commenter @JohnVillalovos