Skip to content

feat(cli): allow options from CLI args or environment variables #1296

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jan 2, 2022

Conversation

nejch
Copy link
Member

@nejch nejch commented Feb 14, 2021

Closes #960
Closes #893

Makes it easier to later introduce #715 hopefully, and maybe #1195.

BREAKING-CHANGE: The gitlab CLI will now accept CLI arguments
and environment variables for its global options in addition
to configuration file options. This may change behavior for
some workflows such as running inside GitLab CI and with
certain environment variables configured.

@codecov-io
Copy link

Codecov Report

Merging #1296 (cab343c) into master (9d6c188) will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##           master    #1296   +/-   ##
=======================================
  Coverage   78.26%   78.26%           
=======================================
  Files          12       12           
  Lines        2894     2894           
=======================================
  Hits         2265     2265           
  Misses        629      629           
Flag Coverage Δ
unit 78.26% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.


Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9d6c188...cab343c. Read the comment docs.

@nejch nejch added this to the v3.0.0 milestone Apr 27, 2021
@nejch nejch mentioned this pull request Jun 14, 2021
15 tasks
@phillid
Copy link

phillid commented Jul 4, 2021

We found this PR after we tried including gitlab-cli in a GitLab CI/CD pipeline job. This PR looks like it will achieve what we wanted to do (drive the GitLab API without creating a config file in each job) 🎉

One outstanding wishlist item we had was if the tool could look for the same environment variables that are set inside a GitLab CI/CD job, i.e. CI_JOB_TOKEN, CI_SERVER_URL and so on 1. If the tool shared these varaibles, it could authenticate with the GitLab instance without any extra code written in CI/CD jobs - is that something that has already been considered?

@nejch
Copy link
Member Author

nejch commented Jul 4, 2021

We found this PR after we tried including gitlab-cli in a GitLab CI/CD pipeline job. This PR looks like it will achieve what we wanted to do (drive the GitLab API without creating a config file in each job) 🎉

One outstanding wishlist item we had was if the tool could look for the same environment variables that are set inside a GitLab CI/CD job, i.e. CI_JOB_TOKEN, CI_SERVER_URL and so on 1. If the tool shared these varaibles, it could authenticate with the GitLab instance without any extra code written in CI/CD jobs - is that something that has already been considered?

Yes, part of this was meant to implement zero-config for CI jobs. But I need to maybe tweak and add tests to really check all scenarios (for some endpoints, a PAT is needed, so we should allow overriding in a sensible precedence between cli args/env/config files/CI env). Currently the precedence planned is:

1. Explicitly provided CLI arguments,
2. Environment variables,
3. Configuration files:
a. explicitly defined config files:
i. via the `--config-file` CLI argument,
ii. via the `PYTHON_GITLAB_CFG` environment variable,
b. user-specific config file,
c. system-level config file,
4. Environment variables always present in CI (CI_SERVER_URL, CI_JOB_TOKEN).

I'll try to get back to this PR in the coming days :)

@phillid
Copy link

phillid commented Jul 6, 2021

Yes, part of this was meant to implement zero-config for CI jobs. But I need to maybe tweak and add tests to really check all scenarios (for some endpoints, a PAT is needed, so we should allow overriding in a sensible precedence between cli args/env/config files/CI env). Currently the precedence planned is:

1. Explicitly provided CLI arguments,
2. Environment variables,
3. Configuration files:
a. explicitly defined config files:
i. via the `--config-file` CLI argument,
ii. via the `PYTHON_GITLAB_CFG` environment variable,
b. user-specific config file,
c. system-level config file,
4. Environment variables always present in CI (CI_SERVER_URL, CI_JOB_TOKEN).

Aha. I'm blind 👀

That does indeed look like exactly the behaviour we wanted.

@codecov-commenter
Copy link

codecov-commenter commented Sep 12, 2021

Codecov Report

Merging #1296 (13467c5) into master (ce4bc0d) will increase coverage by 0.03%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #1296      +/-   ##
==========================================
+ Coverage   91.57%   91.61%   +0.03%     
==========================================
  Files          74       74              
  Lines        4263     4281      +18     
==========================================
+ Hits         3904     3922      +18     
  Misses        359      359              
Flag Coverage Δ
cli_func_v4 81.63% <100.00%> (+0.07%) ⬆️
py_func_v4 80.35% <26.31%> (-0.23%) ⬇️
unit 83.36% <84.21%> (+0.02%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
gitlab/cli.py 87.50% <100.00%> (+1.29%) ⬆️
gitlab/client.py 89.04% <100.00%> (+0.18%) ⬆️

@nejch nejch force-pushed the feat/merge-cli-env-file-config branch from 13467c5 to a998970 Compare December 7, 2021 23:28
@JohnVillalovos
Copy link
Member

What do you think about using argparse.ArgumentParser..add_mutually_exclusive_group() ?

For example:

private_token_group = parser.add_mutually_exclusive_group()
private_token_group.add_argument(
        "--private-token",
        help=("GitLab private token"),
        required=False,
    )
private_token_group.add_argument(
        "--private-token-env-var",
        help=("Name of environment variable containing GitLab private token"),
        required=False,
    )

But now that I look at the code I'm not sure where to do the processing to load the environment variable 😟

I'll just leave this comment anyway and maybe it gives you an idea 🤷‍♂️

@nejch
Copy link
Member Author

nejch commented Dec 10, 2021

.add_mutually_exclusive_group()

Thanks! Yes, I think it might be useful (not for arg/var exclusive I think but maybe for different types of tokens). But yes sometimes we want to validate those a bit later, because things like CI_JOB_TOKEN might always be present 🤦

I think we just need to write a bunch of tests for expected behaviors in all the possible combinations, and write the code based on that.

@nejch nejch force-pushed the feat/merge-cli-env-file-config branch 2 times, most recently from a6390bf to c60ae1f Compare December 14, 2021 00:45
@nejch nejch force-pushed the feat/merge-cli-env-file-config branch 5 times, most recently from 14487b3 to c6de2f5 Compare January 2, 2022 17:09
@nejch nejch marked this pull request as ready for review January 2, 2022 17:19
@nejch nejch requested a review from JohnVillalovos January 2, 2022 17:20
@nejch
Copy link
Member Author

nejch commented Jan 2, 2022

@JohnVillalovos took almost a year to get back to this, but ready for another look 😅

@nejch nejch force-pushed the feat/merge-cli-env-file-config branch from c6de2f5 to fe964dd Compare January 2, 2022 17:55
Copy link
Member

@JohnVillalovos JohnVillalovos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few nits. But overall looks good to me. Thanks!

BREAKING-CHANGE: The gitlab CLI will now accept CLI arguments
and environment variables for its global options in addition
to configuration file options. This may change behavior for
some workflows such as running inside GitLab CI and with
certain environment variables configured.
@nejch nejch force-pushed the feat/merge-cli-env-file-config branch from fe964dd to 1158be3 Compare January 2, 2022 21:11
@nejch nejch requested a review from JohnVillalovos January 2, 2022 21:21
Copy link
Member

@JohnVillalovos JohnVillalovos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @nejch . Looks good.

@JohnVillalovos JohnVillalovos merged commit ca58008 into main Jan 2, 2022
@JohnVillalovos JohnVillalovos deleted the feat/merge-cli-env-file-config branch January 2, 2022 22:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

cli command line arguments for url and token CLI usage without configuration file
5 participants