Skip to content

Specify bash shell in CI Workflows #605

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 21 commits into from
Oct 28, 2022

Conversation

adam-grant-hendry
Copy link
Contributor

@adam-grant-hendry adam-grant-hendry commented Oct 17, 2022

  • Specify shell as bash in Run tests and linters step.
  • Fix failing tests on Windows

Failing Windows Tests (generated with pytest-html plugin):
pytest_report.zip

Fixes: Issue #602, #604

@adam-grant-hendry
Copy link
Contributor Author

@Lee-W I've noticed the poetry error:

Package traitlets (5.2.2.post0) not found

in several workflows and opened the following issues to try to resolve it:

@adam-grant-hendry
Copy link
Contributor Author

adam-grant-hendry commented Oct 17, 2022

@Lee-W Until the issue is resolved, can we add traitlets >= 5.4.0 as a dependency to commitizen in the pyproject.toml? This requires dropping support for Python 3.6 since traitlets requires Python >= 3.7. However, Python 3.6 reached its end of life on 2021-12-23, so it's probably time to drop it (see also PEP 602). We' likely want to add a deprecation notice, so it might warrant additional discussion.

@codecov
Copy link

codecov bot commented Oct 19, 2022

Codecov Report

Base: 98.40% // Head: 97.90% // Decreases project coverage by -0.50% ⚠️

Coverage data is based on head (a079fe4) compared to base (2ba0b73).
Patch coverage: 100.00% of modified lines in pull request are covered.

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #605      +/-   ##
==========================================
- Coverage   98.40%   97.90%   -0.51%     
==========================================
  Files          39       35       -4     
  Lines        1634     1243     -391     
==========================================
- Hits         1608     1217     -391     
  Misses         26       26              
Flag Coverage Δ
unittests 97.90% <100.00%> (-0.51%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
commitizen/__version__.py 100.00% <100.00%> (ø)
commitizen/commands/bump.py 97.18% <100.00%> (+0.14%) ⬆️
commitizen/commands/check.py 100.00% <100.00%> (ø)
commitizen/bump.py
commitizen/changelog.py
commitizen/exceptions.py
commitizen/__init__.py

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@adam-grant-hendry
Copy link
Contributor Author

@Lee-W The traitlets issue has been resolved (the author pulled 5.2.2 as he uploaded to PyPI with faulty metadata.

@Lee-W
Copy link
Member

Lee-W commented Oct 20, 2022

180f4f2 seems to be weird 🤔 Could you please rebase from the main branch and see whether we could remove this commit?

@adam-grant-hendry
Copy link
Contributor Author

180f4f2 seems to be weird 🤔 Could you please rebase from the main branch and see whether we could remove this commit?

Yes, my apologies. I was experimenting between GPG4win and that which ships with Git for Windows. I resigned my previous commits and accidentally hit this commit. I did a git pull --rebase to fix this.

@adam-grant-hendry
Copy link
Contributor Author

180f4f2 seems to be weird 🤔 Could you please rebase from the main branch and see whether we could remove this commit?

Yes, my apologies. I was experimenting between GPG4win and that which ships with Git for Windows. I resigned my previous commits and accidentally hit this commit. I did a git pull --rebase to fix this.

Hmmm...rebasing didn't seem to do the trick, so I reverted that commit just to be safe.

Specify `shell` as `bash` in `Run tests and linters` step.

Fixes: Issue commitizen-tools#604
Switch `#!/bin/sh` to `#!/usr/bin/env sh` so scripts work on Windows as well as Linux and MacOS.
- Change shebang from `#!/bin/sh` to `#!/usr/bin/env sh`
- Set `$PREFIX` depending on OS
This should be used when parsing binary files, not text streams. The
regular expression fails on Windows because the stream contains `\n`,
but `os.linesep` is `\r\n`.

For reference, see https://stackoverflow.com/questions/38074811/what-is-os-linesep-for
Use `os.sep` for file paths. On Windows, this is `\`. On Linux and
MacOS, this is `/`.
Windows uses `\` for path separation, causing python to see the `\U` in
`C:\Users` as a Unicode string. Replace `\\` with `/` in file path
strings.

Also, the colon in drive paths on Windows causes the `partition` logic
in `bump.py` and `commands/bump.py` to fail. Perform the logic on the
path without the drive portion.
`argcomplete` does not support Git Bash on Windows out of the box. For
details, see https://kislyuk.github.io/argcomplete/#git-bash-support.
On Windows, `Popen` with `shell=True` reads the `COMSPEC` environment
variable to determine the shell to run, which defaults to Command
Prompt (see
https://docs.python.org/3/library/subprocess.html#subprocess.run).
For backwards-compatibility for older OS utility scripts, `COMSPEC`
should not be changed on Windows.

Command Prompt requires double-quotes instead of single-quotes. See

  - https://stackoverflow.com/questions/25534995/escape-double-quotes-in-git-config-from-cmd)
  - https://docs.python.org/3/library/subprocess.html#converting-an-argument-sequence-to-a-string-on-windows

Note that although the commitizen shell `scripts` shebangs have
been changed to a portable variety (i.e. `#!/usr/bin/env sh`),
which runs Git Bash on Windows when Git is installed (i.e.
the `sh.exe` executable that ships with Git for Windows),
`subprocess.Popen()` opens a child process from the active
running shell (i.e. via `CreateProcess()`). Hence, it will
still read `COMSPEC` and run a Command Prompt shell.
The Windows path separator is the backslash (i.e. `\\`). This causes
problems when python reads files from the `Users` folder since `\\U` is
the prefix for a Unicode string in python.

Since `os.path` can properly parse forward slashes on all OS's, we
substitute `\\` for `/` wherever `os.path` methods return backslashes.
For some reason, `pytest-mock` does not affect `os.isatty` when
`sys.stdin` is mocked to use `StringIO` (i.e. a file-like object).
Checking `sys.stdin.isatty()` works and is equivalent to checking
`os.isatty(0)` (`stdin` is file descriptor `0`).
For `pydocstyle`, use `--match-dir` switch to prevent traversal into
`venv/` folder.

For `commitizen`, use `cz` instead of `commitizen` as `cz` is present on
all OS's, but `commitizen` is not present in `venv/Scripts` on Windows.
Add error message if `test` script encounters unsupported OS.
`py -m` was added to command in `test_argcomplete_activation` during
experimentation and was forgotten to be removed.
@adam-grant-hendry
Copy link
Contributor Author

180f4f2 seems to be weird 🤔 Could you please rebase from the main branch and see whether we could remove this commit?

Yes, my apologies. I was experimenting between GPG4win and that which ships with Git for Windows. I resigned my previous commits and accidentally hit this commit. I did a git pull --rebase to fix this.

Hmmm...rebasing didn't seem to do the trick, so I reverted that commit just to be safe.

🤦‍♂️ Doh! I didn't actually rebase properly; my mistake. This should NOW be fixed.

Commit `d936262e` switched the `test` script to use `cz` instead of
`commitizen`, but forgot the prefix to the command is
`poetry run python -m`, which looks for a python module.

`cz` is not a module: it is an entry point script (specified in the
`pyproject.toml` table `[tool.poetry.scripts]`).

Switching `$PREFIX` to `venv/bin/` creates two problems:

  - The subdirectory `venv` creates differs depending on OS. On
    Linux/MacOS, the subdirectory containing copies/symlinks to
	Python binaries is `bin`, but on Windows it is `Scripts`.
	Similiarly, the `site-packages` directory on Linux/MacOS is
    `lib/pythonX.Y/site-packages`, but on Windows it is
	`Lib\site-packages`.
	See: https://docs.python.org/3/library/venv.html#module-venv

  - Since `commitizen` isn't the entry-point script, but rather
    `cz`, the command `venv/bin/commitizen` will fail on
	Linux/MacOS and Windows.

Poetry will check to see if it is currently running inside a
virtual environment, and if it is, it will use it directly
without creating a new one. If it isn't, it will use one that
it has already created or create a brand new one for you.

See: https://python-poetry.org/docs/managing-environments/

A better solution than switching the prefix would be to activate
the virtual environment. However, users can create virtual
environments with any name they choose (`venv/`, `.venv/`, or
otherwise). In addition, they can specify whether or not `poetry`
should create a virtual environment in the project folder
`virtualenvs.in-project`), or at all (`virtualenvs.create`).

Since switching to `venv/bin` only affects developers running
scripts locally (the GitHub Actions CI scripts run `poetry install`
and let it manage the virtual environment), it's best to
remove changing the `$PREFIX` and instead add instructions to
`README`/`CONTRIBUTING`/etc. to tell users to configure `poetry`
and activate their virtual environment appropriately if running
these scripts locally.
Command Prompt on Windows requires double-quotes. Otherwise, the
command

`${PREFIX}pydocstyle --convention=google --add-ignore=D1,D415 --match-dir="^(?![.]|venv).*"`

will fail with

'venv).*' is not recognized as an internal or external command

this will not affect the command in other Linux-based/bash shells.
Last CI run ended with a random `poetry` exception on Windows, which
happens from time-to-time.
Pipe appears broken in certain instances within the version of MinGW64
that ships as Git Bash in Git for Windows.

See:
  - https://sourceforge.net/p/mingw/bugs/2303/
  - msys2/MINGW-packages#13662
@adam-grant-hendry
Copy link
Contributor Author

@Lee-W All tests are now passing. Please kindly re-review. Code coverage has dropped 0.5%; are you alright with that?

@adam-grant-hendry
Copy link
Contributor Author

adam-grant-hendry commented Oct 21, 2022

@woile Pinging you as well, in case you want to be another reviewer.

@adam-grant-hendry adam-grant-hendry requested review from woile and Lee-W and removed request for Lee-W and woile October 22, 2022 15:43
@Lee-W
Copy link
Member

Lee-W commented Oct 23, 2022

Code coverage has dropped 0.5%; are you alright with that?
It's ok 👍

Copy link
Member

@Lee-W Lee-W left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I took a brief look and i'm good with these changes. @woile Could you please take a look when you have time? Thanks

@Lee-W
Copy link
Member

Lee-W commented Oct 27, 2022

@woile I plan to merge this one late this week. Please let me know if you want to take a deeper look :)

@Lee-W Lee-W mentioned this pull request Oct 27, 2022
4 tasks
@woile
Copy link
Member

woile commented Oct 27, 2022

LGTM, thanks for the PR

@Lee-W Lee-W merged commit 423c4d6 into commitizen-tools:master Oct 28, 2022
@adam-grant-hendry adam-grant-hendry deleted the fix/ci branch December 3, 2022 02:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants