Skip to content

github/workflows: Add spell checking via codespell. #10911

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Apr 27, 2023

Conversation

dpgeorge
Copy link
Member

@dpgeorge dpgeorge commented Mar 3, 2023

Running codespell on this repo picks up quite a few spelling mistakes that would be good to fix.

It also has lots of false positives...

Is this worth including in the CI?

Does it belong in the code_formatting.yml workflow, or in its own workflow?

@github-actions
Copy link

github-actions bot commented Mar 3, 2023

Code size report:

   bare-arm:    +0 +0.000% 
minimal x86:    +0 +0.000% 
   unix x64:    +0 +0.000% standard
      stm32:    +0 +0.000% PYBV10
        rp2:    +0 +0.000% PICO

@codecov-commenter
Copy link

codecov-commenter commented Mar 3, 2023

Codecov Report

Merging #10911 (867e4dd) into master (07a719a) will not change coverage.
The diff coverage is n/a.

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

@@           Coverage Diff           @@
##           master   #10911   +/-   ##
=======================================
  Coverage   98.49%   98.49%           
=======================================
  Files         155      155           
  Lines       20583    20583           
=======================================
  Hits        20273    20273           
  Misses        310      310           
Impacted Files Coverage Δ
extmod/modussl_mbedtls.c 89.84% <ø> (ø)
py/asmbase.c 91.17% <ø> (ø)
py/compile.c 99.45% <ø> (ø)
py/emitbc.c 100.00% <ø> (ø)
py/emitnative.c 99.31% <ø> (ø)
py/formatfloat.c 100.00% <ø> (ø)
py/gc.c 99.18% <ø> (ø)
py/modbuiltins.c 95.56% <ø> (ø)
py/objexcept.c 96.42% <ø> (ø)
py/objtype.c 100.00% <ø> (ø)
... and 3 more

@stinos
Copy link
Contributor

stinos commented Mar 3, 2023

It also has lots of false positives...

Unless there's an easy way to make it ignore those (something like a rule making it discard all uppercase things, plus lists of things to ignore like 'ure') it seems more like an annoyance.

@dpgeorge
Copy link
Member Author

dpgeorge commented Mar 3, 2023

Unless there's an easy way to make it ignore those

Yes there are lots of options. Like --ignore-regex and the ability to skip files.

@dpgeorge dpgeorge force-pushed the ci-add-codespell branch 3 times, most recently from 458c98a to 6339b7e Compare March 3, 2023 11:41
@dpgeorge
Copy link
Member Author

dpgeorge commented Mar 3, 2023

OK, with some simple exclusions I've reduced the number of reported spelling mistakes down from 850 to 380. And most of these are spelling mistakes that should be corrected.

@stinos
Copy link
Contributor

stinos commented Mar 3, 2023

Yeah that looks good. Well, apart from the spelling errors themselves :)

@Josverl
Copy link
Contributor

Josverl commented Mar 5, 2023

I think it would help if contributors run a check with the same rules before committing.
therefore I would suggest using a config file, there are a few different formats supported, the .codespell.rc can be in the repo root, or in the tools folder if specified
That way it can be included in a pre-commit hook or into an IDE.
Also I think a few more folder exclusions make sense to make this more usable on a dev setup rather than only in CI ( .git,.*_cache,**/build* )
.codespell.rc

[codespell]
skip = ./drivers/cc3100,./ports/cc3200/hal,./lib,.git,.*_cache,**/build*
count =
quiet-level = 3
ignore-words-list = iput,ans,ure,extint 
ignore-regex = '\b[A-Z]{3}\b'

One feature that I miss in codespell is to allow an override / ignore in code
with cspell I can use # spell-checker: disable to override a suggestion, in a specific line / file without needing to add it to a dictionary.

one example is
ports\stm32\usbhost\Class\AUDIO\Src\usbh_audio.c:942: OT ==> TO, OF, OR, NOT

that is just a comment, and can easily be rewritten.
In other cases that will neet to require an addition to the wordlist.

As a last comment, as a non-native-speaker:
Please make it explicit which flavor/flavour of English should be used for the documentation. Im OK with UK , and US , but now I'm wondering if I need to update my internal AU dictionaries as well 🦘

@dpgeorge
Copy link
Member Author

dpgeorge commented Mar 8, 2023

Thanks for the feedback.

I've now make a config file tools/codespell.cfg with the relevant configuration. And the ci_code_spell_run function in tools/ci.sh can be used (eg at the CLI) to run codespell.

I've also only included certain directories at the moment in the spell check, to make it easier to pass CI. I excluded the ports/ directory. We can add that later with spelling fixed (when someone has time to do it).

I also fixed about 64 spelling mistakes in those directories that are included in the code spell CI.

As a last comment, as a non-native-speaker:
Please make it explicit which flavor/flavour of English should be used for the documentation. Im OK with UK , and US , but now I'm wondering if I need to update my internal AU dictionaries as well kangaroo

In AU we usually use UK spelling 😄

I think codespell allows both US and UK spelling, and we can leave it at that (although I prefer UK).

@jimmo
Copy link
Member

jimmo commented Mar 8, 2023

I've now make a config file tools/codespell.cfg with the relevant configuration

This probably won't help IDE auto-detection of the config, which would expect to find .codespellrc (hopefully this would be configurable but it would be good to make it work out-of-the-box which is I think what @Josverl was suggesting).

@dpgeorge
Copy link
Member Author

dpgeorge commented Mar 9, 2023

This probably won't help IDE auto-detection of the config, which would expect to find .codespellrc

Maybe we will go with pyproject.toml, per #10977.

@Josverl does pyproject.toml work for you?

@Josverl
Copy link
Contributor

Josverl commented Mar 9, 2023

That should work. I can test tomorrow. Need to find a decent add= with proper support for the tool ide platform combo

@dpgeorge
Copy link
Member Author

I've moved the config to pyproject.toml in the root directory of this repo. It's now possible to simply run codespell in the root of the repo to check the spelling (ie no arguments needed).

@dpgeorge dpgeorge force-pushed the ci-add-codespell branch 2 times, most recently from 6661815 to 60c8da5 Compare March 13, 2023 01:12
@dpgeorge dpgeorge changed the title RFC: github/workflows: Add spell checking via codespell. github/workflows: Add spell checking via codespell. Mar 13, 2023
@Josverl
Copy link
Contributor

Josverl commented Mar 19, 2023

The config in pyproject.toml is picked up by all tools I have tried with IDE integration in VSCode and Pycharm.
In most cases there are options to provide overrides in IDE settings.
tested:

  • black
  • codespell - included in Trunk
  • pylance/pyright
  • ruff

@dpgeorge
Copy link
Member Author

Great, thanks for confirming pyproject.toml works!

Signed-off-by: Damien George <damien@micropython.org>

tools/ci.sh: Explicitly specify pyproject.toml.

Signed-off-by: Damien George <damien@micropython.org>

tools/ci.sh: Import tomli.

Signed-off-by: Damien George <damien@micropython.org>
Signed-off-by: Damien George <damien@micropython.org>
So that this file doesn't need to be excluded from codespell.

Signed-off-by: Damien George <damien@micropython.org>
Signed-off-by: Damien George <damien@micropython.org>
Signed-off-by: Damien George <damien@micropython.org>
For convenience, eg for IDEs.

Signed-off-by: Damien George <damien@micropython.org>
@dpgeorge dpgeorge merged commit 867e4dd into micropython:master Apr 27, 2023
@dpgeorge dpgeorge deleted the ci-add-codespell branch April 27, 2023 09:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants