Skip to content

global: Prune trailing whitespace. #13777

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

Gadgetoid
Copy link
Contributor

@Gadgetoid Gadgetoid commented Feb 29, 2024

Prune trailing whitespace across the whole project, this is done automatically with:

grep -IUrl --color "[[:blank:]]$" --exclude-dir=.git --exclude=*.exp | xargs sed -i 's/[[:space:]]*$//'

Raised as a draft because this touches 113 files and may break things so I do not expect it to be merged as is, but I think we should be cognisant of these... issues. I've avoided .exp files, but much of the remaining diffs are still near inscrutable.

Though GitHub will filter out whitespace from diffs if you ask nicely- https://github.com/micropython/micropython/pull/13777/files?diff=unified&w=1

If there's any interest in this kind of tidyup and this PR isn’t suitable for merging, I would recommend individual authors using some variation of the above command or editor config to clean up files as they are otherwise modified.

Copy link

codecov bot commented Feb 29, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.36%. Comparing base (90e5178) to head (8f801f3).

Additional details and impacted files
@@           Coverage Diff           @@
##           master   #13777   +/-   ##
=======================================
  Coverage   98.36%   98.36%           
=======================================
  Files         161      161           
  Lines       21084    21084           
=======================================
  Hits        20739    20739           
  Misses        345      345           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link

Code size report:

   bare-arm:    +0 +0.000% 
minimal x86:    +0 +0.000% 
   unix x64:    +0 +0.000% standard
      stm32:    +0 +0.000% PYBV10
     mimxrt:    +0 +0.000% TEENSY40
        rp2:    +0 +0.000% RPI_PICO
       samd:    +0 +0.000% ADAFRUIT_ITSYBITSY_M4_EXPRESS

@dpgeorge
Copy link
Member

Thanks, this is a reasonable thing to do. I always try to remove any trailing space when I review/merge PRs.

But, I suggest to leave the third-party code alone, ie everything in lib/ and drivers/cc3100/.

@Gadgetoid Gadgetoid force-pushed the patch-trailing-whitespace-removal branch from 9df5e33 to 827c321 Compare February 29, 2024 07:52
@Gadgetoid
Copy link
Contributor Author

Done. Updated the commit message to reflect that these directories are avoided intentionally.

@robert-hh
Copy link
Contributor

I always try to remove any trailing space when I review/merge PRs.

The CI yells when there are trailing spaces in the code.
But on the related topic: @projectgus prepares another global change PR by omitting the STATIC macro. A change like that and as well this PR is a HUGE effort for everyone maintaining PRs and trying to keep it mergeable and recent. When rebasing a PR, GIT will have many merge conflicts. For whitespace these could of course be ignored. So please, if both PRs should be merged, do that immediately one after the other, such that we have the effort of updating only once.

@Gadgetoid
Copy link
Contributor Author

A change like that and as well this PR is a HUGE effort for everyone maintaining PRs and trying to keep it mergeable and recent.

That's why this is still a draft, rather than something I would push to be merged. It might be less troublesome to let this cleanup happen naturally, but still useful to surface where trailing whitespace occurs. (I arrived at this PR entirely out of curiosity after experimenting with linting some other codebases.)

The CI yells when there are trailing spaces in the code.

I guess only if it's new code or on a line that has otherwise been changed?

@Gadgetoid Gadgetoid force-pushed the patch-trailing-whitespace-removal branch from 827c321 to df00fe3 Compare February 29, 2024 12:32
@Gadgetoid Gadgetoid changed the title all: Prune trailing whitespace. global: Prune trailing whitespace. Feb 29, 2024
@Gadgetoid
Copy link
Contributor Author

As per my sleuthing on #13763, here are a list of open PRs which I believe add trailing whitespace:

User PR Title
NoosaHydro #13455 tools/pyboard.py: Fix RTS esp32 lockups.
nxp-yilin #13429 ports: mcx: Initial port for MCXN947 series MCU.
cwalther #13367 nrf/modules/machine: Implement deepsleep()
ricksorensen #13276 esp32/modesp32.c: Add mcu_temperature() function for C3/S2/S3 devices.
gitcnd #13253 ports/esp32/boards: Add support for popular ESP32_CAM boards.
andrewleech #13011 examples/usercmodule: Add finaliser to cexample.
iabdalkader #12961 OpenAMP support.
robert-hh #12937 mimxrt: Provide WiFi and BLE support for MIMXRT boards.
agatti #12853 ports: Add RISC-V 32 bit QEMU port.
robert-hh #12347 mimxrt: Add Quadrature Encoder and Pulse Counter classes.
mhidro #12281 ports/rp2/boards: Added support for YD-RP2040 by VCC-GND studio 4 8 16 MB flash which is a pin compatible PICO clone.
Josverl #12149 CODECONVENTIONS.md: Improve docs for pre-commit setup on Windows.
dmazzella #11679 py/objint.c: Add support for int.bit_length().
pschindler #11398 NUCLEO-H7A3ZI-Q
jaenrig-ifx #11365 New port for Infineon PSoC6 microcontrollers
JackAtKitronik #11301 port/rp2: Added WebUSB to RP2040.
keredson #11183 add set_stream to DecompIO
RetiredWizard #10400 ports/nrf: Add RTC support to NRF ports.
wolfpaulus #10324 ports/esp32/boards/PYBOX: Add support SparkFun Thing Plus board.
RetiredWizard #10314 rp2/boards: Add Waveshare rp2040 plus
robert-hh #9624 samd/adc_dac: Implememt adc.read_timed() and dac.write_timed().
H-Grobben #9367 SMT32G4: Add USB, QPSI and AEMICS Board PYglet
Molorius #9356 shared/tinyusb: (just rp2 for now) change USB device settings and add HID
andrewleech #9309 stm32/btstack: Add support for btstack on WB55 / rfcore.
lowfatcode #9221 docs/library/time.rst: fix description of return results in time.mktime()
mdaeron #9140 tools/mpremote: Fix auto-connection bug in macOS and add config option to filter ports eligible for auto-connection.
r4d10n #8637 ports/wch: Add support for WCH CH32V307.
mattytrentini #8493 esp32/boards: Add INKPLATE6 board.
jbbjarnason #8480 pep 570 positional only parameters.
vikmal-sw #8435 tools/pyboard.py: ProcessToSerial cross-platform fix
andrewleech #8318 nrf/bluetooth: Add support for nimble based ubluetooth.
harbaum #8253 Proposal: Esp32 machine.PCNT through Encoder.py and Counter.py
Romaric-RILLET #8236 mimxrt: Allow to choose the pin to use for miso and cs
andrewleech #8231 tools/mpremote: Add manifest function.
alphaFred #8229 mimxrt/mboot: Adds bootloader support.
andrewleech #8152 extmod/re: Use buffer protocol for data to search through.
andrewleech #8027 extmod/modbluetooth: Add gap_indicate_service_changed.
MarceauFillon #7877 extmod/nimble: Support database hash storing
lorcap #7859 Add new module docs/library/datetime
andrewleech #7845 extmod/modbluetooth: Add gap_unpair command.
MrJake222 #7784 ports/esp8266: Software Serial (SoftUART) support.
caco3 #7725 Add support to read/write a single block in the RTC Memory
marcidy #7526 Introduce Non-blocking wifi scan for ESP32
karfas #7133 esp32/machine_rtc: ESP32 machine.RTC().usermem() returning a bytearray.
ekondayan #7048 esp32/ota: Implement ESP-IDF OTA functionality.
steven-michalske #6963 stm32/can: Add support for a software based CAN message buffer.
sanyi #6894 Add RPI PIO example for quadrature encoder.
IhorNehrutsa #6639 ESP32: New hardware PCNT Counter() and Encoder().
nickovs #6389 Support for AES GCM mode
andrewleech #6080 ports/unix: Add full uos.dupterm support.
stinos #6055 windows: Publish executables from Appveyor builds available (v2)
tschmid #5514 esp32/DAC: Add cosine generator capability
glennrub #5104 nrf/nfc: Add basic support for NFC tag 2.
livius2 #3583 Simple font size scaling for framebuf
martinribelotta #3400 lpc: Add basic micropython port to LPC Cortex-M series microcontrollers

@robert-hh
Copy link
Contributor

robert-hh commented Feb 29, 2024

I do not recall at which time the check for trailing whitespace in code files was added to the CI, but since then it should only slip though in documentation files or commit messages. Maybe in makefiles as well.

@projectgus
Copy link
Contributor

Thanks @Gadgetoid! Agree it's best if we can merge this PR and #13763 together to minimise churn.

@@ -3,35 +3,35 @@
//
// MACRO and Function prototypes for TI-RTOS and Free-RTOS API calls
//
// Copyright (C) 2014 Texas Instruments Incorporated - http://www.ti.com/
//
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This simplelink is third-party code. But I guess if we are going to add CI to check for trailing spaces, then probably it's easier to just remove these as well.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if we are going to add CI to check for trailing

Isn't that already done. I have seen some complaints of the code formatting checker about trailing space.

Copy link
Contributor Author

@Gadgetoid Gadgetoid Mar 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it's third party code, then ports/cc3200/simplelink/oslib/osi.h is unlikely to be changed outside of total replacement? Thus it wont trigger the whitespace checks. Is it best left untouched?

afaik the only whitespace enforcement is via uncrustify, and just running that across the whole codebase results in yet another huge list of small code consistency changes- 6140ce1 (edit link updated to remove some third party code, also oof there are a lot of strange macro rewrites in there)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and just running that across the whole codebase results in yet another huge list of small code consistency changes

You must be running it wrong, because the macro indent/dedent changes should not be there. You need to run tools/codeformat.py.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was running:

find . -name *.c -not -path "./lib/*" -not -path "./ports/cc3200/FreeRTOS/*" -not -path "./drivers/cc3100/*" | xargs uncrustify -c tools/uncrustify.cfg -lC --replace --no-backup

But on closer inspection it looks like tools/codeformat.py has code (fixup_c() ) to then undo these erroneous changes.

I tried running codeformat after directly running uncrustify and received a much smaller set of changes:

pimoroni@052e0ad

Weirdly if I just run something like:

find . -name *.c -not -path "./lib/*" -not -path "./ports/cc3200/FreeRTOS/*" -not -path "./drivers/cc3100/*" | ./tools/codeformat.py -v -c -f

I get a long list of files that have ostensibly been considered and changed, but no actual changes... it was always my assumption that codeformat.py only picked up files changed in the last commit, but I can't actually find a mechanism for that.

Poking a little further, it looks like all of my changes are to files that are NOT in PATHS and not in EXCLUSIONS˘, eg: drivers/bus/softspi.corexamples/natmod/btree/btree_c.c`.

I should probably raise another PR to consider these...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR raised here, since there are a few minor but salient issues that shouldn't be lost in this nested chat - #14046

@Gadgetoid Gadgetoid force-pushed the patch-trailing-whitespace-removal branch 3 times, most recently from c78bcf7 to b270998 Compare March 6, 2024 09:54
Prune trailing whitespace across the whole project*, this is done
automatically with:

grep -IUrl --color "[[:blank:]]$" --exclude-dir=.git --exclude=*.exp |\
xargs sed -i 's/[[:space:]]*$//'

* Skip third-party code in lib/ and drivers/cc3100/.
* Skip generated code in bluetooth_init_cc2564C_1.5.c
* Preserve command output whitespace in docs, eg:
  docs/esp8266/tutorial/repl.rst

Signed-off-by: Phil Howard <phil@gadgetoid.com>
@Gadgetoid Gadgetoid force-pushed the patch-trailing-whitespace-removal branch from b270998 to 8f801f3 Compare March 6, 2024 09:59
@Gadgetoid
Copy link
Contributor Author

Rebased, and added details of the additional exceptions to the commit message for posterity.

@dpgeorge dpgeorge added this to the release-1.23.0 milestone Mar 7, 2024
@dpgeorge dpgeorge marked this pull request as ready for review March 7, 2024 05:19
@dpgeorge
Copy link
Member

dpgeorge commented Mar 7, 2024

Merged in dda9b9c

@dpgeorge dpgeorge closed this Mar 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants