Skip to content

cksum: --check can't handle non-UTF-8 in comments #6573

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
BenWiederhake opened this issue Jul 15, 2024 · 4 comments
Closed

cksum: --check can't handle non-UTF-8 in comments #6573

BenWiederhake opened this issue Jul 15, 2024 · 4 comments
Labels
U - cksum

Comments

@BenWiederhake
Copy link
Collaborator

Currently, we convert every line to a String, which has the disadvantage of not properly handling malformed UTF-8, even if it appears in an "irrelevant" place:

$ cat <<EOF | sed -re 's,xxx,\xff,' > sums
SHA256 (README.md) = 4d9b4c7b50b976ec5b0b2dce5a767d2da4b63eb68592017d475c4e11799a333c
# Comment shouldn't matter: >>xxx<<
SHA256 (LICENSE) = 29953405eaa3dcc41c37d1621d55b6a47eee93e05613e439e73295029740b10c
EOF
$ cksum --check sums
README.md: OK
LICENSE: OK
$ cargo run -q cksum --check sums
cksum: stream did not contain valid UTF-8
[$? = 1]

It also isn't supported in other places during check:

$ echo 'U0hBMjU2IChmdW5ref9uYW1lKSA9IDI5OTUzNDA1ZWFhM2RjYzQxYzM3ZDE2MjFkNTViNmE0N2VlZTkzZTA1NjEzZTQzOWU3MzI5NTAyOTc0MGIxMGMK' | base64 -d > sums
$ cksum -c sums 
cksum: 'funky'$'\377''name': No such file or directory
funky�name: FAILED open or read
cksum: WARNING: 1 listed file could not be read
[$? = 1]
$ cargo run -q cksum -c sums 
cksum: stream did not contain valid UTF-8
[$? = 1]
@RenjiSann RenjiSann mentioned this issue Aug 1, 2024
Closed
@RenjiSann
Copy link
Collaborator

This is partially fixed by #6793 and #6822.
A corner case that is yet to be addressed is when a non-utf8 filename does not exist and it is escaped before being displayed

$ ../../../gnu/src/cksum -c CHECKSUM                 
../../../gnu/src/cksum: 'XXX'$'\377''XXX': No such file or directory    # escaped
XXXXXX: FAILED open or read
../../../gnu/src/cksum: WARNING: 1 listed file could not be read

$ ../../target/debug/cksum -c CHECKSUM                                 
../../target/debug/cksum: XXX�XXX: No such file or directory           # not escaped
XXXXXX: FAILED open or read
../../target/debug/cksum: WARNING: 1 listed file could not be read

@RenjiSann RenjiSann mentioned this issue Nov 27, 2024
Open
@RenjiSann
Copy link
Collaborator

@BenWiederhake
I suggest we close this issue, as the initial problem was solved.
I opened #6892 for my last comment. The fix might take some time to come, as it requires MRSV>=1.79

@RenjiSann
Copy link
Collaborator

@sylvestre Can we close this issue ?

@sylvestre
Copy link
Contributor

sorry, sure :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
U - cksum
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

cksum: Accept non UTF-8 inputs RenjiSann/coreutils
3 participants
@sylvestre @BenWiederhake @RenjiSann