Allow non-utf8 paths name as input for most of the programs #8457

sylvestre · 2025-08-09T13:42:58Z

add a fuzzer to verify that we don't regress in the future

github-actions · 2025-08-09T14:18:17Z

GNU testsuite comparison:

GNU test failed: tests/misc/realpath. tests/misc/realpath is passing on 'main'. Maybe you have to rebase?
Skip an intermittent issue tests/misc/stdbuf (fails in this run but passes in the 'main' branch)

drinkcat

Note: didn't review fuzzer, went all to way to expand.

That's... a huge amount of work, thanks! I'll keep reviewing another day ,-P

src/uu/chmod/src/chmod.rs

src/uu/cat/src/cat.rs

src/uu/du/src/du.rs

github-actions · 2025-08-09T15:08:40Z

GNU testsuite comparison:

GNU test failed: tests/misc/realpath. tests/misc/realpath is passing on 'main'. Maybe you have to rebase?
Skip an intermittent issue tests/timeout/timeout (fails in this run but passes in the 'main' branch)

github-actions · 2025-08-09T16:22:36Z

GNU testsuite comparison:

GNU test failed: tests/misc/realpath. tests/misc/realpath is passing on 'main'. Maybe you have to rebase?
Skipping an intermittent issue tests/misc/stdbuf (passes in this run but fails in the 'main' branch)

github-actions · 2025-08-09T17:29:02Z

GNU testsuite comparison:

Skip an intermittent issue tests/misc/tee (fails in this run but passes in the 'main' branch)

drinkcat

expand->pathchk

src/uu/head/src/head.rs

src/uu/install/src/install.rs

drinkcat · 2025-08-10T02:42:39Z

src/uu/more/src/more.rs

            more(
                InputType::File(BufReader::new(opened_file)),
                length > 1,
-                file.to_str(),
-                next_file.copied(),
+                Some(&file.to_string_lossy()),


I think you should fix more and Pager...

i think it requires a bunch of changes, ok if I do that in a different PR ?
(and i don't think i regressed the current state)

src/uu/pathchk/src/pathchk.rs

drinkcat

Note: Reviewed everything until the tests.

src/uu/ptx/src/ptx.rs

src/uu/readlink/src/readlink.rs

src/uu/realpath/src/realpath.rs

drinkcat · 2025-08-10T05:19:11Z

src/uu/sum/src/sum.rs

+            writeln!(
+                stdout,
+                "{sum:0width$} {blocks:width$} {}",
+                file.to_string_lossy()


This is technically wrong, you should print the bytes (but you could just add a TODO).

src/uu/tac/src/tac.rs

src/uu/touch/src/touch.rs

src/uu/tsort/src/tsort.rs

tests/by-util/test_unlink.rs

tests/by-util/test_truncate.rs

tests/by-util/test_touch.rs

src/uu/touch/src/touch.rs

src/uu/split/src/split.rs

src/uu/truncate/src/truncate.rs

tests/by-util/test_shred.rs

tests/by-util/test_realpath.rs

tests/by-util/test_readlink.rs

tests/by-util/test_pathchk.rs

tests/by-util/test_more.rs

src/uu/mktemp/src/mktemp.rs

github-actions · 2025-08-11T23:23:17Z

GNU testsuite comparison:

Skip an intermittent issue tests/timeout/timeout (fails in this run but passes in the 'main' branch)
Skipping an intermittent issue tests/misc/tee (passes in this run but fails in the 'main' branch)

Fixes issue introduced in b965c94 where switching from NonEmptyStringValueParser to OsString parser removed validation that arguments cannot be empty strings. - Add custom NonEmptyOsStringParser that validates OsString is not empty - Use the parser for FILES, --relative-to, and --relative-base arguments - Add test case to verify empty strings are rejected with exit code 1 - Fixes GNU realpath test failure

Co-authored-by: Daniel Hofstetter <daniel.hofstetter@42dh.com>

Co-authored-by: Nicolas Boichat <nicolas@boichat.ch>

github-actions · 2025-08-14T09:10:52Z

GNU testsuite comparison:

Skipping an intermittent issue tests/misc/tee (passes in this run but fails in the 'main' branch)
Skipping an intermittent issue tests/timeout/timeout (passes in this run but fails in the 'main' branch)

github-actions · 2025-08-14T10:05:58Z

GNU testsuite comparison:

Skipping an intermittent issue tests/timeout/timeout (passes in this run but fails in the 'main' branch)

github-actions · 2025-08-14T12:40:37Z

GNU testsuite comparison:

Skip an intermittent issue tests/misc/tee (fails in this run but passes in the 'main' branch)

cakebaker · 2025-08-14T12:53:10Z

Good work :)

sylvestre force-pushed the no-utf8 branch from db36257 to 42cc1dc Compare August 9, 2025 13:56

cakebaker linked an issue Aug 9, 2025 that may be closed by this pull request

Make sure all tools handle non-Unicode filenames as parameters #8368

Closed

sylvestre requested a review from cakebaker August 9, 2025 14:48

drinkcat reviewed Aug 9, 2025

View reviewed changes

src/uu/chmod/src/chmod.rs Outdated Show resolved Hide resolved

src/uu/cat/src/cat.rs Outdated Show resolved Hide resolved

src/uu/du/src/du.rs Outdated Show resolved Hide resolved

src/uu/du/src/du.rs Outdated Show resolved Hide resolved

src/uu/du/src/du.rs Outdated Show resolved Hide resolved

drinkcat reviewed Aug 10, 2025

View reviewed changes

cakebaker reviewed Aug 10, 2025

View reviewed changes

tests/by-util/test_unlink.rs Outdated Show resolved Hide resolved

cakebaker reviewed Aug 10, 2025

View reviewed changes

tests/by-util/test_unlink.rs Outdated Show resolved Hide resolved

cakebaker reviewed Aug 10, 2025

View reviewed changes

tests/by-util/test_truncate.rs Outdated Show resolved Hide resolved

cakebaker reviewed Aug 10, 2025

View reviewed changes

tests/by-util/test_truncate.rs Outdated Show resolved Hide resolved

cakebaker reviewed Aug 10, 2025

View reviewed changes

tests/by-util/test_touch.rs Outdated Show resolved Hide resolved

cakebaker reviewed Aug 10, 2025

View reviewed changes

src/uu/touch/src/touch.rs Outdated Show resolved Hide resolved