-
-
Notifications
You must be signed in to change notification settings - Fork 1.5k
shred: use 4K block #7886
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
shred: use 4K block #7886
Conversation
This commit changes the default block size for the shred utility to make it more consistent with the GNU coreutils version. Issue uutils#7870
GNU testsuite comparison:
|
did you look if it has an impact on performances ? thanks |
@sylvestre Hello. I performed a brief test on shred using two different types of files: one smaller than the block size (1 byte), and one larger (1MB). I used hyperfine to collect information about time and deviation.
I recreated the file before each test. On 1 byte file
1.4 ms ± 0.3 ms - uutils shred [128 bytes] On 1MB file
3.4 ms ± 0.5 ms - uutils shred [2048 bytes]** It seems that with a 1MB file, we have many more Please let me know if more detailed measurements are needed or if different data collection methods should be used. Thank you |
I'm sure it's also possible to avoid the differences with GNU shred in another way. Instead of changing the buffer size, we could modify the number of bytes that are overwritten in do_pass. Something like let size = if exact { bytes_left } else { std::cmp::min(4096, BLOCK_SIZE) }; In this case, the behavior would be the same as with GNU shred, but without the slowdown caused by the extra write calls. What do you think? |
Here is a PR with alignment changes, but without any modification to the block size. |
I'm going to close this PR, as it results in a noticeable slowdown. Fixing the differences between the GNU and uutils versions by adjusting alignment seems to be a better approach. |
About
Issue #7870
This commit changes the default block size for the shred utility to make it more consistent with the GNU coreutils version.
Before
After
Expected
Please note: The original shred utility uses
fstat
along with the macroses to determine the block size. Therefore, the block size is not constant and may vary across platforms. The most common sizes are 512 bytes and 4 KB. Here, I started by changing the constant and collecting test results and observations. However, to replicate the behavior of shred, deeper modifications are required. Please feel free to share your thoughts and opinions on which approach is preferable: simply changing the constant or moving block size detection to runtime.Thank you