Skip to content

cp: require preserve only certain attributes #4099

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 26 commits into from
Jan 19, 2023

Conversation

sssemil
Copy link
Contributor

@sssemil sssemil commented Nov 1, 2022

In GNU cp, there's a lot of special handling of errors based on using -a, -p, --preserve=all, --preserve=xattr,context etc. which is not handled properly at the moment. In this PR, an analogous behavior to that of require_preserve_xattr, require_preserve_context, and require_preserve flags in GNU cp code is implemented.

Somethings left todo are testing xattr permission denied case (outside of Android test), and also context permission denied case. One idea is making a simple user-space fuse FS with advanced xattr modification permission controls. This will be implemented in a separate PR here - #4100.

Original PR: #4095 (started implemeting testfs there, but it's a bit out of scope, so, I'm separating these PR's).

Closes #4079.

@sssemil sssemil force-pushed the require_preserve_4079 branch 4 times, most recently from f43fc85 to 41df37c Compare November 2, 2022 16:19
@github-actions
Copy link

github-actions bot commented Nov 2, 2022

GNU testsuite comparison:

GNU test failed: tests/misc/tee. tests/misc/tee is passing on 'main'. Maybe you have to rebase?

@github-actions
Copy link

github-actions bot commented Nov 9, 2022

GNU testsuite comparison:

GNU test failed: tests/misc/timeout. tests/misc/timeout is passing on 'main'. Maybe you have to rebase?

Copy link
Member

@tertsdiepraam tertsdiepraam left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, I think it'd be good to rethink how we handle attributes, cause I don't the approach with the Vec<Attribute> is great. It feels more complex that it needs to be. Instead of an Attribute enum, I think we need an Attributes struct with a Preserve enum. Let me give a sketch:

struct Attributes {
    ownership: Preserve,
    mode: Preserve,
    timestamps: Preserve,
    context: Preserve,
    links: Preserve,
    xattr: Preserve,
}

enum Preserve {
    No,      // don't preserve
    Try,     // preserve but don't error
    Require, // return error if it doesn't work
}

Alternatively:

enum Preserve {
    No,
    Yes { required: bool },
}

The default would be No unless the attribute is specified to be preserved. In the case of "all", we could then do:

impl Attributes {
    fn all() -> Self {
         Self {
             ownership: Preserve::Require,
             mode: Preserve::Require,
             timestamps: Preserve::Require,
             links: Preserve::Require,
             context: Preserve::Try,
             xattr: Preserve::Try,
         }
    }
}

I think this makes the code much easier to follow.

@sssemil
Copy link
Contributor Author

sssemil commented Nov 22, 2022

an Attributes struct

@tertsdiepraam Nice idea, makes sense :) I wonder what the copy_attributes function should look like then, just manually going through each attribute? One way of avoiding that could be params for enums, what do you think about something like this:

pub enum Attribute {
    #[cfg(unix)]
    Ownership { required: bool },
    Mode { required: bool },
    Timestamps { required: bool },
    #[cfg(feature = "feat_selinux")]
    Context { required: bool },
    Links { required: bool },
    Xattr { required: bool },
}

Although, this would cause a lot of change to unrelated code, and some unnecessary required setting...

@tertsdiepraam
Copy link
Member

I don't see how the required flag would help with not iterating through each attribute? I agree though that this would change too many things at the same time.

@sssemil
Copy link
Contributor Author

sssemil commented Nov 22, 2022

I don't see how the required flag would help with not iterating through each attribute? I agree though that this would change too many things at the same time.

Ah, I was going the opposite direction here. Was concerned that replacing that iteration over attributes vector with handling each separately would be a negative.

I'll try with the struct, let's see how it works out. Thanks!

@tertsdiepraam
Copy link
Member

tertsdiepraam commented Nov 22, 2022

Oh I only see now that you wrote enum and not struct! That makes your message more clear :)

Was concerned that replacing that iteration over attributes vector with handling each separately would be a negative.

I actually want to replace the Vec indeed, so Options becomes

struct Options {
    ...
    attributes: Attributes,
    ...
}

so copy_attributes just becomes (rough sketch):

fn ignore_if_try<E>(p: Preserve, res: Result<(), E>) -> Result<(),E> {
    match res {
       Err(e) if p == Preserve::Require => Err(e)
       _ => (),
    }
}

fn copy_attributes(attrs: Attributes, ...) -> Result<...> {
    if attrs.ownership != Preserve::No {
         let error = {...};
         ignore_if_try(attr.ownership, error)?
    }
    
    if attrs.mode != Preserve::No {
         ...
    }
}

(I made the ignore_if_try up, you might come up with something better)

@sssemil sssemil force-pushed the require_preserve_4079 branch from 935066b to fa3af7a Compare November 23, 2022 12:36
@sssemil
Copy link
Contributor Author

sssemil commented Nov 23, 2022

@tertsdiepraam pushed my version of it :)

Copy link
Member

@tertsdiepraam tertsdiepraam left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's starting to look super clean! Great work. I have a few more suggestions and some bits I think are not quite right, though I might be mistaken.

@sssemil sssemil force-pushed the require_preserve_4079 branch from 0f90522 to ff789e8 Compare November 28, 2022 13:52
@sylvestre
Copy link
Contributor

fails on mac because of:


---- test_cp::test_cp_preserve_xattr stdout ----
current_directory_resolved: 
run: /Users/runner/work/coreutils/coreutils/target/debug/coreutils cp a b --preserve=xattr
thread 'test_cp::test_cp_preserve_xattr' panicked at 'assertion failed: `(left != right)`
  left: `1669645276`,
 right: `1669645276`', tests/by-util/test_cp.rs:963:5

could you have please a look?
thanks

@sssemil
Copy link
Contributor Author

sssemil commented Dec 7, 2022

fails on mac because of:


---- test_cp::test_cp_preserve_xattr stdout ----
current_directory_resolved: 
run: /Users/runner/work/coreutils/coreutils/target/debug/coreutils cp a b --preserve=xattr
thread 'test_cp::test_cp_preserve_xattr' panicked at 'assertion failed: `(left != right)`
  left: `1669645276`,
 right: `1669645276`', tests/by-util/test_cp.rs:963:5

could you have please a look? thanks

Hm, it looks like the modified time is copied in tests but not when running it locally... Is there some unusual file handling in the tests?

GNU cp:

$ rm -rf a b; touch a; sleep 1; cp --preserve=xattr a b; stat a b
  File: a
  Size: 0               Blocks: 0          IO Block: 4096   regular empty file
Device: 0,37    Inode: 20061619    Links: 1
Access: (0644/-rw-r--r--)  Uid: ( 1000/    emil)   Gid: ( 1000/    emil)
Access: 2022-12-07 03:28:14.095178786 +0100
Modify: 2022-12-07 03:28:14.095178786 +0100
Change: 2022-12-07 03:28:14.095178786 +0100
 Birth: 2022-12-07 03:28:14.095178786 +0100
  File: b
  Size: 0               Blocks: 0          IO Block: 4096   regular empty file
Device: 0,37    Inode: 20061624    Links: 1
Access: (0644/-rw-r--r--)  Uid: ( 1000/    emil)   Gid: ( 1000/    emil)
Access: 2022-12-07 03:28:15.101800872 +0100
Modify: 2022-12-07 03:28:15.101800872 +0100
Change: 2022-12-07 03:28:15.101800872 +0100
 Birth: 2022-12-07 03:28:15.101800872 +0100

uutils/coreutils:

$ rm -rf a b; touch a; sleep 1; coreutils cp --preserve=xattr a b; stat a b
  File: a
  Size: 0               Blocks: 0          IO Block: 4096   regular empty file
Device: 0,37    Inode: 20061695    Links: 1
Access: (0644/-rw-r--r--)  Uid: ( 1000/    emil)   Gid: ( 1000/    emil)
Access: 2022-12-07 03:30:23.899427115 +0100
Modify: 2022-12-07 03:30:23.899427115 +0100
Change: 2022-12-07 03:30:23.899427115 +0100
 Birth: 2022-12-07 03:30:23.899427115 +0100
  File: b
  Size: 0               Blocks: 0          IO Block: 4096   regular empty file
Device: 0,37    Inode: 20061699    Links: 1
Access: (0644/-rw-r--r--)  Uid: ( 1000/    emil)   Gid: ( 1000/    emil)
Access: 2022-12-07 03:30:24.902715973 +0100
Modify: 2022-12-07 03:30:24.902715973 +0100
Change: 2022-12-07 03:30:24.902715973 +0100
 Birth: 2022-12-07 03:30:24.902715973 +0100

@sssemil
Copy link
Contributor Author

sssemil commented Dec 7, 2022

Here's what these files look like during the test:

/tmp/.tmpat5LjD $ stat a b
  File: a
  Size: 0               Blocks: 0          IO Block: 4096   regular empty file
Device: 0,39    Inode: 4914        Links: 1
Access: (0500/-r-x------)  Uid: ( 1000/    emil)   Gid: ( 1000/    emil)
Access: 2022-12-07 03:40:10.430054364 +0100
Modify: 2022-12-07 03:40:09.386767394 +0100
Change: 2022-12-07 03:40:09.386767394 +0100
 Birth: 2022-12-07 03:40:09.386767394 +0100
  File: b
  Size: 0               Blocks: 0          IO Block: 4096   regular empty file
Device: 0,39    Inode: 4915        Links: 1
Access: (0500/-r-x------)  Uid: ( 1000/    emil)   Gid: ( 1000/    emil)
Access: 2022-12-07 03:40:10.430054364 +0100
Modify: 2022-12-07 03:40:10.430054364 +0100
Change: 2022-12-07 03:40:10.430054364 +0100
 Birth: 2022-12-07 03:40:10.430054364 +0100

The access time is updates in the test, unlike in the manual run. Maybe difference between how stat and the fs::metadata work. But doesn't explain the mac os error.

@sssemil sssemil force-pushed the require_preserve_4079 branch from ff789e8 to fca82d1 Compare December 7, 2022 04:54
# Conflicts:
#	src/uu/cp/src/copydir.rs
#	src/uu/cp/src/cp.rs
On Android, this cp with explicit preserve of xattr must fail, because of the limitations of the filesystem setup used on Android.
# Conflicts:
#	tests/by-util/test_cp.rs
# Conflicts:
#	src/uu/cp/src/cp.rs
Access timestamps appear to be modified only in this test. Running the command directly does not alter the access timestamp.
@sssemil sssemil force-pushed the require_preserve_4079 branch from 28e7697 to 013bb05 Compare January 18, 2023 13:35
@tertsdiepraam tertsdiepraam merged commit 50c1833 into uutils:main Jan 19, 2023
@tertsdiepraam
Copy link
Member

Excellent, thanks!

@sssemil sssemil deleted the require_preserve_4079 branch January 20, 2023 04:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

test_cp_parents_2_dirs fails on android
3 participants