Skip to content

Incorrect shutil.copytree() behaviour with symlinks #91205

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
vajdaz mannequin opened this issue Mar 17, 2022 · 10 comments
Open

Incorrect shutil.copytree() behaviour with symlinks #91205

vajdaz mannequin opened this issue Mar 17, 2022 · 10 comments
Labels
stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@vajdaz
Copy link
Mannequin

vajdaz mannequin commented Mar 17, 2022

BPO 47049
Nosy @vajdaz
PRs
  • bpo-47049: Fix incorrect shutil.copytree() behaviour with symlinks #31967
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = None
    created_at = <Date 2022-03-17.18:50:19.316>
    labels = ['type-bug', 'library', '3.11']
    title = 'Incorrect shutil.copytree() behaviour with symlinks'
    updated_at = <Date 2022-04-04.22:13:23.663>
    user = 'https://github.com/vajdaz'

    bugs.python.org fields:

    activity = <Date 2022-04-04.22:13:23.663>
    actor = 'vajdaz'
    assignee = 'none'
    closed = False
    closed_date = None
    closer = None
    components = ['Library (Lib)']
    creation = <Date 2022-03-17.18:50:19.316>
    creator = 'vajdaz'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 47049
    keywords = ['patch']
    message_count = 2.0
    messages = ['415437', '416700']
    nosy_count = 1.0
    nosy_names = ['vajdaz']
    pr_nums = ['31967']
    priority = 'normal'
    resolution = None
    stage = 'patch review'
    status = 'open'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue47049'
    versions = ['Python 3.11']

    Linked PRs

    @vajdaz
    Copy link
    Mannequin Author

    vajdaz mannequin commented Mar 17, 2022

    shutil.copytree incorrectly does not copy symlink contents if called
    with symlink=False and ignore_dangling_symlinks=True.

    The wrong behaviour can be reproduced like this:

    $ tree
    .
    └── a
        ├── a.txt
        └── b
            └── a.txt -> ../a.txt
    
    $ python3 -c "import shutil;shutil.copytree('a/b', 'c', symlinks=False, ignore_dangling_symlinks=True)"

    As a result directoy c will be created but it will remain empty.
    Expected result is a file c/a.txt with the contents of a/b/a.txt.

    @vajdaz vajdaz mannequin added 3.11 only security fixes stdlib Python modules in the Lib dir labels Mar 17, 2022
    @vajdaz

    This comment was marked as outdated.

    @vajdaz vajdaz mannequin added type-bug An unexpected behavior, bug, or error labels Apr 4, 2022
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    @eakr9014
    Copy link

    This still seems to be an issue even though the underlying PR was closed. Even though ignore_dangling_symlinks is passed to recursive calls of copy_tree now, the function will still incorrectly classify symlinks as dangling even if they aren't because it attempts to do a os.readlink relative to the current working directory, not the copied directory

    @dabuahoid
    Copy link

    The issue is still present in Python 3.12.5 and must be fixed since copied data with enabled option symlinks=False and ignore_dangling_symlinks=True misses all resolved symbolic links file contents.

    @dabuahoid
    Copy link

    dabuahoid commented Sep 16, 2024

    Fix in shutil _copytree could look like:

    linkPath = linkto if os.path.isabs(linkto) else os.path.join(os.path.dirname(srcname), linkto)
    # ignore dangling symlink if the flag is on
    if not os.path.exists(linkPath) and ignore_dangling_symlinks:

    @dabuahoid
    Copy link

    @hugovk since you closed my duplicate: may I ask when this fix will finally go into Python 3.12 or 3.13? It is annoying to always fix the one line for each new python release to get the correct behavior. Thanks upfront!

    @hugovk
    Copy link
    Member

    hugovk commented Nov 20, 2024

    Hello! The linked PR was closed before merge:

    Seems that this has been solved in #22937 and it is also a duplicate of #82704. Therefore I close this PR.

    If it's still a problem, someone needs to submit a PR first, then we can merge that into main/3.14, and then we can backport it to 3.12 and 3.13, and it would go out in the subsequent releases. So there's currently no timeline on a fix.

    Are you interested in creating a PR to fix it, with tests?

    @dabuahoid
    Copy link

    Hi @hugovk , thanks for the reply - since I am not familiar with the working style I would prefer the experts doing it.
    However, since the bug is still present in Python 3.14 at least one ticket should be available - will you take care here that this small fix is not lost again?
    Thanks and best regards!

    @hugovk
    Copy link
    Member

    hugovk commented Nov 21, 2024

    I'm not planning on working on the fix, but this is the ticket for it.

    @dabuahoid
    Copy link

    Hi all,
    any plan when to fix this issue?
    I consider it as quite critical since the resulting data is not as intended.

    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    4 participants