Skip to content

gh-91205: fix bug in shutil.copytree with relative links and ignore_dangling_symlinks=True #132984

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
+30 −0
Open

gh-91205: fix bug in shutil.copytree with relative links and ignore_dangling_symlinks=True #132984

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
71de858
gh-91205: fix bug in shutil.copytree with relative links and ignore_d…
duaneg Apr 26, 2025
c0686d6
Address reviewer feedback and in particular add a couple of tests for…
duaneg Apr 26, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions Lib/shutil.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -569,6 +569,10 @@ def _copytree(entries, src, dst, symlinks, ignore, copy_function,
os.symlink(linkto, dstname)
copystat(srcobj, dstname, follow_symlinks=not symlinks)
else:
# if the link is not to an absolute path it is relative to
# the source (see gh-91205)
if not os.path.isabs(linkto):
linkto = os.path.join(os.path.dirname(srcname), linkto)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The original patch used os.path.normpath. Do we need it? Also, shouldn't we retest whether linkto is a symlink or not? if not, please also add a test

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should use normpath: it does a string replacement which can change the meaning if symbolic links are involved, which I think would be a bug if it can happen. If it doesn't change the meaning then it shouldn't matter, since we just use the result in one place immediately after, to check whether it exists or not.

We don't need to check whether the result is a link: it may or may not be, but is correctly handled either way by the following code. os.path.exists will return false if it is a dangling link (and we will either skip it or carry on and raise an error depending on ignore_dangling_symlinks). Note that absolute symbolic links are handled the same way, so if this was necessary they would already be broken.

Having said that, good point about testing: AFAICT this case is not currently under test. I'll add a few levels of valid links-to-links to the new test case and expand test_copytree_dangling_symlinks to test dangling links-to-links.

# ignore dangling symlink if the flag is on
if not os.path.exists(linkto) and ignore_dangling_symlinks:
continue
Expand Down
24 changes: 24 additions & 0 deletions Lib/test/test_shutil.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1077,6 +1077,30 @@ def test_copytree_symlink_dir(self):
self.assertTrue(os.path.islink(os.path.join(dst_dir, 'link_to_dir')))
self.assertIn('test.txt', os.listdir(os.path.join(dst_dir, 'link_to_dir')))

@os_helper.skip_unless_symlink
def test_copytree_relative_symlink(self):
# gh-91205: Ensure valid relative symlinks are copied regardless of the
# value of the ``ignore_dangling_symlinks`` flag.
src_dir = self.mkdtemp()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
src_dir = self.mkdtemp()
src_dir = self.mkdtemp()
self.addCleanup(shutil.rmtree, src_dir, ignore_errors=True)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is necessary (here and elsewhere): the self.mkdtemp() method called here calls self.addCleanup(os_helper.rmtree, d) on the result, so unless I'm missing something it shouldn't be required. I note most of the calls to self.mkdtemp() in this module do not add extra cleanup, although some do. The test doesn't warn about files being left behind, so they are being cleaned up properly without this.

dir_a = os.path.join(src_dir, 'a')
dir_a_dir_b = os.path.join(dir_a, 'b')
os.mkdir(dir_a)
os.mkdir(dir_a_dir_b)
create_file(os.path.join(dir_a, 'a.txt'))
# create a symlink from src/a/b/a.txt to ../a.txt
os.symlink(os.path.join(os.pardir, 'a.txt'),
os.path.join(dir_a_dir_b, 'a.txt'))

for ignore_dangling_symlinks in (True, False):
with self.subTest(ignore_dangling_symlinks=ignore_dangling_symlinks):
dst_dir = os.path.join(self.mkdtemp(), 'x')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
dst_dir = os.path.join(self.mkdtemp(), 'x')
dst_dir = os.path.join(self.mkdtemp(), 'x')
self.addCleanup(shutil.rmtree, dst_dir, ignore_errors=True)

shutil.copytree(
dir_a_dir_b, dst_dir, symlinks=False,
ignore_dangling_symlinks=ignore_dangling_symlinks)
self.assertIn('a.txt', os.listdir(dst_dir))
self.assertFalse(
os.path.islink(os.path.join(dst_dir, 'a.txt')))

def test_copytree_return_value(self):
# copytree returns its destination path.
src_dir = self.mkdtemp()
Expand Down
2 changes: 2 additions & 0 deletions Misc/NEWS.d/next/Library/2025-04-26-13-44-46.gh-issue-91205.kYPo51.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Avoid skipping symbolic links to relative paths in :func:`shutil.copytree` when
``ignore_dangling_symlinks=True`` and ``symlinks=False``.
Loading