Skip to content

feat: support raw symlinks from declare_symlink() #930

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 18 commits into from
Apr 11, 2025

Conversation

rickeylev
Copy link
Contributor

Using the ctx.actions.declare_symlink() API, rules can create explicit symlinks to arbitrary
paths, which Bazel will keep as symlinks. The packaging logic treats these
as regular files, so tries to read through them and store the content they point to. This
is incorrect for two reasons: they are supposed to be left as symlinks and may be dangling
symlinks from the perspective of packaging.

To fix, introduce a new type of file entry: raw_symlink. These entries mean the src is
a symlink whose stored path should be stored into the destination verbatim, also as a symlink.
Such artifacts are identified using File.is_symlink, a Bazel 8+ API.

Work towards #929

Copy link
Collaborator

@cgrindel cgrindel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add a test for this?

@rickeylev
Copy link
Contributor Author

Ok, it's been awhile since I could get time to come back to this!

I've added a basic test. PTAL.

@rickeylev
Copy link
Contributor Author

OK, PTAL.

I fixed the buildifier presubmit failure.

The other failure looks unrelated -- I get the same error at clean head. It's specific to the rolling version of bazel and something about workspace support.

@rickeylev
Copy link
Contributor Author

Actually, I partially take that back -- The LTS windows test is failing. Some quick searches indicates windows symlink support should work fine with the values the test is using (relative symlinks that point nowhere).

Diagnosing and fixing this will be a bit hard, but I'll see what I can do. If it comes to it, can we simply mark this test to skip windows?

@rickeylev rickeylev requested a review from meteorcloudy as a code owner March 19, 2025 16:07
@meteorcloudy
Copy link
Member

https://bazel.build/configure/windows#symlink
Do you have --windows_enable_symlinks specified as a startup flag? Otherwise a junction might be created instead of a symlink, which doesn't support relative path.

See https://cs.opensource.google/bazel/bazel/+/master:src/main/java/com/google/devtools/build/lib/windows/WindowsFileSystem.java;l=83-92

@rickeylev
Copy link
Contributor Author

I tried adding --windows_enable_symlinks, but that didn't fix it.

So far, all I can tell is that, when tar building occurs, and python calls os.readlink() to figure out the value contained in the symlink, it gets back an absolute path e.g. \\?\C:\b\3ctu2esx\execroot\_main\bazel-out\x64_windows-fastbuild\bin\tests\tar\link1. That is the correct resolved path (foo_link1 -> ./link1), but yeah, we don't want the resolved path.

I can't tell which of these cases is happening:

  1. Bazel is writing an absolute path
  2. Windows is silently transforming it to an absolute path at write
  3. Windows is silently transforming it to an absolute path at read
  4. Python is transforming it to an absolute path

I won't have direct access to a windows machine until this evening, so I'll have to wait until then to check some of these ideas.

But overall, this isn't sounding promising. I don't think we can work around (1) or (2). A work around for (3) and (4) might be possible by using e.g. ctypes and dropping into lower-level APIs.

@rickeylev
Copy link
Contributor Author

After testing on my personal windows machine, I'm thinking its (1: Bazel writing an absolute path). I can use mklink to create relative symlinks, and python's os.readlink() returns the relative path correctly. When I use an absolute path, it returns a \\?\... looking string, the same as what I'm observing.

@rickeylev
Copy link
Contributor Author

rickeylev commented Mar 20, 2025

Aha, here we go. known issue in Bazel specific to windows: bazelbuild/bazel#14224

@meteorcloudy
Copy link
Member

OK, I guess let's disable the test on Windows for now

@rickeylev
Copy link
Contributor Author

Tests fixed. PTAL

@rickeylev
Copy link
Contributor Author

bump -- this has been sitting for awhile; is there an ETA on when someone can review?

@cgrindel
Copy link
Collaborator

bump -- this has been sitting for awhile; is there an ETA on when someone can review?

In the future, if you hit the "Re-request review" button, that will let the reviewer who has left comments look at the PR again. Looking now

@rickeylev rickeylev requested a review from cgrindel April 10, 2025 23:53
Copy link
Collaborator

@cgrindel cgrindel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. @aiuto Did you want to look before we merge?

@rickeylev
Copy link
Contributor Author

Oh, i had not noticed that thingy before. Done, for at least my own practice :). Thanks!

@tonyaiuto tonyaiuto merged commit e708301 into bazelbuild:main Apr 11, 2025
3 checks passed
@rickeylev rickeylev deleted the feat.raw.symlinks branch April 15, 2025 16:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants