Skip to content

Support alternate solution for bazel based C++ builds #19447

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
keith opened this issue May 1, 2025 · 3 comments
Open

Support alternate solution for bazel based C++ builds #19447

keith opened this issue May 1, 2025 · 3 comments
Labels
question Further information is requested

Comments

@keith
Copy link

keith commented May 1, 2025

Currently the codeql documentation contains this example for use with bazel based projects:

# Navigate to the Bazel workspace.

# Before building, remove cached objects
# and stop all running Bazel server processes.
bazel clean --expunge

# Build using the following Bazel flags, to help CodeQL detect the build:
# `--spawn_strategy=local`: build locally, instead of using a distributed build
# `--nouse_action_cache`: turn off build caching, which might prevent recompilation of source code
# `--noremote_accept_cached`, `--noremote_upload_local_results`: avoid using a remote cache
# `--disk_cache=`: avoid using a disk cache. Note that a disk cache is no longer considered a remote cache as of Bazel 6.
codeql database create new-database --language=<language> \
--command='bazel build --spawn_strategy=local --nouse_action_cache --noremote_accept_cached --noremote_upload_local_results --disk_cache= //path/to/package:target'

# After building, stop all running Bazel server processes.
# This ensures future build commands start in a clean Bazel server process
# without CodeQL attached.
bazel shutdown

The gist of this is that you must start from scratch and build the whole project on the local machine without any caching. For many projects that moved to bazel moving away from this model was one of the core motivations. For large builds that rely on caching / remote execution, it may no longer be feasible to run the entire build on 1 machine, even if it's only on a scheduled cadence. Moreover the user might not even host the infrastructure to do this anymore, as all of their actual developer and CI builds go through remote execution.

It would be great if there was another way to communicate the necessary information to codeql, in a way that could work alongside bazel's model.

I can't tell from the documentation what the codeql CLI is actually pulling from the build, and I imagine it's quite in depth, but I assume there are other models that could work for providing the same information for certain types of builds.

For example for C++ some natural alternatives (from the user perspective) would be to:

  • produce a compile_commands.json, which is also used in many other developer workflows like C++ LSPs
  • produce a clang indexstore, which can be used as a query-able database for the project
  • produce a kythe index
  • produce a scip-clang index

Each of these options would work better in the bazel model, since indexes could be produced remotely, and cached, and compile_commands.json should already be supported for developer workflows.

related:
@keith keith added the question Further information is requested label May 1, 2025
@keith keith mentioned this issue May 1, 2025
Open
@hvitved
Copy link
Contributor

hvitved commented May 1, 2025

Hi

Thanks for your report, and for your good suggestions.

I can't tell from the documentation what the codeql CLI is actually pulling from the build, and I imagine it's quite in depth, but I assume there are other models that could work for providing the same information for certain types of builds.

The C++ extractor detects invocations of the C++ compiler during the build, and uses that information to determine which files need to be extracted (including e.g. compile-time auto-generated files). This is the reason why the build needs to be full, as the resulting CodeQL database will otherwise be incomplete.

An alternative to doing build-based CodeQL database creation is to use build mode None, but I don't think that is officially supported (yet).

CC @github/codeql-c-extractor .

@jketema
Copy link
Contributor

jketema commented May 1, 2025

To add to @hvitved answer and summarize what is scattered in several places, there's 3 pieces of information from the build that we currently use:

  • The compiler invocations. These we could indeed get from something like a compilation database.
  • The linker invocations, as explained in Can codeql binary use compile_database.json for cpp projects? codeql-cli-binaries#9, and which you already linked to. These are not included in compilation databases, and - as far as I can tell - they are also not included in any of the other "index" solutions that you mention.
  • Any source files that your build generates, as Tom mentions.

Build mode none is indeed not supported at this time for C/C++.

In principle we could support distributed builds - as long as everything is getting build - with the help of codeql database import. However, we do not have a paved path for this. If you're interested in this, I'd recommend contacting GitHub Expert Services.

Build caches we do not support at this time, and depending on how this is arranged in Bazel, this might need close integration with Bazel, as (a) we would somehow need to see that something is pulled from the cache, and as (b) we would need to know how it would have been compiled/linked if it was not pulled from the cache. Something like ccache might be easier support in this respect, given that it's effectively a number of symlinks that we could detect.

@keith
Copy link
Author

keith commented May 1, 2025

Thanks for the replies!

An alternative to doing build-based CodeQL database creation is to use build mode None, but I don't think that is officially supported (yet).

I think omitting generated files (if that was the only downside) would be totally acceptable for our use case. I did a quick test of this and got:

  [2025-05-01 17:16:56] [build-stdout] Build mode 'none' is a pre-release feature that is not enabled in this configuration
  [2025-05-01 17:16:56] [build-stderr] cpp/autobuilder: autobuild summary.

off hand i don't see if there's a user facing way to allow this "pre-release feature"

Thanks for the added detail @jketema!

The linker invocations is an interesting one. The other index formats I mentioned definitely are more geared towards querying information about the final artifacts, more than how they were built. In the case of bazel it would be trivial to generate this information the same way compile commands are generated. Realistically all of this information is query-able without even building anything, which is how the compile commands generation works as well. (unless binary artifacts are also needed, which it doesn't sound like it from your message above) The gist of this is we run bazel aquery which dumps all of the commands that are run, and just mundge the output into whatever we need.

I don't think in order to have more close integration with bazel codeql would have to do anything indepth with the caches. I mentioned that piece because if there was some compilation unit specific artifact that we could produce instead, we could just build all of those and aggregate them, which naturally would pull from the cache if nothing has changed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
question Further information is requested
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@keith @hvitved @jketema