Skip to content

[JAVA] [GRADLE] OOM Issue with GitHub Autobuilder for Kotlin #19374

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
JLLeitschuh opened this issue Apr 24, 2025 · 4 comments
Open

[JAVA] [GRADLE] OOM Issue with GitHub Autobuilder for Kotlin #19374

JLLeitschuh opened this issue Apr 24, 2025 · 4 comments
Labels
question Further information is requested

Comments

@JLLeitschuh
Copy link
Contributor

Description of the issue

Currently getting an OOM error with the CodeQL Autobuilder when attempting to compile Kotlin.

https://github.com/JLLeitschuh/ktlint-gradle/actions/runs/14649396135/job/41111305233?pr=861#step:5:121

JLLeitschuh/ktlint-gradle#861

Build Scan: https://scans.gradle.com/s/mfzgqpzmjgq6q

The stack trace error is too long for a GitHub issue, so I've attached it here. It looks like the code injected inside the compiler is leading to an OOM in the JetBrains Kotlin compiler

logs_37644012249.zip

@JLLeitschuh JLLeitschuh added the question Further information is requested label Apr 24, 2025
@redsun82
Copy link
Contributor

redsun82 commented Apr 25, 2025

👋 @JLLeitschuh I'm sorry to hear you had problems with setting up CodeQL for your kotlin project.

I don't think this should happen with autobuild in your case. As far as I could test moving to a manual build was successful, so I opened JLLeitschuh/ktlint-gradle#865 with the proposed changes:

  • move to build-mode manual from autobuild
  • move the gradle setup before codeql init: this is probably not required for it to work, but it's good hygiene, as otherwise setup steps get analyzed by CodeQL as build steps, which adds overhead and may give unwanted data in the CodeQL database
  • provide an explicit command for the build to be analyzed instead of relying on autobuild: you may want to tweak this if it does not cover what you want being analyzed in your project.

In any case I will reach out with this to our internal team, as this might be an autobuild bug.

@JLLeitschuh
Copy link
Contributor Author

The concern I have with this solution is that the init script that the autobuilder uses won't be injected into the Gradle build with this solution. I don't know how essential the init script is or not. There doesn't seem to be a way to get that init script out of the environment as far as I can tell.

This was the contents of that init script as of back in May of 2024:

allprojects { 
  try {
    tasks.withType(JavaCompile).configureEach {
      outputs.doNotCacheIf("CodeQL analysis", {true})
    }

    tasks.withType(ScalaCompile).configureEach {
      outputs.doNotCacheIf("CodeQL analysis", {true})
    }

    tasks.withType(GroovyCompile).configureEach {
      outputs.doNotCacheIf("CodeQL analysis", {true})
    }
  }
  catch(e) {
    println e
    println "Warning: CodeQL: configureEach not supported on this Gradle version. Using fallback instead."
    tasks.withType(JavaCompile) {
      try {
        outputs.cacheIf({false})
      }
      catch(e2) {
        println e2
        println "Warning: CodeQL: cacheIf not supported on this Gradle version."
      }
    }
    tasks.withType(ScalaCompile) {
      try {
        outputs.cacheIf({false})
      }
      catch(e2) {
        println e2
        println "Warning: CodeQL: cacheIf not supported on this Gradle version."
      }
    }
    tasks.withType(GroovyCompile) {
      try {
        outputs.cacheIf({false})
      }
      catch(e2) {
        println e2
        println "Warning: CodeQL: cacheIf not supported on this Gradle version."
      }
    }
  }

  // KotlinCompile comes from a plugin which the build may or may not
  // use, so we can't use it here. We therefore check task names
  // instead.
  try {
    tasks.matching { task -> task.name.equals( 'compileKotlin' ) }.configureEach {
      try {
        outputs.doNotCacheIf("CodeQL analysis", {true})
      }
      catch(e) {
        println e
        println "Warning: CodeQL: Failed to disable cache for Kotlin."
      }
    }
  }
  catch(e) {
    try {
      tasks.matching { task -> task.name.equals( 'compileKotlin' ) }.all {
        try {
          outputs.cacheIf({false})
        }
        catch(e2) {
          println e2
          println "Warning: CodeQL: Fallback failed to disable cache for Kotlin."
        }
      }
    }
    catch(e2) {
      println e
      println e2
      println "Warning: CodeQL: Failed to configure cache disabled for Kotlin."
    }
  }
}

@JLLeitschuh
Copy link
Contributor Author

The downside of using the --no-build-cache solution as you've done in this pull request is that it disables the build cache on all tasks, not just the compiler tasks. The init script provided by the autobuilder seems to explicitly only target the compilation tasks, which is optimal. Especially on massive builds like the build of gradle/gradle which I worked to get CodeQL working on a few years ago.

@redsun82
Copy link
Contributor

I see, let me pull in someone who can help out here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants