1954747 - Investigate and update the default gradle daemon heap size for local builds

Status: RESOLVED FIXED

(Firefox Build System :: Android Studio and Gradle Integration, enhancement, P2)

Comment 1

[Nick Alexander :nalexander [he/him]]

The jvmArgs can be specified on the command line, like https://stackoverflow.com/a/67530759. Alternately, we could override the root setting for automation in the dotgradle-* files.

Comment 2

[Sylvestre Ledru [:Sylvestre]]

Serge, Mike, would it be possible to change the default value ?
(or to adjust the value in mach to the memory available on the system)

Comment 3

[[:sergesanspaille]]

The default value is already set in https://searchfox.org/mozilla-central/source/gradle.properties#10 . It would be nice to have more information about the build that fails if we are to update the default value. Aaditya?

Comment 4

[Alex Hochheiden [:ahochheiden] [he/him]]

I think I have a good handle on this. We spawn too many daemons because of the different sets of jvmargs:
eg: https://searchfox.org/mozilla-central/source/gradle.properties#10
vs: https://searchfox.org/mozilla-central/source/mobile/android/fenix/gradle.properties#13

The jvmargs aren't the same, so the daemons are 'incompatible' and we get a max heap for each, of 7gb and 8gb, instead of just 8gb.

The other things is the default we set is high to specifically fix some CI issue (not sure on the specifics/bug), but as a result local builds now also have the high value set.

Per: https://github.com/gradle/gradle/issues/19501

When adding jvm args by adding them to both the gradle.properties file and through the GRADLE_OPTS env variable, the GRADLE_OPTS env variable wins and overwrites jvm args added in gradle properties.

So, all we need to do is do an override for CI via the GRADLE_OPTS environment variable with the current sizes, and reduce the rest to whatever is reasonable (I don't know what this number is, but I'll do some testing). I think we can just delete the org.gradle.jvmargs= settings in mobile/android/fenix/gradle.properties and mobile/android/android-components/gradle.properties and only keep the one in the top level gradle.properties and only have one source of truth that the rest will inherit.

I assume the only entry point for CI is ./mach gradle, but I need to verify that. If yes, it should be easy: Some logic to check MOZ_AUTOMATION and manipulate GRADLE_OPTS here.

Comment 5

[Nick Alexander :nalexander [he/him]]

Alex's diagnosis is correct, but the original sin here is that we do a bunch of things in different Gradle top-level projects. We want one Gradle project (rooted at $topsrcdir) and then all of the actions, including those in Android Studio, should re-use the same Gradle daemon. That is: let's not spend any time tweaking the daemon parameters until we've removed the scourge of multiple Gradle root directories; once that's gone, we can tune the parameters.

Comment 6

[Alex Hochheiden [:ahochheiden] [he/him]]

Making all projects use the same org.gradle.jvmargs has resolved the “Starting a Gradle Daemon, 2 incompatible Daemons could not be reused” warning, and my tests show reduced memory usage compared to before.

Current JVM arguments:

org.gradle.jvmargs=-Xmx7g -Xms2g -XX:MaxMetaspaceSize=6g -XX:+HeapDumpOnOutOfMemoryError -XX:+UseParallelGC

The top-level had gradle.properties had -Xmx8g, but I went with -Xmx7g since that's what the rest had. We can revisit these values if needed.

Per Nick’s recommendation, I’ve held off on adding the GRADLE_OPTS override for CI. It’s straightforward to introduce later if we decide it’s necessary.

Comment 7

[Alex Hochheiden [:ahochheiden] [he/him]]

Attachment: Bug 1954747 - Standardize `jvmargs` across projects to share Gradle daemon r?nalexander

This changes makes all projects use identical org.gradle.jvmargs, eliminating
the Starting a Gradle Daemon, 2 incompatible Daemons could not be reusedwarning.

By sharing a single daemon with consistent JVM settings, we greatly reduce the
build’s overall memory footprint.

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/0e276c68a1ae
Standardize `jvmargs` across projects to share Gradle daemon r=nalexander,android-reviewers

Comment 9

[Norisz Fay [:noriszfay]]

https://hg.mozilla.org/mozilla-central/rev/0e276c68a1ae