1281900 - [meta] Improve the JNI preprocessing build experience

Status: RESOLVED FIXED

(Firefox Build System :: Android Studio and Gradle Integration, defect, P5)

Description

[Nick Alexander :nalexander [he/him]]

Full disclosure: I don't really understand how this works and how it could be improved -- I just know it's pretty bad.

If I understand correctly, the C++ code built during the compile tier requires some generated code and headers to build.  We check that generated code into the tree and have a manual process to update it [1].  The generated code is produced by build/annotationProcessors, although this is a white lie -- it's not actually a Java Annotation Processor, it's just a Java program that we run.  That program processes the *compiled* Java JAR files; that is, it depends on having built the Java in mobile/android/base to complete.

To improve this, we might:

1) build the Java code before the compiled C++ code, potentially by adding a java tier, alternately by compiling Java during the compile tier and adding some cross directory dependencies.

2) generate the source and header files automatically each build, without checking them into the tree.

3) profit.

This is a fair amount of work.  In the glorious future, we might make the annotation processing code a real Java Annotation Processor so that it fits into the Gradle build system smoothly, in which case we'd really want to build the Java first.

One reason we might not want to do this: building the Java stuff is slow (1.5-2 minutes on my laptop) and largely serial.  Anything that delays getting to the long and highly-parallel compile tier is bad news.

Any other suggestions for improving this?

[1] See https://dxr.mozilla.org/mozilla-central/source/mobile/android/base/Makefile.in?q=path%3Amobile%2Fandroid%2Fbase%2FMakefile.in&redirect_type=single#561 and https://dxr.mozilla.org/mozilla-central/source/mobile/android/base/Makefile.in?q=path%3Amobile%2Fandroid%2Fbase%2FMakefile.in&redirect_type=single#516.

Comment 1

[Nick Alexander :nalexander [he/him]]

Making this a meta ticket to track multiple pieces.

Comment 2

[Firefox Bug Husbandry Bot]

Re-triaging per https://bugzilla.mozilla.org/show_bug.cgi?id=1473195

Needinfo :susheel if you think this bug should be re-triaged.

Comment 3

[Nick Alexander :nalexander [he/him]]

At this point I think we've done what we're likely to do in this space. We now build the Java code as part of the export tier, which is serial, slow, and reliable; the Java generates JNI wrappers; and the compile tiers consume them. Unless we invest in an incremental annotation processor or similar I think we've reached the local optimum.