1317880 - Building Adjust sdk broken on beta 51, keeping mozilla-beta closed

Status: RESOLVED FIXED

(Firefox Build System :: Android Studio and Gradle Integration, defect, P1)

Description

[Phil Ringnalda (:philor)]

Last thing keeping beta closed and keeping us from being able to ship the first beta of 51.

Once bug 1317359 made it possible to actually build Adjust, we now have https://treeherder.mozilla.org/logviewer.html#?job_id=1922227&repo=mozilla-beta

 02:57:31     INFO -  Exception in thread "main" java.lang.NoClassDefFoundError: com/adjust/sdk/OnAttributionChangedListener
 02:57:31     INFO -  	at java.lang.ClassLoader.defineClass1(Native Method)
 02:57:31     INFO -  	at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
 02:57:31     INFO -  	at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
 02:57:31     INFO -  	at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
 02:57:31     INFO -  	at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
 02:57:31     INFO -  	at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
 02:57:31     INFO -  	at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
 02:57:31     INFO -  	at java.security.AccessController.doPrivileged(Native Method)
 02:57:31     INFO -  	at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
 02:57:31     INFO -  	at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
 02:57:31     INFO -  	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
 02:57:31     INFO -  	at java.lang.ClassLoader.loadClass(ClassLoader.java:412)
 02:57:31     INFO -  	at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
 02:57:31     INFO -  	at org.mozilla.gecko.annotationProcessors.classloader.JarClassIterator.fillLookAheadIfPossible(JarClassIterator.java:51)
 02:57:31     INFO -  	at org.mozilla.gecko.annotationProcessors.classloader.JarClassIterator.hasNext(JarClassIterator.java:27)
 02:57:31     INFO -  	at org.mozilla.gecko.annotationProcessors.AnnotationProcessor.main(AnnotationProcessor.java:84)
 02:57:31     INFO -  Caused by: java.lang.ClassNotFoundException: com.adjust.sdk.OnAttributionChangedListener
 02:57:31     INFO -  	at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
 02:57:31     INFO -  	at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
 02:57:31     INFO -  	at java.security.AccessController.doPrivileged(Native Method)
 02:57:31     INFO -  	at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
 02:57:31     INFO -  	at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
 02:57:31     INFO -  	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
 02:57:31     INFO -  	at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
 02:57:31     INFO -  	... 16 more
 02:57:32     INFO -  gmake[5]: *** [GeneratedJNIWrappers.cpp] Error 1

Comment 1

[Phil Ringnalda (:philor)]

[Tracking Requested - why for this release]:

Because it's making releasing it impossible.

Comment 2

[Liz Henry (:lizzard) (relman/hg->git project)]

Sebastian can you help find someone to work on this? It is blocking 51 beta 1 mobile.

Comment 3

[Liz Henry (:lizzard) (relman/hg->git project)]

Brad, maybe you can suggest someone to investigate/fix this. I'm not sure who to needinfo to.

Comment 4

[Brad Lassey [:blassey] (use needinfo?)]

Sebastian is the right person to ask

Comment 5

[Sebastian Kaspari (:sebastian; :pocmo)]

It looks like we do not include the Adjust code at all - which depends on MOZ_INSTALL_TRACKING being set at this point:
https://dxr.mozilla.org/mozilla-central/source/mobile/android/base/moz.build#1007-1044

I'll have a look, but I'll also flag Chris who might already know what's going on here.

Comment 6

[Sebastian Kaspari (:sebastian; :pocmo)]

Andrzej is actually already working on fixing this.

Comment 7

[Andrzej Hunt :ahunt]

Attachment: Part 1: fix annotation processing by including appropriate jars at each stage

Comment 8

[Andrzej Hunt :ahunt]

Attachment: Part 2: reduce field wastage to not exceed dex field limit

Comment 9

[Andrzej Hunt :ahunt]

These two patches allow me to build beta locally, and are intended for applying directly to beta. 

Part 1 contains build changes that will probably need to be landed on central/aurora too before the next merge day, I don't know if the patch I've uploaded would even apply to central/aurora (there's a lot of work going on in the geckoview area, which I'm not entirely up to date on).

Part 2 already exists on nightly/aurora (52/53) as Bug 1306021.

We should probably look into producing builds with adjust enabled on central/aurora to prevent this issue from hitting us again. Sebastian suggested running builds with the beta/release configuration on central/aurora, which would allow us to verify that we don't break anything that might only be enabled with the release config. (Adjust is however the most significant difference, and adds a large chunk of code that is only built on beta/release.)

Comment 10

[Andrzej Hunt :ahunt]

Attachment: Bug 1317880 - Move adjust and thirdparty jars into GECKOVIEW_JARS to fix annotation processing

GECKOVIEW_JARS contains two Adjust classes (within constants.jar). These depend on the rest of adjust,
which was previously within FENNEC_JARS. Without the remaining adjust jars being on the classpath during
annotation processing for GECKOVIEW_JARS, we are unable to correctly process those Adjust classes
(i.e. we get a NoClassDefFoundError).

The minimal fix is to process adjust as part of GECKOVIEW_JARS. Because adjust depends on HttpClientLib,
we also need to move the relevant jars into GECKOVIEW_JARS too (sync-thirparty contains HttpClientLib).

Finally, the javaadons jar was completely missing from the classpath used for annotation processing,
we need to add that here.

This will probably require further untangling, this is a minimal patch to allow beta to actually build.

Review commit: https://reviewboard.mozilla.org/r/93388/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/93388/

Comment 11

[Andrzej Hunt :ahunt]

Comment on attachment 8811179 [details]
Bug 1317880 - Move adjust and thirdparty jars into GECKOVIEW_JARS to fix annotation processing

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/93388/diff/1-2/

Comment 12

[Sebastian Kaspari (:sebastian; :pocmo)]

Comment on attachment 8811179 [details]
Bug 1317880 - Move adjust and thirdparty jars into GECKOVIEW_JARS to fix annotation processing

https://reviewboard.mozilla.org/r/93388/#review93422

Comment 13

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/6b7c1986d7ec
Move adjust and thirdparty jars into GECKOVIEW_JARS to fix annotation processing r=sebastian

Comment 14

[Andrzej Hunt :ahunt]

Comment on attachment 8811179 [details]
Bug 1317880 - Move adjust and thirdparty jars into GECKOVIEW_JARS to fix annotation processing

Approval Request Comment
[Feature/regressing bug #]: Probably Bug 1291375 
[User impact if declined]: can't build beta (== can't build with adjust enabled)
[Describe test coverage new/current, TreeHerder]: manual local builds of both m-c and beta (beta with adjust enabled, m-c without).
[Risks and why]: Medium risk, build system changes that involve moving jars around between GeckoView and Fennec.
[String/UUID change made/needed]: none.

Comment 15

[Gerry Chang [:gchang]]

Comment on attachment 8811179 [details]
Bug 1317880 - Move adjust and thirdparty jars into GECKOVIEW_JARS to fix annotation processing

Fix a blocker for building fennec 51 beta. Beta51+ and Aurora52+.

Comment 16

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/6b7c1986d7ec

Comment 17

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/releases/mozilla-beta/rev/d84659c9d929

Comment 18

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/releases/mozilla-aurora/rev/f52c628bce82