1291519 - mach gradle build fails on first run with "java.util.zip.ZipException: error in opening zip file" (succeeds on second run)

Status: RESOLVED FIXED

(Firefox Build System :: Android Studio and Gradle Integration, defect)

Description

[Andrzej Hunt :ahunt]

Recently I've started getting the following error when building using gradle from a clean build:

I.e. fresh pull from fx-team, do
./mach clobber && ./mach build && ./mach package && ./mach gradle build

Rerunning |mach gradle build| again succeeds without issues.

(I think I've seen the same error in Android Studio too, when not calling |mach gradle build| myself)

--- mozconfig ---
ac_add_options --enable-application=mobile/android
ac_add_options --target=arm-linux-androideabi

ac_add_options --with-android-sdk="/long/path/to/my/own/sdk"

mk_add_options MOZ_OBJDIR=./objdir-frontend AUTOCLOBBER=1
ac_add_options --enable-artifact-builds


--- Build log below ---
:app:transformClassesWithDexForLocalOldDebug FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':app:transformClassesWithDexForLocalOldDebug'.
> com.android.build.api.transform.TransformException: java.util.zip.ZipException: error in opening zip file

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

Total time: 1 mins 31.408 secs

Comment 1

[Jan Henning [:JanH]]

I'm also seeing this when rebuilding, possibly in conjunction with a changed build configuration when switching between different patch series.

Comment 3

[Brian J Murray]

The root cause appears to be related to Bug 1367768. Specifically, dexOptions.jumboMode=true for general app builds, but for the local product flavor, dexOptions.multiDexEnabled=true. These changes are incompatible, so the Android build tools clean out the intermediate jar files between tasks. However, a bug in Android build tool 2.1.3 means that the intermediates are not cleaned out.

I attached a debugger to the gradle daemon and caught the exception. It appears that there are some leftover dexed jars in obj-arm-linux-androideabi/gradle/build/mobile/android/app/intermediates/pre-dexed/localOldAustralis/debug


com.android.support-animated-vector-drawable-23.4.0_b269defe3352798be883bde397e3b6669f7919e2.jar:        Dalvik dex file version 035
com.android.support-appcompat-v7-23.4.0_77e7f29a01616cebea434c18c088b55ef50ff69f.jar:                    Dalvik dex file version 035
com.android.support-cardview-v7-23.4.0_43f282517feeb9d28ac4c8b94d2a34e9f127863d.jar:                     Dalvik dex file version 035
com.android.support-customtabs-23.4.0_639261833cd72587afe856acb90e32ab4fb02356.jar:                      Dalvik dex file version 035
com.android.support-design-23.4.0_877677515f538748765e0862389bc8c443dfbba4.jar:                          Dalvik dex file version 035
com.android.support-mediarouter-v7-23.4.0_f18d0d7c3ccbc35724d96a0d5996e2e48d58af82.jar:                  Dalvik dex file version 035
com.android.support-palette-v7-23.4.0_0333c3e8653dff5a967534008d594dfadf635297.jar:                      Dalvik dex file version 035
com.android.support-recyclerview-v7-23.4.0_6c7240e519900fdb3cfbbb9644aeefb07627ea25.jar:                 Dalvik dex file version 035
com.android.support-support-v4-23.4.0_81a682a7eb734f3eb85708a3d8ac601f94ee7b77.jar:                      Dalvik dex file version 035
com.android.support-support-vector-drawable-23.4.0_4c2a8cd4b33481da457ded2cfba888ee79563b7a.jar:         Dalvik dex file version 035
com.google.android.gms-play-services-base-8.4.0_7f7c2994cc4c062101ee143ac885fb23e198865b.jar:            Dalvik dex file version 035
com.google.android.gms-play-services-basement-8.4.0_e97add2a40fb8c90e751ac2cbddf08c4b31b8daa.jar:        Dalvik dex file version 035
com.google.android.gms-play-services-cast-8.4.0_551aed44ae7ec3195e516a511a750bf014b41f78.jar:            Dalvik dex file version 035
com.google.android.gms-play-services-gcm-8.4.0_bb9e35a04631188524aff07821ea759f72b34290.jar:             Dalvik dex file version 035
com.google.android.gms-play-services-measurement-8.4.0_3c225e6a8606623c3763186357bf2bbf168ed6ea.jar:     Dalvik dex file version 035
com.squareup.leakcanary-leakcanary-android-no-op-1.4-beta1_4aab84f3e6a7954d93d98609d633a2baf91391cb.jar: Java archive data (JAR)
debug_7c7dea4be73a66a54e7e43051f40831eb87a4a6d.jar:                                                      Java archive data (JAR)
internal_impl-23.4.0_5a2feee756911041ee95c6fa684d506023a7fc72.jar:                                       Dalvik dex file version 035
internal_impl-23.4.0_8dbd309271922135afa6debca4dcb17baf459fc4.jar:                                       Dalvik dex file version 035
mozilla-central-geckoview-unspecified_a336edcb753f67de94e9736751ad055c8752dd54.jar:                      Dalvik dex file version 035
mozilla-central-thirdparty-unspecified_863f4f028441e37568f42d8bdbdf871b4a60fdda.jar:                     Dalvik dex file version 035
support-annotations-23.4.0_c0f21bc5fbc449ba086e6f9db5aa217a5ca8d9bb.jar:                                 Dalvik dex file version 035


The root cause appears to be related to a bug in the upstream Android build tools:
https://issuetracker.google.com/issues/37135363

Comment 4

[Brian J Murray]

I was able to fix the issue by cherrypicking this patch into gradle build tools 2.1.3:
https://android.googlesource.com/platform/tools/base/+/f8769cfcd51b45390c3a8bfd6e5455aa6fe9d48e

Unfortunately, all version of Android build tools with this patch break the build for other reasons.

Comment 5

[Brian J Murray]

I was incorrect in comment 3. This bug existed before the Bug 1367768, but the addition of jumbdexing changes the race condition window.

Comment 6

[Brian J Murray]

Attachment: Bug 1291519 Disable multidex for local Fennec gradle app build

Review commit: https://reviewboard.mozilla.org/r/153444/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/153444/

Comment 7

[Nick Alexander :nalexander [he/him] (PTO August 22-September 12)]

Comment on attachment 8882348 [details]
Bug 1291519 Disable multidex for local Fennec gradle app build

https://reviewboard.mozilla.org/r/153444/#review159360

Brian -- sorry for the delayed reply and thanks for the detailed investigation.

I'm going to take this, since it's `local`-only.  I'm going to flag Max Liu as well, since he will be running build system point on the development team that will first see any problems this causes.

Could you confirm that the long-term solution is to upgrade the Android-Gradle build plugin past 2.3, as tracked by https://bugzilla.mozilla.org/show_bug.cgi?id=1366644?

Comment 8

[Nick Alexander :nalexander [he/him] (PTO August 22-September 12)]

maliu: I just want you to be aware of this, in case it bites you (or other active Fennec developers).

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/30091b890dda
Disable multidex for local Fennec gradle app build r=nalexander

Comment 10

[Phil Ringnalda (:philor)]

https://hg.mozilla.org/mozilla-central/rev/30091b890dda

Comment 11

[Brian J Murray]

The long term solution is to upgrade to Gradle 2.3.

Comment 12

[Max Liu [:maliu]]

(In reply to Nick Alexander :nalexander from comment #8)
> maliu: I just want you to be aware of this, in case it bites you (or other
> active Fennec developers).

Thanks a lot, Nick. Hope we can also upgrade gradle plugin to 2.3 or above when our release apk is built from gradle.