1260101 - Android tests get retried indefinitely due to unsigned Fennec apk

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Alin Selagea [:aselagea]]

Noticed lots of retries for the following push: https://treeherder.mozilla.org/#/jobs?repo=try&revision=b54a0369ffcb

It looks like all those jobs are being retried over and over again as they fail to install the fennec apk:

 00:14:03     INFO - >> Install Fennec APK: Attempt #1 of 3
 1467 
 00:14:03     INFO - Running timeout 300 /builds/slave/test/build/android-sdk18/platform-tools/adb -s emulator-5554 install -r /builds/slave/test/build/fennec-48.0a1.en-US.android-arm.apk
 1468 
 2391 KB/s (36200998 bytes in 14.779s)
 1469 
 00:14:23     INFO - pkg: /data/local/tmp/fennec-48.0a1.en-US.android-arm.apk
 1470 
 00:14:23     INFO - Failure [INSTALL_PARSE_FAILED_NO_CERTIFICATES]
 1471 
 00:14:23     INFO - Sleeping 30 seconds
 1472 
 00:14:53     INFO - >> Install Fennec APK: Attempt #2 of 3
 1473 
 00:14:53     INFO - Running timeout 300 /builds/slave/test/build/android-sdk18/platform-tools/adb -s emulator-5554 install -r /builds/slave/test/build/fennec-48.0a1.en-US.android-arm.apk
 1474 
 2362 KB/s (36200998 bytes in 14.965s)
 1475 
 00:15:12     INFO - pkg: /data/local/tmp/fennec-48.0a1.en-US.android-arm.apk
 1476 
 00:15:12     INFO - Failure [INSTALL_PARSE_FAILED_NO_CERTIFICATES]
 1477 
 00:15:12     INFO - Sleeping 30 seconds
 1478 
 00:15:42     INFO - >> Install Fennec APK: Attempt #3 of 3
 1479 
 00:15:42     INFO - Running timeout 300 /builds/slave/test/build/android-sdk18/platform-tools/adb -s emulator-5554 install -r /builds/slave/test/build/fennec-48.0a1.en-US.android-arm.apk
 1480 
 2355 KB/s (36200998 bytes in 15.008s)
 1481 
 00:16:01     INFO - pkg: /data/local/tmp/fennec-48.0a1.en-US.android-arm.apk
 1482 
 00:16:01     INFO - Failure [INSTALL_PARSE_FAILED_NO_CERTIFICATES]
 1483 
 00:16:01    FATAL - INFRA-ERROR: Failed to install /builds/slave/test/build/fennec-48.0a1.en-US.android-arm.apk on test-1
 1484 
 00:16:01    FATAL - Running post_fatal callback...
 1485 
 00:16:01    FATAL - Exiting -1

Comment 1

[Alin Selagea [:aselagea]]

At a first look, it seems that the jar is unsigned:

C:\Program Files\Java\jdk1.8.0_72\bin>jarsigner -verify -verbose -certs fennec-48.0a1.en-US.android-arm.apk

  s = signature was verified
  m = entry is listed in manifest
  k = at least one certificate was found in keystore
  i = at least one certificate was found in identity scope

no manifest.
jar is unsigned. (signatures missing or not parsable)

This is responsible for the backlog age alerts from #buildduty.

Comment 2

[Alin Selagea [:aselagea]]

@Nick: I cancelled the jobs from this push for the reason mentioned above. I will continue investigating why they kept being retried instead of simply failing on their own.

Also, the more recent apks look fine and the jobs don't get re-triggered.

Comment 3

[Nick Alexander :nalexander [he/him]]

(In reply to Alin Selagea [:aselagea][:buildduty] from comment #2)
> @Nick: I cancelled the jobs from this push for the reason mentioned above. I
> will continue investigating why they kept being retried instead of simply
> failing on their own.
> 
> Also, the more recent apks look fine and the jobs don't get re-triggered.

Thanks.  I tried to cancel everything, but the Tree Herder interface is unusable so I gave up.

Comment 4

[Chris AtLee [:catlee]]

Fixed by the move to TC.