Android mochitests and reftests all failing on Beta because of unsigned addons | remoteautomation.py | application timed out after 330 seconds with no output

RESOLVED FIXED

Status

defect
RESOLVED FIXED
3 years ago
3 years ago

People

(Reporter: KWierso, Assigned: kmoir)

Tracking

({intermittent-failure})

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment)

After the merge of 49 to beta, Android tests started failing en masse.

In the logs, I spotted these lines:
08:33:57     INFO -  08-01 08:26:42.836 I/Gecko   ( 2211): 1470065202838	addons.xpi	WARN	Refusing to install staged add-on special-powers@mozilla.org with signed state 0
08:33:57     INFO -  08-01 08:26:43.026 I/Gecko   ( 2211): 1470065203025	addons.xpi	WARN	Refusing to install staged add-on mochikit@mozilla.org with signed state 0

http://archive.mozilla.org/pub/mobile/tinderbox-builds/mozilla-beta-android-api-15/1470060053/mozilla-beta_ubuntu64_vm_armv7_large_test-mochitest-1-bm123-tests1-linux64-build1.txt.gz



Are we requiring signed addons for Fennec now?
Andy, are we requiring signed addons for mobile?
Flags: needinfo?(amckay)
Andy is on PTO this week but we have always required signed addons for all release versions, the big recent change was bug 1186522.  Prior to that change, signing could be disabled in automation, but after that change, it can not be disabled.  There was try run on the beta branch about a month ago to test that change, but is it possible that something changed with the way specialpowers gets loaded on Android since then?
Flags: needinfo?(amckay)
Beta is closed until this can get sorted out, for the record.
Regular desktop mochitests load special powers via Marionette using this code:
https://dxr.mozilla.org/mozilla-central/source/testing/marionette/client/marionette_driver/addons.py#88

The reference to "staged add-on" in the log message above suggests that Fennec uses a different technique, one that is broken by bug 1186522.  But here is an example of a try run that succeeded in the last release cycle:
https://treeherder.mozilla.org/#/jobs?repo=mozilla-beta&revision=35e0d7dd7cd7bff301b75756361b36b275102404
Since that run succeeded, that suggests that something changed with the way tests are run (or specifically, with the way test extensions are loaded) on fennec between 48 and 49?  I have to admit I'm pretty ignorant about how tests work on Fennec, not even sure who to ni? who could help...
I should have read more closely, those try runs weren't actually requiring signing on fennec and given comments on bug 1233199 I guess it is known that there is remaining work to be done before requiring signing on fennec.
Reading through various patches on bug 1186522 I got myself thoroughly confused but I think the changes to mobile/android/config/mozconfigs/common.override from https://hg.mozilla.org/releases/mozilla-beta/rev/35e0d7dd7cd7bff301b75756361b36b275102404 ought to actually be checked in everywhere?  (ie, central, aurora, and beta)
Flags: needinfo?(kmoir)
ahal did some of the initial work on moving Firefox over to use Marionette to inject addons to circumvent the addon signing tyranny.
Android addons should not be signed on any branch.  It looks like that patch that prevented this on beta on fennec was overwritten in the merge.  I'll attach a patch to fix.
Flags: needinfo?(kmoir)
I'll look at the merge script too to see what this happened on uplift.
Attachment #8776929 - Flags: review?(mshal)
Looking at the merge script, it only does replacements.  So I think we should land the previous patch on all other branches until we are ready to enable signing on Fennec, at which point we can update the merge script.
Duplicate of this bug: 1291230
Comment on attachment 8776929 [details] [diff] [review]
bug1290993.patch

It looks like build/mozconfig.common now sets MOZ_REQUIRE_SIGNING to 1 by default on beta. Is that the line that's generated by the merge script?
Attachment #8776929 - Flags: review?(mshal) → review+
Blocks: 1291228
Yes, we want that in the merge script.  The merge script doesn't set the preference on beta in 

mobile/android/config/mozconfigs/common.override to this 
MOZ_REQUIRE_SIGNING=0

which caused the test failures.  So I'll land this patch on trunk and then request permission to land on m-b and m-a. We don't have a date set for when we will enable the preference on beta for Android.
Pushed by kmoir@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/8921d7e94257
Android mochitests and reftests all failing on Beta because of unsigned addons | remoteautomation.py | application timed out after 330 seconds with no output r=mshal DONTBUILD
Comment on attachment 8776929 [details] [diff] [review]
bug1290993.patch

Approval Request Comment
[Feature/regressing bug #]:1290993
[User impact if declined]: test failures will continue on android
[Describe test coverage new/current, TreeHerder]: currently android tests are failing because the preference is enabled for addons on android, no new tests needed
[Risks and why]: if this preference is not enabled, android tests will continue to fail
[String/UUID change made/needed]:n/a
Attachment #8776929 - Flags: approval-mozilla-beta?
Comment on attachment 8776929 [details] [diff] [review]
bug1290993.patch

Approval Request Comment
[Feature/regressing bug #]:1290993
[User impact if declined]: test failures will continue on android
[Describe test coverage new/current, TreeHerder]: currently android tests are failing because the preference is enabled for addons on android, no new tests needed
[Risks and why]: if this preference is not enabled, android tests will continue to fail
[String/UUID change made/needed]:n/a
Attachment #8776929 - Flags: approval-mozilla-aurora?
See Also: → 1290919
Assignee: nobody → kmoir
Comment on attachment 8776929 [details] [diff] [review]
bug1290993.patch

uploaded to m-a and m-b since they are only test issues

Looks like the test failures are fixed on on beta
Attachment #8776929 - Flags: approval-mozilla-beta?
Attachment #8776929 - Flags: approval-mozilla-aurora?
Status: NEW → RESOLVED
Closed: 3 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.