1290993 - Android mochitests and reftests all failing on Beta because of unsigned addons | remoteautomation.py | application timed out after 330 seconds with no output

Status: RESOLVED FIXED

(Testing :: General, defect)

Description

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

After the merge of 49 to beta, Android tests started failing en masse.

In the logs, I spotted these lines:
08:33:57     INFO -  08-01 08:26:42.836 I/Gecko   ( 2211): 1470065202838	addons.xpi	WARN	Refusing to install staged add-on [e-mail] with signed state 0
08:33:57     INFO -  08-01 08:26:43.026 I/Gecko   ( 2211): 1470065203025	addons.xpi	WARN	Refusing to install staged add-on [e-mail] with signed state 0

http://archive.mozilla.org/pub/mobile/tinderbox-builds/mozilla-beta-android-api-15/1470060053/mozilla-beta_ubuntu64_vm_armv7_large_test-mochitest-1-bm123-tests1-linux64-build1.txt.gz



Are we requiring signed addons for Fennec now?

Comment 1

[Jonathan Griffin (:jgriffin)]

Andy, are we requiring signed addons for mobile?

Comment 2

[Andrew Swan [:aswan]]

Andy is on PTO this week but we have always required signed addons for all release versions, the big recent change was bug 1186522.  Prior to that change, signing could be disabled in automation, but after that change, it can not be disabled.  There was try run on the beta branch about a month ago to test that change, but is it possible that something changed with the way specialpowers gets loaded on Android since then?

Comment 3

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

Beta is closed until this can get sorted out, for the record.

Comment 4

[Andrew Swan [:aswan]]

Regular desktop mochitests load special powers via Marionette using this code:
https://dxr.mozilla.org/mozilla-central/source/testing/marionette/client/marionette_driver/addons.py#88

The reference to "staged add-on" in the log message above suggests that Fennec uses a different technique, one that is broken by bug 1186522.  But here is an example of a try run that succeeded in the last release cycle:
https://treeherder.mozilla.org/#/jobs?repo=mozilla-beta&revision=35e0d7dd7cd7bff301b75756361b36b275102404
Since that run succeeded, that suggests that something changed with the way tests are run (or specifically, with the way test extensions are loaded) on fennec between 48 and 49?  I have to admit I'm pretty ignorant about how tests work on Fennec, not even sure who to ni? who could help...

Comment 5

[Andrew Swan [:aswan]]

I should have read more closely, those try runs weren't actually requiring signing on fennec and given comments on bug 1233199 I guess it is known that there is remaining work to be done before requiring signing on fennec.
Reading through various patches on bug 1186522 I got myself thoroughly confused but I think the changes to mobile/android/config/mozconfigs/common.override from https://hg.mozilla.org/releases/mozilla-beta/rev/35e0d7dd7cd7bff301b75756361b36b275102404 ought to actually be checked in everywhere?  (ie, central, aurora, and beta)

Comment 6

[Andreas Tolfsen ❲:ato❳]

ahal did some of the initial work on moving Firefox over to use Marionette to inject addons to circumvent the addon signing tyranny.

Comment 7

[Kim Moir [:kmoir] ET]

Android addons should not be signed on any branch.  It looks like that patch that prevented this on beta on fennec was overwritten in the merge.  I'll attach a patch to fix.

Comment 8

[Kim Moir [:kmoir] ET]

Attachment: bug1290993.patch

I'll look at the merge script too to see what this happened on uplift.

Comment 9

[Kim Moir [:kmoir] ET]

Looking at the merge script, it only does replacements.  So I think we should land the previous patch on all other branches until we are ready to enable signing on Fennec, at which point we can update the merge script.

Comment 11

[Michael Shal [:mshal]]

Comment on attachment 8776929 [details] [diff] [review]
bug1290993.patch

It looks like build/mozconfig.common now sets MOZ_REQUIRE_SIGNING to 1 by default on beta. Is that the line that's generated by the merge script?

Comment 12

[Kim Moir [:kmoir] ET]

Yes, we want that in the merge script.  The merge script doesn't set the preference on beta in 

mobile/android/config/mozconfigs/common.override to this 
MOZ_REQUIRE_SIGNING=0

which caused the test failures.  So I'll land this patch on trunk and then request permission to land on m-b and m-a. We don't have a date set for when we will enable the preference on beta for Android.

Comment 13

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/8921d7e94257
Android mochitests and reftests all failing on Beta because of unsigned addons | remoteautomation.py | application timed out after 330 seconds with no output r=mshal DONTBUILD

Comment 14

[Kim Moir [:kmoir] ET]

Comment on attachment 8776929 [details] [diff] [review]
bug1290993.patch

Approval Request Comment
[Feature/regressing bug #]:1290993
[User impact if declined]: test failures will continue on android
[Describe test coverage new/current, TreeHerder]: currently android tests are failing because the preference is enabled for addons on android, no new tests needed
[Risks and why]: if this preference is not enabled, android tests will continue to fail
[String/UUID change made/needed]:n/a

Comment 15

[Kim Moir [:kmoir] ET]

Comment on attachment 8776929 [details] [diff] [review]
bug1290993.patch

Approval Request Comment
[Feature/regressing bug #]:1290993
[User impact if declined]: test failures will continue on android
[Describe test coverage new/current, TreeHerder]: currently android tests are failing because the preference is enabled for addons on android, no new tests needed
[Risks and why]: if this preference is not enabled, android tests will continue to fail
[String/UUID change made/needed]:n/a

Comment 16

[Kim Moir [:kmoir] ET]

Comment on attachment 8776929 [details] [diff] [review]
bug1290993.patch

uploaded to m-a and m-b since they are only test issues

Looks like the test failures are fixed on on beta

Comment 17

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/8921d7e94257