Closed Bug 1250616 Opened 4 years ago Closed 4 years ago

Sign mozscreenshots and indexedDB-test for mochitest

Categories

(Release Engineering :: General, defect)

defect
Not set

Tracking

(firefox46 fixed, firefox47 fixed, firefox48 fixed)

RESOLVED FIXED
Tracking Status
firefox46 --- fixed
firefox47 --- fixed
firefox48 --- fixed

People

(Reporter: andy+bugzilla, Assigned: ahal)

References

(Blocks 2 open bugs)

Details

Attachments

(3 files, 3 obsolete files)

There are some addons that helpers in the try process that look like they aren't being installed:

12:24:33 INFO - 1455740673691 addons.xpi WARN Refusing to install staged add-on workerbootstrap-test@mozilla.org with signed state 0
12:24:33 INFO - 1455740673715 addons.xpi WARN Refusing to install staged add-on worker-test@mozilla.org with signed state 0
12:24:33 INFO - 1455740673754 addons.xpi WARN Refusing to install staged add-on mozscreenshots@mozilla.org with signed state 0
12:24:33 INFO - 1455740673766 addons.xpi WARN Refusing to install staged add-on indexedDB-test@mozilla.org with signed state 0 

We should probably get those signed so that they remain functional for try users.
workerbootstrap, worker-test and indexedDB-test are all generated at build time. This means we'll need to remove pre-processing like this:
https://dxr.mozilla.org/mozilla-central/source/dom/indexedDB/test/extensions/install.rdf

Luckily the most complicated thing seems to be the allowed version which we can replace with '*'. I'll take a stab at this.
Assignee: nobody → ahalberstadt
Status: NEW → ASSIGNED
I got mozscreenshots and indexedDB-test signed and working, but workerbootstrap-test and worker-test both fail to install in their test runs.

workerbootstrap-test has a javascript exception on install:
>1456345980405	addons.xpi	WARN	Exception running bootstrap method startup on workerbootstrap-test@mozilla.org: [Exception... "Component returned failure code: 0x804b000a (NS_ERROR_MALFORMED_URI) [nsIIOService2.newURI]"  nsresult: "0x804b000a (NS_ERROR_MALFORMED_URI)"  location: "JS frame :: resource://gre/modules/addons/XPIProvider.jsm -> jar:file:///tmp/tmpFSaMex.mozrunner/extensions/workerbootstrap-test@mozilla.org.xpi!/bootstrap.js :: gWorkerAndCallback.start :: line 32"  data: no] Stack trace: gWorkerAndCallback.start()@resource://gre/modules/addons/XPIProvider.jsm -> jar:file:///tmp/tmpFSaMex.mozrunner/extensions/workerbootstrap-test@mozilla.org.xpi!/bootstrap.js:32 < startup()@resource://gre/modules/addons/XPIProvider.jsm -> jar:file:///tmp/tmpFSaMex.mozrunner/extensions/workerbootstrap-test@mozilla.org.xpi!/bootstrap.js:140 < this.XPIProvider.callBootstrapMethod()@resource://gre/modules/addons/XPIProvider.jsm:4627 < this.XPIProvider.startup()@resource://gre/modules/addons/XPIProvider.jsm:2674 < callProvider()@resource://gre/modules/AddonManager.jsm:227 < _startProvider()@resource://gre/modules/AddonManager.jsm:755 < AddonManagerInternal.startup()@resource://gre/modules/AddonManager.jsm:938 < this.AddonManagerPrivate.startup()@resource://gre/modules/AddonManager.jsm:2751 < amManager.prototype.observe()@addonManager.js:57


worker-test fails because __LOCATION__ is undefined:
https://dxr.mozilla.org/mozilla-central/source/dom/workers/test/extensions/traditional/WorkerTest.js#25

__LOCATION__ seems to be something that httpd.js injects somehow. So I guess it makes sense that it's not defined anymore, but I don't know how to work around it.

Sigh, why can't things just work for once.
See Also: → 1251372
I spun up bug 1251371 to tackle the two failing addons. I'll deal with mozscreenshots and indexedDB-test here.
Summary: Sign helper add-ons → Sign mozscreenshots and indexedDB-test for mochitest
Comment on attachment 8723194 [details]
MozReview Request: Bug 1250616 - Sign mozscreenshots extensions for mochitest, r?MattN

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/36401/diff/1-2/
Attachment #8723194 - Attachment description: MozReview Request: Bug 1250616 - Sign extraneous mochitest extensions → MozReview Request: Bug 1250616 - Sign mozscreenshots extensions for mochitest, r?MattN
Attachment #8723194 - Flags: review?(MattN+bmo)
Blocks: 1251633
Comment on attachment 8723194 [details]
MozReview Request: Bug 1250616 - Sign mozscreenshots extensions for mochitest, r?MattN

https://reviewboard.mozilla.org/r/36401/#review33495
Attachment #8723194 - Flags: review?(MattN+bmo) → review+
Whiteboard: [leave-open]
Comment on attachment 8724105 [details]
MozReview Request: Bug 1250616 - Sign indexedDB-test extension for mochitest, r?khuey

I don't review patches in mozreview, sorry.  Please attach a diff to this bug and rerequest review.
Attachment #8724105 - Flags: review?(khuey) → review-
Attachment #8724105 - Attachment is obsolete: true
Attachment #8724716 - Flags: review?(khuey)
Whiteboard: [leave-open]
Flags: needinfo?(ahalberstadt) → needinfo?(gbrown)
I can reproduce the Android failure locally. I see no errors or warnings in logcat except for the test failure. 

With extensions.logging.enabled:

I/GeckoConsole(24937): 1456956588359    addons.xpi      DEBUG   Processing install of indexedDB-test@mozilla.org in app-profile
I/GeckoConsole(24937): 1456956588370    addons.xpi      INFO    Mapping indexedDB-test@mozilla.org to /storage/sdcard/tests/profile/extensions/indexedDB-test@mozilla.org.xpi
I/GeckoConsole(24937): 1456956588509    addons.xpi      DEBUG   New add-on indexedDB-test@mozilla.org in app-profile
I/GeckoConsole(24937): 1456956588511    addons.xpi      DEBUG   getModTime: Recursive scan of indexedDB-test@mozilla.org
I/GeckoConsole(24937): 1456956588525    addons.xpi      DEBUG   getInstallState changed: true, state: {"app-profile":{"workerbootstrap-test@mozilla.org":{"d":"/storage/sdcard/tests/profile/extensions/workerbootstrap-test@mozilla.org","st":1456956587001,"mt":1456956556000},"worker-test@mozilla.org":{"d":"/storage/sdcard/tests/profile/extensions/worker-test@mozilla.org","st":1456956587000,"mt":1456956556000},"special-powers@mozilla.org":{"d":"/storage/sdcard/tests/profile/extensions/special-powers@mozilla.org","st":1456956587000,"mt":1456956556000},"roboextender@mozilla.org":{"d":"/storage/sdcard/tests/profile/extensions/roboextender@mozilla.org","st":1456956587000,"mt":1456956556000},"mochikit@mozilla.org":{"d":"/storage/sdcard/tests/profile/extensions/mochikit@mozilla.org","st":1456956588000,"mt":1456956556000},"indexedDB-test@mozilla.org":{"d":"/storage/sdcard/tests/profile/extensions/indexedDB-test@mozilla.org.xpi","st":1456956588000}}}
I/GeckoConsole(24937): 1456956589590    addons.xpi-utils        DEBUG   New add-on indexedDB-test@mozilla.org installed in app-profile
I/GeckoConsole(24937): 1456956590336    addons.manager  DEBUG   Registering startup change 'installed' for indexedDB-test@mozilla.org
I/GeckoConsole(24937): 1456956590343    addons.xpi      DEBUG   Loading bootstrap scope from /storage/sdcard/tests/profile/extensions/indexedDB-test@mozilla.org.xpi
I/GeckoConsole(24937): 1456956590370    addons.xpi      DEBUG   Calling bootstrap method install on indexedDB-test@mozilla.org version 2016.02.26
I/GeckoConsole(24937): 1456956590376    addons.xpi-utils        DEBUG   Make addon app-profile:indexedDB-test@mozilla.org visible

Without this change, additional messages are logged:

I/GeckoConsole(26891): 1456957543548    addons.xpi      DEBUG   Updating XPIState for {"id":"indexedDB-test@mozilla.org","syncGUID":"_KRsPAJUeZdR","location":"app-profile","version":"2013.10.10","type":"extension","internalName":null,"updateURL":null,"updateKey":null,"optionsURL":null,"optionsType":null,"aboutURL":null,"icons":{},"iconURL":null,"icon64URL":null,"defaultLocale":{"name":"IndexedDBTest","description":"IndexedDB functions for use in testing.","creator":"Mozilla","homepageURL":null},"visible":true,"active":true,"userDisabled":false,"appDisabled":false,"descriptor":"/storage/sdcard/tests/profile/extensions/indexedDB-test@mozilla.org","installDate":1456957541000,"updateDate":1456957541000,"applyBackgroundUpdates":1,"bootstrap":true,"skinnable":false,"size":2917,"sourceURI":null,"releaseNotesURI":null,"softDisabled":false,"foreignInstall":true,"hasBinaryComponents":false,"strictCompatibility":false,"locales":[],"targetApplications":[{"id":"toolkit@mozilla.org","minVersion":"47.0a1","maxVersion":"47.0a1"}],"targetPlatforms":[],"mult
I/GeckoConsole(26891): 1456957543549    addons.xpi      DEBUG   getModTime: Recursive scan of indexedDB-test@mozilla.org
I/GeckoConsole(26891): 1456957543773    addons.xpi      DEBUG   Registering manifest for /storage/sdcard/tests/profile/extensions/indexedDB-test@mozilla.org
I/GeckoConsole(26891): 1456957543775    addons.xpi      DEBUG   Calling bootstrap method startup on indexedDB-test@mozilla.org version 2013.10.10
Landed the mozscreenshots patch on aurora (I guess I should have spun these into separate bugs):
https://hg.mozilla.org/releases/mozilla-aurora/rev/a42a8c60245f
On Android, the IndexedDB xpi is found and processed but is disabled -- appDisabled is true. The add-on is appDisabled because isUsableAddon() returns false, and that's because it fails the compatibility check here: http://hg.mozilla.org/mozilla-central/annotate/c59c022943f6/toolkit/mozapps/extensions/internal/XPIProvider.jsm#l719. It's not compatible because matchingTargetApplication is null here: http://hg.mozilla.org/mozilla-central/annotate/c59c022943f6/toolkit/mozapps/extensions/internal/XPIProvider.jsm#l6663.

matchingTargetApplication() searches the add-on targetApplications for an entry matching Services.appinfo.ID: http://hg.mozilla.org/mozilla-central/annotate/c59c022943f6/toolkit/mozapps/extensions/internal/XPIProvider.jsm#l6714. On desktop, Services.appinfo.ID is {ec8030f7-c20a-464f-9b0e-13a3a9e97384}, which is found in the add-on targetApplications (and seen in the new install.rdf). On Android, Services.appinfo.ID is {aa3c5121-dab2-40e2-81ca-7ea25febc110}, which is not found.

:ahal -- I'm not finding any documentation for the targetApplication for Firefox for Android. Is this making sense to you?
Flags: needinfo?(gbrown) → needinfo?(ahalberstadt)
Found confirmation that Firefox for Android has its own guid: https://addons.mozilla.org/en-US/firefox/pages/appversions/
Ah yeah, that makes sense I didn't realize they were separate applications. Thanks for digging, should be an easy fix! I can probably take it from here if you want.
Flags: needinfo?(ahalberstadt)
Updated patch. Carrying r+ forward.
Attachment #8724716 - Attachment is obsolete: true
Attachment #8726899 - Flags: review+
These extensions are hardcoded in runrobocop.py and my try run neglected robocop.
Attachment #8723194 - Attachment is obsolete: true
https://reviewboard.mozilla.org/r/38721/#review35399

::: testing/mochitest/runrobocop.py:221
(Diff revision 1)
>          """

Here's a try run proving it's fixed:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=1786568df7ea
Attachment #8723194 - Attachment is obsolete: false
Comment on attachment 8727979 [details]
MozReview Request: Bug 1250616 - Update extension names in runrobocop.py, r=gbrown

https://reviewboard.mozilla.org/r/38721/#review35401
Attachment #8727979 - Flags: review?(gbrown) → review+
Backed out in https://hg.mozilla.org/integration/mozilla-inbound/rev/80c6c41be91f

Turns out that http://mxr.mozilla.org/mozilla-central/source/toolkit/mozapps/extensions/test/browser/browser_checkAddonCompatibility.js sets extensions.strictCompatibility, which means that an extension which has min and maxVersion 47.0a1 becomes incompatible six or eight pushes after you pushed it, when the 47-48 version bump was merged from m-c.
This means browser_checkAddonCompatibility.js is going to fail every single uplift. I don't know what to do about this in the short term. I'll file a new bug to carry discussion over there.
(In reply to Andrew Halberstadt [:ahal] from comment #29)
> This means browser_checkAddonCompatibility.js is going to fail every single
> uplift. I don't know what to do about this in the short term. I'll file a
> new bug to carry discussion over there.

I don't understand why browser_checkAddonCompatibility is checking the test only add-ons. I think it would be fine to have it skip those. Also you could just set the version range in the test add-ons to 0 -> * so they are always compatible regardless.
Yeah, I realized the version to * is probably the best bet after posting my last comment. I have a try run in-flight with these changes.
Changes maxVersion to *. Try run:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=a6c4371324e4

Carrying r+ forward.
Attachment #8726899 - Attachment is obsolete: true
Attachment #8728614 - Flags: review+
https://hg.mozilla.org/mozilla-central/rev/ab972bb3658f
https://hg.mozilla.org/mozilla-central/rev/1400a894db71
Status: ASSIGNED → RESOLVED
Closed: 4 years ago
Resolution: --- → FIXED
Component: Release Automation → General Automation
QA Contact: bhearsum → catlee
Component: General Automation → General
You need to log in before you can comment on or make changes to this bug.