1446676 - Migrate non-bootstrapped automation add-ons to bootstrapped

Status: RESOLVED FIXED

(Toolkit :: Add-ons Manager, enhancement)

Description

[Kris Maglione [:kmag]]

We need to start removing support for non-bootstrapped add-ons in order to make certain refactorings easier. The biggest remaining blocker for that is the continued use of non-restartless add-ons in automation.

Comment 1

[Kris Maglione [:kmag]]

Attachment: Bug 1446676: Part 1a - Add stub bootstrap.js script to load overlays from legacy non-restartless add-ons.

Several of our test automation add-ons are non-restartless, but only rely on
loading a single overlay into browser windows. This stub bootstrap.js script
allows us to run those extensions as bootstrapped extensions with the same
behavior as before, and with no other changes.

Review commit: https://reviewboard.mozilla.org/r/228628/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/228628/

Comment 2

[Kris Maglione [:kmag]]

Attachment: Bug 1446676: Part 1b - Convert talos overlay extensions to bootstrapped extensions.

Review commit: https://reviewboard.mozilla.org/r/228630/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/228630/

Comment 3

[Kris Maglione [:kmag]]

Attachment: Bug 1446676: Part 2 - Update non-overlay automation extensions to be bootstrapped.

In order to remove support for non-bootstrapped extensions, the remaining test
automation extensions need to be migrated to bootstrapped extensions. These
extensions all work by loading a single component, either with a
profile-after-change or command line handler. This is a straightforward
conversion of those components to bootstrap.js scripts.

Review commit: https://reviewboard.mozilla.org/r/228632/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/228632/

Comment 4

[Kris Maglione [:kmag]]

Attachment: Bug 1446676: Part 3 - Remove non-bootstrapped worker test add-on.

Non-restartless extensions are no longer supported in the wild, and none of
the remaining automation extension rely on Workers in their components.

Review commit: https://reviewboard.mozilla.org/r/228634/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/228634/

Comment 5

[Kris Maglione [:kmag]]

Comment on attachment 8959863 [details]
Bug 1446676: Part 1a - Add stub bootstrap.js script to load overlays from legacy non-restartless add-ons.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/228628/diff/1-2/

Comment 6

[Kris Maglione [:kmag]]

Comment on attachment 8959864 [details]
Bug 1446676: Part 1b - Convert talos overlay extensions to bootstrapped extensions.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/228630/diff/1-2/

Comment 7

[Kris Maglione [:kmag]]

Comment on attachment 8959865 [details]
Bug 1446676: Part 2 - Update non-overlay automation extensions to be bootstrapped.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/228632/diff/1-2/

Comment 8

[Kris Maglione [:kmag]]

Comment on attachment 8959866 [details]
Bug 1446676: Part 3 - Remove non-bootstrapped worker test add-on.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/228634/diff/1-2/

Comment 9

[Andrew Swan [:aswan]]

Comment on attachment 8959863 [details]
Bug 1446676: Part 1a - Add stub bootstrap.js script to load overlays from legacy non-restartless add-ons.

https://reviewboard.mozilla.org/r/228628/#review234728

::: testing/talos/talos/bootstrap.js:20
(Diff revision 2)
> +    if (defaultConstructor) {
> +      this.defaultConstructor = defaultConstructor;
> +    }

I don't get this, the code below assumes `this.defaultConstructor` is something callable.

Comment 10

[Andrew Swan [:aswan]]

Comment on attachment 8959864 [details]
Bug 1446676: Part 1b - Convert talos overlay extensions to bootstrapped extensions.

https://reviewboard.mozilla.org/r/228630/#review234760

::: commit-message-1f9c9:1
(Diff revision 2)
> +Bug 1446676: Part 1b - Convert talos overlay extensions to bootstrapped extensions. r?aswan

mention mochitest here too

::: testing/mochitest/bootstrap.js:10
(Diff revision 2)
> +ChromeUtils.import("resource://gre/modules/AppConstants.jsm");
> +ChromeUtils.import("resource://gre/modules/NetUtil.jsm");
>  ChromeUtils.import("resource://gre/modules/Services.jsm");
>  ChromeUtils.import("resource://gre/modules/XPCOMUtils.jsm");
>  
> +/////// Android ///////

include the "the canonical copy of this ..." comment here?

Comment 11

[Kris Maglione [:kmag]]

Comment on attachment 8959863 [details]
Bug 1446676: Part 1a - Add stub bootstrap.js script to load overlays from legacy non-restartless add-ons.

https://reviewboard.mozilla.org/r/228628/#review234728

> I don't get this, the code below assumes `this.defaultConstructor` is something callable.

This class is copied from ExtensionUtils.jsm, where sometimes defaultConstructor is provided by a sub-class.

Comment 12

[Andrew Swan [:aswan]]

Comment on attachment 8959866 [details]
Bug 1446676: Part 3 - Remove non-bootstrapped worker test add-on.

https://reviewboard.mozilla.org/r/228634/#review234774

Comment 13

[Kris Maglione [:kmag]]

Comment on attachment 8959864 [details]
Bug 1446676: Part 1b - Convert talos overlay extensions to bootstrapped extensions.

https://reviewboard.mozilla.org/r/228630/#review234760

> mention mochitest here too

Oops. Good catch. I thought I'd said something like "test harness" rather than talos.

> include the "the canonical copy of this ..." comment here?

This is actually slightly different than the talos files. It uses a different observer and waits until DOMContentLoaded to apply the overlay. The talos add-ons wait for domwindowopened and then DOMWindowCreated, which works better on desktop, but alas does not seem to work on Android.

I'd consider trying to merge the two, but try runs take over an hour to give me useful output, and it would take me the better part of a day to get a local Android development environment working again, so I don't think it's worth the effort.

Comment 14

[Dave Townsend [:mossop]]

Comment on attachment 8959865 [details]
Bug 1446676: Part 2 - Update non-overlay automation extensions to be bootstrapped.

https://reviewboard.mozilla.org/r/228632/#review234784

Comment 15

[Kris Maglione [:kmag]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/ad8214a6cf04ac2bcd0ceee09bb08b1a474c3680
Bug 1446676: Part 1a - Add stub bootstrap.js script to load overlays from legacy non-restartless add-ons. r=aswan

https://hg.mozilla.org/integration/mozilla-inbound/rev/aabfc0197692dbb5dff1303fba35eb9b3dec5582
Bug 1446676: Part 1b - Convert automation overlay extensions to bootstrapped extensions. r=aswan

https://hg.mozilla.org/integration/mozilla-inbound/rev/71d65d6b64997bc608a1cb3c924be35bbd460b39
Bug 1446676: Part 2 - Update non-overlay automation extensions to be bootstrapped. r=Mossop

https://hg.mozilla.org/integration/mozilla-inbound/rev/d6bda91c66d6625fb98c427ad62b337696df5acf
Bug 1446676: Part 3 - Remove non-bootstrapped worker test add-on. r=aswan

Comment 16

[Kris Maglione [:kmag]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/5d3eb0526d8b78eba75a3433461ce6724a59b5e3
Bug 1446676: Follow-up: Add talos add-on bootstrap.js files to xperf whitelist. r=bustage

Comment 17

[Tiberius Oros[:tiberius_oros]]

https://hg.mozilla.org/mozilla-central/rev/ad8214a6cf04
https://hg.mozilla.org/mozilla-central/rev/aabfc0197692
https://hg.mozilla.org/mozilla-central/rev/71d65d6b6499
https://hg.mozilla.org/mozilla-central/rev/d6bda91c66d6
https://hg.mozilla.org/mozilla-central/rev/5d3eb0526d8b

Comment 18

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Backed out for timeouts in talos-chrome-profiling on OS X:

https://hg.mozilla.org/mozilla-central/rev/b34dce03b8521d7c0ffa6ebe167fb2341bdeb956

Push which ran failing jobs: https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&revision=e636edf00e6fbdc3206c9df4a1548ae38b3d13fa&filter-resultStatus=testfailed&filter-resultStatus=busted&filter-resultStatus=exception&filter-resultStatus=retry&filter-resultStatus=usercancel&filter-resultStatus=runnable

Failure log: https://treeherder.mozilla.org/logviewer.html#?job_id=169427160&repo=mozilla-central

05:52:15     INFO -  TEST-INFO | started process 1700 (/Users/cltbld/tasks/task_1521633562/build/application/Firefox Nightly.app/Contents/MacOS/firefox -foreground -profile /var/folders/mc/n12pf8tx46d48285wl1j69gm00000w/T/tmp4wTGVV/profile)
05:52:15     INFO -  PID 1700 | Unable to read VR Path Registry from /Users/cltbld/Library/Application Support/OpenVR/.openvr/openvrpaths.vrpath
05:52:16     INFO -  PID 1700 | Unable to read VR Path Registry from /Users/cltbld/Library/Application Support/OpenVR/.openvr/openvrpaths.vrpath
05:52:16     INFO -  PID 1700 |
05:52:16     INFO -  PID 1700 | fnbpaint is not yet available (0), retry number 1...
05:52:16     INFO -  PID 1700 | 2018-03-21 05:52:16.834 plugin-container[1701:10353] *** CFMessagePort: bootstrap_register(): failed 1100 (0x44c) 'Permission denied', port = 0x9553, name = 'com.apple.tsm.portname'
05:52:16     INFO -  PID 1700 | See /usr/include/servers/bootstrap_defs.h for the error codes.
05:52:16     INFO -  PID 1700 | Unable to read VR Path Registry from /Users/cltbld/Library/Application Support/OpenVR/.openvr/openvrpaths.vrpath
05:52:17     INFO -  PID 1700 | 2018-03-21 05:52:17.259 plugin-container[1702:10397] *** CFMessagePort: bootstrap_register(): failed 1100 (0x44c) 'Permission denied', port = 0x9a47, name = 'com.apple.tsm.portname'
05:52:17     INFO -  PID 1700 | See /usr/include/servers/bootstrap_defs.h for the error codes.
05:52:17     INFO -  PID 1700 | received fnbpaint value
05:52:17     INFO -  PID 1700 | Cycle 1(1): loaded about:preferences (next: about:blank)
05:52:32     INFO -  PID 1700 | 2018-03-21 05:52:32.360 firefox[1700:10311] Persistent UI failed to open file file:///Users/cltbld/Library/Saved%20Application%20State/org.mozilla.nightly.savedState/window_1.data: No such file or directory (2)
[taskcluster 2018-03-21T13:10:29.838Z] Aborting task - max run time exceeded!


This also caused central-as-beta simulations to fail on Linux and Windows opt: https://treeherder.mozilla.org/#/jobs?repo=try&revision=1f35b55fcbc3bd4d511fdcc6e82ea76def0b956c&filter-resultStatus=testfailed&filter-resultStatus=busted&filter-resultStatus=exception&filter-resultStatus=retry&filter-resultStatus=usercancel&filter-resultStatus=runnable

Comment 19

[Kris Maglione [:kmag]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/c6c3e59498fb0aa83a33ce88613cff881f7e5dc9
Bug 1446676: Part 1a - Add stub bootstrap.js script to load overlays from legacy non-restartless add-ons. r=aswan

https://hg.mozilla.org/integration/mozilla-inbound/rev/c62d96da27e11e0dcbbeaa07e10495135a51d9db
Bug 1446676: Part 1b - Convert automation overlay extensions to bootstrapped extensions. r=aswan

https://hg.mozilla.org/integration/mozilla-inbound/rev/18546e2e45c42258dbedd4819b20c47edd8d9c26
Bug 1446676: Part 2 - Update non-overlay automation extensions to be bootstrapped. r=Mossop

https://hg.mozilla.org/integration/mozilla-inbound/rev/95b7ecfc7fe708eb4e720b0a68b11bf81629e774
Bug 1446676: Part 3 - Remove non-bootstrapped worker test add-on. r=aswan

Comment 20

[Arthur Iakab [arthur_iakab]]

https://hg.mozilla.org/mozilla-central/rev/c6c3e59498fb
https://hg.mozilla.org/mozilla-central/rev/c62d96da27e1
https://hg.mozilla.org/mozilla-central/rev/18546e2e45c4
https://hg.mozilla.org/mozilla-central/rev/95b7ecfc7fe7