1837185 - web-ext run -f nightly cannot start Nightly 116 with add-ons

Status: RESOLVED FIXED

(WebExtensions :: General, defect, P1)

Description

[Rob Wu [:robwu]]

Original report at https://github.com/mozilla/web-ext/issues/2780; I confirmed that this is happening.

After starting Firefox Nightly 116 (latest version, buildid 20230607033033), I get web-ext run -f nightly

Error: Could not install add-on at '/private/tmp/shared-worker-port': [Exception... "AddonManager is not initialized" nsresult: "0xc1f30001 (NS_ERROR_NOT_INITIALIZED)" location: "JS frame :: resource://gre/modules/AddonManager.sys.mjs :: installTemporaryAddon :: line 2658" data: no]

Comment 1

[Rob Wu [:robwu]]

mozregression output:

5:21.62 INFO: Last good revision: 81afd61afe48ea4877eff969a6eda908bcf79231
5:21.62 INFO: First bad revision: 8a4b7b46101dab1ab91d388939059d21319cd034
5:21.62 INFO: Pushlog:
https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=81afd61afe48ea4877eff969a6eda908bcf79231&tochange=8a4b7b46101dab1ab91d388939059d21319cd034

So rewriting one of the imports has resulted in this bug.

Comment 2

[Rob Wu [:robwu]]

Note that the regression is associated with the commit that changed the imports from .jsm to .sys.mjs: https://hg.mozilla.org/integration/autoland/rev/8a4b7b46101dab1ab91d388939059d21319cd034
( not the preceding one that rewrote the file itself from .jsm to .sys.mjs )

Comment 3

[Rob Wu [:robwu]]

Strange, the issue goes away upon reverting the ESM to JSM import at https://searchfox.org/mozilla-central/diff/3c0c3a26765c1c4143f23dd52afa66caf749158b/devtools/server/actors/addon/addons.js#11-14

Comment 4

[Rob Wu [:robwu]]

I have manually verified that the above fixes the issue; I'm trying to reproduce the issue in a unit test, but want to already get the fix out without being blocked on the unit test.

Comment 5

[Rob Wu [:robwu]]

Attachment: Bug 1837185 - Import AddonManager.jsm instead of ESM in devtools/

Importing JSM as before, instead of ESM fixes the regression.
This effectively reverts the changes in devtools/ starting from
https://hg.mozilla.org/mozilla-central/rev/8a4b7b46101d#l43.1

This commit may be reverted again when the root cause has been
addressed, in bug 1837245.

Comment 6

[Rob Wu [:robwu]]

Attachment: Bug 1837185 - Use realistic loader in test_webext_apis.js

test_webext_apis.js was not realistic because it did not use a dedicated
loader like DevToolsStartup. That prevented the test from catching the
regression. This patch fixes that by replicating the logic at
https://searchfox.org/mozilla-central/rev/aec3a901e6f6b3041b5ec457c9111a042cef1fb1/devtools/startup/DevToolsStartup.sys.mjs#1075-1101

Comment 7

[Rob Wu [:robwu]]

Attachment: Bug 1837185 - Correct loader for extension imports in WebExtension descriptor +tests

The WebExtensionDescriptorActor loaded extension modules with the
dedicated devtools loader, which returns a different instance than
the real module in the browser. Besides the code not being designed to
be loaded twice in the same browser instance, this has the following
devtools-protocol-observable consequences:

• Uninitialized AddonManager -> Unable to detect addon updates/uninstalls.
• Detached ExtensionParent -> Unable to observe the actual extension,
  including not seeing extension state such as warnings.

Comment 8

[Rob Wu [:robwu]]

Attachment: Bug 1837185 - Correct remaining extension imports in devtools modules

Extension modules are singletons and must never be loaded through the
dedicated DevTools loader. This happens when --start-debugger-server is
used.

These files in question are primarily designed to support the Firefox
devtools, so I haven't created separate unit tests for these cases.
The kind of problem associated with this import pattern has been covered
by other patches in this bug.

Comment 9

[Rob Wu [:robwu]]

Attachment: Revert "Bug 1837185 - Import AddonManager.jsm instead of ESM in devtools/"

This reverts commit 73c9ad27476afad842324b7fbcee210c8c23c311.

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/8f0425e45173
Use realistic loader in test_webext_apis.js + fix failures r=devtools-reviewers,jdescottes
https://hg.mozilla.org/integration/autoland/rev/a8aefc39fedd
Correct loader for extension imports in WebExtension descriptor +tests r=rpl,devtools-reviewers,jdescottes
https://hg.mozilla.org/integration/autoland/rev/54b03590d32a
Correct remaining extension imports in devtools modules r=rpl,devtools-reviewers,jdescottes

Comment 12

[Cristina Horotan [:chorotan]]

https://hg.mozilla.org/mozilla-central/rev/8f0425e45173
https://hg.mozilla.org/mozilla-central/rev/a8aefc39fedd
https://hg.mozilla.org/mozilla-central/rev/54b03590d32a

Comment 13

[Rob Wu [:robwu]]

Comment on attachment 9337932 [details]
Bug 1837185 - Use realistic loader in test_webext_apis.js

Beta/Release Uplift Approval Request

• User impact if declined: Users cannot install add-ons through remote debugging / web-ext.
• Is this code covered by automated tests?: Yes
• Has the fix been verified in Nightly?: No
• Needs manual test from QE?: No
• If yes, steps to reproduce:
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): Targeted fix in devtools/, to effectively revert to the behavior before the regression without side effects. Problem is well-understood and covered by automated tests. I have also verified the behavior manually before landing the patch.
• String changes made/needed: none
• Is Android affected?: Unknown

Comment 14

[Donal Meehan [:dmeehan]]

Comment on attachment 9337932 [details]
Bug 1837185 - Use realistic loader in test_webext_apis.js

Approved for 115.0b4.

Comment 15

[Donal Meehan [:dmeehan]]

Comment on attachment 9337994 [details]
Bug 1837185 - Correct loader for extension imports in WebExtension descriptor +tests

Approved for 115.0b4.

Comment 16

[Donal Meehan [:dmeehan]]

Comment on attachment 9337995 [details]
Bug 1837185 - Correct remaining extension imports in devtools modules

Approved for 115.0b4.

Comment 17

[Donal Meehan [:dmeehan]]

https://hg.mozilla.org/releases/mozilla-beta/rev/10aa949f3b16
https://hg.mozilla.org/releases/mozilla-beta/rev/378711852278
https://hg.mozilla.org/releases/mozilla-beta/rev/2f27f8b0f598