1766114 - Provide ChromeUtils.import in autoconfig script

Status: RESOLVED FIXED

(Core :: AutoConfig (Mission Control Desktop), task)

Description

[Tooru Fujisawa [:arai]]

Derived from bug 1765167 comment #16 and bug 1765167 comment #17.

Cu.import is going to be deprecated, at least for in-tree browser code and test code, in bug 1765167, and ChromeUtils.import with single parameter is a replacement for it.

Then, ChromeUtils isn't available in autoconfig script.
If we enforce ChromeUtils.import also in autoconfig script, the sandbox creation code and nsXPConnect::CreateSandbox need to be modified to pass/receive option to define ChromeUtils.

Comment 1

[Tooru Fujisawa [:arai]]

Then, the purpose of switching to ChromeUtils.import is to prepare for switching to ES module (bug 1308512),
and, ChromeUtils,import with the current JSM semantics is also going to be deprecated in the long term.

So, whether and how to address this bug depends on how Cu.import is used in the AutoConfig scripts.

Comment 2

[Tooru Fujisawa [:arai]]

:mkaply, what's the policy around breaking change and AutoConfig?

• (a) is a breaking change acceptable if we document how to rewrite code snippet inside AutoConfig?
• (b) can breaking changes be scattered across multiple releases (so, people needs to rewrite AutoConfig for each release) ? or should it be done at once as much as possible (so, people need to rewrite AutoConfig only once) ?
• (c) should there be some release cycles that supports both old code and new code? (not sure if we can provide for this case tho)
• (d) also, do we have data about what modules are widely used in AutoConfig ?

Currently the following 3 breaking changes are expected:

1. the return value of Cu.import("...", {}) stops having some properties (see bug 1610653 comment #27)
2. JSM => ES module (bug 1308512)
3. Delete Cu.import and ChromeUtils.import

Step 1 can be avoided as separate change, if we delay it until step 2,
or, we could make Cu.import return a proxy that has both global properties and exported symbols.

If we do this at separate step, the AutoConfig script that uses the return value of Cu.import needs to be rewritten in the following way:

// Before :  receive global object and get property
var { BrowserWindowTracker } = Cu.import("resource:///modules/BrowserWindowTracker.jsm", {});

// After : receive "exports" object and get property, with Cu.import
var obj = {};
Cu.import("resource:///modules/BrowserWindowTracker.jsm", obj);
var { BrowserWindowTracker } = obj;

// After : receive "exports" object and get property, with ChromeUtils.import
var { BrowserWindowTracker } = ChromeUtils.import("resource:///modules/BrowserWindowTracker.jsm");

At least for now, no change is expected for Services.jsm and AboutNewTab.jsm that I saw many snippets on the web.
Also, if Cu.import is called with single parameter and return value is not used, no change is required, given exported symbols are copied to global variable.

Step 2 definitely happens, gradually for each module, and it will likely with filename change (e.g. .jsm => .mjs), with different function name (ChromeUtils.importModule("...")).
At this point, Cu.import for the module will stop working unless we add shim.

If we don't add shim, the above code needs to be rewritten to the following:

// After : receive "exports" object and get property, with ChromeUtils.importModule for .mjs
var { BrowserWindowTracker } = ChromeUtils.importModule("resource:///modules/BrowserWindowTracker.mjs");

Step 3 is optional, but there won't be in-tree consumer of them after the transition.

Comment 3

[Mike Kaply [:mkaply]]

What is your timeline for these changes?

As long as we document things really well, I don't see a problem with these changes.

I'm comfortable with making people rewrite Autoconfig stuff (we've done it before).

But I would rather see it done all it once to avoid multiple rewrites.

If you want to see common uses of Cu.import in Autoconfig, you can look at my cck2wizard repo which a lot of people look at:

https://github.com/mkaply/cck2wizard/search?q=Cu.import

or scripts I've written here:

https://mike.kaply.com/2015/02/10/installing-certificates-into-firefox/

Comment 4

[Tooru Fujisawa [:arai]]

Thank you for the response and the links! the those resources helps a lot :)

(In reply to Mike Kaply [:mkaply] from comment #3)

What is your timeline for these changes?

We can start rewriting JSM with ES module (ESM) maybe from next month, after bug 1311728 gets fixed.
Rewriting all modules at once isn't something realistic (there are 1200+ files), so it will happen gradually for each module or subtree, and the entire process will take months (maybe 3 months or more?).

So, possible option here to reduce the AutoConfig rewrite is to provide a wrapper JSM file that imports corresponding ESM file and export the same thing, until the rewrite finishes (if we're fine with having 1200+ wrapper files in-tree, or maybe less if we provide it only for widely-used JSMs).
And ask for AutoConfig rewrite once all modules are rewritten with ESM, and then remove the wrapper JSM files after that.

The change for the return value of Cu.import() can be anytime between right now and the "JSM to ESM" rewrite.
The patch is ready in bug 1610653, and if possible (no major conflict with widely used cases), I'd like to land all changes at once, in next month.
If this is not possible, the change can be delayed to the "JSM to ESM" rewrite, for each file.

Deleting Cu.import and ChromeUtils.import will be after all above.

Comment 5

[Mike Kaply [:mkaply]]

OK, the reason I asked is because the next ESR is 102, so if we could do no breaking changes before then, they would give us more time to do the work.

But we can certainly encourage folks to transition to the new method in the 102 release notes.

Comment 6

[Tooru Fujisawa [:arai]]

I see. I'll try not to let breaking changes ride the 101 and 102 trains,
and I'll think about the transition document, and wrapper JSM way.

And in this bug, exposing ChromeUtils in AutoConfig sandbox should be the right way to go.

Comment 7

[Tooru Fujisawa [:arai]]

Attachment: Bug 1766114 - Part 1: Define ChromeUtils in privideged AutoConfig sandbox. r?mkaply!

Comment 8

[Tooru Fujisawa [:arai]]

Attachment: Bug 1766114 - Part 2: Use Services.jsm in AutoConfig prefcalls.js. r?mkaply!

Depends on D144939

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/281a38d258ae
Part 1: Define ChromeUtils in privideged AutoConfig sandbox. r=mkaply
https://hg.mozilla.org/integration/autoland/rev/caadeaea532f
Part 2: Use Services.jsm in AutoConfig prefcalls.js. r=mkaply

Comment 10

[Norisz Fay [:noriszfay]]

https://hg.mozilla.org/mozilla-central/rev/281a38d258ae
https://hg.mozilla.org/mozilla-central/rev/caadeaea532f

Comment 11

[Tooru Fujisawa [:arai]]

(In reply to Mike Kaply [:mkaply] from comment #5)

OK, the reason I asked is because the next ESR is 102, so if we could do no breaking changes before then, they would give us more time to do the work.

But we can certainly encourage folks to transition to the new method in the 102 release notes.

Sorry, I misread the last sentence.

102 ESR won't be ready for the transition on the consumer side, given the transition can be started only after the in-tree transition finishes, and in-tree transition will take months.
So, it will be at least next-next ESR.

Then, I have one more question about the transition.
We're going to have some release cycles that is compatible both with current API and new API, so that both unmodified AutoConfig and updated AutoConfig works, so that users can rewrite their AutoConfig during that period.
Does ESR also need a major cycle that is compatible both with current API and new API?

for example, is (a) acceptable, or is (b) required?
(a) ESR 102 is compatible only with current way, and ESR 110 (?) is compatible only with new way
(b) ESR 102 is compatible only with current way, and ESR 110 (?) is compatible both with current and new ways, and ESR 120 (?) is compatible only with new way.

Comment 12

[Mike Kaply [:mkaply]]

a is acceptable, but I thought you made ChromeUtils.import work now?

The next ESR is much farther out (mid next year).

Or are you saying nothing is changing for 102 at all?

Comment 13

[Tooru Fujisawa [:arai]]

For 102 ESR, nothing visible will be changed.

Sorry, my question was unclear,
it's about until when we support the current way, that means when to remove the backward compatibility, for example:

• when to remove the shim that's going to be added by bug 1766761
• when to apply bug 1758481 patch that removes the deprecated feature

If (a) is acceptable, those changes can be done before 110, so, for example, the following transition plan is possible (to be clear, just an example):

• 102 - 105: rewrite in-tree code (during this cycles, "current" way is supported, and "new" way is partially supported)
• 106 - 108: rewrite not-in-tree consumer (during this cycles, both "current" and "new" ways are supported)
• 109: remove backward compatibility (after this cycle, only "new " way is supported)

and in that case, ESR will be:

• 102: only "current" way is supported
• 110: only "new" way is supported

If (b) is required, those changes need to be deferred until 111+ cycle.

Comment 14

[Mike Kaply [:mkaply]]

The next ESR will probably more like Firefox 114.

I think the answer here is that we do not need an ESR with backward compatibility.

Once these changes are made, we'll notify in our normal places that there are compatibility changes for Autoconfig.

So I think you should just proceed as normal and I'll document along the way as people need to change.

I've been meaning to restart my blog and this might be a good post to start that when we make the change.

Comment 15

[Tooru Fujisawa [:arai]]

Okay, thank you!