Closed Bug 1247687 Opened 7 years ago Closed 21 days ago

Implement worker modules

Categories

(Core :: DOM: Workers, task)

task

Tracking

()

RESOLVED FIXED
111 Branch
Tracking Status
firefox111 --- fixed

People

(Reporter: jonco, Assigned: yulia)

References

(Depends on 1 open bug, Blocks 16 open bugs)

Details

(Keywords: dev-doc-needed, DevAdvocacy)

Attachments

(15 files, 22 obsolete files)

48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
Allow workers to execute module scripts too.

Proposed spec: https://github.com/whatwg/html/pull/608
That would be really useful
See Also: → 1473463
Keywords: DevAdvocacy

Does this actually need JS engine changes?

Flags: needinfo?(jcoppeard)

No, all the work is in the embedding.

Component: JavaScript Engine → DOM: Workers
Flags: needinfo?(jcoppeard)

Chromium will send an intent to ship ES modules for Shared Workers.
(crbug: https://bugs.chromium.org/p/chromium/issues/detail?id=824646)

We are considering FireFox is positive for supporting this feature since the specification discussion (https://github.com/whatwg/html/pull/608#issuecomment-183003447).
If there is misunderstanding or any updates, please let us know.

As part of overhauling the scriptloader to support modules we'd also like to try and reduce code duplication between the worker scriptloader and the main-thread window scriptloader. Much of the complexity in the worker scriptloader relates to 2 things:

  1. Channels can't be opened on worker threads. Bug 1528285 will address this so we want to wait on that.
  2. Loading ServiceWorker scripts from Cache API storage does not look like using a channel. For this, a tentative plan to investigate is to create a dedicated channel type for loading scripts from the Cache API. This could help normalize things from the script loader's perspective.
    • This may or may not also dovetail with the need for there to be a Worker equivalent to DocumentChannel for ServiceWorker and SharedWorker cases where we need to know the COEP header value before we select the process to spin up the worker.
Depends on: omt-networking
Depends on: 1311726
See Also: → 1572644

A quick update on this ticket would be appreciated!

The missing JS module support still affects the neo.mjs framework dev mode.
This one now runs fine in Chrome as well as Safari Tech Preview, while FF can only use the dist versions.

I got invited into the working draft podcast this evening to talk about workers.
So far I can only say: "No clue if and when this will happen for FF", which feels bad.

Thx for your input!
Tobias

I am currently working on the dependency (https://bugzilla.mozilla.org/show_bug.cgi?id=1311726), so this work should follow shortly. The prep work is pretty tricky. It will happen, can't give an absolute time estimate right now.

Assignee: nobody → ystartsev

Hi Yulia, thanks for the fast response! Very much appreciated.

As a dev, I would never ask other devs for absolute time estimates :)
It sounds great, that this ticket is finally getting picked up.

In case you need some rather complex testing scenarios, feel free to use the dev version of the neo online examples:
https://neomjs.github.io/pages/node_modules/neo.mjs/dist/production/apps/website/index.html#mainview=examples&childview=devmode

Best regards,
Tobias

Yulia: thanks for the last update on this! May I ask if "this work should follow shortly" is still accurate?
https://bugzilla.mozilla.org/show_bug.cgi?id=1311726 also references import maps, and I'd understand if that's more important.

I want to encourage/contribute to bundlers to improve their worker support by treating new Worker(new URL("./file.js"), import.meta.url)) similar to a dynamic import in their dependency graph. Library authors like me currently have to work around a lot of issues, but module workers would solve a lot of them.
(I don't want to gripe too much, but I'm speaking from a lot of experience. I've spent literally hundreds of hours finding hacks so that I can provide web worker libraries that are compatible across common JS environments: https://github.com/lgarron/web-worker-compat-problems)

Unfortunately, I think it will hard to get bundlers to do something until Firefox at least experimentally supports module workers, because it's very hard to transpile ESM to something that works in classic workers. It's really valuable if module workers are available in the stable versions of all modern browsers, and Firefox will soon be the only one without support: https://caniuse.com/?search=module%20worker
For example, Snowpack explicitly states:

ESM syntax (import/export) in Web Workers is still not supported in all modern browsers. Snowpack v3.0.0 and the Snowpack Webpack v5 plugin will both support automatic bundling once released.
https://www.snowpack.dev/guides/web-worker/

Is there something that contributors can do to help, if we don't have a lot of knowledge of the Firefox codebase?

I've started work on a contribution to esbuild (for
https://github.com/evanw/esbuild/issues/312) to automatically handle new Worker(new URL("./file.js", import.meta.url)) but this will definitely rely on module worker support.

It seems iOS 15 is shipping in 4 days, at which point Firefox will unfortunately be the only modern browser that requires projects to maintain workarounds for classic modules. 😭

Could I ask again whether we can still hope for Firefox to support module workers in the near future? 🥺

Bug 1311726 seems active, which is currently a blocker of this feature.

Attached file WIP: Bug 1247687 - WIP (obsolete) —

Depends on D134052

Blocks: 1350370

Big Ups for this magnificent enhancement. Hope its going well.

Attachment #9255754 - Attachment description: WIP: Bug 1247687 - (WIP) Update WebIDL to accept module workers → WIP: Bug 1247687 - Update WebIDL to accept module workers
Blocks: 1771082
Blocks: 1775574
Depends on: 1778678
Attachment #9278186 - Attachment description: WIP: Bug 1247687 - Implement Module Loading for Workers → WIP: Bug 1247687 - Implement Static Module Loading for Workers

Comment on attachment 9278178 [details]
WIP: Bug 1247687 - Implement IsTopLevel to handle cases where we have a child module

Revision D147319 was moved to bug 1784457. Setting attachment 9278178 [details] to obsolete.

Attachment #9278178 - Attachment is obsolete: true

Comment on attachment 9278182 [details]
WIP: Bug 1247687 - Make ModuleLoaderBase thread-insensitive

Revision D147323 was moved to bug 1784477. Setting attachment 9278182 [details] to obsolete.

Attachment #9278182 - Attachment is obsolete: true

Comment on attachment 9278175 [details]
WIP: Bug 1247687 - Move mClientInfo and mController to the WorkerScriptLoader constructor

Revision D147316 was moved to bug 1784476. Setting attachment 9278175 [details] to obsolete.

Attachment #9278175 - Attachment is obsolete: true

Comment on attachment 9278176 [details]
WIP: Bug 1247687 - Make ClientInfo persistant for main scripts on WorkerScriptLoader

Revision D147317 was moved to bug 1784476. Setting attachment 9278176 [details] to obsolete.

Attachment #9278176 - Attachment is obsolete: true

Comment on attachment 9278177 [details]
WIP: Bug 1247687 - Move creation of ScriptLoadRequests into own functions

Revision D147318 was moved to bug 1784482. Setting attachment 9278177 [details] to obsolete.

Attachment #9278177 - Attachment is obsolete: true

Comment on attachment 9278181 [details]
WIP: Bug 1247687 - Move shutdown operation to be always after ProcessRequests

Revision D147322 was moved to bug 1784482. Setting attachment 9278181 [details] to obsolete.

Attachment #9278181 - Attachment is obsolete: true

Comment on attachment 9278184 [details]
WIP: Bug 1247687 - Implement corrected base URI for module redirection

Revision D147325 was moved to bug 1784482. Setting attachment 9278184 [details] to obsolete.

Attachment #9278184 - Attachment is obsolete: true

Comment on attachment 9278180 [details]
WIP: Bug 1247687 - Implement single script loading for main script

Revision D147321 was moved to bug 1784482. Setting attachment 9278180 [details] to obsolete.

Attachment #9278180 - Attachment is obsolete: true

Comment on attachment 9278172 [details]
WIP: Bug 1247687 - Implement ScriptLoaderInterface Skeleton in WorkerScriptLoader

Revision D147313 was moved to bug 1784481. Setting attachment 9278172 [details] to obsolete.

Attachment #9278172 - Attachment is obsolete: true

Comment on attachment 9278173 [details]
WIP: Bug 1247687 - Use ScriptLoaderInterface method for reporting errors

Revision D147314 was moved to bug 1784481. Setting attachment 9278173 [details] to obsolete.

Attachment #9278173 - Attachment is obsolete: true

Comment on attachment 9278174 [details]
WIP: Bug 1247687 - Use ScriptLoaderInterface method for filling compile options

Revision D147315 was moved to bug 1784481. Setting attachment 9278174 [details] to obsolete.

Attachment #9278174 - Attachment is obsolete: true
Attachment #9255755 - Attachment is obsolete: true
Attachment #9255753 - Attachment is obsolete: true
Attachment #9255754 - Attachment description: WIP: Bug 1247687 - Update WebIDL to accept module workers → Bug 1247687 - Update WebIDL to accept module workers; r=asuth
Attachment #9278183 - Attachment description: WIP: Bug 1247687 - Add WorkerModuleLoader skeleton → Bug 1247687 - Add WorkerModuleLoader skeleton; r=jonco
Attachment #9278185 - Attachment description: WIP: Bug 1247687 - Implement InitModuleLoader methods for WorkerScriptLoader and WorkerGlobalScope → Bug 1247687 - Implement InitModuleLoader methods for WorkerScriptLoader and WorkerGlobalScope; r=jonco
Attachment #9278186 - Attachment description: WIP: Bug 1247687 - Implement Static Module Loading for Workers → Bug 1247687 - Implement initial Static Module Loading for Workers; r=jonco

There is a follow on that needs to implement the csp correctly, and also update the wpt tests. I'm
running those now. I will likely flag workers off initially, so that I can land the csp for workers
separately, unless this isn't seen as a major blocker.

Depends on D155566

This is a slightly annoying thing that can happen. When we abruptly cancel (such as an infinitely
running script being forcibly terminated) we will be in a state where the EvaluateModule call will
finish after the loader is destroyed. So, instead we track if there has been a forcible
cancelation, and exit early.

Depends on D155567

Attachment #9278188 - Attachment description: WIP: Bug 1247687 - Disallow ImportScripts from module workers → Bug 1247687 - Disallow ImportScripts from module workers; r=jonco
Attachment #9278187 - Attachment is obsolete: true
Attachment #9291733 - Attachment is obsolete: true
Attachment #9291729 - Attachment description: WIP: Bug 1247687 - Use referrer to set secFlags for modules → WIP: Bug 1247687 - Use requestCredentials to set secFlags for modules
Attachment #9291730 - Attachment description: WIP: Bug 1247687 - Initial wpt-tests → WIP: Bug 1247687 - Initial wpt-tests; r=jonco,asuth
Attachment #9291731 - Attachment description: WIP: Bug 1247687 - (WIP) Show failing csp tests → Bug 1247687 - enable CSP WPT tests for Static Module Workers; r=evilpie,asuth
Attachment #9291729 - Attachment description: WIP: Bug 1247687 - Use requestCredentials to set secFlags for modules → WIP: Bug 1247687 - Use requestCredentials to set secFlags for modules; r=jonco
Attachment #9291730 - Attachment description: WIP: Bug 1247687 - Initial wpt-tests; r=jonco,asuth → Bug 1247687 - Initial wpt-tests; r=jonco,asuth
Attachment #9291729 - Attachment description: WIP: Bug 1247687 - Use requestCredentials to set secFlags for modules; r=jonco → Bug 1247687 - Use requestCredentials to set secFlags for modules; r=jonco
Blocks: 1792824
Blocks: 1792825
Attachment #9292501 - Attachment description: Bug 1247687 - Fix csp for Module Workers; r=evilpie,asuth → Bug 1247687 - Implement csp for Module Workers; r=evilpie,asuth
Severity: normal → S3
Attachment #9291731 - Attachment is obsolete: true
Blocks: 1798554
Attachment #9300631 - Attachment is obsolete: true
Attached file WIP: Bug 1247687 - wip service workers (obsolete) —

Depends on D156103

Attachment #9305626 - Attachment is obsolete: true
Blocks: 1805676
No longer depends on: 1805676
Pushed by ystartsev@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/85b827971a48
Call EnsureAndLoadStringBundle from Worker ScriptLoaderRunnable; r=allstarschh
https://hg.mozilla.org/integration/autoland/rev/34680059b9f0
Update WebIDL to accept module workers; r=asuth
https://hg.mozilla.org/integration/autoland/rev/4b0a4fcc6894
Add WorkerModuleLoader skeleton; r=jonco
https://hg.mozilla.org/integration/autoland/rev/6c129bd72b61
Implement InitModuleLoader methods for WorkerScriptLoader and WorkerGlobalScope; r=jonco
https://hg.mozilla.org/integration/autoland/rev/6ee318df6641
Disallow ImportScripts from module workers; r=jonco
https://hg.mozilla.org/integration/autoland/rev/9276c7e1ddd9
Implement initial Static Module Loading for Workers; r=jonco
https://hg.mozilla.org/integration/autoland/rev/4dbd510fb042
Implement correct referrer for modules; r=jonco
https://hg.mozilla.org/integration/autoland/rev/e7b103c79b1a
Implement requestCredentials for Module Workers; r=jonco
https://hg.mozilla.org/integration/autoland/rev/0c9650a1ac48
Use requestCredentials to set secFlags for modules; r=jonco
https://hg.mozilla.org/integration/autoland/rev/40b8abaf1c0b
Handle cancellation of long running modules; r=jonco
https://hg.mozilla.org/integration/autoland/rev/354711cf113a
Initial wpt-tests; r=jonco,asuth
https://hg.mozilla.org/integration/autoland/rev/55f13fb4ee3f
Implement csp for Module Workers; r=evilpie,asuth,rpl,ckerschb
https://hg.mozilla.org/integration/autoland/rev/2429599729cb
Copy over csp related web-platform test files and metadata; r=asuth
https://hg.mozilla.org/integration/autoland/rev/5d05ab0c7cde
Update wpt code to enable worker static-import templates to set CSP; r=asuth
https://hg.mozilla.org/integration/autoland/rev/c6c5750cf713
Update mozilla wpt meta files with new expectations; r=asuth
https://hg.mozilla.org/integration/autoland/rev/721f612fd09f
1247687, 1247687, 1247687, 1247687, 1247687, 1247687, 1247687, 1247687, 1247687, 1247687, 1247687, 1247687, 1247687, 1247687: apply code formatting via Lando
Created web-platform-tests PR https://github.com/web-platform-tests/wpt/pull/37501 for changes under testing/web-platform/tests

Comment on attachment 9292502 [details]
WIP: Bug 1247687 - Implement modules for shared workers

Revision D156103 was moved to bug 1805676. Setting attachment 9292502 [details] to obsolete.

Attachment #9292502 - Attachment is obsolete: true

Comment on attachment 9308022 [details]
WIP: Bug 1247687 - Update WPT tests for shared module workers

Revision D164573 was moved to bug 1805676. Setting attachment 9308022 [details] to obsolete.

Attachment #9308022 - Attachment is obsolete: true

Comment on attachment 9304751 [details]
WIP: Bug 1247687 - wip service workers

Revision D162744 was moved to bug 1360870. Setting attachment 9304751 [details] to obsolete.

Attachment #9304751 - Attachment is obsolete: true
Regressions: 1805884

Backed out for frequent string bundle related crashes with PDF viewer (bug 1806064)

Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Target Milestone: 110 Branch → ---
Upstream PR was closed without merging
Pushed by ystartsev@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/0ab366c6eaaf
Update WebIDL to accept module workers; r=asuth
https://hg.mozilla.org/integration/autoland/rev/c94a9dc60dff
Add WorkerModuleLoader skeleton; r=jonco
https://hg.mozilla.org/integration/autoland/rev/68b476066248
Implement InitModuleLoader methods for WorkerScriptLoader and WorkerGlobalScope; r=jonco
https://hg.mozilla.org/integration/autoland/rev/7770ec4717fd
Disallow ImportScripts from module workers; r=jonco
https://hg.mozilla.org/integration/autoland/rev/0ae1fd421d4f
Implement initial Static Module Loading for Workers; r=jonco
https://hg.mozilla.org/integration/autoland/rev/49f18430c465
Implement correct referrer for modules; r=jonco
https://hg.mozilla.org/integration/autoland/rev/59207e959b7c
Implement requestCredentials for Module Workers; r=jonco
https://hg.mozilla.org/integration/autoland/rev/45de9ffeec19
Use requestCredentials to set secFlags for modules; r=jonco
https://hg.mozilla.org/integration/autoland/rev/9153182c650d
Handle cancellation of long running modules; r=jonco
https://hg.mozilla.org/integration/autoland/rev/31888ffde37a
Initial wpt-tests; r=jonco,asuth
https://hg.mozilla.org/integration/autoland/rev/1700d5b79273
Implement csp for Module Workers; r=evilpie,asuth,rpl,ckerschb
https://hg.mozilla.org/integration/autoland/rev/0bc871906e97
Copy over csp related web-platform test files and metadata; r=asuth
https://hg.mozilla.org/integration/autoland/rev/2adf67f910e8
Update wpt code to enable worker static-import templates to set CSP; r=asuth
https://hg.mozilla.org/integration/autoland/rev/d698041e5174
Update mozilla wpt meta files with new expectations; r=asuth
https://hg.mozilla.org/integration/autoland/rev/81d052cabf84
1247687, 1247687, 1247687, 1247687, 1247687, 1247687, 1247687, 1247687, 1247687, 1247687, 1247687, 1247687, 1247687: apply code formatting via Lando
Regressions: 1806738
Flags: needinfo?(ystartsev)
Upstream PR was closed without merging
Pushed by ystartsev@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/87b6a9494bfb
Update WebIDL to accept module workers; r=asuth
https://hg.mozilla.org/integration/autoland/rev/ebeb65b42212
Add WorkerModuleLoader skeleton; r=jonco
https://hg.mozilla.org/integration/autoland/rev/997e9ca2e1e9
Implement InitModuleLoader methods for WorkerScriptLoader and WorkerGlobalScope; r=jonco
https://hg.mozilla.org/integration/autoland/rev/15eb9daca661
Disallow ImportScripts from module workers; r=jonco
https://hg.mozilla.org/integration/autoland/rev/474668f6615e
Implement initial Static Module Loading for Workers; r=jonco
https://hg.mozilla.org/integration/autoland/rev/7327e6438343
Implement correct referrer for modules; r=jonco
https://hg.mozilla.org/integration/autoland/rev/f3204b9ec0dc
Implement requestCredentials for Module Workers; r=jonco
https://hg.mozilla.org/integration/autoland/rev/c585aea29f0c
Use requestCredentials to set secFlags for modules; r=jonco
https://hg.mozilla.org/integration/autoland/rev/a1900e8b640c
Handle cancellation of long running modules; r=jonco
https://hg.mozilla.org/integration/autoland/rev/78addb6af24d
Initial wpt-tests; r=jonco,asuth
https://hg.mozilla.org/integration/autoland/rev/98a751f6f41d
Implement csp for Module Workers; r=evilpie,asuth,rpl,ckerschb
https://hg.mozilla.org/integration/autoland/rev/9e97be53fb49
Copy over csp related web-platform test files and metadata; r=asuth
https://hg.mozilla.org/integration/autoland/rev/060c0649c2f5
Update wpt code to enable worker static-import templates to set CSP; r=asuth
https://hg.mozilla.org/integration/autoland/rev/cb4aa6f21ff7
Update mozilla wpt meta files with new expectations; r=asuth
https://hg.mozilla.org/integration/autoland/rev/ad56233b442a
1247687, 1247687, 1247687, 1247687, 1247687, 1247687, 1247687, 1247687, 1247687, 1247687, 1247687, 1247687, 1247687: apply code formatting via Lando
Attachment #9278179 - Attachment is obsolete: true
Flags: needinfo?(ystartsev)
You need to log in before you can comment on or make changes to this bug.