843720 - Organize scripts by module instead of URL

Status: NEW

(DevTools :: Debugger, enhancement, P5)

Description

[James Burke [:jrburke]]

It is common for performance to combine multiple files into one file. However those combined files are hard to debug. One mechanism to do so is to use //@ sourceURL, but this requires the combined code to be stringified and then evaled.

This can be awkward to use because:

* Some Content Security Policy (CSP) uses forbid eval. This is comes up directly for certain types of Web Apps.
* It causes an awkward interaction with source maps, makes it harder to get minified code.

With the coming use of modules in ECMAScript, and the use of AMD modules today, it is possible to use those module boundaries instead. This has a few benefits:

* No eval in play.
* as ES modules get adopted, there will be modules that are made up of tinier, scoped interior modules. Being able to report to the library author the module ID that had the problem will give better targeting of issues.
* //@ sourceURL relies on a path. With modules that use module IDs, paths become less important, and in some cases confusing in the case of bundled modules distributed and used in other projects: those paths will likely not match the application paths.

While ES modules are not done yet, they seem to have settled on using string IDs, like AMD and CommonJS/node use. In the ES case, a file containing many modules will have boundaries like this:

module 'some/thing' {
  // module code in here
}

For AMD, it looks like this:

define('some/thing', [optional array], function () {
  // module code in here
});

The developer tool debugger could look for these boundaries then use the module ID in the list of scripts ("some/thing" in the above example). Error line numbers would be reported relative to the module wrapper, with the first line number being the first line inside the module wrapper.

While ES modules are not complete yet, this can be put into use today, and work out any wrinkles by targeting AMD modules, as they are in widespread use.

I will likely need some pointers, but I am willing to write test cases if it helps.

Comment 1

[James Burke [:jrburke]]

Also created a similar ticket for webkit inspector here:
https://bugs.webkit.org/show_bug.cgi?id=110493

Comment 2

[Eddy Bruel [:ejpbruel]]

(In reply to James Burke [:jrburke] from comment #0)
> It is common for performance to combine multiple files into one file.
> However those combined files are hard to debug. One mechanism to do so is to
> use //@ sourceURL, but this requires the combined code to be stringified and
> then evaled.
> 
> This can be awkward to use because:
> 
> * Some Content Security Policy (CSP) uses forbid eval. This is comes up
> directly for certain types of Web Apps.
> * It causes an awkward interaction with source maps, makes it harder to get
> minified code.
> 
> With the coming use of modules in ECMAScript, and the use of AMD modules
> today, it is possible to use those module boundaries instead. This has a few
> benefits:
> 
> * No eval in play.
> * as ES modules get adopted, there will be modules that are made up of
> tinier, scoped interior modules. Being able to report to the library author
> the module ID that had the problem will give better targeting of issues.
> * //@ sourceURL relies on a path. With modules that use module IDs, paths
> become less important, and in some cases confusing in the case of bundled
> modules distributed and used in other projects: those paths will likely not
> match the application paths.
> 
> While ES modules are not done yet, they seem to have settled on using string
> IDs, like AMD and CommonJS/node use. In the ES case, a file containing many
> modules will have boundaries like this:
> 
> module 'some/thing' {
>   // module code in here
> }
> 
> For AMD, it looks like this:
> 
> define('some/thing', [optional array], function () {
>   // module code in here
> });
> 
> The developer tool debugger could look for these boundaries then use the
> module ID in the list of scripts ("some/thing" in the above example). Error
> line numbers would be reported relative to the module wrapper, with the
> first line number being the first line inside the module wrapper.
> 
> While ES modules are not complete yet, this can be put into use today, and
> work out any wrinkles by targeting AMD modules, as they are in widespread
> use.
> 
> I will likely need some pointers, but I am willing to write test cases if it
> helps.

I like the idea, but this is unlikely to become a priority for us until ES modules become a reality. Leaving this bug open for future reference.

Comment 3

[Nick Fitzgerald [:fitzgen] [⏰PST; UTC-8]]

Note that the latest ES6 spec only allows for one module per file, so this bug would only be for third party module systems. This makes me unsure we should ever implement it. A better solution is to use source maps.

Comment 4

[James Burke [:jrburke]]

The current state of the ES6 spec is designed that way, but I do not believe it has had enough real world implementation and is betting on too many other things to take the place of file concatenation, where I do not think they will satisfy the use cases. I wrote up some background on that here:

https://github.com/jrburke/module/blob/master/docs/inlining.md

As for using source maps, for fxos apps, the size of the deliverable is constrained enough, at least for gaia apps, that we would not deliver the source map files in the bundles due to taking up more space on the device.

All that said, feel free to close this if you prefer a bug triage system that is targeted on things that have a clearer, more immediate implementation pathways.