274651 - Provide streaming API to JS parser (pass data in chunks to js engine)

Status: RESOLVED DUPLICATE of bug 1061886

(Core :: JavaScript Engine, enhancement)

Description

[Darin Fisher]

Provide streaming API to JS parser

Right now, it is not possible (according to shaver) to stream chunks of JS to
the JS parser.  It would probably help speed up loading of pages that reference
external JS files if we did not have to defer parsing until the entire JS file
is received.  This would probably help with memory usage too for large JS files
since currently we are required to build one large buffer to store the JS file
in (and that buffer must be unicode).

The potential consumer of this new API is nsScriptLoader.cpp

Comment 1

[Brendan Eich [:brendan]]

The JS parser is recursive descent, and that's not gonna change.  So unless you
want it to do the buffering that nsScriptLoader does currently (worst case), or
at least buffering up till the next complete top-level statement or declaration
(more code), the only way to fix this short of rewriting it to avoid recursion
is to use a thread.

Is there a way to estimate the performance gain to be had here?

/be

Comment 2

[Darin Fisher]

I didn't realize the parser was recursive descent.  That does pose an
interesting challenge.

I think the potential gain would be to mitigate the cost of parsing JS since we
could be parsing the current chunk of JS while fetching the next chunk of JS
instead of waiting to parse until after we have received the entire JS document.

Given that loading of external JS files blocks the HTML parser, solving this bug
might be a significant improvement in page load time.

Comment 3

[Christian :Biesinger (don't email me, ping me on IRC)]

*** Bug 297247 has been marked as a duplicate of this bug. ***

Comment 4

[Christian :Biesinger (don't email me, ping me on IRC)]

hm... so, the specific use case I had in mind for Bug 297247 could also be done
differently. The CSS parser is also a recursive descent parser, yet deals fine
with nsIConverterInputStream: It just asks the stream for more data when it
needs it.

however, the specific request in this bug is probably still a good thing, since
it avoids storing all the data in memory, even if possibly in several chunks.

Comment 5

[Bob Clary [:bc] (inactive)]

-> default qa

Comment 6

[Boris Zbarsky [:bzbarsky]]

Dupping forward to more recent bug with better ideas on how to do this.