Huge amounts of CPU time spent in memcpy when using XMLHttpRequest.responseText

NEW
Unassigned

Status

()

defect
6 years ago
5 months ago

People

(Reporter: kael, Unassigned)

Tracking

(Blocks 1 bug)

24 Branch
x86_64
Windows 7
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [games:p?][diamond], )

When using XMLHttpRequest to stream large files, .responseText accesses seem to produce tons and tons of reallocations and memcpy operations. In my profiles the memcpy traffic alone accounts for 10-13% of CPU time according to the SPS profile.

The reallocations also create a lot of GC pressure and pauses.

Things that would help here:

Avoid reallocating the responseText string's buffer repeatedly (this has other benefits, in particular reducing the consequences of the responseText leak)

Being able to stream an arraybuffer response (last time I checked this doesn't work?)

Being able to request a string or array buffer representing a subset of the response (instead of getting back a 40MB string representing an entire 40MB file)

Being able to adjust the chunk size the browser uses when notifying you about updates via onreadystatechange.


For reference, this is the library I am using to stream TAR files via XHR:
https://github.com/kevingadd/JSIL/blob/tar/Libraries/multifile.js

Let me know if you need a full test case to test against and I can upload one (it's big, though)
Whiteboard: games:p? → [games:p?]
There's a test case up that demonstrates the problem:
http://hildr.luminance.org/bugs/3/Lumberjack/Lumberjack.html?disableSound
Talk to ack -- he has some tar+IndexedDB stuff.  Also, yeah, XHR will realloc, though so will arraybuffer.  responseText is doubly awful though, because it's ucs2; why use responseText and not an arraybuffer? (you can do chunked arraybuffer)

In the future we also plan to honor content-length and preallocate a buffer if possible.
> Avoid reallocating the responseText string's buffer repeatedly

Not really doable if JS asks for the .responseText repeatedly, because it's sharing the underlying buffer, which therefore cannot be modified in-place....

> Being able to stream an arraybuffer response (last time I checked this doesn't work?)

chunked-arraybuffer should work, though it's Gecko-only last I checked.

> Being able to request a string or array buffer representing a subset of the response

Does using HTTP Range headers not work?  Note that this may not play nice with gzip encoding, though.

> In the future we also plan to honor content-length and preallocate a buffer if possible.

That won't help if the web page polls .responseText on progress, unless the JS GC is very expeditious about dropping those strings; see above.
Whiteboard: [games:p?] → [games:p3]
Whiteboard: [games:p3] → [games:p3][diamond]
Whiteboard: [games:p3][diamond] → [games:p?][diamond]
Component: DOM → DOM: Core & HTML
You need to log in before you can comment on or make changes to this bug.