487948 - Web Worker Threads allocate too much memory, causing GC stuttering.

Status: RESOLVED WORKSFORME

(Core :: JavaScript Engine, defect)

Description

[Hans Schmucker]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2a1pre) Gecko/20090411 Firefox/3.0
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2a1pre) Gecko/20090411

Can't really give much info here besides that a script was ported straight from a normal inline one to one using workers. Almost nothing was changed, but the WebWorkers version allocates an additional 100MB or so per second, driving the GC crazy. You can see the normal version at http://www.tapper-ware.net/devel/js/js.metatunnel/ . The webworkers version simply adds an additional parameter to render only part of the image (in this case 4 threads are created and each renders 1/4 of the image) and encodes the output into characters using charCodeFrom (an earlier version used arrays and JSON, the result is the same) and finally sends it back to the main script. Almost all variables are global to avoid as much GC activity as possible.


Reproducible: Always

Steps to Reproduce:
1. Open the supplied link
2. Open the taskmanager
Actual Results:  
The memory consumption rises by about 100MB per second and then gets GCed.

Expected Results:  
It should function exactly like the non worker version.

Comment 1

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

Can you attach the web workers version here?

Comment 2

[Hans Schmucker]

Attachment: Worker version, main document

Comment 3

[Hans Schmucker]

Attachment: Worker version, worker script.

Comment 4

[Ben Turner (not reading bugmail, use the needinfo flag!)]

So I do see the increased memory usage (going from about 57MB idle to a max of 130MB) but it gets collected rather quickly. Profiling this I see that the choppiness isn't actually due to GC per se, but to bug 454435. Each worker thread is spending ~60-70% of its time in RefillDoubleFreeList... :(

Comment 5

[Hans Schmucker]

To be honest, I barely understand a word that was said in the other bug, as the JS mozilla classes are usually as low-level as I get to work, maybe you can put me on the right track: The way I understand it, the messages are not actually the problem (at least not in this case, as the messages are relatively small), but the variables used in the worker, as the worker is constantly saved and recreated in order to put it onto whichever OS thread is currently available... is that about right? That would mean that any worker that needs a lot of memory is going to end up with the same behaviour I get... which is hardly acceptable.

Comment 6

[Mike Beltzner [:beltzner, not reading bugmail]]

As per the discussion in bug 454435 comment 49 and onwards, doesn't look like this will block. Stuart: feel free to make your case there (or here) by renominating.

Comment 7

[Ben Turner (not reading bugmail, use the needinfo flag!)]

Pushing this to JSeng, maybe should be dup'd to bug 454435.

Comment 8

[Igor Bukanov]

(In reply to comment #7)
> Pushing this to JSeng, maybe should be dup'd to bug 454435.

The changes from the bug 454435 does not affect how much memory is allocated on a particular thread. As far as I understand the issue is that the workers do not run the GC itself. So if the main thread does not execute the JS code for a while, it would mean that a lot of garbage will be accumulated.

Comment 9

[Ben Turner (not reading bugmail, use the needinfo flag!)]

The worker test case here looks great with bug 649537. I think it will resolve this bug to everyone's satisfaction.