680847 - FileReader doesn't clear the memory when reused, until tab is switched

Status: RESOLVED INVALID

(Core :: DOM: Core & HTML, defect)

Description

[Davit Barbakadze]

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8) AppleWebKit/535.1 (KHTML, like Gecko) Chrome/13.0.782.112 Safari/535.1

Steps to reproduce:

Consider a code like this:

<input id="file" type="file" onchange="sliceMe()" />

<script>
function sliceMe() {
    var file = document.getElementById('file').files[0], 
        fr,
        chunkSize = 2097152, 
        chunks = Math.ceil(file.size / chunkSize), 
        chunk = 0;

    function loadNext() {
       var start, end,
           blobSlice = File.prototype.mozSlice || File.prototype.webkitSlice;

       start = chunk * chunkSize;
       end = start + chunkSize >= file.size ? file.size : start + chunkSize;

       fr = new FileReader;
       fr.onload = function() {      
          if (++chunk < chunks) {
             fr = null;
             // shortcut - in production upload happens here and then loadNext() is called again
             loadNext(); 
          }
       };
       fr.readAsBinaryString(blobSlice.call(file, start, end));
    }

    loadNext();
}
</script>

It was an experiment to overcome the bug with FormData + Blob in Gecko 6- (https://bugzilla.mozilla.org/show_bug.cgi?id=649150). We thought that it still might be a good solution for chunked upload, since it didn't require whole file to be loaded in memory, only separate chunks, and in idea could give us some additional performance.



Actual results:

Memory usage quickly crawls up, as if we were loading whole file in memory, and even worse then that, comparing to the former approach, it hogs almost all the cpu time. Client-side quickly becomes unresponsive and then unusable. 


Expected results:

We expected that memory would be purged as we reuse the FileReader object, so that in general memory usage will stay on the same level on average for the whole process (that's what happens in Chrome btw). I've asked the same question on mozilla developer forums (https://developer.mozilla.org/forums/viewtopic.php?f=4&t=393), but no one seems to be visiting Open Web section on there, so... and on StackOverflow (http://stackoverflow.com/questions/7137760/is-it-possible-to-clean-memory-after-filereader), where one guy noticed that if FileReader object is set to null after each cycle and then neighbour tab is clicked in browser, memory clears in few seconds (as one would expect).

Here is the fiddle example for quick reference: http://jsfiddle.net/WK6H6/5/

Comment 1

[Kyle Huey (Exited; not receiving bugmail, old account, do not use)]

What makes you think you're reusing the FileReader object?  You're creating a new one each time you call loadNext.  You create a FileReader for each chunk, and effectively load the whole file into memory until the GC runs.  If you hoist the FileReader construction out of loadNext and into the beginning of sliceMe things behave as you expect.



That said, there is a Gecko side bug here, which is that our GC heuristics for DOM things that consume lots of memory need work.  I've filed Bug 681479 for that.

Comment 2

[Kyle Huey (Exited; not receiving bugmail, old account, do not use)]

Also, you see the memory being cleared when you switch tabs because that triggers the GC.

Comment 3

[Davit Barbakadze]

This is a great news! Thank you Kyle for bringing this to our attention. Sorry for bugging you for invalid things, hope it will still help you make things better.

Comment 4

[Norah Smith]

I'm currently trying to reuse a FileReader object with the intent of uploading chunks of a file. Reusing the object for each blob actually seems to be leaking at least as much memory as creating a new FileReader for each blob.

Additionally, reusing the FileReader object doesn't work at all in Chrome, while new creating new FileReaders works as intended. 

Anyway, do you think Bug 681479 would account for the memory issues I'm seeing with reusing the FileReader instance?

Comment 5

[Kyle Huey (Exited; not receiving bugmail, old account, do not use)]

(In reply to Norah Smith from comment #4)
> I'm currently trying to reuse a FileReader object with the intent of
> uploading chunks of a file. Reusing the object for each blob actually seems
> to be leaking at least as much memory as creating a new FileReader for each
> blob.

Can you attach or link to some example code?

> Additionally, reusing the FileReader object doesn't work at all in Chrome,
> while new creating new FileReaders works as intended. 

You should report this to the Chromium folks at http://crbug.com

> Anyway, do you think Bug 681479 would account for the memory issues I'm
> seeing with reusing the FileReader instance?

No.  That bug would account for memory issues you would see when *not* reusing the FileReader instance.

Comment 6

[Davit Barbakadze]

Norah Smith, if you proceed to upload stage, then it might be XMLHttpRequest that leaks. Are you doing something similar to this: https://bugzilla.mozilla.org/show_bug.cgi?id=681931?

Comment 7

[Norah Smith]

Hi Kyle and Davit, thanks for your replies. I'm not able to link to a workable example, as there's lots of app specific code intertwined in the upload logic, but the basic strategy is similar to the one Davit showed at the top, but with variable chunk sizes, and a reused FileReader.

And Kyle, I just took the upload logic out of the mix and just had the browser loading the file chunk by chunk and the memory usage stayed constant within about 20ish MB, so I'm sufficiently convinced FileReader isn't the source of the leak.

Davit, I'll take a look at the other bug ticket, but I think I ended up finding a workaround that doesn't involve Firefox memory usage creeping too horribly high over the course of the upload.