607313 - Progress handler and event.total wrong total file size when uploading files > 4GB

Status: RESOLVED DUPLICATE of bug 215450

(Core :: DOM: Core & HTML, defect)

Description

[Vivien Schilis]

User-Agent:       Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_4; en-us) AppleWebKit/533.18.1 (KHTML, like Gecko) Version/5.0.2 Safari/533.18.5
Build Identifier: 3.6.11 and 4.0 b6

When using the XMLHttpRequest() and binding on the progress event we discover a strange behaviour on the event.total which returns correctly the total size for small/medium size (tested: 500MB works!) But on a very big file like 4GB it correctly returns the file size at the first event callback but an incorrect file size on subsequent calls.

idea: integer overflow when saved in memory?

note: Same behaviour on Webkit browsers! (Fix it before them :D)

Reproducible: Always

Steps to Reproduce:
1. Uploading a Big file I would say > 1GB But I only tested on 4GB

Actual Results:  
progress callback

event.total = 818372608

Expected Results:  
event.total = 5113339904

Contact me if you need to reproduce it.

Comment 1

[Vivien Schilis]

Allow me to rectify: the first call doesn't return the good file size neither.

Comment 2

[Mounir Lamouri (:mounir)]

It would be nice to have a test case.

Comment 3

[Olli Pettay [:smaug][[e-mail]]]

Yeah, testcase would be great.
In which way are you uploading the file?

Comment 4

[Vivien Schilis]

Ok i'll deploy an example on heroku between today and tomorrow morning.

Comment 5

[Vivien Schilis]

What do you mean by which way? 
Input File and XMLHttpRequest
I will give more details within the example

Comment 6

[Olli Pettay [:smaug][[e-mail]]]

There are many ways to upload file data using XHR.
But sounds you use File object.

Comment 7

[Vivien Schilis]

Indeed, I use File object

Comment 8

[Boris Zbarsky [:bzbarsky]]

This is presumably the relevant code:

2488       PRUint32 uploadTotal = 0;
2489       postDataStream->Available(&uploadTotal);
2490       mUploadTotal = uploadTotal;

So anything over 4GB will get wrapped around.

I don't seem to see any existing bugs on the fact that streams' available() is 32-bit....  However I _do_ see language in the API that says that the return value must be PR_UINT32_MAX if the actual length is bigger than that.  It sounds like some stream is not doing that right, at least.

We should really fix the stream thing, though; it causes us to send and incorrect Content-Length when uploading files > 4GB too.

Comment 9

[Vivien Schilis]

Here is a simple website with upload and prints:
http://onprogress-filesize-bug.heroku.com

If you select a very large file it will print a correct file size on load and a wrong one on progress.

Comment 10

[Makoto Kato [:m_kato]]

Vivien, does this still occur on current Nightly?

nsIInputStream handles 64-bit length from 17.0, this may be fixed on Aurora/Nightly.

Comment 11

[pablobm]

I can confirm this is fixed now.

The test case above (http://onprogress-filesize-bug.heroku.com/) reproduces the bug on Firefox 4, but not on Firefox 18.

Comment 12

[Makoto Kato [:m_kato]]

(In reply to pablo from comment #11)
> I can confirm this is fixed now.
> 
> The test case above (http://onprogress-filesize-bug.heroku.com/) reproduces
> the bug on Firefox 4, but not on Firefox 18.

Thank you for verified.  dup of original fix.