1374489 - [e10s] Very poor large file upload performance in mega.nz

Status: RESOLVED WORKSFORME

(Core :: Networking, defect, P1)

Description

[Mez Rep]

User Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:55.0) Gecko/20100101 Firefox/55.0
Build ID: 20170615070049

Steps to reproduce:

With multiprocess enabled, visit https://mega.nz and try to upload a large file (for example a 30GB file on a machine with 16GB of RAM)


Actual results:

Tab stalls/freezes, system reports terabytes of file I/O


Expected results:

Upload should work fine, just like it does with e10s disabled

Comment 1

[Dragana Damjanovic [:dragana]]

Jason does anyone has time to take a look?

It works properly on non-e10s!

Comment 2

[Jason Duell]

Daniel, can you have a look at this?

Comment 3

[--]

Do you experience this "freeze" at a specific file size or after a certain time or something? Or put another way: does it work for you in any circumstances and if so which?

I've tried to upload several different things using Firefox Nightly 56 on both Windows (64 bit) and Linux as well as Firefox 54 release (64 bit) on Windows (all e10s enabled) and I cannot see any speed degradation or problems with them - even if I get fairly slow upload speeds (1.2 - 2.4 MB/s on a 100mbit uplink).

The site does uploads with XHR by sending the upload as a large number of 1MB chunks

Comment 4

[Mez Rep]

It works for me for small files, e.g. a 100 MB file is OK. For larger files, depends on how long you wait.

I just tried a 1280414115-byte file. Transfer gets stuck at "Initializing" and the content process does 77 GB of I/O reads (as reported by the OS) (this is ~64x the size of the file). This is fast (~1 minute?) since it is served from cache. Then the transfer actually starts and proceeds fine. For larger files that don't fit on RAM, I didn't wait long enough (my guess is that it would eventually work, but the machine is unusable in the meanwhile).

The same file without e10s causes less than 5 MB of IO before the transfer starts (and it starts more or less immediately, as opposed to taking a minute).

This is on a clean profile with the build in the first post.

Comment 5

[Firefox Bug Husbandry Bot]

Bulk priority update: https://bugzilla.mozilla.org/show_bug.cgi?id=1399258

Comment 6

[Jason Duell]

Mezrep,

Sorry for delay here.

Our underlying architecture has changed here--we're now sandboxing child processes from doing I/O (instead the parent should do it for them).   Could you try your scenario again with Firefox nightly?

Comment 7

[Jason Duell]

Keeping this at P1, because if we have systematic issues uploading large data we should fix ASAP.

Comment 8

[Valentin Gosu [:valentin]]

Trying to reproduce.

Comment 9

[Mez Rep]

Hello,

I can confirm this does not happen in Nightly anymore.

I just tested (to make sure this was not workarounded on Mega's end):

55-series Nightly singleprocess: works
55-series Nightly multiprocess: broken (stall at the beginning)
57-series Nightly multiprocess: seems to be working correctly (I did not try to wait for the upload to finish, but at least it started correctly)

I think this bug is now fixed; thank you.

Comment 10

[Valentin Gosu [:valentin]]

Indeed, I tested this a few days ago - upload speed was on par with or over Chrome and Edge. (1.1 Mbps vs 600 & 500 Kbps)
There were also no issues with the size - I managed to upload 15Gb on a computer with 4Gb of RAM.