1584898 - Large downloads from MEGA freezes the entire browser

Status: VERIFIED FIXED

(Core :: DOM: File, defect, P2)

Description

[sheelgautam995]

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:71.0) Gecko/20100101 Firefox/71.0

Steps to reproduce:

So, I went to mega.nz and downloaded a big .zip file of around 1 GB in size.

Actual results:

Towards the end of download (after 99%) the entire browser froze and began to run like a slideshow. It became responsive when the file got saved into the Downloads folder. It happens only with large file sizes (1 GB or greater)

Expected results:

The browser should not become unresponsive. Chrome and Safari download just fine from mega.nz

Comment 1

[sheelgautam995]

My computer is a 2014 Non Retina Macbook Air. It has 4GB RAM.

Comment 2

[sheelgautam995]

Attachment: about:support

Comment 3

[Liviu Seplecan]

Hi @sheelgautam995, tested the issue in Mac Book Pro Mojave 8GB 10_14_6 - the download occurs with no issue. Please test if the issue is reproducible in safe mode, here is a link that can help you:
https://support.mozilla.org/en-US/kb/troubleshoot-firefox-issues-using-safe-mode
Further, I will add a component, if ins't the right one please fell free to change it.
Regards,
Liviu

Comment 4

[sheelgautam995]

Attachment: Recording.mp4

The browser crashes when downloading from mega.nz in safe mode.
So, I downloaded a file in normal mode while making a recording.
As you can see, the browser freezes towards the end of the download.
Also, QuickTime crashed midway during the recording. I couldn't complete the recording.

Comment 5

[sheelgautam995]

Is Firefox saving the download in memory instead of the disk ?

Comment 6

[:Gijs (he/him)]

(In reply to sheelgautam995 from comment #5)

Is Firefox saving the download in memory instead of the disk ?

It doesn't normally, no.

Are there any disk permissions / disk types that differ between the target directory where you're saving the file and your temporary folders?

Could you try using the profiler ( https://profiler.firefox.com/ ) to get a performance profile of the end of the download? If you start profiling right before the freeze happens and finish/capture the profile when Firefox returns to normal, then use "Share..." to get a link that you can share here, we can have a look at what is making the process slow on your machine - off-hand it's hard to be sure what the issue is...

Comment 7

[sheelgautam995]

Here is the profile. I hope I did it right.
https://perfht.ml/31qJgpk
Sometimes Mega.nz says "Firefox has an insufficient buffer to decrypt data in the browser". IDK what that means.

Comment 8

[:Gijs (he/him)]

Reporter: The profile is interesting, thank you. There's some bits missing at the beginning (possibly because the profiling buffer became full) which means I'm not sure how you ended up in this state / what happened that tripped the bad behaviour - can you provide more (step by step) details about how to reproduce this? Is the encryption perhaps something you turn on within mega.nz or something like that?

There's definitely some weird jankiness going on, and it looks like mega.nz uses blobs in a weird way. :baku, looks like we're janking marshaling (large?) blobs between processes, can you take a look at the profile in comment #7 and clarify if there's enough here for you to investigate/improve things?

Comment 9

[:Gijs (he/him)]

Tentatively moving this to Core: DOM: File because the profile has scripts coming from blob:https://mega.nz/{some-guid} as well as lots of jank from PChildToParentStream code - feel free to move along to IPC or another more appropriate component.

Comment 10

[sheelgautam995]

Here is another profile which I think is clearer than the last one.
https://perfht.ml/2IZvmnS
I got this profile with fresh boot of Firefox with only single tab open (i.e. mega.nz). I started profiling when about thirty seconds were left for the download to be finished.

The stutter starts to happen when the file is about to be saved to the disk.
I don't observe stutter with smaller file sizes.

I did not turn on any special features on mega.nz. I think mega.nz has some inbuilt encryption feature. I you download from mega.nz, you will notice that it doesn't download straight away. I think it first downloads the file in some temporary location and then decrypts it (this is purely a hunch) and then transfers it to the Downloads folder.

During the previous profiling, I was browsing even when it began to stutter. This time, I did not touch the computer during the profiling.

Comment 11

[sheelgautam995]

When I download in safe mode from mega.nz. I sometime get this crash. I filed a separate bug for the crash. Take a look.
https://bugzilla.mozilla.org/show_bug.cgi?id=1586255

Comment 12

[sheelgautam995]

Also, I think the first profile got messed up probably because I ran it several times before the actual profiling.I had not idea it would mess up the result.

Comment 13

[Andrea Marchesini [:baku]]

Am I right to say that the majority of the time is taken by nsPipe to release data?
Would be possible to call the 'free()' on a separate thread?

Comment 14

[(no longer active)]

(In reply to Andrea Marchesini [:baku] from comment #13)

Am I right to say that the majority of the time is taken by nsPipe to release data?

Yes, exactly. This is the input stream data of course (1GB buffer). This is freed from here

Would be possible to call the 'free()' on a separate thread?

It would be an improvement... But it won't be a complete solution, since for example we'll probably be holding the allocator lock during some of this time, so if the main thread tries to (de-)allocate some memory while this 1GB buffer is being deallocated in the background thread that would stall.

A better question to ask ourselves is why do we need to keep 1GB of data in memory while downloading a file... (I'm using the peak memory usage of 1,019MB in the profile to draw the conclusion that the entire downloaded file is stored in the parent process' memory while the download is in progress, up to 5 seconds afterwards too.) Can we occasionally flush this buffer or something to that effect?

Comment 15

[Andrea Marchesini [:baku]]

A better question to ask ourselves is why do we need to keep 1GB of data in memory while downloading a file... (I'm using the peak memory usage of 1,019MB in the profile to draw the conclusion that the entire downloaded file is stored in the parent process' memory while the download is in progress, up to 5 seconds afterwards too.) Can we occasionally flush this buffer or something to that effect?

This is how mega works: It creates a huge blob object in memory. Then, it decrypts it, and then it allows the user to download it.
Talking about those 5 secs, I would like to get rid of them, but we don't have a nice way to keep blobs alive during the loading and all the async steps. Let's keep this issue separate.

Comment 16

[Andrea Marchesini [:baku]]

Attachment: Bug 1584898 - Free nsPipe memory on a separate thread, r?ehsan

Comment 17

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/d68244d4b2dc
Free nsPipe memory on a separate thread, r=Ehsan

Comment 18

[Stefan Hindli [:stefan_hindli]]

https://hg.mozilla.org/mozilla-central/rev/d68244d4b2dc

Comment 19

[Maria Berlinger [:mberlinger], Services QA]

Hello,
I've tried to reproduce this issue using Firefox 71.0a1 (2019-09-29) but without success.
I've followed the str from comment 1 but Firefox doesn't became unresponsive.

sheelgautam995, could you please confirm that this is fixed using Firefox 72.0.2? (https://archive.mozilla.org/pub/firefox/candidates/72.0.2-candidates/build1/)

Comment 20

[sheelgautam995]

This seems to be fixed. I am using stable 73.0 right now. MEGA.nz seems to be a poorly designed service because its slow on pretty much every browser.

Comment 21

[Cristian Fogel [:cfogel]]

Updating bug status based on the previous comment.
@sheelgautam995 thank you for the reply!

Comment 22

[N. de Jonge]

It is still very much broken.
Consistently crashes the tab for me in 76.0 (64-bit).

Cristian, you should not have marked this fixed based on one person's comment that it "seems to be fixed".

The only way to properly test this bug is to download a file from MEGA that is almost as large or larger than the available system memory.
In other words, if you have 4GB memory, then try to download a 3GB+ file from MEGA.

Not fixed, still a bug.

Comment 23

[:Gijs (he/him)]

(In reply to N. de Jonge from comment #22)

It is still very much broken.
Consistently crashes the tab for me in 76.0 (64-bit).

You get a crashed tab error page? That's a new/different issue. Please file a separate bug.

The only way to properly test this bug is to download a file from MEGA that is almost as large or larger than the available system memory.
In other words, if you have 4GB memory, then try to download a 3GB+ file from MEGA.

This also sounds different - in comment #0, the download succeeded eventually, but the browser was very slow in the meantime. That's not what you're describing.

Comment 24

[sheelgautam995]

N. de Jonge, is the crash same as this https://bugzilla.mozilla.org/show_bug.cgi?id=1586255 ?

Comment 25

[UnPossible-Me]

Yep, I have the exact same issue.

Browser: Firefox v90.0
Host: Arch Linux

It seems it takes a file roughly half the size of your RAM to crash the tab.
I'm noticing a large consumption of RAM as well while downloading confirming Andrea's statement.

Comment 26

[:Gijs (he/him)]

(In reply to UnPossible-Me from comment #25)

Yep, I have the exact same issue.

Browser: Firefox v90.0
Host: Arch Linux

It seems it takes a file roughly half the size of your RAM to crash the tab.
I'm noticing a large consumption of RAM as well while downloading confirming Andrea's statement.

bug 1700187 tracks remaining issues here.

Comment 27

[Runner2022]

(In reply to UnPossible-Me from comment #25)

Yep, I have the exact same issue.

Browser: Firefox v90.0
Host: Arch Linux

It seems it takes a file roughly half the size of your RAM to crash the tab.
I'm noticing a large consumption of RAM as well while downloading confirming Andrea's statement.

I have the same issue as you.

Browser: Firefox
Host: Windows 11

The whole browser and the system froze at the end of the download.
I tried it three times. It crashed every time.
It seemed to be a RAM problem but I checked it backstage and RAM had plenty of space.

Comment 28

[kahnwiley]

I am having a similar issue; reproduced it in Firefox, Chrome, and Edge. Downloading large files, multiple files via zip, or having multiple mega tabs open simultaneously not only causes the browser to lock up, but hard locks my entire system and I cannot even move the mouse or access the security options through ctrl+alt+del.

Sent a bug report to Mega, their response (after trying to blame my DIMM slots or my fan):

"Yes - browsers are memory hogs and eat gigabytes as if they were cheap.

However, that will not cause correctly functioning computers running
a correctly functioning OS to lock up.

It will cause a slowdown, kill the tab, kill the browser, kill other apps,
but neither would the entire OS crash, nor would it freeze."

I was rather incensed by this response as I spent some time specifically detailing my replication of the bug in multiple browsers. Seems like they don't want to admit their processes reserve way too much memory for my poor computer to keep up.