Closed Bug 1584898 Opened 1 year ago Closed 1 year ago

Large downloads from MEGA freezes the entire browser

Categories

(Core :: DOM: File, defect, P2)

71 Branch
defect

Tracking

()

VERIFIED FIXED
mozilla72
Tracking Status
firefox72 --- fixed

People

(Reporter: sheelgautam995, Assigned: baku, NeedInfo)

References

Details

Attachments

(3 files)

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:71.0) Gecko/20100101 Firefox/71.0

Steps to reproduce:

So, I went to mega.nz and downloaded a big .zip file of around 1 GB in size.

Actual results:

Towards the end of download (after 99%) the entire browser froze and began to run like a slideshow. It became responsive when the file got saved into the Downloads folder. It happens only with large file sizes (1 GB or greater)

Expected results:

The browser should not become unresponsive. Chrome and Safari download just fine from mega.nz

My computer is a 2014 Non Retina Macbook Air. It has 4GB RAM.

Attached file about:support

Hi @sheelgautam995, tested the issue in Mac Book Pro Mojave 8GB 10_14_6 - the download occurs with no issue. Please test if the issue is reproducible in safe mode, here is a link that can help you:
https://support.mozilla.org/en-US/kb/troubleshoot-firefox-issues-using-safe-mode
Further, I will add a component, if ins't the right one please fell free to change it.
Regards,
Liviu

Component: Untriaged → File Handling
Flags: needinfo?(sheelgautam995)
Attached video Recording.mp4

The browser crashes when downloading from mega.nz in safe mode.
So, I downloaded a file in normal mode while making a recording.
As you can see, the browser freezes towards the end of the download.
Also, QuickTime crashed midway during the recording. I couldn't complete the recording.

Flags: needinfo?(sheelgautam995)

Is Firefox saving the download in memory instead of the disk ?

(In reply to sheelgautam995 from comment #5)

Is Firefox saving the download in memory instead of the disk ?

It doesn't normally, no.

Are there any disk permissions / disk types that differ between the target directory where you're saving the file and your temporary folders?

Could you try using the profiler ( https://profiler.firefox.com/ ) to get a performance profile of the end of the download? If you start profiling right before the freeze happens and finish/capture the profile when Firefox returns to normal, then use "Share..." to get a link that you can share here, we can have a look at what is making the process slow on your machine - off-hand it's hard to be sure what the issue is...

Flags: needinfo?(sheelgautam995)

Here is the profile. I hope I did it right.
https://perfht.ml/31qJgpk
Sometimes Mega.nz says "Firefox has an insufficient buffer to decrypt data in the browser". IDK what that means.

Flags: needinfo?(sheelgautam995)

Reporter: The profile is interesting, thank you. There's some bits missing at the beginning (possibly because the profiling buffer became full) which means I'm not sure how you ended up in this state / what happened that tripped the bad behaviour - can you provide more (step by step) details about how to reproduce this? Is the encryption perhaps something you turn on within mega.nz or something like that?

There's definitely some weird jankiness going on, and it looks like mega.nz uses blobs in a weird way. :baku, looks like we're janking marshaling (large?) blobs between processes, can you take a look at the profile in comment #7 and clarify if there's enough here for you to investigate/improve things?

Flags: needinfo?(sheelgautam995)
Flags: needinfo?(amarchesini)

Tentatively moving this to Core: DOM: File because the profile has scripts coming from blob:https://mega.nz/{some-guid} as well as lots of jank from PChildToParentStream code - feel free to move along to IPC or another more appropriate component.

Component: File Handling → DOM: File
Product: Firefox → Core

Here is another profile which I think is clearer than the last one.
https://perfht.ml/2IZvmnS
I got this profile with fresh boot of Firefox with only single tab open (i.e. mega.nz). I started profiling when about thirty seconds were left for the download to be finished.

The stutter starts to happen when the file is about to be saved to the disk.
I don't observe stutter with smaller file sizes.

I did not turn on any special features on mega.nz. I think mega.nz has some inbuilt encryption feature. I you download from mega.nz, you will notice that it doesn't download straight away. I think it first downloads the file in some temporary location and then decrypts it (this is purely a hunch) and then transfers it to the Downloads folder.

During the previous profiling, I was browsing even when it began to stutter. This time, I did not touch the computer during the profiling.

Flags: needinfo?(sheelgautam995)

When I download in safe mode from mega.nz. I sometime get this crash. I filed a separate bug for the crash. Take a look.
https://bugzilla.mozilla.org/show_bug.cgi?id=1586255

Also, I think the first profile got messed up probably because I ran it several times before the actual profiling.I had not idea it would mess up the result.

Priority: -- → P2

Am I right to say that the majority of the time is taken by nsPipe to release data?
Would be possible to call the 'free()' on a separate thread?

Flags: needinfo?(amarchesini) → needinfo?(ehsan)

(In reply to Andrea Marchesini [:baku] from comment #13)

Am I right to say that the majority of the time is taken by nsPipe to release data?

Yes, exactly. This is the input stream data of course (1GB buffer). This is freed from here

Would be possible to call the 'free()' on a separate thread?

It would be an improvement... But it won't be a complete solution, since for example we'll probably be holding the allocator lock during some of this time, so if the main thread tries to (de-)allocate some memory while this 1GB buffer is being deallocated in the background thread that would stall.

A better question to ask ourselves is why do we need to keep 1GB of data in memory while downloading a file... (I'm using the peak memory usage of 1,019MB in the profile to draw the conclusion that the entire downloaded file is stored in the parent process' memory while the download is in progress, up to 5 seconds afterwards too.) Can we occasionally flush this buffer or something to that effect?

Flags: needinfo?(ehsan) → needinfo?(amarchesini)

A better question to ask ourselves is why do we need to keep 1GB of data in memory while downloading a file... (I'm using the peak memory usage of 1,019MB in the profile to draw the conclusion that the entire downloaded file is stored in the parent process' memory while the download is in progress, up to 5 seconds afterwards too.) Can we occasionally flush this buffer or something to that effect?

This is how mega works: It creates a huge blob object in memory. Then, it decrypts it, and then it allows the user to download it.
Talking about those 5 secs, I would like to get rid of them, but we don't have a nice way to keep blobs alive during the loading and all the async steps. Let's keep this issue separate.

Flags: needinfo?(amarchesini)
Pushed by amarchesini@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/d68244d4b2dc
Free nsPipe memory on a separate thread, r=Ehsan
Status: UNCONFIRMED → RESOLVED
Closed: 1 year ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla72
Assignee: nobody → amarchesini
Flags: qe-verify+
Regressions: 1602986

Hello,
I've tried to reproduce this issue using Firefox 71.0a1 (2019-09-29) but without success.
I've followed the str from comment 1 but Firefox doesn't became unresponsive.

sheelgautam995, could you please confirm that this is fixed using Firefox 72.0.2? (https://archive.mozilla.org/pub/firefox/candidates/72.0.2-candidates/build1/)

Flags: needinfo?(sheelgautam995)

This seems to be fixed. I am using stable 73.0 right now. MEGA.nz seems to be a poorly designed service because its slow on pretty much every browser.

Flags: needinfo?(sheelgautam995)

Updating bug status based on the previous comment.
@sheelgautam995 thank you for the reply!

Status: RESOLVED → VERIFIED
Flags: qe-verify+

It is still very much broken.
Consistently crashes the tab for me in 76.0 (64-bit).

Cristian, you should not have marked this fixed based on one person's comment that it "seems to be fixed".

The only way to properly test this bug is to download a file from MEGA that is almost as large or larger than the available system memory.
In other words, if you have 4GB memory, then try to download a 3GB+ file from MEGA.

Not fixed, still a bug.

(In reply to N. de Jonge from comment #22)

It is still very much broken.
Consistently crashes the tab for me in 76.0 (64-bit).

You get a crashed tab error page? That's a new/different issue. Please file a separate bug.

The only way to properly test this bug is to download a file from MEGA that is almost as large or larger than the available system memory.
In other words, if you have 4GB memory, then try to download a 3GB+ file from MEGA.

This also sounds different - in comment #0, the download succeeded eventually, but the browser was very slow in the meantime. That's not what you're describing.

Flags: needinfo?(mail)

N. de Jonge, is the crash same as this https://bugzilla.mozilla.org/show_bug.cgi?id=1586255 ?

Regressions: 1667581
You need to log in before you can comment on or make changes to this bug.