Closed Bug 1469126 Opened Last year Closed Last year

DoS: A couple of nested Blobs may cause the browser to consume all available memory and crash

Categories

(Core :: DOM: File, defect)

60 Branch
defect
Not set

Tracking

()

VERIFIED FIXED
mozilla62
Tracking Status
firefox62 --- verified
firefox64 --- verified
firefox65 --- verified
firefox66 --- verified

People

(Reporter: viktor.dragomiretskyy, Assigned: baku)

References

Details

Attachments

(2 files)

User Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0
Build ID: 20180607194748

Steps to reproduce:

** WARNING - following these steps may freeze your computer **
1. Go to https://cdn.rawgit.com/uunicorn/blob-experiments/6a34a95b/ubuntu-cow-bomb.html
2. Wait for a minute or so.


Notes:

The script is only doing 30 iterations. This loop does not consume a lot of time nor memory. URL.createObjectURL(blob) on the other hand seems to be doing something very memory extensive on the resulting blob. 

The blob represents a binary tree with a depth of 30. Each layer of this tree is backed by a single Blob instance. The leaf nodes are represented by a single Blob instance of size 1 character, but there are 2^30 different paths to get to the leaf. 
So, in summary:
- the script only creates 31 Blobs instances;
- there is only one instance of Blob which has some data in it and the size of this blob is 1 character;
- the total size of the root blob is 2^30 bytes (1GB).

Oh, the latest Chrome seems to be vulnerable to this DoS attack as well.


Actual results:

The Firefox memory footprint grows rapidly up to the point when everything freezes and I have to hard-power-off the laptop. For some reason kernel OOM killer does not do its job (Ubuntu 16.04.4).

I've got similar experience on MacBook Pro as well, although Mac had a chance to ask me which process to kill (right before freezing completely).


Expected results:

FF should not crash and let me use my Blob filled with 1GB of 1's.
Flags: needinfo?(amarchesini)
Duplicate of this bug: 1469182
Attached patch blobSet.patchSplinter Review
Couple of issues here:

1. GetAllocationSize was counting the size of the same blob over and over again. Now I introduce a visitedBlob array to count the size of unique blobs.

2. BlobSet.appendBlobImpl() should return NS_ERROR_OUT_OF_MEMORY.

3. The return value of BlobSet.AppendFoo() should be checked everywhere.
Assignee: nobody → amarchesini
Flags: needinfo?(amarchesini)
Attachment #8985842 - Flags: review?(bugs)
Let's try to be smarter when creating an inputStream for Multiplex Blobs.
Attachment #8985857 - Flags: review?(bugs)
Hi Andrea,

I'm looking at the patches and I'm not sure they fully address the problem.
If you modify the test script from https://bugzilla.mozilla.org/show_bug.cgi?id=1469182 to look like this:

var blob = new Blob(['1']);
for(;;)
    blob = new Blob([blob, '2']);

it will probably still cause the unbounded recursion.

---------------------------------------------------------

Regarding the first patch which implements GetAllocationSize() memoisation: can you try your fix against the script which is actually using the URL returned by URL.createObjectURL(blob) ?
Here's an example: https://cdn.rawgit.com/uunicorn/blob-experiments/933a4a62/1gb-of-ones.html
With a loop count of 10 I can easily download a file filled with 1KB of 1's now.
With a count of 30 it should be possible to download a file with 1GB of 1's.


Regards,
Viktor
Attachment #8985857 - Flags: review?(bugs) → review+
Attachment #8985842 - Flags: review?(bugs) → review+
Pushed by amarchesini@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/4a137fd2fcaf
GetAllocationSize should report unique blobs when dealing with nested blobs, r=smaug
https://hg.mozilla.org/integration/mozilla-inbound/rev/9dc030b148fa
Multipart Blobs should use nsMultiplexInputStream only when needed, r=smaug
> var blob = new Blob(['1']);
> for(;;)
>     blob = new Blob([blob, '2']);

> it will probably still cause the unbounded recursion.

Sure, but at least we report the correct allocation size. This was the main goal of my first patch.
Your script will end up with a OOM allocation and now, we should report it correctly (see the BlobSet part of the patch).
We should not crash anymore.

> Regarding the first patch which implements GetAllocationSize() memoisation:
> can you try your fix against the script which is actually using the URL
> returned by URL.createObjectURL(blob) ?

I would consider this script a DoS attach. If firefox manages to allocate all the memory, it should work, otherwise, it terminates the script with a OOM error eventually.
https://hg.mozilla.org/mozilla-central/rev/4a137fd2fcaf
https://hg.mozilla.org/mozilla-central/rev/9dc030b148fa
Status: UNCONFIRMED → RESOLVED
Closed: Last year
Resolution: --- → FIXED
Target Milestone: --- → mozilla62
Problem is not fully fixed.
After applying both patches I can crash the tab using the following script:
https://cdn.rawgit.com/uunicorn/blob-experiments/f15fe8e6/deep-recursion-2-bomb.html

Shall I open a new bug for this one?
Status: RESOLVED → UNCONFIRMED
Resolution: FIXED → ---
Yes. please. File a separate bug. What you are proposing here a DoS script, that crashes with a OOM error.
Status: UNCONFIRMED → RESOLVED
Closed: Last yearLast year
Resolution: --- → FIXED
Flags: qe-verify+
I’ve reproduced this issue on Firefox 60.0.2 (20180605171542) under Ubuntu 16.04 x64 with the link from bug’s description. Memory increased gradually and my computer froze at 81 %. 
Loading the site on the latest Beta 62.0b7(20180709172241) or latest Nightly 63.0a1 (2018-07-12) on the same platform, I ended up in the same situation. When memory reached around 80-85%, my computer froze again. 
On Ubuntu 16.04, but on 32 bits, the browser crashed with the OOM signature (on 62.0b7).
Am I missing something here? Not sure, though, what exactly should be the fix that I have to see (excluding computer freezing and crashes.
I’ve retested this bug on the latest Beta (62.0b12), using the link from comment 0, with the following results:
- Ubuntu 16.04 x64 - “out of memory: 0x0000000000000080 bytes requested” message inside the terminal, after the computer seemed to freeze for a few moments
- Ubuntu 16.04 x32 - tab crashes after several seconds with OOM | small signature
- Windows 10 x64 - tab crashed after several minutes with OOM | small signature
- macOS 10.13 - tab crashed after several minutes with IPCError-browser | ShutDownKill signature

Andrea, can you please tell me what exactly the expected behavior should be here?
Flags: needinfo?(amarchesini)
> Andrea, can you please tell me what exactly the expected behavior should be
> here?

Anca, can you please send me some crash-reports? Thanks. I want to see if we can do something for that small signature crashes.
Flags: needinfo?(amarchesini) → needinfo?(anca.soncutean)
Blocks: 1479407
I filed a new bug to work on this crash. See bug 1479407.
Hi, I can Confirm this issue as fixed in Firefox 64.0, Beta 65.0b6 and Nightly 66, the Issue no longer freezes the computer on either ubuntu, mac or windows, it does however get the tab crash mentioned in Comment 11. I will mark this as Verified since there already is a different issue logged for the crash.
Status: RESOLVED → VERIFIED
Flags: qe-verify+
You need to log in before you can comment on or make changes to this bug.