1392355 - High memory usage on a github wiki page

Status: RESOLVED INVALID

(Core :: DOM: Security, defect)

Description

[Jagan]

User Agent: Mozilla/5.0 (Windows NT 6.1; rv:50.0) Gecko/20100101 Firefox/50.0
Build ID: 20100101

Steps to reproduce:

Go to https://github.com/robbyrussell/oh-my-zsh/wiki/External-themes.


Actual results:

Memory usage went up and system became unresponsive (htop showed 2.4GB memory usage).  It was confirmed by some people in #nightly and this affects stable as well. 


Expected results:

Should work normally like always.

Comment 1

[Clément Lefèvre]

Confirming the bug. Reproduced on at least two computers with Nightly on different OS.
I didn't mesured the increase, but it was fast and lead to unresponsiveness quickly so I quickly closed/killed the tab.

Comment 2

[alex_mayorga]

¡Hola Jagan!

Please provide about:memory?verbose report of this issue.

¡Gracias!
Alex

Comment 3

[Jagan]

Attachment: about:memory logs

Comment 4

[Marius Coman [:mcoman], Ecosystem QA]

User Agent 	Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:57.0) Gecko/20100101 Firefox/57.0
Firefox: 57.0a1, Build ID 	20170815100349

I have tested this issue on Windows 10 x64, Ubuntu 14.04 x64 and macOS 10.12, with the latest Firefox release (55.0.2) and the latest Nightly (57.0a1) build. I have managed to reproduce it only on Windows 10 x64, while on macOS and Linux, the memory remains the same when visiting the provided page, or other web pages.
Considering this, using the Mozregression tool, I have managed to find a regression window. From the pushlog, it seems that the bug 1302539 has caused this.
 
Last good revision: c9a069260d72f3db448e705540f2708a55e7e0f1
First bad revision: 9eb7e1d20e40eea17611cd254c5e0aca17e16831
Pushlog:
https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=c9a069260d72f3db448e705540f2708a55e7e0f1&tochange=9eb7e1d20e40eea17611cd254c5e0aca17e16831

Christoph, can you please take a look at this?

Comment 5

[Clément Lefèvre]

May I suggest to remove Windows from OS?
I did managed to reproduce it on my Mac.

It's not the same version as you though, as this is OS X 10.9.
I couldn't reproduce it on my Archlinux though.

PS: This was tested using Stylo in every cases, if that does matter.

Comment 6

[Marius Coman [:mcoman], Ecosystem QA]

I have tested this issue on macOS 10.9 and managed to reproduce it as well. As result I'm changing the affected OSs to "All".

Comment 7

[Christoph Kerschbaumer [:ckerschb}]

When visiting the site [1] I see that the page ships with the header 'X-Content-Type-Options nosniff', but I don't see any messages on the console that XCTO nosniff caused blocking of some content. You mentioned that Bug 1302539 is responsible for causing the memory overhead - we had to disable XCTO for images within Bug 1302539 because it caused some compatibility issues between other browsers and Firefox. For the sake of completeness, XCTO nosniff was implemented within Bug 471020.

Anyway, I flipped the following pref to true:
> pref("security.xcto_nosniff_block_images", false);
and visited [1] again. I see four error messages on the console (see underneath). I assume that loading those images causes the memory overhead. In other words, I think it's not something that Firefox can fix, but rather the webpage.

[1] https://github.com/robbyrussell/oh-my-zsh/wiki/External-themes.

%-----snip-----

The resource from “https://camo.githubusercontent.com/e6cf18e92309e419510c24c13eab47732a6dc123/68747470733a2f2f646c2e64726f70626f7875736572636f6e74656e742e636f6d2f752f363734313338382f63696163686f2d7a73682d7468656d652e706e67” was blocked due to MIME type mismatch (X-Content-Type-Options: nosniff).[Learn More]  (unknown)

The resource from “https://camo.githubusercontent.com/6e3b868846d2e9546912afe473adc4681f6e4dc8/687474703a2f2f773333746d617269636963682e636f6d2f696d616765732f656e6c69676874656e6d656e742e706e67” was blocked due to MIME type mismatch (X-Content-Type-Options: nosniff).[Learn More]  (unknown)

The resource from “https://raw.githubusercontent.com/tylerreckart/hyperzsh/master/screenshots/demo.gif” was blocked due to MIME type mismatch (X-Content-Type-Options: nosniff).[Learn More]  (unknown)

The resource from “https://raw.githubusercontent.com/denysdovhan/spaceship-zsh-theme/master/preview.gif” was blocked due to MIME type mismatch (X-Content-Type-Options: nosniff).[Learn More]

Comment 8

[Clément Lefèvre]

(In reply to Christoph Kerschbaumer [:ckerschb] from comment #7)
> When visiting the site [1] I see that the page ships with the header
> 'X-Content-Type-Options nosniff', but I don't see any messages on the
> console that XCTO nosniff caused blocking of some content. You mentioned
> that Bug 1302539 is responsible for causing the memory overhead - we had to
> disable XCTO for images within Bug 1302539 because it caused some
> compatibility issues between other browsers and Firefox. For the sake of
> completeness, XCTO nosniff was implemented within Bug 471020.
> 
> Anyway, I flipped the following pref to true:
> > pref("security.xcto_nosniff_block_images", false);
> and visited [1] again. I see four error messages on the console (see
> underneath). I assume that loading those images causes the memory overhead.
> In other words, I think it's not something that Firefox can fix, but rather
> the webpage.
> 
> [1] https://github.com/robbyrussell/oh-my-zsh/wiki/External-themes.
> 
> %-----snip-----
> 
> The resource from
> “https://camo.githubusercontent.com/e6cf18e92309e419510c24c13eab47732a6dc123/
> 68747470733a2f2f646c2e64726f70626f7875736572636f6e74656e742e636f6d2f752f36373
> 4313338382f63696163686f2d7a73682d7468656d652e706e67” was blocked due to MIME
> type mismatch (X-Content-Type-Options: nosniff).[Learn More]  (unknown)
> 
> The resource from
> “https://camo.githubusercontent.com/6e3b868846d2e9546912afe473adc4681f6e4dc8/
> 687474703a2f2f773333746d617269636963682e636f6d2f696d616765732f656e6c696768746
> 56e6d656e742e706e67” was blocked due to MIME type mismatch
> (X-Content-Type-Options: nosniff).[Learn More]  (unknown)
> 
> The resource from
> “https://raw.githubusercontent.com/tylerreckart/hyperzsh/master/screenshots/
> demo.gif” was blocked due to MIME type mismatch (X-Content-Type-Options:
> nosniff).[Learn More]  (unknown)
> 
> The resource from
> “https://raw.githubusercontent.com/denysdovhan/spaceship-zsh-theme/master/
> preview.gif” was blocked due to MIME type mismatch (X-Content-Type-Options:
> nosniff).[Learn More]

Well, I don't really know what 'X-Content-Type-Options nosniff' are. However, even if there's something faulty on the visited webpage, I guess overloading memory like that to the point it can crash the whole computer isn't acceptable, so maybe should there have some security/failsafe?

Comment 9

[Christoph Kerschbaumer [:ckerschb}]

(In reply to Clément Lefèvre from comment #8)
> Well, I don't really know what 'X-Content-Type-Options nosniff' are.
> However, even if there's something faulty on the visited webpage, I guess
> overloading memory like that to the point it can crash the whole computer
> isn't acceptable, so maybe should there have some security/failsafe?

Just to make sure, does flipping the pref to true:
> pref("security.xcto_nosniff_block_images", false);
make the problem go away?

Comment 10

[Jagan]

I confirm that changing that pref to true does make the page load and does not increase memory usage.

Comment 11

[Daniel Veditz [:dveditz]]

This change did not "cause" the memory usage: The page has lots of images and uses lots of memory. It is entirely possible for a web page to use up all your memory and crash a tab -- a web page can be infinite.

Formerly we were blocking things that claimed to be <img> that didn't have a valid image mime type. This broke some pages (and it apparently breaking this one, though you didn't notice), and of course if images are blocked then they're not using any memory. The change in the regressing bug was simply to revert to the behavior before Bug 471020 wrt images.

Comment 12

[Clément Lefèvre]

(In reply to Daniel Veditz [:dveditz] from comment #11)
> This change did not "cause" the memory usage: The page has lots of images
> and uses lots of memory. It is entirely possible for a web page to use up
> all your memory and crash a tab -- a web page can be infinite.
> 
> Formerly we were blocking things that claimed to be <img> that didn't have a
> valid image mime type. This broke some pages (and it apparently breaking
> this one, though you didn't notice), and of course if images are blocked
> then they're not using any memory. The change in the regressing bug was
> simply to revert to the behavior before Bug 471020 wrt images.

Do you mean there is no failsafe or security measure to limit the memory a tab is taking?
I think there should be one, even if set very high in percentage of system memory, something like that.

And in the same time, even considering your explanation, 2.4GB seems pretty high for just a bunch of images (honnestly, there aren't that much, and those are not top-quality images)

Comment 13

[Boris Zbarsky [:bzbarsky]]

> honnestly, there aren't that much, and those are not top-quality images

https://raw.githubusercontent.com/tylerreckart/hyperzsh/master/screenshots/demo.gif is a 24MB gif file.  That's 24MB of _compressed_ data that needs to be decompressed.  When I just tried opening it in an image viewer program, the memory used by that program shot up to at least 10GB (I killed it at that point).  The problem is that it's an animated gif; each frame is 1596x1160 but there are 1242 of them.  Total memory usage needed to decode this image fully is 1242*1596*1160*4 bytes, or about 9GB.  One could play tricks with only decoding part of the image at a time, but I'm not sure browsers do that, because for normal uses of animated gifs it's not needed and can cause problems with stuttering.

https://raw.githubusercontent.com/denysdovhan/spaceship-zsh-theme/master/preview.gif is similarly a 6MB animated gif file with 1200x850 frames and 364 frames.  Total memory is 1200*850*364*4 bytes or about 1.5GB.

So no, 2.4GB doesn't seem all that high for those images.

Comment 14

[Clément Lefèvre]

(In reply to Boris Zbarsky [:bz] (still digging out from vacation mail) from comment #13)
> > honnestly, there aren't that much, and those are not top-quality images
> 
> https://raw.githubusercontent.com/tylerreckart/hyperzsh/master/screenshots/
> demo.gif is a 24MB gif file.  That's 24MB of _compressed_ data that needs to
> be decompressed.  When I just tried opening it in an image viewer program,
> the memory used by that program shot up to at least 10GB (I killed it at
> that point).  The problem is that it's an animated gif; each frame is
> 1596x1160 but there are 1242 of them.  Total memory usage needed to decode
> this image fully is 1242*1596*1160*4 bytes, or about 9GB.  One could play
> tricks with only decoding part of the image at a time, but I'm not sure
> browsers do that, because for normal uses of animated gifs it's not needed
> and can cause problems with stuttering.
> 
> https://raw.githubusercontent.com/denysdovhan/spaceship-zsh-theme/master/
> preview.gif is similarly a 6MB animated gif file with 1200x850 frames and
> 364 frames.  Total memory is 1200*850*364*4 bytes or about 1.5GB.
> 
> So no, 2.4GB doesn't seem all that high for those images.

My bad then. Didn't realized that.

Question is still up for some failsafe instead of crashing the whole browser if not OS though, what do you think about it?

Comment 15

[Boris Zbarsky [:bzbarsky]]

Not crashing the whole browser is obviously good, though I'd expect crashes to be limited to the content process, not the whole browser.

As for the OS, that's the kernel's job, I would think.

Comment 16

[Clément Lefèvre]

(In reply to Boris Zbarsky [:bz] (still digging out from vacation mail) from comment #15)
> Not crashing the whole browser is obviously good, though I'd expect crashes
> to be limited to the content process, not the whole browser.
> 
> As for the OS, that's the kernel's job, I would think.

Well, the problem to expect that the content process is going to crash is that in many cases, and it was for me, the memory overload already result in system swap and/or memory compression, leading to a fully unresponding browser.
Which is why we had to kill it quickly.

Comment 17

[Jagan]

Can the page be processed in a way that the GIFs are decoded only when they are viewed and thus reducing the memory usage. Memory usage for playing the GIFs when I tested using mpv, it takes about 100M to play the first GIF referenced by :bz instead of 10G. So I don't know why the whole GIF has to be loaded in memory as we don't load whole uncompressed videos into memory.

Comment 18

[Boris Zbarsky [:bzbarsky]]

I mentioned this in comment 13.  See the sentence starting "One could play tricks".