941823 - Large images may cause "Image corrupt or truncated" error

Status: RESOLVED DUPLICATE of bug 370763

(Core :: Graphics: ImageLib, defect)

Description

[David [:Ceremony]]

Attachment: imagetest.zip

User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:26.0) Gecko/20100101 Firefox/26.0 (Beta/Release)
Build ID: 20131118212339

Steps to reproduce:

When using high resolution images (eg. 15000x15000 pixels), Firefox behaves inconsistently and fails displaying said images. You can test it with the file I attached.

Note, that imagetype and filesize doesn't seem to matter, only the actual dimensions of the picture.


Actual results:

It can result in the "Image corrupt or truncated" error or just refuse to display them without any error. However, it does sometimes work and display those large images.


Expected results:

Firefox should be consistent and always display the images and not just sometimes.

Comment 1

[Elbart]

Upon a fresh start of the browser in Win7-64bit, I can at least open one of the images.
No chance on a 32bit-OS, though.

I assume it's because the largest contiguous block of available virtual memory (VM) after a fresh start is big enough on Win7-64bit due to the higher VM-limit (4GB for a 32bit-application compared to 2GB on a 32bit-Windows).

In my tests, the largest block on Win7-64bit was around 1.7GB, but on Vista-32bit around 400MB, which is obviously way too small for a 15k*15k-image (~640MB uncompressed).

I also get the "image corrupt or truncated"-error for smaller images when the current Firefox-session has been open for a while and is slowly approaching the VM-limit.

The older your Firefox-session is, the smaller the biggest free VM-block will be, and the more often the "Image truncated"-error will pop up, especially if you're browsing image-heavy sites.

I.e right now this session I'm using (running for six hours) on a 32bit-Win is at 1.5GB VM and the largest free VM-block is 130MB, plus some blocks at 40MB and 20MB.
Opening a gallery with a few images in the 1k*1k-range leaves me with a lot of gaps without images.

Example: http://imgur.com/gallery/LpuE8

And what it looks like now: http://i.imgur.com/9BjPxTl.jpg

Comment 2

[Elbart]

Test done with the testcase on Windows 7 64bit, but I removed the GIF due to lack of memory in this machine:
http://elbart.bplaced.net/vm_32bit_vs_64bit.png

32bit-Firefox: Only displays test.png, but there would be enough free memory and virtual memory to display test.jpg too. The largest free VM-block after decoding the PNG is too small for the JPG, which would take the same amount of memory as the decoded PNG.
After removing the LARGEADDRESSAWARE-flag from the firefox.exe-executable, you can't even display that one image (largest free block after start: 450MiB), although the whole browser including the decoded image would only take up ~1.4GB of memory.

64bit-Firefox: Both images are shown. The whole browser's memory-footprint is smaller than the VM-limit of a 32bit-program on Windows-64bit, so in theory the 32bit-Firefox should be able to display both images too.

Comment 3

[Nicholas Nethercote [inactive]]

Can any other 32-bit browsers open this file? Maybe Chrome can due to its use of separate processes, which makes it more likely that the necessary contiguous VM space is available? It's a tough one... even if electrolysis (a.k.a. multiple processes) land in Firefox, it's likely that the use of separate processes will be less aggressive than Chrome's, e.g. one chrome process and one content process, so it wouldn't help that much.

A 64-bit build of Firefox on Windows is a much more likely solution. The ETA on that is unknown, unfortunately.

Comment 4

[Elbart]

(In reply to Nicholas Nethercote [:njn] from comment #3)
> Can any other 32-bit browsers open this file? Maybe Chrome can due to its
> use of separate processes, which makes it more likely that the necessary
> contiguous VM space is available?

On my machine (Win7-64, 4GiB RAM), Chrome 33 Beta can show the page when one of the images is commented, otherwise the process crashes.
Nightly (2014-02-02) shows one image, the other one throws the "image corrupt or truncated"-error in the browser console.
IE11 can't even show one image, except for a short flash after finishing its decoding.

Comment 5

[David [:Ceremony]]

While reading webcomics online, I encountered a proper example of this issue with FF being unable to display this oversized image (691x50759): http://www.batoto.net/read/_/182566/dice-the-cube-that-changes-everything_ch8_by_soul-scans

Even a 64bit build of FF nightly is unable to handle the picture. A clean profile was used for both tests.
Chrome handles is very well with a fairly low memory footprint of around 300MB (all processes combined)

Comment 6

[Elbart]

I think that's a different bug, as Firefox can't open any JPG or PNG with either width or height larger than 32,767px.

Should be bug 591822.

Comment 7

[Mr Kirk]

Do we have time frame for when this issue will be fixed?

Comment 9

[Elbart]

Undo the DUPE!
The test-files are clearly nowhere near the 32767px-limit mentioned in bug 59182!
Bug 591822 has nothing to do with the memory-constraints mentioned in this bug!

Comment 10

[Virtual_ManPL [:Virtual] 🇵🇱 - (please needinfo? me - so I will see your comment/reply/question/etc.)]

Ahhh, sorry, you're right! My bad!

Comment 11

[Jared Wein [:jaws] (please needinfo? me)]

(firefox-backlog flag cleared because this isn't in the scope of the firefox desktop team)

Comment 12

[Seth Fowler [:seth] [:s2h]]

This should be fixed by downscale-during-decode.

Comment 13

[Arnou]

Yes this tip fix the problem on Windows XP x86, but not in Windows Seven x86

Comment 14

[HC]

I am using Firefox 39 on Windows 7 64.
Most of my camera pictures are problematic. The concerned images start to appear on loading at the upper fourth of screen (e.g. by dragging them from explorer to Firefox window) and then disappear to black. An error message is shown. Other images of "valid", smaller sizes are shown correctly. 
The process is reproducible: If I drag the same image onto Firefox, a part of it appears and then switches to error or even only the error message appears with a black background.
But: It once happened that the image was shown correctly when I brought Firefox back to top. So there may not be a static limitation on image size but some kind of memory management issue.
I found that there may be a limitation on the number of pixels of around 13.5MP, as all problematic images I found were larger than, for instance, 4600 x 3000 pixels. So this is not an "esotheric" test image size but about 1/3 the image size my camera can produce...

Comment 15

[HC]

(In reply to HC from comment #14)
> I am using Firefox 39 on Windows 7 64.
> Most of my camera pictures are problematic. The concerned images start to
> appear on loading at the upper fourth of screen (e.g. by dragging them from
> explorer to Firefox window) and then disappear to black. An error message is
> shown. Other images of "valid", smaller sizes are shown correctly. 
> The process is reproducible: If I drag the same image onto Firefox, a part
> of it appears and then switches to error or even only the error message
> appears with a black background.
> But: It once happened that the image was shown correctly when I brought
> Firefox back to top. So there may not be a static limitation on image size
> but some kind of memory management issue.
> I found that there may be a limitation on the number of pixels of around
> 13.5MP, as all problematic images I found were larger than, for instance,
> 4600 x 3000 pixels. So this is not an "esotheric" test image size but about
> 1/3 the image size my camera can produce...

I should have mentioned that all the images I have tested are displayed properly in Chrome 43.

Comment 16

[Elbart]

(In reply to HC from comment #14)

> But: It once happened that the image was shown correctly when I brought
> Firefox back to top. So there may not be a static limitation on image size
> but some kind of memory management issue.
> I found that there may be a limitation on the number of pixels of around
> 13.5MP, as all problematic images I found were larger than, for instance,
> 4600 x 3000 pixels. So this is not an "esotheric" test image size but about
> 1/3 the image size my camera can produce...

There's no size-limitation going on, other than bug 591822.

AFIAK Fx needs continuous memory for images to decode properly.
Using VMMap from Sysinternals, you can look up the memory of the firefox-process.
Select "free" and sort by size.

In your case 4600x3000x4bytes per pixel = 52.6MiB.

If the largest continuous chunk of free memory is smaller than 52.6MiB, the image-decoding stops and shows the "image corrupt or truncated" error.

Why Chrome isn't having this problem is simple: Multi-process.

Currently release Firefox-versions (32bit) only have one pool of virtual memory, either 2GB (on 32bit-Windows) or 4GB (on 64bit-Windows).
The longer the browser-session goes on, the more fragmented the memory becomes, the smaller the largest continuous free memory-blocks become.

Chrome is using one process (chrome.exe) per tab. So each tab can use the whole 2GB or 4GB.
Close the tab and open a new one, and you get a new clean pool of 2GB and 4GB.

Once Firefox gets multi-process-support ("Electrolysis", e10s), Firefox should behave the same as Chrome.
And once Firefox is released in 64bit, the only real limiting factor should be your machine's RAM (and currently bug 591822).

Comment 17

[David [:Ceremony]]

(In reply to Elbart from comment #16)
> Once Firefox gets multi-process-support ("Electrolysis", e10s), Firefox
> should behave the same as Chrome.
> And once Firefox is released in 64bit, the only real limiting factor should
> be your machine's RAM (and currently bug 591822).

Not sure how far e10s is yet, though in the current 64bit nightly edition (41.0a2), enabling e10s doesn't solve anything: The images from testcase will not show up simultaneously. More like one out of three, tops!

Comment 18

[u279076]

As Seth mentioned in comment 12, implementing bug 1045926 should resolve this issue and work is still progressing in that bug. I would not expect to see this issue resolved until that work is completed.

Comment 19

[marcino245]

I confirm the bug (Windows XP SP3 32bit, 4GB RAM, latest Nightly 47.0a1) - I loaded index.html first time from the attached imagetest.zip and all of the 3 thumbnails do not display (no errors - just empty white squares). However I found several reproductable behaviours:

1) after refreshing (F5 key) the tab, the PNG and GIF are displayed and the JPG still not. 
2) after refreshing again and again and so on, still the PNG and GIF are displayed and the JPG still not.
3) after closing the tab, clearing the history (ticking "browsing & download history" + "cache" is enought, you don't need to tick anything more), and then opening the tab again, again all of the 3 thumbnails do not display, as for the first time. (And then after refreshing the tab, again the PNG and GIF are displayed and the JPG still not.) 

And also several reproductable sub-behaviours:

4) after doing the 1), the PGN and GIF thumbnails are displayed, and also they are displayed after clicking by right mouse button on them and selecting "view image" (but this is still not the regular biggest size, just a size fit to the display), but since then, if you click on the image by left mouse button to zoom it (to a full size), it will fail and show just a black color, and there is no error message.

5) loading the thumbnails by "view image" works only for the first time and not for the second time or more (opposite to the thumbnails case which don't display for the first time, but display for the second time and more (after refreshing). And the error message appears: "The image "XXX" cannot be displayed because it contains errors"

So, to sum up briefly:

a) all of the 3 thumbnails are not displayed for the first time, but the PGN and GIF are displayed for the second time or more (after refreshing), JPG still fails

b) after refreshing as in a), the PGN and GIF thumbnails are also displayed correctly by selecting "view image" (they are fit to display) but fails when zooming to the orginal (the biggest) size, instead of the image, just a black color is displayed, and there is no error message.

c) opening the thumbnails (after refreshing as in a) ) by "view image" works only for the first time and not for the second time or more (an opposite to the a) ). The error message appears: "The image "XXX" cannot be displayed because it contains errors".


It would be good if someone confirmed the reproducibility of the bug behaviour.

Comment 20

[marcino245]

As for 5), it also shows a bad texture sometimes instead of an error message.

Comment 21

[marcino245]

Attachment: First time - all of the 3 thumbnails not displayed - white squares.png

Comment 22

[marcino245]

Attachment: Second time (after refreshing the tab) - PNG And GIF thumbnails displayed but JPG still not displayed.png

Comment 23

[marcino245]

Attachment: Opening the thumbnails by View Image (fit to display) works for the first time.png

Comment 24

[marcino245]

Attachment: Opening the thumbnails by View Image (fit to display) fails for the second time - an error message.png

Comment 25

[marcino245]

Attachment: Opening the thumbnails by View Image (fit to display) fails for the second time - a bad texture.png

Comment 26

[marcino245]

Attachment: Zooming after View Image, not works and shows a black color.png

Comment 27

[marcino245]

And finally, I will only add that after several refreshes while in "View Image" (fit to display), it  sometimes works after it previoulsy failed and fails after it precioulsy worked (alternately).

So varied behavior of the entire bug is difficult and complex.

Comment 28

[marcino245]

I just noticed that this bug is a duplicate of bug 370763, someone should mark this as a duplicate.

Comment 29

[u279076]

(In reply to marcino245 from comment #28)
> I just noticed that this bug is a duplicate of bug 370763, someone should
> mark this as a duplicate.

Definitely looks like it could be a dupe. I'll close this bug and mark that bug as blocked by bug 1045926.