Closed Bug 160453 Opened 22 years ago Closed 22 years ago

libpng - Too much data in IDAT chunks - image cannot be displayed

Categories

(Core :: Graphics: ImageLib, defect)

Product:
Core
Component:
Graphics: ImageLib
Platform:
x86
Linux
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

()

Status:
VERIFIED DUPLICATE of bug 170352

People

(Reporter: jmd, Assigned: pavlov)

References

Details

Attachments

(3 files)

test image - 640 x 480, 8-bit/color RGB, non-interlaced
68.02 KB, image/png
Details
TCP flow dump
1.80 KB, text/plain
Details
TCP flow dump 2
2.66 KB, text/plain
Details 
Mozilla is having issues with a lot of PNG files that every other view is
capable of handling.

Attachment for bug 150317 is one case.
Attaching another image here.

This image shows fine in Netscape 4.x, ee, gqview, kview, and GIMP.
Attached file TCP flow dump — 
When trying to view the attachment, Mozilla displays the following:

The image "http://bugzilla.org/attachment.cgi?id=93531&action=view" cannot be
displayed, because it contains errors.

The TCP flow dump attached shows that two requests are actually made. I have
seen this behavior before. Will get back to this shortly, as it is related.
Attached file TCP flow dump 2 — 
This is (maybe) somewhat related to the previous attachment. The ADULT link
http://www.ultimatecalendargirls.com/pics1/sondra_hall/sonhbkblubnch025.jpg
will be displayed, but is followed by a 302 direct and a GIF. Mozilla says that
the requested JPEG cannot be displayed, because it contains errors.

Disabling HTTP 1.1 and pipelining solves this one, but not the previous.
The PNG files with "too many IDAT's" are in fact invalid.  Since Mozilla
is supposed to be standards-compliant, I don't think it should go too
far out of its way to render invalid image files.

Some time ago I gave pngcrush the ability to remove the extra IDAT chunks,
by intercepting libpng's error-processing feature (it checks the text of
the error message, and if it's "Too many IDAT's" it treats the error as a
warning and skips the extra IDAT).  Mozilla could swipe the pngcrush code
and do the same, but I don't recommend it.  Better just to complain to the
authors of the sites with nonstandard PNGs.

PS I know the apostrophe doesn't belong there.  I inherited it from previous
libpng maintainers.

I was going to CONFIRM, but for some reason the only choice offered me
is "(o) Leave as NEW"

Glenn
This is the same as bug #159498.  The exact error reported by libpng is
as the bug title says: "Too much data in IDAT chunks".  Interestingly,
libpng only notices this when the progressive reader is used.  Compare the
results of libpng's contrib/gregbook/rpng-x (sequential) versus rpng2-x
(progressive).

Glenn

The "Too much data in IDAT chunks" error was added recently in the process
of fixing bug #154996.  Previously, PNG files with a small amount of extra
data would harmlessly overflow into the row buffer and the image would be
displayed, while malevolent PNG files with a large amount of extra data
would crash; as the security people like to say, a carefully crafted
file could potentially be used for remote exploitation.  Now that
bug #154996 was fixed, all PNG files with too much data are trapped and
a png_error is issued.

Glenn
Based on Info-ZIP experience, I'm guessing "too much data" is a fairly common
occurrence in PNG images (i.e., deflate streams).  If so, a warning would be
more appropriate--as we've seen countless times in the past, anal decoders get
blamed for the problems rather than the broken encoders, because "it works in X,
Y and Z!"

(And, in general, the philosophy should be to be hyper-strict in encoding but as
permissive as possible in decoding.  Mozilla falls into the latter class.)

Greg
Libpng-1.2.5beta1, just released at http://libpng.sf.net addresses this
problem.  The png_error() about "Too much data" is changed to a png_warning()
and there is a bugfix to prevent the crashing that originally was the subject
of bug 154996.  Libpng is thus able to display the image, if one actually
exists in the corrupted file.  The remainder of the IDAT chunk or chunks is
skipped, so only one or two warnings are generated.

It would be useful if someone would build Mozilla with libpng-1.2.5beta1 and
see if the situation is indeed improved.

Glenn
Can someone explain exactly what is incorrect about this image 
(http://bugzilla.org/attachment.cgi?id=93531&action=view) ?  It has 9
IDAT chunks, the first 8 with 8192 bytes each, and the last with 3903 bytes.  
The 69439 bytes deflate to 922080 bytes output.  This is exactly the same as
(480 lines)*(1 + 3*(640 pixels per line) ).
I don't see where the "extra data" is.
           - Julian Brookes
> The 69439 bytes deflate to 922080 bytes output

How did you determine that?  Note that if you use libpng's non-progressive
reader, it stops when it has decoded enough, which would be 922080 bytes.
The progressive reader wouldn't stop until it ran out of data.  libpng-1.2.5beta1
will skip any extra bytes and only return the 922080 legitimate bytes.

Glenn
The 922080 is from examining the total_out field of the zlib struct after
inflating all the IDAT data.  I've posted the program I used at
http://www.martnet.com/~ysmaps/extrad/index.htm as well as another image that I
believe doesn't have extra data (which Mozilla also won't display), and another
test file which DOES have extra data.  The program I used skips over the IHDR so
it doesn't know how much inflated data is supposed to be there, just inflates
all IDAT and shows the byte count.   - Julian Brookes
Indeed, sometimes libpng erroneously says "Too much data in IDAT chunks"
when zlib returns Z_BUF_ERROR and there is no more data.  This will be
fixed in libpng-1.2.5.

Thanks.

Glenn
*** Bug 162819 has been marked as a duplicate of this bug. ***
A quick remedy for this problem is to change

   png_error(png_ptr, "Too much datas in IDAT chunks");

to
    if (png_ptr->zstream.avail_in)
        png_error(png_ptr, "Too much data in IDAT chunks");

in pngpread.c.

Libpng-1.2.5 will have a more extensive modification that makes
it safe to use png_warning() instead, and skip over the extra
data when it is present, thus avoiding any crash.

Glenn
Libpng-1.2.5rc1 is available now from the PNG sites such as
http://libpng.sf.net/ .  It fixes this bug.  Libpng-1.0.15rc1 also has the same
bugfix, if you're not ready to migrate from 1.0.x.  The "production" libpng
releases 1.2.5 and 1.0.15 are scheduled for 20 September.

Glenn
Any idea what encoders or encoding circumstances are putting this extra data
into the IDAT chunks? We should find out and file bugs against them.
I got the errors with images generated by RRDtool (more precisely, RRDs::graph
from a Perl script).
Since libpng-1.0.9 (patched) is erroneously identifying some files as having
too much data, it'd be best to build Mozilla with libpng-1.2.5rc1 and use that
to find the ones that are really invalid.  Alternatively, just use rpng2-x
in the libpng/contrib/gregbook directory of the libpng-1.2.5rc1 or the
libpng-1.0.15rc1 distribution.

Why has no one changed this to CONFIRMED?  I don't seem to be sufficiently
empowered to do it.

Glenn
I think I am seeing this error on a PNG image generated by PHP 4.2.1 on
www.advancedpowers.com
I'm not sure if the php bug is in PHP, GD or libpng.
See my comments above.  The error is in libpng and is fixed in libpng-1.2.5rc1.
Libpng-1.2.5 is scheduled to be released on September 20 and I suggest that
Mozilla upgrade to version 1.2.5 from the current patched version 1.0.9.  If
that is too much of a leap, upgrade to version 1.0.15 which will also be released
on September 20 and also fixes this error.

Glenn
*** Bug 159498 has been marked as a duplicate of this bug. ***
*** Bug 169392 has been marked as a duplicate of this bug. ***
WFM with libpng 1.2.5rc3 (btw, that setup needs some font help).
Depends on: 170352
WFM Moz 1.2beta, Build ID 2002101612.  Why is this bug still open?

Glenn
Because I forgot to close it when bug 170352 was checked in.

Closing...


*** This bug has been marked as a duplicate of 170352 ***
Status: NEW → RESOLVED
Closed: 22 years ago
Resolution: --- → DUPLICATE
Status: RESOLVED → VERIFIED
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: