Closed Bug 411626 Opened 16 years ago Closed 12 years ago

Need regression tests for JPEG image decoding


(Core :: Graphics: ImageLib, enhancement)

Not set





(Reporter: Dolske, Unassigned)


(Depends on 1 open bug)



(1 file, 1 obsolete file)

Spun off from bug 386651... We should have some tests to ensure that various flavors/options of JPEG are correctly decoded.

I looked around, and can't find any kind of existing JPEG test suite. So, we should probably just one from scratch. Things to test might include:
* Simple, baseline JPEG
* Progressive JPEG
* Different compression levels (this is mostly an encoder thing, not sure if
  it really ends up exercising the decoder differently)
* Inclusion of metadata (EXIF, thumbnail)
* Inclusion of a color profile
* 12-bit JPEG (unsupported? but don't barf on it.)
* CMYK JPEG (bug 44781)

Previously fixed JPEG bugs may have interesting images for a test suite as well.
Blocks: 44781
Attached file Basic test set (obsolete) —
Here's a set of basic test images...
* Sizes (1x1 to 33x33)
* Progressive
* grayscale
* cmyk

Just need to reftest-ize them with PNG references.
Corrected some test images, and added PNG references.
Attachment #303442 - Attachment is obsolete: true

These reftests all pass on OS X and Solaris, but on Windows most of them fail with slight color differences. This isn't the same problem as bug 405384, so I'm guessing it could be due to the SSE2 stuff (ala bug 411379)? I don't think that's wrong (iirc the JPEG spec even explicitly allows some variation in decoding), but it makes life more difficult for reftests.

Example: With the jpeg-size-2x2.jpg, the values are...

          Reference PNG                           Test JPG
  rgb(179,179,93)  rgb(143,143,57)     rgb(178,178,92) rgb(143,143,57)
  rgb(106,106,20)  rgb( 80, 80, 0)     rgb(106,106,20) rgb( 80, 80, 0)

Note the slight difference in the upper-left pixel. The 1x1, 3x3, 4x4, 6x6, and 8x8 images are the only ones which pass on Windows.

For now, I'll probably land these as-is, but marked as fails-if() on Windows [as was done in bug 405384 while those issues were worked out]. Assuming the color differences are expected, we can then either (1) land reference images specific to Windows [and mark them as failing on other platforms], or (2) fix the general problem by adding some capability to reftest to allow for fuzzy comparisons. This would probably be useful in other places as well. For example, the canvas tests added in bug 407049 have a number of tests that pass if the actual value matches the expected value +/- a certain range.
Regarding bug 411379, the SSE2 color processing hasn't been added yet. The only color-related code in 411379 is in using a static table for the color tables instead of computing them for every image.

Do you have a date when the tests started failing?
Attachment #303464 - Attachment description: Basic test set v.2 → [checked in] Basic test set v.2
Oh. Hmm. I haven't been following that closely, and thought it was already in.

I put up a testcase at, which reports any differences between the 16x16 JPEG and PNG used in the reftests. With a current trunk nightly on Windows, I get:

Pixel (14,9) differs.
 jpg: rgb(21,102,132)
 png: rgb(21,102,131)
Pixel (14,10) differs.
 jpg: rgb(20,85,149)
 png: rgb(20,86,147)
Pixel (15,10) differs.
 jpg: rgb(10,84,157)
 png: rgb(10,84,155)
Pixel (13,11) differs.
 jpg: rgb(37,70,149)
 png: rgb(37,70,147)
Pixel (14,11) differs.
 jpg: rgb(21,69,167)
 png: rgb(21,70,165)
Test complete. Images are NOT identical.

Could you see if you get different results with your SSE2 patch? We could also check to see if this used to work in the past, but if the SSE2 patch is going to change things anyway, that might not really matter.
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9b4pre) Gecko/2008022809 Minefield/3.0b4pre

With this build and a recent pull of the tree, I ran the reftests and got these results:

REFTEST PASS: file:///D:/moz/mozilla/modules/libpr0n/test/reftest/jpeg/jpg-size-1x1.jpg
REFTEST UNEXPECTED PASS: file:///D:/moz/mozilla/modules/libpr0n/test/reftest/jpeg/jpg-size-2x2.jpg
REFTEST PASS: file:///D:/moz/mozilla/modules/libpr0n/test/reftest/jpeg/jpg-size-3x3.jpg
REFTEST PASS: file:///D:/moz/mozilla/modules/libpr0n/test/reftest/jpeg/jpg-size-4x4.jpg
REFTEST UNEXPECTED PASS: file:///D:/moz/mozilla/modules/libpr0n/test/reftest/jpeg/jpg-size-5x5.jpg
REFTEST PASS: file:///D:/moz/mozilla/modules/libpr0n/test/reftest/jpeg/jpg-size-6x6.jpg
REFTEST UNEXPECTED PASS: file:///D:/moz/mozilla/modules/libpr0n/test/reftest/jpeg/jpg-size-7x7.jpg
REFTEST PASS: file:///D:/moz/mozilla/modules/libpr0n/test/reftest/jpeg/jpg-size-8x8.jpg
REFTEST UNEXPECTED PASS: file:///D:/moz/mozilla/modules/libpr0n/test/reftest/jpeg/jpg-size-9x9.jpg
REFTEST UNEXPECTED PASS: file:///D:/moz/mozilla/modules/libpr0n/test/reftest/jpeg/jpg-size-15x15.jpg
REFTEST UNEXPECTED PASS: file:///D:/moz/mozilla/modules/libpr0n/test/reftest/jpeg/jpg-size-16x16.jpg
REFTEST UNEXPECTED PASS: file:///D:/moz/mozilla/modules/libpr0n/test/reftest/jpeg/jpg-size-17x17.jpg
REFTEST UNEXPECTED PASS: file:///D:/moz/mozilla/modules/libpr0n/test/reftest/jpeg/jpg-size-31x31.jpg
REFTEST UNEXPECTED PASS: file:///D:/moz/mozilla/modules/libpr0n/test/reftest/jpeg/jpg-size-32x32.jpg
REFTEST UNEXPECTED PASS: file:///D:/moz/mozilla/modules/libpr0n/test/reftest/jpeg/jpg-size-33x33.jpg
REFTEST UNEXPECTED PASS: file:///D:/moz/mozilla/modules/libpr0n/test/reftest/jpeg/jpg-progressive.jpg
REFTEST UNEXPECTED PASS: file:///D:/moz/mozilla/modules/libpr0n/test/reftest/jpeg/jpg-gray.jpg
REFTEST UNEXPECTED PASS: file:///D:/moz/mozilla/modules/libpr0n/test/reftest/jpeg/jpg-cmyk-1.jpg
REFTEST UNEXPECTED PASS: file:///D:/moz/mozilla/modules/libpr0n/test/reftest/jpeg/jpg-cmyk-2.jpg

WinXP SP2, 1280x1024x32 @ 85Hz, GeForce 6800GT, Forceware 169.21 drivers, AGPx4, 256MB Video memory, video BIOS AthlonXP2400, 1GB DDR RAM.
Closed: 12 years ago
Resolution: --- → FIXED
Woo close-to-4-year-old patches!

But we shouldn't resolved fixed until this lands on central, no?
Resolution: FIXED → ---
Oops, used the wrong tree for my link.

This landed in 2008 :)
Closed: 12 years ago12 years ago
Resolution: --- → FIXED
Depends on: 797042
You need to log in before you can comment on or make changes to this bug.