Closed Bug 867594 Opened 11 years ago Closed 8 months ago

getImageData returns incorrect data for sRGB images

Categories

(Core :: Graphics: Canvas2D, defect)

Product:
Core
Component:
Graphics: Canvas2D
Version:
22 Branch
Platform:
x86_64
Windows 7
Type:
defect

Tracking

()

Status:
RESOLVED INACTIVE

People

(Reporter: kael, Unassigned)

References

(
URL
)

Details

When a sRGB PNG image is loaded in Firefox and drawn to a canvas, calling getImageData on the canvas returns image data that has been converted from sRGB into the current display color space.

In the linked test case, I consider it correct that the images look different in Firefox, and even look different in the canvas, but getting their 'data' should definitely not produce different results.

For reference, Chrome just ignores the sRGB chunk in the PNG, so this test case works.

This being broken makes it very difficult to use images to store arbitrary data because now anyone creating an image has to know how to remove color profile data from their PNGs and JPEGs; non-trivial.

Bug 611371 claims that this is behavior is correct but I disagree. If the method were called 'getDisplayedColors' or something I think it would be more reasonable for it to factor in color profiles, but the current behavior irreversibly discards valuable information from images. That bug also claims there is a workaround, but I've never seen it documented - what is it? Is it standard?

Anyway, as I see it the problem is there are two use cases:
1. Get pixel data as displayed to the user given their current color profile
2. Get raw pixel data from a decoded image

Right now Chrome supports 1 but not 2, and Firefox supports 2 but not 1. I don't think either use case should be discarded, but 2 is almost certainly more important for real-world games and apps.

The current behavior in FF also makes it impossible to perform lossless image modification using canvas (i.e. draw an image to a canvas, modify it, then get a data URL to save it) because colorspace conversions can't be avoided. That sucks.
I got the two use cases backwards in the comment:
Right now Chrome supports 2, not 1. Firefox supports 1 but not 2.
There's no good way to do this with the current 2D API.  With WebGL, we added the UNPACK_COLORSPACE_CONVERSION_WEBGL and UNPACK_PREMULTIPLY_ALPHA_WEBGL parameters for precisely this reason, because there is no way to know what the use case is.

getImageData returns the contents of the canvas buffer, not any image.  It doesn't make sense to have a <canvas> right next to an <img> look different when the image is drawn into the canvas.  The canvas itself is always stored in the current display color space (because you can draw images with different color profiles into it, so we have to pick a single color space for the canvas).

If you're using an image to store arbitrary data, you're already doing something hacky and not trivial -- you need to make sure to remove color profile data, because having it is not correct; there *is* no profile for your data.
If it's possible to do in WebGL why can't it be possible in Canvas? Both have an underlying representation which is defined as a pixel buffer containing rgba data, without an associated colorspace. You can store data into a WebGL texture, and in fact that is done all the time.

I agree that the 'root cause' fix for this is to remove color profile data, but the reality is that it is not enough. This is a bug that will currently only affect people in Firefox on configurations where a color profile conversion occurs, which means anyone building an ordinary HTML5 app that relies on image data or image processing will build broken software without knowing it.

Could the color profile transformation be disabled for offscreen canvases, perhaps? Then you could at least do image processing and analysis without being tripped up by color profiles. 

Alternately, could the API be extended?

It's okay if this isn't fixed in the short term, but leaving it broken forever sucks.
Because WebGL has the notion of a "texture object", and the flags only apply to how the data is copied from the Image into the texture.  They don't apply to the <canvas> itself.  The 2D API has no such object (the HTML Image object is the equivalent, which already has data with colorspace etc. info).  A WebGL texture itself has no colorspace, unlike the <canvas> element which does.

The API could be extended, but to do what you want I suspect you really just want a getImageData equivalent directly on Image elements.
A way to read image data from images directly seems like the best solution, yes. Something like this is necessary for any image manipulation use cases - for example if I wanted to build a photo editor in HTML5 I would need a way to get at raw image data, otherwise every time an image got edited in my webapp, it would get mangled by the end user's color profile.
Would making the canvas backing store explicitly sRGB fix the problem? CSS already says sRGB is the default, and if the display profile is set to sRGB then the round tripping works.

It'd at least be more consistent than exposing the device colour space to web content.
If everything is sRGB, that would at least let you round-trip sRGB content. On the other hand, wouldn't that cause non-sRGB images to get mangled when you draw them? Hm.
It seems like bug 870026 would help here.
(In reply to Kevin Gadd (:kael) from comment #0)
> For reference, Chrome just ignores the sRGB chunk in the PNG, so this test
> case works.

Chrome (at least Chrome 22+ on Windows) does NOT ignore the image color profile. It's just that Blink (formerly the WebKit Chromium port) supports only iCCP chunk.

> This being broken makes it very difficult to use images to store arbitrary
> data

Don't use images as general-purpose storage, period.

> because now anyone creating an image has to know how to remove color
> profile data from their PNGs and JPEGs; non-trivial.

Users will have to learn about the color profile anyway. Or rather, I couldn't imagine that users manipulating images don't have to know about the color profile.
Even if the users don't know about the color profile, they will not touch the display color profile settings (because they don't know about the existence). So the settings will be left to the Windows default (a.k.a. sRGB).

> The current behavior in FF also makes it impossible to perform lossless
> image modification using canvas (i.e. draw an image to a canvas, modify it,
> then get a data URL to save it) because colorspace conversions can't be
> avoided. That sucks.

HTML5 image editors can strip chunks from the image by themselves before drawing it to the canvas. I admit it's hacky and suboptimal, so your request will be valid.
However, I really want to see a spec proposal. It will not be so useful if it's implemented as a Mozilla-specific extension.
A chromium bug about the new ImageBitmap type (defined in the canvas spec, I think?) suggests extending it to allow turning colorspace conversion on/off:

http://code.google.com/p/chromium/issues/detail?id=166658

This would be an elegant way to solve this problem; ImageBitmap already provides a clean way to decode image data (from blobs, etc), so it would allow you to load an image and explicitly choose whether you want colorspace conversion done, and then draw that loaded image to a canvas for safe manipulation and round-tripping.
i just ran into this today. many aggravating hours later narrowed it down to getImageData returning different results, major WTF. google landed me here.

stripping the color profile yeilded expected results. is there any way to detect color profiles via the Canvas API, so at least to warn user of inconsistency?
Your best bet is to use CORS to do a XHR in binary mode to load the image and then parse the headers. PNG headers are well specified and not too hard to read; not sure about JPEG but I assume it's not awful. I don't think GIFs can have color profile data.
Kevin, that sounds just aweful.

meanwhile, take a look at this strange case: http://ricardocabello.com/files/blog/mrdoob_4pointgradient4.png

as-is, getImageData on that gradient returns different results (ff good, chrome poor). stripping the profile in photoshop makes both return the same, poor result. re-applying sRGB in photoshop afterwards restores consistency, good result.

maybe the profile is stored in a way that chrome considers it corrupted, but ff does not?

In the process of migrating remaining bugs to the new severity system, the severity for this bug cannot be automatically determined. Please retriage this bug using the new severity system.

Severity: major → --

The severity field is not set for this bug.
:lsalzman, could you have a look please?

For more information, please visit BugBot documentation.

Flags: needinfo?(lsalzman)
Status: NEW → RESOLVED
Closed: 8 months ago
Flags: needinfo?(lsalzman)
Resolution: --- → INACTIVE
You need to log in before you can comment on or make changes to this bug.