Closed Bug 444014 Opened 12 years ago Closed 12 years ago

Firefox honors embedded ICC intent flag, and probably shouldn't

Categories

(Firefox :: General, defect)

PowerPC
macOS
defect
Not set

Tracking

()

RESOLVED FIXED

People

(Reporter: bugzilla, Assigned: bholley)

References

Details

Attachments

(4 files, 2 obsolete files)

User-Agent:       Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_3; en-us) AppleWebKit/527+ (KHTML, like Gecko) Version/3.1.1 Safari/525.20
Build Identifier: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9) Gecko/2008061004 Firefox/3.0

In certain situations, images will display incorrectly in Firefox even though they display correctly in other applications. This is primarily due to Firefox honoring the ICC profile intent flag. Specifically honoring the intent if set to "absolute colorimetric" rendering.

If the intent flag is set to "absolute colorimetric", and the wtpt tag in the two ICC profiles aren't the same (i.e. wtpt XYZ values are D50 for display, and D65 for the embedded profile color space), the image will shift dramatically blue or yellow. (D65 image to D50 display = blue shift; D50 image to D65 display = yellow shift).

Reproducible: Always

Steps to Reproduce:
1. Install and set display profile using a profile with wtpt tag set to D50.
2. gfx.color_management.enabled set to true and app relaunched.
3. Open in Firefox a JPEG with embedded sRGB ICC profile intent flag set to absolute colorimetric.
4. Open in Fiefox a duplicate JPEG with embedded sRGB ICC profile intent flag set to relative colorimetric
Actual Results:  
Image with intent=absolute colorimetric will display bluer than it should.

Expected Results:  
All other apps ignore this intent and use an intent appropriate for use of the image. For display, Relative Colorimetric + Black Point compensation is what Photoshop does by default. Next most appropriate is Perceptual.

Bug 418538 depends on this bug.
This likely applies to all platforms and OSs.

Otherwise useless information: for display profiles the ICC spec states that the wtpt tag should always contain XYZ values for D50. If the actual white point is not D50, then the use of the chad tag is required. There are a number of malformed profiles that don't follow this esoteric and debatable rule in the spec. But the result is that it's possible to get chromatic adaptation between different white points where it is not desired. The simple logic is that the user is chromatically adapted to the display, and further chromatic adaptation isn't needed, which is what is implied by use of absolute colorimetric rendering.

I'd recommend that contingency be made in coding for content and context specific rendering intents. For example sRGB images displayed to sRGB displays are best rendered using relative colorimetric. sRGB images for smaller gamut displays such as laptops are best rendered using perceptual. HDR images (when we get there) rendered to HDR displays are best rendered using absolute colorimetric, etc. So boolean logic should be employed to get the best results. In the meantime we probably should use Perceptual.
The white point of the display is actually native (I think close to D65), but as per ICC spec the primaries are adapted to D50. wtpt tag is thus also D50. This profile conforms to the clarified and stricter language in the version 4 ICC specification, although it is a version 2.4.0 profile.
This image is displaying incorrectly compared to all other color managed applications, which use either RelCol + BPC to display this image to screen, or Perceptual. Firefox is clearly honoring the embedded intent flag, and should ignore it (at least if AbsCol is set)
Displays correctly in FF. Matches other applications.
It is possible to argue this sRGB profile included in Mac OS X and by Adobe is malformed per the ICC specification. The wtpt tag in the profile is not set to D50, but rather to D65. That too is why this problem occurs in Firefox. This is made more clear in the ICC v4 spec. I'm not sure if Adobe is inclined to update and rerelease the sRGB profile or not as a v2 profile.

In wouldn't matter in any event. There are a ton of images floating around with this particular profile embedded in it. Most of them have intent set to Relative Colorimetric. A small percent may have AbsCol set. I think the problem will be rare, but as it is an inappropriate intent for display for JPEGs (given their current color space and dynamic range), intent=abscol should be ignored.
Duplicate of this bug: 441239
Marking as blocking bug 418538
Blocks: 418538
http://www.w3.org/TR/css3-color/
3.4 rendering intent property

Here we go. That paragraph exactly describes how FireFox 3 is behaving. I think it's a bad idea. No other application works this way. If Firefox worked this way today, it would unlikely be a problem the vast majority of the time. But the CSS3 spec falls flat in how we need this stuff to work in practice. And considering CSS3 has now been worked on for two internet eternities, if it asks us to do the wrong things, we'll be doing it for a decade at least.

Maybe we need to get some people involved in CSS3 into this conversation? Or maybe an offline introduction and conversation should occur?

More CSS3 complaints:

3.1.1 regarding gamma, has the same inappropriate recommendations found in CSS2, and in my view it should be removed from CSS3 entirely. At best it's based on old data, at worst it's based on incorrect data. It also seems to be in conflict with section 3.3 ICC color profile.

3.3 ICC color profile
The property value by default is auto, which causes sRGB to be used as source for everything that doesn't have an embedded profile. That's fine.

However there is an explicit and rather easy mechanism to override embedded profiles in images, which is total heresy, and is in conflict with the stated intended behavior of color management in Firefox 3. This is a specification that causes *except and only by default* the ignoring of important image metadata designed to preserve its color appearance, and is in a sense, data loss. That is really inappropriate for an application to actually behave this way, let alone for a spec to demand it.

CSS3 also does not appear to have any means of specifying profiles on a per object basis, rather than having to embed multiple redundant profiles, these could be specified per object via URL.
To discuss css3-color, subscribe to www-style@w3.org and send comments there, with "[css3-color]" at the beginning of the Subject line.  (If you don't subscribe, your mail will get stuck in moderation, perhaps forever.)  See archives at http://lists.w3.org/Archives/Public/www-style/ .

I'd note that a new working draft should be published within the next few weeks based on what's here http://dev.w3.org/csswg/css3-color/ .  Note that the color profile features have been dropped due to lack of implementation.  That said, we're still interested in comments on it.
Chris - I think you make some good points about the CSS3 spec. This probably isn't the best place to discuss the other CSS3 complaints, so you should probably take dbaron's advice and bring it up on the w3c lists. For now, let's just talk about the intent flag to keep the bug on topic.

I'm a bit confused as to what you mean by "No other application works this way." I assume that apps like photoshop honor the image's intent flag don't they? To me, the entire issue seems less of a problem with what we'll be dealing with for the next 10 years (since web designers and photographers will eventually get the message and chose a more appropriate intent flag), but rather with what will happen when we roll this stuff out (hopefully with firefox 3.1 if we get through all these bugs and if I can fix the perf issues).

I've been discussing the intent flag issue with Stuart and Vlad. In general, it seems kind of sucky to override something like an intent flag (since it is, or rather _should be_, what the creator intended), but I think in practice the net effect of not doing so would be to occasionally have whites rendered with a blueish tint. Given that I'm guessing the number of people who will ever proof on a web browser (supposedly the main use of absolute colorimetric intent, though you probably know way more about it than I do) can probably be counted on one hand, I think it's probably a smarter default to override absolute intent by default and to have an about:config option to enable it. It's unclear as to whether or not Safari respects intent flags at all, but if it does it certainly overrides absolute colorimetric intent (the test cases display properly).

Do you forsee a problem with any of the other intents (relative, perceptual, saturation)? If not, it's probably best to leave them be and only regulate the absolute flag. What would suggest changing it to? My gut vote would be to go for perceptual, but again I think you have more experience in this department.
(In reply to comment #9)
> This probably
> isn't the best place to discuss the other CSS3 complaints, so you should
> probably take dbaron's advice and bring it up on the w3c lists.

In lieu of the revised CSS3 color module, per David Baron's post, if that remains as proposed, in my view Mozilla can, and should, make their own path with respect to color management in the web browser. Someone has to be a leader in this venue, and at this point, W3C is abdicating its role. Of course, that's my opinion.

> I'm a bit confused as to what you mean by "No other application works this
> way." I assume that apps like photoshop honor the image's intent flag don't
> they?

No. It's ignored. All images opened in Photoshop, by default are displayed using source (embedded or assumed), to destination (current display profile) using Relative Colorimetric + Black Point compensation.




> To me, the entire issue seems less of a problem with what we'll be
> dealing with for the next 10 years (since web designers and photographers will
> eventually get the message and chose a more appropriate intent flag),

Forget it. I've been doing this for quite a while, and in just the past few days stumbled on Photoshop even writing an intent based on what was set in Color Settings, at the time the document was opened, or created. It's extraordinarily obscure.


> but
> rather with what will happen when we roll this stuff out (hopefully with
> firefox 3.1 if we get through all these bugs and if I can fix the perf issues).

perf?

In terms of imminent release, if you cannot implement RelCol + BPC, the next closesest intent is Perceptual. I would recommend ignoring the intent flag and using either of those (in order).

 
> I've been discussing the intent flag issue with Stuart and Vlad. In general, it
> seems kind of sucky to override something like an intent flag (since it is, or
> rather _should be_, what the creator intended),

Well it's a b.s. tag in that it's really the absolute last resort rendering intent. It has very little practical usage, especially in context where we know what the correct intent to use is. Give we're talking about a web browser, RelCol + BPC is most appropriate for mimicking Photoshop. If Perceptual worked as well as everyone would like, and maybe as well as v4 promises, then I'd suggested Perceptual as the default. It's better if the designer could specify this in the document, rather than the image because the image is a different thing than its usage in context.



> but I think in practice the net
> effect of not doing so would be to occasionally have whites rendered with a
> blueish tint. Given that I'm guessing the number of people who will ever proof
> on a web browser (supposedly the main use of absolute colorimetric intent,
> though you probably know way more about it than I do) can probably be counted
> on one hand, I think it's probably a smarter default to override absolute
> intent by default and to have an about:config option to enable it.

It's esoteric, there are other ways that already exist to deal with this. Another solution outside of a CSS3 context is not needed in my opinion. I think W3C's backtracking on ICC implementation is unfortunate, although I'm glad to see what was suggested has been canned. I think there is a rational middle ground that will not take much coding, is forward thinking and extensible.

> It's unclear
> as to whether or not Safari respects intent flags at all, but if it does it
> certainly overrides absolute colorimetric intent (the test cases display
> properly).

I think it ignores the intent flag, and uses Perceptual.


> Do you forsee a problem with any of the other intents (relative, perceptual,
> saturation)?

Rare.

In the case of current sRGB profiles that conform to the ICC v2 spec, and display profiles that largely also conform to the ICC v2 spec, there is no difference between the other three intents when using a conventional CMM, including lcms. For now. There is plenty of allowance in all versions of the ICC spec to perform CMM based intelligent gamut and dynamic range mapping, which could be different depending on the intent selected. In my view it's unlikely this will be an issue over the course of the next 3 years.

However, with ICC v4 source and or destination profiles, there can be differences between the intents. most of the time they are subtle.

In any event. the use case here is more strongly in favor for Perceptual as a default unless explicitly set to something else.

Note that RelCol + BPC is in effect a fifth rendering intent because it is a default intent in Adobe applications both for display and for print. And yet it has no official status in the ICC spec, therefore it isn't an option for the intent flag. The vast majority of users are expecting RelCol + BPC, and what is embedded in the images by Photoshop is RelCol (no placeholder for BPC). So if you cannot replicate BPC, you should use Perceptual in any case.

An argument could be made for the remaining intent: saturation. But most vendors duplicate the perceptual intent table in the saturation intent table.

 *shrug* 

I will bet there are 5 people on the whole planet, minus Adobe employes, myself, and anyone reading this, that has any notion that they can set the intent flag using Photoshop...

I'd pick Perceptual as a default, and compel the W3C to implement optional support for overriding it via a CSS3 property.


> If not, it's probably best to leave them be and only regulate the
> absolute flag. What would suggest changing it to? My gut vote would be to go
> for perceptual, but again I think you have more experience in this department.

sRGB to most display profiles today there will be no difference unless AbsCol is called.

Moving forward with v4 display profiles, and increasingly a v4 sRGB now in public beta by the ICC, Perceptual. As a default. I still think CSS3 needs to allow for overriding this.

But one thing is for sure is that 3.1.1 section on gamma compensation is useless. No one has ever bothered to pay attention to it, except for me, and I'm complaining about it.
>In lieu of the revised CSS3 color module, per David Baron's post, if that
>remains as proposed, in my view Mozilla can, and should, make their own path
>with respect to color management in the web browser. Someone has to be a leader
>in this venue, and at this point, W3C is abdicating its role. Of course, that's
>my opinion.

There's currently no code in mozilla (and probably not in any other browser) that implements the CSS3 color management spec. Given the debate over the issue, it sounds like the best course of action is to leave it there for the time being (indeed, it seems like your recommendation is mostly to ignore those properties anyway). I'd encourage you to work with the W3C to make sure that, whenever the spec is re-introduced, it is satisfactory. We'll revisit it from the mozilla side later on.

>perf?

Sorry, I haven't been doing a good job at keeping bugzilla complete and up to date (hopefully this will be fixed today). Basically, the two reasons color management was turned off by default in ff3.0 were the bugs (which you are intimately familiar with, and your expertise is proving invaluable in the triage process), and performance issues. When he turned it on, Stuart recorded a 30% performance hit, which was unacceptable for release. There wasn't time before the 3.0 release to work on it, so it was turned off and saved for 3.1. My job is to get it running fast. ;-) I've been hand-working assembly over the past few days and am making some good progress.


As for this bug itself, sometime in the next week or so I hope to write a patch that uses RelCol + BPC (a cursory look suggests that lcms does BPC) by default and adds an about:config to let people turn on respect for the intent flag if they desire. I'm assuming that should take care of the issues in this bug?
Status: UNCONFIRMED → ASSIGNED
Ever confirmed: true
Assignee: nobody → bholley
Status: ASSIGNED → NEW
Yep. There are some off list conversations occurring on this with some members of the ICC to see if there will be a recommendation in this regard. But it is a committee for whatever that's worth.

I think RelCol+BPC is safe so long as we get the same results as Photoshop. My understanding of lcms's implementation is it uses different steps than Adobe's method, but the results are the same. (?) I haven't tested it.

Anyway, most of the time (converting from most intermediate color spaces to most typical non-custom display profiles) there is no difference in any of the intents, except AbsCol. So that's the one really to avoid, and Saturation is not being paid attention to that much. But for now, in this ICC v2 to v4 transition phase, I think RelCol + BPC is a good idea, and matches up with Photoshop so at least the more hard core content creators will be getting essentially the same thing in Firefox and Photoshop and I think that's pretty cool.
I've attached a patch I wrote to fix this issue. I did some analysis and determined that black point compensation takes us through some code paths with really poor performance, so I went with perceptual rendering (as a default). The embedded intent flag can be honored by changing gfx.color_management.honor_embedded_intent in about:config).

I've thrown the patch up on the try server to generate builds for each platform. They should show up here:

https://build.mozilla.org/tryserver-builds/?C=M;O=D

Chris - when the build is done (in about an hour and a half or so), would you be willing to try it out and make sure it's satisfactory?
Using a 24 color test image (JPG)
1.untagged
2.tagged sRGB RC
3.tagged sRGB AC

With one exception, I get identical RGB values to the display from all three test images. Clearly the AC intent flag in the JPEG's ICC profile is being ignored in favor of the default. 

Compared to Photoshop I get identical values in most of the 24 sampled colors, with perhaps 1/3 having 1 to 2 level differences in one or two channels. Tonal reproduction and gray balance are identical.

(The one exception is with a light yellow color, and the untagged JPEG, where the blue channel is off by 1 level. Interesting, but in my view not presently relevant and worth chasing.)

For CMYK in FF compared to Photoshop is where the lack of BPC really comes into play, so there are greater differences numerically but visually they are, I think minor. The difference between FF and Photoshop color managed CMYK will be invisible to all but the most sophisticated users, and FF is not a soft proofing application. Compared to color management off for tagged CMYK, the difference is astounding.
Comment on attachment 330077 [details] [diff] [review]
patch to override embedded intents with perceptual rendering (unless user specifies otherwise in about:config)

great! flagging vlad for review
Attachment #330077 - Flags: review?(vladimir)
Comment on attachment 330077 [details] [diff] [review]
patch to override embedded intents with perceptual rendering (unless user specifies otherwise in about:config)

Let's not put a pref lookup inside the critical path for image decoding :)  Parse this pref in gfx init, where we decide on/off for cms in general.
Attachment #330077 - Flags: review?(vladimir) → review-
added an updated patch. Flagging vlad for review.
Attachment #330077 - Attachment is obsolete: true
Attachment #330137 - Flags: review?(vladimir)
added another patch to address vlad's concerns (from IRC). Hopefully this'll be the last.
Attachment #330137 - Attachment is obsolete: true
Attachment #330173 - Flags: review?(vladimir)
Attachment #330137 - Flags: review?(vladimir)
Comment on attachment 330173 [details] [diff] [review]
yet another iteration to please vlad. You can now specify the override intent in about:config

Looks great, thanks!
Attachment #330173 - Flags: review?(vladimir) → review+
thanks vlad - flagging as checkin-needed
Status: NEW → ASSIGNED
Keywords: checkin-needed
Pushed to mozilla-central:
http://hg.mozilla.org/mozilla-central/index.cgi/rev/68b0caac284d
Status: ASSIGNED → RESOLVED
Closed: 12 years ago
Keywords: checkin-needed
Resolution: --- → FIXED
(In reply to comment #21)
Is this push means we must use internal lcms and cannot use system lcms?
Matsuura - yes, among other patches. I"m not sure if there was previously an option to use a system lcms, but until the changes we're making to lcms are pushed upstream you'll need to use the bundled library. It's only ~100k though. ;-)
(In reply to comment #23)
I see. I'll use bundled version of lcms until upstream will release the new version with this fix.
Has this fix been pushed?

Yes. mozilla/configure has the option of "--with-system-lcms" and use system lcms if system has 1.17 or higher version of lcms.
yes, it's been pushed (changeset 68b0caac284d).
Sorry for confusing.

> Has this fix been pushed?
means your change in lcms.h is also useful for lcms itself and has pushed to "lcms maintainer(s)".

It seems that your modification in lcms.h is used only in gfxPlatform.cpp. If your modification in lcms.h is useful only for mozilla, I think it should be put in other part (gfxPlatform.cpp?).

I don't like the discussion of the file size because it is sometimes apt to get excited like "remove MNG from mozilla".

We should not break compatibilities with original component easily unless "We strongly believe that our modification makes all users of original component happy but upstream doesn't accept it. (ie. aPNG)"
Sorry for the lack of context. The reason I broke compatibility so easily has to do with the fact that compatibility is broken in much more significant ways in other patches that will land soon (see bug 444661). We hope to push the changes upstream once they're all done, but we're not at that point yet. Stay tuned.
Hi Bobby,
Thank you for your comment.

I have understood your plan to push.
I hope both users/developers of mozilla and the dependent software will be happy.
Blocks: 469556
No longer blocks: 469556
You need to log in before you can comment on or make changes to this bug.