Closed Bug 321548 Opened 19 years ago Closed 19 years ago

Certain scaled images are now blurry

Categories

(Core Graveyard :: GFX: Mac, defect)

Product:
Core Graveyard
Component:
GFX: Mac
Version:
1.8 Branch
Platform:
PowerPC
macOS
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED WONTFIX

People

(Reporter: ceo, Assigned: jaas)

Details

(Keywords: regression)

Attachments

(5 files)

screenshot (looks ok to me)
98.60 KB, image/png
Details
Screen shot from Netscape
69.28 KB, application/pdf
Details
Screenshot on Linux (SeaMonkey trunk)
96.01 KB, image/png
Details
gif89 - with interpolation
296.30 KB, image/png
Details
gif89 - no interpolation
165.20 KB, image/png
Details 
User-Agent:       Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8) Gecko/20051111 Firefox/1.5
Build Identifier: Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8) Gecko/20051111 Firefox/1.5

I run a project whose aim is to develop stroke order diagrams for each of the 6,355 Japanese Kanji in the JIS208 set. Our software creates a diagram by taking one "frame" from the diagram and sending it to the browser with the width and height expanded to 7 X magnification.  Every single browser known to us will simply magnify the small image by perfect mulitplication... each pixel in the original image becomes a 7 X 7 square in the magnified image.  It is critical to how we perform changes to the image with our click and point tools that this is consistent on all browser platforms.

Firefox 1.5 however has deviated from the norm.  We no longer see our image scaled with mathmatical precision.  Instead it has been blurred, and it is nearly impossible to continue using Firefox with our project.  What we would like is a way to turn off the image blurring when small images are magnified into exactly proportional large images.

Reproducible: Always

Steps to Reproduce:
1. is now a feature of v. 1.5




I can be called at 412-343-0113 to discuss this.
Please attach a small HTML testcase that demonstrates the problem,
using this link:
https://bugzilla.mozilla.org/attachment.cgi?bugid=321548&action=enter
Email from Reporter:

There isn't a way to show you the bug unless you have an account with the project:

Go to:

http://www.kanjicafe.com

Register an Ice Mocha account... then click on "SODER Login" from the home page with the login ID and password from Ice Mocha.

Jim



WFM Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8) Gecko/20051224 Firefox/1.5

I'll attach a screenshot showing what I see after I log in.  Is it different from what you see?
Keywords: regression
Summary: Change in Image Resizing Method Endangers Our Project → Scaled images are now blurry
Btw, please reply by adding a comment to the bug report, not by email.
IF you examine the same image in an older version of Firefox, or ANY other web browser, the difference will be glaring.  The image showing is not a perfect mathematical multiple of the image being served, but is being altered by the browser, making it impossible to make precise editing decisions on the root image.   PLEASE compare this image you are showing here with one from any other browser including older versions of Firefox or Firebird.
As far as I can tell, the big image looks identical (pixel for pixel) in Safari.  I don't see anything gthat looks "blurry" in the screenshot I attached.

Can you attach something that looks different from my screenshot (use Cmd+Shift+4 and drag a rectangle to take a screenshot) and tell me what browser you used to get it?
This PDF is a screen shot from Netscape.  This is what the main image is supposed to look like ( a 50 X 50 pixel image stretched to 350 X 350 )
My screenshot and your screenshot look pretty much the same to me: the 350x350 image consists of 7x7 blocks, each of which is not blurry.  What am I missing?
I don't know how to respond to your last comment.  These images are nothing alike.  For example, look at the black dotted border around the image.  In Netscape et al, every pixel of that border is 000000 black, but in Firefox 1.5/Safari, the edges of EVERY black dash has been tampered with to produce a grey.  Its some kind of Macintosh smoothing effect that renders every color's edge a blur between it and the color next to it, which is supposed to make it look less bit-mapped for a photo of a flower or something, but in the case of this project, ruins the accuracy of the magnification.
Here's what I see on Linux - it does not look blurred to me.
The previous screenshots do look blurred compared to what I see on Linux.
YES... your LINUX screenshot is exactly what it is SUPPOSED to look like... and Mac Firefox prior to 1.5 looked like this too... but now Firefox does some kind of Safari like image trickery to smooth out the image.  Compare your linux image to the 1st screenshot shared.
MacOSX GFX regression?
Assignee: nobody → joshmoz
Component: General → GFX: Mac
Product: Firefox → Core
QA Contact: general → mac
Version: unspecified → 1.8 Branch
The blurring is easiest to see on the dotted outline of the images.

Maybe a regression from when we went to Quartz for images (bug 245407).
We need to know when this regressed.
I see the difference now.  There's a slight blurring of the edges between the 7x7 blocks in the first screenshot.  Comparing it with the third screenshot, the difference is indeed visible.  The blurring is visible with the dashed border, and I can confirm with Colorzilla that the other edges are blurred as well.
Just tested this out, and it is due to the Quartz changes.  Specifically, the interpolation for the Quartz context defaults to medium-quality interpolation (http://developer.apple.com/qa/qa2001/qa1186.html).  Calling |::CGContextSetInterpolationQuality(context, kCGInterpolationNone);| in |nsDrawingSurfaceMac::StartQuartzDrawing()| makes KanjiCafe work as Jim expects.

But I'm not sure if this is a change that we want.  Interpolation does make images look nicer, and Safari does what Firefox currently does, so the latest Safari also has this problem with KanjiCafe.

As an example, look at http://www.mozilla.org/quality/browser/front-end/testcases/imaging/img-gif-89/img-gif-89.html, and compare with and without the interpolation change: the images look much better with interpolation.  (And as mentioned above, Safari seems to also use the default interpolation.)
Status: UNCONFIRMED → NEW
Ever confirmed: true
Current Firefox 1.5.
Firefox trunk with |kCGInterpolationNone| change.
You've raised the million dollar question.  Do you trust that the web developer is sending the image he/she intends to display, or do you make the assumption that he always wants you to run your dictated image alteration routine and its inherently irreversable affects. Trust me, if I wanted my edges blurred, I have the power to blur them.  Hard wiring what amounts to a bag of PhotoShop tricks into the browser's image display might make the programmer feel omnipotent, but its disruptive to the web developer's intentions. A browser should not force its image altering routines on people.  At the bare minimum, give us the ability not to use these tricks on our images.  I want a true representation of the image... and when you are building image manipulating tools, its not an option.  If all Firefox is is a glittery tool to show off some hot new interpolation, its not a serious browser.  It makes Firefox a less useful tool.  If Apple wants Safari to alter images this way to trick people into thinking web sites look cooler on Safari... well that's a regrettable decision on their part. I build tools...not toys.
Using nearest-neighbor scaling "alters the image" too, and usually in a more perceptable way.  The real question is which scaling algorithm works best for most scaled images on the Web.

The vast majority of images that a browser has to scale are not scaled in order to show visitors the precise colors of each pixel in the image, but to make the image bigger.  I think the burden for sites to do server-side scaling should be on the rare case, not the common case.

There might be cross-platform consistency or backwards-compatibility arguments for going back to nearest-neighbor, at least for Firefox 1.5.0.x / Firefox 2.  (Cairoification in Firefox 3 might bring interpolating scaling to all platforms, btw.)

See also bug 98971, "Prettier image resizing & scaling (Bilinear, Bicubic, anything better than Nearest Neighbor)".
Wow, I think you're the first person to ever actually ask for images to be scaled using nearest neighbor and not something better.  I don't really see this as a regression but as forward progress.  We're planning on moving to bicubic scaling on all platforms for Firefox 3.

Maybe a CSS property is needed for specifying how you want to scale an image so that we could let you revert back to nearest neighbor for your application.  Possible values to include would be nearest-neighbor, bilinear, bicubic.  The default value in Firefox should be bicubic.

This would give you the flexability you need.  cc'ing some layout people to see what they think.
IMHO if you want images to be scaled using a certain algorithm, you should do it server-side.
I think comment 22 is right on the nose for web pages.  The only reason browsers have historically used linear scaling is that nonlinear scaling was too performance-intensive to be usable on the hardware that was available.  But by now, that's very much less an issue.  And nonlinear scaling gives far better output for the vast majority of images on the web.

That said, in terms of web-based applications (as opposed to web pages), perhaps having the ability to perform linear scaling is desirable in some cases.  If nothing else, it allows scaling without server round-tripping.  I do still think this is a fairly minor edge case and we should be defaulting to nonlinear scaling in general.

If we do decide to have a way of specifying the scaling algorithm, I'm not sure a CSS property is the right way to do it.  Adding yet another CSS property has certain costs (in memory and performance) that don't seem justified by the very limited use case here...  More importantly, what scaling algorithm would work best is an intrinsic property of the image data and the way that the <img> is being used in the web page; that is, of the content.  So it seems to me that an attribute (HTML5?) would be more relevant.

I think you guys are in love with wiz bang, and not seeing this in a principled fashion.  Any scaling which makes arbitrary decisions on how the image should look when its scaled, and then forces the programmer's preferences, and alters the image itself without the developer's permission is a flawed form of thinking - and a little bit unethical.  The scaling used in MSIE, Opera, Netscape, old Firefox, etc, gives the web site developer an exactly magnified version of any image whose width and height settings are an exact multiple of the original.  When you start making assumptions about how I want my image to appear because you like how interpolation makes a few pet pics look nice to your eye, you're crossing the line.  That's not your decision to make.  There is no reason for the web developer to not send the exact image he wants to.  People like me, who scale the image in exact multiples, do so because we are making repeated banging on the server and need to reduce the bandwidth.  If I have to send every pixel of a 7 X version of the image, then the wait time is unbearable and the rate of failure skyrockets.  There isn't a good case for somebody to want to scale an image 7X if they are looking for pretty pictures.  That's just nuts.  They can send exactly what they want (and in fact they do).  Scaling only applied to people who are trying to magnify an image to see what is there -> so if you go and toy with the image, you defeat the purpose of magnification.

At the very least you should add a toggle in the browser preferences to select a less drastic form of scaling.
> Any scaling which makes arbitrary decisions on how the image should
> look when its scaled

That would be any scaling by a factor other than 1.

> and alters the image itself

Any scaling does this.

> gives the web site developer an exactly magnified version

What does "exactly magnified version" mean?  If the original image has a diagonal line that's as straight as possible given the granularity of the image, should scaling attempt to preserve the straightness intended by the image creator or the jagginess introduced by the image format limitations?  You claim the latter is the more important property and should be preserved at all costs.  I don't buy it.

> There is no reason for the web developer to not send the exact image he wants
> to.

Absolutely agreed.  That's what Robert said in comment 22 as well.  If you send the exact image you want and tell us to just render it, we will.  The problems are arising because you're sending an image, then asking us to perform an operation on it (scaling) which distorts the image in various ways.  Then you're not happy that we have chosen some sorts of distortion over others.  But the point is, distortion is inherent in the scaling process.

> There isn't a good case for somebody to want to scale an image 7X if they are
> looking for pretty pictures.

I'm not sure I buy this blanket statement.  But if true, where do you draw the line for where "there isn't a good case"?  Clearly scaling by a factor of up to 1.5, say, happens all the time. 

> Scaling only applied to people who are trying to magnify an image to see what
> is there 

The point is that "what is there" is not just "a set of pixels" but "an image".  And very often the latter is better served by nonlinear scaling (whereas linear scaling favors the former).

> At the very least you should add a toggle in the browser preferences

It's not going in the UI, I can guarantee that (since 99.99% of users would have no idea what the heck it's talking about).  And at that point, having it is pointless.  It's not like you can control what preferences your users have set anyway.
OK I've got an idea that will make everyone happy.  Use interpolation whenever the image scaling is less than 2X, and use the old way when its 2X or greater.  I'll agree that I might try to make a very small change to an images size when I'm parking it next to another image which is slightly different sizewise, and I would welcome interpolation in those cases, but nobody is posting pictures of their cat and scaling them 7X.  That's just a silly notion and a silly excuse to keep interpolation.
If you send the 7X image from the server, it should compress very well ... it should be far less than 49 times the size of the original image, if you use PNG format.

Jim, you have internalized the idea that nearest-neighbour scaling is the one true scaling algorithm and all other scaling algorithms are deviations from the truth. I'm afraid that just isn't so.
Robert, in Firefox prior to 1.5, and all other browsers except Safari, if you are using the browser as a tool to magnify an image, and you are scaling by an exact multiple of both the width and height, the magnified image is in fact an exact replica of the smaller image.  The original pixel is now a perfect 49 pixel square.  It may not have been the original idea, but by mathematical happenstance, it operates as a perfect microscope.

If that is nearest-neighbour scaling, then whatever its esthetical shortcomings in minor scaling operations, for large magnification scaling, it IS the one true scaling algorithm.
(In reply to comment #28)
> If that is nearest-neighbour scaling, then whatever its esthetical shortcomings
> in minor scaling operations, for large magnification scaling, it IS the one
> true scaling algorithm.

That's only true if you assume that the original raster image is a perfect and complete representation of the underlying data, which is generally not the case (e.g., photos, diagrams with diagonal lines, logos with diagonal lines or curves, text included in an image which almost always has diagonal lines and curves).

As a bit of a side comment, I've heard that the height and width attributes on image were originally not designed for scaling, but instead were originally designed to provide the intrinsic dimensions of the image so that the browser wouldn't have to re-layout the page when each image loaded.  A mismatch between height or width and the intrinsic dimensions of the image is thus an error case that all browsers happen to handle by scaling; in hindsight some people think that was a bad choice of error handling behavior.
>That's only true if you assume that the original raster image is a perfect and
>complete representation of the underlying data, which is generally not the case
>(e.g., photos, diagrams with diagonal lines, logos with diagonal lines or
>curves, text included in an image which almost always has diagonal lines and
>curves).

I cannot understand your argument.

>As a bit of a side comment, I've heard that the height and width attributes on
>image were originally not designed for scaling, but instead were originally
>designed to provide the intrinsic dimensions of the image so that the browser
>wouldn't have to re-layout the page when each image loaded.

Yes... by giving each image a width and height attribute, the page will not have to reload itself after the image downloads... 

>A mismatch between height or width and the intrinsic dimensions of the image is >thus an error case that all browsers happen to handle by scaling; in hindsight >some people think that was a bad choice of error handling behavior.

That's a non-sequitur.  If it was an error, nobody would have written a scaling routine... they would send a broken image icon.  The scaling doesn't JUST HAPPEN to be there... it was put there for the purpose of matching an image of any size to the desired one.  That's no more of an error than if a browser crashed in the absense of a favicon.ico.  There's no reason to call a power and attribute of the browser an error.  Nonsense.  The browser used to have the power to serve as a magnifier, and now it doesn't.  Its simply less useful now.  Period.

Besides the point is that nobody is ever going to want interpolation for a 7 X scaled image, and by not setting a limit to its use, you are making my favorite browser useless to our project.  If you simply curb interpolation to situations where scaling is less than 2X, you will save the browser from undo calculating on really huge scalings, keep all the interpolation benefits where they belong, and save those of us who magnify in order to inspect from having our images corrupted.  I can't imagine anyone magnifying more than 2X unless they want to know what the pixels underlying the image look like.
> I can't imagine anyone magnifying more than 2X unless they want to know what the
> pixels underlying the image look like.

I can. For example, someone using an image to create a gradient effect on a Web page, scaled to the height of the content, where the Web page is unexpectedly tall. Or someone with a very high resolution screen, or printer, where we scale everything up to make it appear normal size.

We are not going to add some hack to force a dumb scaling algorithm because it suits one particular application. Scale images server-side instead if it matters to you. Further discussion will waste time for all of us.
Status: NEW → RESOLVED
Closed: 19 years ago
Resolution: --- → INVALID
Sigh.  Not too smart.  The only hack is what you did to Firefox.  We're switching browsers... so I guess you win... and Firefox now joins the ranks of browsers to avoid.
I think INVALID in the face of a use-case broken by an intentional change is the wrong resolution.  WONTFIX is the right one.  Reopening to fix that.  But I did want to say more, in case Jim can change his mind and stick around a bit longer.

I joined Netscape in early April, 1995, and I can testify that img height and width were both (a) to allow layout to proceed before enough of the src had loaded to determine dimensions from image data, and (b) to support scaling.  Scaling was a feature.  A feature implemented a certain way becomes a norm, which is to say a compatibility constraint.

A norm is not always normative, and better scaling than nearest neighbor does look better for most images.  So the fruitful direction, if there is one, is pointed at by bz's comment 23.

Hixie, any initial thoughts?  We should have a new bug asking for an enhancement if there is a WHATWG plan of record to address this incompatible chnage.  It may be that Apple folks would want such an extension too.  The extension should be backward compatible, so Jim's content could use it without affecting old browsers.

/be
Status: RESOLVED → REOPENED
Resolution: INVALID → ---
Status: REOPENED → RESOLVED
Closed: 19 years ago → 19 years ago
Resolution: --- → WONTFIX
Summary: Scaled images are now blurry → Certain scaled images are now blurry
(In reply to comment #23)
> I think comment 22 is right on the nose for web pages.  The only reason
> browsers have historically used linear scaling is that nonlinear scaling was
> too performance-intensive to be usable on the hardware that was available.  But
> by now, that's very much less an issue.  And nonlinear scaling gives far better
> output for the vast majority of images on the web.
> 
> That said, in terms of web-based applications (as opposed to web pages),
> perhaps having the ability to perform linear scaling is desirable in some
> cases.  If nothing else, it allows scaling without server round-tripping.  I do
> still think this is a fairly minor edge case and we should be defaulting to
> nonlinear scaling in general.
> 
> If we do decide to have a way of specifying the scaling algorithm, I'm not sure
> a CSS property is the right way to do it.  Adding yet another CSS property has
> certain costs (in memory and performance) that don't seem justified by the very
> limited use case here...  More importantly, what scaling algorithm would work
> best is an intrinsic property of the image data and the way that the <img> is
> being used in the web page; that is, of the content.  So it seems to me that an
> attribute (HTML5?) would be more relevant.
> 

If you are saying that computers now have the power and resources to handle interpolation, then you should have no problem adding a css attribute for it. If the argument truly is resources, then I find it rather arbitrary to justify one but not the other, especially considering the dramatic change it is bringing to the landscape. I think you underestimate the number of cases this will apply to, both including application and web page situations. If you are going to approach the question of interpolation alternatives given machines can handle it, then at the very least provide a variety of solutions, which would reflect the maturity of the software.

Not everybody uses dashed lines via css (insert your favorite odd css attribute here), but it's there because someone does use it. Why don't we throw out every style-based feature in CSS since apparently software engineers are smarter than everyone else and know what's best for all of us? Websites that like to allow people to change schemes and customize are more likely to benefit from having interpolation options within CSS, and not HTML. Interpolation really is like a filter... and I could imagine filter support being handled via css somewhere down the line.

I concede interpolation as a default would be good for most images, but pixel artists and people posting diagrams should have an option, and that option should probably be css based (vector image formats is not a solution). The very same power that is allowing interpolation as a realistic consideration is also allowing for javascript based web game design, and it can be argued that this singular choice for interpolation is actually hindering progress and diversity in fields such as this.

Technically speaking, there are no curves in bitmap images, and if that's what you want, why don't you work with vectors?? A next-generation jpg-like image format would resolve this problem, without having to break existing solutions or 'consume resources' as is being suggested. Sometimes doing the right thing means waiting for media codifiers to get with the times, and not forcing a feature upon everyone. How many people are clamorously seeking interpolation, versus those who've cited problems they have with this here and now? The populist argument makes sense, but I side with actuals vs potentials in this matter. Even if Jim Rose got it wrong in one place or another (arbitrary scale interpolation application), the argument of principle he makes is still important.

John
John?  What's the upshot of this comment?  That it would be nice to have a way to affect this on the part of the page?

Isn't that what I said in the comment you quoted?  If that's what you want, please file a bug requesting it....
(In reply to comment #35)
> John?  What's the upshot of this comment?  That it would be nice to have a way
> to affect this on the part of the page?
> 
> Isn't that what I said in the comment you quoted?  If that's what you want,
> please file a bug requesting it....
> 

I'm probably not tech savvy enough to articulate the best solution and report a bug on it, but the dialog is helping me to understand that side and see what does work. I can put in my two cents and reiterate the points from my perspective, however.

You did make a point that including a preferences pane on interpolation features wouldn't be sensible, assuming people are too dense to figure it out, but we've seen features like this for anti-aliasing of fonts at specified sizes. If we get interpolation options, I think it goes hand-in-hand with UI support.

John
Product: Core → Core Graveyard
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: