Closed Bug 1023323 Opened 10 years ago Closed 10 years ago

Intermittent 413361-1.html | image comparison (==), max difference: 5, number of differing pixels: 1

Categories

(Core :: Graphics: ImageLib, defect)

Product:
Core
Component:
Graphics: ImageLib
Platform:
ARM
Gonk (Firefox OS)
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

()

Status:
RESOLVED FIXED
Milestone:
mozilla35
Tracking Flags:
Tracking Status
firefox33 --- fixed
firefox34 --- fixed
firefox35 --- fixed
firefox-esr31 --- unaffected

People

(Reporter: emorley, Unassigned, NeedInfo)

Details

(Keywords: intermittent-failure)

Attachments

(2 files)

reftest log
22.24 KB, text/plain
Details
reftest-analyzer screenshot
19.82 KB, image/png
Details
Attached file reftest log — 
b2g_emulator_vm mozilla-inbound opt test reftest-4 on 2014-06-10 02:22:16 PDT for push 6d172e86ce2e

slave: tst-linux64-spot-794

https://tbpl.mozilla.org/php/getParsedLog.php?id=41431583&tree=Mozilla-Inbound

{
03:34:23     INFO -  REFTEST TEST-START | http://10.0.2.2:8888/tests/layout/reftests/bugs/413361-1.html
03:34:32     INFO -  REFTEST TEST-LOAD | http://10.0.2.2:8888/tests/layout/reftests/bugs/413361-1.html | 663 / 711 (93%)
03:34:32     INFO -  REFTEST TEST-LOAD | http://10.0.2.2:8888/tests/layout/reftests/bugs/413361-1-ref.html | 663 / 711 (93%)
03:34:32     INFO -  REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8888/tests/layout/reftests/bugs/413361-1.html | image comparison (==), max difference: 5, number of differing pixels: 1
}
1 pixel with each RGB component differ -5 from ref.
The failure in comment 2 (different slave) is *exactly* the same.
("The failure in comment 2" should have said "comment 1".)
I suspect this is another instance of "images don't scale deterministically
on some hardware" bug.
Component: Layout: Block and Inline → ImageLib
Android certainly doesn't scale images deterministically. I don't know how many times I've opened the reftest analyzer and seen that same image with a line of fail through the middle, and then much later seen either a fuzzy-if or a change to a solid image because "oh, I wasn't trying to test what the middle of the image looked like anyway."
dbaron, any reason not to switch to http://mxr.mozilla.org/mozilla-central/source/layout/reftests/bugs/blue-600x58.png like we have for half a dozen other things that were using mozilla-banner.gif just because it was handy?
Flags: needinfo?(dbaron)
If it really wasn't deterministic that would seem reasonable, although it seems somewhat odd to say "images don't scale deterministically" when they match 99.9% of the time and fail 0.1% of the time.  I'm curious if seth knows what's up.  (Probably not a huge deal, but also, if it's a repeated pattern of low-frequency intermittent failure in reftests, might not be a bad idea to fix the underlying problem.)

I guess the change to another same-dimension image would probably be fine for what this test was intended to test, though.  But part of what's great about tests is that they test things other than what was originally intended.
Flags: needinfo?(dbaron) → needinfo?(seth)
I guess one other question here:  is "some hardware" equal to "a particular small number of test machines that have some bad RAM", or is it evenly distributed across Android machines?  (I ask because being so close to deterministic seems odd.)
Flags: needinfo?(philringnalda)
Usually the cause of images not scaling the same way is that in one case we are drawing into a context the size of the whole page, and in the other case we are drawing into a context that is only the size of the image (because we are doing a paint where only it was invalidated). So on platforms where we draw into contexts like that to get deterministic results for a scaled image all you need to do is consider the offset of the image as well as the size. The seeming-non-determinism just comes from not considering the offset of the image.
Which midaired a comment where I said that I didn't understand bug 863658 comment 22 either, but I didn't see any sign of machine-specific there or in bug 948389 and bug 919482 and bug 909305. Personally I wouldn't be surprised if, in the fantasy world where someone would ever actually look at the tests which were blindly disabled to turn on reftests on Android, and the tests which were blindly disabled to turn on reftests on b2g, a fair number of things we skip-if are actually just resizing images without caring about the contents. mozilla-banner.gif's only in 48 reftests, but there's no reason to think there aren't other images, or other copies of it with different names.
Flags: needinfo?(philringnalda)
The thing that is machine specific is that we usually draw into a context the size of the whole window. Only on some backends do we size the context to the size of the area to draw.
Right, but we're trying to disambiguate "machine-specific, we should disable panda-0500 and pretend that we have diagnostics that can detect something wrong with it" from "platform-specific, you just can't reliably test that on this platform, so you can either change the test or have it disabled on this platform."
Oh, sorry, I was only speaking to platform-specific concerns.
Matt, do you know if we actually derive any benefit from the behavior that Timothy mentions in comment 13? If it's not buying us something,  I wonder if it wouldn't be better to fix the underlying behavior, since in the long term we are going to keep having this issue - even if we fix all existing tests, people will keep checking in more tests with non-solid-color images.
Flags: needinfo?(matt.woodrow)
(In reply to Seth Fowler [:seth] from comment #18)
> Matt, do you know if we actually derive any benefit from the behavior that
> Timothy mentions in comment 13? If it's not buying us something,  I wonder
> if it wouldn't be better to fix the underlying behavior, since in the long
> term we are going to keep having this issue - even if we fix all existing
> tests, people will keep checking in more tests with non-solid-color images.

I think we'd need to know exactly what the issue was to answer this.

Android definitely allocates surfaces the size of the invalidated region to draw into, but it's not clear why image scaling would differ based on the destination surface size. The only thing I can think of would be if we end up drawing with an integer offset instead of a partial pixel offset. We should always be rounded out to the nearest pixel though, and that problem would affect a lot more tests than just some image drawing. This also isn't something that I'd expect to vary 0.1% of the time.
Flags: needinfo?(matt.woodrow)
Do we include the time of day in the decision for how to round? That pattern of clustering together at the same time, across multiple trees, usually only happens to things that are hitting the network.
But we're still learnin' stuff with all these failures, right? They aren't just pointless noise pointlessly angering people?
(In reply to Phil Ringnalda (:philor) from comment #96)
> But we're still learnin' stuff with all these failures, right? They aren't
> just pointless noise pointlessly angering people?

Yup, enough fun for one bug. Marking as fuzzy on Android 4.0.
https://hg.mozilla.org/mozilla-central/rev/862c93f77047
https://hg.mozilla.org/releases/mozilla-aurora/rev/b2aa0c1397ae
https://hg.mozilla.org/releases/mozilla-beta/rev/8338468a7588

I'm resolving this bug under the assumption that nothing else is going to be done here. Feel free to reopen if that's wrong.
Status: NEW → RESOLVED
Closed: 10 years ago
status-firefox33: --- → fixed
status-firefox34: --- → fixed
status-firefox35: --- → fixed
status-firefox-esr31: --- → unaffected
Resolution: --- → FIXED
Target Milestone: --- → mozilla35
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: