Intermittent 413361-1.html | image comparison (==), max difference: 5, number of differing pixels: 1

RESOLVED FIXED in Firefox 33
(NeedInfo from)

Status

()

Core
ImageLib
RESOLVED FIXED
4 years ago
4 years ago

People

(Reporter: emorley, Unassigned, NeedInfo)

Tracking

({intermittent-failure})

Trunk
mozilla35
ARM
Gonk (Firefox OS)
intermittent-failure
Points:
---

Firefox Tracking Flags

(firefox33 fixed, firefox34 fixed, firefox35 fixed, firefox-esr31 unaffected)

Details

Attachments

(2 attachments)

Created attachment 8437708 [details]
reftest log

b2g_emulator_vm mozilla-inbound opt test reftest-4 on 2014-06-10 02:22:16 PDT for push 6d172e86ce2e

slave: tst-linux64-spot-794

https://tbpl.mozilla.org/php/getParsedLog.php?id=41431583&tree=Mozilla-Inbound

{
03:34:23     INFO -  REFTEST TEST-START | http://10.0.2.2:8888/tests/layout/reftests/bugs/413361-1.html
03:34:32     INFO -  REFTEST TEST-LOAD | http://10.0.2.2:8888/tests/layout/reftests/bugs/413361-1.html | 663 / 711 (93%)
03:34:32     INFO -  REFTEST TEST-LOAD | http://10.0.2.2:8888/tests/layout/reftests/bugs/413361-1-ref.html | 663 / 711 (93%)
03:34:32     INFO -  REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8888/tests/layout/reftests/bugs/413361-1.html | image comparison (==), max difference: 5, number of differing pixels: 1
}
Comment hidden (Treeherder Robot)
Created attachment 8438661 [details]
reftest-analyzer screenshot

1 pixel with each RGB component differ -5 from ref.
The failure in comment 2 (different slave) is *exactly* the same.
("The failure in comment 2" should have said "comment 1".)
I suspect this is another instance of "images don't scale deterministically
on some hardware" bug.
Component: Layout: Block and Inline → ImageLib
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Android certainly doesn't scale images deterministically. I don't know how many times I've opened the reftest analyzer and seen that same image with a line of fail through the middle, and then much later seen either a fuzzy-if or a change to a solid image because "oh, I wasn't trying to test what the middle of the image looked like anyway."
dbaron, any reason not to switch to http://mxr.mozilla.org/mozilla-central/source/layout/reftests/bugs/blue-600x58.png like we have for half a dozen other things that were using mozilla-banner.gif just because it was handy?
Flags: needinfo?(dbaron)
Comment hidden (Treeherder Robot)
If it really wasn't deterministic that would seem reasonable, although it seems somewhat odd to say "images don't scale deterministically" when they match 99.9% of the time and fail 0.1% of the time.  I'm curious if seth knows what's up.  (Probably not a huge deal, but also, if it's a repeated pattern of low-frequency intermittent failure in reftests, might not be a bad idea to fix the underlying problem.)

I guess the change to another same-dimension image would probably be fine for what this test was intended to test, though.  But part of what's great about tests is that they test things other than what was originally intended.
Flags: needinfo?(dbaron) → needinfo?(seth)
I guess one other question here:  is "some hardware" equal to "a particular small number of test machines that have some bad RAM", or is it evenly distributed across Android machines?  (I ask because being so close to deterministic seems odd.)
Flags: needinfo?(philringnalda)
Usually the cause of images not scaling the same way is that in one case we are drawing into a context the size of the whole page, and in the other case we are drawing into a context that is only the size of the image (because we are doing a paint where only it was invalidated). So on platforms where we draw into contexts like that to get deterministic results for a scaled image all you need to do is consider the offset of the image as well as the size. The seeming-non-determinism just comes from not considering the offset of the image.
Which midaired a comment where I said that I didn't understand bug 863658 comment 22 either, but I didn't see any sign of machine-specific there or in bug 948389 and bug 919482 and bug 909305. Personally I wouldn't be surprised if, in the fantasy world where someone would ever actually look at the tests which were blindly disabled to turn on reftests on Android, and the tests which were blindly disabled to turn on reftests on b2g, a fair number of things we skip-if are actually just resizing images without caring about the contents. mozilla-banner.gif's only in 48 reftests, but there's no reason to think there aren't other images, or other copies of it with different names.
Flags: needinfo?(philringnalda)
The thing that is machine specific is that we usually draw into a context the size of the whole window. Only on some backends do we size the context to the size of the area to draw.
Right, but we're trying to disambiguate "machine-specific, we should disable panda-0500 and pretend that we have diagnostics that can detect something wrong with it" from "platform-specific, you just can't reliably test that on this platform, so you can either change the test or have it disabled on this platform."
Oh, sorry, I was only speaking to platform-specific concerns.
Matt, do you know if we actually derive any benefit from the behavior that Timothy mentions in comment 13? If it's not buying us something,  I wonder if it wouldn't be better to fix the underlying behavior, since in the long term we are going to keep having this issue - even if we fix all existing tests, people will keep checking in more tests with non-solid-color images.
Flags: needinfo?(matt.woodrow)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
(In reply to Seth Fowler [:seth] from comment #18)
> Matt, do you know if we actually derive any benefit from the behavior that
> Timothy mentions in comment 13? If it's not buying us something,  I wonder
> if it wouldn't be better to fix the underlying behavior, since in the long
> term we are going to keep having this issue - even if we fix all existing
> tests, people will keep checking in more tests with non-solid-color images.

I think we'd need to know exactly what the issue was to answer this.

Android definitely allocates surfaces the size of the invalidated region to draw into, but it's not clear why image scaling would differ based on the destination surface size. The only thing I can think of would be if we end up drawing with an integer offset instead of a partial pixel offset. We should always be rounded out to the nearest pixel though, and that problem would affect a lot more tests than just some image drawing. This also isn't something that I'd expect to vary 0.1% of the time.
Flags: needinfo?(matt.woodrow)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Do we include the time of day in the decision for how to round? That pattern of clustering together at the same time, across multiple trees, usually only happens to things that are hitting the network.
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
But we're still learnin' stuff with all these failures, right? They aren't just pointless noise pointlessly angering people?
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
(In reply to Phil Ringnalda (:philor) from comment #96)
> But we're still learnin' stuff with all these failures, right? They aren't
> just pointless noise pointlessly angering people?

Yup, enough fun for one bug. Marking as fuzzy on Android 4.0.
https://hg.mozilla.org/mozilla-central/rev/862c93f77047
https://hg.mozilla.org/releases/mozilla-aurora/rev/b2aa0c1397ae
https://hg.mozilla.org/releases/mozilla-beta/rev/8338468a7588

I'm resolving this bug under the assumption that nothing else is going to be done here. Feel free to reopen if that's wrong.
Status: NEW → RESOLVED
Last Resolved: 4 years ago
status-firefox33: --- → fixed
status-firefox34: --- → fixed
status-firefox35: --- → fixed
status-firefox-esr31: --- → unaffected
Resolution: --- → FIXED
Target Milestone: --- → mozilla35
Comment hidden (Treeherder Robot)
You need to log in before you can comment on or make changes to this bug.