The default bug view has changed. See this FAQ.

Intermittent WinXP (and no other platform except WinXP) TEST-UNEXPECTED-PASS (not TEST-UNEXPECTED-FAIL, TEST-UNEXPECTED-PASS) | dynamic-text-04.svg | image comparison (==)

RESOLVED FIXED in mozilla12

Status

()

Core
SVG
RESOLVED FIXED
6 years ago
4 years ago

People

(Reporter: philor, Unassigned)

Tracking

({intermittent-failure})

Trunk
mozilla12
All
Windows XP
intermittent-failure
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [test marked as random])

Attachments

(1 obsolete attachment)

(Reporter)

Description

6 years ago
http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1298487014.1298488395.13658.gz
Rev3 WINNT 5.1 mozilla-central opt test reftest on 2011/02/23 10:50:14
s: talos-r3-xp-013

REFTEST INFO | Loading a blank page
REFTEST TEST-START | file:///C:/talos-slave/test/build/reftest/tests/layout/reftests/svg/dynamic-text-04.svg
REFTEST TEST-START | file:///C:/talos-slave/test/build/reftest/tests/layout/reftests/svg/dynamic-text-04-ref.svg
REFTEST TEST-UNEXPECTED-PASS | file:///C:/talos-slave/test/build/reftest/tests/layout/reftests/svg/dynamic-text-04.svg | image comparison (==)
REFTEST   IMAGE: data:image/png;base64,iVBOR...
REFTEST INFO | Loading a blank page

I would expect this extremely intermittent pass of the test that bug 623405 marked as failing on WinXP to just be, as I said mistakenly of the other test there,

[[[
Honestly, I don't have the slightest idea what made me think that "drawn in a
random spot" somehow meant "drawn in a random spot which will never be the
right spot."
]]]

except that I'm a little puzzled by the way that the log only shows one data URI, rather than the two that I think I've seen the few times I've actually paid any attention to the details of reftest failures.
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)

Comment 5

6 years ago
Created attachment 526782 [details] [diff] [review]
hg changeset patch

Updated

6 years ago
Keywords: checkin-needed
Why do we want to have this test marked as random? It doesn't sound to fail THAT much...
Keywords: checkin-needed
Hardware: x86 → All
I think you mean: it doesn't seem to *pass* that much. :)

It's nearly-perma-failing on Windows XP -- so it marked as a failing -- but it occasionally randomly passes. (hence this bug)

IMHO, flagging it as random-on-windows-XP would be accurate and appropriate in this case.
Though it also would be interesting to find out if the test ever started (reliably) passing, so I can see the case for leaving the annotation as-is, too. (given the relatively low level of orange-noise that it's causing now)
Oh, my bad... I didn't see it was changing fails-if to random-if, I was too focused on the random-if.
(In reply to comment #8)
> Though it also would be interesting to find out if the test ever started
> (reliably) passing, so I can see the case for leaving the annotation as-is,
> too. (given the relatively low level of orange-noise that it's causing now)

I agree. The fact it's currently marked fails-if only reverse the issue: we should only mark something as orange if we really have no other choices.
(Reporter)

Comment 11

6 years ago
If someone wants to write that test, which isn't this bug, and which ought to be named something like bug_421587_unfixed and ought to be in bug 421587, I'd think that all they would need to do would be to have something like four iframes with rotated text in them. Since we're only accidentally getting it right four out of several thousand times, making sure we never get four right at once ought to be enough to tell us that bug 421587 still exists.
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)

Updated

6 years ago
Attachment #526782 - Attachment is obsolete: true
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
The changes in bug 598482 will likely make this happen more, because the test will paint fewer times.

That could be "fixed" by putting the actual rotation off on a requestAnimationFrame callback or three, I suspect.
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
https://tbpl.mozilla.org/php/getParsedLog.php?id=8561837&tree=Mozilla-Inbound
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)

Comment 77

5 years ago
I suspect that bug 525095 has much reduced the amount of randomness resulting in this test passing much more frequently than it used to.
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Summary: Intermittent WinXP TEST-UNEXPECTED-PASS | dynamic-text-04.svg → Intermittent WinXP TEST-UNEXPECTED-PASS | dynamic-text-04.svg | image comparison (==)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
(In reply to Robert Longson from comment #77)
> I suspect that bug 525095 has much reduced the amount of randomness
> resulting in this test passing much more frequently than it used to.

Yeah, it seems to be passing half the time, and failing half the time.

I can't see any visual difference between the failing image and the reference, so this seems low priority. For now I marked the test as random:

https://hg.mozilla.org/integration/mozilla-inbound/rev/eea95e86541f
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Test annotation landed:
https://hg.mozilla.org/mozilla-central/rev/eea95e86541f

Leaving this bug open for now, but you can resolve it if there's no more work planned here.
Whiteboard: [orange] → [orange][test marked as random]
Target Milestone: --- → mozilla12
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)

Comment 173

5 years ago
Any work on why this is random would take place in bug 421587.
Status: NEW → RESOLVED
Last Resolved: 5 years ago
Resolution: --- → FIXED
(and to be clear, we shouldn't be getting any more oranges from this, since it was marked random in comment 167)
Depends on: 421587
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
(Reporter)

Comment 190

5 years ago
Nice try, but it's not that easy to get out of filing new failures.
Status: REOPENED → RESOLVED
Last Resolved: 5 years ago5 years ago
Resolution: --- → FIXED
Summary: Intermittent WinXP TEST-UNEXPECTED-PASS | dynamic-text-04.svg | image comparison (==) → Intermittent WinXP (and no other platform except WinXP) TEST-UNEXPECTED-PASS (not TEST-UNEXPECTED-FAIL, TEST-UNEXPECTED-PASS) | dynamic-text-04.svg | image comparison (==)
(Reporter)

Updated

5 years ago
Depends on: 768297
(Assignee)

Updated

4 years ago
Keywords: intermittent-failure
(Assignee)

Updated

4 years ago
Whiteboard: [orange][test marked as random] → [test marked as random]
You need to log in before you can comment on or make changes to this bug.