Closed Bug 791480 Opened 12 years ago Closed 10 years ago

Intermittent REFTEST TEST-UNEXPECTED-FAIL | layout/reftests/printing/745025-1.html | image comparison (==), max difference: 255, number of differing pixels: 100

Categories

(Core :: Printing: Output, defect)

Product:
Core
Component:
Printing: Output
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

()

Status:
RESOLVED FIXED
Milestone:
mozilla31
Tracking Flags:
Tracking Status
firefox31 --- fixed
firefox-esr24 --- wontfix

People

(Reporter: philor, Assigned: roc)

References

Details

(Keywords: intermittent-failure)

Attachments

(2 files)

reftest log
143.27 KB, text/plain
Details
fix
4.96 KB, patch
dbaron
: review+
Details | Diff | Splinter Review
Attached file reftest log — 
https://tbpl.mozilla.org/php/getParsedLog.php?id=15244667&tree=Mozilla-Inbound
Rev3 WINNT 5.1 mozilla-inbound opt test reftest on 2012-09-15 11:12:17 PDT for push edbfd9e7b324
slave: talos-r3-xp-014

REFTEST TEST-START | file:///c:/talos-slave/test/build/reftest/tests/layout/reftests/printing/745025-1.html | 6026 / 8378 (71%)
REFTEST TEST-START | file:///c:/talos-slave/test/build/reftest/tests/layout/reftests/printing/745025-1-ref.html | 6026 / 8378 (71%)
REFTEST TEST-UNEXPECTED-FAIL | file:///c:/talos-slave/test/build/reftest/tests/layout/reftests/printing/745025-1.html | image comparison (==), max difference: 255, number of differing pixels: 100
(data URLs that ask the question "Dude, where's my little red square?")
REFTEST INFO | Saved log: START file:///c:/talos-slave/test/build/reftest/tests/layout/reftests/printing/745025-1.html
REFTEST INFO | Saved log: [CONTENT] OnDocumentLoad triggering WaitForTestEnd
REFTEST INFO | Saved log: [CONTENT] WaitForTestEnd: Adding listeners
REFTEST INFO | Saved log: Initializing canvas snapshot
REFTEST INFO | Saved log: DoDrawWindow 0,0,800,1000
REFTEST INFO | Saved log: [CONTENT] WaitForTestEnd: Adding listeners
REFTEST INFO | Saved log: [CONTENT] MakeProgress: STATE_WAITING_TO_FIRE_INVALIDATE_EVENT
REFTEST INFO | Saved log: [CONTENT] MakeProgress: waiting for MozAfterPaint
REFTEST INFO | Saved log: [CONTENT] AfterPaintListener in file:///c:/talos-slave/test/build/reftest/tests/layout/reftests/printing/745025-1.html
REFTEST INFO | Saved log: Updating canvas for invalidation
REFTEST INFO | Saved log: DoDrawWindow 0,0,800,1000
REFTEST INFO | Saved log: [CONTENT] MakeProgress: STATE_WAITING_TO_FIRE_INVALIDATE_EVENT
REFTEST INFO | Saved log: [CONTENT] MakeProgress: waiting for MozAfterPaint
REFTEST INFO | Saved log: [CONTENT] AfterPaintListener in file:///c:/talos-slave/test/build/reftest/tests/layout/reftests/printing/745025-1.html
REFTEST INFO | Saved log: [CONTENT] AttrModifiedListener fired
REFTEST INFO | Saved log: [CONTENT] MakeProgress: STATE_WAITING_TO_FIRE_INVALIDATE_EVENT
REFTEST INFO | Saved log: [CONTENT] MakeProgress: dispatching MozReftestInvalidate
REFTEST INFO | Saved log: [CONTENT] MakeProgress: STATE_WAITING_FOR_REFTEST_WAIT_REMOVAL
REFTEST INFO | Saved log: [CONTENT] MakeProgress: setting up print mode
REFTEST INFO | Saved log: [CONTENT] FlushRendering generated paint for window file:///c:/talos-slave/test/build/reftest/tests/layout/reftests/printing/745025-1.html
REFTEST INFO | Saved log: [CONTENT] MakeProgress: STATE_WAITING_TO_FINISH
REFTEST INFO | Saved log: [CONTENT] MakeProgress: waiting for MozAfterPaint
REFTEST INFO | Saved log: [CONTENT] AfterPaintListener in file:///c:/talos-slave/test/build/reftest/tests/layout/reftests/printing/745025-1.html
REFTEST INFO | Saved log: Updating canvas for invalidation
REFTEST INFO | Saved log: DoDrawWindow 0,0,800,1000
REFTEST INFO | Saved log: [CONTENT] AttrModifiedListener fired
REFTEST INFO | Saved log: [CONTENT] MakeProgress: STATE_WAITING_TO_FIRE_INVALIDATE_EVENT
REFTEST INFO | Saved log: [CONTENT] MakeProgress: dispatching MozReftestInvalidate
REFTEST INFO | Saved log: [CONTENT] MakeProgress: STATE_WAITING_FOR_REFTEST_WAIT_REMOVAL
REFTEST INFO | Saved log: [CONTENT] MakeProgress: setting up print mode
REFTEST INFO | Saved log: [CONTENT] FlushRendering generated paint for window file:///c:/talos-slave/test/build/reftest/tests/layout/reftests/printing/745025-1.html
REFTEST INFO | Saved log: [CONTENT] MakeProgress: STATE_WAITING_TO_FINISH
REFTEST INFO | Saved log: [CONTENT] MakeProgress: waiting for MozAfterPaint
REFTEST INFO | Saved log: [CONTENT] AfterPaintListener in file:///c:/talos-slave/test/build/reftest/tests/layout/reftests/printing/745025-1.html
REFTEST INFO | Saved log: [CONTENT] MakeProgress: STATE_WAITING_TO_FINISH
REFTEST INFO | Saved log: [CONTENT] MakeProgress: Completed
REFTEST INFO | Saved log: [CONTENT] MakeProgress: STATE_COMPLETED
REFTEST INFO | Saved log: [CONTENT] RecordResult fired
REFTEST INFO | Saved log: RecordResult fired
REFTEST INFO | Saved log: START file:///c:/talos-slave/test/build/reftest/tests/layout/reftests/printing/745025-1-ref.html
REFTEST INFO | Saved log: [CONTENT] OnDocumentLoad setting up print mode
REFTEST INFO | Saved log: [CONTENT] OnDocumentLoad triggering AfterOnLoadScripts
REFTEST INFO | Saved log: Initializing canvas snapshot
REFTEST INFO | Saved log: DoDrawWindow 0,0,800,1000
REFTEST INFO | Saved log: [CONTENT] RecordResult fired
REFTEST INFO | Saved log: RecordResult fired
Blocks: 745025
Summary: Intermittent REFTEST TEST-UNEXPECTED-FAIL | file:///c:/talos-slave/test/build/reftest/tests/layout/reftests/printing/745025-1.html | image comparison (==), max difference: 255, number of differing pixels: 100 → Intermittent REFTEST TEST-UNEXPECTED-FAIL | layout/reftests/printing/745025-1.html | image comparison (==), max difference: 255, number of differing pixels: 100
Whiteboard: [orange]
Brendan, please may you take a look at this?

At the moment, the failure rate of this test is high enough that it will be one of the ones disabled for too many intermittent failures, once I return from vacation in a few days time.

Cheers :-)
Flags: needinfo?(bdahl)
(In reply to Ed Morley [UTC+0; email:edmorley@moco] from comment #75)
> Brendan, please may you take a look at this?
> 
> At the moment, the failure rate of this test is high enough that it will be
> one of the ones disabled for too many intermittent failures, once I return
> from vacation in a few days time.
> 
> Cheers :-)

I'll try to check it out on Monday. Last time I tried to fix it I could not reproduce locally though.
Flags: needinfo?(bdahl)
Failure rate too high; marked as fuzzy-if on Windows7 whilst Brendan investigates:
https://hg.mozilla.org/integration/mozilla-inbound/rev/1c384f9d78e9
Whiteboard: [test marked as fuzzy-if on Windows][leave open]
https://tbpl.mozilla.org/php/getParsedLog.php?id=34820470&tree=Mozilla-Inbound
WINNT 6.2 mozilla-inbound opt test reftest-no-accel on 2014-02-17 21:09:45 PST for push bcba561c4e53
(continued from previous comment)
REFTEST TEST-UNEXPECTED-FAIL | file:///C:/slave/test/build/tests/reftest/tests/layout/reftests/printing/745025-1.html | image comparison (==), max difference: 255, number of differing pixels: 100
Though actually, that Windows NT 6.2 reftest-noaccel failure is a new 100% of the time failure triggered by the landing of bug 972713.
(In reply to David Baron [:dbaron] (needinfo? me) (UTC-8) from comment #153)
> Though actually, that Windows NT 6.2 reftest-noaccel failure is a new 100%
> of the time failure triggered by the landing of bug 972713.

When did the policy change to annotating away new test failures instead of backing out the regressing changeset?

Anyway, roc backed out bug 972713, so I've backed out this annotation as well. If it's determined in that bug to be the best way forward, he can add it to his patch when he re-lands.
https://hg.mozilla.org/integration/mozilla-inbound/rev/9254485c7f9d
Flags: needinfo?(dbaron)
Sorry, it didn't; I asked roc to fix or back out, but I failed to make it clear that the annotation should be backed out too.
Flags: needinfo?(dbaron)
(And you're right that I should have backed it out right after realizing it rather than waiting for roc; I think I've had the "don't waste build/test cycles" drilled into me too hard, and I didn't want to trigger another wasteful full round of builds.)
No worries, thanks for the replies :)
Changing the test to be
      canvas.mozPrintCallback = function(obj) {
        setTimeout(function() {
          var ctx = obj.context;
          ctx.fillStyle = 'rgb(255, 0, 0)';
          ctx.fillRect(0, 0, 10, 10);
          obj.done();
        }, 3000);
      };
makes the problem obvious. There is nothing to make reftests wait for all print callbacks to have completed via the "done()" method before taking the snapshot.
Attached patch fix — — Splinter Review 
This allows reftest-print tests to remove reftest-wait after any canvas-print-callbacks have completed.

An alternative approach would have been to make reftests automatically wait for all canvas print callbacks to complete, but that seems like unnecessary complexity.
Attachment #8387344 - Flags: review?(dbaron)
I was arguing against something somewhat similar to this in bug 428037.

I guess there are currently only two tests that use the combination:
  layout/reftests/printing/test-async-print.html
  layout/reftests/printing/745025-1.html

and this one seems to prefer being able to switch to reftest-print before reftest-wait is removed, as did Mats's use case.  So I guess, even though it's not related to the way printing actually works, it probably makes more sense for testing to switch it to be this way.
Comment on attachment 8387344 [details] [diff] [review]
fix

Can you remove the 3000ms timeout?  (Can it be 0?  If not, why not?)

And please remove the test disabling as well.

r=dbaron with that, I guess
Attachment #8387344 - Flags: review?(dbaron) → review+
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
Assignee: nobody → roc
status-firefox20: affected → ---
status-firefox31: --- → fixed
status-firefox-esr24: --- → wontfix
Whiteboard: [test marked as fuzzy-if on Windows][leave open]
Target Milestone: --- → mozilla31
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: