Closed Bug 1579339 Opened 5 years ago Closed 5 years ago

Intermittent (tier2) TinderboxPrint: Per-test run of ...\resize-reflow-001.ref.html | content-inserted-002.ref.xhtml<br/>: FAILURE

Categories

(Testing :: General, defect, P5)

Product:
Testing
Component:
General
Type:
defect

Tracking

(Not tracked)

Status:
RESOLVED DUPLICATE of bug 1528471

People

(Reporter: intermittent-bug-filer, Unassigned)

References

(Regression)

Details

(Keywords: intermittent-failure, regression)

Filed by: nerli [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=265293089&repo=mozilla-central
Full log: https://queue.taskcluster.net/v1/task/NjHou9N8ToynQnOyFEHYMw/runs/0/artifacts/public/logs/live_backing.log
Reftest URL: https://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://queue.taskcluster.net/v1/task/NjHou9N8ToynQnOyFEHYMw/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1

[task 2019-09-06T05:33:38.160Z] 05:33:38 INFO - REFTEST SUITE-START | Running 0 tests
[task 2019-09-06T05:33:38.161Z] 05:33:38 INFO - REFTEST INFO | Result summary:
[task 2019-09-06T05:33:38.161Z] 05:33:38 INFO - REFTEST INFO | Successful: 0 (0 pass, 0 load only)
[task 2019-09-06T05:33:38.162Z] 05:33:38 INFO - REFTEST INFO | Unexpected: 0 (0 unexpected fail, 0 unexpected pass, 0 unexpected asserts, 0 failed load, 0 exception)
[task 2019-09-06T05:33:38.162Z] 05:33:38 INFO - REFTEST INFO | Known problems: 0 (0 known fail, 0 known asserts, 0 random, 0 skipped, 0 slow)
[task 2019-09-06T05:33:38.162Z] 05:33:38 INFO - REFTEST SUITE-END | Shutdown
[task 2019-09-06T05:33:38.163Z] 05:33:38 INFO - REFTEST INFO | :::
[task 2019-09-06T05:33:38.163Z] 05:33:38 INFO - REFTEST INFO | ::: Test verification summary for:
[task 2019-09-06T05:33:38.164Z] 05:33:38 INFO - REFTEST INFO | :::
[task 2019-09-06T05:33:38.164Z] 05:33:38 INFO - REFTEST INFO | ::: Z:/task_1567744601/build/tests/reftest/tests/layout/reftests/pagination/resize-reflow-001.ref.html
[task 2019-09-06T05:33:38.164Z] 05:33:38 INFO - REFTEST INFO | :::
[task 2019-09-06T05:33:38.164Z] 05:33:38 INFO - REFTEST INFO | ::: 1. Run each test 10 times in one browser. : Pass
[task 2019-09-06T05:33:38.164Z] 05:33:38 INFO - REFTEST INFO | ::: 2. Run each test 5 times in a new browser each time. : Pass
[task 2019-09-06T05:33:38.165Z] 05:33:38 INFO - REFTEST INFO | ::: 3. Run each test 10 times in one browser, in chaos mode. : Pass
[task 2019-09-06T05:33:38.165Z] 05:33:38 INFO - REFTEST INFO | ::: 4. Run each test 5 times in a new browser each time, in chaos mode. : Pass
[task 2019-09-06T05:33:38.165Z] 05:33:38 INFO - REFTEST INFO | :::
[task 2019-09-06T05:33:38.165Z] 05:33:38 INFO - REFTEST INFO | ::: Test verification PASSED
[task 2019-09-06T05:33:38.165Z] 05:33:38 INFO - REFTEST INFO | :::
[task 2019-09-06T05:33:38.187Z] 05:33:38 INFO - Return code: 0
[task 2019-09-06T05:33:38.188Z] 05:33:38 INFO - TinderboxPrint: reftest-reftest<br/><em class="testfail">T-FAIL</em>
[task 2019-09-06T05:33:38.188Z] 05:33:38 ERROR - # TBPL FAILURE #
[task 2019-09-06T05:33:38.188Z] 05:33:38 WARNING - setting return code to 2
[task 2019-09-06T05:33:38.188Z] 05:33:38 ERROR - TinderboxPrint: Per-test run of ...\resize-reflow-001.ref.html<br/>: FAILURE

This seems to be a regression from https://bugzilla.mozilla.org/show_bug.cgi?id=1308587

Flags: needinfo?(aethanyc)
Regressed by: 1308587

Another failure that was triggered: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=265293036&repo=mozilla-central&lineNumber=3338

Summary: Intermittent TinderboxPrint: Per-test run of ...\resize-reflow-001.ref.html<br/>: FAILURE → Intermittent (tier2) TinderboxPrint: Per-test run of ...\resize-reflow-001.ref.html | content-inserted-002.ref.xhtml<br/>: FAILURE

I don't think bug 1308587 makes any behavioral difference to the tests.

From the failure log in comment 0, resize-reflow-001.ref.html is a reference file of a reftest, and its corresponded test file isn't modified. And it seems nothing is executed per REFTEST INFO | Successful: 0 (0 pass, 0 load only)?

Also, from the log [task 2019-09-06T05:33:38.188Z] 05:33:38 INFO - TinderboxPrint: reftest-reftest<br/><em class="testfail">T-FAIL</em>, it seems that the log printing is fishy, because the code print_summary() can only print such message if unexpected_count is T-FAIL.

Geoffrey, any idea?

Flags: needinfo?(aethanyc) → needinfo?(gbrown)

That's exactly the issue: test-verify is trying to run the reftest harness against a reference file. There are a lot of cases where TV will recognize reference files and avoid verification, or where it can avoid the failure otherwise, but it isn't foolproof. It's a long-standing issue. Sorry.

Status: NEW → RESOLVED
Closed: 5 years ago
Component: Layout → General
Flags: needinfo?(gbrown)
Product: Core → Testing
Resolution: --- → DUPLICATE
Has Regression Range: --- → yes
You need to log in before you can comment on or make changes to this bug.