Closed Bug 1512464 Opened 10 months ago Closed 9 months ago

Intermittent Tier 2 test-verify TinderboxPrint: Per-test run of ...\mochitest\general\test_offsets.js<br/>: FAILURE

Categories

(Core :: DOM: Core & HTML, defect, P2)

defect

Tracking

()

RESOLVED FIXED
mozilla66
Tracking Status
firefox65 --- fixed
firefox66 --- fixed

People

(Reporter: intermittent-bug-filer, Assigned: gbrown)

References

Details

(Keywords: intermittent-failure, Whiteboard: [stockwell needswork:owner])

Attachments

(1 file)

Filed by: rgurzau [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=215765445&repo=mozilla-inbound

https://queue.taskcluster.net/v1/task/HWQ9XSNYSs2ZbSLF8awpDQ/runs/0/artifacts/public/logs/live_backing.log

16:39:13     INFO - SUITE-START | Running 0 tests
16:39:13     INFO - :::
16:39:13     INFO - ::: Test verification summary for:
16:39:13     INFO - :::
16:39:13  WARNING - Warning: test_offsets.js from manifest Z:\task_1544113872\build\tests\mochitest\chrome\dom\tests\mochitest\general\chrome.ini is not a valid test
16:39:13     INFO - :::
16:39:13     INFO - ::: 1. Run each test 10 times in one browser. : FAIL
16:39:13     INFO - ::: 2. Run each test 5 times in a new browser each time. : not run / incomplete
16:39:13     INFO - ::: 3. Run each test 10 times in one browser, in chaos mode. : not run / incomplete
16:39:13     INFO - ::: 4. Run each test 5 times in a new browser each time, in chaos mode. : not run / incomplete
16:39:13     INFO - :::
16:39:13     INFO - ::: Test verification FAILED!
16:39:13     INFO - :::
16:39:13     INFO - Buffered messages finished
16:39:13     INFO - SUITE-END | took 0s
16:39:13     INFO - Return code: 0
16:39:13    ERROR - No checks run.
16:39:13     INFO - TinderboxPrint: mochitest-chrome<br/><em class="testfail">T-FAIL</em>
16:39:13    ERROR - # TBPL FAILURE #
16:39:13  WARNING - setting return code to 2
16:39:13    ERROR - TinderboxPrint: Per-test run of ...\mochitest\general\test_offsets.js<br/>: FAILURE
16:39:13     INFO - Structured output parser in use for mochitest.
That push was backed out, how can it be its fault?
Flags: needinfo?(emilio)
@Emilio

These Tv failures with "TinderboxPrint: Per-test run of ...\mochitest\general\test_offsets.js<br/>: FAILURE" error message are occurring only on your pushes. Please see this link: https://tinyurl.com/yapkbpyv
Flags: needinfo?(emilio)
That is because my patch touched the test, and it failed in TV. But I don't think the TV failure was related to my patch.
Flags: needinfo?(emilio)
(In reply to Emilio Cobos Álvarez (:emilio) from comment #3)
> That push was backed out, how can it be its fault?

That's what i was hoping to find out :D. they fail even after the backout though.
That means that there's a bug in the test which causes it to fail in verify, but it was covered up just because nobody touches that test often :)
Is there a simpler way to find out who covered it?  or what exactly the source would be?
Well, chances are that the test doesn't pass in verify since verify was introduced, so not really.

The way to fix this is actually trying to reproduce that failure locally and debug it. I can try to prioritize investigating this test if you think it's very important, but otherwise I'm not sure I'd get to it very soon, specially since the problem is likely to be a test problem, and is likely to be there since forever (so not a regression).
Duplicate of this bug: 1512895
Assignee: nobody → gbrown
Priority: -- → P2
The unexpected entries in dom/tests/mochitest/general/chrome.ini seem to go back to bug 920223. I note https://bugzilla.mozilla.org/show_bug.cgi?id=920223#c6 and https://bugzilla.mozilla.org/show_bug.cgi?id=920223#c7 -- perhaps the source of confusion.
The problem here is that test_offsets.js was listed as a test file in chrome.ini, but it isn't actually a test - it is a support file for test_offsets.html and test_offsets.xul. Correcting the manifest allows the file to be modified without causing a failure in test verification.

https://treeherder.mozilla.org/#/jobs?repo=try&tier=1%2C2%2C3&revision=672e6209864e8b9a732a9373cbbcab2d5970d699
Attachment #9030616 - Flags: review?(htsai)
Comment on attachment 9030616 [details] [diff] [review]
remove non-test files from list of tests in chrome.ini

Review of attachment 9030616 [details] [diff] [review]:
-----------------------------------------------------------------

Thanks Geoffrey! I'd like to get a DOM peer review on this, though the change looks good to me.
Attachment #9030616 - Flags: review?(htsai)
Attachment #9030616 - Flags: review?(bugs)
Attachment #9030616 - Flags: feedback+
Comment on attachment 9030616 [details] [diff] [review]
remove non-test files from list of tests in chrome.ini

oh, those files shouldn't have had test_* prefix but something like file_* or so. Oh well, this is fine too.
Attachment #9030616 - Flags: review?(bugs) → review+
Pushed by gbrown@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/e19e818041e3
Remove invalid entries from dom/tests/mochitest/general/chrome.ini; r=smaug
https://hg.mozilla.org/mozilla-central/rev/e19e818041e3
Status: NEW → RESOLVED
Closed: 9 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla66
Component: DOM → DOM: Core & HTML
You need to log in before you can comment on or make changes to this bug.